Wikis as a tool for fostering emergence of communities

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Wikis as a tool for fostering emergence of communities

alain_desilets
I need a good solid reference to substantiate the following claim:

"Besides leading to high quality content, wikis have been shown to be good tools for fostering the emergence of active communities"

Does anyone know of a good research paper that looks specifically at this kind of impact of wikis?

Thx.


----
Alain Désilets, National Research Council of Canada
Chair, WikiSym 2007

2007 International Symposium on Wikis
Wikis at Work in the World:
Open, Organic, Participatory Media for the 21st Century

http://www.wikisym.org/ws2007/


_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergence of communities

Brian J Mingus
I think the first part of your claim needs to be substantiated first! Almost all of the content on the English Wikipedia, for example, is of low quality.

http://en.wikipedia.org/wiki/Wikipedia:Version_1.0_Editorial_Team/Index


On 8/29/07, Desilets, Alain <[hidden email]> wrote:
I need a good solid reference to substantiate the following claim:

"Besides leading to high quality content, wikis have been shown to be good tools for fostering the emergence of active communities"

Does anyone know of a good research paper that looks specifically at this kind of impact of wikis?

Thx.


----
Alain Désilets, National Research Council of Canada
Chair, WikiSym 2007

2007 International Symposium on Wikis
Wikis at Work in the World:
Open, Organic, Participatory Media for the 21st Century

http://www.wikisym.org/ws2007/


_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergence ofcommunities

alain_desilets
Alain Désilets wrote:

> I need a good solid reference to substantiate the following claim:
>
> "Besides leading to high quality content, wikis have been shown to be good tools
> for fostering the emergence of active communities"
>
> Does anyone know of a good research paper that looks specifically at this kind of
> impact of wikis?

Brian replied:

> I think the first part of your claim needs to be substantiated first! Almost all of the
> content on the English Wikipedia, for example, is of low quality.
>
> http://en.wikipedia.org/wiki/Wikipedia:Version_1.0_Editorial_Team/Index

I already have a solid reference to substantiate the fact that Wikipedia has quality content, i.e. the Nature Magazne study of 2005:

http://www.nature.com/news/2005/051212/full/438900a.html

This study found that the quality of Wikipedia content was comparable to that of Britannica (on a sample of 42 pages in various scientific domains).

The page you quote above does not constitute a scientific assessment of the quality of WikiPedia pages. For one thing, it's not a random sample. These are pages that people on the editorial board selected for review, and it could be that they naturally focused on pages that were a-priori more likely to present quality issues. Second, it does not compare the quality of wikipedia pages to the quality of corresponding pages on more traditional resources.

The Nature study on the other hand was scientifically sound on both of those points. It did a random sample of pages on a broad range of scientific topics. And it compared the quality of those pages to comparable pages on Brittanica.

The Nature study is also consistent with my personal experience (and that of most people I talk to) as a user of Wikipedia. In other words, whenever I go to a Wikipedia page on a topic that I know well, I find it to be good quality and accurate.

So... Until someone shows me a scientifically sound demonstration that Wikipedia pages are of significantly lower quality than other more traditional resources, I will continue claiming that wikis like Wikipedia can lead to quality content.


----
Alain Désilets, National Research Council of Canada
Chair, WikiSym 2007

2007 International Symposium on Wikis
Wikis at Work in the World:
Open, Organic, Participatory Media for the 21st Century

http://www.wikisym.org/ws2007/




_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergence ofcommunities

Andrea Forte
I don't have a specific reference for you, just some comments. :) I
think perhaps your original claim is too strong--on both counts.
Including the word "can" as you did in your reply is important. Wikis
CAN lead to high-quality content. And CAN lead to active communities.
(Clearly there are examples of active communities and examples of
wikis that never get used much.) I am not convinced that the
technology itself leads to high-quality articles in Wikipedia, and I
don't think that the Nature article supports that claim in any way.
One could hypothesize about characteristics of wiki that help support
the goal of producing high-quality articles and try to empirically
verify the claim, but to my knowledge no one has done so in any
systematic fashion.


On 8/29/07, Desilets, Alain <[hidden email]> wrote:

> Alain Désilets wrote:
>
> >               I need a good solid reference to substantiate the following claim:
> >
> >               "Besides leading to high quality content, wikis have been shown to be good tools
> > for fostering the emergence of active communities"
> >
> >               Does anyone know of a good research paper that looks specifically at this kind of
> > impact of wikis?
>
> Brian replied:
>
> >       I think the first part of your claim needs to be substantiated first! Almost all of the
> > content on the English Wikipedia, for example, is of low quality.
> >
> >       http://en.wikipedia.org/wiki/Wikipedia:Version_1.0_Editorial_Team/Index
>
> I already have a solid reference to substantiate the fact that Wikipedia has quality content, i.e. the Nature Magazne study of 2005:
>
> http://www.nature.com/news/2005/051212/full/438900a.html
>
> This study found that the quality of Wikipedia content was comparable to that of Britannica (on a sample of 42 pages in various scientific domains).
>
> The page you quote above does not constitute a scientific assessment of the quality of WikiPedia pages. For one thing, it's not a random sample. These are pages that people on the editorial board selected for review, and it could be that they naturally focused on pages that were a-priori more likely to present quality issues. Second, it does not compare the quality of wikipedia pages to the quality of corresponding pages on more traditional resources.
>
> The Nature study on the other hand was scientifically sound on both of those points. It did a random sample of pages on a broad range of scientific topics. And it compared the quality of those pages to comparable pages on Brittanica.
>
> The Nature study is also consistent with my personal experience (and that of most people I talk to) as a user of Wikipedia. In other words, whenever I go to a Wikipedia page on a topic that I know well, I find it to be good quality and accurate.
>
> So... Until someone shows me a scientifically sound demonstration that Wikipedia pages are of significantly lower quality than other more traditional resources, I will continue claiming that wikis like Wikipedia can lead to quality content.
>
>
> ----
> Alain Désilets, National Research Council of Canada
> Chair, WikiSym 2007
>
> 2007 International Symposium on Wikis
> Wikis at Work in the World:
> Open, Organic, Participatory Media for the 21st Century
>
> http://www.wikisym.org/ws2007/
>
>
>
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>

_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergence of communities

Piotr Konieczny-2
In reply to this post by alain_desilets
Desilets, Alain wrote:

> I need a good solid reference to substantiate the following claim:
>
> "Besides leading to high quality content, wikis have been shown to be good tools for fostering the emergence of active communities"
>
> Does anyone know of a good research paper that looks specifically at this kind of impact of wikis?

Check: WP:ACST for a list of articles with abstracts. There should be a
few suitable.


--
Piotr Konieczny

"The problem about Wikipedia is, that it just works in reality, not in
theory."

_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergence of communities

Reid Priedhorsky-2
In reply to this post by alain_desilets
Desilets, Alain wrote:
>
> I need a good solid reference to substantiate the following claim:
>
> "Besides leading to high quality content, wikis have been shown to be
> good tools for fostering the emergence of active communities"
>
> Does anyone know of a good research paper that looks specifically at
> this kind of impact of wikis?

Hi Alain,

The following work by Dan Cosley et al. argues that wikis and
traditional review-before-publication result in the same quality, but
wiki gets there faster:

Cosley, D., Frankowski, D., Terveen, L., & Riedl, J. (2006). Using
Intelligent Task Routing and Contribution Review to Help Communities
Build Artifacts of Lasting Value. Proc. CHI 2006.
http://grouplens.org/papers/pdf/itr-chi2006.pdf

(Full disclosure: this work is a product of my own research group.)

HTH,

Reid

_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergence of communities

Brian J Mingus
I am thinking more along the lines of the loss of quality of previously high quality articles, which are already incredibly small in proportion, such as "featured articles." Traditional content production methods asymptote in quality, but the editing process in place at Wikipedia (which is only one possible wiki process, and also one of the most successful, but does not necessarily speak about wikis in general) encourages articles to gradually increase in quality, and then again decrease. It is unknown if they will stabilize (which brings about thoughts of a 1.0)

There are plenty of examples of this phenomenon: http://en.wikipedia.org/wiki/Wikipedia:Former_featured_articles

This could be due to changing featured article criteria, but in general, the claim that simply starting a wiki encourages high quality content is lacking evidence. If anything, wikis encourage the addition of noise to high quality content. Adding noise to turing complete wiki syntax can quickly snowball, turning into an aggregation of media that lacks coherence.

On 8/29/07, Reid Priedhorsky <[hidden email]> wrote:
Desilets, Alain wrote:
>
> I need a good solid reference to substantiate the following claim:
>
> "Besides leading to high quality content, wikis have been shown to be
> good tools for fostering the emergence of active communities"
>
> Does anyone know of a good research paper that looks specifically at
> this kind of impact of wikis?

Hi Alain,

The following work by Dan Cosley et al. argues that wikis and
traditional review-before-publication result in the same quality, but
wiki gets there faster:

Cosley, D., Frankowski, D., Terveen, L., & Riedl, J. (2006). Using
Intelligent Task Routing and Contribution Review to Help Communities
Build Artifacts of Lasting Value. Proc. CHI 2006.
http://grouplens.org/papers/pdf/itr-chi2006.pdf

(Full disclosure: this work is a product of my own research group.)

HTH,

Reid

_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergence ofcommunities

phoebe ayers
In reply to this post by alain_desilets
On 8/29/07, Desilets, Alain <[hidden email]> wrote:

> Alain Désilets wrote:
>
> >               I need a good solid reference to substantiate the following claim:
> >
> >               "Besides leading to high quality content, wikis have been shown to be good tools
> > for fostering the emergence of active communities"
> >
> >               Does anyone know of a good research paper that looks specifically at this kind of
> > impact of wikis?
>
> Brian replied:
>
> >       I think the first part of your claim needs to be substantiated first! Almost all of the
> > content on the English Wikipedia, for example, is of low quality.
> >
> >       http://en.wikipedia.org/wiki/Wikipedia:Version_1.0_Editorial_Team/Index
>
> I already have a solid reference to substantiate the fact that Wikipedia has quality content, i.e. the Nature Magazne study of 2005:
>
> http://www.nature.com/news/2005/051212/full/438900a.html
>
> This study found that the quality of Wikipedia content was comparable to that of Britannica (on a sample of 42 pages in various scientific domains).
>
> The page you quote above does not constitute a scientific assessment of the quality of WikiPedia pages. For one thing, it's not a random sample. These are pages that people on the editorial board selected for review, and it could be that they naturally focused on pages that were a-priori more likely to present quality issues. Second, it does not compare the quality of wikipedia pages to the quality of corresponding pages on more traditional resources.
>
> The Nature study on the other hand was scientifically sound on both of those points. It did a random sample of pages on a broad range of scientific topics. And it compared the quality of those pages to comparable pages on Brittanica.
>
> The Nature study is also consistent with my personal experience (and that of most people I talk to) as a user of Wikipedia. In other words, whenever I go to a Wikipedia page on a topic that I know well, I find it to be good quality and accurate.
>
> So... Until someone shows me a scientifically sound demonstration that Wikipedia pages are of significantly lower quality than other more traditional resources, I will continue claiming that wikis like Wikipedia can lead to quality content.

They lead to high-quality content. They also lead to low-quality
content. They lead to all sorts of content.

The vast majority of Wikipedia articles are low-to-mid quality, with
most being also quite short; but this is simply because there are so
very many of them. Despite the wiki technology, most of these articles
only have one or two primary authors, as well. Most of this "long
tail" of short articles are not comparable to Britannica or other
encyclopedias, simply because Britannica does not include topics such
as every episode of popular TV shows as "articles". I don't know of
any published studies "proving" this distribution of quality per se --
in part because it's really hard to measure -- but it's pretty well
known by contributors. You also, of course, have to come up with a
good definition of "quality" -- obviously, Wikipedia meets some
definitions, but completely fails others (like written by experts in
the field).

In part because of this, and because of other factors, the English
Wikipedia is not, perhaps, the best wiki to look at when trying to
make a statement about wiki technology in general. EN:WP has a highly
developed ruleset, culture, practices and visibility that can't
necessary be carried over to talking about other wikis, such as those
with a smaller or restricted community. For communities, are you
talking about wikis that foster communities that already exist (such
as within a company) or wikis that *create* community by existing
(like Wikipedia?) The latter seems true enough by example; I'm not
sure about the former.

By the way, the 1.0 project that Brian pointed to was rating by topic
(trying to find good articles, not poor ones) and trying to find core
articles that could be included in a CD version of WP. They worked
with the WikiProjects and on their own to come up with lists within
subjects. They've also been working on a "core topics" list:
http://en.wikipedia.org/wiki/Wikipedia:Version_1.0_Editorial_Team/Core_topics
(and a variation) http://en.wikipedia.org/wiki/Wikipedia:Core_topics_-_1%2C000

Considering that the Nature study actually relied on a very similar
methodology -- asking experts to subjectively rate articles -- it
seems like a pretty valid comparison. The reviewers working on 1.0 are
academics and experienced Wikipedians; the people working in the
WikiProjects presumably have some subject knowledge.

(I think Brian and I are proving the point here that if you want
detailed criticism of Wikipedia, ask some long-time contributors :) )

best,
phoebe

_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergence of communities

phoebe ayers
In reply to this post by Piotr Konieczny-2
On 8/29/07, Piotr Konieczny <[hidden email]> wrote:

> Desilets, Alain wrote:
>
> > I need a good solid reference to substantiate the following claim:
> >
> > "Besides leading to high quality content, wikis have been shown to be good tools for fostering the emergence of active communities"
> >
> > Does anyone know of a good research paper that looks specifically at this kind of impact of wikis?
>
> Check: WP:ACST for a list of articles with abstracts. There should be a
> few suitable.
>
Also, for those who don't know, the Wiki Research Database is the most
comprehensive collection around:
http://bibliography.wikimedia.de/

There's also more studies linked here: http://meta.wikimedia.org/wiki/Research

-- phoebe

_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergence of communities

Janet Hawtin
wiki editing is a skill, it is a technical skill but more so a social skill.

different wikis will have different kinds of material and community based on

- the initial goal of the project,

- how clearly that is defined and whether

- the participants understand and agree with the purpose and therefore
the criteria for editing..this would be the case with a wikipedia
modelled project.
- if the project uses different criteria for editing, eg has
authorised people or some other means of managing choices the outcomes
and participation will vary accordingly.

- the size diversity and commitment/interest of the audience

- the skills of the participants to express themselves effectively in
a space where
negotiating included content on the basis of fit for purpose, or
whatever model is used for
choosing and editing material.

_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergenceofcommunities

alain_desilets
In reply to this post by Andrea Forte
> I don't have a specific reference for you, just some
> comments. :) I think perhaps your original claim is too
> strong--on both counts.
> Including the word "can" as you did in your reply is
> important. Wikis CAN lead to high-quality content. And CAN
> lead to active communities.

Yes, I came to the same realization after writing my reply. I have
started quite a few wiki sites that never achieved either quality nor
active communities ;-).

> (Clearly there are examples of active communities and
> examples of wikis that never get used much.) I am not
> convinced that the technology itself leads to high-quality
> articles in Wikipedia, and I don't think that the Nature
> article supports that claim in any way.

When I talk about "wiki", I am not talking about just the technology. I
am also talking about the wiki way and philosophy (which, btw, I
consider much more significant than the technology that we currently use
to support that wiki way and philosophy).

> One could hypothesize about characteristics of wiki that help
> support the goal of producing high-quality articles and try
> to empirically verify the claim, but to my knowledge no one
> has done so in any systematic fashion.

Well, there are a number of wiki patterns and anti-patterns that have
been written here:

www.wikipatterns.com

But that's not exactly a scientific analysis of what works and doesn't.

Oh welll... Maybe I'll just leave it as a "bold" claim.

Thx Andrea.

Alain

_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: [wiki-research] Wikis as a tool for fostering emergence ofcommunities

alain_desilets
In reply to this post by alain_desilets
Yes, I would like a copy of the paper, thx.

> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of Derek Hansen
> Sent: August 29, 2007 1:28 PM
> To: Discussion of wiki research and practice
> Cc: Research into Wikimedia content and communities
> Subject: Re: [wiki-research] Wikis as a tool for fostering
> emergence ofcommunities
>
> I will be presenting a paper "Virtual Community Maintenance
> with a Repository" at the ASIST conference in October that
> discusses the ways in which a wiki repository has helped
> strengthen an email-based technical support community. The
> abstract is found at
> http://www.asis.org/Conferences/AM07/papers/72.html
>
> If you would like a copy of the paper let me know.
>
> Derek L. Hansen
> Assistant Professor
> University of Maryland
>
> On 8/29/07, Desilets, Alain <[hidden email]> wrote:
> > I need a good solid reference to substantiate the following claim:
> >
> > "Besides leading to high quality content, wikis have been
> shown to be good tools for fostering the emergence of active
> communities"
> >
> > Does anyone know of a good research paper that looks
> specifically at this kind of impact of wikis?
> >
> > Thx.
> >
> >
> > ----
> > Alain Désilets, National Research Council of Canada Chair, WikiSym
> > 2007
> >
> > 2007 International Symposium on Wikis
> > Wikis at Work in the World:
> > Open, Organic, Participatory Media for the 21st Century
> >
> > http://www.wikisym.org/ws2007/
> >
> > _______________________________________________
> >
> > wiki-research mailing list, [hidden email]
> > http://www.wikisym.org/mailman/listinfo/wiki-research
> >
> > For the wiki-research, wiki-standards, wikisym-announce
> mailing lists, please see:
> > http://www.wikisym.org/cgi-bin/mailman/listinfo
> >
> _______________________________________________
>
> wiki-research mailing list, [hidden email]
> http://www.wikisym.org/mailman/listinfo/wiki-research
>
> For the wiki-research, wiki-standards, wikisym-announce
> mailing lists, please see:
> http://www.wikisym.org/cgi-bin/mailman/listinfo
>

_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergence of communities

alain_desilets
In reply to this post by Piotr Konieczny-2
> Check: WP:ACST for a list of articles with abstracts. There
> should be a few suitable.

Sorry, what does WP:ACST?

> "The problem about Wikipedia is, that it just works in
> reality, not in
> theory."

BTW: I love that quote. Is it yours? I'll be using it a lot in the future!


----
Alain Désilets, National Research Council of Canada
Chair, WikiSym 2007

2007 International Symposium on Wikis
Wikis at Work in the World:
Open, Organic, Participatory Media for the 21st Century

http://www.wikisym.org/ws2007/


_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergence ofcommunities

alain_desilets
In reply to this post by Brian J Mingus
> I am thinking more along the lines of the loss of quality of
previously high quality
> articles, which are already incredibly small in proportion, such as
"featured articles."
> Traditional content production methods asymptote in quality, but the
editing process in
> place at Wikipedia (which is only one possible wiki process, and also
one of the most
> successful, but does not necessarily speak about wikis in general)
encourages articles to
> gradually increase in quality, and then again decrease. It is unknown
if they will
> stabilize (which brings about thoughts of a 1.0)
> There are plenty of examples of this phenomenon:
> http://en.wikipedia.org/wiki/Wikipedia:Former_featured_articles

I'm sure there are plenty of examples of that too.

Does that mean that Wikipedia as a whole is low quality? No.

The demoted articles could still be very high quality, and comparable to
equivalent entries in Brittanica.

The demoted articles could be a very small portion of the featured
articles.

> This could be due to changing featured article criteria, but in
general, the claim that
> simply starting a wiki encourages high quality content is lacking
evidence. If anything,
> wikis encourage the addition of noise to high quality content. Adding
noise to turing
> complete wiki syntax can quickly snowball, turning into an aggregation
of media that lacks
> coherence.

The fact that wiki technology and social process were able to reproduce
Brittanica in 3 years is good enough evidence for me. And it's an
evidence that has been measured objectively by the Nature study.

In fact, I would say that by now, the burden of the proof is on those
who claim that Wikipedia is poor quality and that the wiki process and
technology is not working. To date, I haven't seen any solid objective
empirical evidence to that effect.

But I am opened to persuasion.

Alain

BTW: I agree with Andrea that wiki does not AUTOMATICALLY lead to
quality content and active communities. Creating a community that works
is still a bit of a black art. The point is that it CAN work, which came
as a surprise to most people, including myself.

_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergenceofcommunities

alain_desilets
In reply to this post by phoebe ayers
> The vast majority of Wikipedia articles are low-to-mid
> quality, with most being also quite short; but this is simply
> because there are so very many of them. Despite the wiki
> technology, most of these articles only have one or two
> primary authors, as well. Most of this "long tail" of short
> articles are not comparable to Britannica or other
> encyclopedias, simply because Britannica does not include
> topics such as every episode of popular TV shows as
> "articles".

I agree. That's why when we talk about the quality of WP versus traditional sources, it's important to focus on content that is covered by both types of resources.

The fact that Wikipedia covers content that is not covered by Brittanica should not be held against the approach. It should be considered a strength, not a weakness!

> You also, of course, have to come up with a
> good definition of "quality" -- obviously, Wikipedia meets
> some definitions, but completely fails others (like written
> by experts in the field).

"Written by experts in the field" is a process-oriented definition of quality. Such definitions are always highly suspicious. They're often promoted by tool and process vendors who want to equate quality with their particular brand of tool or process.

It's much better to define quality of a product in terms of attributes of the end product than in terms of the process by which it was achieved.

Rereading the Nature article, I noticed that they talk about two dimensions of quality:

- Accuracy of the information
- Clarity of presentation

While WP seems to do very well on accuracy, it seems many of the expert reviewers in the study complained about Clarity. So maybe the claim should be that:

"Experience shows that wikis *can* lead to *comprehensive and accurate* content"

> In part because of this, and because of other factors, the
> English Wikipedia is not, perhaps, the best wiki to look at
> when trying to make a statement about wiki technology in
> general. EN:WP has a highly developed ruleset, culture,
> practices and visibility that can't necessary be carried over
> to talking about other wikis, such as those with a smaller or
> restricted community. For communities, are you talking about
> wikis that foster communities that already exist (such as
> within a company) or wikis that *create* community by
> existing (like Wikipedia?) The latter seems true enough by
> example; I'm not sure about the former.

I'm making the claim in the context of this article:

http://wikimania2007.wikimedia.org/wiki/Proceedings:AD1

Which talks about creating a world-wide community of translators.


>
> By the way, the 1.0 project that Brian pointed to was rating
> by topic (trying to find good articles, not poor ones) and
> trying to find core articles that could be included in a CD
> version of WP. They worked with the WikiProjects and on their
> own to come up with lists within subjects. They've also been
> working on a "core topics" list:
> http://en.wikipedia.org/wiki/Wikipedia:Version_1.0_Editorial_T
> eam/Core_topics
> (and a variation)
> http://en.wikipedia.org/wiki/Wikipedia:Core_topics_-_1%2C000
>
> Considering that the Nature study actually relied on a very
> similar methodology -- asking experts to subjectively rate
> articles -- it seems like a pretty valid comparison. The
> reviewers working on 1.0 are academics and experienced
> Wikipedians; the people working in the WikiProjects
> presumably have some subject knowledge.

OK, thx for the precisions. One way that this differs from the Nature study is that it does not compare WP's quality on those articles to the quality level on Brittanica.

What is your impression on that? Would the low quality articles on that list fail the test of being comparable to Brittanica in terms of accuracy?

> (I think Brian and I are proving the point here that if you
> want detailed criticism of Wikipedia, ask some long-time
> contributors :) )

Such self-criticism is a sure sign of a healthy community!

----
Alain Désilets, National Research Council of Canada
Chair, WikiSym 2007

2007 International Symposium on Wikis
Wikis at Work in the World:
Open, Organic, Participatory Media for the 21st Century

http://www.wikisym.org/ws2007/


_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergence of communities

Angela-5
In reply to this post by alain_desilets
On 8/30/07, Desilets, Alain <[hidden email]> wrote:
> > Check: WP:ACST for a list of articles with abstracts. There
> > should be a few suitable.
>
> Sorry, what does WP:ACST?

It means the "Wikipedia in academic studies" page at
<http://en.wikipedia.org/wiki/Wikipedia:Wikipedia_in_academic_studies>

WP:ACST is a shortcut link there - wikipedia.org/wiki/WP:ACST

Angela

_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergence ofcommunities

alain_desilets
Thx.

> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On
> Behalf Of Angela
> Sent: August 30, 2007 8:44 AM
> To: Research into Wikimedia content and communities
> Subject: Re: [Wiki-research-l] Wikis as a tool for fostering
> emergence ofcommunities
>
> On 8/30/07, Desilets, Alain <[hidden email]> wrote:
> > > Check: WP:ACST for a list of articles with abstracts.
> There should
> > > be a few suitable.
> >
> > Sorry, what does WP:ACST?
>
> It means the "Wikipedia in academic studies" page at
> <http://en.wikipedia.org/wiki/Wikipedia:Wikipedia_in_academic_studies>
>
> WP:ACST is a shortcut link there - wikipedia.org/wiki/WP:ACST
>
> Angela
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>

_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergence of communities

Andrea Forte
In reply to this post by alain_desilets
I see that quote all over. :) It's kind of funny to me because it
often seems to be used in a way that encourages the idea that
wikipedia working is mysterious and magical. As an academic it seems
like a pretty plain and straightforward statement: Our theories don't
seem to account for this, so there's more work to be done on our
theories. Yay.

(I actually think there's a significant body of theory we can use to
explain a lot of what's going on in Wikipedia, but it's distributed
among different disciplines with different vocabularies and difficult
to synthesize. :) )


On 8/30/07, Desilets, Alain <[hidden email]> wrote:

> > Check: WP:ACST for a list of articles with abstracts. There
> > should be a few suitable.
>
> Sorry, what does WP:ACST?
>
> > "The problem about Wikipedia is, that it just works in
> > reality, not in
> > theory."
>
> BTW: I love that quote. Is it yours? I'll be using it a lot in the future!
>
>
> ----
> Alain Désilets, National Research Council of Canada
> Chair, WikiSym 2007
>
> 2007 International Symposium on Wikis
> Wikis at Work in the World:
> Open, Organic, Participatory Media for the 21st Century
>
> http://www.wikisym.org/ws2007/
>
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>

_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergence of communities

Piotr Konieczny-2
In reply to this post by alain_desilets
Desilets, Alain wrote:

> Sorry, what does WP:ACST?

It's one of WP:SHORT's :) Which mean, shortcuts
(http://en.wikipedia.org/wiki/Wikipedia:Shortcut); this one leads to
http://en.wikipedia.org/wiki/Wikipedia:Wikipedia_in_academic_studies, a
project I adopted and updated this summer: in theory, it's supposed to
hold a list of all academic works mentioning Wikipedia - in practice, we
all need to keep updating it, just as any wiki page :)

>>"The problem about Wikipedia is, that it just works in
>>reality, not in
>>theory."
>
>
> BTW: I love that quote. Is it yours? I'll be using it a lot in the future!

According to http://en.wikiquote.org/wiki/Wikipedia it is attributed
(but not sourced!) to Stephen Colbert. I may not be a fan of the man,
but the quote *is* good (I use it as a motto of my Wikipedia MT and
article based on it, recently submitted for review).

--
Piotr Konieczny

"The problem about Wikipedia is, that it just works in reality, not in
theory."

_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikis as a tool for fostering emergence of communities

Kat Walsh-4
In reply to this post by Brian J Mingus
On 8/29/07, Brian <[hidden email]> wrote:

> I am thinking more along the lines of the loss of quality of previously high
> quality articles, which are already incredibly small in proportion, such as
> "featured articles." Traditional content production methods asymptote in
> quality, but the editing process in place at Wikipedia (which is only one
> possible wiki process, and also one of the most successful, but does not
> necessarily speak about wikis in general) encourages articles to gradually
> increase in quality, and then again decrease. It is unknown if they will
> stabilize (which brings about thoughts of a 1.0)
>
> There are plenty of examples of this phenomenon:
> http://en.wikipedia.org/wiki/Wikipedia:Former_featured_articles
>
> This could be due to changing featured article criteria, but in general, the
> claim that simply starting a wiki encourages high quality content is lacking
> evidence. If anything, wikis encourage the addition of noise to high quality
> content. Adding noise to turing complete wiki syntax can quickly snowball,
> turning into an aggregation of media that lacks coherence.

I think that monitoring featured articles is a poor metric for quality
of Wikipedia articles in general, as getting an article "featured" on
English Wikipedia is not solely a function of article quality -- some
poor work gets through and some good work will never get through
(because the subject doesn't merit a long article, for example), or in
some cases those writing articles have no desire to put them through
the featured article process.

The ratings by the individual wikiprojects, while still wildly
variable and idiosyncratic, are probably a better guide.

-Kat

--
Wikimedia needs you: http://wikimediafoundation.org/wiki/Fundraising
* *  * *  * *  * *  * *  * *  * *  * *  * *  * *  * *  * *  * *  * *
http://en.wikipedia.org/wiki/User:Mindspillage | (G)AIM:Mindspillage
mindspillage or mind|wandering on irc.freenode.net | email for phone

_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l
12