Quality on different language version

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Quality on different language version

Anders Wennersten-2
(reposted from Wikimedia-i)

I have several times asked for a professional quality study of our
different language versions, but not seen it exist or being done,
perhaps you know more on this list?. before we start the strategy work
I  believe we should have basic facts on the table like this one

I therefor list here my subjective impression after daily looking into
the different version for 5-15 articles (new ones being created on
sv.wp) (I list them in order how often I use them to calibrate the svwp
articles).

enwp- a magnitude better then any other. main weakeness are articles on
marginal subjects that seems to be allowed to exist there, even if
rather bad, and without templates (noone cares to patrol these?)

eswp - a very  good version, which in the general discussion are not
getting appropriate credit

dewp - good when the articles exist, but many serious holes. Is the
elitist way of running it, discouraging new editors in non obvious
subjects (that after time passes gets very relevant)?
frwp - also good, but somewhat scattered quality both in coverage and
the different articles (even in same subject area)
nlwp - very good coverage in the geographic subjects, decent quality on
articles but limited "world" coverage in areas like biographies
itwp - good articles but a bit italiancentered,

nowp - small but decent articles. Their short focused articletext
sometimes give more easyaccessed knowledge then an overly long one in
other languages

ptwp - the real disappointment. it is among the top ten in volume and
accesses but clearly missing a lot, and even existing articles are
uneven. I now use it even less then Ukrainian and Russian which I use
very seldom as the different alphabet makes it hard to understand the
article content

dawp,fiwp and plwp -Ok but only used by me for articles related to the
country

(arabic, chinese and japanese I almost never use, too complicated)

(I also use some smaller ones like sqwp , in these versions I have seen
serious quality problems not to be found in any of the above ones, I am
not sure they even have basic patrolling in place)

Anders

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Quality on different language version

Heather Ford-3
Hi Anders,

Yes, it's a great question! Mark Graham and I are currently working on a project around how to determine quality within and between Wikipedias and I've been looking around for literature. I'm only just starting the literature review but I've found some interesting studies by Callahan & Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority of quality studies, we find, have been done on English Wikipedia (starting with the famous 2005 Nature study) but there have been few studies that assess of quality between languages. If you find anything else, let us know!

Thanks!

Best,



On 10 June 2014 07:58, Anders Wennersten <[hidden email]> wrote:
(reposted from Wikimedia-i)

I have several times asked for a professional quality study of our different language versions, but not seen it exist or being done, perhaps you know more on this list?. before we start the strategy work I  believe we should have basic facts on the table like this one

I therefor list here my subjective impression after daily looking into the different version for 5-15 articles (new ones being created on sv.wp) (I list them in order how often I use them to calibrate the svwp articles).

enwp- a magnitude better then any other. main weakeness are articles on marginal subjects that seems to be allowed to exist there, even if rather bad, and without templates (noone cares to patrol these?)

eswp - a very  good version, which in the general discussion are not getting appropriate credit

dewp - good when the articles exist, but many serious holes. Is the elitist way of running it, discouraging new editors in non obvious subjects (that after time passes gets very relevant)?
frwp - also good, but somewhat scattered quality both in coverage and the different articles (even in same subject area)
nlwp - very good coverage in the geographic subjects, decent quality on articles but limited "world" coverage in areas like biographies
itwp - good articles but a bit italiancentered,

nowp - small but decent articles. Their short focused articletext sometimes give more easyaccessed knowledge then an overly long one in other languages

ptwp - the real disappointment. it is among the top ten in volume and accesses but clearly missing a lot, and even existing articles are uneven. I now use it even less then Ukrainian and Russian which I use very seldom as the different alphabet makes it hard to understand the article content

dawp,fiwp and plwp -Ok but only used by me for articles related to the country

(arabic, chinese and japanese I almost never use, too complicated)

(I also use some smaller ones like sqwp , in these versions I have seen serious quality problems not to be found in any of the above ones, I am not sure they even have basic patrolling in place)

Anders

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Quality on different language version

Anders Wennersten-2
Thanks for answer

Your answer confirm my "fear", that focus is almost completly to en:wp and how it is compared with an ideal perfect Q

My interest and what I believe the movement need before we dig into next round of strategy round is
*what versions are dysfunctional. These represent a risk for the movement as they can jeoprdaize the brand name, as they are not living up to basic Q (and NPOV)
*what can we learn from each other, why are some better in some aspects and worse in others?

I would recommend a research approach much more basic just collecting some few data on each version (and forget about enwp)

Anders



Heather Ford skrev 2014-06-10 13:09:
Hi Anders,

Yes, it's a great question! Mark Graham and I are currently working on a project around how to determine quality within and between Wikipedias and I've been looking around for literature. I'm only just starting the literature review but I've found some interesting studies by Callahan & Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority of quality studies, we find, have been done on English Wikipedia (starting with the famous 2005 Nature study) but there have been few studies that assess of quality between languages. If you find anything else, let us know!

Thanks!

Best,



On 10 June 2014 07:58, Anders Wennersten <[hidden email]> wrote:
(reposted from Wikimedia-i)

I have several times asked for a professional quality study of our different language versions, but not seen it exist or being done, perhaps you know more on this list?. before we start the strategy work I  believe we should have basic facts on the table like this one

I therefor list here my subjective impression after daily looking into the different version for 5-15 articles (new ones being created on sv.wp) (I list them in order how often I use them to calibrate the svwp articles).

enwp- a magnitude better then any other. main weakeness are articles on marginal subjects that seems to be allowed to exist there, even if rather bad, and without templates (noone cares to patrol these?)

eswp - a very  good version, which in the general discussion are not getting appropriate credit

dewp - good when the articles exist, but many serious holes. Is the elitist way of running it, discouraging new editors in non obvious subjects (that after time passes gets very relevant)?
frwp - also good, but somewhat scattered quality both in coverage and the different articles (even in same subject area)
nlwp - very good coverage in the geographic subjects, decent quality on articles but limited "world" coverage in areas like biographies
itwp - good articles but a bit italiancentered,

nowp - small but decent articles. Their short focused articletext sometimes give more easyaccessed knowledge then an overly long one in other languages

ptwp - the real disappointment. it is among the top ten in volume and accesses but clearly missing a lot, and even existing articles are uneven. I now use it even less then Ukrainian and Russian which I use very seldom as the different alphabet makes it hard to understand the article content

dawp,fiwp and plwp -Ok but only used by me for articles related to the country

(arabic, chinese and japanese I almost never use, too complicated)

(I also use some smaller ones like sqwp , in these versions I have seen serious quality problems not to be found in any of the above ones, I am not sure they even have basic patrolling in place)

Anders

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Quality on different language version

Juliana Bastos Marques
This topic comes in handy for my research on Featured Articles in WP:PT. Maybe some of you may remember my request a little while ago about studies on Wikipedias other than English. Well, not that I believe that the Featured Article requirements are a good evaluation per se, in terms of quality of content.

Anders, what are the articles you evaluated? I'm curious to find out what was so bad in the Portuguese Wikipedia. Indeed, there are many problems there, but I'm surprised to hear that it looks so bad. I know it's a drop in the ocean, but I've been fixing some new articles that are translations from bad English ones - which look good, but analyzing the content reveals many problems.

Juliana.


On Tue, Jun 10, 2014 at 8:29 AM, Anders Wennersten <[hidden email]> wrote:
Thanks for answer

Your answer confirm my "fear", that focus is almost completly to en:wp and how it is compared with an ideal perfect Q

My interest and what I believe the movement need before we dig into next round of strategy round is
*what versions are dysfunctional. These represent a risk for the movement as they can jeoprdaize the brand name, as they are not living up to basic Q (and NPOV)
*what can we learn from each other, why are some better in some aspects and worse in others?

I would recommend a research approach much more basic just collecting some few data on each version (and forget about enwp)

Anders



Heather Ford skrev 2014-06-10 13:09:
Hi Anders,

Yes, it's a great question! Mark Graham and I are currently working on a project around how to determine quality within and between Wikipedias and I've been looking around for literature. I'm only just starting the literature review but I've found some interesting studies by Callahan & Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority of quality studies, we find, have been done on English Wikipedia (starting with the famous 2005 Nature study) but there have been few studies that assess of quality between languages. If you find anything else, let us know!

Thanks!

Best,



On 10 June 2014 07:58, Anders Wennersten <[hidden email]> wrote:
(reposted from Wikimedia-i)

I have several times asked for a professional quality study of our different language versions, but not seen it exist or being done, perhaps you know more on this list?. before we start the strategy work I  believe we should have basic facts on the table like this one

I therefor list here my subjective impression after daily looking into the different version for 5-15 articles (new ones being created on sv.wp) (I list them in order how often I use them to calibrate the svwp articles).

enwp- a magnitude better then any other. main weakeness are articles on marginal subjects that seems to be allowed to exist there, even if rather bad, and without templates (noone cares to patrol these?)

eswp - a very  good version, which in the general discussion are not getting appropriate credit

dewp - good when the articles exist, but many serious holes. Is the elitist way of running it, discouraging new editors in non obvious subjects (that after time passes gets very relevant)?
frwp - also good, but somewhat scattered quality both in coverage and the different articles (even in same subject area)
nlwp - very good coverage in the geographic subjects, decent quality on articles but limited "world" coverage in areas like biographies
itwp - good articles but a bit italiancentered,

nowp - small but decent articles. Their short focused articletext sometimes give more easyaccessed knowledge then an overly long one in other languages

ptwp - the real disappointment. it is among the top ten in volume and accesses but clearly missing a lot, and even existing articles are uneven. I now use it even less then Ukrainian and Russian which I use very seldom as the different alphabet makes it hard to understand the article content

dawp,fiwp and plwp -Ok but only used by me for articles related to the country

(arabic, chinese and japanese I almost never use, too complicated)

(I also use some smaller ones like sqwp , in these versions I have seen serious quality problems not to be found in any of the above ones, I am not sure they even have basic patrolling in place)

Anders

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l




--
www.domusaurea.org

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Quality on different language version

Anders Wennersten-2
My starting point have been the newly created articles on svwp. They will represent the usual bunch of football playser, tv-stars, computergames, films  etc where svwp are behind most versions but where enwp is excellent. The interesting comparisons comes from the next levels of articles that can be almost anything, a footballstadium in Kazan Russia, an albanian poet, a church in Venize, a specie with unclear taxonomy, the american solider who perhaps deserted etc. In these cases I only often find a corresponding article in enwp, but also very often (around 20%) I find it in another version and no presence in enwp.

And when enwp is not giving me support, I most often find support in eswp and frwp, sometimes in dewp, but almost never in ptwp. For exemple taxanomical threes  with name in native and latin is about the weakest in ptwp. But I can be wrong and I would love to be part in a more complete research on Q comparisons for the different versions


Anders





Juliana Bastos Marques skrev 2014-06-10 14:06:
This topic comes in handy for my research on Featured Articles in WP:PT. Maybe some of you may remember my request a little while ago about studies on Wikipedias other than English. Well, not that I believe that the Featured Article requirements are a good evaluation per se, in terms of quality of content.

Anders, what are the articles you evaluated? I'm curious to find out what was so bad in the Portuguese Wikipedia. Indeed, there are many problems there, but I'm surprised to hear that it looks so bad. I know it's a drop in the ocean, but I've been fixing some new articles that are translations from bad English ones - which look good, but analyzing the content reveals many problems.

Juliana.


On Tue, Jun 10, 2014 at 8:29 AM, Anders Wennersten <[hidden email]> wrote:
Thanks for answer

Your answer confirm my "fear", that focus is almost completly to en:wp and how it is compared with an ideal perfect Q

My interest and what I believe the movement need before we dig into next round of strategy round is
*what versions are dysfunctional. These represent a risk for the movement as they can jeoprdaize the brand name, as they are not living up to basic Q (and NPOV)
*what can we learn from each other, why are some better in some aspects and worse in others?

I would recommend a research approach much more basic just collecting some few data on each version (and forget about enwp)

Anders



Heather Ford skrev 2014-06-10 13:09:
Hi Anders,

Yes, it's a great question! Mark Graham and I are currently working on a project around how to determine quality within and between Wikipedias and I've been looking around for literature. I'm only just starting the literature review but I've found some interesting studies by Callahan & Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority of quality studies, we find, have been done on English Wikipedia (starting with the famous 2005 Nature study) but there have been few studies that assess of quality between languages. If you find anything else, let us know!

Thanks!

Best,



On 10 June 2014 07:58, Anders Wennersten <[hidden email]> wrote:
(reposted from Wikimedia-i)

I have several times asked for a professional quality study of our different language versions, but not seen it exist or being done, perhaps you know more on this list?. before we start the strategy work I  believe we should have basic facts on the table like this one

I therefor list here my subjective impression after daily looking into the different version for 5-15 articles (new ones being created on sv.wp) (I list them in order how often I use them to calibrate the svwp articles).

enwp- a magnitude better then any other. main weakeness are articles on marginal subjects that seems to be allowed to exist there, even if rather bad, and without templates (noone cares to patrol these?)

eswp - a very  good version, which in the general discussion are not getting appropriate credit

dewp - good when the articles exist, but many serious holes. Is the elitist way of running it, discouraging new editors in non obvious subjects (that after time passes gets very relevant)?
frwp - also good, but somewhat scattered quality both in coverage and the different articles (even in same subject area)
nlwp - very good coverage in the geographic subjects, decent quality on articles but limited "world" coverage in areas like biographies
itwp - good articles but a bit italiancentered,

nowp - small but decent articles. Their short focused articletext sometimes give more easyaccessed knowledge then an overly long one in other languages

ptwp - the real disappointment. it is among the top ten in volume and accesses but clearly missing a lot, and even existing articles are uneven. I now use it even less then Ukrainian and Russian which I use very seldom as the different alphabet makes it hard to understand the article content

dawp,fiwp and plwp -Ok but only used by me for articles related to the country

(arabic, chinese and japanese I almost never use, too complicated)

(I also use some smaller ones like sqwp , in these versions I have seen serious quality problems not to be found in any of the above ones, I am not sure they even have basic patrolling in place)

Anders

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l




--
www.domusaurea.org


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Quality on different language version

Scott Hale
This thread seems really to be dealing with two distinct issues: coverage and quality.

On the coverage side, the best work I know of is from Brent Hecht and Darren Gergle. Especially [1] but also [2,3]. The English edition may be the greatest in size, but in reality most other languages contain many unique articles not found in English. All the editions to some extent suffer from a self-focus bias where regions where the language is spoken are better represented than regions where the language is not spoken.

Quality of existing articles is completely separate from coverage and is harder to evaluate at scale. I understand that some of the work at GroupLens (Minnesota) around SuggestBot is trying to develop metrics that can measure quality at scale in different languages (and working to develop/use different metrics in different languages as appropriate because not all editions have the same conventions for referencing styles, etc.). I don't have a specific reference here, but would say keep an eye on this area.

~Scott


[1] Hecht, B., & Gergle, D. (2010). The Tower of Babel meets Web 2.0: User-generated content and its applications in a multilingual context. Proceedings of the 28th International Conference on Human Factors in Computing Systems (pp. 291-300). New York, NY, USA: ACM. doi:http://doi.acm.org/10.1145/1753326.1753370

[2] Hecht, B., & Gergle, D. (2009). Measuring self-focus bias in community-maintained knowledge repositories. Proceedings of the Fourth International Conference on Communities and Technologies (pp. 11-20). New York, NY, USA: ACM. doi:http://doi.acm.org/10.1145/1556460.1556463

[3] Bao, P., Hecht, B., Carton, S., Quaderi, M., Horn, M., & Gergle, D. (2012). Omnipedia: Bridging the Wikipedia Language Gap. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1075-1084). New York, NY, USA: ACM. doi:10.1145/2207676.2208553



On Tue, Jun 10, 2014 at 7:34 AM, Anders Wennersten <[hidden email]> wrote:
My starting point have been the newly created articles on svwp. They will represent the usual bunch of football playser, tv-stars, computergames, films  etc where svwp are behind most versions but where enwp is excellent. The interesting comparisons comes from the next levels of articles that can be almost anything, a footballstadium in Kazan Russia, an albanian poet, a church in Venize, a specie with unclear taxonomy, the american solider who perhaps deserted etc. In these cases I only often find a corresponding article in enwp, but also very often (around 20%) I find it in another version and no presence in enwp.

And when enwp is not giving me support, I most often find support in eswp and frwp, sometimes in dewp, but almost never in ptwp. For exemple taxanomical threes  with name in native and latin is about the weakest in ptwp. But I can be wrong and I would love to be part in a more complete research on Q comparisons for the different versions


Anders





Juliana Bastos Marques skrev 2014-06-10 14:06:
This topic comes in handy for my research on Featured Articles in WP:PT. Maybe some of you may remember my request a little while ago about studies on Wikipedias other than English. Well, not that I believe that the Featured Article requirements are a good evaluation per se, in terms of quality of content.

Anders, what are the articles you evaluated? I'm curious to find out what was so bad in the Portuguese Wikipedia. Indeed, there are many problems there, but I'm surprised to hear that it looks so bad. I know it's a drop in the ocean, but I've been fixing some new articles that are translations from bad English ones - which look good, but analyzing the content reveals many problems.

Juliana.


On Tue, Jun 10, 2014 at 8:29 AM, Anders Wennersten <[hidden email]> wrote:
Thanks for answer

Your answer confirm my "fear", that focus is almost completly to en:wp and how it is compared with an ideal perfect Q

My interest and what I believe the movement need before we dig into next round of strategy round is
*what versions are dysfunctional. These represent a risk for the movement as they can jeoprdaize the brand name, as they are not living up to basic Q (and NPOV)
*what can we learn from each other, why are some better in some aspects and worse in others?

I would recommend a research approach much more basic just collecting some few data on each version (and forget about enwp)

Anders



Heather Ford skrev 2014-06-10 13:09:
Hi Anders,

Yes, it's a great question! Mark Graham and I are currently working on a project around how to determine quality within and between Wikipedias and I've been looking around for literature. I'm only just starting the literature review but I've found some interesting studies by Callahan & Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority of quality studies, we find, have been done on English Wikipedia (starting with the famous 2005 Nature study) but there have been few studies that assess of quality between languages. If you find anything else, let us know!

Thanks!

Best,



On 10 June 2014 07:58, Anders Wennersten <[hidden email]> wrote:
(reposted from Wikimedia-i)

I have several times asked for a professional quality study of our different language versions, but not seen it exist or being done, perhaps you know more on this list?. before we start the strategy work I  believe we should have basic facts on the table like this one

I therefor list here my subjective impression after daily looking into the different version for 5-15 articles (new ones being created on sv.wp) (I list them in order how often I use them to calibrate the svwp articles).

enwp- a magnitude better then any other. main weakeness are articles on marginal subjects that seems to be allowed to exist there, even if rather bad, and without templates (noone cares to patrol these?)

eswp - a very  good version, which in the general discussion are not getting appropriate credit

dewp - good when the articles exist, but many serious holes. Is the elitist way of running it, discouraging new editors in non obvious subjects (that after time passes gets very relevant)?
frwp - also good, but somewhat scattered quality both in coverage and the different articles (even in same subject area)
nlwp - very good coverage in the geographic subjects, decent quality on articles but limited "world" coverage in areas like biographies
itwp - good articles but a bit italiancentered,

nowp - small but decent articles. Their short focused articletext sometimes give more easyaccessed knowledge then an overly long one in other languages

ptwp - the real disappointment. it is among the top ten in volume and accesses but clearly missing a lot, and even existing articles are uneven. I now use it even less then Ukrainian and Russian which I use very seldom as the different alphabet makes it hard to understand the article content

dawp,fiwp and plwp -Ok but only used by me for articles related to the country

(arabic, chinese and japanese I almost never use, too complicated)

(I also use some smaller ones like sqwp , in these versions I have seen serious quality problems not to be found in any of the above ones, I am not sure they even have basic patrolling in place)

Anders

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l




--
www.domusaurea.org


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l




--
Scott Hale
Oxford Internet Institute
University of Oxford
http://www.scotthale.net/
[hidden email]

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Quality on different language version

Ziko van Dijk-3
In reply to this post by Anders Wennersten-2
Dear Anders,

Thank you for bringing this up. My experience is that there is still a
huge gap between computer-based quantity-oriented studies and
human-based selective sample quality-oriented studies.

I published in 2009 a paper on "small" or "weak" articles but I am
afraid that it was too much a numbers' game. It contained a table that
differentiated between large, middle-sized, small and mini Wikipedias,
with some assumptions on the quality and the power to cover topics.

Last year I started with a paper but the publishers seemed not to
finish their project. I compared the notability criteria of en, de,
nl, af (Afrikaans) and fo (Frisian) and found out that they are
actually very comparable, as far as they can be compared at all. The
often assumed "severity" of de.wp on notability seems to be a myth,
maybe based on anti German cliche.

Now I have made for my lectures a table of small and larger
encyclopedic articles in order to compare a topic in different
reference works. Reason for this is also my contribution to the
Historians' Convention (Historikertag) later this year. My basic
question is whether Wikipedia is a good starting point for a historic
topic, following the research of early deceased Swiss historian Peter
Haber.

Haber made his point i.a. at the example of [[de:Frühmittelalter]]
(early middle ages) in 2010. That article, he complained, contained no
real inaccuracies, but still it was useless for a student. No good
structure, some facts put one after the other etc. His explanation: if
you want to write an article about a person, say about Henri Dunant,
you take some biographies and write from his birth to his death and
legacy. That's relatively easy and can be done by any good writer. But
for a comprehensive article on the early middle ages, you must be a
skilled historian very familiar with the period.

(I now experience the same with a series of Wikipedia articles I write
about a certain period in German history. Just following the (older)
standard reference works would simply not make me happy, not be a
really valuable contribution to Wikipedia. With (nearly) every new
work I get from the inter library loan I see that it is good to wait
with publication of an article until I have together the set of works
I deem necessary. - I consider to write a kind of report about this
series.)

It would be great to have a set of criteria for an article typology,
based partially on function of the article (overview, or registration
of an item in a row etc.) and the inner quality (structure,
comprehensiveness, based on literature etc.).

Kind regards
Ziko




























2014-06-10 14:34 GMT+02:00 Anders Wennersten <[hidden email]>:

> My starting point have been the newly created articles on svwp. They will
> represent the usual bunch of football playser, tv-stars, computergames,
> films  etc where svwp are behind most versions but where enwp is excellent.
> The interesting comparisons comes from the next levels of articles that can
> be almost anything, a footballstadium in Kazan Russia, an albanian poet, a
> church in Venize, a specie with unclear taxonomy, the american solider who
> perhaps deserted etc. In these cases I only often find a corresponding
> article in enwp, but also very often (around 20%) I find it in another
> version and no presence in enwp.
>
> And when enwp is not giving me support, I most often find support in eswp
> and frwp, sometimes in dewp, but almost never in ptwp. For exemple
> taxanomical threes  with name in native and latin is about the weakest in
> ptwp. But I can be wrong and I would love to be part in a more complete
> research on Q comparisons for the different versions
>
>
> Anders
>
>
>
>
>
> Juliana Bastos Marques skrev 2014-06-10 14:06:
>
> This topic comes in handy for my research on Featured Articles in WP:PT.
> Maybe some of you may remember my request a little while ago about studies
> on Wikipedias other than English. Well, not that I believe that the Featured
> Article requirements are a good evaluation per se, in terms of quality of
> content.
>
> Anders, what are the articles you evaluated? I'm curious to find out what
> was so bad in the Portuguese Wikipedia. Indeed, there are many problems
> there, but I'm surprised to hear that it looks so bad. I know it's a drop in
> the ocean, but I've been fixing some new articles that are translations from
> bad English ones - which look good, but analyzing the content reveals many
> problems.
>
> Juliana.
>
>
> On Tue, Jun 10, 2014 at 8:29 AM, Anders Wennersten
> <[hidden email]> wrote:
>>
>> Thanks for answer
>>
>> Your answer confirm my "fear", that focus is almost completly to en:wp and
>> how it is compared with an ideal perfect Q
>>
>> My interest and what I believe the movement need before we dig into next
>> round of strategy round is
>> *what versions are dysfunctional. These represent a risk for the movement
>> as they can jeoprdaize the brand name, as they are not living up to basic Q
>> (and NPOV)
>> *what can we learn from each other, why are some better in some aspects
>> and worse in others?
>>
>> I would recommend a research approach much more basic just collecting some
>> few data on each version (and forget about enwp)
>>
>> Anders
>>
>>
>>
>> Heather Ford skrev 2014-06-10 13:09:
>>
>> Hi Anders,
>>
>> Yes, it's a great question! Mark Graham and I are currently working on a
>> project around how to determine quality within and between Wikipedias and
>> I've been looking around for literature. I'm only just starting the
>> literature review but I've found some interesting studies by Callahan &
>> Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority of
>> quality studies, we find, have been done on English Wikipedia (starting with
>> the famous 2005 Nature study) but there have been few studies that assess of
>> quality between languages. If you find anything else, let us know!
>>
>> Thanks!
>>
>> Best,
>> heather.
>>
>> [1] http://onlinelibrary.wiley.com/doi/10.1002/asi.21577/abstract
>>
>> [2]
>> http://www.researchgate.net/publication/200773220_Issues_of_cross-contextual_information_quality_evaluation_-_The_case_of_Arabic_English_and_Korean_Wikipedia/file/60b7d51ae682e9912a.pdf
>>
>>
>>
>>
>>
>> Heather Ford
>> Oxford Internet Institute Doctoral Programme
>> EthnographyMatters | Oxford Digital Ethnography Group
>> http://hblog.org | @hfordsa
>>
>>
>>
>>
>> On 10 June 2014 07:58, Anders Wennersten <[hidden email]> wrote:
>>>
>>> (reposted from Wikimedia-i)
>>>
>>> I have several times asked for a professional quality study of our
>>> different language versions, but not seen it exist or being done, perhaps
>>> you know more on this list?. before we start the strategy work I  believe we
>>> should have basic facts on the table like this one
>>>
>>> I therefor list here my subjective impression after daily looking into
>>> the different version for 5-15 articles (new ones being created on sv.wp) (I
>>> list them in order how often I use them to calibrate the svwp articles).
>>>
>>> enwp- a magnitude better then any other. main weakeness are articles on
>>> marginal subjects that seems to be allowed to exist there, even if rather
>>> bad, and without templates (noone cares to patrol these?)
>>>
>>> eswp - a very  good version, which in the general discussion are not
>>> getting appropriate credit
>>>
>>> dewp - good when the articles exist, but many serious holes. Is the
>>> elitist way of running it, discouraging new editors in non obvious subjects
>>> (that after time passes gets very relevant)?
>>> frwp - also good, but somewhat scattered quality both in coverage and the
>>> different articles (even in same subject area)
>>> nlwp - very good coverage in the geographic subjects, decent quality on
>>> articles but limited "world" coverage in areas like biographies
>>> itwp - good articles but a bit italiancentered,
>>>
>>> nowp - small but decent articles. Their short focused articletext
>>> sometimes give more easyaccessed knowledge then an overly long one in other
>>> languages
>>>
>>> ptwp - the real disappointment. it is among the top ten in volume and
>>> accesses but clearly missing a lot, and even existing articles are uneven. I
>>> now use it even less then Ukrainian and Russian which I use very seldom as
>>> the different alphabet makes it hard to understand the article content
>>>
>>> dawp,fiwp and plwp -Ok but only used by me for articles related to the
>>> country
>>>
>>> (arabic, chinese and japanese I almost never use, too complicated)
>>>
>>> (I also use some smaller ones like sqwp , in these versions I have seen
>>> serious quality problems not to be found in any of the above ones, I am not
>>> sure they even have basic patrolling in place)
>>>
>>> Anders
>>>
>>> _______________________________________________
>>> Wiki-research-l mailing list
>>> [hidden email]
>>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>
>>
>>
>>
>> _______________________________________________
>> Wiki-research-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>
>>
>>
>> _______________________________________________
>> Wiki-research-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>
>
>
>
> --
> www.domusaurea.org
>
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Quality on different language version

Juliana Bastos Marques
Ziko, could you please supply the full reference to Haber's article? Indeed, what I observe in History-related articles is almost a tendency towards positivism, histoire événementielle - hence, for instance, the vast number of battle themes. Discussion of historiographic approaches to concepts is usually quite rare and badly written.

Anyone willing to conduct comparative research on quality of History-related articles, please drop me a note!

Juliana.


On Tue, Jun 10, 2014 at 11:12 AM, Ziko van Dijk <[hidden email]> wrote:
Dear Anders,

Thank you for bringing this up. My experience is that there is still a
huge gap between computer-based quantity-oriented studies and
human-based selective sample quality-oriented studies.

I published in 2009 a paper on "small" or "weak" articles but I am
afraid that it was too much a numbers' game. It contained a table that
differentiated between large, middle-sized, small and mini Wikipedias,
with some assumptions on the quality and the power to cover topics.

Last year I started with a paper but the publishers seemed not to
finish their project. I compared the notability criteria of en, de,
nl, af (Afrikaans) and fo (Frisian) and found out that they are
actually very comparable, as far as they can be compared at all. The
often assumed "severity" of de.wp on notability seems to be a myth,
maybe based on anti German cliche.

Now I have made for my lectures a table of small and larger
encyclopedic articles in order to compare a topic in different
reference works. Reason for this is also my contribution to the
Historians' Convention (Historikertag) later this year. My basic
question is whether Wikipedia is a good starting point for a historic
topic, following the research of early deceased Swiss historian Peter
Haber.

Haber made his point i.a. at the example of [[de:Frühmittelalter]]
(early middle ages) in 2010. That article, he complained, contained no
real inaccuracies, but still it was useless for a student. No good
structure, some facts put one after the other etc. His explanation: if
you want to write an article about a person, say about Henri Dunant,
you take some biographies and write from his birth to his death and
legacy. That's relatively easy and can be done by any good writer. But
for a comprehensive article on the early middle ages, you must be a
skilled historian very familiar with the period.

(I now experience the same with a series of Wikipedia articles I write
about a certain period in German history. Just following the (older)
standard reference works would simply not make me happy, not be a
really valuable contribution to Wikipedia. With (nearly) every new
work I get from the inter library loan I see that it is good to wait
with publication of an article until I have together the set of works
I deem necessary. - I consider to write a kind of report about this
series.)

It would be great to have a set of criteria for an article typology,
based partially on function of the article (overview, or registration
of an item in a row etc.) and the inner quality (structure,
comprehensiveness, based on literature etc.).

Kind regards
Ziko




























2014-06-10 14:34 GMT+02:00 Anders Wennersten <[hidden email]>:
> My starting point have been the newly created articles on svwp. They will
> represent the usual bunch of football playser, tv-stars, computergames,
> films  etc where svwp are behind most versions but where enwp is excellent.
> The interesting comparisons comes from the next levels of articles that can
> be almost anything, a footballstadium in Kazan Russia, an albanian poet, a
> church in Venize, a specie with unclear taxonomy, the american solider who
> perhaps deserted etc. In these cases I only often find a corresponding
> article in enwp, but also very often (around 20%) I find it in another
> version and no presence in enwp.
>
> And when enwp is not giving me support, I most often find support in eswp
> and frwp, sometimes in dewp, but almost never in ptwp. For exemple
> taxanomical threes  with name in native and latin is about the weakest in
> ptwp. But I can be wrong and I would love to be part in a more complete
> research on Q comparisons for the different versions
>
>
> Anders
>
>
>
>
>
> Juliana Bastos Marques skrev 2014-06-10 14:06:
>
> This topic comes in handy for my research on Featured Articles in WP:PT.
> Maybe some of you may remember my request a little while ago about studies
> on Wikipedias other than English. Well, not that I believe that the Featured
> Article requirements are a good evaluation per se, in terms of quality of
> content.
>
> Anders, what are the articles you evaluated? I'm curious to find out what
> was so bad in the Portuguese Wikipedia. Indeed, there are many problems
> there, but I'm surprised to hear that it looks so bad. I know it's a drop in
> the ocean, but I've been fixing some new articles that are translations from
> bad English ones - which look good, but analyzing the content reveals many
> problems.
>
> Juliana.
>
>
> On Tue, Jun 10, 2014 at 8:29 AM, Anders Wennersten
> <[hidden email]> wrote:
>>
>> Thanks for answer
>>
>> Your answer confirm my "fear", that focus is almost completly to en:wp and
>> how it is compared with an ideal perfect Q
>>
>> My interest and what I believe the movement need before we dig into next
>> round of strategy round is
>> *what versions are dysfunctional. These represent a risk for the movement
>> as they can jeoprdaize the brand name, as they are not living up to basic Q
>> (and NPOV)
>> *what can we learn from each other, why are some better in some aspects
>> and worse in others?
>>
>> I would recommend a research approach much more basic just collecting some
>> few data on each version (and forget about enwp)
>>
>> Anders
>>
>>
>>
>> Heather Ford skrev 2014-06-10 13:09:
>>
>> Hi Anders,
>>
>> Yes, it's a great question! Mark Graham and I are currently working on a
>> project around how to determine quality within and between Wikipedias and
>> I've been looking around for literature. I'm only just starting the
>> literature review but I've found some interesting studies by Callahan &
>> Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority of
>> quality studies, we find, have been done on English Wikipedia (starting with
>> the famous 2005 Nature study) but there have been few studies that assess of
>> quality between languages. If you find anything else, let us know!
>>
>> Thanks!
>>
>> Best,
>> heather.
>>
>> [1] http://onlinelibrary.wiley.com/doi/10.1002/asi.21577/abstract
>>
>> [2]
>> http://www.researchgate.net/publication/200773220_Issues_of_cross-contextual_information_quality_evaluation_-_The_case_of_Arabic_English_and_Korean_Wikipedia/file/60b7d51ae682e9912a.pdf
>>
>>
>>
>>
>>
>> Heather Ford
>> Oxford Internet Institute Doctoral Programme
>> EthnographyMatters | Oxford Digital Ethnography Group
>> http://hblog.org | @hfordsa
>>
>>
>>
>>
>> On 10 June 2014 07:58, Anders Wennersten <[hidden email]> wrote:
>>>
>>> (reposted from Wikimedia-i)
>>>
>>> I have several times asked for a professional quality study of our
>>> different language versions, but not seen it exist or being done, perhaps
>>> you know more on this list?. before we start the strategy work I  believe we
>>> should have basic facts on the table like this one
>>>
>>> I therefor list here my subjective impression after daily looking into
>>> the different version for 5-15 articles (new ones being created on sv.wp) (I
>>> list them in order how often I use them to calibrate the svwp articles).
>>>
>>> enwp- a magnitude better then any other. main weakeness are articles on
>>> marginal subjects that seems to be allowed to exist there, even if rather
>>> bad, and without templates (noone cares to patrol these?)
>>>
>>> eswp - a very  good version, which in the general discussion are not
>>> getting appropriate credit
>>>
>>> dewp - good when the articles exist, but many serious holes. Is the
>>> elitist way of running it, discouraging new editors in non obvious subjects
>>> (that after time passes gets very relevant)?
>>> frwp - also good, but somewhat scattered quality both in coverage and the
>>> different articles (even in same subject area)
>>> nlwp - very good coverage in the geographic subjects, decent quality on
>>> articles but limited "world" coverage in areas like biographies
>>> itwp - good articles but a bit italiancentered,
>>>
>>> nowp - small but decent articles. Their short focused articletext
>>> sometimes give more easyaccessed knowledge then an overly long one in other
>>> languages
>>>
>>> ptwp - the real disappointment. it is among the top ten in volume and
>>> accesses but clearly missing a lot, and even existing articles are uneven. I
>>> now use it even less then Ukrainian and Russian which I use very seldom as
>>> the different alphabet makes it hard to understand the article content
>>>
>>> dawp,fiwp and plwp -Ok but only used by me for articles related to the
>>> country
>>>
>>> (arabic, chinese and japanese I almost never use, too complicated)
>>>
>>> (I also use some smaller ones like sqwp , in these versions I have seen
>>> serious quality problems not to be found in any of the above ones, I am not
>>> sure they even have basic patrolling in place)
>>>
>>> Anders
>>>
>>> _______________________________________________
>>> Wiki-research-l mailing list
>>> [hidden email]
>>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>
>>
>>
>>
>> _______________________________________________
>> Wiki-research-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>
>>
>>
>> _______________________________________________
>> Wiki-research-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>
>
>
>
> --
> www.domusaurea.org
>
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



--
www.domusaurea.org

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Quality on different language version

Ziko van Dijk-3
Indeed, Juliana, one example: for the complex process with regard to a
German emperor in 1848/1849, it would have been possible to write one
or several articles on e.g. the debates in the National Assembly. But
what did de.wp? Elected with [[Kaiserdeputation]], the Assembly's
delegation to the Prussian King, the most visible element of the
process.

I now see that https://de.wikipedia.org/wiki/Reichsgr%C3%BCndung is a
similar example, also an article suffering from the
Frühmittelalter-problem and other issues. "Foundation of the Empire".

* With the picture of von Werner about the proclamation on January 18,
it highlights the most visible element.

* "Staatsgründung": Very basically about the (more important) legal
proceedings of the parliament

* A huge part about the proclamation

* "Sichtweise der süddeutschen Staaten": good part, but not much
connected to the others

* "Folgen und Bewertung": a mix of what followed and a judgement. With
good elements, partially two long, some inaccuracies or improper
wordings

* Following a paragraph on the Franco-German relations (not quite
suitable here, or in larger European context of the event in question,
the foundation of the Empire). Following a list and map of the single
German states, which we already have elsewhere

* A list with literature, partially not directly related to the topic or dated

* Surprisingly many footnotes, with a certain diversity of sometimes
very general works.

Haber is this person: https://de.wikipedia.org/wiki/Peter_Haber_%28Historiker%29

In English short: http://www.hist.net/index.php?id=39&L=1

http://wiki.histnet.ch/index.php/Werkstattgespr%C3%A4ch_Wien_2010

E.g.: http://derstandard.at/1277337531926/Wer-viel-Zeit-hat-hat-bei-Wikipedia-das-Sagen

I can't find anything more specific at the moment.

Kind regards
Ziko

2014-06-10 17:07 GMT+02:00 Juliana Bastos Marques <[hidden email]>:

> Ziko, could you please supply the full reference to Haber's article? Indeed,
> what I observe in History-related articles is almost a tendency towards
> positivism, histoire événementielle - hence, for instance, the vast number
> of battle themes. Discussion of historiographic approaches to concepts is
> usually quite rare and badly written.
>
> Anyone willing to conduct comparative research on quality of History-related
> articles, please drop me a note!
>
> Juliana.
>
>
> On Tue, Jun 10, 2014 at 11:12 AM, Ziko van Dijk <[hidden email]> wrote:
>>
>> Dear Anders,
>>
>> Thank you for bringing this up. My experience is that there is still a
>> huge gap between computer-based quantity-oriented studies and
>> human-based selective sample quality-oriented studies.
>>
>> I published in 2009 a paper on "small" or "weak" articles but I am
>> afraid that it was too much a numbers' game. It contained a table that
>> differentiated between large, middle-sized, small and mini Wikipedias,
>> with some assumptions on the quality and the power to cover topics.
>>
>> Last year I started with a paper but the publishers seemed not to
>> finish their project. I compared the notability criteria of en, de,
>> nl, af (Afrikaans) and fo (Frisian) and found out that they are
>> actually very comparable, as far as they can be compared at all. The
>> often assumed "severity" of de.wp on notability seems to be a myth,
>> maybe based on anti German cliche.
>>
>> Now I have made for my lectures a table of small and larger
>> encyclopedic articles in order to compare a topic in different
>> reference works. Reason for this is also my contribution to the
>> Historians' Convention (Historikertag) later this year. My basic
>> question is whether Wikipedia is a good starting point for a historic
>> topic, following the research of early deceased Swiss historian Peter
>> Haber.
>>
>> Haber made his point i.a. at the example of [[de:Frühmittelalter]]
>> (early middle ages) in 2010. That article, he complained, contained no
>> real inaccuracies, but still it was useless for a student. No good
>> structure, some facts put one after the other etc. His explanation: if
>> you want to write an article about a person, say about Henri Dunant,
>> you take some biographies and write from his birth to his death and
>> legacy. That's relatively easy and can be done by any good writer. But
>> for a comprehensive article on the early middle ages, you must be a
>> skilled historian very familiar with the period.
>>
>> (I now experience the same with a series of Wikipedia articles I write
>> about a certain period in German history. Just following the (older)
>> standard reference works would simply not make me happy, not be a
>> really valuable contribution to Wikipedia. With (nearly) every new
>> work I get from the inter library loan I see that it is good to wait
>> with publication of an article until I have together the set of works
>> I deem necessary. - I consider to write a kind of report about this
>> series.)
>>
>> It would be great to have a set of criteria for an article typology,
>> based partially on function of the article (overview, or registration
>> of an item in a row etc.) and the inner quality (structure,
>> comprehensiveness, based on literature etc.).
>>
>> Kind regards
>> Ziko
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> 2014-06-10 14:34 GMT+02:00 Anders Wennersten <[hidden email]>:
>> > My starting point have been the newly created articles on svwp. They
>> > will
>> > represent the usual bunch of football playser, tv-stars, computergames,
>> > films  etc where svwp are behind most versions but where enwp is
>> > excellent.
>> > The interesting comparisons comes from the next levels of articles that
>> > can
>> > be almost anything, a footballstadium in Kazan Russia, an albanian poet,
>> > a
>> > church in Venize, a specie with unclear taxonomy, the american solider
>> > who
>> > perhaps deserted etc. In these cases I only often find a corresponding
>> > article in enwp, but also very often (around 20%) I find it in another
>> > version and no presence in enwp.
>> >
>> > And when enwp is not giving me support, I most often find support in
>> > eswp
>> > and frwp, sometimes in dewp, but almost never in ptwp. For exemple
>> > taxanomical threes  with name in native and latin is about the weakest
>> > in
>> > ptwp. But I can be wrong and I would love to be part in a more complete
>> > research on Q comparisons for the different versions
>> >
>> >
>> > Anders
>> >
>> >
>> >
>> >
>> >
>> > Juliana Bastos Marques skrev 2014-06-10 14:06:
>> >
>> > This topic comes in handy for my research on Featured Articles in WP:PT.
>> > Maybe some of you may remember my request a little while ago about
>> > studies
>> > on Wikipedias other than English. Well, not that I believe that the
>> > Featured
>> > Article requirements are a good evaluation per se, in terms of quality
>> > of
>> > content.
>> >
>> > Anders, what are the articles you evaluated? I'm curious to find out
>> > what
>> > was so bad in the Portuguese Wikipedia. Indeed, there are many problems
>> > there, but I'm surprised to hear that it looks so bad. I know it's a
>> > drop in
>> > the ocean, but I've been fixing some new articles that are translations
>> > from
>> > bad English ones - which look good, but analyzing the content reveals
>> > many
>> > problems.
>> >
>> > Juliana.
>> >
>> >
>> > On Tue, Jun 10, 2014 at 8:29 AM, Anders Wennersten
>> > <[hidden email]> wrote:
>> >>
>> >> Thanks for answer
>> >>
>> >> Your answer confirm my "fear", that focus is almost completly to en:wp
>> >> and
>> >> how it is compared with an ideal perfect Q
>> >>
>> >> My interest and what I believe the movement need before we dig into
>> >> next
>> >> round of strategy round is
>> >> *what versions are dysfunctional. These represent a risk for the
>> >> movement
>> >> as they can jeoprdaize the brand name, as they are not living up to
>> >> basic Q
>> >> (and NPOV)
>> >> *what can we learn from each other, why are some better in some aspects
>> >> and worse in others?
>> >>
>> >> I would recommend a research approach much more basic just collecting
>> >> some
>> >> few data on each version (and forget about enwp)
>> >>
>> >> Anders
>> >>
>> >>
>> >>
>> >> Heather Ford skrev 2014-06-10 13:09:
>> >>
>> >> Hi Anders,
>> >>
>> >> Yes, it's a great question! Mark Graham and I are currently working on
>> >> a
>> >> project around how to determine quality within and between Wikipedias
>> >> and
>> >> I've been looking around for literature. I'm only just starting the
>> >> literature review but I've found some interesting studies by Callahan &
>> >> Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The
>> >> majority of
>> >> quality studies, we find, have been done on English Wikipedia (starting
>> >> with
>> >> the famous 2005 Nature study) but there have been few studies that
>> >> assess of
>> >> quality between languages. If you find anything else, let us know!
>> >>
>> >> Thanks!
>> >>
>> >> Best,
>> >> heather.
>> >>
>> >> [1] http://onlinelibrary.wiley.com/doi/10.1002/asi.21577/abstract
>> >>
>> >> [2]
>> >>
>> >> http://www.researchgate.net/publication/200773220_Issues_of_cross-contextual_information_quality_evaluation_-_The_case_of_Arabic_English_and_Korean_Wikipedia/file/60b7d51ae682e9912a.pdf
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> Heather Ford
>> >> Oxford Internet Institute Doctoral Programme
>> >> EthnographyMatters | Oxford Digital Ethnography Group
>> >> http://hblog.org | @hfordsa
>> >>
>> >>
>> >>
>> >>
>> >> On 10 June 2014 07:58, Anders Wennersten <[hidden email]>
>> >> wrote:
>> >>>
>> >>> (reposted from Wikimedia-i)
>> >>>
>> >>> I have several times asked for a professional quality study of our
>> >>> different language versions, but not seen it exist or being done,
>> >>> perhaps
>> >>> you know more on this list?. before we start the strategy work I
>> >>> believe we
>> >>> should have basic facts on the table like this one
>> >>>
>> >>> I therefor list here my subjective impression after daily looking into
>> >>> the different version for 5-15 articles (new ones being created on
>> >>> sv.wp) (I
>> >>> list them in order how often I use them to calibrate the svwp
>> >>> articles).
>> >>>
>> >>> enwp- a magnitude better then any other. main weakeness are articles
>> >>> on
>> >>> marginal subjects that seems to be allowed to exist there, even if
>> >>> rather
>> >>> bad, and without templates (noone cares to patrol these?)
>> >>>
>> >>> eswp - a very  good version, which in the general discussion are not
>> >>> getting appropriate credit
>> >>>
>> >>> dewp - good when the articles exist, but many serious holes. Is the
>> >>> elitist way of running it, discouraging new editors in non obvious
>> >>> subjects
>> >>> (that after time passes gets very relevant)?
>> >>> frwp - also good, but somewhat scattered quality both in coverage and
>> >>> the
>> >>> different articles (even in same subject area)
>> >>> nlwp - very good coverage in the geographic subjects, decent quality
>> >>> on
>> >>> articles but limited "world" coverage in areas like biographies
>> >>> itwp - good articles but a bit italiancentered,
>> >>>
>> >>> nowp - small but decent articles. Their short focused articletext
>> >>> sometimes give more easyaccessed knowledge then an overly long one in
>> >>> other
>> >>> languages
>> >>>
>> >>> ptwp - the real disappointment. it is among the top ten in volume and
>> >>> accesses but clearly missing a lot, and even existing articles are
>> >>> uneven. I
>> >>> now use it even less then Ukrainian and Russian which I use very
>> >>> seldom as
>> >>> the different alphabet makes it hard to understand the article content
>> >>>
>> >>> dawp,fiwp and plwp -Ok but only used by me for articles related to the
>> >>> country
>> >>>
>> >>> (arabic, chinese and japanese I almost never use, too complicated)
>> >>>
>> >>> (I also use some smaller ones like sqwp , in these versions I have
>> >>> seen
>> >>> serious quality problems not to be found in any of the above ones, I
>> >>> am not
>> >>> sure they even have basic patrolling in place)
>> >>>
>> >>> Anders
>> >>>
>> >>> _______________________________________________
>> >>> Wiki-research-l mailing list
>> >>> [hidden email]
>> >>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>> >>
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> Wiki-research-l mailing list
>> >> [hidden email]
>> >> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> Wiki-research-l mailing list
>> >> [hidden email]
>> >> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>> >>
>> >
>> >
>> >
>> > --
>> > www.domusaurea.org
>> >
>> >
>> > _______________________________________________
>> > Wiki-research-l mailing list
>> > [hidden email]
>> > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>> >
>> >
>> >
>> > _______________________________________________
>> > Wiki-research-l mailing list
>> > [hidden email]
>> > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>> >
>>
>> _______________________________________________
>> Wiki-research-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>
>
>
> --
> www.domusaurea.org
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
fn
Reply | Threaded
Open this post in threaded view
|

Re: Quality on different language version

fn
In reply to this post by Heather Ford-3
Dear Heather,


In our WikiLit systematic reviews we found a few publications. I have
just made a semantic query on the WikiLit site to give you an overview:

http://wikilit.referata.com/wiki/WikiLit:Quality

There are not that many. You should find them described in our review on
research on Wikipedia content: "The sum of all human knowledge": a
systematic review of scholarly research on the content of Wikipedia

http://spectrum.library.concordia.ca/978618/1/WikiLit_Content_%2D_open_access_version.pdf

http://neuro.compute.dtu.dk/wiki/%22The_sum_of_all_human_knowledge%22:_a_systematic_review_of_scholarly_research_on_the_content_of_Wikipedia

best
Finn Årup Nielsen


On 06/10/2014 01:09 PM, Heather Ford wrote:

> Hi Anders,
>
> Yes, it's a great question! Mark Graham and I are currently working on a
> project around how to determine quality within and between Wikipedias
> and I've been looking around for literature. I'm only just starting the
> literature review but I've found some interesting studies by Callahan &
> Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The
> majority of quality studies, we find, have been done on English
> Wikipedia (starting with the famous 2005 Nature study) but there have
> been few studies that assess of quality between languages. If you find
> anything else, let us know!
>
> Thanks!
>
> Best,
> heather.
>
> [1] http://onlinelibrary.wiley.com/doi/10.1002/asi.21577/abstract
>
> [2]
> http://www.researchgate.net/publication/200773220_Issues_of_cross-contextual_information_quality_evaluation_-_The_case_of_Arabic_English_and_Korean_Wikipedia/file/60b7d51ae682e9912a.pdf
>
>
>
>
> Heather Ford
> Oxford Internet Institute <http://www.oii.ox.ac.uk> Doctoral Programme
> EthnographyMatters <http://ethnographymatters.net> | Oxford Digital
> Ethnography Group <http://www.oii.ox.ac.uk/research/projects/?id=115>
> http://hblog.org <http://hblog.org/> | @hfordsa
> <http://www.twitter.com/hfordsa>
>
>
>
>
> On 10 June 2014 07:58, Anders Wennersten <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     (reposted from Wikimedia-i)
>
>     I have several times asked for a professional quality study of our
>     different language versions, but not seen it exist or being done,
>     perhaps you know more on this list?. before we start the strategy
>     work I  believe we should have basic facts on the table like this one
>
>     I therefor list here my subjective impression after daily looking
>     into the different version for 5-15 articles (new ones being created
>     on sv.wp) (I list them in order how often I use them to calibrate
>     the svwp articles).
>
>     enwp- a magnitude better then any other. main weakeness are articles
>     on marginal subjects that seems to be allowed to exist there, even
>     if rather bad, and without templates (noone cares to patrol these?)
>
>     eswp - a very  good version, which in the general discussion are not
>     getting appropriate credit
>
>     dewp - good when the articles exist, but many serious holes. Is the
>     elitist way of running it, discouraging new editors in non obvious
>     subjects (that after time passes gets very relevant)?
>     frwp - also good, but somewhat scattered quality both in coverage
>     and the different articles (even in same subject area)
>     nlwp - very good coverage in the geographic subjects, decent quality
>     on articles but limited "world" coverage in areas like biographies
>     itwp - good articles but a bit italiancentered,
>
>     nowp - small but decent articles. Their short focused articletext
>     sometimes give more easyaccessed knowledge then an overly long one
>     in other languages
>
>     ptwp - the real disappointment. it is among the top ten in volume
>     and accesses but clearly missing a lot, and even existing articles
>     are uneven. I now use it even less then Ukrainian and Russian which
>     I use very seldom as the different alphabet makes it hard to
>     understand the article content
>
>     dawp,fiwp and plwp -Ok but only used by me for articles related to
>     the country
>
>     (arabic, chinese and japanese I almost never use, too complicated)
>
>     (I also use some smaller ones like sqwp , in these versions I have
>     seen serious quality problems not to be found in any of the above
>     ones, I am not sure they even have basic patrolling in place)
>
>     Anders
>
>     _________________________________________________
>     Wiki-research-l mailing list
>     [hidden email]
>     <mailto:[hidden email]>
>     https://lists.wikimedia.org/__mailman/listinfo/wiki-__research-l
>     <https://lists.wikimedia.org/mailman/listinfo/wiki-research-l>
>
>
>
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Quality on different language version

Anders Wennersten-2
In reply to this post by Ziko van Dijk-3
Thanks a lot for all feedback and links I will look into.

My feeling, though, is that I am more interested in what is "below the
belt", ie rottenness creeping into different versions, more then the
more acceptable quality aspect we are used looking into.

I will see if I, after reading your links, can formulate what I would
like to see (which perhaps I even an do myself) in a kind a "research
proposal" and hope I can preesnt it here to get feedback on it

Anders




Ziko van Dijk skrev 2014-06-10 16:12:

> Dear Anders,
>
> Thank you for bringing this up. My experience is that there is still a
> huge gap between computer-based quantity-oriented studies and
> human-based selective sample quality-oriented studies.
>
> I published in 2009 a paper on "small" or "weak" articles but I am
> afraid that it was too much a numbers' game. It contained a table that
> differentiated between large, middle-sized, small and mini Wikipedias,
> with some assumptions on the quality and the power to cover topics.
>
> Last year I started with a paper but the publishers seemed not to
> finish their project. I compared the notability criteria of en, de,
> nl, af (Afrikaans) and fo (Frisian) and found out that they are
> actually very comparable, as far as they can be compared at all. The
> often assumed "severity" of de.wp on notability seems to be a myth,
> maybe based on anti German cliche.
>
> Now I have made for my lectures a table of small and larger
> encyclopedic articles in order to compare a topic in different
> reference works. Reason for this is also my contribution to the
> Historians' Convention (Historikertag) later this year. My basic
> question is whether Wikipedia is a good starting point for a historic
> topic, following the research of early deceased Swiss historian Peter
> Haber.
>
> Haber made his point i.a. at the example of [[de:Frühmittelalter]]
> (early middle ages) in 2010. That article, he complained, contained no
> real inaccuracies, but still it was useless for a student. No good
> structure, some facts put one after the other etc. His explanation: if
> you want to write an article about a person, say about Henri Dunant,
> you take some biographies and write from his birth to his death and
> legacy. That's relatively easy and can be done by any good writer. But
> for a comprehensive article on the early middle ages, you must be a
> skilled historian very familiar with the period.
>
> (I now experience the same with a series of Wikipedia articles I write
> about a certain period in German history. Just following the (older)
> standard reference works would simply not make me happy, not be a
> really valuable contribution to Wikipedia. With (nearly) every new
> work I get from the inter library loan I see that it is good to wait
> with publication of an article until I have together the set of works
> I deem necessary. - I consider to write a kind of report about this
> series.)
>
> It would be great to have a set of criteria for an article typology,
> based partially on function of the article (overview, or registration
> of an item in a row etc.) and the inner quality (structure,
> comprehensiveness, based on literature etc.).
>
> Kind regards
> Ziko
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> 2014-06-10 14:34 GMT+02:00 Anders Wennersten <[hidden email]>:
>> My starting point have been the newly created articles on svwp. They will
>> represent the usual bunch of football playser, tv-stars, computergames,
>> films  etc where svwp are behind most versions but where enwp is excellent.
>> The interesting comparisons comes from the next levels of articles that can
>> be almost anything, a footballstadium in Kazan Russia, an albanian poet, a
>> church in Venize, a specie with unclear taxonomy, the american solider who
>> perhaps deserted etc. In these cases I only often find a corresponding
>> article in enwp, but also very often (around 20%) I find it in another
>> version and no presence in enwp.
>>
>> And when enwp is not giving me support, I most often find support in eswp
>> and frwp, sometimes in dewp, but almost never in ptwp. For exemple
>> taxanomical threes  with name in native and latin is about the weakest in
>> ptwp. But I can be wrong and I would love to be part in a more complete
>> research on Q comparisons for the different versions
>>
>>
>> Anders
>>
>>
>>
>>
>>
>> Juliana Bastos Marques skrev 2014-06-10 14:06:
>>
>> This topic comes in handy for my research on Featured Articles in WP:PT.
>> Maybe some of you may remember my request a little while ago about studies
>> on Wikipedias other than English. Well, not that I believe that the Featured
>> Article requirements are a good evaluation per se, in terms of quality of
>> content.
>>
>> Anders, what are the articles you evaluated? I'm curious to find out what
>> was so bad in the Portuguese Wikipedia. Indeed, there are many problems
>> there, but I'm surprised to hear that it looks so bad. I know it's a drop in
>> the ocean, but I've been fixing some new articles that are translations from
>> bad English ones - which look good, but analyzing the content reveals many
>> problems.
>>
>> Juliana.
>>
>>
>> On Tue, Jun 10, 2014 at 8:29 AM, Anders Wennersten
>> <[hidden email]> wrote:
>>> Thanks for answer
>>>
>>> Your answer confirm my "fear", that focus is almost completly to en:wp and
>>> how it is compared with an ideal perfect Q
>>>
>>> My interest and what I believe the movement need before we dig into next
>>> round of strategy round is
>>> *what versions are dysfunctional. These represent a risk for the movement
>>> as they can jeoprdaize the brand name, as they are not living up to basic Q
>>> (and NPOV)
>>> *what can we learn from each other, why are some better in some aspects
>>> and worse in others?
>>>
>>> I would recommend a research approach much more basic just collecting some
>>> few data on each version (and forget about enwp)
>>>
>>> Anders
>>>
>>>
>>>
>>> Heather Ford skrev 2014-06-10 13:09:
>>>
>>> Hi Anders,
>>>
>>> Yes, it's a great question! Mark Graham and I are currently working on a
>>> project around how to determine quality within and between Wikipedias and
>>> I've been looking around for literature. I'm only just starting the
>>> literature review but I've found some interesting studies by Callahan &
>>> Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority of
>>> quality studies, we find, have been done on English Wikipedia (starting with
>>> the famous 2005 Nature study) but there have been few studies that assess of
>>> quality between languages. If you find anything else, let us know!
>>>
>>> Thanks!
>>>
>>> Best,
>>> heather.
>>>
>>> [1] http://onlinelibrary.wiley.com/doi/10.1002/asi.21577/abstract
>>>
>>> [2]
>>> http://www.researchgate.net/publication/200773220_Issues_of_cross-contextual_information_quality_evaluation_-_The_case_of_Arabic_English_and_Korean_Wikipedia/file/60b7d51ae682e9912a.pdf
>>>
>>>
>>>
>>>
>>>
>>> Heather Ford
>>> Oxford Internet Institute Doctoral Programme
>>> EthnographyMatters | Oxford Digital Ethnography Group
>>> http://hblog.org | @hfordsa
>>>
>>>
>>>
>>>
>>> On 10 June 2014 07:58, Anders Wennersten <[hidden email]> wrote:
>>>> (reposted from Wikimedia-i)
>>>>
>>>> I have several times asked for a professional quality study of our
>>>> different language versions, but not seen it exist or being done, perhaps
>>>> you know more on this list?. before we start the strategy work I  believe we
>>>> should have basic facts on the table like this one
>>>>
>>>> I therefor list here my subjective impression after daily looking into
>>>> the different version for 5-15 articles (new ones being created on sv.wp) (I
>>>> list them in order how often I use them to calibrate the svwp articles).
>>>>
>>>> enwp- a magnitude better then any other. main weakeness are articles on
>>>> marginal subjects that seems to be allowed to exist there, even if rather
>>>> bad, and without templates (noone cares to patrol these?)
>>>>
>>>> eswp - a very  good version, which in the general discussion are not
>>>> getting appropriate credit
>>>>
>>>> dewp - good when the articles exist, but many serious holes. Is the
>>>> elitist way of running it, discouraging new editors in non obvious subjects
>>>> (that after time passes gets very relevant)?
>>>> frwp - also good, but somewhat scattered quality both in coverage and the
>>>> different articles (even in same subject area)
>>>> nlwp - very good coverage in the geographic subjects, decent quality on
>>>> articles but limited "world" coverage in areas like biographies
>>>> itwp - good articles but a bit italiancentered,
>>>>
>>>> nowp - small but decent articles. Their short focused articletext
>>>> sometimes give more easyaccessed knowledge then an overly long one in other
>>>> languages
>>>>
>>>> ptwp - the real disappointment. it is among the top ten in volume and
>>>> accesses but clearly missing a lot, and even existing articles are uneven. I
>>>> now use it even less then Ukrainian and Russian which I use very seldom as
>>>> the different alphabet makes it hard to understand the article content
>>>>
>>>> dawp,fiwp and plwp -Ok but only used by me for articles related to the
>>>> country
>>>>
>>>> (arabic, chinese and japanese I almost never use, too complicated)
>>>>
>>>> (I also use some smaller ones like sqwp , in these versions I have seen
>>>> serious quality problems not to be found in any of the above ones, I am not
>>>> sure they even have basic patrolling in place)
>>>>
>>>> Anders
>>>>
>>>> _______________________________________________
>>>> Wiki-research-l mailing list
>>>> [hidden email]
>>>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>>
>>>
>>>
>>> _______________________________________________
>>> Wiki-research-l mailing list
>>> [hidden email]
>>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>>
>>>
>>>
>>> _______________________________________________
>>> Wiki-research-l mailing list
>>> [hidden email]
>>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>>
>>
>>
>> --
>> www.domusaurea.org
>>
>>
>> _______________________________________________
>> Wiki-research-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>
>>
>>
>> _______________________________________________
>> Wiki-research-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Quality on different language version

Kerry Raymond
Having followed this thread, I am somewhat confused about what is meant by
the term "article quality", even in a single language, yet alone multiple
languages.

Sticking just to a single language for the moment ...

Do we mean that the facts presented are correct? That the kings and queens
were born and died on the dates stated?

Do we mean spelling and grammar is correct? Do we mean some kind of logical
structure? Do we mean some kind of narrative flow that "tells the story" of
the topic in a natural and engaging way?

Do we mean the use of citations? Do we mean whether the citation used
actually contains information that supports what is said by the text in the
article with which it is associated?

Do we mean some kind of "completeness" of an article? That is, it has "all"
the information. If so, what do we do if the topic is split across a number
of articles {{main|...}}}? Do we assess the group of articles? And what do
we mean by "all" anyway?

Do we mean it meets all the WP policies? Notability? Appropriate use of
external links? That the Manual of Style has been carefully followed?

Or do we mean whether it has been assessed as a stub/start/.../good article
by some review process?

Whenever I find myself in a discussion about "quality" (on any subject, not
just Wikipedia), it pretty much always boils down to "fitness for purpose as
perceived by the user". This is why surveying of users is often used to
measure quality. "How well did we serve you today?" If anyone has been
through Singapore Airport recently, you will have encountered the touch
screens asking to rate on a 1-5 scale just about everything you could
imagine, every toilet block, every immigration queue, etc. And it does have
the cleanest toilets and the fastest immigration queues, so maybe there's
something to be said for the approach.

I think we need to have some common understanding of what we mean by
quality, before we try to compare it across languages. And when we do
compare across languages, then we have to observe that the set of users
changes and presumably their needs change too.

It is interesting to note that en.WP page views have dropped consistently
since Google Knowledge (which generally displays the first para from the
en.WP article) was introduced. What this tells us is that a certain
percentage of readers of an article simply want the most basic facts, which
would be delivered even by a stub article. "Suriname is a country on the
northeastern Atlantic coast of South America" certainly met my information
needs adequately (I heard it mentioned on the TV news in connection with a
hurricane). After finding out where it was in the world, I could have gone
on to read about its colonial history, its demographic sexuality, and its
biodiversity, but I didn't because I didn't have a need to know at that
moment. My point here is that while we would not generally regard a stub as
"quality", but a percentage of the readers of a stub are probably completely
satisfied.

Of course, doing surveys of articles with real users is somewhat difficult
for a research project. But it might be useful to see how user perceptions
of quality compare with other metrics (particularly those which can be more
easily generated for a research project). Starting with other metrics,
without knowing that they are a good proxy for user perception, is probably
a waste of time.

Kerry




_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Quality on different language version

Heather Ford-3
Totally agree with you, Kerry - that there are *very* different ideas about what constitutes quality. The large diversity in research about quality taking very different variables into account is testament to that. I'm interested in your note about page views after Google Knowledge Graph. According to these stats http://stats.wikimedia.org/EN/SummaryEN.htm. Page views went up from mid-2012 to beginning of 2013 and then they went down quite sharply but seem to start rising again at the end of 2013. But perhaps you're seeing other data? Would love to hear your thoughts!

btw, for those asking about historiography, Brendan Luyt [1] has done some great work on how Wikipedia represents dominant and alternative historiographies [e.g. 2]. 

And thanks, Finn, for the great work you continue to do with Wikilit :)

Best,
Heather.




On 11 June 2014 01:19, Kerry Raymond <[hidden email]> wrote:
Having followed this thread, I am somewhat confused about what is meant by
the term "article quality", even in a single language, yet alone multiple
languages.

Sticking just to a single language for the moment ...

Do we mean that the facts presented are correct? That the kings and queens
were born and died on the dates stated?

Do we mean spelling and grammar is correct? Do we mean some kind of logical
structure? Do we mean some kind of narrative flow that "tells the story" of
the topic in a natural and engaging way?

Do we mean the use of citations? Do we mean whether the citation used
actually contains information that supports what is said by the text in the
article with which it is associated?

Do we mean some kind of "completeness" of an article? That is, it has "all"
the information. If so, what do we do if the topic is split across a number
of articles {{main|...}}}? Do we assess the group of articles? And what do
we mean by "all" anyway?

Do we mean it meets all the WP policies? Notability? Appropriate use of
external links? That the Manual of Style has been carefully followed?

Or do we mean whether it has been assessed as a stub/start/.../good article
by some review process?

Whenever I find myself in a discussion about "quality" (on any subject, not
just Wikipedia), it pretty much always boils down to "fitness for purpose as
perceived by the user". This is why surveying of users is often used to
measure quality. "How well did we serve you today?" If anyone has been
through Singapore Airport recently, you will have encountered the touch
screens asking to rate on a 1-5 scale just about everything you could
imagine, every toilet block, every immigration queue, etc. And it does have
the cleanest toilets and the fastest immigration queues, so maybe there's
something to be said for the approach.

I think we need to have some common understanding of what we mean by
quality, before we try to compare it across languages. And when we do
compare across languages, then we have to observe that the set of users
changes and presumably their needs change too.

It is interesting to note that en.WP page views have dropped consistently
since Google Knowledge (which generally displays the first para from the
en.WP article) was introduced. What this tells us is that a certain
percentage of readers of an article simply want the most basic facts, which
would be delivered even by a stub article. "Suriname is a country on the
northeastern Atlantic coast of South America" certainly met my information
needs adequately (I heard it mentioned on the TV news in connection with a
hurricane). After finding out where it was in the world, I could have gone
on to read about its colonial history, its demographic sexuality, and its
biodiversity, but I didn't because I didn't have a need to know at that
moment. My point here is that while we would not generally regard a stub as
"quality", but a percentage of the readers of a stub are probably completely
satisfied.

Of course, doing surveys of articles with real users is somewhat difficult
for a research project. But it might be useful to see how user perceptions
of quality compare with other metrics (particularly those which can be more
easily generated for a research project). Starting with other metrics,
without knowing that they are a good proxy for user perception, is probably
a waste of time.

Kerry




_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
fn
Reply | Threaded
Open this post in threaded view
|

Re: Quality on different language version

fn
In reply to this post by Kerry Raymond

Dear Kerry,


Concerning: "I think we need to have some common understanding of what
we mean by quality, before we try to compare it across languages."

When reviewing Wikipedia research we already did that. Our categories
were very much like those you suggested. Under quality
http://wikilit.referata.com/wiki/Category:Quality we have:

1) Comprehensiveness‎ (i.e., completeness or coverage)

2) Currency‎ (i.e., up-to-dateness)

3) Readability and style‎ (spelling and grammar could go here)

4) Reliability‎ (accuracy, e.g., factual errors)


And not really quality per se:

a) Antecedents of quality‎

b) Featured articles‎

In our JASIST paper in print '"The sum of all human knowledge": a
systematic review of scholarly research on the content of Wikipedia' we
furthermore has a section called 'verifiability' (i.e., 'use of sources')

In my review ("Wikipedia research and tools: Review and comments") I
have this list: accuracy (no factual errors), coverage, bias,
conciseness, readability, up-to-dateness, usable/suitable
and whether the articles are well-illustrated and
well-sourced.


Note that in our review we distinguish between "real" quality and user
perception of quality. You will see our list of studies on perception of
quality here:

http://wikilit.referata.com/wiki/Category:Reader_perceptions_of_credibility

These studies are discussed in "Wikipedia in the eyes of its beholders:
a systematic review of scholarly research on Wikipedia readers and
readership" (page 14+)


The reviews are (perhaps!?) available from (The webservers have had
problems. If a link does not work try the other one or contact us):

'"The sum of all human knowledge": a systematic review of scholarly
research on the content of Wikipedia'
http://www2.compute.dtu.dk/pubdb/views/edoc_download.php/6784/pdf/imm6784.pdf

http://spectrum.library.concordia.ca/978618/1/WikiLit_Content_%2D_open_access_version.pdf

http://neuro.compute.dtu.dk/wiki/%22The_sum_of_all_human_knowledge%22:_a_systematic_review_of_scholarly_research_on_the_content_of_Wikipedia


"Wikipedia research and tools: Review and comments"
http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/6012/pdf/imm6012.pdf


"Wikipedia in the eyes of its beholders: a systematic review of
scholarly research on Wikipedia readers and readership"

http://www2.compute.dtu.dk/pubdb/views/edoc_download.php/6785/pdf/imm6785.pdf

http://spectrum.library.concordia.ca/978617/1/Wikipedia_Readership_-_JASIST_-_open_access_version.pdf

http://neuro.compute.dtu.dk/wiki/Wikipedia_in_the_eyes_of_its_beholders:_a_systematic_review_of_scholarly_research_on_Wikipedia_readers_and_readership


best
Finn Årup Nielsen



On 06/11/2014 02:19 AM, Kerry Raymond wrote:

> Having followed this thread, I am somewhat confused about what is meant by
> the term "article quality", even in a single language, yet alone multiple
> languages.
>
> Sticking just to a single language for the moment ...
>
> Do we mean that the facts presented are correct? That the kings and queens
> were born and died on the dates stated?
>
> Do we mean spelling and grammar is correct? Do we mean some kind of logical
> structure? Do we mean some kind of narrative flow that "tells the story" of
> the topic in a natural and engaging way?
>
> Do we mean the use of citations? Do we mean whether the citation used
> actually contains information that supports what is said by the text in the
> article with which it is associated?
>
> Do we mean some kind of "completeness" of an article? That is, it has "all"
> the information. If so, what do we do if the topic is split across a number
> of articles {{main|...}}}? Do we assess the group of articles? And what do
> we mean by "all" anyway?
>
> Do we mean it meets all the WP policies? Notability? Appropriate use of
> external links? That the Manual of Style has been carefully followed?
>
> Or do we mean whether it has been assessed as a stub/start/.../good article
> by some review process?
>
> Whenever I find myself in a discussion about "quality" (on any subject, not
> just Wikipedia), it pretty much always boils down to "fitness for purpose as
> perceived by the user". This is why surveying of users is often used to
> measure quality. "How well did we serve you today?" If anyone has been
> through Singapore Airport recently, you will have encountered the touch
> screens asking to rate on a 1-5 scale just about everything you could
> imagine, every toilet block, every immigration queue, etc. And it does have
> the cleanest toilets and the fastest immigration queues, so maybe there's
> something to be said for the approach.
>
> I think we need to have some common understanding of what we mean by
> quality, before we try to compare it across languages. And when we do
> compare across languages, then we have to observe that the set of users
> changes and presumably their needs change too.
>
> It is interesting to note that en.WP page views have dropped consistently
> since Google Knowledge (which generally displays the first para from the
> en.WP article) was introduced. What this tells us is that a certain
> percentage of readers of an article simply want the most basic facts, which
> would be delivered even by a stub article. "Suriname is a country on the
> northeastern Atlantic coast of South America" certainly met my information
> needs adequately (I heard it mentioned on the TV news in connection with a
> hurricane). After finding out where it was in the world, I could have gone
> on to read about its colonial history, its demographic sexuality, and its
> biodiversity, but I didn't because I didn't have a need to know at that
> moment. My point here is that while we would not generally regard a stub as
> "quality", but a percentage of the readers of a stub are probably completely
> satisfied.
>
> Of course, doing surveys of articles with real users is somewhat difficult
> for a research project. But it might be useful to see how user perceptions
> of quality compare with other metrics (particularly those which can be more
> easily generated for a research project). Starting with other metrics,
> without knowing that they are a good proxy for user perception, is probably
> a waste of time.
>
> Kerry
>
>
>
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Quality on different language version

Kerry Raymond
In reply to this post by Heather Ford-3

My comment on Google Knowledge relates to a presentation in one of the WMF Metrics meetings and subsequent discussion (all of which can be seen on video since about 2012):

 

https://meta.wikimedia.org/wiki/Metrics_and_activities_meetings    (Heather, this is a better URL than the one I sent you by accident a moment ago)

 

There are agendas but, while they tell you who will be speaking and about what topic, they don’t provide enough info for me to say which meeting was the one when they had discussed Google Knowledge as it was probably part of an agenda item labelled something like “top line metrics”.

 

I find it’s worth viewing these videos as they give the best insight into what’s happening at WMF at the moment. And of course there is always a metrics presentation, discussion of features in development, in beta, rolled out etc.

 

Kerry

 

 


From: Heather Ford [mailto:[hidden email]]
Sent: Wednesday, 11 June 2014 6:53 PM
To: [hidden email]; Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] Quality on different language version

 

Totally agree with you, Kerry - that there are *very* different ideas about what constitutes quality. The large diversity in research about quality taking very different variables into account is testament to that. I'm interested in your note about page views after Google Knowledge Graph. According to these stats http://stats.wikimedia.org/EN/SummaryEN.htm. Page views went up from mid-2012 to beginning of 2013 and then they went down quite sharply but seem to start rising again at the end of 2013. But perhaps you're seeing other data? Would love to hear your thoughts!

 

btw, for those asking about historiography, Brendan Luyt [1] has done some great work on how Wikipedia represents dominant and alternative historiographies [e.g. 2]. 

 

And thanks, Finn, for the great work you continue to do with Wikilit :)

 

Best,

Heather.

 


 

On 11 June 2014 01:19, Kerry Raymond <[hidden email]> wrote:

Having followed this thread, I am somewhat confused about what is meant by
the term "article quality", even in a single language, yet alone multiple
languages.

Sticking just to a single language for the moment ...

Do we mean that the facts presented are correct? That the kings and queens
were born and died on the dates stated?

Do we mean spelling and grammar is correct? Do we mean some kind of logical
structure? Do we mean some kind of narrative flow that "tells the story" of
the topic in a natural and engaging way?

Do we mean the use of citations? Do we mean whether the citation used
actually contains information that supports what is said by the text in the
article with which it is associated?

Do we mean some kind of "completeness" of an article? That is, it has "all"
the information. If so, what do we do if the topic is split across a number
of articles {{main|...}}}? Do we assess the group of articles? And what do
we mean by "all" anyway?

Do we mean it meets all the WP policies? Notability? Appropriate use of
external links? That the Manual of Style has been carefully followed?

Or do we mean whether it has been assessed as a stub/start/.../good article
by some review process?

Whenever I find myself in a discussion about "quality" (on any subject, not
just Wikipedia), it pretty much always boils down to "fitness for purpose as
perceived by the user". This is why surveying of users is often used to
measure quality. "How well did we serve you today?" If anyone has been
through Singapore Airport recently, you will have encountered the touch
screens asking to rate on a 1-5 scale just about everything you could
imagine, every toilet block, every immigration queue, etc. And it does have
the cleanest toilets and the fastest immigration queues, so maybe there's
something to be said for the approach.

I think we need to have some common understanding of what we mean by
quality, before we try to compare it across languages. And when we do
compare across languages, then we have to observe that the set of users
changes and presumably their needs change too.

It is interesting to note that en.WP page views have dropped consistently
since Google Knowledge (which generally displays the first para from the
en.WP article) was introduced. What this tells us is that a certain
percentage of readers of an article simply want the most basic facts, which
would be delivered even by a stub article. "Suriname is a country on the
northeastern Atlantic coast of South America" certainly met my information
needs adequately (I heard it mentioned on the TV news in connection with a
hurricane). After finding out where it was in the world, I could have gone
on to read about its colonial history, its demographic sexuality, and its
biodiversity, but I didn't because I didn't have a need to know at that
moment. My point here is that while we would not generally regard a stub as
"quality", but a percentage of the readers of a stub are probably completely
satisfied.

Of course, doing surveys of articles with real users is somewhat difficult
for a research project. But it might be useful to see how user perceptions
of quality compare with other metrics (particularly those which can be more
easily generated for a research project). Starting with other metrics,
without knowing that they are a good proxy for user perception, is probably
a waste of time.

Kerry





_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Quality on different language version

Oliver Keyes-4
Actually, we're still doing work on whether the KnowledgeGraph stuff is having an impact, or whether it's a problem with the data source, or, or..


On 11 June 2014 14:21, Kerry Raymond <[hidden email]> wrote:

My comment on Google Knowledge relates to a presentation in one of the WMF Metrics meetings and subsequent discussion (all of which can be seen on video since about 2012):

 

https://meta.wikimedia.org/wiki/Metrics_and_activities_meetings    (Heather, this is a better URL than the one I sent you by accident a moment ago)

 

There are agendas but, while they tell you who will be speaking and about what topic, they don’t provide enough info for me to say which meeting was the one when they had discussed Google Knowledge as it was probably part of an agenda item labelled something like “top line metrics”.

 

I find it’s worth viewing these videos as they give the best insight into what’s happening at WMF at the moment. And of course there is always a metrics presentation, discussion of features in development, in beta, rolled out etc.

 

Kerry

 

 


From: Heather Ford [mailto:[hidden email]]
Sent: Wednesday, 11 June 2014 6:53 PM
To: [hidden email]; Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] Quality on different language version

 

Totally agree with you, Kerry - that there are *very* different ideas about what constitutes quality. The large diversity in research about quality taking very different variables into account is testament to that. I'm interested in your note about page views after Google Knowledge Graph. According to these stats http://stats.wikimedia.org/EN/SummaryEN.htm. Page views went up from mid-2012 to beginning of 2013 and then they went down quite sharply but seem to start rising again at the end of 2013. But perhaps you're seeing other data? Would love to hear your thoughts!

 

btw, for those asking about historiography, Brendan Luyt [1] has done some great work on how Wikipedia represents dominant and alternative historiographies [e.g. 2]. 

 

And thanks, Finn, for the great work you continue to do with Wikilit :)

 

Best,

Heather.

 


 

On 11 June 2014 01:19, Kerry Raymond <[hidden email]> wrote:

Having followed this thread, I am somewhat confused about what is meant by
the term "article quality", even in a single language, yet alone multiple
languages.

Sticking just to a single language for the moment ...

Do we mean that the facts presented are correct? That the kings and queens
were born and died on the dates stated?

Do we mean spelling and grammar is correct? Do we mean some kind of logical
structure? Do we mean some kind of narrative flow that "tells the story" of
the topic in a natural and engaging way?

Do we mean the use of citations? Do we mean whether the citation used
actually contains information that supports what is said by the text in the
article with which it is associated?

Do we mean some kind of "completeness" of an article? That is, it has "all"
the information. If so, what do we do if the topic is split across a number
of articles {{main|...}}}? Do we assess the group of articles? And what do
we mean by "all" anyway?

Do we mean it meets all the WP policies? Notability? Appropriate use of
external links? That the Manual of Style has been carefully followed?

Or do we mean whether it has been assessed as a stub/start/.../good article
by some review process?

Whenever I find myself in a discussion about "quality" (on any subject, not
just Wikipedia), it pretty much always boils down to "fitness for purpose as
perceived by the user". This is why surveying of users is often used to
measure quality. "How well did we serve you today?" If anyone has been
through Singapore Airport recently, you will have encountered the touch
screens asking to rate on a 1-5 scale just about everything you could
imagine, every toilet block, every immigration queue, etc. And it does have
the cleanest toilets and the fastest immigration queues, so maybe there's
something to be said for the approach.

I think we need to have some common understanding of what we mean by
quality, before we try to compare it across languages. And when we do
compare across languages, then we have to observe that the set of users
changes and presumably their needs change too.

It is interesting to note that en.WP page views have dropped consistently
since Google Knowledge (which generally displays the first para from the
en.WP article) was introduced. What this tells us is that a certain
percentage of readers of an article simply want the most basic facts, which
would be delivered even by a stub article. "Suriname is a country on the
northeastern Atlantic coast of South America" certainly met my information
needs adequately (I heard it mentioned on the TV news in connection with a
hurricane). After finding out where it was in the world, I could have gone
on to read about its colonial history, its demographic sexuality, and its
biodiversity, but I didn't because I didn't have a need to know at that
moment. My point here is that while we would not generally regard a stub as
"quality", but a percentage of the readers of a stub are probably completely
satisfied.

Of course, doing surveys of articles with real users is somewhat difficult
for a research project. But it might be useful to see how user perceptions
of quality compare with other metrics (particularly those which can be more
easily generated for a research project). Starting with other metrics,
without knowing that they are a good proxy for user perception, is probably
a waste of time.

Kerry





_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l




--
Oliver Keyes
Research Analyst
Wikimedia Foundation

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l