The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

classic Classic list List threaded Threaded
64 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Thomas Morton
>
> Jokes aside :) the problem here is exemplary of what Wikipedia *doesn't*
> do well, which is to find ways to assess the legitimacy of
> not-yet-legitimised knowledge


I'm not seeing a good argument that we *should* assess the legitimacy. This
seems to be being cast in the light of "verifiability not truth" (a really
silly maxim) but, in reality, it goes more back to our idea of "we use
reliable sources because they are *peer reviewed*".

The implicit suggestion here is that Wikipedia could/should act as that
form of peer review for so called "not-yet-legitimised knowledge".

Although it would be nice to have that role it isn't actually all that
practical for several reasons:

- We already have enough disagreement over sourcing as it is
- Very few of us are truly subject matter experts
- Even fewer of us have experience of peer review and critical examination
of work (this is especially critical in the sciences)
- Taking on the role of peer review puts us at odds with our main aim; of
providing a summary resource.

The main thing it would do is open up Wikipedia as an avenue to push (and
legitimise) fringe material.


> whether the 'truth' is new analysis backed up by serious scholarship (as
> in this case), or things that have not yet made it to reliable print
> scholarship (knowledge that's circulated orally, whether in conversations
> or social media). The core of the problem would appear to be our insistence
> on the narrowest and smallest possible definition of 'legitimate
> knowledge'.


Is it? Lets look at what happened here.

- Someone posted information apparently based on their own analysis - it's
not unreasonable to remove this
- He began to defend his additions on the talk page and some were
incorporated
- He gave up further attempts
- The next day a lot of those comments were incorporated (if you read
through the detail very carefully, to as much of an extent as the published
literature allowed) based on the inconsistencies he raised
- He went away and wrote a book which forwards a number of new theories and
updates our understanding of the topic.

Has anyone actually read through the points raised? The problem is not a
case of "well this factual thing disproves what is in the article". It is
much more a case of disagreement over the established *interpretation* of
events and over the *extent* to which views expressed by the previously top
level source were recorded (for example; "no evidence" was a mistaken
summary of the view raised by the source, a point which was then corrected).


> And I'd imagine that the solution is to find a workable, sensible and
> cross-culturally translatable version of legitimacy that is a lot better,
> bigger and more generous than what we have.


No it isn't.

We have a good sourcing policy; one which does cover a very wide range of
sources and can be relaxed and restricted as required to fit the topic
based on good editorial judgement.

However, for the topic of *history* (in which I have an interest, and where
I work on articles at the moment) we definitely should stick to well
reviewed, published material.

What *was* at issue here is how we treat new users; the discussion was
approached (on the part of our editors) either as a battleground/fight, or
in a quite patronising way. The issue here was that someone was put off
from raising the issues.

I do know of academics who are frustrated by what they see
as inaccuracies in Wikipedia articles; and when they try to correct them
from their own knowledge get reverted. That, coupled with a lack of
understanding of how Wikipedia works from a technical perspective, can make
the experience very frustrating - and the opportunity to explain the
rational viewpoint (i.e. peer reviewed sourcing) is lost.

If you read the article this is what he is saying; that academics should
follow the peer review route before trying to get their material
included. He also notes that even when he had taken this route he was put
off because of his treatment the last time.

The failure here is *not* our content policy. But the behavioural.

Tom
_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Neil Babbage
"What *was* at issue here is how we treat new users; the discussion was
approached (on the part of our editors) either as a battleground/fight, or in a quite patronising way. The issue here was that someone was put off from raising the issues."

The "expertise" that is most valued at Wikipedia is expertise in Wikipedia itself  - its policies, procedures, technology, etc - rather than expertise in the content. That's a fundamental cultural flaw if the project is to succeed.

In reference to other comments here about the treatment of new editors, there has been a noticeable (to me at least) shift away from the role of administrators and "senior editors" from helping newcomers overcome the challenges to finding them a nuisance. On smaller projects the "it's no big deal" approach to the sysop flag still dominates and the administrators spend their time correcting naming errors, moving pages, merging histories, adding templates and also adding content to help new work survive. I don't see that at the English Wikipedia any more.

Sent from my BlackBerry® wireless device

-----Original Message-----
From: Thomas Morton <[hidden email]>
Sender: [hidden email]
Date: Wed, 22 Feb 2012 10:15:20
To: Wikimedia Foundation Mailing List<[hidden email]>
Reply-To: Wikimedia Foundation Mailing List <[hidden email]>
Subject: Re: [Foundation-l] The 'Undue Weight' of Truth on Wikipedia (from
 the Chronicle) + some citation discussions

>
> Jokes aside :) the problem here is exemplary of what Wikipedia *doesn't*
> do well, which is to find ways to assess the legitimacy of
> not-yet-legitimised knowledge


I'm not seeing a good argument that we *should* assess the legitimacy. This
seems to be being cast in the light of "verifiability not truth" (a really
silly maxim) but, in reality, it goes more back to our idea of "we use
reliable sources because they are *peer reviewed*".

The implicit suggestion here is that Wikipedia could/should act as that
form of peer review for so called "not-yet-legitimised knowledge".

Although it would be nice to have that role it isn't actually all that
practical for several reasons:

- We already have enough disagreement over sourcing as it is
- Very few of us are truly subject matter experts
- Even fewer of us have experience of peer review and critical examination
of work (this is especially critical in the sciences)
- Taking on the role of peer review puts us at odds with our main aim; of
providing a summary resource.

The main thing it would do is open up Wikipedia as an avenue to push (and
legitimise) fringe material.


> whether the 'truth' is new analysis backed up by serious scholarship (as
> in this case), or things that have not yet made it to reliable print
> scholarship (knowledge that's circulated orally, whether in conversations
> or social media). The core of the problem would appear to be our insistence
> on the narrowest and smallest possible definition of 'legitimate
> knowledge'.


Is it? Lets look at what happened here.

- Someone posted information apparently based on their own analysis - it's
not unreasonable to remove this
- He began to defend his additions on the talk page and some were
incorporated
- He gave up further attempts
- The next day a lot of those comments were incorporated (if you read
through the detail very carefully, to as much of an extent as the published
literature allowed) based on the inconsistencies he raised
- He went away and wrote a book which forwards a number of new theories and
updates our understanding of the topic.

Has anyone actually read through the points raised? The problem is not a
case of "well this factual thing disproves what is in the article". It is
much more a case of disagreement over the established *interpretation* of
events and over the *extent* to which views expressed by the previously top
level source were recorded (for example; "no evidence" was a mistaken
summary of the view raised by the source, a point which was then corrected).


> And I'd imagine that the solution is to find a workable, sensible and
> cross-culturally translatable version of legitimacy that is a lot better,
> bigger and more generous than what we have.


No it isn't.

We have a good sourcing policy; one which does cover a very wide range of
sources and can be relaxed and restricted as required to fit the topic
based on good editorial judgement.

However, for the topic of *history* (in which I have an interest, and where
I work on articles at the moment) we definitely should stick to well
reviewed, published material.

What *was* at issue here is how we treat new users; the discussion was
approached (on the part of our editors) either as a battleground/fight, or
in a quite patronising way. The issue here was that someone was put off
from raising the issues.

I do know of academics who are frustrated by what they see
as inaccuracies in Wikipedia articles; and when they try to correct them
from their own knowledge get reverted. That, coupled with a lack of
understanding of how Wikipedia works from a technical perspective, can make
the experience very frustrating - and the opportunity to explain the
rational viewpoint (i.e. peer reviewed sourcing) is lost.

If you read the article this is what he is saying; that academics should
follow the peer review route before trying to get their material
included. He also notes that even when he had taken this route he was put
off because of his treatment the last time.

The failure here is *not* our content policy. But the behavioural.

Tom
_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Thomas Morton
>
> "What *was* at issue here is how we treat new users; the discussion was
> approached (on the part of our editors) either as a battleground/fight, or
> in a quite patronising way. The issue here was that someone was put off
> from raising the issues."
>
> The "expertise" that is most valued at Wikipedia is expertise in Wikipedia
> itself  - its policies, procedures, technology, etc - rather than expertise
> in the content. That's a fundamental cultural flaw if the project is to
> succeed.
>

In a sense; though, as one academic pointed out to me, writing
an encyclopaedia is a skill in itself. And just because one is a topic area
expert does not immediately make them the most capable of writing the
article (in some respects it makes them less capable than an interested
layman).


> In reference to other comments here about the treatment of new editors,
> there has been a noticeable (to me at least) shift away from the role of
> administrators and "senior editors" from helping newcomers overcome the
> challenges to finding them a nuisance.


I don't think this is an issue of sysops or "senior editors" - it is
ingrained in the vast majority of the community.

For example we know it is common in newer/younger editors to "bite" or
otherwise apply policy too strongly - because with regularity we have to
deal with the fall out (i.e. mentor them).

I see the same issues with content editors as well; with resistance to
anyone trying to add content to articles they've invested in (I don't just
mean subject matter experts).

Realistically *we are all part of the problem*. You, me, etc. because the
problem is the entire ecosystem. Even stuff we think is polite and sensible
might be incomprehensible to a newbie. Simple things like linking to, or
quoting, parts of policy instead of taking time to write a simple
explanation. The use of templates. The resistance to listen to arguments.
It all adds up into a confusing user experience.

This is not a new problem; many online communities suffer, and have
suffered, from it.

All of the things I mentioned are useful once your dealing with editors
aware of the workings - it's not "new and scary" at that point and acts as
a useful shortcut to streamline our interaction. The key thing to work on,
I think, is easing newbies into that process without bombarding them with
too much of it at once.

Tom
_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Mike Christie
On Wed, Feb 22, 2012 at 7:05 AM, Thomas Morton <[hidden email]
> wrote:

> Realistically *we are all part of the problem*. You, me, etc. because the
> problem is the entire ecosystem. Even stuff we think is polite and sensible
> might be incomprehensible to a newbie. Simple things like linking to, or
> quoting, parts of policy instead of taking time to write a simple
> explanation. The use of templates. The resistance to listen to arguments.
> It all adds up into a confusing user experience.
>
> This is not a new problem; many online communities suffer, and have
> suffered, from it.
>
> All of the things I mentioned are useful once your dealing with editors
> aware of the workings - it's not "new and scary" at that point and acts as
> a useful shortcut to streamline our interaction. The key thing to work on,
> I think, is easing newbies into that process without bombarding them with
> too much of it at once.
>

This is part of the reason why I have been advocating that the education
programs take an active role in encouraging the academics who teach classes
on Wikipedia to become contributors themselves.  If we can provide
high-quality one-on-one mentoring to academics in the workings of Wikipedia
we could increase the percentage of users who have a foot in both worlds.
 Editors without subject matter expertise will always be needed but to
solve some of the problems on Wikipedia, particularly those regarding undue
weight and comprehensiveness of coverage, we have to attract experts and
help them become editors.

Mike
_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Thomas Morton
On 22 February 2012 12:44, Mike Christie <[hidden email]> wrote:

> On Wed, Feb 22, 2012 at 7:05 AM, Thomas Morton <
> [hidden email]
> > wrote:
>
> > Realistically *we are all part of the problem*. You, me, etc. because the
> > problem is the entire ecosystem. Even stuff we think is polite and
> sensible
> > might be incomprehensible to a newbie. Simple things like linking to, or
> > quoting, parts of policy instead of taking time to write a simple
> > explanation. The use of templates. The resistance to listen to arguments.
> > It all adds up into a confusing user experience.
> >
> > This is not a new problem; many online communities suffer, and have
> > suffered, from it.
> >
> > All of the things I mentioned are useful once your dealing with editors
> > aware of the workings - it's not "new and scary" at that point and acts
> as
> > a useful shortcut to streamline our interaction. The key thing to work
> on,
> > I think, is easing newbies into that process without bombarding them with
> > too much of it at once.
> >
>
> This is part of the reason why I have been advocating that the education
> programs take an active role in encouraging the academics who teach classes
> on Wikipedia to become contributors themselves.  If we can provide
> high-quality one-on-one mentoring to academics in the workings of Wikipedia
> we could increase the percentage of users who have a foot in both worlds.
>  Editors without subject matter expertise will always be needed but to
> solve some of the problems on Wikipedia, particularly those regarding undue
> weight and comprehensiveness of coverage, we have to attract experts and
> help them become editors.


That sounds like a great idea! I've always wondered about how our eduction
programs focus on students, but not on the academics that teach them - or
on professional organisations.

During the last Board elections one of the things I kept saying is that we
need to focus on subject matter experts; be they academics
or professionals and get their input.

Tom
_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Achal Prabhala-2
In reply to this post by Thomas Morton


On Wednesday 22 February 2012 03:45 PM, Thomas Morton wrote:
>> Jokes aside :) the problem here is exemplary of what Wikipedia *doesn't*
>> do well, which is to find ways to assess the legitimacy of
>> not-yet-legitimised knowledge
>
> I'm not seeing a good argument that we *should* assess the legitimacy. This
> seems to be being cast in the light of "verifiability not truth" (a really
> silly maxim) but, in reality, it goes more back to our idea of "we use
> reliable sources because they are *peer reviewed*".

Well actually, we use newspaper sources very frequently, as well as
non-scholarly (and therefore non-peer-reviewed) books, so in fact, we
rely on *printing* (or to put it more kindly, publishing) as a signal
for peer-review, not peer-review itself. In my opinion, this is a poor
signal.

>
> The implicit suggestion here is that Wikipedia could/should act as that
> form of peer review for so called "not-yet-legitimised knowledge".
>
> Although it would be nice to have that role it isn't actually all that
> practical for several reasons:
>
> - We already have enough disagreement over sourcing as it is
> - Very few of us are truly subject matter experts
> - Even fewer of us have experience of peer review and critical examination
> of work (this is especially critical in the sciences)
> - Taking on the role of peer review puts us at odds with our main aim; of
> providing a summary resource.
>
> The main thing it would do is open up Wikipedia as an avenue to push (and
> legitimise) fringe material.

I completely agree that we need a system that doesn't throw a spanner in
the works - but if you're suggesting that the only workable signal for
legitimacy is printing, then that seems odd; and it is odder that while
we ourselves rely on a range of filtered non-printed sources for our own
information (social media, conversations) that we shouldn't attempt to
find a way to bring Wikipedia into these very old and very new systems
of legitimate knowledge that we've fundamentally accepted ourselves.

>
>
>> whether the 'truth' is new analysis backed up by serious scholarship (as
>> in this case), or things that have not yet made it to reliable print
>> scholarship (knowledge that's circulated orally, whether in conversations
>> or social media). The core of the problem would appear to be our insistence
>> on the narrowest and smallest possible definition of 'legitimate
>> knowledge'.
>
> Is it? Lets look at what happened here.
>
> - Someone posted information apparently based on their own analysis - it's
> not unreasonable to remove this
> - He began to defend his additions on the talk page and some were
> incorporated
> - He gave up further attempts
> - The next day a lot of those comments were incorporated (if you read
> through the detail very carefully, to as much of an extent as the published
> literature allowed) based on the inconsistencies he raised
> - He went away and wrote a book which forwards a number of new theories and
> updates our understanding of the topic.
>
> Has anyone actually read through the points raised? The problem is not a
> case of "well this factual thing disproves what is in the article". It is
> much more a case of disagreement over the established *interpretation* of
> events and over the *extent* to which views expressed by the previously top
> level source were recorded (for example; "no evidence" was a mistaken
> summary of the view raised by the source, a point which was then corrected).
>
>
>> And I'd imagine that the solution is to find a workable, sensible and
>> cross-culturally translatable version of legitimacy that is a lot better,
>> bigger and more generous than what we have.
>
> No it isn't.
>
> We have a good sourcing policy; one which does cover a very wide range of
> sources and can be relaxed and restricted as required to fit the topic
> based on good editorial judgement.

Consider these two points:

1) Bad behaviour needs a back-up, and inadequately updated/ incompletely
thought out policies serve as a bulwark against weeding out bad behaviour.

2) If, for instance, 'no original research' was to keep physics cranks
out, as seems the case, then it's succeeded - the physics cranks are
out. Given that it was put in place ten years ago though, and given that
it may have been very useful circa 2001, in a Wikipedia with limited
geographical contribution and use, things are very different now. Might
we not benefit from assessing the cost of policies that guard against
enemies who no longer exist?

>
> However, for the topic of *history* (in which I have an interest, and where
> I work on articles at the moment) we definitely should stick to well
> reviewed, published material.
>
> What *was* at issue here is how we treat new users; the discussion was
> approached (on the part of our editors) either as a battleground/fight, or
> in a quite patronising way. The issue here was that someone was put off
> from raising the issues.
>
> I do know of academics who are frustrated by what they see
> as inaccuracies in Wikipedia articles; and when they try to correct them
> from their own knowledge get reverted. That, coupled with a lack of
> understanding of how Wikipedia works from a technical perspective, can make
> the experience very frustrating - and the opportunity to explain the
> rational viewpoint (i.e. peer reviewed sourcing) is lost.
>
> If you read the article this is what he is saying; that academics should
> follow the peer review route before trying to get their material
> included. He also notes that even when he had taken this route he was put
> off because of his treatment the last time.
>
> The failure here is *not* our content policy. But the behavioural.

I respect where you're coming from and it's very helpful in furthering
my own understanding of the situation. But: I think the 'behavioural' is
distinctly affected by 'policy' - especially when the policy is
malleable, loose and archaic enough to interpreted (usually hawkishly)
at will by those already in the know.

>
> Tom
> _______________________________________________
> foundation-l mailing list
> [hidden email]
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>

_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Thomas Morton
On 22 February 2012 13:11, Achal Prabhala <[hidden email]> wrote:

>
>
> On Wednesday 22 February 2012 03:45 PM, Thomas Morton wrote:
>
>> Jokes aside :) the problem here is exemplary of what Wikipedia *doesn't*
>>> do well, which is to find ways to assess the legitimacy of
>>> not-yet-legitimised knowledge
>>>
>>
>> I'm not seeing a good argument that we *should* assess the legitimacy.
>> This
>> seems to be being cast in the light of "verifiability not truth" (a really
>> silly maxim) but, in reality, it goes more back to our idea of "we use
>> reliable sources because they are *peer reviewed*".
>>
>
> Well actually, we use newspaper sources very frequently, as well as
> non-scholarly (and therefore non-peer-reviewed) books, so in fact, we rely
> on *printing* (or to put it more kindly, publishing) as a signal for
> peer-review, not peer-review itself. In my opinion, this is a poor signal.


Well realistically, yes, we consider something that has been reputably
published to have a basic level of reliability. But that is not the end of
the test.

This idea of "published" can (and is) relaxed though. Indeed it is my
perception that in many topic areas we rely far too heavily on online
sources - there can be a distinct prejudice against offline source material.

However I am interested in whether you have a specific idea of what you
would change? Can you express a reason for why using the published test is
a poor signal?

Tom
_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

David Gerard-2
On 22 February 2012 13:29, Thomas Morton <[hidden email]> wrote:

> However I am interested in whether you have a specific idea of what you
> would change? Can you express a reason for why using the published test is
> a poor signal?


It produces a rich crop of both false positives and false negatives. I
can't think of a better test off the top of my head, but that doesn't
mean it's defects are somehow not gross and obvious. No-one who's ever
been quoted by the media could ever hear them being called a "reliable
source" and keep a straight face.


- d.

_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Achal Prabhala-2
In reply to this post by Thomas Morton


On Wednesday 22 February 2012 06:59 PM, Thomas Morton wrote:

> On 22 February 2012 13:11, Achal Prabhala<[hidden email]>  wrote:
>
>>
>> On Wednesday 22 February 2012 03:45 PM, Thomas Morton wrote:
>>
>>> Jokes aside :) the problem here is exemplary of what Wikipedia *doesn't*
>>>> do well, which is to find ways to assess the legitimacy of
>>>> not-yet-legitimised knowledge
>>>>
>>> I'm not seeing a good argument that we *should* assess the legitimacy.
>>> This
>>> seems to be being cast in the light of "verifiability not truth" (a really
>>> silly maxim) but, in reality, it goes more back to our idea of "we use
>>> reliable sources because they are *peer reviewed*".
>>>
>> Well actually, we use newspaper sources very frequently, as well as
>> non-scholarly (and therefore non-peer-reviewed) books, so in fact, we rely
>> on *printing* (or to put it more kindly, publishing) as a signal for
>> peer-review, not peer-review itself. In my opinion, this is a poor signal.
>
> Well realistically, yes, we consider something that has been reputably
> published to have a basic level of reliability. But that is not the end of
> the test.
>
> This idea of "published" can (and is) relaxed though. Indeed it is my
> perception that in many topic areas we rely far too heavily on online
> sources - there can be a distinct prejudice against offline source material.
>
> However I am interested in whether you have a specific idea of what you
> would change? Can you express a reason for why using the published test is
> a poor signal?
>
> Tom


I think it's a poor signal when it's the only signal, when it wholly
occupies the phrase 'legitimate knowledge'. In a cross-cultural context,
and especially on English Wikipedia, it's notoriously fraught - it's
very difficult for someone with no experience of a place to distinguish
between 'printed' and 'respectably published' - or even more simply,
between a lunatic fringe newsletter and a mainstream newspaper. I
thought what Tom Morris had to say here was very useful:
http://en.wikipedia.org/wiki/User:Tom_Morris/The_Reliability_Delusion -
that we could well deepen our own understanding of currently
unimpeachable sources - like the Guardian or the Observer.

So the helpful starting point here is that printed, published work is
fallible and variably reliable too.

In real life, each of us has figured out ways to filter the legitimate
from the illegitimate in terms of received knowledge, whether in
newspapers, conversations, or on twitter. But on Wikipedia, we've only
figured out a way to sort the published, and maybe a little but more.
Published knowledge though, is a fraction of what there is to know as a
whole. That sounds terribly high-minded but it's not really, and some
more on this is available here:
http://meta.wikimedia.org/wiki/Research:Oral_Citations

My point is not that we should discard what we have in terms of
policies. My point is that we may benefit from acknowledging what the
policies lead us *not to do well*. And that would be to find a system to
sort out the unreliable and fake from the reliable and legitimate when
it comes to oral citations, or social media citations or primary sources
- in exactly the same way as we've figured out a system to sort the
unreliable from the reliable in another fallible knowledge system -
printed publishing. And if we think that these things we don't do well
are important and that we can figure a way to bring them in, then we
should find that way. (Which is to say - to add to what we've got, not
to forego the current system).

An aside: there are millions of oral testimonies hosted at thousands of
extremely reputable organisations - on Native American life at the
Smithsonian, or Holocaust history at Yale - which currently have no
place on Wikipedia, because they're primary sources. Often but not
always, these primary sources relate to power relations - and so you are
far more likely to find the lives of women, Native Americans, Holocaust
survivors or Jazz musicians in oral testimony than in the printed word.
Sometimes, foregoing these primary sources may be the right decision,
but other times this will not be so - and by disallowing primary sources
in entirety, or not figuring out a system to use them sensibly, I think
we're throwing the baby out with the bathwater.

Related point: there's this project proposal that you might be
interested in -
http://meta.wikimedia.org/wiki/Wikimedia_Fellowships/Project_Ideas/InCite


> _______________________________________________
> foundation-l mailing list
> [hidden email]
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>

_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Fred Bauder-2
In reply to this post by Thomas Morton
>>
>> "What *was* at issue here is how we treat new users; the discussion was
>> approached (on the part of our editors) either as a battleground/fight,
>> or
>> in a quite patronising way. The issue here was that someone was put off
>> from raising the issues."
>>
>> The "expertise" that is most valued at Wikipedia is expertise in
>> Wikipedia
>> itself  - its policies, procedures, technology, etc - rather than
>> expertise
>> in the content. That's a fundamental cultural flaw if the project is to
>> succeed.
>>
>
> In a sense; though, as one academic pointed out to me, writing
> an encyclopaedia is a skill in itself. And just because one is a topic
> area
> expert does not immediately make them the most capable of writing the
> article (in some respects it makes them less capable than an interested
> layman).
>
Of course, but that is not a reason to heap abuse on them rather than
assisting them.

>
>> In reference to other comments here about the treatment of new editors,
>> there has been a noticeable (to me at least) shift away from the role
>> of
>> administrators and "senior editors" from helping newcomers overcome the
>> challenges to finding them a nuisance.
>
>
> I don't think this is an issue of sysops or "senior editors" - it is
> ingrained in the vast majority of the community.
>
> For example we know it is common in newer/younger editors to "bite" or
> otherwise apply policy too strongly - because with regularity we have to
> deal with the fall out (i.e. mentor them).
>
> I see the same issues with content editors as well; with resistance to
> anyone trying to add content to articles they've invested in (I don't
> just
> mean subject matter experts).

That is what is going on at the Haymarket article. Editing that article
successfully is harder than the D-Day Landing.

>
> Realistically *we are all part of the problem*. You, me, etc. because the
> problem is the entire ecosystem. Even stuff we think is polite and
> sensible
> might be incomprehensible to a newbie. Simple things like linking to, or
> quoting, parts of policy instead of taking time to write a simple
> explanation. The use of templates. The resistance to listen to arguments.
> It all adds up into a confusing user experience.
>
> This is not a new problem; many online communities suffer, and have
> suffered, from it.
>
> All of the things I mentioned are useful once your dealing with editors
> aware of the workings - it's not "new and scary" at that point and acts
> as
> a useful shortcut to streamline our interaction. The key thing to work
> on,
> I think, is easing newbies into that process without bombarding them with
> too much of it at once.
>
> Tom

And we do have a problem with academics such as this one who are not
patient enough or too busy to get up to speed. Note, however, that he is
not too busy to write an article on the Academic Chronicle or appear on
NPR.

Now, in effect we have moved a Wikipedia policy discussion off our policy
pages onto The Academic Chronicle and NPR which most of us have no access
too. Our policy process is broken, and, in fact, effectively jammed.

Fred


_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Thomas Morton
In reply to this post by Achal Prabhala-2
>
> I think it's a poor signal when it's the only signal, when it wholly
> occupies the phrase 'legitimate knowledge'. In a cross-cultural context,
> and especially on English Wikipedia, it's notoriously fraught - it's very
> difficult for someone with no experience of a place to distinguish between
> 'printed' and 'respectably published' - or even more simply, between a
> lunatic fringe newsletter and a mainstream newspaper. I thought what Tom
> Morris had to say here was very useful: http://en.wikipedia.org/wiki/**
> User:Tom_Morris/The_**Reliability_Delusion<http://en.wikipedia.org/wiki/User:Tom_Morris/The_Reliability_Delusion>- that we could well deepen our own understanding of currently
> unimpeachable sources - like the Guardian or the Observer.
>
> So the helpful starting point here is that printed, published work is
> fallible and variably reliable too.
>

I absolutely agree with that. One of the big bug bears I have is that, when
discussing sourcing, people put them into categories such as "newspaper"
and declare them therefore reliable.

IMO our sourcing policy is very good at laying out how to consider the
reliability of the source - for example reminding us to think of it in
terms of not only the content but author and publisher (is the author known
for attacking X, is the publisher criticised for publishing poor quality
work?).

This is the key usefulness of publishing - in that it involves other
people/entities in the process.

So if anything we should boil the sourcing policy down to "lets see who
your friends are".


In real life, each of us has figured out ways to filter the legitimate from
> the illegitimate in terms of received knowledge, whether in newspapers,
> conversations, or on twitter.


Ugh, no. Don't get me started on this :) The lack of critical thinking
within the wider population is dire - and the spoon fed rubbish we get from
every side is disheartening. Things like the prevalence of homoeopathy are
examples of this issue.

We have filters; but they are subjective and usually not good.


> But on Wikipedia, we've only figured out a way to sort the published, and
> maybe a little but more. Published knowledge though, is a fraction of what
> there is to know as a whole. That sounds terribly high-minded but it's not
> really, and some more on this is available here:
> http://meta.wikimedia.org/**wiki/Research:Oral_Citations<http://meta.wikimedia.org/wiki/Research:Oral_Citations>


The oral citations stuff is cool - and I now see where you are going with
your thought process.

For what its worth I don't think we preclude such sourcing - but I do think
that the community often misunderstands (or fails to read) the actual
policy (as opposed to, say, the summary).

My point is not that we should discard what we have in terms of policies.

> My point is that we may benefit from acknowledging what the policies lead
> us *not to do well*. And that would be to find a system to sort out the
> unreliable and fake from the reliable and legitimate when it comes to oral
> citations, or social media citations or primary sources - in exactly the
> same way as we've figured out a system to sort the unreliable from the
> reliable in another fallible knowledge system - printed publishing. And if
> we think that these things we don't do well are important and that we can
> figure a way to bring them in, then we should find that way. (Which is to
> say - to add to what we've got, not to forego the current system).
>

This is probably where we disagree. The amount of editorial decision making
in terms of "what weight should I give this material". If you have a set of
oral accounts of an event how do you present that - which parts as fact?
Which as opinion? What weighting? Finding expert people to do that review
stage - and have the work reviewed - is absolutely critical to writing an
Encyclopaedia.


> An aside: there are millions of oral testimonies hosted at thousands of
> extremely reputable organisations - on Native American life at the
> Smithsonian, or Holocaust history at Yale - which currently have no place
> on Wikipedia, because they're primary sources.


And this is what I meant about misunderstanding policies. Because nothing
in our policies precludes the use of primary sources. What you can't do is
use them for interpretation or analysis. So to make up an example; if you
have an oral citation from someone who was arrested under an oppressive
regime - and questioned at length on his choice of blonde hair color and
whether he dyed it. You could relate that experience, but you
couldn't necessarily say something like "The regime persecuted people with
blond hair, or those who dyed it".

So if there are oral recordings of at the Smithsonian & Yale (surely that
means they are published?? It certainly fits our explicit criteria for
published) then we can and should be using them.

One example of published primary sources we do use is court proceedings.

Tom
_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Fred Bauder-2
In reply to this post by Thomas Morton

> This idea of "published" can (and is) relaxed though. Indeed it is my
> perception that in many topic areas we rely far too heavily on online
> sources - there can be a distinct prejudice against offline source
> material.

> Tom

Journals pose a particular problem as they are often, as in the case of
the three journal articles in this case, behind pay walls. Those are peer
reviewed, while his book by a commercial publisher has not received
academic reviews.

Someone did send me a copy of one of the academic journal articles. But I
have yet to see the other two which cost quite a bit.

Fred



_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Fred Bauder-2
In reply to this post by Thomas Morton

>
> And this is what I meant about misunderstanding policies. Because nothing
> in our policies precludes the use of primary sources. What you can't do
> is
> use them for interpretation or analysis. So to make up an example; if you
> have an oral citation from someone who was arrested under an oppressive
> regime - and questioned at length on his choice of blonde hair color and
> whether he dyed it. You could relate that experience, but you
> couldn't necessarily say something like "The regime persecuted people
> with
> blond hair, or those who dyed it".
>
> So if there are oral recordings of at the Smithsonian & Yale (surely that
> means they are published?? It certainly fits our explicit criteria for
> published) then we can and should be using them.
>
> One example of published primary sources we do use is court proceedings.
>
> Tom

Interesting because in the Haymarket case there is a 3,000 page
transcript of the trial on line. I thought we could not use it directly.
What can we use it for? Can it be used as a reference for itself, in the
sense that the fact that there was a lengthy hearing with a great number
of prosecution witnesses being heard, as well as many defense witness?

From Identifying reliable sources:

Primary sources are often difficult to use appropriately. While they can
be both reliable and useful in certain situations, they must be used with
caution in order to avoid original research. Material based purely on
primary sources should be avoided. All interpretive claims, analyses, or
synthetic claims about primary sources must be referenced to a secondary
source, rather than original analysis of the primary-source material by
Wikipedia editors.

Fred


_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Thomas Morton
>
> Interesting because in the Haymarket case there is a 3,000 page
> transcript of the trial on line. I thought we could not use it directly.
> What can we use it for?

Can it be used as a reference for itself, in the
> sense that the fact that there was a lengthy hearing with a great number
> of prosecution witnesses being heard, as well as many defense witness?
>

No; because that is interpretive/analysis.

But you could use it to, say, list the witnesses called.

Tom
_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Sarah-128
In reply to this post by Achal Prabhala-2
On Wed, Feb 22, 2012 at 11:01 AM, Achal Prabhala <[hidden email]> wrote:
> An aside: there are millions of oral testimonies hosted at thousands of
> extremely reputable organisations - on Native American life at the
> Smithsonian, or Holocaust history at Yale - which currently have no place on
> Wikipedia, because they're primary sources.

There is no policy disallowing the use of primary sources on the
English Wikipedia. They have to be used carefully, because it's easy
to misuse them, but they're definitely allowed. See the NOR policy --
http://en.wikipedia.org/w/index.php?title=Wikipedia:No_original_research&oldid=478167288#Primary.2C_secondary_and_tertiary_sources

Sarah

_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Andrew Lih
In reply to this post by Fred Bauder-2
An update: Steven Walling will be with me on NPR's Talk of the Nation,
today at 3pm US Eastern time talking about this issue.

In preparation for the show, I looked up Messer-Kruse's book on Amazon, and
I am pasting in the first two sentences of the blurb (bold emphasis mine).

In this *controversial* and *groundbreaking* new history, Timothy
Messer-Kruse *rewrites* the standard narrative of the most iconic event in
American labor history: the Haymarket Bombing and Trial of 1886. Using
thousands of pages of previously unexamined materials, Messer-Kruse
demonstrates that, *contrary* to longstanding historical opinion, the trial
was not the “travesty of justice” it has *commonly* been *depicted* as.

I am sympathetic to Messer-Kruse's plight, but these key words highlight
perhaps why this case is the perfect storm of conditions (ie. Achilles
Heel) for a clash in editing.

The ability of Wikipedia to absorb leading edge, "groundbreaking new
history" research is limited, given the emergent norms and accrual of
policy that has primarily served to make sure things are verified as a
majority view before it makes it into Wikipedia. There are good reasons for
this, since every hour of every day Wikipedia is bombarded by vandalism and
crackpot contributions.

But I do share Mike Godwin's concerns on what this means for attracting
editors and for Wikipedia's public image.

More and more, I'm convinced Wikipedia must focus on embracing a new
complementary culture -- an "invitation culture" that Sarah Stierch (of
GLAMwiki fame) really brought to my attention at Wikimania Haifa. We have
to recognize Wikipedia has a huge monoculture problem when the editor
survey says 91% of active editors are male.

Sarah told me as a female, she would never have participated in Wikipedia
without someone else inviting her first. And that there were many great
folks out there that felt the same thing because on the face of it,
Wikipedia is not putting in neon lights that it's soliciting participation
and there are many reasons to describe newbie experiences as "jarring" or
even "unwelcoming."

That's basically what GLAM can address and I think it is crucial to
Wikipedia's future. It reaches out directly to people who share Wikipedia's
mission about education and quality by approaching them as valued members
of a knowledge creation community to make Wikipedia participation
accessible. Wikipedians in Residence have served as the liaisons to make
that introductory experience smooth and empowering. Another area ripe for
collaboration is journalism, by finding a way to engage journalists in
creating content such as for the Oral Citations project. And,
coincidentally, both of these fields have a high percentage of females.

We cannot bank Wikipedia's future solely on the prospective lone
contributor toughing it out against the obstacles of complex wikimarkup, a
cumbersome talk page/discussion system, demoralizing edit reverts, policy
pages gone wild, and if he or she gets that far, a frightening
administrator hazing ritual.

For more on the GLAM, see the two page summary produced at GLAMcampDC last
week:

http://commons.wikimedia.org/wiki/File:GLAM_One-Pager.pdf


-Andrew
_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Steven Walling
On Wed, Feb 22, 2012 at 10:13 AM, Andrew Lih <[hidden email]> wrote:

> But I do share Mike Godwin's concerns on what this means for attracting
> editors and for Wikipedia's public image.
>

This is where I disagree. But we can talk about this later. ;)

Steven
_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Andrew Lih
On Wed, Feb 22, 2012 at 10:20 AM, Steven Walling
<[hidden email]>wrote:

> On Wed, Feb 22, 2012 at 10:13 AM, Andrew Lih <[hidden email]> wrote:
>
> > But I do share Mike Godwin's concerns on what this means for attracting
> > editors and for Wikipedia's public image.
> >
>
> This is where I disagree. But we can talk about this later. ;)
>

Awesome! I was afraid we'd be on the show agreeing on everything. :)
_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Achal Prabhala-2
In reply to this post by Fred Bauder-2


On Wednesday 22 February 2012 08:08 PM, Fred Bauder wrote:

> Journals pose a particular problem as they are often, as in the case of
> the three journal articles in this case, behind pay walls. Those are peer
> reviewed, while his book by a commercial publisher has not received
> academic reviews.
>
> Someone did send me a copy of one of the academic journal articles. But I
> have yet to see the other two which cost quite a bit.
>
> Fred
>
>


I was a student at really well-resourced US universities for a short
part of my life and then spent the rest of it in far flung parts of the
third world with little access to the kind of knowledge I had access to
while in the US - a situation that continues here in India - and so I
particularly identify with the access problem you've raised. Journal
corporations like Reed Elsevier and services like JSTOR and Project Muse
provide negligibly small entry to non-paying consumers outside their
traditional base - rich universities in the US, Europe and a few other
parts of the world.

This creates a weird anomaly, reflected - I am sure - on Wikipedia. Open
Access journals - and just generally, any knowledge resource whose text
is available to see freely on the internet - probably gets far more
citation use on Wikipedia and elsewhere than a journal behind a paywall.
(And in many ways this is really good - the reward for sharing or going
OA is greater circulation and more citations).

But I can't imagine that either closed journal companies or closed
journal article authors are pleased with this. If enough of us see some
value in it, I wonder if we can ask someone at the Foundation to
negotiate with these services for some kind of preferential/free access?
Perhaps a limited amount of free browsing with a registered Wikipedia
login or something like that. It would certainly help the work of
editing - both in terms of citing well as well as in terms of looking up
that citation or checking up on it. The journals market is so
centralised, there are literally two companies and two services to talk
to for just about everything under the sun.

A related problem is what currently happens to material on Google books.
You follow a citation link on a Wikipedia page, say to a particular
page, and you find that the page in question is disbarred - as it has
not been made available under the (usually minimal) free page views that
the copyright holder of the book has authorised Google to allow. This is
a shame because my understanding of the situation is that even when
something like 10% of the book is allowed to be seen, the Google books
process is somewhat random, and doesn't necessarily include the one page
you want in your session. But - if this were technically possible and if
someone at the Foundation was interested in talking to Google about this
- if each Google books citation link from Wikipedia were to assuredly
take us to that page (assuming some minimal viewing permission, so this
wouldn't apply to books where the copyright holder has provided *no*
permissions) then that would be really helpful for editors, both those
making the citation as well as others checking up on it. (And probably
turn a lot of the non-linking citations to pages in a book into links
that take you somewhere).



> _______________________________________________
> foundation-l mailing list
> [hidden email]
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>

_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Reply | Threaded
Open this post in threaded view
|

Re: The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

Achal Prabhala-2
In reply to this post by Sarah-128
Thank you Tom, and Sarah, for your very helpful explanations - they are
extremely useful.

There's a discussion on at the reliable sources notice board, for
instance, which highlights some of the interpretive problems you raise:
http://en.wikipedia.org/wiki/Wikipedia:Reliable_sources/Noticeboard#Oral_Citations

Can I ask you how you would analyse the work of the oral citations
project (http://meta.wikimedia.org/wiki/Research:Oral_Citations) in
terms of our policies on original research, and verifiability?

And further, how these policies might apply to the idea of social media,
as well as more private archives, say, corporate archives, being used as
citations? (And on that point, is there a difference between the the
Native American folk archive at the Smithsonian and the corporate
archives of the Michelin corporation in France, for our purposes?)

Okay, I know that's asking a lot :)


On Wednesday 22 February 2012 08:56 PM, Sarah wrote:

> On Wed, Feb 22, 2012 at 11:01 AM, Achal Prabhala<[hidden email]>  wrote:
>> An aside: there are millions of oral testimonies hosted at thousands of
>> extremely reputable organisations - on Native American life at the
>> Smithsonian, or Holocaust history at Yale - which currently have no place on
>> Wikipedia, because they're primary sources.
> There is no policy disallowing the use of primary sources on the
> English Wikipedia. They have to be used carefully, because it's easy
> to misuse them, but they're definitely allowed. See the NOR policy --
> http://en.wikipedia.org/w/index.php?title=Wikipedia:No_original_research&oldid=478167288#Primary.2C_secondary_and_tertiary_sources
>
> Sarah
>
> _______________________________________________
> foundation-l mailing list
> [hidden email]
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>

_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
1234