The general population & AfD

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

The general population & AfD

Gwern Branwen
http://www-users.cs.umn.edu/~lam/papers/lam_group2010_wikipedia-group-decisions.pdf
:

> "We also found that there have been two bots (computer programs that edit Wikipedia)—BJBot and Jayden54Bot—that automatically automatically notified article editors about AfD discussions and recruited them to participate per the established policy. These bots performed AfD notifications for several months, and offer us an opportunity to study the effect of recruitment that is purely policy driven. We use a process like one described above to detect successful instances of bot-initiated recruitment: if a recruitment bot edited a user’s talk page, and that user !voted in an AfD within two days, then we consider that user to have been recruited by the bot.
> Using the above processes, we identified 8,464 instances of successful recruiting. Table 2 shows a summary of who did the recruiting, and how their recruits !voted. We see large differences in !voting behavior, which suggests that there is bias in who people choose to recruit. (From these data we cannot tell whether the bias is an intentional effort to influence consensus, or the result of social network homophily [14].) Participants recruited by keep !voters were about four times less likely to support deletion as those recruited by delete !voters. The participants that bots recruited also appear unlikely to support deletion, which reflects the policy bias we observed earlier."

--
gwern
http://www.gwern.net

_______________________________________________
WikiEN-l mailing list
[hidden email]
To unsubscribe from this mailing list, visit:
https://lists.wikimedia.org/mailman/listinfo/wikien-l
Reply | Threaded
Open this post in threaded view
|

Re: The general population & AfD

metasj
This is a nicely competent paper.  Thanks for the heads up!  SJ

On Fri, May 20, 2011 at 10:12 AM, Gwern Branwen <[hidden email]> wrote:

> http://www-users.cs.umn.edu/~lam/papers/lam_group2010_wikipedia-group-decisions.pdf
> :
>
>> "We also found that there have been two bots (computer programs that edit Wikipedia)—BJBot and Jayden54Bot—that automatically automatically notified article editors about AfD discussions and recruited them to participate per the established policy. These bots performed AfD notifications for several months, and offer us an opportunity to study the effect of recruitment that is purely policy driven. We use a process like one described above to detect successful instances of bot-initiated recruitment: if a recruitment bot edited a user’s talk page, and that user !voted in an AfD within two days, then we consider that user to have been recruited by the bot.
>> Using the above processes, we identified 8,464 instances of successful recruiting. Table 2 shows a summary of who did the recruiting, and how their recruits !voted. We see large differences in !voting behavior, which suggests that there is bias in who people choose to recruit. (From these data we cannot tell whether the bias is an intentional effort to influence consensus, or the result of social network homophily [14].) Participants recruited by keep !voters were about four times less likely to support deletion as those recruited by delete !voters. The participants that bots recruited also appear unlikely to support deletion, which reflects the policy bias we observed earlier."
>
> --
> gwern
> http://www.gwern.net
>
> _______________________________________________
> WikiEN-l mailing list
> [hidden email]
> To unsubscribe from this mailing list, visit:
> https://lists.wikimedia.org/mailman/listinfo/wikien-l
>



--
Samuel Klein          identi.ca:sj           w:user:sj          +1 617 529 4266

_______________________________________________
WikiEN-l mailing list
[hidden email]
To unsubscribe from this mailing list, visit:
https://lists.wikimedia.org/mailman/listinfo/wikien-l
Reply | Threaded
Open this post in threaded view
|

Re: The general population & AfD

Gwern Branwen
On Fri, May 27, 2011 at 9:19 PM, Samuel Klein <[hidden email]> wrote:
> This is a nicely competent paper.  Thanks for the heads up!  SJ

Re-reading, I'm not sure I understand what the results mean. To
continue the above quote:

> Participants recruited by keep !voters were about four times less likely to support deletion as those recruited by delete !voters. The participants that bots recruited also appear unlikely to support deletion, which reflects the policy bias we observed earlier.
>
> To see what effect participant recruitment has on decision quality, we introduce four binary variables: BotRecruit, NomRecruit, DeleteRecruit, and KeepRecruit. These variables indicate whether a bot, the AfD nominator, a delete !voter, or a keep !voter successfully recruited somebody to the group, respectively.
>
> Looking back to table 1, we find that regardless of the decision, none of the first three variables has a statistically significant effect. On the other hand, when a keep !voter recruited someone to the discussion, we see a significant effect: delete decisions are more likely to be reversed.

So:

1. people recruited by a !keep voter (KeepRecruit) also tend to vote !keep
2. people recruited by a !delete voter (DeleteRecruit) tend neither
way, both !delete and !keep
    2.5. likewise for people recruited (NomRecruit) by the nominator
(almost always a !delete voter, obviously)
3. people recruited by a bot (BotRecruit), like 2 & 2.5, have no
'statistically significant effect'

This is a little troubling for anyone who wants to argue that deleting
fewer articles is the will of the people - the BotRecruits should then
have been more likely to be !keepers.

>     We offer two possible explanations: the first is that recruitment by keep !voters, biased as it may appear, is a sign of positive community interest, and suggests that the article should be kept. If the community decides otherwise and deletes the article, then decision quality suffers. An alternative explanation is that keep !voter recruitment is a sign of activism among those who prefer to keep the article. These proponents may be especially persistent in maintaining the article’s existence in Wikipedia, even if it requires working to reverse a delete decision."

Obviously I prefer the first interpretation. With that one, the story
becomes an article in an obscure niche is put for deletion by a
boorish deletionist; in come the specialists who are not ignorant of
the topic and literature and save it. If I saw an anime article that
should not be deleted up for deletion, I wouldn't ask random
Wikipedians to help, I'd go to what pass for anime experts on
Wikipedia like Timothy Perper, who can look through the academic
literature and have better access to media both English and Japanese.
Looks like bias, smells like homophily, but really just the system
working.

--
gwern
http://www.gwern.net

_______________________________________________
WikiEN-l mailing list
[hidden email]
To unsubscribe from this mailing list, visit:
https://lists.wikimedia.org/mailman/listinfo/wikien-l
Reply | Threaded
Open this post in threaded view
|

Re: The general population & AfD

WereSpielChequers-2
Interesting,

If I understand this correctly the bots are notifying the wikiprojects
based on the wikiproject tags, and in my experience  the !voters that
attracts will tend to have a good idea as to who is and isn't notable
to their project. So I'm not surprised they are a mix of delete and
keep.

The Keep !voters are I would hope being selective, and only bothering
to try and rescue what they believe is worth rescuing and possible to
rescue. So I'm not surprised that they get more keep voters than the
average AFD "recruiter".

Amongst the people notified by the nominator I assume you have the
author and major editors, and they I would expect to tend to vote
keep.  Otherwise you have the people who voted in previous AFDs on
that subject - that can range from ones who are likely to vote keep
because the previous AFD closed as keep, to the probably rarer
scenario where the article was deleted, has been recreated but doesn't
meet G4. To counterbalance that I'm not sure who else the nominator or
a delete voter would be informing who would tend to vote delete.

It would be interesting to see the voting tendency of the other two
major groups - regular AFD voters who weren't specifically canvassed
but who vote in many AFDs; and watchlisters who for whatever reason
have watchlisted the article or the talkpage of its creator. I know
I've found myself at AFD trying to rescue an article after seeing an
AFD notification on a usertalk page I've been watchlisting.

As for the argument that "deleting fewer articles is the will of the
people" I'm not sure we get that many mistakes at AFD. I'm more
worried about CSD, and to a lesser extent prods and turning articles
into redirects.

WereSpielChequers

On 1 June 2011 20:07, Gwern Branwen <[hidden email]> wrote:

> On Fri, May 27, 2011 at 9:19 PM, Samuel Klein <[hidden email]> wrote:
>> This is a nicely competent paper.  Thanks for the heads up!  SJ
>
> Re-reading, I'm not sure I understand what the results mean. To
> continue the above quote:
>
>> Participants recruited by keep !voters were about four times less likely to support deletion as those recruited by delete !voters. The participants that bots recruited also appear unlikely to support deletion, which reflects the policy bias we observed earlier.
>>
>> To see what effect participant recruitment has on decision quality, we introduce four binary variables: BotRecruit, NomRecruit, DeleteRecruit, and KeepRecruit. These variables indicate whether a bot, the AfD nominator, a delete !voter, or a keep !voter successfully recruited somebody to the group, respectively.
>>
>> Looking back to table 1, we find that regardless of the decision, none of the first three variables has a statistically significant effect. On the other hand, when a keep !voter recruited someone to the discussion, we see a significant effect: delete decisions are more likely to be reversed.
>
> So:
>
> 1. people recruited by a !keep voter (KeepRecruit) also tend to vote !keep
> 2. people recruited by a !delete voter (DeleteRecruit) tend neither
> way, both !delete and !keep
>    2.5. likewise for people recruited (NomRecruit) by the nominator
> (almost always a !delete voter, obviously)
> 3. people recruited by a bot (BotRecruit), like 2 & 2.5, have no
> 'statistically significant effect'
>
> This is a little troubling for anyone who wants to argue that deleting
> fewer articles is the will of the people - the BotRecruits should then
> have been more likely to be !keepers.
>
>>     We offer two possible explanations: the first is that recruitment by keep !voters, biased as it may appear, is a sign of positive community interest, and suggests that the article should be kept. If the community decides otherwise and deletes the article, then decision quality suffers. An alternative explanation is that keep !voter recruitment is a sign of activism among those who prefer to keep the article. These proponents may be especially persistent in maintaining the article’s existence in Wikipedia, even if it requires working to reverse a delete decision."
>
> Obviously I prefer the first interpretation. With that one, the story
> becomes an article in an obscure niche is put for deletion by a
> boorish deletionist; in come the specialists who are not ignorant of
> the topic and literature and save it. If I saw an anime article that
> should not be deleted up for deletion, I wouldn't ask random
> Wikipedians to help, I'd go to what pass for anime experts on
> Wikipedia like Timothy Perper, who can look through the academic
> literature and have better access to media both English and Japanese.
> Looks like bias, smells like homophily, but really just the system
> working.
>
> --
> gwern
> http://www.gwern.net
>
> _______________________________________________
> WikiEN-l mailing list
> [hidden email]
> To unsubscribe from this mailing list, visit:
> https://lists.wikimedia.org/mailman/listinfo/wikien-l
>

_______________________________________________
WikiEN-l mailing list
[hidden email]
To unsubscribe from this mailing list, visit:
https://lists.wikimedia.org/mailman/listinfo/wikien-l