Motivations to Contribute to Wikipedia

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Motivations to Contribute to Wikipedia

Audrey Abeyta
Hello all, 

I am an undergraduate student at the University of California, Santa Barbara, conducting a senior honors thesis on users' motivations to contribute to Wikipedia. A more detailed description of the project can be read here: http://meta.wikimedia.org/wiki/Research:Motivations_to_Contribute_to_Wikipedia

My project's success is dependent on the valuable responses of Wikipedia contributors, which I am collecting through an online questionnaire. This brief questionnaire is completely anonymous and should take approximately 10 minutes to complete. If any of you are willing to complete this questionnaire, it can be accessed here: https://us1.us.qualtrics.com/SE/?SID=SV_8ixU9RkozemzC4s

Please feel free to contact me with any questions or concerns.

Thank you in advance for your help!

Sincerely,

Audrey Abeyta


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Motivations to Contribute to Wikipedia

ENWP Pine
I have concerns about this survey. I will address one set of comments to Audrey, and a second set of comments and a question to the Research Committee.
 
Audrey: thanks for your interest in Wikipedia. I suggest that you look at the other research that has previously discussed motivations of Wikipedia contributors and factors that can effect that motivation, such as http://meta.wikimedia.org/wiki/Wikipedia_Editors_Survey_2011 and http://meta.wikimedia.org/wiki/Research:Wikimedia_Summer_of_Research_2011/Summary_of_Findings. On your research Meta page, I disagree with your characterization of extant literature as “lacking,” because while it isn’t comprehensive it also shouldn’t be dismissed. Also, I am wondering why you would use a 2006 source for information about Wikipedia user contribution activity because 2006 was a long time ago in the context of Wikipedia’s lifetime. Regarding surveys of Wikipedians in general, I am skeptical about the reliability of surveys in measuring the motivations of Wikipedia contributors because so many people are not the kind of dedicated volunteer who would be likely to read Research-l or volunteer ten minutes of their time to participate in a study about their motives. Also, you will need to consider bad actors like vandals, spammers, POV pushers, and PR manipulators. Your survey might reveal interesting characteristics of certain classes of editors, but I would be very surprised if your survey results were representative of the entire population of Wikipedia editors. Another complicating factor is that motivations of any single editor can change over time. Finally, and perhaps most significantly, I have some procedural concerns. Did you discuss your survey with anyone in the Wikipedia research community before you announced it here? Your page on Meta says that you “will also request the Research Committee's support in recruiting subjects.” Your section on “Wikimedia Policies, Ethics, and Human Subjects Protection” says nothing about consultation with or approval of the Research Committee, and the most recent published minutes from the Research Committee (that I was able to find) don’t appear to show that your research was discussed by them. I think that they might have had valuable ideas that could have helped you in designing your survey and understanding the existing work on editor motivation. It is my understanding that Research Committee approval is required before soliciting Wikipedia subjects for surveys (see my question below).
 
RCOM members: I would appreciate an official reply to the following concerns. Is it policy that surveys which recruit participants (instead of passively examining editor contributions) must be approved by RCOM before they are sent to Wikimedia mailing lists and/or announced to the broader Wikimedia community (beyond a relatively limited scope such as a single wikiproject, such as GOCE on EN, which might give its approval to the survey only within the scope of that wikiproject)? I am under the impression from the December 12, 2011 RCOM meeting minutes that RCOM approval is required for surveys such as the one that Audrey made. My personal view is that surveyors should get RCOM’s approval before making broad public announcements which recruit research participants, because even well intended researchers can experience difficulties due to questionable assumptions built into the design a study, a limited understanding about the Wikipedia community, or a lack of knowledge about significant existing research. Also, there can be privacy and copyright concerns regarding survey data, and those reasons alone seem sufficient to require that RCOM’s approval is necessary in addition to the approval of any academic institution that is associated with a survey. Also, I am under the impression that permission from the WMF’s legal department is required, in addition to RCOM approval. In the absence of RCOM approval and WMF Legal approval, should information about such an unapproved survey be removed from Meta?
 
Thanks,
 
Pine
 
 
 
Sent: Saturday, 17 March, 2012 13:01
Subject: [Wiki-research-l] Motivations to Contribute to Wikipedia
 
Hello all, 
 
I am an undergraduate student at the University of California, Santa Barbara, conducting a senior honors thesis on users' motivations to contribute to Wikipedia. A more detailed description of the project can be read here: http://meta.wikimedia.org/wiki/Research:Motivations_to_Contribute_to_Wikipedia
 
My project's success is dependent on the valuable responses of Wikipedia contributors, which I am collecting through an online questionnaire. This brief questionnaire is completely anonymous and should take approximately 10 minutes to complete. If any of you are willing to complete this questionnaire, it can be accessed here: https://us1.us.qualtrics.com/SE/?SID=SV_8ixU9RkozemzC4s.
 
Please feel free to contact me with any questions or concerns.
 
Thank you in advance for your help!
 
Sincerely,
 
Audrey Abeyta
 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Motivations to Contribute to Wikipedia

James Salsman-2
In reply to this post by Audrey Abeyta
En Pine <[hidden email]> wrote:
>
>... It is my understanding that Research Committee approval is
> required before soliciting Wikipedia subjects for surveys....

Although Dario Taraborelli has claimed that such Committee approval is
required, he has refused to say where such a policy statement exists:

http://meta.wikimedia.org/wiki/Research_talk:FAQ

The only minutes I was able to find wherein the Research Committee
discussed the possibility of reviewing and approving research, doing
so was explicitly rejected, including by Dario, in favor of issuing
guidelines instead:

http://meta.wikimedia.org/wiki/Research:Committee/Meetings/Meeting_2010-09-18/Log

A Wikipedia Research Policy was announced in April 2010, but never
formally approved by the English Wikipedia community or the Foundation:

http://lists.wikimedia.org/pipermail/wiki-research-l/2010-April/000955.html

At one point it included a proposal for a "Subject Recruitment Approvals
Group" but that was removed in September 2010:

http://en.wikipedia.org/w/index.php?title=Wikipedia%3AResearch&action=historysubmit&diff=382382612&oldid=382338985

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Motivations to Contribute to Wikipedia

Dario Taraborelli-3
In reply to this post by ENWP Pine
A few clarifications on the RCom review procedure (cc'ing Lane as he was raising a similar point here [1])

The current review process for subject recruitment (SR) requests (as well as other types of research requests) has been set up since the creation of the Research Index. It has been adopted since then by the Research Committee and by the Foundation as the de facto standard process through which all such requests are processed – whether they are from external researchers, WMF staff or community members. Despite the lack of a formal policy – which is partly due to the fact that a broadly agreed solution on how to manage SR requests has never been found in years [2] – this process was introduced and applied to all SR proposals as the minimum requirement to:

• help document these requests and the credentials of their authors
• ensure that proposals are legitimate, and that recruitment messages are not used for abuse 
• ensure that they meet basic requirements of privacy, data retention and data licensing
• assess whether the proposed recruitment strategy and sampling requirements are sensible
• identify, wherever possible, redundant or potentially disruptive research
• point the researcher to existing work on the topic
• mitigate the "survey burnout" that affects our editor community

It should be stressed that, precisely due to the lack of a formal policy, the RCom has never been in a position to grant any kind of "definitive approval" to recruit participants: the best we can do is to flag a proposal as "reviewed" or help identify and report patently abusive requests. All proposals that are submitted to our attention are automatically marked as "pending review" via a dedicated template (which will display a yellow SR icon and add the proposal to the appropriate category [3]). We strive to provide the above kind of support and assessment to the different requests we receive and once a project is reviewed we change the support flag to "reviewed" . This is not to say that the process is 100% error-free or very efficient (we unfortunately have little bandwidth to dedicate to this process and review all requests in a timely way), but the review itself (if you haven't come across one [4]) tends to be quite serious and exhaustive. 

So to briefly answer Pine's questions:

- yes, going through RCom review is the standard procedure we expect all proposals to comply with
- no, a proposal should not be removed from Meta if it hasn't been reviewed, it should only be flagged as pending review using the WMF-support template. This also means that Audrey fully complied with the expected procedure to submit a SR request.
- yes, there are privacy concerns, and this is the reason why we take the review of data collection/retention/licensing terms in the proposal very seriously. As these surveys do not fall under the WMF's privacy policy they are not reviewed by WMF Legal team unless they are considered potentially abusive. The only privacy terms that apply are those displayed on the landing page of a survey and our goal is to support best practices in setting up these terms (for example, by making sure that these terms are explicitly accepted by the participants before entering the survey/experiment, particularly in the case of non-academic studies that are not backed by an explicit IRB approval).

Hope this helps address your concerns

Best,
Dario

[2] http://meta.wikimedia.org/wiki/Research:Committee/Areas_of_interest/Subject_recruitment_processes

On Mar 18, 2012, at 3:40 AM, En Pine wrote:

I have concerns about this survey. I will address one set of comments to Audrey, and a second set of comments and a question to the Research Committee.
 
Audrey: thanks for your interest in Wikipedia. I suggest that you look at the other research that has previously discussed motivations of Wikipedia contributors and factors that can effect that motivation, such as http://meta.wikimedia.org/wiki/Wikipedia_Editors_Survey_2011 and http://meta.wikimedia.org/wiki/Research:Wikimedia_Summer_of_Research_2011/Summary_of_Findings. On your research Meta page, I disagree with your characterization of extant literature as “lacking,” because while it isn’t comprehensive it also shouldn’t be dismissed. Also, I am wondering why you would use a 2006 source for information about Wikipedia user contribution activity because 2006 was a long time ago in the context of Wikipedia’s lifetime. Regarding surveys of Wikipedians in general, I am skeptical about the reliability of surveys in measuring the motivations of Wikipedia contributors because so many people are not the kind of dedicated volunteer who would be likely to read Research-l or volunteer ten minutes of their time to participate in a study about their motives. Also, you will need to consider bad actors like vandals, spammers, POV pushers, and PR manipulators. Your survey might reveal interesting characteristics of certain classes of editors, but I would be very surprised if your survey results were representative of the entire population of Wikipedia editors. Another complicating factor is that motivations of any single editor can change over time. Finally, and perhaps most significantly, I have some procedural concerns. Did you discuss your survey with anyone in the Wikipedia research community before you announced it here? Your page on Meta says that you “will also request the Research Committee's support in recruiting subjects.” Your section on “Wikimedia Policies, Ethics, and Human Subjects Protection” says nothing about consultation with or approval of the Research Committee, and the most recent published minutes from the Research Committee (that I was able to find) don’t appear to show that your research was discussed by them. I think that they might have had valuable ideas that could have helped you in designing your survey and understanding the existing work on editor motivation. It is my understanding that Research Committee approval is required before soliciting Wikipedia subjects for surveys (see my question below).
 
RCOM members: I would appreciate an official reply to the following concerns. Is it policy that surveys which recruit participants (instead of passively examining editor contributions) must be approved by RCOM before they are sent to Wikimedia mailing lists and/or announced to the broader Wikimedia community (beyond a relatively limited scope such as a single wikiproject, such as GOCE on EN, which might give its approval to the survey only within the scope of that wikiproject)? I am under the impression from the December 12, 2011 RCOM meeting minutes that RCOM approval is required for surveys such as the one that Audrey made. My personal view is that surveyors should get RCOM’s approval before making broad public announcements which recruit research participants, because even well intended researchers can experience difficulties due to questionable assumptions built into the design a study, a limited understanding about the Wikipedia community, or a lack of knowledge about significant existing research. Also, there can be privacy and copyright concerns regarding survey data, and those reasons alone seem sufficient to require that RCOM’s approval is necessary in addition to the approval of any academic institution that is associated with a survey. Also, I am under the impression that permission from the WMF’s legal department is required, in addition to RCOM approval. In the absence of RCOM approval and WMF Legal approval, should information about such an unapproved survey be removed from Meta?
 
Thanks,
 
Pine
 
 
 
Sent: Saturday, 17 March, 2012 13:01
Subject: [Wiki-research-l] Motivations to Contribute to Wikipedia
 
Hello all, 
 
I am an undergraduate student at the University of California, Santa Barbara, conducting a senior honors thesis on users' motivations to contribute to Wikipedia. A more detailed description of the project can be read here: http://meta.wikimedia.org/wiki/Research:Motivations_to_Contribute_to_Wikipedia
 
My project's success is dependent on the valuable responses of Wikipedia contributors, which I am collecting through an online questionnaire. This brief questionnaire is completely anonymous and should take approximately 10 minutes to complete. If any of you are willing to complete this questionnaire, it can be accessed here: https://us1.us.qualtrics.com/SE/?SID=SV_8ixU9RkozemzC4s.
 
Please feel free to contact me with any questions or concerns.
 
Thank you in advance for your help!
 
Sincerely,
 
Audrey Abeyta
 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Motivations to Contribute to Wikipedia

Jodi Schneider-3
Hi Dario & all,

Thanks for that!

RCom review is still confusing to me (and I assume to many). I'm not sure how new researchers find out about the need for it (or even about the meta documentation procedures). I think more information/publicity about this could help. One possible and partial approach would be to have a regular (monthly?) summary of and reminder about RCom review sent to this email list.

-Jodi

On Mon, Mar 19, 2012 at 6:52 AM, Dario Taraborelli <[hidden email]> wrote:So to briefly answer Pine's questions:

- yes, going through RCom review is the standard procedure we expect all proposals to comply with
- no, a proposal should not be removed from Meta if it hasn't been reviewed, it should only be flagged as pending review using the WMF-support template. This also means that Audrey fully complied with the expected procedure to submit a SR request.
- yes, there are privacy concerns, and this is the reason why we take the review of data collection/retention/licensing terms in the proposal very seriously. As these surveys do not fall under the WMF's privacy policy they are not reviewed by WMF Legal team unless they are considered potentially abusive. The only privacy terms that apply are those displayed on the landing page of a survey and our goal is to support best practices in setting up these terms (for example, by making sure that these terms are explicitly accepted by the participants before entering the survey/experiment, particularly in the case of non-academic studies that are not backed by an explicit IRB approval).

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Motivations to Contribute to Wikipedia

James Salsman-2
In reply to this post by Audrey Abeyta
Dario Taraborelli <[hidden email]> wrote:
>
>... due to the lack of a formal policy, the RCom has never been
> in a position to grant any kind of "definitive approval" to recruit
> participants....

I appreciate that clarification, but it strictly contradicts this edit from
11 days ago:

http://meta.wikimedia.org/w/index.php?title=Research:Subject_recruitment&diff=3546474&oldid=2703471

about which Dario said, "I appreciate the documentation on the
review procedure" at
http://meta.wikimedia.org/wiki/Research_talk:Subject_recruitment

I think there are some very serious ethical issues here.  Requiring
Research Committee approval to contact editors or users was
explicitly rejected by the Research Committee:

http://meta.wikimedia.org/wiki/Research:Committee/Meetings/Meeting_2010-09-18/Log

As far as I can tell, the Research Committee has not discussed the
topic since.

I wonder what the community thinks of this new requirement.

Sincerely,
James Salsman

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Motivations to Contribute to Wikipedia

Lane Rasberry
Hey all,

James: I made the edit stating the research should get approval, and I did that by jumping into the game and just making the edit based on what I read in discussion boards. I did not consider it to be a new requirement as you called it. On the talk page I proposed a kind of approval - actually it is what Dario is calling "flagging", meaning that someone checks to see that the researcher has submitted requested information but does not check to see whether the information meets any criteria. 

I think flagging is a good idea to make sure that researchers meet minimum disclosure requirements, and I would be happy to be one of the reviewers of research in this way. I trust that it would be noncontroversial just to ask researchers to complete a form and then check to see if it is completed; surely any other review system would start with that step, right?

Jodi: Most researchers do not know about research submission for approval on Wikipedia, but I think if we had a landing page with instructions then the community would become aware of it and start directing researchers to it. Researchers know that they are supposed to seek community approval before doing work.

Is such a flagging system already in place? If not, shall we start one?

This is what I imagine is what we have consensus to do - is this how it is supposed to work?
  1. Researcher jumps on Wikipedia unannounced and starts recruiting for surveys
  2. Some Wikipedian tells the researcher to submit their project for review
  3. Researcher goes to landing page and completes a form for their proposal
  4. The proposal is posted publicly
  5. Any volunteer can check the proposal to see if all fields are completed
  6. Volunteers tag the form as being completed or incomplete - no quality review
  7. Completed forms eventually get reviewed by RCom according to criteria which are currently undefined
  8. Approved projects get a template to stick on their project page. 
  9. Researchers must show their research page to all research recruitment candidates, who would be able to see the completed form, the flagging by a volunteer, and the approval by RCom. The approval template would also link to more information about research on Wikipedia.
  10. Research subjects would only be able to agree to participate in research by following instructions at the bottom of the research description form, so they would see default notices like "unflagged" or "unreviewed" if no one has checked it.

On Mon, Mar 19, 2012 at 5:15 AM, James Salsman <[hidden email]> wrote:
Dario Taraborelli <[hidden email]> wrote:
>
>... due to the lack of a formal policy, the RCom has never been
> in a position to grant any kind of "definitive approval" to recruit
> participants....

I appreciate that clarification, but it strictly contradicts this edit from
11 days ago:

http://meta.wikimedia.org/w/index.php?title=Research:Subject_recruitment&diff=3546474&oldid=2703471

about which Dario said, "I appreciate the documentation on the
review procedure" at
http://meta.wikimedia.org/wiki/Research_talk:Subject_recruitment

I think there are some very serious ethical issues here.  Requiring
Research Committee approval to contact editors or users was
explicitly rejected by the Research Committee:

http://meta.wikimedia.org/wiki/Research:Committee/Meetings/Meeting_2010-09-18/Log

As far as I can tell, the Research Committee has not discussed the
topic since.

I wonder what the community thinks of this new requirement.

Sincerely,
James Salsman

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



--
Lane Rasberry
206.801.0814
[hidden email]

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Motivations to Contribute to Wikipedia

James Salsman-2
In reply to this post by Audrey Abeyta
Lane,

Thanks for your message:

> James: I made the edit stating the research should get approval,
> and I did that by jumping into the game and just making the edit
> based on what I read in discussion boards. I did not consider it
> to be a new requirement....

For the benefit of those who haven't clicked on the link, you edited
[[meta:Research:Subject recruitment]] to read, at the top:

"If you are doing research which involves contacting Wikimedia project
editors or users then you must first notify the Wikimedia Research
Committee by describing your project. After your project gets approval
then you may begin."

How could that not be seen as a requirement?  Do you think there is a
way to phrase it so that it would not be seen as a requirement?

Certainly this is not your fault.  As you read, Dario Taraborelli
stated on February 15, "this is a policy that we're enforcing ...
approval is required"
http://meta.wikimedia.org/w/index.php?title=Research_talk%3AFAQ&diff=3441309&oldid=3440848

And after you made that edit, Dario thanked you for it, saying, "I
appreciate the documentation on the review procedure" -- even though
the Research Committee had explicitly rejected an approval policy
requirement in September 2010, has not discussed it since, and neither
the community or the Foundation has ever endorsed any of the earlier
policy proposals.

I would not be so upset about this if I hadn't been repeatedly accused
of misconduct in failing to obtain RCom approval.

Given the ease and lack of remorse with which Dr. Taraborelli, Mr.
Walling, and Mr. Beaudette have all repeatedly lied about me while
accusing me of misconduct, I have lost all confidence in the ability
of Foundation staff to adhere to basic ethics. I intend to continue to
raise this issue until it is addressed sufficiently.

Sincerely,
James Salsman

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Motivations to Contribute to Wikipedia

Dario Taraborelli-3
James,

I think I have replied consistently to your requests, both on wiki and by mail, stressing that this is the de facto standard procedure that was introduced with the creation of the RCom, pending a formal (as in voted) policy, and that the expectation is for whoever runs a survey or subject recruitment campaign to comply with this procedure. I appreciate that it implies a bit of bureaucracy but it's the best solution we can offer to help the community understand who runs a study and what for and help the researcher/investigator meet some basic requirements.

Dario

On Mar 19, 2012, at 10:06 AM, James Salsman wrote:

> Lane,
>
> Thanks for your message:
>
>> James: I made the edit stating the research should get approval,
>> and I did that by jumping into the game and just making the edit
>> based on what I read in discussion boards. I did not consider it
>> to be a new requirement....
>
> For the benefit of those who haven't clicked on the link, you edited
> [[meta:Research:Subject recruitment]] to read, at the top:
>
> "If you are doing research which involves contacting Wikimedia project
> editors or users then you must first notify the Wikimedia Research
> Committee by describing your project. After your project gets approval
> then you may begin."
>
> How could that not be seen as a requirement?  Do you think there is a
> way to phrase it so that it would not be seen as a requirement?
>
> Certainly this is not your fault.  As you read, Dario Taraborelli
> stated on February 15, "this is a policy that we're enforcing ...
> approval is required"
> http://meta.wikimedia.org/w/index.php?title=Research_talk%3AFAQ&diff=3441309&oldid=3440848
>
> And after you made that edit, Dario thanked you for it, saying, "I
> appreciate the documentation on the review procedure" -- even though
> the Research Committee had explicitly rejected an approval policy
> requirement in September 2010, has not discussed it since, and neither
> the community or the Foundation has ever endorsed any of the earlier
> policy proposals.
>
> I would not be so upset about this if I hadn't been repeatedly accused
> of misconduct in failing to obtain RCom approval.
>
> Given the ease and lack of remorse with which Dr. Taraborelli, Mr.
> Walling, and Mr. Beaudette have all repeatedly lied about me while
> accusing me of misconduct, I have lost all confidence in the ability
> of Foundation staff to adhere to basic ethics. I intend to continue to
> raise this issue until it is addressed sufficiently.
>
> Sincerely,
> James Salsman
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Motivations to Contribute to Wikipedia

Dario Taraborelli-3
In reply to this post by Jodi Schneider-3
Jodi,

you're right, we should make this documentation more visible and accessible to researchers. It's currently linked from the landing page of the Research Index as well as from the FAQ, any help to improve the documentation is very welcom. WP:BOLD ;)

Dario

On Mar 19, 2012, at 1:37 AM, Jodi Schneider wrote:

Hi Dario & all,

Thanks for that!

RCom review is still confusing to me (and I assume to many). I'm not sure how new researchers find out about the need for it (or even about the meta documentation procedures). I think more information/publicity about this could help. One possible and partial approach would be to have a regular (monthly?) summary of and reminder about RCom review sent to this email list.

-Jodi

On Mon, Mar 19, 2012 at 6:52 AM, Dario Taraborelli <[hidden email]> wrote:So to briefly answer Pine's questions:

- yes, going through RCom review is the standard procedure we expect all proposals to comply with
- no, a proposal should not be removed from Meta if it hasn't been reviewed, it should only be flagged as pending review using the WMF-support template. This also means that Audrey fully complied with the expected procedure to submit a SR request.
- yes, there are privacy concerns, and this is the reason why we take the review of data collection/retention/licensing terms in the proposal very seriously. As these surveys do not fall under the WMF's privacy policy they are not reviewed by WMF Legal team unless they are considered potentially abusive. The only privacy terms that apply are those displayed on the landing page of a survey and our goal is to support best practices in setting up these terms (for example, by making sure that these terms are explicitly accepted by the participants before entering the survey/experiment, particularly in the case of non-academic studies that are not backed by an explicit IRB approval).
_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Motivations to Contribute to Wikipedia

Dario Taraborelli-3
In reply to this post by Lane Rasberry
Hi Lane,

your proposed workflow is a good description of how I would like the SR procedure to function in an ideal world. I am not myself at the forefront of SR discussions, but I'd definitely like to see a more streamlined process and a better way of signaling to participants which projects are flagged as reviewed and which aren't. Part of the discussion that we had during the last RCom meeting of the RCom was precisely focused on this issue [1].

If you want to contribute to the SR discussion, I strongly recommend you post your proposal on this page [2] so it can be seen and discussed by others. It would also probably make sense to move the entire SR discussion to a dedicated list as I suspect many wiki-research-l subscribers are not interested in following this thread. I'll also forward this to the RCom members who have been involved in SR as they will be able to make a better judgment than mine on these matters


Dario



Is such a flagging system already in place? If not, shall we start one?

This is what I imagine is what we have consensus to do - is this how it is supposed to work?
  1. Researcher jumps on Wikipedia unannounced and starts recruiting for surveys
  2. Some Wikipedian tells the researcher to submit their project for review
  3. Researcher goes to landing page and completes a form for their proposal
  4. The proposal is posted publicly
  5. Any volunteer can check the proposal to see if all fields are completed
  6. Volunteers tag the form as being completed or incomplete - no quality review
  7. Completed forms eventually get reviewed by RCom according to criteria which are currently undefined
  8. Approved projects get a template to stick on their project page. 
  9. Researchers must show their research page to all research recruitment candidates, who would be able to see the completed form, the flagging by a volunteer, and the approval by RCom. The approval template would also link to more information about research on Wikipedia.
  10. Research subjects would only be able to agree to participate in research by following instructions at the bottom of the research description form, so they would see default notices like "unflagged" or "unreviewed" if no one has checked it.


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: [Foundation-l] Motivations to Contribute to Wikipedia

James Salsman-2
In reply to this post by James Salsman-2
Dario Taraborelli <[hidden email]> wrote:

> James, I think I have replied consistently to your requests, both on wiki
> and by mail....

Anyone can judge for themselves whether this is true by looking at
http://meta.wikimedia.org/wiki/Research_talk:FAQ

There you claimed that research approval was a mandatory policy, and
much more recently you thanked a third party for an edit which clearly
implies that it is strictly mandatory. But in September 2010 you
agreed with the rest of the RCom that research subject recruitment
approval should not be mandatory in favor of published guidelines
instead. And when called on the inconsistency, you wrote that approval
is not in fact mandatory. Both can not be true.

Instead of apologizing for your lie with which you attempted to impugn
my integrity, you have been trying to cover it up with rhetoric.

Is that behavior considered acceptable at the Wikimedia Foundation?

Sincerely,
James Salsman

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: [Foundation-l] Motivations to Contribute to Wikipedia

Joe Corneli-3
@Audrey, are you sure you want to study motivations to contribute to
Wikipedia?  Maybe you should study flamewars on Wikipedia mailing
lists instead, it promises to be a subject on which you could get lots
of observational data!

On Mon, Mar 19, 2012 at 6:11 PM, James Salsman <[hidden email]> wrote:

> Dario Taraborelli <[hidden email]> wrote:
>
>> James, I think I have replied consistently to your requests, both on wiki
>> and by mail....
>
> Anyone can judge for themselves whether this is true by looking at
> http://meta.wikimedia.org/wiki/Research_talk:FAQ
>
> There you claimed that research approval was a mandatory policy, and
> much more recently you thanked a third party for an edit which clearly
> implies that it is strictly mandatory. But in September 2010 you
> agreed with the rest of the RCom that research subject recruitment
> approval should not be mandatory in favor of published guidelines
> instead. And when called on the inconsistency, you wrote that approval
> is not in fact mandatory. Both can not be true.
>
> Instead of apologizing for your lie with which you attempted to impugn
> my integrity, you have been trying to cover it up with rhetoric.
>
> Is that behavior considered acceptable at the Wikimedia Foundation?
>
> Sincerely,
> James Salsman
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Motivations to Contribute to Wikipedia

Emilio J. Rodríguez-Posada
In reply to this post by Audrey Abeyta
Hi Audrey;

Have you added a free text entry in your survey? I have seen many pre-selected replies but not places for free responses.

On WikiPapers we have some publications about that subject http://wikipapers.referata.com/wiki/Motivations

I hope you publish your thesis and your dataset (anonymized?) online when finished. Good luck.

Regards,
emijrp

2012/3/17 Audrey Abeyta <[hidden email]>
Hello all, 

I am an undergraduate student at the University of California, Santa Barbara, conducting a senior honors thesis on users' motivations to contribute to Wikipedia. A more detailed description of the project can be read here: http://meta.wikimedia.org/wiki/Research:Motivations_to_Contribute_to_Wikipedia

My project's success is dependent on the valuable responses of Wikipedia contributors, which I am collecting through an online questionnaire. This brief questionnaire is completely anonymous and should take approximately 10 minutes to complete. If any of you are willing to complete this questionnaire, it can be accessed here: https://us1.us.qualtrics.com/SE/?SID=SV_8ixU9RkozemzC4s

Please feel free to contact me with any questions or concerns.

Thank you in advance for your help!

Sincerely,

Audrey Abeyta


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: [Foundation-l] Motivations to Contribute to Wikipedia

James Salsman-2
In reply to this post by Audrey Abeyta
Yaroslav M. Blanter <[hidden email]> wrote:

> actually, I am pretty sure we did discuss the procedure which requires
> endorsement (we did not call it approval), either at the extraordinary
> meeting in December (related to the survey banner story) or in the RCom
> mailing list in the thread related to the same story....

The discussion in the RCom mailing list archives is fairly summarized by
http://lists.wikimedia.org/pipermail/rcom-l/2011-December/000506.html
"I rather suspect that there isn't consensus on this committee to restrict
researchers in their requesting community members to complete research
questionnaires."

The subsequent etherpad minutes for the December 22, 2011 meeting,
discuss a "future subject recruitment policy" which the participants
state they do not understand how to craft.

> do not call other people liars unless you have very strong proofs....

Is there any way to interpret these two statements such that one of
them is not a lie?

"this is a policy that we're enforcing ... approval is required"
http://meta.wikimedia.org/w/index.php?title=Research_talk%3AFAQ&diff=3441309&oldid=3440848

"due to the lack of a formal policy, the RCom has never been in a
position to grant any kind of "definitive approval" to recruit
participants"
http://lists.wikimedia.org/pipermail/wiki-research-l/2012-March/001896.html

The first of those two statements was made in an effort to accuse me
of misconduct. I stand by my statements, and I am certain that I have
acted ethically.

It would be best if this issue were addressed as a mistake on the part
of those who have contradicted themselves.

Sincerely,
James Salsman

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: [Foundation-l] Motivations to Contribute to Wikipedia

Aaron Halfaker-2
James, 

Please discontinue hijacking this discussion about Audrey's proposal for your crusade.  If you'd like to challenge Dario publicly, please start your own thread. 

-Aaron 

On Mon, Mar 19, 2012 at 6:54 PM, James Salsman <[hidden email]> wrote:
Yaroslav M. Blanter <[hidden email]> wrote:

> actually, I am pretty sure we did discuss the procedure which requires
> endorsement (we did not call it approval), either at the extraordinary
> meeting in December (related to the survey banner story) or in the RCom
> mailing list in the thread related to the same story....

The discussion in the RCom mailing list archives is fairly summarized by
http://lists.wikimedia.org/pipermail/rcom-l/2011-December/000506.html
"I rather suspect that there isn't consensus on this committee to restrict
researchers in their requesting community members to complete research
questionnaires."

The subsequent etherpad minutes for the December 22, 2011 meeting,
discuss a "future subject recruitment policy" which the participants
state they do not understand how to craft.

> do not call other people liars unless you have very strong proofs....

Is there any way to interpret these two statements such that one of
them is not a lie?

"this is a policy that we're enforcing ... approval is required"
http://meta.wikimedia.org/w/index.php?title=Research_talk%3AFAQ&diff=3441309&oldid=3440848

"due to the lack of a formal policy, the RCom has never been in a
position to grant any kind of "definitive approval" to recruit
participants"
http://lists.wikimedia.org/pipermail/wiki-research-l/2012-March/001896.html

The first of those two statements was made in an effort to accuse me
of misconduct. I stand by my statements, and I am certain that I have
acted ethically.

It would be best if this issue were addressed as a mistake on the part
of those who have contradicted themselves.

Sincerely,
James Salsman

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: [Foundation-l] Motivations to Contribute to Wikipedia

Lane Rasberry
Hey guys.

I wrote a proposal for reviewing all research done on human subjects on Wikipedia. Please check it out here.

Dario suggested that conversation about subject recruitment leave this mailing list. Could I ask that someone set up a human subject research mailing list and then describe how I may join it?

Thanks.

On Mon, Mar 19, 2012 at 5:36 PM, Aaron Halfaker <[hidden email]> wrote:
James, 

Please discontinue hijacking this discussion about Audrey's proposal for your crusade.  If you'd like to challenge Dario publicly, please start your own thread. 

-Aaron 


On Mon, Mar 19, 2012 at 6:54 PM, James Salsman <[hidden email]> wrote:
Yaroslav M. Blanter <[hidden email]> wrote:

> actually, I am pretty sure we did discuss the procedure which requires
> endorsement (we did not call it approval), either at the extraordinary
> meeting in December (related to the survey banner story) or in the RCom
> mailing list in the thread related to the same story....

The discussion in the RCom mailing list archives is fairly summarized by
http://lists.wikimedia.org/pipermail/rcom-l/2011-December/000506.html
"I rather suspect that there isn't consensus on this committee to restrict
researchers in their requesting community members to complete research
questionnaires."

The subsequent etherpad minutes for the December 22, 2011 meeting,
discuss a "future subject recruitment policy" which the participants
state they do not understand how to craft.

> do not call other people liars unless you have very strong proofs....

Is there any way to interpret these two statements such that one of
them is not a lie?

"this is a policy that we're enforcing ... approval is required"
http://meta.wikimedia.org/w/index.php?title=Research_talk%3AFAQ&diff=3441309&oldid=3440848

"due to the lack of a formal policy, the RCom has never been in a
position to grant any kind of "definitive approval" to recruit
participants"
http://lists.wikimedia.org/pipermail/wiki-research-l/2012-March/001896.html

The first of those two statements was made in an effort to accuse me
of misconduct. I stand by my statements, and I am certain that I have
acted ethically.

It would be best if this issue were addressed as a mistake on the part
of those who have contradicted themselves.

Sincerely,
James Salsman

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l




--
Lane Rasberry
206.801.0814
[hidden email]

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
fn
Reply | Threaded
Open this post in threaded view
|

Re: Motivations to Contribute to Wikipedia

fn
In reply to this post by Emilio J. Rodríguez-Posada
Dear Audrey,


On 03/19/2012 08:43 PM, emijrp wrote:

> On WikiPapers we have some publications about that subject
> http://wikipapers.referata.com/wiki/Motivations

Regarding: "Extant literature on this topic is lacking".

As you see Emijrp has collected 10 papers categorized under motivation
in his WikiPapers, and in the recently announce Wikilit group we have
also many papers on motivation (mostly collected by the four other
members of the Wikilit group), see e.g., this category with 30 papers:

http://wikilit.referata.com/wiki/Category:Contributor_motivation

Also the parent category:

http://wikilit.referata.com/wiki/Category:Antecedents_of_participation

52 papers are under this category. There is room for an entire review on
its own. :-)


Cheers,
Finn


> 2012/3/17 Audrey Abeyta <[hidden email]
> <mailto:[hidden email]>>
>
>     Hello all,
>
>     I am an undergraduate student at the University of California, Santa
>     Barbara, conducting a senior honors thesis on users' motivations to
>     contribute to Wikipedia. A more detailed description of the project
>     can be read here:
>     http://meta.wikimedia.org/wiki/Research:Motivations_to_Contribute_to_Wikipedia
>
>     My project's success is dependent on the valuable responses of
>     Wikipedia contributors, which I am collecting through an online
>     questionnaire. This brief questionnaire is completely anonymous and
>     should take approximately 10 minutes to complete. If any of you are
>     willing to complete this questionnaire, it can be accessed here:
>     https://us1.us.qualtrics.com/SE/?SID=SV_8ixU9RkozemzC4s.
>
>     Please feel free to contact me with any questions or concerns.
>
>     Thank you in advance for your help!
>
>     Sincerely,
>
>     Audrey Abeyta
>
>
>     _______________________________________________
>     Wiki-research-l mailing list
>     [hidden email]
>     <mailto:[hidden email]>
>     https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Motivations to Contribute to Wikipedia

Audrey Abeyta
Dear Finn,

Thank you so much for the literature recommendations regarding users' motivations to contribute to Wikipedia. I was able to find some of these articles on my own, but many I had missed and will incorporate into my literature review.

I should probably clarify what I meant when I said that "extant literature is lacking". I in no way meant to dismiss the work that has been done in this field (which is more extensive than I originally realized). Instead, I was referring to the inconclusive results some of the articles I've read have found. In many of the articles I've referenced, researchers look at motivation from an intrinsic/extrinsic motivation perspective, which is a good way of measuring motivation, but maybe does not explore other important facets.

Thanks again for the literature! I could not complete this project without the help of you and others in the Wikipedia community.

Best,

Audrey

On Mar 20, 2012, at 8:52 AM, Finn Årup Nielsen wrote:

> Dear Audrey,
>
>
> On 03/19/2012 08:43 PM, emijrp wrote:
>
>> On WikiPapers we have some publications about that subject
>> http://wikipapers.referata.com/wiki/Motivations
>
> Regarding: "Extant literature on this topic is lacking".
>
> As you see Emijrp has collected 10 papers categorized under motivation in his WikiPapers, and in the recently announce Wikilit group we have also many papers on motivation (mostly collected by the four other members of the Wikilit group), see e.g., this category with 30 papers:
>
> http://wikilit.referata.com/wiki/Category:Contributor_motivation
>
> Also the parent category:
>
> http://wikilit.referata.com/wiki/Category:Antecedents_of_participation
>
> 52 papers are under this category. There is room for an entire review on its own. :-)
>
>
> Cheers,
> Finn
>
>
>> 2012/3/17 Audrey Abeyta <[hidden email]
>> <mailto:[hidden email]>>
>>
>>    Hello all,
>>
>>    I am an undergraduate student at the University of California, Santa
>>    Barbara, conducting a senior honors thesis on users' motivations to
>>    contribute to Wikipedia. A more detailed description of the project
>>    can be read here:
>>    http://meta.wikimedia.org/wiki/Research:Motivations_to_Contribute_to_Wikipedia
>>
>>    My project's success is dependent on the valuable responses of
>>    Wikipedia contributors, which I am collecting through an online
>>    questionnaire. This brief questionnaire is completely anonymous and
>>    should take approximately 10 minutes to complete. If any of you are
>>    willing to complete this questionnaire, it can be accessed here:
>>    https://us1.us.qualtrics.com/SE/?SID=SV_8ixU9RkozemzC4s.
>>
>>    Please feel free to contact me with any questions or concerns.
>>
>>    Thank you in advance for your help!
>>
>>    Sincerely,
>>
>>    Audrey Abeyta
>>
>>
>>    _______________________________________________
>>    Wiki-research-l mailing list
>>    [hidden email]
>>    <mailto:[hidden email]>
>>    https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>
>>
>
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l