discussion about wikipedia surveys

classic Classic list List threaded Threaded
55 messages Options
123
Reply | Threaded
Open this post in threaded view
|

discussion about wikipedia surveys

phoebe ayers-3
Hi all,

FYI, for those who do not regularly follow wikimedia-l, there's a discussion going on there about Wikipedia surveys (sparked off by one particular survey) that may be of interest to this list. See http://lists.wikimedia.org/pipermail/wikimedia-l/2014-July/073367.html

Briefly, the meta-question seems to be: we set up some researcher best practices such that researchers should get approval via RCOM, but that process is now not active. So now what? What should researchers do?

(Personally, I think the answer should be to resuscitate RCOM, but that's easy to say and harder to do!)

-- phoebe

--
* I use this address for lists; send personal messages to phoebe.ayers <at> gmail.com *

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Federico Leva (Nemo)
phoebe ayers, 16/07/2014 19:21:
> (Personally, I think the answer should be to resuscitate RCOM, but
> that's easy to say and harder to do!)

IMHO in the meanwhile the most useful thing folks can do is subscribing
to the feed of new research pages:
<https://meta.wikimedia.org/w/index.php?title=Special:NewPages&feed=atom&hidebots=1&hideredirs=1&limit=500&offset=&namespace=202>
It's easier to build a functioning RCOM out of an active community of
"reviewers", than the other way round.

Nemo

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Aaron Halfaker-3
RCOM review is still alive and looking for new reviewers (really, coordinators).  Researchers can be directed to me or Dario ([hidden email]) to be assigned a reviewer.  There is also a proposed policy on enwiki that could use some eyeballs: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment


On Wed, Jul 16, 2014 at 11:16 AM, Federico Leva (Nemo) <[hidden email]> wrote:
phoebe ayers, 16/07/2014 19:21:
> (Personally, I think the answer should be to resuscitate RCOM, but
> that's easy to say and harder to do!)

IMHO in the meanwhile the most useful thing folks can do is subscribing
to the feed of new research pages:
<https://meta.wikimedia.org/w/index.php?title=Special:NewPages&feed=atom&hidebots=1&hideredirs=1&limit=500&offset=&namespace=202>
It's easier to build a functioning RCOM out of an active community of
"reviewers", than the other way round.

Nemo

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Kerry Raymond

Just saying here what I already put on the Talk page:

 

I am a little bothered by the opening sentence "This page documents the process that researchers must follow before asking Wikipedia contributors to participate in research studies such as surveys, interviews and experiments."

WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes. What WMF does own is its communication channels to me as a contributor and WMF has a right to control what occurs on those channels. Also I think WMF probably should be concerned about both its readers and its contributors being recruited through its channels (as either might be being recruited). I think this distinction should be made, e.g.

"This page documents the process that researchers must follow if they wish to use Wikipedia's (WMF's?) communication channels to recruit people to participate in research studies such as surveys, interviews and experiments. Communication channels include its mailing lists, its Project pages, Talk pages, and User Talk pages [and whatever else I've forgotten]." 

 

If researchers want to recruit WPians via non-WMF means, I don’t think it’s any business of WMF’s. An example might be a researcher who wanted to contact WPians via chapters or thorgs; I would leave it for the chapter/thorg to decide if they wanted to assist the researcher via their communication channels.

 

Of course, the practical reality of it is that some researchers (oblivious of WMF’s concerns in relation to recruitment of WPians to research projects) will simply use WMF’s channels without asking nicely first. Obviously we can remove such requests on-wiki and follow up any email requests with the commentary that this was not an approved request. In my category of [whatever else I’ve forgotten], I guess there are things like Facebook groups and any other social media presence.

 

Also to be practical, if WMF is to have a process to vet research surveys, I think it has to be sufficiently fast and not be overly demanding to avoid the possibility of the researcher giving up (“too hard to deal with these people”) and simply spamming email, project pages, social media in the hope of recruiting some participants regardless. That is, if we make it too slow/hard to do the right thing, we effectively encourage doing the wrong thing. Also, what value-add can we give them to reward those who do the right thing? It’s nice to have a carrot as well as a stick when it comes to onerous processes J

 

Because of the criticism of “not giving back”, could we perhaps do things to try to make the researcher feel part of the community to make “giving back” more likely? For example, could we give them a slot every now and again to talk about their project in the R&D Showcase? Encourage them to be on this mailing list. Are we at a point where it might make sense to organise a Wikipedia research conference to help build a research community? Just thinking aloud here …

 

Kerry

 

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Aaron Halfaker
Sent: Thursday, 17 July 2014 6:59 AM
To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

RCOM review is still alive and looking for new reviewers (really, coordinators).  Researchers can be directed to me or Dario ([hidden email]) to be assigned a reviewer.  There is also a proposed policy on enwiki that could use some eyeballs: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

 

On Wed, Jul 16, 2014 at 11:16 AM, Federico Leva (Nemo) <[hidden email]> wrote:

phoebe ayers, 16/07/2014 19:21:

> (Personally, I think the answer should be to resuscitate RCOM, but
> that's easy to say and harder to do!)

IMHO in the meanwhile the most useful thing folks can do is subscribing
to the feed of new research pages:
<https://meta.wikimedia.org/w/index.php?title=Special:NewPages&feed=atom&hidebots=1&hideredirs=1&limit=500&offset=&namespace=202>
It's easier to build a functioning RCOM out of an active community of
"reviewers", than the other way round.

Nemo

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Amir E. Aharoni
> WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes.

I don't think that it really should be about WMF. The WMF shouldn't enforce anything. The community can formulate good practices for researchers and _advise_ community members not to cooperate with researchers who don't follow these practices. Not much more is needed.


--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
‪“We're living in pieces,
I want to live in peace.” – T. Moore‬


2014-07-17 8:24 GMT+03:00 Kerry Raymond <[hidden email]>:

Just saying here what I already put on the Talk page:

 

I am a little bothered by the opening sentence "This page documents the process that researchers must follow before asking Wikipedia contributors to participate in research studies such as surveys, interviews and experiments."

WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes. What WMF does own is its communication channels to me as a contributor and WMF has a right to control what occurs on those channels. Also I think WMF probably should be concerned about both its readers and its contributors being recruited through its channels (as either might be being recruited). I think this distinction should be made, e.g.

"This page documents the process that researchers must follow if they wish to use Wikipedia's (WMF's?) communication channels to recruit people to participate in research studies such as surveys, interviews and experiments. Communication channels include its mailing lists, its Project pages, Talk pages, and User Talk pages [and whatever else I've forgotten]." 

 

If researchers want to recruit WPians via non-WMF means, I don’t think it’s any business of WMF’s. An example might be a researcher who wanted to contact WPians via chapters or thorgs; I would leave it for the chapter/thorg to decide if they wanted to assist the researcher via their communication channels.

 

Of course, the practical reality of it is that some researchers (oblivious of WMF’s concerns in relation to recruitment of WPians to research projects) will simply use WMF’s channels without asking nicely first. Obviously we can remove such requests on-wiki and follow up any email requests with the commentary that this was not an approved request. In my category of [whatever else I’ve forgotten], I guess there are things like Facebook groups and any other social media presence.

 

Also to be practical, if WMF is to have a process to vet research surveys, I think it has to be sufficiently fast and not be overly demanding to avoid the possibility of the researcher giving up (“too hard to deal with these people”) and simply spamming email, project pages, social media in the hope of recruiting some participants regardless. That is, if we make it too slow/hard to do the right thing, we effectively encourage doing the wrong thing. Also, what value-add can we give them to reward those who do the right thing? It’s nice to have a carrot as well as a stick when it comes to onerous processes J

 

Because of the criticism of “not giving back”, could we perhaps do things to try to make the researcher feel part of the community to make “giving back” more likely? For example, could we give them a slot every now and again to talk about their project in the R&D Showcase? Encourage them to be on this mailing list. Are we at a point where it might make sense to organise a Wikipedia research conference to help build a research community? Just thinking aloud here …

 

Kerry

 

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Aaron Halfaker
Sent: Thursday, 17 July 2014 6:59 AM
To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

RCOM review is still alive and looking for new reviewers (really, coordinators).  Researchers can be directed to me or Dario ([hidden email]) to be assigned a reviewer.  There is also a proposed policy on enwiki that could use some eyeballs: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

 

On Wed, Jul 16, 2014 at 11:16 AM, Federico Leva (Nemo) <[hidden email]> wrote:

phoebe ayers, 16/07/2014 19:21:

> (Personally, I think the answer should be to resuscitate RCOM, but
> that's easy to say and harder to do!)

IMHO in the meanwhile the most useful thing folks can do is subscribing
to the feed of new research pages:
<https://meta.wikimedia.org/w/index.php?title=Special:NewPages&feed=atom&hidebots=1&hideredirs=1&limit=500&offset=&namespace=202>
It's easier to build a functioning RCOM out of an active community of
"reviewers", than the other way round.

Nemo

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Kerry Raymond

Yes, I meant the community/communities of WMF. But the authority of the community derives from WMF, which chooses to delegate such matters. I think that “advise” is a good word to use.

 

Kerry

 

 


From: Amir E. Aharoni [mailto:[hidden email]]
Sent: Thursday, 17 July 2014 5:37 PM
To: [hidden email]; Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

> WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes.

I don't think that it really should be about WMF. The WMF shouldn't enforce anything. The community can formulate good practices for researchers and _advise_ community members not to cooperate with researchers who don't follow these practices. Not much more is needed.



--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
“We're living in pieces,
I want to live in peace.” – T. Moore

 

2014-07-17 8:24 GMT+03:00 Kerry Raymond <[hidden email]>:

Just saying here what I already put on the Talk page:

 

I am a little bothered by the opening sentence "This page documents the process that researchers must follow before asking Wikipedia contributors to participate in research studies such as surveys, interviews and experiments."

WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes. What WMF does own is its communication channels to me as a contributor and WMF has a right to control what occurs on those channels. Also I think WMF probably should be concerned about both its readers and its contributors being recruited through its channels (as either might be being recruited). I think this distinction should be made, e.g.

"This page documents the process that researchers must follow if they wish to use Wikipedia's (WMF's?) communication channels to recruit people to participate in research studies such as surveys, interviews and experiments. Communication channels include its mailing lists, its Project pages, Talk pages, and User Talk pages [and whatever else I've forgotten]." 

 

If researchers want to recruit WPians via non-WMF means, I don’t think it’s any business of WMF’s. An example might be a researcher who wanted to contact WPians via chapters or thorgs; I would leave it for the chapter/thorg to decide if they wanted to assist the researcher via their communication channels.

 

Of course, the practical reality of it is that some researchers (oblivious of WMF’s concerns in relation to recruitment of WPians to research projects) will simply use WMF’s channels without asking nicely first. Obviously we can remove such requests on-wiki and follow up any email requests with the commentary that this was not an approved request. In my category of [whatever else I’ve forgotten], I guess there are things like Facebook groups and any other social media presence.

 

Also to be practical, if WMF is to have a process to vet research surveys, I think it has to be sufficiently fast and not be overly demanding to avoid the possibility of the researcher giving up (“too hard to deal with these people”) and simply spamming email, project pages, social media in the hope of recruiting some participants regardless. That is, if we make it too slow/hard to do the right thing, we effectively encourage doing the wrong thing. Also, what value-add can we give them to reward those who do the right thing? It’s nice to have a carrot as well as a stick when it comes to onerous processes J

 

Because of the criticism of “not giving back”, could we perhaps do things to try to make the researcher feel part of the community to make “giving back” more likely? For example, could we give them a slot every now and again to talk about their project in the R&D Showcase? Encourage them to be on this mailing list. Are we at a point where it might make sense to organise a Wikipedia research conference to help build a research community? Just thinking aloud here …

 

Kerry

 

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Aaron Halfaker
Sent: Thursday, 17 July 2014 6:59 AM
To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

RCOM review is still alive and looking for new reviewers (really, coordinators).  Researchers can be directed to me or Dario ([hidden email]) to be assigned a reviewer.  There is also a proposed policy on enwiki that could use some eyeballs: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

 

On Wed, Jul 16, 2014 at 11:16 AM, Federico Leva (Nemo) <[hidden email]> wrote:

phoebe ayers, 16/07/2014 19:21:

> (Personally, I think the answer should be to resuscitate RCOM, but
> that's easy to say and harder to do!)

IMHO in the meanwhile the most useful thing folks can do is subscribing
to the feed of new research pages:
<https://meta.wikimedia.org/w/index.php?title=Special:NewPages&feed=atom&hidebots=1&hideredirs=1&limit=500&offset=&namespace=202>
It's easier to build a functioning RCOM out of an active community of
"reviewers", than the other way round.

Nemo

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Heather Ford-3
Agree with Kerry that we really need to have a more flexible process that speaks to the main problem that (I think) RCOM was started to solve i.e. that Wikipedians were getting tired of being continually contacted by researchers to fill out *surveys*. I'm not sure where feelings are about that right now (I certainly haven't seen a huge amount of surveys myself) but I guess the big question right now is whether RCOM is actually active or not. I must say that I was surprised, Aaron, when I read that it is active because I had heard from others in your team about a year or two ago that this wasn't going to be the vehicle for obtaining permission going forward and that a new, more lightweight process was being designed. As Nathan discusses on the Wikimedia-l list, there aren't many indications that RCOM is still active. Perhaps there has been a recent decision to resuscitate it? If that's the case, let us know about it :) And then we can discuss what needs to happen to build a good process. 

One immediate requirement that I've been talking to others about is finding ways of making the case to the WMF as a group of researchers for the anonymization of country level data, for example. I've spoken to a few researchers (and I myself made a request about a year ago that hasn't been responded to) and it seems like some work is required by the foundation to do this anonymisation but that there are a few of us who would be really keen to use this data to produce research very valuable to Wikipedia - especially from smaller language versions/developing countries. Having an official process that assesses how worthwhile this investment of time would be to the Foundation would be a great idea, I think, but right now there seems to be a general focus on the research that the Foundation does itself rather than enabling researchers outside. I know how busy Aaron and Dario (and others in the team) are so perhaps this requires a new position to coordinate between researchers and Foundation resources?

Anyway, I think the big question right now is whether there are any plans for RCOM that have been made by the research team and the only people who can answer that are folks in the research team :)

Best,
Heather.



On 17 July 2014 08:49, Kerry Raymond <[hidden email]> wrote:

Yes, I meant the community/communities of WMF. But the authority of the community derives from WMF, which chooses to delegate such matters. I think that “advise” is a good word to use.

 

Kerry

 

 


From: Amir E. Aharoni [mailto:[hidden email]]
Sent: Thursday, 17 July 2014 5:37 PM
To: [hidden email]; Research into Wikimedia content and communities


Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

> WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes.

I don't think that it really should be about WMF. The WMF shouldn't enforce anything. The community can formulate good practices for researchers and _advise_ community members not to cooperate with researchers who don't follow these practices. Not much more is needed.



--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
“We're living in pieces,
I want to live in peace.” – T. Moore

 

2014-07-17 8:24 GMT+03:00 Kerry Raymond <[hidden email]>:

Just saying here what I already put on the Talk page:

 

I am a little bothered by the opening sentence "This page documents the process that researchers must follow before asking Wikipedia contributors to participate in research studies such as surveys, interviews and experiments."

WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes. What WMF does own is its communication channels to me as a contributor and WMF has a right to control what occurs on those channels. Also I think WMF probably should be concerned about both its readers and its contributors being recruited through its channels (as either might be being recruited). I think this distinction should be made, e.g.

"This page documents the process that researchers must follow if they wish to use Wikipedia's (WMF's?) communication channels to recruit people to participate in research studies such as surveys, interviews and experiments. Communication channels include its mailing lists, its Project pages, Talk pages, and User Talk pages [and whatever else I've forgotten]." 

 

If researchers want to recruit WPians via non-WMF means, I don’t think it’s any business of WMF’s. An example might be a researcher who wanted to contact WPians via chapters or thorgs; I would leave it for the chapter/thorg to decide if they wanted to assist the researcher via their communication channels.

 

Of course, the practical reality of it is that some researchers (oblivious of WMF’s concerns in relation to recruitment of WPians to research projects) will simply use WMF’s channels without asking nicely first. Obviously we can remove such requests on-wiki and follow up any email requests with the commentary that this was not an approved request. In my category of [whatever else I’ve forgotten], I guess there are things like Facebook groups and any other social media presence.

 

Also to be practical, if WMF is to have a process to vet research surveys, I think it has to be sufficiently fast and not be overly demanding to avoid the possibility of the researcher giving up (“too hard to deal with these people”) and simply spamming email, project pages, social media in the hope of recruiting some participants regardless. That is, if we make it too slow/hard to do the right thing, we effectively encourage doing the wrong thing. Also, what value-add can we give them to reward those who do the right thing? It’s nice to have a carrot as well as a stick when it comes to onerous processes J

 

Because of the criticism of “not giving back”, could we perhaps do things to try to make the researcher feel part of the community to make “giving back” more likely? For example, could we give them a slot every now and again to talk about their project in the R&D Showcase? Encourage them to be on this mailing list. Are we at a point where it might make sense to organise a Wikipedia research conference to help build a research community? Just thinking aloud here …

 

Kerry

 

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Aaron Halfaker
Sent: Thursday, 17 July 2014 6:59 AM
To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

RCOM review is still alive and looking for new reviewers (really, coordinators).  Researchers can be directed to me or Dario ([hidden email]) to be assigned a reviewer.  There is also a proposed policy on enwiki that could use some eyeballs: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

 

On Wed, Jul 16, 2014 at 11:16 AM, Federico Leva (Nemo) <[hidden email]> wrote:

phoebe ayers, 16/07/2014 19:21:

> (Personally, I think the answer should be to resuscitate RCOM, but
> that's easy to say and harder to do!)

IMHO in the meanwhile the most useful thing folks can do is subscribing
to the feed of new research pages:
<https://meta.wikimedia.org/w/index.php?title=Special:NewPages&feed=atom&hidebots=1&hideredirs=1&limit=500&offset=&namespace=202>
It's easier to build a functioning RCOM out of an active community of
"reviewers", than the other way round.

Nemo

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Aaron Halfaker-2
Aaron, when I read that it is active because I had heard from others in your team about a year or two ago that this wasn't going to be the vehicle for obtaining permission going forward and that a new, more lightweight process was being designed.

1) If anyone told you that we are no longer active, they were wrong.
2) The "lightweight" process you refer to is what I linked to in enwiki in my previous response.  See again: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

Generally, there seems to be a misconception that RCom == paid WMF activities.  While RCom involves a relationship with the Wikimedia Foundation, our activities as part of RCom are 100% volunteer and open to participation from other Wikipedians (seriously, let me know if you want to help out!), and as such, our backlog tends to suffer when our available volunteer time does.  FWIW, I became involved in this work as a volunteer (before I started working with the WMF).  With that in mind, it seems like we are not discussing RCom itself which is mostly inactive -- so much as we are discussing the subject recruitment review process which is still active.  Let me state this clearly: If you send an email to me or Dario about a research project that you would like reviewed, we will help you coordinate a review.  Our job as review coordinators is to make sure that the study is adequately documented and that Wikipedians and other researchers are pulled in to discuss the material.  We don't just welcome broad involvement -- we need it!  We all suffer from the lack of it.  Please show up help us!

To give you some context on the current stats and situation, I should probably give a bit of history.  I've been working to improve subject recruitment review -- with the goal of improving interactions between researchers and Wikipedians -- for years.  Let me first say that I'm game to make this better.  In my experience, the biggest issue to documenting the a review/endorsement/whatever process that I have come across is this: there seems to be a lot of people who feel that minimizing process description provides power and adaptability to intended processes[1].  It's these people that I've regularly battled in my frequent efforts to increase the formalization around the subject recruitment proposal vetting process (e.g. SRAG had a structured appeals process and stated timelines).  The result of these battles is the severely under-documented process "described" in meta:R:FAQ.

Here's some links to my previous work on subject recruitment process that will show these old discussions about process creep
For new work, see my current (but stalled for about 1.5 years) push for a structured process on English Wikipedia. https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment  See also the checklist I have been working on with Lane. https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment/Wikipedian_checklist

When you review these docs and the corresponding conversations, please keep in mind that I was a new Wikipedian for the development of WP:SRAG and WP:Research, so I made some really critical mistakes -- like taking hyperbolic criticism of the proposals personally. :\ 

So what now?  Well, in the meantime, if you let me know about some subject recruitment you want to do, I'll help you find someone to coordinate a review that fits within the process described in the RCom docs.  In the short term, are any of you folks interested in going through some iterations of the new WP:Research_recruitment policy doc?  

-Aaron


On Thu, Jul 17, 2014 at 2:38 AM, Heather Ford <[hidden email]> wrote:
Agree with Kerry that we really need to have a more flexible process that speaks to the main problem that (I think) RCOM was started to solve i.e. that Wikipedians were getting tired of being continually contacted by researchers to fill out *surveys*. I'm not sure where feelings are about that right now (I certainly haven't seen a huge amount of surveys myself) but I guess the big question right now is whether RCOM is actually active or not. I must say that I was surprised, Aaron, when I read that it is active because I had heard from others in your team about a year or two ago that this wasn't going to be the vehicle for obtaining permission going forward and that a new, more lightweight process was being designed. As Nathan discusses on the Wikimedia-l list, there aren't many indications that RCOM is still active. Perhaps there has been a recent decision to resuscitate it? If that's the case, let us know about it :) And then we can discuss what needs to happen to build a good process. 

One immediate requirement that I've been talking to others about is finding ways of making the case to the WMF as a group of researchers for the anonymization of country level data, for example. I've spoken to a few researchers (and I myself made a request about a year ago that hasn't been responded to) and it seems like some work is required by the foundation to do this anonymisation but that there are a few of us who would be really keen to use this data to produce research very valuable to Wikipedia - especially from smaller language versions/developing countries. Having an official process that assesses how worthwhile this investment of time would be to the Foundation would be a great idea, I think, but right now there seems to be a general focus on the research that the Foundation does itself rather than enabling researchers outside. I know how busy Aaron and Dario (and others in the team) are so perhaps this requires a new position to coordinate between researchers and Foundation resources?

Anyway, I think the big question right now is whether there are any plans for RCOM that have been made by the research team and the only people who can answer that are folks in the research team :)

Best,
Heather.



On 17 July 2014 08:49, Kerry Raymond <[hidden email]> wrote:

Yes, I meant the community/communities of WMF. But the authority of the community derives from WMF, which chooses to delegate such matters. I think that “advise” is a good word to use.

 

Kerry

 

 


From: Amir E. Aharoni [mailto:[hidden email]]
Sent: Thursday, 17 July 2014 5:37 PM
To: [hidden email]; Research into Wikimedia content and communities


Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

> WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes.

I don't think that it really should be about WMF. The WMF shouldn't enforce anything. The community can formulate good practices for researchers and _advise_ community members not to cooperate with researchers who don't follow these practices. Not much more is needed.



--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
“We're living in pieces,
I want to live in peace.” – T. Moore

 

2014-07-17 8:24 GMT+03:00 Kerry Raymond <[hidden email]>:

Just saying here what I already put on the Talk page:

 

I am a little bothered by the opening sentence "This page documents the process that researchers must follow before asking Wikipedia contributors to participate in research studies such as surveys, interviews and experiments."

WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes. What WMF does own is its communication channels to me as a contributor and WMF has a right to control what occurs on those channels. Also I think WMF probably should be concerned about both its readers and its contributors being recruited through its channels (as either might be being recruited). I think this distinction should be made, e.g.

"This page documents the process that researchers must follow if they wish to use Wikipedia's (WMF's?) communication channels to recruit people to participate in research studies such as surveys, interviews and experiments. Communication channels include its mailing lists, its Project pages, Talk pages, and User Talk pages [and whatever else I've forgotten]." 

 

If researchers want to recruit WPians via non-WMF means, I don’t think it’s any business of WMF’s. An example might be a researcher who wanted to contact WPians via chapters or thorgs; I would leave it for the chapter/thorg to decide if they wanted to assist the researcher via their communication channels.

 

Of course, the practical reality of it is that some researchers (oblivious of WMF’s concerns in relation to recruitment of WPians to research projects) will simply use WMF’s channels without asking nicely first. Obviously we can remove such requests on-wiki and follow up any email requests with the commentary that this was not an approved request. In my category of [whatever else I’ve forgotten], I guess there are things like Facebook groups and any other social media presence.

 

Also to be practical, if WMF is to have a process to vet research surveys, I think it has to be sufficiently fast and not be overly demanding to avoid the possibility of the researcher giving up (“too hard to deal with these people”) and simply spamming email, project pages, social media in the hope of recruiting some participants regardless. That is, if we make it too slow/hard to do the right thing, we effectively encourage doing the wrong thing. Also, what value-add can we give them to reward those who do the right thing? It’s nice to have a carrot as well as a stick when it comes to onerous processes J

 

Because of the criticism of “not giving back”, could we perhaps do things to try to make the researcher feel part of the community to make “giving back” more likely? For example, could we give them a slot every now and again to talk about their project in the R&D Showcase? Encourage them to be on this mailing list. Are we at a point where it might make sense to organise a Wikipedia research conference to help build a research community? Just thinking aloud here …

 

Kerry

 

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Aaron Halfaker
Sent: Thursday, 17 July 2014 6:59 AM
To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

RCOM review is still alive and looking for new reviewers (really, coordinators).  Researchers can be directed to me or Dario ([hidden email]) to be assigned a reviewer.  There is also a proposed policy on enwiki that could use some eyeballs: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

 

On Wed, Jul 16, 2014 at 11:16 AM, Federico Leva (Nemo) <[hidden email]> wrote:

phoebe ayers, 16/07/2014 19:21:

> (Personally, I think the answer should be to resuscitate RCOM, but
> that's easy to say and harder to do!)

IMHO in the meanwhile the most useful thing folks can do is subscribing
to the feed of new research pages:
<https://meta.wikimedia.org/w/index.php?title=Special:NewPages&feed=atom&hidebots=1&hideredirs=1&limit=500&offset=&namespace=202>
It's easier to build a functioning RCOM out of an active community of
"reviewers", than the other way round.

Nemo

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Aaron Halfaker-3


Kerry said:
Because of the criticism of “not giving back”, could we perhaps do things to try to make the researcher feel part of the community to make “giving back” more likely? For example, could we give them a slot every now and again to talk about their project in the R&D Showcase? Encourage them to be on this mailing list. Are we at a point where it might make sense to organise a Wikipedia research conference to help build a research community? Just thinking aloud here …

This is a bit different than the main topic, so I wanted to break it out into another reply.   

We just had Nate Matias[0] from the MIT media lab present on his work at the last showcase[1].  We also just sent out a survey about the showcase that includes a call for recommended speakers at future showcases[2].  As for a Wikipedia research conference, see OpenSym[3] (formerly WikiSym) and Wikimania[4] (not as researchy, but a great venue to maximize wiki research impact). 



On Thu, Jul 17, 2014 at 8:30 AM, Aaron Halfaker <[hidden email]> wrote:
Aaron, when I read that it is active because I had heard from others in your team about a year or two ago that this wasn't going to be the vehicle for obtaining permission going forward and that a new, more lightweight process was being designed.

1) If anyone told you that we are no longer active, they were wrong.
2) The "lightweight" process you refer to is what I linked to in enwiki in my previous response.  See again: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

Generally, there seems to be a misconception that RCom == paid WMF activities.  While RCom involves a relationship with the Wikimedia Foundation, our activities as part of RCom are 100% volunteer and open to participation from other Wikipedians (seriously, let me know if you want to help out!), and as such, our backlog tends to suffer when our available volunteer time does.  FWIW, I became involved in this work as a volunteer (before I started working with the WMF).  With that in mind, it seems like we are not discussing RCom itself which is mostly inactive -- so much as we are discussing the subject recruitment review process which is still active.  Let me state this clearly: If you send an email to me or Dario about a research project that you would like reviewed, we will help you coordinate a review.  Our job as review coordinators is to make sure that the study is adequately documented and that Wikipedians and other researchers are pulled in to discuss the material.  We don't just welcome broad involvement -- we need it!  We all suffer from the lack of it.  Please show up help us!

To give you some context on the current stats and situation, I should probably give a bit of history.  I've been working to improve subject recruitment review -- with the goal of improving interactions between researchers and Wikipedians -- for years.  Let me first say that I'm game to make this better.  In my experience, the biggest issue to documenting the a review/endorsement/whatever process that I have come across is this: there seems to be a lot of people who feel that minimizing process description provides power and adaptability to intended processes[1].  It's these people that I've regularly battled in my frequent efforts to increase the formalization around the subject recruitment proposal vetting process (e.g. SRAG had a structured appeals process and stated timelines).  The result of these battles is the severely under-documented process "described" in meta:R:FAQ.

Here's some links to my previous work on subject recruitment process that will show these old discussions about process creep
For new work, see my current (but stalled for about 1.5 years) push for a structured process on English Wikipedia. https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment  See also the checklist I have been working on with Lane. https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment/Wikipedian_checklist

When you review these docs and the corresponding conversations, please keep in mind that I was a new Wikipedian for the development of WP:SRAG and WP:Research, so I made some really critical mistakes -- like taking hyperbolic criticism of the proposals personally. :\ 

So what now?  Well, in the meantime, if you let me know about some subject recruitment you want to do, I'll help you find someone to coordinate a review that fits within the process described in the RCom docs.  In the short term, are any of you folks interested in going through some iterations of the new WP:Research_recruitment policy doc?  

-Aaron


On Thu, Jul 17, 2014 at 2:38 AM, Heather Ford <[hidden email]> wrote:
Agree with Kerry that we really need to have a more flexible process that speaks to the main problem that (I think) RCOM was started to solve i.e. that Wikipedians were getting tired of being continually contacted by researchers to fill out *surveys*. I'm not sure where feelings are about that right now (I certainly haven't seen a huge amount of surveys myself) but I guess the big question right now is whether RCOM is actually active or not. I must say that I was surprised, Aaron, when I read that it is active because I had heard from others in your team about a year or two ago that this wasn't going to be the vehicle for obtaining permission going forward and that a new, more lightweight process was being designed. As Nathan discusses on the Wikimedia-l list, there aren't many indications that RCOM is still active. Perhaps there has been a recent decision to resuscitate it? If that's the case, let us know about it :) And then we can discuss what needs to happen to build a good process. 

One immediate requirement that I've been talking to others about is finding ways of making the case to the WMF as a group of researchers for the anonymization of country level data, for example. I've spoken to a few researchers (and I myself made a request about a year ago that hasn't been responded to) and it seems like some work is required by the foundation to do this anonymisation but that there are a few of us who would be really keen to use this data to produce research very valuable to Wikipedia - especially from smaller language versions/developing countries. Having an official process that assesses how worthwhile this investment of time would be to the Foundation would be a great idea, I think, but right now there seems to be a general focus on the research that the Foundation does itself rather than enabling researchers outside. I know how busy Aaron and Dario (and others in the team) are so perhaps this requires a new position to coordinate between researchers and Foundation resources?

Anyway, I think the big question right now is whether there are any plans for RCOM that have been made by the research team and the only people who can answer that are folks in the research team :)

Best,
Heather.



On 17 July 2014 08:49, Kerry Raymond <[hidden email]> wrote:

Yes, I meant the community/communities of WMF. But the authority of the community derives from WMF, which chooses to delegate such matters. I think that “advise” is a good word to use.

 

Kerry

 

 


From: Amir E. Aharoni [mailto:[hidden email]]
Sent: Thursday, 17 July 2014 5:37 PM
To: [hidden email]; Research into Wikimedia content and communities


Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

> WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes.

I don't think that it really should be about WMF. The WMF shouldn't enforce anything. The community can formulate good practices for researchers and _advise_ community members not to cooperate with researchers who don't follow these practices. Not much more is needed.



--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
“We're living in pieces,
I want to live in peace.” – T. Moore

 

2014-07-17 8:24 GMT+03:00 Kerry Raymond <[hidden email]>:

Just saying here what I already put on the Talk page:

 

I am a little bothered by the opening sentence "This page documents the process that researchers must follow before asking Wikipedia contributors to participate in research studies such as surveys, interviews and experiments."

WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes. What WMF does own is its communication channels to me as a contributor and WMF has a right to control what occurs on those channels. Also I think WMF probably should be concerned about both its readers and its contributors being recruited through its channels (as either might be being recruited). I think this distinction should be made, e.g.

"This page documents the process that researchers must follow if they wish to use Wikipedia's (WMF's?) communication channels to recruit people to participate in research studies such as surveys, interviews and experiments. Communication channels include its mailing lists, its Project pages, Talk pages, and User Talk pages [and whatever else I've forgotten]." 

 

If researchers want to recruit WPians via non-WMF means, I don’t think it’s any business of WMF’s. An example might be a researcher who wanted to contact WPians via chapters or thorgs; I would leave it for the chapter/thorg to decide if they wanted to assist the researcher via their communication channels.

 

Of course, the practical reality of it is that some researchers (oblivious of WMF’s concerns in relation to recruitment of WPians to research projects) will simply use WMF’s channels without asking nicely first. Obviously we can remove such requests on-wiki and follow up any email requests with the commentary that this was not an approved request. In my category of [whatever else I’ve forgotten], I guess there are things like Facebook groups and any other social media presence.

 

Also to be practical, if WMF is to have a process to vet research surveys, I think it has to be sufficiently fast and not be overly demanding to avoid the possibility of the researcher giving up (“too hard to deal with these people”) and simply spamming email, project pages, social media in the hope of recruiting some participants regardless. That is, if we make it too slow/hard to do the right thing, we effectively encourage doing the wrong thing. Also, what value-add can we give them to reward those who do the right thing? It’s nice to have a carrot as well as a stick when it comes to onerous processes J

 

Because of the criticism of “not giving back”, could we perhaps do things to try to make the researcher feel part of the community to make “giving back” more likely? For example, could we give them a slot every now and again to talk about their project in the R&D Showcase? Encourage them to be on this mailing list. Are we at a point where it might make sense to organise a Wikipedia research conference to help build a research community? Just thinking aloud here …

 

Kerry

 

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Aaron Halfaker
Sent: Thursday, 17 July 2014 6:59 AM
To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

RCOM review is still alive and looking for new reviewers (really, coordinators).  Researchers can be directed to me or Dario ([hidden email]) to be assigned a reviewer.  There is also a proposed policy on enwiki that could use some eyeballs: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

 

On Wed, Jul 16, 2014 at 11:16 AM, Federico Leva (Nemo) <[hidden email]> wrote:

phoebe ayers, 16/07/2014 19:21:

> (Personally, I think the answer should be to resuscitate RCOM, but
> that's easy to say and harder to do!)

IMHO in the meanwhile the most useful thing folks can do is subscribing
to the feed of new research pages:
<https://meta.wikimedia.org/w/index.php?title=Special:NewPages&feed=atom&hidebots=1&hideredirs=1&limit=500&offset=&namespace=202>
It's easier to build a functioning RCOM out of an active community of
"reviewers", than the other way round.

Nemo

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

phoebe ayers-3
In reply to this post by Heather Ford-3

On Thu, Jul 17, 2014 at 2:38 AM, Heather Ford <[hidden email]> wrote:
Agree with Kerry that we really need to have a more flexible process that speaks to the main problem that (I think) RCOM was started to solve i.e. that Wikipedians were getting tired of being continually contacted by researchers to fill out *surveys*.


That's correct, afaik that was the original motivation, along with some of the concerns that Lane/Nathan raised in the other list -- i.e. that it was difficult for contributors to tell if a survey was ethical, vetted, etc. Frankly, I think some long-term contributors just felt jaded -- for a while it seemed there were lots of surveys and studies to try to find out things that seemed intuitively obvious if you were a participant in the community. I think Heather is right that it seems like there have been fewer surveys in recent years, for whatever reason.

Part of the problem is a somewhat subtle demographic one: while contributors to Wikipedia do turn over, so newer contributors will not necessarily have seen lots of surveys, very heavy editors and admins (who are often easier to identify) tend to be long-term participants who might have been surveyed many times. Additionally, the people who follow mailing lists, social media, etc. (or at least the people who speak up on those channels) skew towards very-long-term contributors who have strong opinions and have seen it all before. So, if you advertise your survey on the mailing list, that's the population you get, and that's the feedback you get. (But it's a catch-22; there's not really other obvious mass channels).

Anyway, this is a hard problem without super-obvious solutions, and not one that there's a lot of models for -- very few online projects are simultaneously as open with their data and as interesting for research purposes!

best,
Phoebe


--
* I use this address for lists; send personal messages to phoebe.ayers <at> gmail.com *

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Jonathan Morgan
In reply to this post by Heather Ford-3


First, I wanted to highlight the important issue that Heather raises here, because although it's a separate issue, it's an important one:

On Thu, Jul 17, 2014 at 2:38 AM, Heather Ford <[hidden email]> wrote:
...

One immediate requirement that I've been talking to others about is finding ways of making the case to the WMF as a group of researchers for the anonymization of country level data, for example. I've spoken to a few researchers (and I myself made a request about a year ago that hasn't been responded to) and it seems like some work is required by the foundation to do this anonymisation but that there are a few of us who would be really keen to use this data to produce research very valuable to Wikipedia - especially from smaller language versions/developing countries. Having an official process that assesses how worthwhile this investment of time would be to the Foundation would be a great idea, I think, but right now there seems to be a general focus on the research that the Foundation does itself rather than enabling researchers outside. I know how busy Aaron and Dario (and others in the team) are so perhaps this requires a new position to coordinate between researchers and Foundation resources?

Anyway, I think the big question right now is whether there are any plans for RCOM that have been made by the research team and the only people who can answer that are folks in the research team :)

Best,
Heather.



As a community-run group, RCOM doesn't have any role in making non-public data available to researchers. However, Aaron and I are putting together a proposal for a workshop that would address issues like this. That's work we're doing in an official capacity, as opposed to the RCOM work, which is volunteer.

On RCOM more generally... I think clarifying the role of the committee, and getting a larger and more diverse set of people involved, might help make RCOM work. But as Aaron can attest, it is difficult to get people to agree on what RCOMs role should be, let alone get them to work for RCOM.

I've been involved with RCOM for a while, albeit not very actively. Unfortunately, I think that the fact that the only people who "review" requests happen to be* WMF staffers contributes to confusion about RCOM's role and it's authority. IMO, if RCOM or any other subject recruitment review process is to succeed, we need:
  • more wiki-researchers who are willing to regularly participate in both peer review and in developing better process guidelines and standards (it's really just Aaron right now)
  • more Wikipedians who are willing to do the same
  • some degree of buy-in from the Wikimedia community as a whole. RCOM needs legitimacy. But where, and from whom? Subject recruitment is a global concern, but the proposed subject recruitment process is focused on en-wiki (mostly because that's where most of the relevant research activities that we are aware of are happening). How to make RCOM more global?
RCOM is in a tough spot right now. We can't force researchers to submit their proposals, or abide by the suggestions/recommendations/decisions/whatever that result from their review. But because we look like an official body, it's easy to blame us for failing to prevent disruptive research (if you're a community member), for "rubber stamping" research that we like (ditto), or for drowning research in red tape (if you're a wiki-researcher). 


- J

*we were wiki-researchers first!


 


On 17 July 2014 08:49, Kerry Raymond <[hidden email]> wrote:

Yes, I meant the community/communities of WMF. But the authority of the community derives from WMF, which chooses to delegate such matters. I think that “advise” is a good word to use.

 

Kerry

 

 


From: Amir E. Aharoni [mailto:[hidden email]]
Sent: Thursday, 17 July 2014 5:37 PM
To: [hidden email]; Research into Wikimedia content and communities


Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

> WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes.

I don't think that it really should be about WMF. The WMF shouldn't enforce anything. The community can formulate good practices for researchers and _advise_ community members not to cooperate with researchers who don't follow these practices. Not much more is needed.



--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
“We're living in pieces,
I want to live in peace.” – T. Moore

 

2014-07-17 8:24 GMT+03:00 Kerry Raymond <[hidden email]>:

Just saying here what I already put on the Talk page:

 

I am a little bothered by the opening sentence "This page documents the process that researchers must follow before asking Wikipedia contributors to participate in research studies such as surveys, interviews and experiments."

WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes. What WMF does own is its communication channels to me as a contributor and WMF has a right to control what occurs on those channels. Also I think WMF probably should be concerned about both its readers and its contributors being recruited through its channels (as either might be being recruited). I think this distinction should be made, e.g.

"This page documents the process that researchers must follow if they wish to use Wikipedia's (WMF's?) communication channels to recruit people to participate in research studies such as surveys, interviews and experiments. Communication channels include its mailing lists, its Project pages, Talk pages, and User Talk pages [and whatever else I've forgotten]." 

 

If researchers want to recruit WPians via non-WMF means, I don’t think it’s any business of WMF’s. An example might be a researcher who wanted to contact WPians via chapters or thorgs; I would leave it for the chapter/thorg to decide if they wanted to assist the researcher via their communication channels.

 

Of course, the practical reality of it is that some researchers (oblivious of WMF’s concerns in relation to recruitment of WPians to research projects) will simply use WMF’s channels without asking nicely first. Obviously we can remove such requests on-wiki and follow up any email requests with the commentary that this was not an approved request. In my category of [whatever else I’ve forgotten], I guess there are things like Facebook groups and any other social media presence.

 

Also to be practical, if WMF is to have a process to vet research surveys, I think it has to be sufficiently fast and not be overly demanding to avoid the possibility of the researcher giving up (“too hard to deal with these people”) and simply spamming email, project pages, social media in the hope of recruiting some participants regardless. That is, if we make it too slow/hard to do the right thing, we effectively encourage doing the wrong thing. Also, what value-add can we give them to reward those who do the right thing? It’s nice to have a carrot as well as a stick when it comes to onerous processes J

 

Because of the criticism of “not giving back”, could we perhaps do things to try to make the researcher feel part of the community to make “giving back” more likely? For example, could we give them a slot every now and again to talk about their project in the R&D Showcase? Encourage them to be on this mailing list. Are we at a point where it might make sense to organise a Wikipedia research conference to help build a research community? Just thinking aloud here …

 

Kerry

 

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Aaron Halfaker
Sent: Thursday, 17 July 2014 6:59 AM
To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

RCOM review is still alive and looking for new reviewers (really, coordinators).  Researchers can be directed to me or Dario ([hidden email]) to be assigned a reviewer.  There is also a proposed policy on enwiki that could use some eyeballs: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

 

On Wed, Jul 16, 2014 at 11:16 AM, Federico Leva (Nemo) <[hidden email]> wrote:

phoebe ayers, 16/07/2014 19:21:

> (Personally, I think the answer should be to resuscitate RCOM, but
> that's easy to say and harder to do!)

IMHO in the meanwhile the most useful thing folks can do is subscribing
to the feed of new research pages:
<https://meta.wikimedia.org/w/index.php?title=Special:NewPages&feed=atom&hidebots=1&hideredirs=1&limit=500&offset=&namespace=202>
It's easier to build a functioning RCOM out of an active community of
"reviewers", than the other way round.

Nemo

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l




--
Jonathan T. Morgan
Learning Strategist
Wikimedia Foundation


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Kerry Raymond
In reply to this post by Aaron Halfaker-3

I guess I was not so much thinking of an general invitation to the R&D Showcase but a specific “expectation” (albeit couched as an invitation) on those given permission to recruit via WMF channels to give a few short (or long as appropriate to the stage of their research) talks on their project. Ditto research projects supported through IEG or similar.

 

I agree that OpenSym is available as a research conference but it is not run by our community and therefore doesn’t help to create a sense of community with the researchers in question. Wikimania is run by our community but isn’t a research conference (would not count as a publication for academic purposes). But I don’t know if it’s realistic to try to establish another conference in terms of the volunteer effort to run it.

 

Kerry

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Aaron Halfaker
Sent: Friday, 18 July 2014 1:45 AM
To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

 

 

Kerry said:

Because of the criticism of “not giving back”, could we perhaps do things to try to make the researcher feel part of the community to make “giving back” more likely? For example, could we give them a slot every now and again to talk about their project in the R&D Showcase? Encourage them to be on this mailing list. Are we at a point where it might make sense to organise a Wikipedia research conference to help build a research community? Just thinking aloud here …

 

This is a bit different than the main topic, so I wanted to break it out into another reply.   

 

We just had Nate Matias[0] from the MIT media lab present on his work at the last showcase[1].  We also just sent out a survey about the showcase that includes a call for recommended speakers at future showcases[2].  As for a Wikipedia research conference, see OpenSym[3] (formerly WikiSym) and Wikimania[4] (not as researchy, but a great venue to maximize wiki research impact). 

 

 

On Thu, Jul 17, 2014 at 8:30 AM, Aaron Halfaker <[hidden email]> wrote:

Aaron, when I read that it is active because I had heard from others in your team about a year or two ago that this wasn't going to be the vehicle for obtaining permission going forward and that a new, more lightweight process was being designed.

 

1) If anyone told you that we are no longer active, they were wrong.

2) The "lightweight" process you refer to is what I linked to in enwiki in my previous response.  See again: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

 

Generally, there seems to be a misconception that RCom == paid WMF activities.  While RCom involves a relationship with the Wikimedia Foundation, our activities as part of RCom are 100% volunteer and open to participation from other Wikipedians (seriously, let me know if you want to help out!), and as such, our backlog tends to suffer when our available volunteer time does.  FWIW, I became involved in this work as a volunteer (before I started working with the WMF).  With that in mind, it seems like we are not discussing RCom itself which is mostly inactive -- so much as we are discussing the subject recruitment review process which is still active.  Let me state this clearly: If you send an email to me or Dario about a research project that you would like reviewed, we will help you coordinate a review.  Our job as review coordinators is to make sure that the study is adequately documented and that Wikipedians and other researchers are pulled in to discuss the material.  We don't just welcome broad involvement -- we need it!  We all suffer from the lack of it.  Please show up help us!

 

To give you some context on the current stats and situation, I should probably give a bit of history.  I've been working to improve subject recruitment review -- with the goal of improving interactions between researchers and Wikipedians -- for years.  Let me first say that I'm game to make this better.  In my experience, the biggest issue to documenting the a review/endorsement/whatever process that I have come across is this: there seems to be a lot of people who feel that minimizing process description provides power and adaptability to intended processes[1].  It's these people that I've regularly battled in my frequent efforts to increase the formalization around the subject recruitment proposal vetting process (e.g. SRAG had a structured appeals process and stated timelines).  The result of these battles is the severely under-documented process "described" in meta:R:FAQ.

 

Here's some links to my previous work on subject recruitment process that will show these old discussions about process creep

·  https://en.wikipedia.org/wiki/Wikipedia:Subject_Recruitment_Approvals_Group

o https://en.wikipedia.org/wiki/Wikipedia_talk:Subject_Recruitment_Approvals_Group

·  https://en.wikipedia.org/w/index.php?title=Wikipedia:Research&oldid=354600173

o https://en.wikipedia.org/wiki/Wikipedia_talk:Research/Archive_1

o https://en.wikipedia.org/wiki/Wikipedia_talk:Research/Archive_2 -- Note that this was actually an enwiki policy for about 5 hours before the RfC was overturned due to too few editors being involved in the straw poll.

For new work, see my current (but stalled for about 1.5 years) push for a structured process on English Wikipedia. https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment  See also the checklist I have been working on with Lane. https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment/Wikipedian_checklist

 

When you review these docs and the corresponding conversations, please keep in mind that I was a new Wikipedian for the development of WP:SRAG and WP:Research, so I made some really critical mistakes -- like taking hyperbolic criticism of the proposals personally. :\ 

 

So what now?  Well, in the meantime, if you let me know about some subject recruitment you want to do, I'll help you find someone to coordinate a review that fits within the process described in the RCom docs.  In the short term, are any of you folks interested in going through some iterations of the new WP:Research_recruitment policy doc?  

 

-Aaron

 

On Thu, Jul 17, 2014 at 2:38 AM, Heather Ford <[hidden email]> wrote:

Agree with Kerry that we really need to have a more flexible process that speaks to the main problem that (I think) RCOM was started to solve i.e. that Wikipedians were getting tired of being continually contacted by researchers to fill out *surveys*. I'm not sure where feelings are about that right now (I certainly haven't seen a huge amount of surveys myself) but I guess the big question right now is whether RCOM is actually active or not. I must say that I was surprised, Aaron, when I read that it is active because I had heard from others in your team about a year or two ago that this wasn't going to be the vehicle for obtaining permission going forward and that a new, more lightweight process was being designed. As Nathan discusses on the Wikimedia-l list, there aren't many indications that RCOM is still active. Perhaps there has been a recent decision to resuscitate it? If that's the case, let us know about it :) And then we can discuss what needs to happen to build a good process. 

 

One immediate requirement that I've been talking to others about is finding ways of making the case to the WMF as a group of researchers for the anonymization of country level data, for example. I've spoken to a few researchers (and I myself made a request about a year ago that hasn't been responded to) and it seems like some work is required by the foundation to do this anonymisation but that there are a few of us who would be really keen to use this data to produce research very valuable to Wikipedia - especially from smaller language versions/developing countries. Having an official process that assesses how worthwhile this investment of time would be to the Foundation would be a great idea, I think, but right now there seems to be a general focus on the research that the Foundation does itself rather than enabling researchers outside. I know how busy Aaron and Dario (and others in the team) are so perhaps this requires a new position to coordinate between researchers and Foundation resources?

 

Anyway, I think the big question right now is whether there are any plans for RCOM that have been made by the research team and the only people who can answer that are folks in the research team :)

 

Best,

Heather.


 

On 17 July 2014 08:49, Kerry Raymond <[hidden email]> wrote:

Yes, I meant the community/communities of WMF. But the authority of the community derives from WMF, which chooses to delegate such matters. I think that “advise” is a good word to use.

 

Kerry

 

 


From: Amir E. Aharoni [mailto:[hidden email]]
Sent: Thursday, 17 July 2014 5:37 PM
To: [hidden email]; Research into Wikimedia content and communities


Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

> WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes.

I don't think that it really should be about WMF. The WMF shouldn't enforce anything. The community can formulate good practices for researchers and _advise_ community members not to cooperate with researchers who don't follow these practices. Not much more is needed.



--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
“We're living in pieces,
I want to live in peace.” – T. Moore

 

2014-07-17 8:24 GMT+03:00 Kerry Raymond <[hidden email]>:

Just saying here what I already put on the Talk page:

 

I am a little bothered by the opening sentence "This page documents the process that researchers must follow before asking Wikipedia contributors to participate in research studies such as surveys, interviews and experiments."

WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes. What WMF does own is its communication channels to me as a contributor and WMF has a right to control what occurs on those channels. Also I think WMF probably should be concerned about both its readers and its contributors being recruited through its channels (as either might be being recruited). I think this distinction should be made, e.g.

"This page documents the process that researchers must follow if they wish to use Wikipedia's (WMF's?) communication channels to recruit people to participate in research studies such as surveys, interviews and experiments. Communication channels include its mailing lists, its Project pages, Talk pages, and User Talk pages [and whatever else I've forgotten]." 

 

If researchers want to recruit WPians via non-WMF means, I don’t think it’s any business of WMF’s. An example might be a researcher who wanted to contact WPians via chapters or thorgs; I would leave it for the chapter/thorg to decide if they wanted to assist the researcher via their communication channels.

 

Of course, the practical reality of it is that some researchers (oblivious of WMF’s concerns in relation to recruitment of WPians to research projects) will simply use WMF’s channels without asking nicely first. Obviously we can remove such requests on-wiki and follow up any email requests with the commentary that this was not an approved request. In my category of [whatever else I’ve forgotten], I guess there are things like Facebook groups and any other social media presence.

 

Also to be practical, if WMF is to have a process to vet research surveys, I think it has to be sufficiently fast and not be overly demanding to avoid the possibility of the researcher giving up (“too hard to deal with these people”) and simply spamming email, project pages, social media in the hope of recruiting some participants regardless. That is, if we make it too slow/hard to do the right thing, we effectively encourage doing the wrong thing. Also, what value-add can we give them to reward those who do the right thing? It’s nice to have a carrot as well as a stick when it comes to onerous processes J

 

Because of the criticism of “not giving back”, could we perhaps do things to try to make the researcher feel part of the community to make “giving back” more likely? For example, could we give them a slot every now and again to talk about their project in the R&D Showcase? Encourage them to be on this mailing list. Are we at a point where it might make sense to organise a Wikipedia research conference to help build a research community? Just thinking aloud here …

 

Kerry

 

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Aaron Halfaker
Sent: Thursday, 17 July 2014 6:59 AM
To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

RCOM review is still alive and looking for new reviewers (really, coordinators).  Researchers can be directed to me or Dario ([hidden email]) to be assigned a reviewer.  There is also a proposed policy on enwiki that could use some eyeballs: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

 

On Wed, Jul 16, 2014 at 11:16 AM, Federico Leva (Nemo) <[hidden email]> wrote:

phoebe ayers, 16/07/2014 19:21:

> (Personally, I think the answer should be to resuscitate RCOM, but
> that's easy to say and harder to do!)

IMHO in the meanwhile the most useful thing folks can do is subscribing
to the feed of new research pages:
<https://meta.wikimedia.org/w/index.php?title=Special:NewPages&feed=atom&hidebots=1&hideredirs=1&limit=500&offset=&namespace=202>
It's easier to build a functioning RCOM out of an active community of
"reviewers", than the other way round.

Nemo

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Pine W
If RCOM needs more volunteer Wikimedians, the alive and well IEG Committee includes a Research Working Group that reviews grant proposals for WMF funding through the IEG program, so RCOM could reach out to IEGCom. I'm on IEGCom and the RWG but I can't speak for RCOM. (:

Pine


On Thu, Jul 17, 2014 at 3:10 PM, Kerry Raymond <[hidden email]> wrote:

I guess I was not so much thinking of an general invitation to the R&D Showcase but a specific “expectation” (albeit couched as an invitation) on those given permission to recruit via WMF channels to give a few short (or long as appropriate to the stage of their research) talks on their project. Ditto research projects supported through IEG or similar.

 

I agree that OpenSym is available as a research conference but it is not run by our community and therefore doesn’t help to create a sense of community with the researchers in question. Wikimania is run by our community but isn’t a research conference (would not count as a publication for academic purposes). But I don’t know if it’s realistic to try to establish another conference in terms of the volunteer effort to run it.

 

Kerry

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Aaron Halfaker
Sent: Friday, 18 July 2014 1:45 AM


To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

 

 

Kerry said:

Because of the criticism of “not giving back”, could we perhaps do things to try to make the researcher feel part of the community to make “giving back” more likely? For example, could we give them a slot every now and again to talk about their project in the R&D Showcase? Encourage them to be on this mailing list. Are we at a point where it might make sense to organise a Wikipedia research conference to help build a research community? Just thinking aloud here …

 

This is a bit different than the main topic, so I wanted to break it out into another reply.   

 

We just had Nate Matias[0] from the MIT media lab present on his work at the last showcase[1].  We also just sent out a survey about the showcase that includes a call for recommended speakers at future showcases[2].  As for a Wikipedia research conference, see OpenSym[3] (formerly WikiSym) and Wikimania[4] (not as researchy, but a great venue to maximize wiki research impact). 

 

 

On Thu, Jul 17, 2014 at 8:30 AM, Aaron Halfaker <[hidden email]> wrote:

Aaron, when I read that it is active because I had heard from others in your team about a year or two ago that this wasn't going to be the vehicle for obtaining permission going forward and that a new, more lightweight process was being designed.

 

1) If anyone told you that we are no longer active, they were wrong.

2) The "lightweight" process you refer to is what I linked to in enwiki in my previous response.  See again: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

 

Generally, there seems to be a misconception that RCom == paid WMF activities.  While RCom involves a relationship with the Wikimedia Foundation, our activities as part of RCom are 100% volunteer and open to participation from other Wikipedians (seriously, let me know if you want to help out!), and as such, our backlog tends to suffer when our available volunteer time does.  FWIW, I became involved in this work as a volunteer (before I started working with the WMF).  With that in mind, it seems like we are not discussing RCom itself which is mostly inactive -- so much as we are discussing the subject recruitment review process which is still active.  Let me state this clearly: If you send an email to me or Dario about a research project that you would like reviewed, we will help you coordinate a review.  Our job as review coordinators is to make sure that the study is adequately documented and that Wikipedians and other researchers are pulled in to discuss the material.  We don't just welcome broad involvement -- we need it!  We all suffer from the lack of it.  Please show up help us!

 

To give you some context on the current stats and situation, I should probably give a bit of history.  I've been working to improve subject recruitment review -- with the goal of improving interactions between researchers and Wikipedians -- for years.  Let me first say that I'm game to make this better.  In my experience, the biggest issue to documenting the a review/endorsement/whatever process that I have come across is this: there seems to be a lot of people who feel that minimizing process description provides power and adaptability to intended processes[1].  It's these people that I've regularly battled in my frequent efforts to increase the formalization around the subject recruitment proposal vetting process (e.g. SRAG had a structured appeals process and stated timelines).  The result of these battles is the severely under-documented process "described" in meta:R:FAQ.

 

Here's some links to my previous work on subject recruitment process that will show these old discussions about process creep

·  https://en.wikipedia.org/wiki/Wikipedia:Subject_Recruitment_Approvals_Group

o https://en.wikipedia.org/wiki/Wikipedia_talk:Subject_Recruitment_Approvals_Group

·  https://en.wikipedia.org/w/index.php?title=Wikipedia:Research&oldid=354600173

o https://en.wikipedia.org/wiki/Wikipedia_talk:Research/Archive_1

o https://en.wikipedia.org/wiki/Wikipedia_talk:Research/Archive_2 -- Note that this was actually an enwiki policy for about 5 hours before the RfC was overturned due to too few editors being involved in the straw poll.

For new work, see my current (but stalled for about 1.5 years) push for a structured process on English Wikipedia. https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment  See also the checklist I have been working on with Lane. https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment/Wikipedian_checklist

 

When you review these docs and the corresponding conversations, please keep in mind that I was a new Wikipedian for the development of WP:SRAG and WP:Research, so I made some really critical mistakes -- like taking hyperbolic criticism of the proposals personally. :\ 

 

So what now?  Well, in the meantime, if you let me know about some subject recruitment you want to do, I'll help you find someone to coordinate a review that fits within the process described in the RCom docs.  In the short term, are any of you folks interested in going through some iterations of the new WP:Research_recruitment policy doc?  

 

-Aaron

 

On Thu, Jul 17, 2014 at 2:38 AM, Heather Ford <[hidden email]> wrote:

Agree with Kerry that we really need to have a more flexible process that speaks to the main problem that (I think) RCOM was started to solve i.e. that Wikipedians were getting tired of being continually contacted by researchers to fill out *surveys*. I'm not sure where feelings are about that right now (I certainly haven't seen a huge amount of surveys myself) but I guess the big question right now is whether RCOM is actually active or not. I must say that I was surprised, Aaron, when I read that it is active because I had heard from others in your team about a year or two ago that this wasn't going to be the vehicle for obtaining permission going forward and that a new, more lightweight process was being designed. As Nathan discusses on the Wikimedia-l list, there aren't many indications that RCOM is still active. Perhaps there has been a recent decision to resuscitate it? If that's the case, let us know about it :) And then we can discuss what needs to happen to build a good process. 

 

One immediate requirement that I've been talking to others about is finding ways of making the case to the WMF as a group of researchers for the anonymization of country level data, for example. I've spoken to a few researchers (and I myself made a request about a year ago that hasn't been responded to) and it seems like some work is required by the foundation to do this anonymisation but that there are a few of us who would be really keen to use this data to produce research very valuable to Wikipedia - especially from smaller language versions/developing countries. Having an official process that assesses how worthwhile this investment of time would be to the Foundation would be a great idea, I think, but right now there seems to be a general focus on the research that the Foundation does itself rather than enabling researchers outside. I know how busy Aaron and Dario (and others in the team) are so perhaps this requires a new position to coordinate between researchers and Foundation resources?

 

Anyway, I think the big question right now is whether there are any plans for RCOM that have been made by the research team and the only people who can answer that are folks in the research team :)

 

Best,

Heather.


 

On 17 July 2014 08:49, Kerry Raymond <[hidden email]> wrote:

Yes, I meant the community/communities of WMF. But the authority of the community derives from WMF, which chooses to delegate such matters. I think that “advise” is a good word to use.

 

Kerry

 

 


From: Amir E. Aharoni [mailto:[hidden email]]
Sent: Thursday, 17 July 2014 5:37 PM
To: [hidden email]; Research into Wikimedia content and communities


Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

> WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes.

I don't think that it really should be about WMF. The WMF shouldn't enforce anything. The community can formulate good practices for researchers and _advise_ community members not to cooperate with researchers who don't follow these practices. Not much more is needed.



--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
“We're living in pieces,
I want to live in peace.” – T. Moore

 

2014-07-17 8:24 GMT+03:00 Kerry Raymond <[hidden email]>:

Just saying here what I already put on the Talk page:

 

I am a little bothered by the opening sentence "This page documents the process that researchers must follow before asking Wikipedia contributors to participate in research studies such as surveys, interviews and experiments."

WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes. What WMF does own is its communication channels to me as a contributor and WMF has a right to control what occurs on those channels. Also I think WMF probably should be concerned about both its readers and its contributors being recruited through its channels (as either might be being recruited). I think this distinction should be made, e.g.

"This page documents the process that researchers must follow if they wish to use Wikipedia's (WMF's?) communication channels to recruit people to participate in research studies such as surveys, interviews and experiments. Communication channels include its mailing lists, its Project pages, Talk pages, and User Talk pages [and whatever else I've forgotten]." 

 

If researchers want to recruit WPians via non-WMF means, I don’t think it’s any business of WMF’s. An example might be a researcher who wanted to contact WPians via chapters or thorgs; I would leave it for the chapter/thorg to decide if they wanted to assist the researcher via their communication channels.

 

Of course, the practical reality of it is that some researchers (oblivious of WMF’s concerns in relation to recruitment of WPians to research projects) will simply use WMF’s channels without asking nicely first. Obviously we can remove such requests on-wiki and follow up any email requests with the commentary that this was not an approved request. In my category of [whatever else I’ve forgotten], I guess there are things like Facebook groups and any other social media presence.

 

Also to be practical, if WMF is to have a process to vet research surveys, I think it has to be sufficiently fast and not be overly demanding to avoid the possibility of the researcher giving up (“too hard to deal with these people”) and simply spamming email, project pages, social media in the hope of recruiting some participants regardless. That is, if we make it too slow/hard to do the right thing, we effectively encourage doing the wrong thing. Also, what value-add can we give them to reward those who do the right thing? It’s nice to have a carrot as well as a stick when it comes to onerous processes J

 

Because of the criticism of “not giving back”, could we perhaps do things to try to make the researcher feel part of the community to make “giving back” more likely? For example, could we give them a slot every now and again to talk about their project in the R&D Showcase? Encourage them to be on this mailing list. Are we at a point where it might make sense to organise a Wikipedia research conference to help build a research community? Just thinking aloud here …

 

Kerry

 

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Aaron Halfaker
Sent: Thursday, 17 July 2014 6:59 AM
To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

RCOM review is still alive and looking for new reviewers (really, coordinators).  Researchers can be directed to me or Dario ([hidden email]) to be assigned a reviewer.  There is also a proposed policy on enwiki that could use some eyeballs: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

 

On Wed, Jul 16, 2014 at 11:16 AM, Federico Leva (Nemo) <[hidden email]> wrote:

phoebe ayers, 16/07/2014 19:21:

> (Personally, I think the answer should be to resuscitate RCOM, but
> that's easy to say and harder to do!)

IMHO in the meanwhile the most useful thing folks can do is subscribing
to the feed of new research pages:
<https://meta.wikimedia.org/w/index.php?title=Special:NewPages&feed=atom&hidebots=1&hideredirs=1&limit=500&offset=&namespace=202>
It's easier to build a functioning RCOM out of an active community of
"reviewers", than the other way round.

Nemo

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Federico Leva (Nemo)
In reply to this post by Jonathan Morgan
Jonathan Morgan, 17/07/2014 23:37:
> But because we /look like /an official body, it's easy to blame us for
> failing to prevent disruptive research (if you're a community member),
> for "rubber stamping" research that we like (ditto), or for drowning
> research in red tape (if you're a wiki-researcher).

RCOM doesn't *look like* an official body, it claims to be one. With its
current structure, it looks like a WMF staff committee.
https://wikimediafoundation.org/wiki/Resolution:Wikimedia_Committees#Staff_committees

If you don't want it to look official, it's easy: call it "interest
group", add a "draft" template, add a "under pilot" warning, call it a
subcommittee of the communications committee (a rather common format).

Nemo

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Heather Ford-3
In reply to this post by phoebe ayers-3
On 17 July 2014 17:55, phoebe ayers <[hidden email]> wrote:

Part of the problem is a somewhat subtle demographic one: while contributors to Wikipedia do turn over, so newer contributors will not necessarily have seen lots of surveys, very heavy editors and admins (who are often easier to identify) tend to be long-term participants who might have been surveyed many times. Additionally, the people who follow mailing lists, social media, etc. (or at least the people who speak up on those channels) skew towards very-long-term contributors who have strong opinions and have seen it all before. So, if you advertise your survey on the mailing list, that's the population you get, and that's the feedback you get. (But it's a catch-22; there's not really other obvious mass channels).

This is a really important insight, thanks for sharing it, Phoebe. It's important to work out what the problem is that we're trying to solve before we try solving it! If the key problem here is that Wikipedians need to be protected from researchers constantly surveying them, and actually the wide-ranging surveys are really rare these days, then maybe the problem is with heavy editors and admins being constantly 'surveyed' (although I'm guessing that this is not the only research method being used as I talk about below). 

Does anyone know whether this is actually a problem with editors these days? I know that I have interviewed a bunch of editors over the years without RCOM approval (some with RCOM approval) and I have only had good experiences. Sure there were people who didn't want to be interviewed, but they just ignored my requests - I'm not sure that they would say that they were bothered enough that an entire process needed to be developed to approve projects. 

I think part of the problem here is that there is a bias towards particular types of research projects in the way that RCOM was designed. I do both quantitative and qualitative research on WP and the quantitative research nowadays focuses mostly on capturing large-scale user actions using the API or the dumps - I have a feeling that's why there are fewer surveys these days - more researchers are using the data to conduct research and (right now) that doesn't require any permissions beyond what is required by uni ethics board (and all the problems that come with that!).

The projects I do as a qualitative researcher tend to be exploratory. I will interview people on skype, for example, about their work on particular articles before I know that I have a project. I could certainly develop a proposal to RCOM but it would be so wide-ranging that I'm unsure what the actual benefit was. I think that a much bigger problem is actually developing community guidelines around ethical treatment of subjects who don't often realise that their comments and interactions can be legally (but, I believe not necessarily ethically) used without their permission (I wrote something about my thoughts on this here [1]). 

Basically, I think that we need to reassess what kinds of problems are the most important ones right now that we want to solve rather than resuscitating a process that was designed to address a specific type of problem that was prevalent a long time ago. The new problems that I see right now that a research community is best placed to solve are things like:

- developing community guidelines for the representation of editors' identities in research (similar, perhaps to the AOIR guidelines [2]); 
- finding ways of making responsible requests to the WMF for data that they hold that might benefit research outside the WMF;  
- developing opportunities for researchers to collaborate and share what they're doing with the wider research community (as Kerry suggests). 


Best,
Heather.

 

Anyway, this is a hard problem without super-obvious solutions, and not one that there's a lot of models for -- very few online projects are simultaneously as open with their data and as interesting for research purposes!

best,
Phoebe


--
* I use this address for lists; send personal messages to phoebe.ayers <at> gmail.com *

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Heather Ford-3
In reply to this post by Jonathan Morgan

On 17 July 2014 22:37, Jonathan Morgan <[hidden email]> wrote:

First, I wanted to highlight the important issue that Heather raises here, because although it's a separate issue, it's an important one:

On Thu, Jul 17, 2014 at 2:38 AM, Heather Ford <[hidden email]> wrote:
...


One immediate requirement that I've been talking to others about is finding ways of making the case to the WMF as a group of researchers for the anonymization of country level data, for example. I've spoken to a few researchers (and I myself made a request about a year ago that hasn't been responded to) and it seems like some work is required by the foundation to do this anonymisation but that there are a few of us who would be really keen to use this data to produce research very valuable to Wikipedia - especially from smaller language versions/developing countries. Having an official process that assesses how worthwhile this investment of time would be to the Foundation would be a great idea, I think, but right now there seems to be a general focus on the research that the Foundation does itself rather than enabling researchers outside. I know how busy Aaron and Dario (and others in the team) are so perhaps this requires a new position to coordinate between researchers and Foundation resources?

As a community-run group, RCOM doesn't have any role in making non-public data available to researchers. However, Aaron and I are putting together a proposal for a workshop that would address issues like this. That's work we're doing in an official capacity, as opposed to the RCOM work, which is volunteer.

Jonathan, it looks like this will be a great workshop and I think CSCW is a great venue! but I don't think it addresses the issue unless there's something I'm missing (like an invitation, for example! ;) I see that the workshop is forward-facing but its aim seems to be to work with a bunch of different communities like Reddit and GalaxyZoo. What we need are better channels as Wikipedia researchers to communicate our needs as researchers operating outside the WMF. And preferably in a way that doesn't require us to have to travel to Canada to a workshop to do it!

And, I offered it as a joke but it reminds me of a small, subtle point, I think it would be nice if you could offer an invitation to the researchers on this list to join the workshop and/or workshop planning when you advertise the work you're doing on this. I know it's a wiki and anyone could probably join, but I feel like there is enormous possibility for the group represented here to feel involved and recognised, and I, for one, would like to be invited sometimes.. to the fun stuff, that is, not just the hard, arduous stuff :) 

Best,
Heather.

 

On RCOM more generally... I think clarifying the role of the committee, and getting a larger and more diverse set of people involved, might help make RCOM work. But as Aaron can attest, it is difficult to get people to agree on what RCOMs role should be, let alone get them to work for RCOM.

I've been involved with RCOM for a while, albeit not very actively. Unfortunately, I think that the fact that the only people who "review" requests happen to be* WMF staffers contributes to confusion about RCOM's role and it's authority. IMO, if RCOM or any other subject recruitment review process is to succeed, we need:
  • more wiki-researchers who are willing to regularly participate in both peer review and in developing better process guidelines and standards (it's really just Aaron right now)
  • more Wikipedians who are willing to do the same
  • some degree of buy-in from the Wikimedia community as a whole. RCOM needs legitimacy. But where, and from whom? Subject recruitment is a global concern, but the proposed subject recruitment process is focused on en-wiki (mostly because that's where most of the relevant research activities that we are aware of are happening). How to make RCOM more global?
RCOM is in a tough spot right now. We can't force researchers to submit their proposals, or abide by the suggestions/recommendations/decisions/whatever that result from their review. But because we look like an official body, it's easy to blame us for failing to prevent disruptive research (if you're a community member), for "rubber stamping" research that we like (ditto), or for drowning research in red tape (if you're a wiki-researcher). 


- J

*we were wiki-researchers first!


 


On 17 July 2014 08:49, Kerry Raymond <[hidden email]> wrote:

Yes, I meant the community/communities of WMF. But the authority of the community derives from WMF, which chooses to delegate such matters. I think that “advise” is a good word to use.

 

Kerry

 

 


From: Amir E. Aharoni [mailto:[hidden email]]
Sent: Thursday, 17 July 2014 5:37 PM
To: [hidden email]; Research into Wikimedia content and communities


Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

> WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes.

I don't think that it really should be about WMF. The WMF shouldn't enforce anything. The community can formulate good practices for researchers and _advise_ community members not to cooperate with researchers who don't follow these practices. Not much more is needed.



--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
“We're living in pieces,
I want to live in peace.” – T. Moore

 

2014-07-17 8:24 GMT+03:00 Kerry Raymond <[hidden email]>:

Just saying here what I already put on the Talk page:

 

I am a little bothered by the opening sentence "This page documents the process that researchers must follow before asking Wikipedia contributors to participate in research studies such as surveys, interviews and experiments."

WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes. What WMF does own is its communication channels to me as a contributor and WMF has a right to control what occurs on those channels. Also I think WMF probably should be concerned about both its readers and its contributors being recruited through its channels (as either might be being recruited). I think this distinction should be made, e.g.

"This page documents the process that researchers must follow if they wish to use Wikipedia's (WMF's?) communication channels to recruit people to participate in research studies such as surveys, interviews and experiments. Communication channels include its mailing lists, its Project pages, Talk pages, and User Talk pages [and whatever else I've forgotten]." 

 

If researchers want to recruit WPians via non-WMF means, I don’t think it’s any business of WMF’s. An example might be a researcher who wanted to contact WPians via chapters or thorgs; I would leave it for the chapter/thorg to decide if they wanted to assist the researcher via their communication channels.

 

Of course, the practical reality of it is that some researchers (oblivious of WMF’s concerns in relation to recruitment of WPians to research projects) will simply use WMF’s channels without asking nicely first. Obviously we can remove such requests on-wiki and follow up any email requests with the commentary that this was not an approved request. In my category of [whatever else I’ve forgotten], I guess there are things like Facebook groups and any other social media presence.

 

Also to be practical, if WMF is to have a process to vet research surveys, I think it has to be sufficiently fast and not be overly demanding to avoid the possibility of the researcher giving up (“too hard to deal with these people”) and simply spamming email, project pages, social media in the hope of recruiting some participants regardless. That is, if we make it too slow/hard to do the right thing, we effectively encourage doing the wrong thing. Also, what value-add can we give them to reward those who do the right thing? It’s nice to have a carrot as well as a stick when it comes to onerous processes J

 

Because of the criticism of “not giving back”, could we perhaps do things to try to make the researcher feel part of the community to make “giving back” more likely? For example, could we give them a slot every now and again to talk about their project in the R&D Showcase? Encourage them to be on this mailing list. Are we at a point where it might make sense to organise a Wikipedia research conference to help build a research community? Just thinking aloud here …

 

Kerry

 

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Aaron Halfaker
Sent: Thursday, 17 July 2014 6:59 AM
To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

RCOM review is still alive and looking for new reviewers (really, coordinators).  Researchers can be directed to me or Dario ([hidden email]) to be assigned a reviewer.  There is also a proposed policy on enwiki that could use some eyeballs: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

 

On Wed, Jul 16, 2014 at 11:16 AM, Federico Leva (Nemo) <[hidden email]> wrote:

phoebe ayers, 16/07/2014 19:21:

> (Personally, I think the answer should be to resuscitate RCOM, but
> that's easy to say and harder to do!)

IMHO in the meanwhile the most useful thing folks can do is subscribing
to the feed of new research pages:
<https://meta.wikimedia.org/w/index.php?title=Special:NewPages&feed=atom&hidebots=1&hideredirs=1&limit=500&offset=&namespace=202>
It's easier to build a functioning RCOM out of an active community of
"reviewers", than the other way round.

Nemo

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l




--
Jonathan T. Morgan
Learning Strategist
Wikimedia Foundation


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Aaron Halfaker-2
Does anyone know whether this is actually a problem with editors these days?

Yes.  We regularly see requests to survey the most active Wikipedians about their motivations to edit.  These requests are problematic for some very obvious reasons.  See this proposal[1] for an example of a study that was halted in review due to the disruption it would have caused.

The projects I do as a qualitative researcher tend to be exploratory. I will interview people on skype, for example, about their work on particular articles before I know that I have a project.
  
Do you document your study on wiki and ask for feedback about disruption before moving forward?  Regardless of the process around it, I think we might all agree that is good behavior for any research activity.  This might be obvious to you as someone who has been doing ethnographic work in Wikimedia communities for a long time, but it is apprently less obvious to more junior wiki researchers.   

This good-faith documentation and discussion describes the whole RCom subject recruitment process.  You refer to RCom as "heavy weight", but as far as I can tell, the weight is entirely on the RCom coordinator -- a burden I'll gladly accept to help good research take place without disruption.  Researchers should have already documented their research and prepared themselves to discuss the work with their subjects before they arrive. 

I don't know of a single study that has passed stalled in RCom's process that has resulted in substantial disruption or stalled for more than two weeks.  I welcome you to provide counter examples.  

I don't think [the CSCW workshop proposal] addresses the issue unless there's something I'm missing (like an invitation, for example!

One of the ways that researchers can be supported is through groups that help them socialize their research activities with community members (and minimize disruption for community members).  Despite the tone of this conversation, we have been highly successful in this regard.  

I think it would be nice if you could offer an invitation to the researchers on this list

That's the plan.  We're just getting to a point where we have a solid idea of what we want to accomplish.  An announcement will come soon.  

Basically, I think that we need to reassess what kinds of problems are the most important ones right now that we want to solve rather than resuscitating a process that was designed to address a specific type of problem that was prevalent a long time ago

As I pointed out previously, the subject recruitment process is alive and does not need to be "resuscitated ".  It is also solving a relevant problem.  I welcome Lane Rasberry (if he has time) to share his substantial concerns about undocumented, undiscussed research taking place on-wiki.  


-Aaron 


On Fri, Jul 18, 2014 at 1:27 AM, Heather Ford <[hidden email]> wrote:

On 17 July 2014 22:37, Jonathan Morgan <[hidden email]> wrote:

First, I wanted to highlight the important issue that Heather raises here, because although it's a separate issue, it's an important one:

On Thu, Jul 17, 2014 at 2:38 AM, Heather Ford <[hidden email]> wrote:
...


One immediate requirement that I've been talking to others about is finding ways of making the case to the WMF as a group of researchers for the anonymization of country level data, for example. I've spoken to a few researchers (and I myself made a request about a year ago that hasn't been responded to) and it seems like some work is required by the foundation to do this anonymisation but that there are a few of us who would be really keen to use this data to produce research very valuable to Wikipedia - especially from smaller language versions/developing countries. Having an official process that assesses how worthwhile this investment of time would be to the Foundation would be a great idea, I think, but right now there seems to be a general focus on the research that the Foundation does itself rather than enabling researchers outside. I know how busy Aaron and Dario (and others in the team) are so perhaps this requires a new position to coordinate between researchers and Foundation resources?

As a community-run group, RCOM doesn't have any role in making non-public data available to researchers. However, Aaron and I are putting together a proposal for a workshop that would address issues like this. That's work we're doing in an official capacity, as opposed to the RCOM work, which is volunteer.

Jonathan, it looks like this will be a great workshop and I think CSCW is a great venue! but I don't think it addresses the issue unless there's something I'm missing (like an invitation, for example! ;) I see that the workshop is forward-facing but its aim seems to be to work with a bunch of different communities like Reddit and GalaxyZoo. What we need are better channels as Wikipedia researchers to communicate our needs as researchers operating outside the WMF. And preferably in a way that doesn't require us to have to travel to Canada to a workshop to do it!

And, I offered it as a joke but it reminds me of a small, subtle point, I think it would be nice if you could offer an invitation to the researchers on this list to join the workshop and/or workshop planning when you advertise the work you're doing on this. I know it's a wiki and anyone could probably join, but I feel like there is enormous possibility for the group represented here to feel involved and recognised, and I, for one, would like to be invited sometimes.. to the fun stuff, that is, not just the hard, arduous stuff :) 

Best,
Heather.

 

On RCOM more generally... I think clarifying the role of the committee, and getting a larger and more diverse set of people involved, might help make RCOM work. But as Aaron can attest, it is difficult to get people to agree on what RCOMs role should be, let alone get them to work for RCOM.

I've been involved with RCOM for a while, albeit not very actively. Unfortunately, I think that the fact that the only people who "review" requests happen to be* WMF staffers contributes to confusion about RCOM's role and it's authority. IMO, if RCOM or any other subject recruitment review process is to succeed, we need:
  • more wiki-researchers who are willing to regularly participate in both peer review and in developing better process guidelines and standards (it's really just Aaron right now)
  • more Wikipedians who are willing to do the same
  • some degree of buy-in from the Wikimedia community as a whole. RCOM needs legitimacy. But where, and from whom? Subject recruitment is a global concern, but the proposed subject recruitment process is focused on en-wiki (mostly because that's where most of the relevant research activities that we are aware of are happening). How to make RCOM more global?
RCOM is in a tough spot right now. We can't force researchers to submit their proposals, or abide by the suggestions/recommendations/decisions/whatever that result from their review. But because we look like an official body, it's easy to blame us for failing to prevent disruptive research (if you're a community member), for "rubber stamping" research that we like (ditto), or for drowning research in red tape (if you're a wiki-researcher). 


- J

*we were wiki-researchers first!


 


On 17 July 2014 08:49, Kerry Raymond <[hidden email]> wrote:

Yes, I meant the community/communities of WMF. But the authority of the community derives from WMF, which chooses to delegate such matters. I think that “advise” is a good word to use.

 

Kerry

 

 


From: Amir E. Aharoni [mailto:[hidden email]]
Sent: Thursday, 17 July 2014 5:37 PM
To: [hidden email]; Research into Wikimedia content and communities


Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

> WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes.

I don't think that it really should be about WMF. The WMF shouldn't enforce anything. The community can formulate good practices for researchers and _advise_ community members not to cooperate with researchers who don't follow these practices. Not much more is needed.



--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
“We're living in pieces,
I want to live in peace.” – T. Moore

 

2014-07-17 8:24 GMT+03:00 Kerry Raymond <[hidden email]>:

Just saying here what I already put on the Talk page:

 

I am a little bothered by the opening sentence "This page documents the process that researchers must follow before asking Wikipedia contributors to participate in research studies such as surveys, interviews and experiments."

WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes. What WMF does own is its communication channels to me as a contributor and WMF has a right to control what occurs on those channels. Also I think WMF probably should be concerned about both its readers and its contributors being recruited through its channels (as either might be being recruited). I think this distinction should be made, e.g.

"This page documents the process that researchers must follow if they wish to use Wikipedia's (WMF's?) communication channels to recruit people to participate in research studies such as surveys, interviews and experiments. Communication channels include its mailing lists, its Project pages, Talk pages, and User Talk pages [and whatever else I've forgotten]." 

 

If researchers want to recruit WPians via non-WMF means, I don’t think it’s any business of WMF’s. An example might be a researcher who wanted to contact WPians via chapters or thorgs; I would leave it for the chapter/thorg to decide if they wanted to assist the researcher via their communication channels.

 

Of course, the practical reality of it is that some researchers (oblivious of WMF’s concerns in relation to recruitment of WPians to research projects) will simply use WMF’s channels without asking nicely first. Obviously we can remove such requests on-wiki and follow up any email requests with the commentary that this was not an approved request. In my category of [whatever else I’ve forgotten], I guess there are things like Facebook groups and any other social media presence.

 

Also to be practical, if WMF is to have a process to vet research surveys, I think it has to be sufficiently fast and not be overly demanding to avoid the possibility of the researcher giving up (“too hard to deal with these people”) and simply spamming email, project pages, social media in the hope of recruiting some participants regardless. That is, if we make it too slow/hard to do the right thing, we effectively encourage doing the wrong thing. Also, what value-add can we give them to reward those who do the right thing? It’s nice to have a carrot as well as a stick when it comes to onerous processes J

 

Because of the criticism of “not giving back”, could we perhaps do things to try to make the researcher feel part of the community to make “giving back” more likely? For example, could we give them a slot every now and again to talk about their project in the R&D Showcase? Encourage them to be on this mailing list. Are we at a point where it might make sense to organise a Wikipedia research conference to help build a research community? Just thinking aloud here …

 

Kerry

 

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Aaron Halfaker
Sent: Thursday, 17 July 2014 6:59 AM
To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

RCOM review is still alive and looking for new reviewers (really, coordinators).  Researchers can be directed to me or Dario ([hidden email]) to be assigned a reviewer.  There is also a proposed policy on enwiki that could use some eyeballs: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

 

On Wed, Jul 16, 2014 at 11:16 AM, Federico Leva (Nemo) <[hidden email]> wrote:

phoebe ayers, 16/07/2014 19:21:

> (Personally, I think the answer should be to resuscitate RCOM, but
> that's easy to say and harder to do!)

IMHO in the meanwhile the most useful thing folks can do is subscribing
to the feed of new research pages:
<https://meta.wikimedia.org/w/index.php?title=Special:NewPages&feed=atom&hidebots=1&hideredirs=1&limit=500&offset=&namespace=202>
It's easier to build a functioning RCOM out of an active community of
"reviewers", than the other way round.

Nemo

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l




--
Jonathan T. Morgan
Learning Strategist
Wikimedia Foundation


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Jonathan Morgan
In reply to this post by Heather Ford-3



On Fri, Jul 18, 2014 at 1:27 AM, Heather Ford <[hidden email]> wrote:

Jonathan, it looks like this will be a great workshop and I think CSCW is a great venue! but I don't think it addresses the issue unless there's something I'm missing (like an invitation, for example! ;) I see that the workshop is forward-facing but its aim seems to be to work with a bunch of different communities like Reddit and GalaxyZoo. What we need are better channels as Wikipedia researchers to communicate our needs as researchers operating outside the WMF. And preferably in a way that doesn't require us to have to travel to Canada to a workshop to do it!

And, I offered it as a joke but it reminds me of a small, subtle point, I think it would be nice if you could offer an invitation to the researchers on this list to join the workshop and/or workshop planning when you advertise the work you're doing on this. I know it's a wiki and anyone could probably join, but I feel like there is enormous possibility for the group represented here to feel involved and recognised, and I, for one, would like to be invited sometimes.. to the fun stuff, that is, not just the hard, arduous stuff :) 


For me, it's just a matter of bandwidth. I get a lot of personal requests for advice, pointers, data, etc from wiki-researchers. I try to answer them, but I can't personally work with every researcher who wants special access beyond public dumps, APIs, databases... or who wants an "in" on subject recruitment. I don't scale all that well :)

Plus, that's an opaque and ad-hoc process, and it doesn't contribute to the formation of public standards and guidelines that benefit other researchers. Hence, the workshop. We will try to get a bunch of researchers together in one place, figure out what their needs are, and get them involved in developing a better process for quenching their data thirst!

Not sure I get your invite question, tho? Are you asking to be a co-author on our workshop proposal? ;) 

Best,
Heather.

 

On RCOM more generally... I think clarifying the role of the committee, and getting a larger and more diverse set of people involved, might help make RCOM work. But as Aaron can attest, it is difficult to get people to agree on what RCOMs role should be, let alone get them to work for RCOM.

I've been involved with RCOM for a while, albeit not very actively. Unfortunately, I think that the fact that the only people who "review" requests happen to be* WMF staffers contributes to confusion about RCOM's role and it's authority. IMO, if RCOM or any other subject recruitment review process is to succeed, we need:
  • more wiki-researchers who are willing to regularly participate in both peer review and in developing better process guidelines and standards (it's really just Aaron right now)
  • more Wikipedians who are willing to do the same
  • some degree of buy-in from the Wikimedia community as a whole. RCOM needs legitimacy. But where, and from whom? Subject recruitment is a global concern, but the proposed subject recruitment process is focused on en-wiki (mostly because that's where most of the relevant research activities that we are aware of are happening). How to make RCOM more global?
RCOM is in a tough spot right now. We can't force researchers to submit their proposals, or abide by the suggestions/recommendations/decisions/whatever that result from their review. But because we look like an official body, it's easy to blame us for failing to prevent disruptive research (if you're a community member), for "rubber stamping" research that we like (ditto), or for drowning research in red tape (if you're a wiki-researcher). 


- J

*we were wiki-researchers first!


 


On 17 July 2014 08:49, Kerry Raymond <[hidden email]> wrote:

Yes, I meant the community/communities of WMF. But the authority of the community derives from WMF, which chooses to delegate such matters. I think that “advise” is a good word to use.

 

Kerry

 

 


From: Amir E. Aharoni [mailto:[hidden email]]
Sent: Thursday, 17 July 2014 5:37 PM
To: [hidden email]; Research into Wikimedia content and communities


Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

> WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes.

I don't think that it really should be about WMF. The WMF shouldn't enforce anything. The community can formulate good practices for researchers and _advise_ community members not to cooperate with researchers who don't follow these practices. Not much more is needed.



--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
“We're living in pieces,
I want to live in peace.” – T. Moore

 

2014-07-17 8:24 GMT+03:00 Kerry Raymond <[hidden email]>:

Just saying here what I already put on the Talk page:

 

I am a little bothered by the opening sentence "This page documents the process that researchers must follow before asking Wikipedia contributors to participate in research studies such as surveys, interviews and experiments."

WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes. What WMF does own is its communication channels to me as a contributor and WMF has a right to control what occurs on those channels. Also I think WMF probably should be concerned about both its readers and its contributors being recruited through its channels (as either might be being recruited). I think this distinction should be made, e.g.

"This page documents the process that researchers must follow if they wish to use Wikipedia's (WMF's?) communication channels to recruit people to participate in research studies such as surveys, interviews and experiments. Communication channels include its mailing lists, its Project pages, Talk pages, and User Talk pages [and whatever else I've forgotten]." 

 

If researchers want to recruit WPians via non-WMF means, I don’t think it’s any business of WMF’s. An example might be a researcher who wanted to contact WPians via chapters or thorgs; I would leave it for the chapter/thorg to decide if they wanted to assist the researcher via their communication channels.

 

Of course, the practical reality of it is that some researchers (oblivious of WMF’s concerns in relation to recruitment of WPians to research projects) will simply use WMF’s channels without asking nicely first. Obviously we can remove such requests on-wiki and follow up any email requests with the commentary that this was not an approved request. In my category of [whatever else I’ve forgotten], I guess there are things like Facebook groups and any other social media presence.

 

Also to be practical, if WMF is to have a process to vet research surveys, I think it has to be sufficiently fast and not be overly demanding to avoid the possibility of the researcher giving up (“too hard to deal with these people”) and simply spamming email, project pages, social media in the hope of recruiting some participants regardless. That is, if we make it too slow/hard to do the right thing, we effectively encourage doing the wrong thing. Also, what value-add can we give them to reward those who do the right thing? It’s nice to have a carrot as well as a stick when it comes to onerous processes J

 

Because of the criticism of “not giving back”, could we perhaps do things to try to make the researcher feel part of the community to make “giving back” more likely? For example, could we give them a slot every now and again to talk about their project in the R&D Showcase? Encourage them to be on this mailing list. Are we at a point where it might make sense to organise a Wikipedia research conference to help build a research community? Just thinking aloud here …

 

Kerry

 

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Aaron Halfaker
Sent: Thursday, 17 July 2014 6:59 AM
To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

RCOM review is still alive and looking for new reviewers (really, coordinators).  Researchers can be directed to me or Dario ([hidden email]) to be assigned a reviewer.  There is also a proposed policy on enwiki that could use some eyeballs: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

 

On Wed, Jul 16, 2014 at 11:16 AM, Federico Leva (Nemo) <[hidden email]> wrote:

phoebe ayers, 16/07/2014 19:21:

> (Personally, I think the answer should be to resuscitate RCOM, but
> that's easy to say and harder to do!)

IMHO in the meanwhile the most useful thing folks can do is subscribing
to the feed of new research pages:
<https://meta.wikimedia.org/w/index.php?title=Special:NewPages&feed=atom&hidebots=1&hideredirs=1&limit=500&offset=&namespace=202>
It's easier to build a functioning RCOM out of an active community of
"reviewers", than the other way round.

Nemo

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l




--
Jonathan T. Morgan
Learning Strategist
Wikimedia Foundation


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l




--
Jonathan T. Morgan
Learning Strategist
Wikimedia Foundation


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Jonathan Morgan
In reply to this post by Federico Leva (Nemo)
On Thu, Jul 17, 2014 at 11:36 PM, Federico Leva (Nemo) <[hidden email]> wrote:
Jonathan Morgan, 17/07/2014 23:37:
> But because we /look like /an official body, it's easy to blame us for
> failing to prevent disruptive research (if you're a community member),
> for "rubber stamping" research that we like (ditto), or for drowning
> research in red tape (if you're a wiki-researcher).

RCOM doesn't *look like* an official body, it claims to be one. With its
current structure, it looks like a WMF staff committee.
https://wikimediafoundation.org/wiki/Resolution:Wikimedia_Committees#Staff_committees

If you don't want it to look official, it's easy: call it "interest
group", add a "draft" template, add a "under pilot" warning, call it a
subcommittee of the communications committee (a rather common format).


Heh. Well, I wasn't aware it was I was participating in an official body. Does that mean I get to review proposals under my staff account now? :P

I suppose if I'm this confused about RCOM's role, I shouldn't be surprised that others are too.

- J

--
Jonathan T. Morgan
Learning Strategist
Wikimedia Foundation


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: discussion about wikipedia surveys

Jonathan Morgan
In reply to this post by Pine W

On Thu, Jul 17, 2014 at 7:50 PM, Pine W <[hidden email]> wrote:
If RCOM needs more volunteer Wikimedians, the alive and well IEG Committee includes a Research Working Group that reviews grant proposals for WMF funding through the IEG program, so RCOM could reach out to IEGCom. I'm on IEGCom and the RWG but I can't speak for RCOM. (:

Thanks, Pine. I'll likely hold you to that offer ;)
 

On Thu, Jul 17, 2014 at 3:10 PM, Kerry Raymond <[hidden email]> wrote:

I guess I was not so much thinking of an general invitation to the R&D Showcase but a specific “expectation” (albeit couched as an invitation) on those given permission to recruit via WMF channels to give a few short (or long as appropriate to the stage of their research) talks on their project. Ditto research projects supported through IEG or similar.

 

I agree that OpenSym is available as a research conference but it is not run by our community and therefore doesn’t help to create a sense of community with the researchers in question. Wikimania is run by our community but isn’t a research conference (would not count as a publication for academic purposes). But I don’t know if it’s realistic to try to establish another conference in terms of the volunteer effort to run it.

 

Kerry

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Aaron Halfaker
Sent: Friday, 18 July 2014 1:45 AM


To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

 

 

Kerry said:

Because of the criticism of “not giving back”, could we perhaps do things to try to make the researcher feel part of the community to make “giving back” more likely? For example, could we give them a slot every now and again to talk about their project in the R&D Showcase? Encourage them to be on this mailing list. Are we at a point where it might make sense to organise a Wikipedia research conference to help build a research community? Just thinking aloud here …

 

This is a bit different than the main topic, so I wanted to break it out into another reply.   

 

We just had Nate Matias[0] from the MIT media lab present on his work at the last showcase[1].  We also just sent out a survey about the showcase that includes a call for recommended speakers at future showcases[2].  As for a Wikipedia research conference, see OpenSym[3] (formerly WikiSym) and Wikimania[4] (not as researchy, but a great venue to maximize wiki research impact). 

 

 

On Thu, Jul 17, 2014 at 8:30 AM, Aaron Halfaker <[hidden email]> wrote:

Aaron, when I read that it is active because I had heard from others in your team about a year or two ago that this wasn't going to be the vehicle for obtaining permission going forward and that a new, more lightweight process was being designed.

 

1) If anyone told you that we are no longer active, they were wrong.

2) The "lightweight" process you refer to is what I linked to in enwiki in my previous response.  See again: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

 

Generally, there seems to be a misconception that RCom == paid WMF activities.  While RCom involves a relationship with the Wikimedia Foundation, our activities as part of RCom are 100% volunteer and open to participation from other Wikipedians (seriously, let me know if you want to help out!), and as such, our backlog tends to suffer when our available volunteer time does.  FWIW, I became involved in this work as a volunteer (before I started working with the WMF).  With that in mind, it seems like we are not discussing RCom itself which is mostly inactive -- so much as we are discussing the subject recruitment review process which is still active.  Let me state this clearly: If you send an email to me or Dario about a research project that you would like reviewed, we will help you coordinate a review.  Our job as review coordinators is to make sure that the study is adequately documented and that Wikipedians and other researchers are pulled in to discuss the material.  We don't just welcome broad involvement -- we need it!  We all suffer from the lack of it.  Please show up help us!

 

To give you some context on the current stats and situation, I should probably give a bit of history.  I've been working to improve subject recruitment review -- with the goal of improving interactions between researchers and Wikipedians -- for years.  Let me first say that I'm game to make this better.  In my experience, the biggest issue to documenting the a review/endorsement/whatever process that I have come across is this: there seems to be a lot of people who feel that minimizing process description provides power and adaptability to intended processes[1].  It's these people that I've regularly battled in my frequent efforts to increase the formalization around the subject recruitment proposal vetting process (e.g. SRAG had a structured appeals process and stated timelines).  The result of these battles is the severely under-documented process "described" in meta:R:FAQ.

 

Here's some links to my previous work on subject recruitment process that will show these old discussions about process creep

·  https://en.wikipedia.org/wiki/Wikipedia:Subject_Recruitment_Approvals_Group

o https://en.wikipedia.org/wiki/Wikipedia_talk:Subject_Recruitment_Approvals_Group

·  https://en.wikipedia.org/w/index.php?title=Wikipedia:Research&oldid=354600173

o https://en.wikipedia.org/wiki/Wikipedia_talk:Research/Archive_1

o https://en.wikipedia.org/wiki/Wikipedia_talk:Research/Archive_2 -- Note that this was actually an enwiki policy for about 5 hours before the RfC was overturned due to too few editors being involved in the straw poll.

For new work, see my current (but stalled for about 1.5 years) push for a structured process on English Wikipedia. https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment  See also the checklist I have been working on with Lane. https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment/Wikipedian_checklist

 

When you review these docs and the corresponding conversations, please keep in mind that I was a new Wikipedian for the development of WP:SRAG and WP:Research, so I made some really critical mistakes -- like taking hyperbolic criticism of the proposals personally. :\ 

 

So what now?  Well, in the meantime, if you let me know about some subject recruitment you want to do, I'll help you find someone to coordinate a review that fits within the process described in the RCom docs.  In the short term, are any of you folks interested in going through some iterations of the new WP:Research_recruitment policy doc?  

 

-Aaron

 

On Thu, Jul 17, 2014 at 2:38 AM, Heather Ford <[hidden email]> wrote:

Agree with Kerry that we really need to have a more flexible process that speaks to the main problem that (I think) RCOM was started to solve i.e. that Wikipedians were getting tired of being continually contacted by researchers to fill out *surveys*. I'm not sure where feelings are about that right now (I certainly haven't seen a huge amount of surveys myself) but I guess the big question right now is whether RCOM is actually active or not. I must say that I was surprised, Aaron, when I read that it is active because I had heard from others in your team about a year or two ago that this wasn't going to be the vehicle for obtaining permission going forward and that a new, more lightweight process was being designed. As Nathan discusses on the Wikimedia-l list, there aren't many indications that RCOM is still active. Perhaps there has been a recent decision to resuscitate it? If that's the case, let us know about it :) And then we can discuss what needs to happen to build a good process. 

 

One immediate requirement that I've been talking to others about is finding ways of making the case to the WMF as a group of researchers for the anonymization of country level data, for example. I've spoken to a few researchers (and I myself made a request about a year ago that hasn't been responded to) and it seems like some work is required by the foundation to do this anonymisation but that there are a few of us who would be really keen to use this data to produce research very valuable to Wikipedia - especially from smaller language versions/developing countries. Having an official process that assesses how worthwhile this investment of time would be to the Foundation would be a great idea, I think, but right now there seems to be a general focus on the research that the Foundation does itself rather than enabling researchers outside. I know how busy Aaron and Dario (and others in the team) are so perhaps this requires a new position to coordinate between researchers and Foundation resources?

 

Anyway, I think the big question right now is whether there are any plans for RCOM that have been made by the research team and the only people who can answer that are folks in the research team :)

 

Best,

Heather.


 

On 17 July 2014 08:49, Kerry Raymond <[hidden email]> wrote:

Yes, I meant the community/communities of WMF. But the authority of the community derives from WMF, which chooses to delegate such matters. I think that “advise” is a good word to use.

 

Kerry

 

 


From: Amir E. Aharoni [mailto:[hidden email]]
Sent: Thursday, 17 July 2014 5:37 PM
To: [hidden email]; Research into Wikimedia content and communities


Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

> WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes.

I don't think that it really should be about WMF. The WMF shouldn't enforce anything. The community can formulate good practices for researchers and _advise_ community members not to cooperate with researchers who don't follow these practices. Not much more is needed.



--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
“We're living in pieces,
I want to live in peace.” – T. Moore

 

2014-07-17 8:24 GMT+03:00 Kerry Raymond <[hidden email]>:

Just saying here what I already put on the Talk page:

 

I am a little bothered by the opening sentence "This page documents the process that researchers must follow before asking Wikipedia contributors to participate in research studies such as surveys, interviews and experiments."

WMF does not "own" me as a contributor; it does not decide who can and cannot recruit me for whatever purposes. What WMF does own is its communication channels to me as a contributor and WMF has a right to control what occurs on those channels. Also I think WMF probably should be concerned about both its readers and its contributors being recruited through its channels (as either might be being recruited). I think this distinction should be made, e.g.

"This page documents the process that researchers must follow if they wish to use Wikipedia's (WMF's?) communication channels to recruit people to participate in research studies such as surveys, interviews and experiments. Communication channels include its mailing lists, its Project pages, Talk pages, and User Talk pages [and whatever else I've forgotten]." 

 

If researchers want to recruit WPians via non-WMF means, I don’t think it’s any business of WMF’s. An example might be a researcher who wanted to contact WPians via chapters or thorgs; I would leave it for the chapter/thorg to decide if they wanted to assist the researcher via their communication channels.

 

Of course, the practical reality of it is that some researchers (oblivious of WMF’s concerns in relation to recruitment of WPians to research projects) will simply use WMF’s channels without asking nicely first. Obviously we can remove such requests on-wiki and follow up any email requests with the commentary that this was not an approved request. In my category of [whatever else I’ve forgotten], I guess there are things like Facebook groups and any other social media presence.

 

Also to be practical, if WMF is to have a process to vet research surveys, I think it has to be sufficiently fast and not be overly demanding to avoid the possibility of the researcher giving up (“too hard to deal with these people”) and simply spamming email, project pages, social media in the hope of recruiting some participants regardless. That is, if we make it too slow/hard to do the right thing, we effectively encourage doing the wrong thing. Also, what value-add can we give them to reward those who do the right thing? It’s nice to have a carrot as well as a stick when it comes to onerous processes J

 

Because of the criticism of “not giving back”, could we perhaps do things to try to make the researcher feel part of the community to make “giving back” more likely? For example, could we give them a slot every now and again to talk about their project in the R&D Showcase? Encourage them to be on this mailing list. Are we at a point where it might make sense to organise a Wikipedia research conference to help build a research community? Just thinking aloud here …

 

Kerry

 

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Aaron Halfaker
Sent: Thursday, 17 July 2014 6:59 AM
To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] discussion about wikipedia surveys

 

RCOM review is still alive and looking for new reviewers (really, coordinators).  Researchers can be directed to me or Dario ([hidden email]) to be assigned a reviewer.  There is also a proposed policy on enwiki that could use some eyeballs: https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment

 

On Wed, Jul 16, 2014 at 11:16 AM, Federico Leva (Nemo) <[hidden email]> wrote:

phoebe ayers, 16/07/2014 19:21:

> (Personally, I think the answer should be to resuscitate RCOM, but
> that's easy to say and harder to do!)

IMHO in the meanwhile the most useful thing folks can do is subscribing
to the feed of new research pages:
<https://meta.wikimedia.org/w/index.php?title=Special:NewPages&feed=atom&hidebots=1&hideredirs=1&limit=500&offset=&namespace=202>
It's easier to build a functioning RCOM out of an active community of
"reviewers", than the other way round.

Nemo

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l




--
Jonathan T. Morgan
Learning Strategist
Wikimedia Foundation


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
123