Editor participation rates in surveys

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Editor participation rates in surveys

Juliana Bastos
Hi all! I am looking for any discussions/data about participation rates in
research surveys directed towards editors. I'd like to see if there's a
consistent rate, or not, in responses per target group. Can anybody help me
with this?

Thank you,
Juliana



--
www.domusaurea.org
_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Editor participation rates in surveys

Jonathan Morgan
Hi Juliana,

Can you give a little more info about what you're looking for, and a little
context about why your asking?

I don't know of any research that has specifically asked whether there is a
difference in response rate per target group. Anecdotally (I've run a lot
of editor surveys), I can say that in my experience:

   - very new editors often don't respond to surveys at a high rate,
   probably because they're less committed to/invested in Wikipedia and/or
   they have already lost interest (or stopped participating for other
   reasons) by the time they get the survey
   - how you deliver the survey matters a lot: for example, direct email
   vs. talkpage message vs. newsletter/mailing list message vs. invitation at
   a live f2f event
   - the topic and goal of your survey matters a lot: if it's something
   that people care about, they're more likely to respond. If people feel that
   it's important or personally useful to tell you what they know or what they
   think, they're more likely to respond. If you're asking for very personal
   information, or information that is not clearly relevant to your stated
   goals, they're often less likely to respond.
   - who you are and why you're asking matters a lot: do the editors trust
   you? do they have preconceived notions (correct or not) about who you are,
   what the data will be used for, how it will be stored and published, how
   privacy and anonymity will be ensured (if applicable)... these all matter a
   whole lot.
   - in general, smaller-scale surveys targeted at a very specific group
   and which are clearly relevant to the expertise and goals of that group,
   and follow scientific best practices for open and ethical research, seem to
   work pretty well (with all the above caveats)

Hope that helps,
Jonathan

On Tue, Oct 31, 2017 at 1:36 PM, Juliana Bastos Marques <
[hidden email]> wrote:

> Hi all! I am looking for any discussions/data about participation rates in
> research surveys directed towards editors. I'd like to see if there's a
> consistent rate, or not, in responses per target group. Can anybody help me
> with this?
>
> Thank you,
> Juliana
>
>
>
> --
> www.domusaurea.org
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>



--
Jonathan T. Morgan
Senior Design Researcher
Wikimedia Foundation
User:Jmorgan (WMF) <https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF)>
_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Editor participation rates in surveys

Juliana Bastos
Thanks for your reply, Jonathan. I was wondering if anybody has ever
conducted a systematic research on the variables that you listed.

I had a sample of the 200 editors with more edits (30 days in Sept/Oct) on
Portuguese Wikipedia, and 11 of these participated. I could have adopted
other criteria - for instance, only 34 of these are admins, 2 are bots -,
but for my purposes I just wanted a small sample with an objective
selection. Indeed, the participation rate of 5,5% was expected, but I was
wondering if there are any studies that can corroborate this.

Thanks,
Juliana

On Tue, Oct 31, 2017 at 5:27 PM, Jonathan Morgan <[hidden email]>
wrote:

> Hi Juliana,
>
> Can you give a little more info about what you're looking for, and a little
> context about why your asking?
>
> I don't know of any research that has specifically asked whether there is a
> difference in response rate per target group. Anecdotally (I've run a lot
> of editor surveys), I can say that in my experience:
>
>    - very new editors often don't respond to surveys at a high rate,
>    probably because they're less committed to/invested in Wikipedia and/or
>    they have already lost interest (or stopped participating for other
>    reasons) by the time they get the survey
>    - how you deliver the survey matters a lot: for example, direct email
>    vs. talkpage message vs. newsletter/mailing list message vs. invitation
> at
>    a live f2f event
>    - the topic and goal of your survey matters a lot: if it's something
>    that people care about, they're more likely to respond. If people feel
> that
>    it's important or personally useful to tell you what they know or what
> they
>    think, they're more likely to respond. If you're asking for very
> personal
>    information, or information that is not clearly relevant to your stated
>    goals, they're often less likely to respond.
>    - who you are and why you're asking matters a lot: do the editors trust
>    you? do they have preconceived notions (correct or not) about who you
> are,
>    what the data will be used for, how it will be stored and published, how
>    privacy and anonymity will be ensured (if applicable)... these all
> matter a
>    whole lot.
>    - in general, smaller-scale surveys targeted at a very specific group
>    and which are clearly relevant to the expertise and goals of that group,
>    and follow scientific best practices for open and ethical research,
> seem to
>    work pretty well (with all the above caveats)
>
> Hope that helps,
> Jonathan
>
> On Tue, Oct 31, 2017 at 1:36 PM, Juliana Bastos Marques <
> [hidden email]> wrote:
>
> > Hi all! I am looking for any discussions/data about participation rates
> in
> > research surveys directed towards editors. I'd like to see if there's a
> > consistent rate, or not, in responses per target group. Can anybody help
> me
> > with this?
> >
> > Thank you,
> > Juliana
> >
> >
> >
> > --
> > www.domusaurea.org
> > _______________________________________________
> > Wiki-research-l mailing list
> > [hidden email]
> > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
> >
>
>
>
> --
> Jonathan T. Morgan
> Senior Design Researcher
> Wikimedia Foundation
> User:Jmorgan (WMF) <https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF)>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>



--
www.domusaurea.org
_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Editor participation rates in surveys

Kerry Raymond
Frankly that sounds like quite a high participation rate to me. I'd be surprised if a random call on English Wikipedians would have produced that level of response.  But, as Jonathon has already said, participation rate is going to depend on a lot of variables.

I don't think anyone has conducted the "meta-research" on the willingness of Wikipedians to be surveyed and in what circumstances. Indeed, I doubt it can be studied. Even if you had access to every study ever done on Wikipedians, it is unlikely that each study occurs independent of any other. Almost certainly, some of the studies started with enquiries to other researchers about how best to solicit participation, so I would suspect that many studies adopted methods of recruitment influenced by the experience of previous studies. This means that surveys may be recruiting based on greatest likelihood of finding respondents rather than whether those respondents are a representative sample within the desired cohort of subjects. And, let's face it, a researcher who has to produce a result to get a PhD, tenure, promotion, or their next grant, a large number of non-representatives respondents at least gives you enough data to draw some conclusions about, whereas a very small number of representative respondents might not. "We surveyed 2 people and they said ..." (ouch!).  Let's never forget that we don't do research just to make the world a better place.

Indeed, when it comes to Wikipedians, I don't think we know what a representative sample should look like in any case. Even the WMF's own editor surveys had relatively low participation rates (5000 for the 2011 editor survey) which is a drop in the bucket of the millions of registered user accounts (and the almost unknowable number of editors contributing anonymously. In contrast when you do a random population study or within staff of an organisation, you do generally have other data (e.g. census, HR records) to tell you how representative your sample is on a number of the standard demographic variables. No wonder we like to study university students so much (known demographics and such a convenient sample!).

Kerry

-----Original Message-----
From: Wiki-research-l [mailto:[hidden email]] On Behalf Of Juliana Bastos Marques
Sent: Wednesday, 1 November 2017 7:38 AM
To: Research into Wikimedia content and communities <[hidden email]>
Subject: Re: [Wiki-research-l] Editor participation rates in surveys

Thanks for your reply, Jonathan. I was wondering if anybody has ever conducted a systematic research on the variables that you listed.

I had a sample of the 200 editors with more edits (30 days in Sept/Oct) on Portuguese Wikipedia, and 11 of these participated. I could have adopted other criteria - for instance, only 34 of these are admins, 2 are bots -, but for my purposes I just wanted a small sample with an objective selection. Indeed, the participation rate of 5,5% was expected, but I was wondering if there are any studies that can corroborate this.

Thanks,
Juliana

On Tue, Oct 31, 2017 at 5:27 PM, Jonathan Morgan <[hidden email]>
wrote:

> Hi Juliana,
>
> Can you give a little more info about what you're looking for, and a
> little context about why your asking?
>
> I don't know of any research that has specifically asked whether there
> is a difference in response rate per target group. Anecdotally (I've
> run a lot of editor surveys), I can say that in my experience:
>
>    - very new editors often don't respond to surveys at a high rate,
>    probably because they're less committed to/invested in Wikipedia and/or
>    they have already lost interest (or stopped participating for other
>    reasons) by the time they get the survey
>    - how you deliver the survey matters a lot: for example, direct email
>    vs. talkpage message vs. newsletter/mailing list message vs.
> invitation at
>    a live f2f event
>    - the topic and goal of your survey matters a lot: if it's something
>    that people care about, they're more likely to respond. If people
> feel that
>    it's important or personally useful to tell you what they know or
> what they
>    think, they're more likely to respond. If you're asking for very
> personal
>    information, or information that is not clearly relevant to your stated
>    goals, they're often less likely to respond.
>    - who you are and why you're asking matters a lot: do the editors trust
>    you? do they have preconceived notions (correct or not) about who
> you are,
>    what the data will be used for, how it will be stored and published, how
>    privacy and anonymity will be ensured (if applicable)... these all
> matter a
>    whole lot.
>    - in general, smaller-scale surveys targeted at a very specific group
>    and which are clearly relevant to the expertise and goals of that group,
>    and follow scientific best practices for open and ethical research,
> seem to
>    work pretty well (with all the above caveats)
>
> Hope that helps,
> Jonathan
>
> On Tue, Oct 31, 2017 at 1:36 PM, Juliana Bastos Marques <
> [hidden email]> wrote:
>
> > Hi all! I am looking for any discussions/data about participation
> > rates
> in
> > research surveys directed towards editors. I'd like to see if
> > there's a consistent rate, or not, in responses per target group.
> > Can anybody help
> me
> > with this?
> >
> > Thank you,
> > Juliana
> >
> >
> >
> > --
> > www.domusaurea.org
> > _______________________________________________
> > Wiki-research-l mailing list
> > [hidden email]
> > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
> >
>
>
>
> --
> Jonathan T. Morgan
> Senior Design Researcher
> Wikimedia Foundation
> User:Jmorgan (WMF)
> <https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF)>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>



--
www.domusaurea.org
_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

FW: Re: Editor participation rates in surveys

Kerry Raymond
I just received this response to my message to this wiki-research mailing list. Do we think it acceptable that people sign up to this list with this kind of demand for payment? I don’t.

 

Sure I can just delete the email, but I don’t believe this is acceptable behaviour when this person has presumably willingly signed up to this list and should therefore be willing to receive legitimate messages sent to it by other list members.

 

Kerry

 

From: REMOVED BY KERRY
Sent: Wednesday, 1 November 2017 11:15 AM
To: [hidden email]
Subject: Re: Re: [Wiki-research-l] Editor participation rates in surveys

 



               
               
               
               

 

               
               

Hello!

I use a new email filtering service called BitBounce to better filter my spam. To deliver your email to my inbox, please click the link below and pay the small Bitcoin fee. Thank you!

 <https://bitbounce.com?ref=bitbounce> You can sign up or get more info about BitBounce by clicking here.

To deliver your email:

 <https://bitbounce.com/pay_bitbounce_fee/1302462732> We’ve never met. I’ll pay your fee.

 <https://bitbounce.com/whitelist/request_access/32440322?email_address=kerry.raymond%40gmail.com&ref=bitbounce> I know you. Add me to your whitelist.

  <https://bitbounce.com/images/landing/bitbounce_logo.png>

Email that pays.



BitBounce is a product by:

Turing Cloud

 <https://bitbounce.com> BitBounce.com

Redwood City, CA

BitBounce integrates with:

CoinBase

 <https://www.coinbase.com> CoinBase.com

San Francisco, CA

                       
                       
                       
                       

 

                       
                       
                       

 

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: Re: Editor participation rates in surveys

Juliana Bastos
Yes, I also received this horrible spam.

Juliana

On Tue, Oct 31, 2017 at 9:58 PM, Kerry Raymond <[hidden email]>
wrote:

> I just received this response to my message to this wiki-research mailing
> list. Do we think it acceptable that people sign up to this list with this
> kind of demand for payment? I don’t.
>
>
>
> Sure I can just delete the email, but I don’t believe this is acceptable
> behaviour when this person has presumably willingly signed up to this list
> and should therefore be willing to receive legitimate messages sent to it
> by other list members.
>
>
>
> Kerry
>
>
>
> From: REMOVED BY KERRY
> Sent: Wednesday, 1 November 2017 11:15 AM
> To: [hidden email]
> Subject: Re: Re: [Wiki-research-l] Editor participation rates in surveys
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Hello!
>
> I use a new email filtering service called BitBounce to better filter my
> spam. To deliver your email to my inbox, please click the link below and
> pay the small Bitcoin fee. Thank you!
>
>  <https://bitbounce.com?ref=bitbounce> You can sign up or get more info
> about BitBounce by clicking here.
>
> To deliver your email:
>
>  <https://bitbounce.com/pay_bitbounce_fee/1302462732> We’ve never met.
> I’ll pay your fee.
>
>  <https://bitbounce.com/whitelist/request_access/
> 32440322?email_address=kerry.raymond%40gmail.com&ref=bitbounce> I know
> you. Add me to your whitelist.
>
>   <https://bitbounce.com/images/landing/bitbounce_logo.png>
>
> Email that pays.
>
>
>
> BitBounce is a product by:
>
> Turing Cloud
>
>  <https://bitbounce.com> BitBounce.com
>
> Redwood City, CA
>
> BitBounce integrates with:
>
> CoinBase
>
>  <https://www.coinbase.com> CoinBase.com
>
> San Francisco, CA
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>



--
www.domusaurea.org
_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l