FW: What works for increasing editor engagement?

classic Classic list List threaded Threaded
39 messages Options
12
Reply | Threaded
Open this post in threaded view
|

FW: What works for increasing editor engagement?

Kerry Raymond

 

 


From: Kerry Raymond [mailto:[hidden email]]
Sent: Tuesday, 16 September 2014 12:23 PM
To: Research into Wikimedia content and communities
Cc: Editor Engagement
Subject: Re: [Wiki-research-l] What works for increasing editor engagement?

 

With the mood bar, the communication back to the editor was through their user page and email (when known). Do you have any data to show where they saw it (or from where they responded to it)? I've long suspected that new users don't know about User Talk and this frustrates our efforts to communicate with them. so I would be interested to know if there was any difference in reaction from those communicated with via user talk alone and those who also got email and what that might say about user talk as a means to communicate with new users. I note that on the mobile interface running on my ipad, I cannot find a way to get to my User Talk page (as far as I can see), short of entering the URL manually or switching to the desktop interface, which makes user talk pretty useless way of communicating with mobile users.

Sent from my iPad


On 16 Sep 2014, at 6:02 am, Giovanni Luca Ciampaglia <[hidden email]> wrote:

Hi Pine,

 

to answer your question on results about improving editor retention, there is a new paper authored by me and Dario coming out soon about MoodBar, an early EE experiment whose aim was to elicit feedback from newly registered editors, that shows that lightweight socialization (e.g. reporting feedback about editing experience and receiving help from more experienced users) improves long-term editor retention.

 

The pre-print of the paper is up on the arxiv http://arxiv.org/abs/1409.1496 

I also gave a talk about it at the Mediawiki metrics meeting earlier this summer: https://www.youtube.com/watch?v=Rn4-cBYxttA

 

Cheers,


Giovanni Luca Ciampaglia

919 E 10th ∙ Bloomington 47408 IN ∙ USA
http://www.glciampaglia.com/
+1 812 855-7261

[hidden email]

 

2014-09-11 2:00 GMT-04:00 Pine W <[hidden email]>:

Hello research colleagues,

When I look at the WMF Report Card, it appears to me that the global active editor stats and the number of new accounts being registered per month has been relatively flat since at least 2011.

Those of you who work in EE research and analytics, I would like to ask if there is a summary of techniques that you have found that do produce statistically significant results in improving editor retention. I know that some of you write tools, design projects, or pull and analyze data about editors. It looks to me like WMF is investing significant effort in research and tool creation, but we're not moving the needle to create the results that we had hoped to achieve. So I'd like to ask what have we learned from all of our time working on editor engagement about techniques and programs that do improve the EE stats significant ways, so that we can hopefully accelerate the implementation of programs and techniques that have demonstrated success.

 

I'd also like to ask what barriers you think prevent us from becoming more effective at improving the number of users who register and the number of active editors. For example, are users who go through GettingStarted often being deterred by quickly being confronted by experienced editors in ways that make the newbies want to leave? If that is a significant problem, how do you suggest addressing this?

One of my concerns about investing further in developing Flow, analytics tools like like WIkimetrics, and further complex editor engagement research projects, is that the most important challenges related to editor engagement may be problems that can only be solved through primarily interpersonal and social means rather than the use of software tools and mass communications. I like Wikimetrics and I use it, and I think there's an important place for analytics and tool development in EE work, but I wonder if WMF should scale up the emphasis on grassroots social and interpersonal efforts, particularly in the context of the 2015+ Strategic Plan and Jimmy's speech at the 2014 Wikimania. What do you think,and if your answer is yes, how do you think WMF can do this while respecting the autonomy and social processes of the volunteer projects?

Thanks,

Pine


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editor engagement?

Giovanni Luca Ciampaglia-3
Hi Kerry, this is an interesting question but unfortunately no such data was collected from MoodBar. However, we knew whether users had an authenticated email address, and found that this is associated to a 24% higher edit count, as measured in the first 30 days, compared to other users who reported feedback. MoodBar users were also reminded of the possibility of registering an email address, in case they had none, and in this case we observed no difference. Don't know if this answers your question.  

Cheers,

G

 

Giovanni Luca Ciampaglia

✎ 919 E 10th ∙ Bloomington 47408 IN ∙ USA
http://www.glciampaglia.com/
✆ +1 812 855-7261
[hidden email]

2014-09-16 0:47 GMT-04:00 Kerry Raymond <[hidden email]>:

 

 


From: Kerry Raymond [mailto:[hidden email]]
Sent: Tuesday, 16 September 2014 12:23 PM
To: Research into Wikimedia content and communities
Cc: Editor Engagement
Subject: Re: [Wiki-research-l] What works for increasing editor engagement?

 

With the mood bar, the communication back to the editor was through their user page and email (when known). Do you have any data to show where they saw it (or from where they responded to it)? I've long suspected that new users don't know about User Talk and this frustrates our efforts to communicate with them. so I would be interested to know if there was any difference in reaction from those communicated with via user talk alone and those who also got email and what that might say about user talk as a means to communicate with new users. I note that on the mobile interface running on my ipad, I cannot find a way to get to my User Talk page (as far as I can see), short of entering the URL manually or switching to the desktop interface, which makes user talk pretty useless way of communicating with mobile users.

Sent from my iPad


On 16 Sep 2014, at 6:02 am, Giovanni Luca Ciampaglia <[hidden email]> wrote:

Hi Pine,

 

to answer your question on results about improving editor retention, there is a new paper authored by me and Dario coming out soon about MoodBar, an early EE experiment whose aim was to elicit feedback from newly registered editors, that shows that lightweight socialization (e.g. reporting feedback about editing experience and receiving help from more experienced users) improves long-term editor retention.

 

The pre-print of the paper is up on the arxiv http://arxiv.org/abs/1409.1496 

I also gave a talk about it at the Mediawiki metrics meeting earlier this summer: https://www.youtube.com/watch?v=Rn4-cBYxttA

 

Cheers,


Giovanni Luca Ciampaglia

919 E 10th ∙ Bloomington 47408 IN ∙ USA
http://www.glciampaglia.com/
+1 <a href="tel:812%20855-7261" value="+18128557261" target="_blank">812 855-7261

[hidden email]

 

2014-09-11 2:00 GMT-04:00 Pine W <[hidden email]>:

Hello research colleagues,

When I look at the WMF Report Card, it appears to me that the global active editor stats and the number of new accounts being registered per month has been relatively flat since at least 2011.

Those of you who work in EE research and analytics, I would like to ask if there is a summary of techniques that you have found that do produce statistically significant results in improving editor retention. I know that some of you write tools, design projects, or pull and analyze data about editors. It looks to me like WMF is investing significant effort in research and tool creation, but we're not moving the needle to create the results that we had hoped to achieve. So I'd like to ask what have we learned from all of our time working on editor engagement about techniques and programs that do improve the EE stats significant ways, so that we can hopefully accelerate the implementation of programs and techniques that have demonstrated success.

 

I'd also like to ask what barriers you think prevent us from becoming more effective at improving the number of users who register and the number of active editors. For example, are users who go through GettingStarted often being deterred by quickly being confronted by experienced editors in ways that make the newbies want to leave? If that is a significant problem, how do you suggest addressing this?

One of my concerns about investing further in developing Flow, analytics tools like like WIkimetrics, and further complex editor engagement research projects, is that the most important challenges related to editor engagement may be problems that can only be solved through primarily interpersonal and social means rather than the use of software tools and mass communications. I like Wikimetrics and I use it, and I think there's an important place for analytics and tool development in EE work, but I wonder if WMF should scale up the emphasis on grassroots social and interpersonal efforts, particularly in the context of the 2015+ Strategic Plan and Jimmy's speech at the 2014 Wikimania. What do you think,and if your answer is yes, how do you think WMF can do this while respecting the autonomy and social processes of the volunteer projects?

Thanks,

Pine


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editor engagement?

Kerry Raymond

Well, it’s interesting that there was a 24% higher edit counts for those with an email address. The question is what does it tell us. Is there a dependency? Are they both dependent on something else we don’t know about?

 

It might mean (as I am speculating) that those with email addresses were more likely to actually see the notifications of responses to their feedback and more likely to edit more productively from receiving those responses leading to higher edit counts.

 

But it might also mean something completely different. For example, those who envisaged themselves as making many contributions might have thought it worth the extra keystrokes to provide their email address on signup, whereas those who envisaged themselves as just fixing a couple of things might not have thought it worth the effort. In which case, the 24% difference in edit count might just reflect the difference in intention (self-selecting).

 

Kerry

 


From: Giovanni Luca Ciampaglia [mailto:[hidden email]]
Sent: Wednesday, 17 September 2014 11:48 PM
To: [hidden email]; Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] FW: What works for increasing editor engagement?

 

Hi Kerry, this is an interesting question but unfortunately no such data was collected from MoodBar. However, we knew whether users had an authenticated email address, and found that this is associated to a 24% higher edit count, as measured in the first 30 days, compared to other users who reported feedback. MoodBar users were also reminded of the possibility of registering an email address, in case they had none, and in this case we observed no difference. Don't know if this answers your question.  

 

Cheers,

 

G

 

 


Giovanni Luca Ciampaglia

919 E 10th ∙ Bloomington 47408 IN ∙ USA
http://www.glciampaglia.com/
+1 812 855-7261

[hidden email]

 

2014-09-16 0:47 GMT-04:00 Kerry Raymond <[hidden email]>:

 

 


From: Kerry Raymond [mailto:[hidden email]]
Sent: Tuesday, 16 September 2014 12:23 PM
To: Research into Wikimedia content and communities
Cc: Editor Engagement
Subject: Re: [Wiki-research-l] What works for increasing editor engagement?

 

With the mood bar, the communication back to the editor was through their user page and email (when known). Do you have any data to show where they saw it (or from where they responded to it)? I've long suspected that new users don't know about User Talk and this frustrates our efforts to communicate with them. so I would be interested to know if there was any difference in reaction from those communicated with via user talk alone and those who also got email and what that might say about user talk as a means to communicate with new users. I note that on the mobile interface running on my ipad, I cannot find a way to get to my User Talk page (as far as I can see), short of entering the URL manually or switching to the desktop interface, which makes user talk pretty useless way of communicating with mobile users.

Sent from my iPad


On 16 Sep 2014, at 6:02 am, Giovanni Luca Ciampaglia <[hidden email]> wrote:

Hi Pine,

 

to answer your question on results about improving editor retention, there is a new paper authored by me and Dario coming out soon about MoodBar, an early EE experiment whose aim was to elicit feedback from newly registered editors, that shows that lightweight socialization (e.g. reporting feedback about editing experience and receiving help from more experienced users) improves long-term editor retention.

 

The pre-print of the paper is up on the arxiv http://arxiv.org/abs/1409.1496 

I also gave a talk about it at the Mediawiki metrics meeting earlier this summer: https://www.youtube.com/watch?v=Rn4-cBYxttA

 

Cheers,


Giovanni Luca Ciampaglia

919 E 10th ∙ Bloomington 47408 IN ∙ USA
http://www.glciampaglia.com/
+1 <a href="tel:812%20855-7261" target="_blank" value="+18128557261">812 855-7261

[hidden email]

 

2014-09-11 2:00 GMT-04:00 Pine W <[hidden email]>:

Hello research colleagues,

When I look at the WMF Report Card, it appears to me that the global active editor stats and the number of new accounts being registered per month has been relatively flat since at least 2011.

Those of you who work in EE research and analytics, I would like to ask if there is a summary of techniques that you have found that do produce statistically significant results in improving editor retention. I know that some of you write tools, design projects, or pull and analyze data about editors. It looks to me like WMF is investing significant effort in research and tool creation, but we're not moving the needle to create the results that we had hoped to achieve. So I'd like to ask what have we learned from all of our time working on editor engagement about techniques and programs that do improve the EE stats significant ways, so that we can hopefully accelerate the implementation of programs and techniques that have demonstrated success.

 

I'd also like to ask what barriers you think prevent us from becoming more effective at improving the number of users who register and the number of active editors. For example, are users who go through GettingStarted often being deterred by quickly being confronted by experienced editors in ways that make the newbies want to leave? If that is a significant problem, how do you suggest addressing this?

One of my concerns about investing further in developing Flow, analytics tools like like WIkimetrics, and further complex editor engagement research projects, is that the most important challenges related to editor engagement may be problems that can only be solved through primarily interpersonal and social means rather than the use of software tools and mass communications. I like Wikimetrics and I use it, and I think there's an important place for analytics and tool development in EE work, but I wonder if WMF should scale up the emphasis on grassroots social and interpersonal efforts, particularly in the context of the 2015+ Strategic Plan and Jimmy's speech at the 2014 Wikimania. What do you think,and if your answer is yes, how do you think WMF can do this while respecting the autonomy and social processes of the volunteer projects?

Thanks,

Pine


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editor engagement?

Giovanni Luca Ciampaglia-3
Hi Kerry, yes, we took into account the possibility for self-selection when we designed the study. That's actually quite a crucial aspect. That is: is MoodBar conducive to increased productivity and retention (because by using it people can seek help from other editors), or is it just that those who were more motivated and perhaps tech-savvy to begin with were also more likely to use MoodBar to report feedback? For example, many people who managed to complete their first edit reported a "happy" mood, usually with a nice sweet message thanking Wikipedia. 

This is all discussed in the paper, which you are more than welcome to download and read: http://arxiv.org/abs/1409.1496 

I also mentioned it in the talk, which you can watch here: https://www.youtube.com/watch?v=Rn4-cBYxttA

Cheers,

G​​


Giovanni Luca Ciampaglia

✎ 919 E 10th ∙ Bloomington 47408 IN ∙ USA
http://www.glciampaglia.com/
✆ +1 812 855-7261
[hidden email]

2014-09-17 17:39 GMT-04:00 Kerry Raymond <[hidden email]>:

Well, it’s interesting that there was a 24% higher edit counts for those with an email address. The question is what does it tell us. Is there a dependency? Are they both dependent on something else we don’t know about?

 

It might mean (as I am speculating) that those with email addresses were more likely to actually see the notifications of responses to their feedback and more likely to edit more productively from receiving those responses leading to higher edit counts.

 

But it might also mean something completely different. For example, those who envisaged themselves as making many contributions might have thought it worth the extra keystrokes to provide their email address on signup, whereas those who envisaged themselves as just fixing a couple of things might not have thought it worth the effort. In which case, the 24% difference in edit count might just reflect the difference in intention (self-selecting).

 

Kerry

 


From: Giovanni Luca Ciampaglia [mailto:[hidden email]]
Sent: Wednesday, 17 September 2014 11:48 PM
To: [hidden email]; Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] FW: What works for increasing editor engagement?

 

Hi Kerry, this is an interesting question but unfortunately no such data was collected from MoodBar. However, we knew whether users had an authenticated email address, and found that this is associated to a 24% higher edit count, as measured in the first 30 days, compared to other users who reported feedback. MoodBar users were also reminded of the possibility of registering an email address, in case they had none, and in this case we observed no difference. Don't know if this answers your question.  

 

Cheers,

 

G

 

 


Giovanni Luca Ciampaglia

919 E 10th ∙ Bloomington 47408 IN ∙ USA
http://www.glciampaglia.com/
+1 <a href="tel:812%20855-7261" value="+18128557261" target="_blank">812 855-7261

[hidden email]

 

2014-09-16 0:47 GMT-04:00 Kerry Raymond <[hidden email]>:

 

 


From: Kerry Raymond [mailto:[hidden email]]
Sent: Tuesday, 16 September 2014 12:23 PM
To: Research into Wikimedia content and communities
Cc: Editor Engagement
Subject: Re: [Wiki-research-l] What works for increasing editor engagement?

 

With the mood bar, the communication back to the editor was through their user page and email (when known). Do you have any data to show where they saw it (or from where they responded to it)? I've long suspected that new users don't know about User Talk and this frustrates our efforts to communicate with them. so I would be interested to know if there was any difference in reaction from those communicated with via user talk alone and those who also got email and what that might say about user talk as a means to communicate with new users. I note that on the mobile interface running on my ipad, I cannot find a way to get to my User Talk page (as far as I can see), short of entering the URL manually or switching to the desktop interface, which makes user talk pretty useless way of communicating with mobile users.

Sent from my iPad


On 16 Sep 2014, at 6:02 am, Giovanni Luca Ciampaglia <[hidden email]> wrote:

Hi Pine,

 

to answer your question on results about improving editor retention, there is a new paper authored by me and Dario coming out soon about MoodBar, an early EE experiment whose aim was to elicit feedback from newly registered editors, that shows that lightweight socialization (e.g. reporting feedback about editing experience and receiving help from more experienced users) improves long-term editor retention.

 

The pre-print of the paper is up on the arxiv http://arxiv.org/abs/1409.1496 

I also gave a talk about it at the Mediawiki metrics meeting earlier this summer: https://www.youtube.com/watch?v=Rn4-cBYxttA

 

Cheers,


Giovanni Luca Ciampaglia

919 E 10th ∙ Bloomington 47408 IN ∙ USA
http://www.glciampaglia.com/
+1 <a href="tel:812%20855-7261" value="+18128557261" target="_blank">812 855-7261

[hidden email]

 

2014-09-11 2:00 GMT-04:00 Pine W <[hidden email]>:

Hello research colleagues,

When I look at the WMF Report Card, it appears to me that the global active editor stats and the number of new accounts being registered per month has been relatively flat since at least 2011.

Those of you who work in EE research and analytics, I would like to ask if there is a summary of techniques that you have found that do produce statistically significant results in improving editor retention. I know that some of you write tools, design projects, or pull and analyze data about editors. It looks to me like WMF is investing significant effort in research and tool creation, but we're not moving the needle to create the results that we had hoped to achieve. So I'd like to ask what have we learned from all of our time working on editor engagement about techniques and programs that do improve the EE stats significant ways, so that we can hopefully accelerate the implementation of programs and techniques that have demonstrated success.

 

I'd also like to ask what barriers you think prevent us from becoming more effective at improving the number of users who register and the number of active editors. For example, are users who go through GettingStarted often being deterred by quickly being confronted by experienced editors in ways that make the newbies want to leave? If that is a significant problem, how do you suggest addressing this?

One of my concerns about investing further in developing Flow, analytics tools like like WIkimetrics, and further complex editor engagement research projects, is that the most important challenges related to editor engagement may be problems that can only be solved through primarily interpersonal and social means rather than the use of software tools and mass communications. I like Wikimetrics and I use it, and I think there's an important place for analytics and tool development in EE work, but I wonder if WMF should scale up the emphasis on grassroots social and interpersonal efforts, particularly in the context of the 2015+ Strategic Plan and Jimmy's speech at the 2014 Wikimania. What do you think,and if your answer is yes, how do you think WMF can do this while respecting the autonomy and social processes of the volunteer projects?

Thanks,

Pine


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editor engagement?

Aaron Halfaker-3
Hey folks,

I've been hesitant to chime in because there is so much to discuss wrapped up in this question that I'd probably not get any work done for a few days if I attempted answering it.  So, I propose a project where we work together to generate such a summary so that I can call it "work".  

I've started a stub here: https://meta.wikimedia.org/wiki/Research:New_editor_engagement_strategies  Please feel welcome to contribute boldly.  We can work out the details on the talk page.  

-Aaron


On Wed, Sep 17, 2014 at 5:08 PM, Giovanni Luca Ciampaglia <[hidden email]> wrote:
Hi Kerry, yes, we took into account the possibility for self-selection when we designed the study. That's actually quite a crucial aspect. That is: is MoodBar conducive to increased productivity and retention (because by using it people can seek help from other editors), or is it just that those who were more motivated and perhaps tech-savvy to begin with were also more likely to use MoodBar to report feedback? For example, many people who managed to complete their first edit reported a "happy" mood, usually with a nice sweet message thanking Wikipedia. 

This is all discussed in the paper, which you are more than welcome to download and read: http://arxiv.org/abs/1409.1496 

I also mentioned it in the talk, which you can watch here: https://www.youtube.com/watch?v=Rn4-cBYxttA

Cheers,

G​​


Giovanni Luca Ciampaglia

✎ 919 E 10th ∙ Bloomington 47408 IN ∙ USA
http://www.glciampaglia.com/
✆ <a href="tel:%2B1%20812%20855-7261" value="+18128557261" target="_blank">+1 812 855-7261
[hidden email]

2014-09-17 17:39 GMT-04:00 Kerry Raymond <[hidden email]>:

Well, it’s interesting that there was a 24% higher edit counts for those with an email address. The question is what does it tell us. Is there a dependency? Are they both dependent on something else we don’t know about?

 

It might mean (as I am speculating) that those with email addresses were more likely to actually see the notifications of responses to their feedback and more likely to edit more productively from receiving those responses leading to higher edit counts.

 

But it might also mean something completely different. For example, those who envisaged themselves as making many contributions might have thought it worth the extra keystrokes to provide their email address on signup, whereas those who envisaged themselves as just fixing a couple of things might not have thought it worth the effort. In which case, the 24% difference in edit count might just reflect the difference in intention (self-selecting).

 

Kerry

 


From: Giovanni Luca Ciampaglia [mailto:[hidden email]]
Sent: Wednesday, 17 September 2014 11:48 PM
To: [hidden email]; Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] FW: What works for increasing editor engagement?

 

Hi Kerry, this is an interesting question but unfortunately no such data was collected from MoodBar. However, we knew whether users had an authenticated email address, and found that this is associated to a 24% higher edit count, as measured in the first 30 days, compared to other users who reported feedback. MoodBar users were also reminded of the possibility of registering an email address, in case they had none, and in this case we observed no difference. Don't know if this answers your question.  

 

Cheers,

 

G

 

 


Giovanni Luca Ciampaglia

919 E 10th ∙ Bloomington 47408 IN ∙ USA
http://www.glciampaglia.com/
+1 <a href="tel:812%20855-7261" value="+18128557261" target="_blank">812 855-7261

[hidden email]

 

2014-09-16 0:47 GMT-04:00 Kerry Raymond <[hidden email]>:

 

 


From: Kerry Raymond [mailto:[hidden email]]
Sent: Tuesday, 16 September 2014 12:23 PM
To: Research into Wikimedia content and communities
Cc: Editor Engagement
Subject: Re: [Wiki-research-l] What works for increasing editor engagement?

 

With the mood bar, the communication back to the editor was through their user page and email (when known). Do you have any data to show where they saw it (or from where they responded to it)? I've long suspected that new users don't know about User Talk and this frustrates our efforts to communicate with them. so I would be interested to know if there was any difference in reaction from those communicated with via user talk alone and those who also got email and what that might say about user talk as a means to communicate with new users. I note that on the mobile interface running on my ipad, I cannot find a way to get to my User Talk page (as far as I can see), short of entering the URL manually or switching to the desktop interface, which makes user talk pretty useless way of communicating with mobile users.

Sent from my iPad


On 16 Sep 2014, at 6:02 am, Giovanni Luca Ciampaglia <[hidden email]> wrote:

Hi Pine,

 

to answer your question on results about improving editor retention, there is a new paper authored by me and Dario coming out soon about MoodBar, an early EE experiment whose aim was to elicit feedback from newly registered editors, that shows that lightweight socialization (e.g. reporting feedback about editing experience and receiving help from more experienced users) improves long-term editor retention.

 

The pre-print of the paper is up on the arxiv http://arxiv.org/abs/1409.1496 

I also gave a talk about it at the Mediawiki metrics meeting earlier this summer: https://www.youtube.com/watch?v=Rn4-cBYxttA

 

Cheers,


Giovanni Luca Ciampaglia

919 E 10th ∙ Bloomington 47408 IN ∙ USA
http://www.glciampaglia.com/
+1 <a href="tel:812%20855-7261" value="+18128557261" target="_blank">812 855-7261

[hidden email]

 

2014-09-11 2:00 GMT-04:00 Pine W <[hidden email]>:

Hello research colleagues,

When I look at the WMF Report Card, it appears to me that the global active editor stats and the number of new accounts being registered per month has been relatively flat since at least 2011.

Those of you who work in EE research and analytics, I would like to ask if there is a summary of techniques that you have found that do produce statistically significant results in improving editor retention. I know that some of you write tools, design projects, or pull and analyze data about editors. It looks to me like WMF is investing significant effort in research and tool creation, but we're not moving the needle to create the results that we had hoped to achieve. So I'd like to ask what have we learned from all of our time working on editor engagement about techniques and programs that do improve the EE stats significant ways, so that we can hopefully accelerate the implementation of programs and techniques that have demonstrated success.

 

I'd also like to ask what barriers you think prevent us from becoming more effective at improving the number of users who register and the number of active editors. For example, are users who go through GettingStarted often being deterred by quickly being confronted by experienced editors in ways that make the newbies want to leave? If that is a significant problem, how do you suggest addressing this?

One of my concerns about investing further in developing Flow, analytics tools like like WIkimetrics, and further complex editor engagement research projects, is that the most important challenges related to editor engagement may be problems that can only be solved through primarily interpersonal and social means rather than the use of software tools and mass communications. I like Wikimetrics and I use it, and I think there's an important place for analytics and tool development in EE work, but I wonder if WMF should scale up the emphasis on grassroots social and interpersonal efforts, particularly in the context of the 2015+ Strategic Plan and Jimmy's speech at the 2014 Wikimania. What do you think,and if your answer is yes, how do you think WMF can do this while respecting the autonomy and social processes of the volunteer projects?

Thanks,

Pine


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editor engagement?

James Salsman-2
In reply to this post by Kerry Raymond
Hi Aaron,

available for correlation with the number of new articles each user created?

Aaron Halfaker <[hidden email]> wrote:
>...
> I propose a project where we work together to generate 
> a summary so that I can call it "work." I've started a stub here:

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editor engagement?

Aaron Halfaker-2
Sure!  You'll find the hand-coded set of users here within the next half hour (cron job copies datasets over).  

Categories: 
  1. Vandals - Purposefully malicious, out to cause harm
  2. Bad-faith - Trying to be funny, not here to help or harm
  3. Good-faith - Trying to be productive, but failing
  4. Golden - Successfully contributing productively
-Aaron

On Wed, Sep 24, 2014 at 1:15 PM, James Salsman <[hidden email]> wrote:
Hi Aaron,

available for correlation with the number of new articles each user created?

Aaron Halfaker <[hidden email]> wrote:
>...
> I propose a project where we work together to generate 
> a summary so that I can call it "work." I've started a stub here:

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editor engagement?

James Salsman-2
In reply to this post by Kerry Raymond
Thanks, Aaron, that is very helpful!

Do you also have data sets for number of still-extant non-redirect
articles created for the good-faith and golden users?

I can figure that out from the API, but I'm busy. Also, I have
previously hypothesized that creating an article is strongly
correlated with survival of desirable newcomers, and you probably have
people working for you who have no preconceived notions on the
question, or if you don't you can find them.

Best regards,
James Salsman

Aaron wrote:
>... You'll find the hand-coded set of users here
> http://datasets.wikimedia.org/public-datasets/enwiki/rise-and-decline
>...
> Categories:
>
>   1. Vandals - Purposefully malicious, out to cause harm
>   2. Bad-faith - Trying to be funny, not here to help or harm
>   3. Good-faith - Trying to be productive, but failing
>   4. Golden - Successfully contributing productively....

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editor engagement?

Aaron Halfaker-3
Regretfully I don't have that stat.  I'd be interested in digging into this with you though.  Could you write up a brief description of a Research Idea here: https://meta.wikimedia.org/wiki/Research:Labs2/Ideas and ping me?  

-Aaron 

On Wed, Sep 24, 2014 at 7:35 PM, James Salsman <[hidden email]> wrote:
Thanks, Aaron, that is very helpful!

Do you also have data sets for number of still-extant non-redirect
articles created for the good-faith and golden users?

I can figure that out from the API, but I'm busy. Also, I have
previously hypothesized that creating an article is strongly
correlated with survival of desirable newcomers, and you probably have
people working for you who have no preconceived notions on the
question, or if you don't you can find them.

Best regards,
James Salsman

Aaron wrote:
>... You'll find the hand-coded set of users here
> http://datasets.wikimedia.org/public-datasets/enwiki/rise-and-decline
>...
> Categories:
>
>   1. Vandals - Purposefully malicious, out to cause harm
>   2. Bad-faith - Trying to be funny, not here to help or harm
>   3. Good-faith - Trying to be productive, but failing
>   4. Golden - Successfully contributing productively....

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editor engagement?

WereSpielChequers-2
In reply to this post by Aaron Halfaker-2
Hi, if we are analysing new editors creating new articles then there are two groups I would expect to see, individual spammers and corporate spammers. 

The individual spammers are the kids writing about the band that will be the next big thing on the ....... Scene in their town, and their first gig is next Tuesday provided they can recruit a drummer.

The corporate spammers are writing about some business, but in a style that makes a corporate flyer look neutral. 


There are some implicit assumptions that we make as new page patrollers, and which some research might help by proving or disproving.

1 people whose first article is an attack page or vandalism will very rarely turn into productive editors. I have met or heard of three former vandals who made good editors, there may be more, but I've never heard of a good editor who started out creating attack pages.

2 people who are paid to spam this site are unlikely to turn into good editors, but a large proportion of good editors made COI edits among their early edits. The classic wikipedian is a time rich altruist with Internet access that they can make personal use of, I'm not seeing much overlap there with corporate spammers, but people who make COI edits other than about their employment clearly have personal access to the Internet.

3 neutral point of view is something that new editors need to pick up, and POV editing amongst a new editor's early contributions is so common that if a new account writes neutrally they are  assumed by some to be sock puppets or returning editors.

4 we live in a copy paste world, and while a student plagiarising to try to get better marks is clearly doing so in bad faith, a wikipedia contributor using copy paste is still a good faith contributor, they just need to be taught not to use copy and paste. Presumably these people are in Aaron's group 3, again we have a lot of former copyright violators in the community, and a lot of new articles get deleted as copyright violations.

My suspicion is that much of our tolerance of vandals is wasted effort. We would save a lot of time and not lose any good editors by moving from the current five strikes and you are out approach to vandals to one of blocking any new editor after their first edit if that edit was blatant vandalism. Copyvio by contrast is something where we could probably retain more of the editors by trying different approaches to explaining why their edits had to be rejected and how they could contribute.


Regards

Jonathan Cardy


On 25 Sep 2014, at 00:19, Aaron Halfaker <[hidden email]> wrote:

Sure!  You'll find the hand-coded set of users here within the next half hour (cron job copies datasets over).  

Categories: 
  1. Vandals - Purposefully malicious, out to cause harm
  2. Bad-faith - Trying to be funny, not here to help or harm
  3. Good-faith - Trying to be productive, but failing
  4. Golden - Successfully contributing productively
-Aaron

On Wed, Sep 24, 2014 at 1:15 PM, James Salsman <[hidden email]> wrote:
Hi Aaron,

available for correlation with the number of new articles each user created?

Aaron Halfaker <[hidden email]> wrote:
>...
> I propose a project where we work together to generate 
> a summary so that I can call it "work." I've started a stub here:

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editorengagement?

Kerry Raymond

This pretty much matches my own observations.

 

I think it’s not uncommon for new editors to first write about something they know well, which often means some level of conflict of interest. But that doesn’t necessarily mean the person is writing lies or being completely over-the-top with lavish praise; they can write quite reasonable content too. And generally new editors don’t know that CoI editing isn’t welcome. Similarly a lot get blocked for 3 reverts and get accused of edit warring, but I suspect it is often not the case. I suspect often they look back at the article and can’t see the edit they did, so they just do it again because it didn’t “stick”. I am not convinced they do know that other people are reverting their edits because I don’t think they know about the article history and the edit summaries and the user talk pages.

 

We make some awfully big assumptions that new editors know what we do; it’s amazing how much stuff in a UI you mentally filter out. I did a talk about Wikipedia about a year ago for a group of librarians. They asked for some “tips and tricks” to be included, so for the first time I systematically scanned the article UI and found cute little features I never knew were there (even on menus I often used). And of course the UI changes over time and probably I didn’t notice new things appear. But not noticing is consistent with “mission focus”, which has been studied in retail settings. A customer “on a mission” (a specific transaction that must be dealt with, as opposed to a customer “just browsing”) is oblivious to promotions until the “mission” is complete and then becomes aware of their surroundings as they turn to leave the store (which tells you where to put promotional signs relative to your “mission” products). I think Wikipedia editing is very mission-focussed (I must fix that error) so we see nothing but what we want to see on the way in and then have no “exit screen” to talk to the editor as they leave (unlike a bricks-and-mortar retailer). It’s hard to communicate with new editors on-wiki.

 

I actually think doing a user experience experiment (ideally with eye tracking) with new users would be very informative. Given them a “mission” and watch what they look at, what they do and if they succeed in their task, and whether their edits survives over time (and if it doesn’t survive, why is that). Indeed, seeing what they do when their edit disappears would be interesting too.

 

It is difficult for many of us as experienced Wikipedians to see Wikipedia as the new contributor sees it. I have some sense of the new editor experience because I do edit training and it certainly shows why nothing is fool-proof because fools are so ingenious. It’s amazing how new editors dive into parts of the UI I’ve never even noticed and then tie themselves up in knots. They often get so interested in Preview that they never Save (and then suddenly oops, all their work is gone). They generate continual edit conflicts (often entirely on their own, I think by going back in the browser and editing the article in multiple browser windows, leading to their contributions being smeared across a number of edit windows, none of which can be saved, or so it seems). And if they are editing in mainspace, they panic if the article changes in some mysterious way and they don’t know how they did it (it was another concurrent editor). In fact, if there is one thing I would like is for the watchlist notification to be slower than normal when a new editor is involved as having someone alerted by the watchlist coming and “fixing” things (even if it’s well-meant) puts the new editor into a spin because they don’t understand what’s going on (they have no experience with collaborative document editing). And, a perennial favourite, why is the Reference section empty when they’ve added all those citations (they don’t expect them to be in-line, they expect them in the References section). When I look at the constant questions and problems I have to deal with in edit training, I am not surprised that people trying to learn on their own give up. Seen through the eyes of the beginner, it’s so much harder than we realise both technically and collaboratively.

 

Kerry

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of WereSpielChequers
Sent: Thursday, 25 September 2014 2:18 PM
To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] FW: What works for increasing editorengagement?

 

Hi, if we are analysing new editors creating new articles then there are two groups I would expect to see, individual spammers and corporate spammers. 

The individual spammers are the kids writing about the band that will be the next big thing on the ....... Scene in their town, and their first gig is next Tuesday provided they can recruit a drummer.

 

The corporate spammers are writing about some business, but in a style that makes a corporate flyer look neutral. 

 

 

There are some implicit assumptions that we make as new page patrollers, and which some research might help by proving or disproving.

 

1 people whose first article is an attack page or vandalism will very rarely turn into productive editors. I have met or heard of three former vandals who made good editors, there may be more, but I've never heard of a good editor who started out creating attack pages.

 

2 people who are paid to spam this site are unlikely to turn into good editors, but a large proportion of good editors made COI edits among their early edits. The classic wikipedian is a time rich altruist with Internet access that they can make personal use of, I'm not seeing much overlap there with corporate spammers, but people who make COI edits other than about their employment clearly have personal access to the Internet.

 

3 neutral point of view is something that new editors need to pick up, and POV editing amongst a new editor's early contributions is so common that if a new account writes neutrally they are  assumed by some to be sock puppets or returning editors.

 

4 we live in a copy paste world, and while a student plagiarising to try to get better marks is clearly doing so in bad faith, a wikipedia contributor using copy paste is still a good faith contributor, they just need to be taught not to use copy and paste. Presumably these people are in Aaron's group 3, again we have a lot of former copyright violators in the community, and a lot of new articles get deleted as copyright violations.

 

My suspicion is that much of our tolerance of vandals is wasted effort. We would save a lot of time and not lose any good editors by moving from the current five strikes and you are out approach to vandals to one of blocking any new editor after their first edit if that edit was blatant vandalism. Copyvio by contrast is something where we could probably retain more of the editors by trying different approaches to explaining why their edits had to be rejected and how they could contribute.


Regards

 

Jonathan Cardy

 


On 25 Sep 2014, at 00:19, Aaron Halfaker <[hidden email]> wrote:

Sure!  You'll find the hand-coded set of users here within the next half hour (cron job copies datasets over).  

 

Categories: 

  1. Vandals - Purposefully malicious, out to cause harm
  2. Bad-faith - Trying to be funny, not here to help or harm
  3. Good-faith - Trying to be productive, but failing
  4. Golden - Successfully contributing productively

-Aaron

 

On Wed, Sep 24, 2014 at 1:15 PM, James Salsman <[hidden email]> wrote:

Hi Aaron,

 

available for correlation with the number of new articles each user created?


Aaron Halfaker <[hidden email]> wrote:
>...

> I propose a project where we work together to generate 

> a summary so that I can call it "work." I've started a stub here:


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editorengagement?

Pine W
This is a great discussion.

Kerry, I believe that Steven Walling and what was called the E3 team, now known as the Growth team, use eye tracking experiments. It would be interesting to hear about what they're learning, and tie those observations together with yours and WSC's.

Also, I am happy to discuss the idea of teaming up with one or more people in this discussion to create an Individual Engagement Grant proposal that would help us to explore some of the material that we're discussing here. Aaron and I are already putting together a proposal related to how editors interact with each other, and I'd be happy to work on concurrent projects related to research on new editors if I can make the scheduling and financing work.

Pine

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editorengagement?

WereSpielChequers-2
In reply to this post by Kerry Raymond
Yes, training newbies is a great way to learn and to see the flaws that we mentally blank out. I also found that I need to keep a vanilla account for demonstrating things to newbies, if I use my WereSpielChequers account the various extra buttons confuse people. 

I wouldn't worry too much about watchlisters making edits and causing edit conflicts, most of the time you aren't going to be in the same time zone, watchlisters even the most active ones are unlikely to check their watch list more than a few times a day. So as long as you don't start newbies on highly watched articles like Sarah Palin you should be OK with them. But edit conflicts are a real problem for newbies and especially those creating new articles. The new page patrol people need to look at articles as they are created in order to pick up attack pages etc, and when they find OK articles they tend to at least categorise them. Some of this could be fixed by improving the software for handling edit conflicts, for example it would be nice if adding a category and changing some text were not treated as a conflict. However this is the sort of Bugzilla request that gets closed as won't fix because of the implicit assumption that any editor should be able to handle an edit conflict in their stride. My preferred solution is to teach newbies to always start new articles in sandboxes and move them to main space when they are ready, alternatively creating a references section means that categories can be added without causing an edit conflict.



Re Pine's comment. When it comes to research topics It would be great to know how many editors are driven away by edit conflicts. But I don't know if that info is actually logged, and if you ask people why they gave up they don't necessarily know whether their edit conflict was with a machine or another editor. I once had to convince an angry editor that they hadn't been reverted by another editor, both edits had the same time stamp and theirs had been lost by the system without the other editor knowing. So if you ask former editors why they went and they blame other editors it may partly be that they don't know the difference between a conflict with the system and with another editor deliberately reverting them.


By the way, if anyone is around in London drop me a line and I will tell you if we have any editathons going on. Editathons have many roles in the community, one being as a focus group on the user interface.

Regards

Jonathan Cardy


On 25 Sep 2014, at 10:09, Kerry Raymond <[hidden email]> wrote:

This pretty much matches my own observations.

 

I think it’s not uncommon for new editors to first write about something they know well, which often means some level of conflict of interest. But that doesn’t necessarily mean the person is writing lies or being completely over-the-top with lavish praise; they can write quite reasonable content too. And generally new editors don’t know that CoI editing isn’t welcome. Similarly a lot get blocked for 3 reverts and get accused of edit warring, but I suspect it is often not the case. I suspect often they look back at the article and can’t see the edit they did, so they just do it again because it didn’t “stick”. I am not convinced they do know that other people are reverting their edits because I don’t think they know about the article history and the edit summaries and the user talk pages.

 

We make some awfully big assumptions that new editors know what we do; it’s amazing how much stuff in a UI you mentally filter out. I did a talk about Wikipedia about a year ago for a group of librarians. They asked for some “tips and tricks” to be included, so for the first time I systematically scanned the article UI and found cute little features I never knew were there (even on menus I often used). And of course the UI changes over time and probably I didn’t notice new things appear. But not noticing is consistent with “mission focus”, which has been studied in retail settings. A customer “on a mission” (a specific transaction that must be dealt with, as opposed to a customer “just browsing”) is oblivious to promotions until the “mission” is complete and then becomes aware of their surroundings as they turn to leave the store (which tells you where to put promotional signs relative to your “mission” products). I think Wikipedia editing is very mission-focussed (I must fix that error) so we see nothing but what we want to see on the way in and then have no “exit screen” to talk to the editor as they leave (unlike a bricks-and-mortar retailer). It’s hard to communicate with new editors on-wiki.

 

I actually think doing a user experience experiment (ideally with eye tracking) with new users would be very informative. Given them a “mission” and watch what they look at, what they do and if they succeed in their task, and whether their edits survives over time (and if it doesn’t survive, why is that). Indeed, seeing what they do when their edit disappears would be interesting too.

 

It is difficult for many of us as experienced Wikipedians to see Wikipedia as the new contributor sees it. I have some sense of the new editor experience because I do edit training and it certainly shows why nothing is fool-proof because fools are so ingenious. It’s amazing how new editors dive into parts of the UI I’ve never even noticed and then tie themselves up in knots. They often get so interested in Preview that they never Save (and then suddenly oops, all their work is gone). They generate continual edit conflicts (often entirely on their own, I think by going back in the browser and editing the article in multiple browser windows, leading to their contributions being smeared across a number of edit windows, none of which can be saved, or so it seems). And if they are editing in mainspace, they panic if the article changes in some mysterious way and they don’t know how they did it (it was another concurrent editor). In fact, if there is one thing I would like is for the watchlist notification to be slower than normal when a new editor is involved as having someone alerted by the watchlist coming and “fixing” things (even if it’s well-meant) puts the new editor into a spin because they don’t understand what’s going on (they have no experience with collaborative document editing). And, a perennial favourite, why is the Reference section empty when they’ve added all those citations (they don’t expect them to be in-line, they expect them in the References section). When I look at the constant questions and problems I have to deal with in edit training, I am not surprised that people trying to learn on their own give up. Seen through the eyes of the beginner, it’s so much harder than we realise both technically and collaboratively.

 

Kerry

 


From: [hidden email] [[hidden email]] On Behalf Of WereSpielChequers
Sent: Thursday, 25 September 2014 2:18 PM
To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] FW: What works for increasing editorengagement?

 

Hi, if we are analysing new editors creating new articles then there are two groups I would expect to see, individual spammers and corporate spammers. 

The individual spammers are the kids writing about the band that will be the next big thing on the ....... Scene in their town, and their first gig is next Tuesday provided they can recruit a drummer.

 

The corporate spammers are writing about some business, but in a style that makes a corporate flyer look neutral. 

 

 

There are some implicit assumptions that we make as new page patrollers, and which some research might help by proving or disproving.

 

1 people whose first article is an attack page or vandalism will very rarely turn into productive editors. I have met or heard of three former vandals who made good editors, there may be more, but I've never heard of a good editor who started out creating attack pages.

 

2 people who are paid to spam this site are unlikely to turn into good editors, but a large proportion of good editors made COI edits among their early edits. The classic wikipedian is a time rich altruist with Internet access that they can make personal use of, I'm not seeing much overlap there with corporate spammers, but people who make COI edits other than about their employment clearly have personal access to the Internet.

 

3 neutral point of view is something that new editors need to pick up, and POV editing amongst a new editor's early contributions is so common that if a new account writes neutrally they are  assumed by some to be sock puppets or returning editors.

 

4 we live in a copy paste world, and while a student plagiarising to try to get better marks is clearly doing so in bad faith, a wikipedia contributor using copy paste is still a good faith contributor, they just need to be taught not to use copy and paste. Presumably these people are in Aaron's group 3, again we have a lot of former copyright violators in the community, and a lot of new articles get deleted as copyright violations.

 

My suspicion is that much of our tolerance of vandals is wasted effort. We would save a lot of time and not lose any good editors by moving from the current five strikes and you are out approach to vandals to one of blocking any new editor after their first edit if that edit was blatant vandalism. Copyvio by contrast is something where we could probably retain more of the editors by trying different approaches to explaining why their edits had to be rejected and how they could contribute.


Regards

 

Jonathan Cardy

 


On 25 Sep 2014, at 00:19, Aaron Halfaker <[hidden email]> wrote:

Sure!  You'll find the hand-coded set of users here within the next half hour (cron job copies datasets over).  

 

Categories: 

  1. Vandals - Purposefully malicious, out to cause harm
  2. Bad-faith - Trying to be funny, not here to help or harm
  3. Good-faith - Trying to be productive, but failing
  4. Golden - Successfully contributing productively

-Aaron

 

On Wed, Sep 24, 2014 at 1:15 PM, James Salsman <[hidden email]> wrote:

Hi Aaron,

 

available for correlation with the number of new articles each user created?


Aaron Halfaker <[hidden email]> wrote:
>...

> I propose a project where we work together to generate 

> a summary so that I can call it "work." I've started a stub here:


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editor engagement?

James Salsman-2
In reply to this post by Kerry Raymond
Aaron, would you please post the script you used to create
https://commons.wikimedia.org/wiki/File:Desirable_newcomer_survival_over_time.png
?

I would be happy to modify it to also collect the number of extant
non-redirect articles each desirable user created.

> Aaron wrote:
> >... You'll find the hand-coded set of users here
> > http://datasets.wikimedia.org/public-datasets/enwiki/rise-and-decline
> >...
> > Categories:
> >
> >   1. Vandals - Purposefully malicious, out to cause harm
> >   2. Bad-faith - Trying to be funny, not here to help or harm
> >   3. Good-faith - Trying to be productive, but failing
> >   4. Golden - Successfully contributing productively....

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editor engagement?

Luca de Alfaro-5
Re. the edit conflicts happening when a new user is editing: 

Can't one add some AJAX to the editor that notifies that one still has the editing window open? Maybe editors could wait to modify work in progress, if they had that indication, and if the content does not seem vandalism? 

Luca

On Thu, Sep 25, 2014 at 12:17 PM, James Salsman <[hidden email]> wrote:
Aaron, would you please post the script you used to create
https://commons.wikimedia.org/wiki/File:Desirable_newcomer_survival_over_time.png
?

I would be happy to modify it to also collect the number of extant
non-redirect articles each desirable user created.

> Aaron wrote:
> >... You'll find the hand-coded set of users here
> > http://datasets.wikimedia.org/public-datasets/enwiki/rise-and-decline
> >...
> > Categories:
> >
> >   1. Vandals - Purposefully malicious, out to cause harm
> >   2. Bad-faith - Trying to be funny, not here to help or harm
> >   3. Good-faith - Trying to be productive, but failing
> >   4. Golden - Successfully contributing productively....

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editor engagement?

James Salsman-2
In reply to this post by Kerry Raymond
Luca wrote:
>
> Re. the edit conflicts happening when a new user is editing:
>
> Can't one add some AJAX to the editor that notifies that one
> still has the editing window open? Maybe editors could wait to
> modify work in progress, if they had that indication, and if the
> content does not seem vandalism?

Instead of asking editors to wait, we could improve the merge
algorithm to avoid conflicts:

https://en.wikipedia.org/wiki/Merge_(revision_control)

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editor engagement?

Luca de Alfaro-5
Better merging would be welcome.  But also less aggressive editing/policing. 

When I edit openstreetmap I have a better overall experience: the edits may or may not go live immediately, but I don't have the impression that there is someone aggressively vetting/refining my edits while I am still doing them.  I feel welcome there. 

To make Wikipedia more welcoming, we could do a few things. 

We could allow users to save drafts.  In this way, people could work for a while at their own pace, and then publish the changes.  Currently, saving is the only way to avoid risking losing changes, but it has the very undesired effect of inviting editors/vetters to the page before one is really done. 

We could also allow a time window (even 30 minutes) before edits went live after one is done editing (using above Ajax mechanism to track when editor open), experienced editors would not need to swoop in quite so fast on the work of new users, and the whole editing atmosphere would be more relaxed and welcoming. 

The fact is that the Wikipedia editor, with its lack of ability to save drafts, poor merging, and swooping editors, feels incredibly outdated and unwelcoming - downright aggressive - to anyone used to WordPress / Google Docs / Blogger / ...

Luca

On Thu, Sep 25, 2014 at 12:35 PM, James Salsman <[hidden email]> wrote:
Luca wrote:
>
> Re. the edit conflicts happening when a new user is editing:
>
> Can't one add some AJAX to the editor that notifies that one
> still has the editing window open? Maybe editors could wait to
> modify work in progress, if they had that indication, and if the
> content does not seem vandalism?

Instead of asking editors to wait, we could improve the merge
algorithm to avoid conflicts:

https://en.wikipedia.org/wiki/Merge_(revision_control)

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editorengagement?

Kerry Raymond
In reply to this post by WereSpielChequers-2

Australian outreach events generally edit Australian content. Other Australian editors are likely to be on the watch list and are likely to be in the timezone. And plenty of non-Australian editors are sitting in their pyjamas at all hours of the day and night waiting to pounce. Believe me, new editors encounter other editors very quickly (although sometimes they don’t realise it). They often think it “rude” that other people are editing the article “while I am in the middle of working on it”. Their mental model of collaborative editing is like a shared lawn mower. You have sole use for a while; then it is passed on to the next person.

 

Not sure I can help you with London editathons. But I do have a couple of edit training days coming up in Oakey, Queensland in a few weeks:

 

https://en.wikipedia.org/wiki/Oakey,_Queensland

 

Kerry

 

 


From: WereSpielChequers [mailto:[hidden email]]
Sent: Thursday, 25 September 2014 9:38 PM
To: [hidden email]; Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] FW: What works for increasing editorengagement?

 

Yes, training newbies is a great way to learn and to see the flaws that we mentally blank out. I also found that I need to keep a vanilla account for demonstrating things to newbies, if I use my WereSpielChequers account the various extra buttons confuse people. 

 

I wouldn't worry too much about watchlisters making edits and causing edit conflicts, most of the time you aren't going to be in the same time zone, watchlisters even the most active ones are unlikely to check their watch list more than a few times a day. So as long as you don't start newbies on highly watched articles like Sarah Palin you should be OK with them. But edit conflicts are a real problem for newbies and especially those creating new articles. The new page patrol people need to look at articles as they are created in order to pick up attack pages etc, and when they find OK articles they tend to at least categorise them. Some of this could be fixed by improving the software for handling edit conflicts, for example it would be nice if adding a category and changing some text were not treated as a conflict. However this is the sort of Bugzilla request that gets closed as won't fix because of the implicit assumption that any editor should be able to handle an edit conflict in their stride. My preferred solution is to teach newbies to always start new articles in sandboxes and move them to main space when they are ready, alternatively creating a references section means that categories can be added without causing an edit conflict.

 

 

 

Re Pine's comment. When it comes to research topics It would be great to know how many editors are driven away by edit conflicts. But I don't know if that info is actually logged, and if you ask people why they gave up they don't necessarily know whether their edit conflict was with a machine or another editor. I once had to convince an angry editor that they hadn't been reverted by another editor, both edits had the same time stamp and theirs had been lost by the system without the other editor knowing. So if you ask former editors why they went and they blame other editors it may partly be that they don't know the difference between a conflict with the system and with another editor deliberately reverting them.

 

 

By the way, if anyone is around in London drop me a line and I will tell you if we have any editathons going on. Editathons have many roles in the community, one being as a focus group on the user interface.

Regards

 

Jonathan Cardy

 


On 25 Sep 2014, at 10:09, Kerry Raymond <[hidden email]> wrote:

This pretty much matches my own observations.

 

I think it’s not uncommon for new editors to first write about something they know well, which often means some level of conflict of interest. But that doesn’t necessarily mean the person is writing lies or being completely over-the-top with lavish praise; they can write quite reasonable content too. And generally new editors don’t know that CoI editing isn’t welcome. Similarly a lot get blocked for 3 reverts and get accused of edit warring, but I suspect it is often not the case. I suspect often they look back at the article and can’t see the edit they did, so they just do it again because it didn’t “stick”. I am not convinced they do know that other people are reverting their edits because I don’t think they know about the article history and the edit summaries and the user talk pages.

 

We make some awfully big assumptions that new editors know what we do; it’s amazing how much stuff in a UI you mentally filter out. I did a talk about Wikipedia about a year ago for a group of librarians. They asked for some “tips and tricks” to be included, so for the first time I systematically scanned the article UI and found cute little features I never knew were there (even on menus I often used). And of course the UI changes over time and probably I didn’t notice new things appear. But not noticing is consistent with “mission focus”, which has been studied in retail settings. A customer “on a mission” (a specific transaction that must be dealt with, as opposed to a customer “just browsing”) is oblivious to promotions until the “mission” is complete and then becomes aware of their surroundings as they turn to leave the store (which tells you where to put promotional signs relative to your “mission” products). I think Wikipedia editing is very mission-focussed (I must fix that error) so we see nothing but what we want to see on the way in and then have no “exit screen” to talk to the editor as they leave (unlike a bricks-and-mortar retailer). It’s hard to communicate with new editors on-wiki.

 

I actually think doing a user experience experiment (ideally with eye tracking) with new users would be very informative. Given them a “mission” and watch what they look at, what they do and if they succeed in their task, and whether their edits survives over time (and if it doesn’t survive, why is that). Indeed, seeing what they do when their edit disappears would be interesting too.

 

It is difficult for many of us as experienced Wikipedians to see Wikipedia as the new contributor sees it. I have some sense of the new editor experience because I do edit training and it certainly shows why nothing is fool-proof because fools are so ingenious. It’s amazing how new editors dive into parts of the UI I’ve never even noticed and then tie themselves up in knots. They often get so interested in Preview that they never Save (and then suddenly oops, all their work is gone). They generate continual edit conflicts (often entirely on their own, I think by going back in the browser and editing the article in multiple browser windows, leading to their contributions being smeared across a number of edit windows, none of which can be saved, or so it seems). And if they are editing in mainspace, they panic if the article changes in some mysterious way and they don’t know how they did it (it was another concurrent editor). In fact, if there is one thing I would like is for the watchlist notification to be slower than normal when a new editor is involved as having someone alerted by the watchlist coming and “fixing” things (even if it’s well-meant) puts the new editor into a spin because they don’t understand what’s going on (they have no experience with collaborative document editing). And, a perennial favourite, why is the Reference section empty when they’ve added all those citations (they don’t expect them to be in-line, they expect them in the References section). When I look at the constant questions and problems I have to deal with in edit training, I am not surprised that people trying to learn on their own give up. Seen through the eyes of the beginner, it’s so much harder than we realise both technically and collaboratively.

 

Kerry

 


From: [hidden email] [[hidden email]] On Behalf Of WereSpielChequers
Sent: Thursday, 25 September 2014 2:18 PM
To: Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] FW: What works for increasing editorengagement?

 

Hi, if we are analysing new editors creating new articles then there are two groups I would expect to see, individual spammers and corporate spammers. 

The individual spammers are the kids writing about the band that will be the next big thing on the ....... Scene in their town, and their first gig is next Tuesday provided they can recruit a drummer.

 

The corporate spammers are writing about some business, but in a style that makes a corporate flyer look neutral. 

 

 

There are some implicit assumptions that we make as new page patrollers, and which some research might help by proving or disproving.

 

1 people whose first article is an attack page or vandalism will very rarely turn into productive editors. I have met or heard of three former vandals who made good editors, there may be more, but I've never heard of a good editor who started out creating attack pages.

 

2 people who are paid to spam this site are unlikely to turn into good editors, but a large proportion of good editors made COI edits among their early edits. The classic wikipedian is a time rich altruist with Internet access that they can make personal use of, I'm not seeing much overlap there with corporate spammers, but people who make COI edits other than about their employment clearly have personal access to the Internet.

 

3 neutral point of view is something that new editors need to pick up, and POV editing amongst a new editor's early contributions is so common that if a new account writes neutrally they are  assumed by some to be sock puppets or returning editors.

 

4 we live in a copy paste world, and while a student plagiarising to try to get better marks is clearly doing so in bad faith, a wikipedia contributor using copy paste is still a good faith contributor, they just need to be taught not to use copy and paste. Presumably these people are in Aaron's group 3, again we have a lot of former copyright violators in the community, and a lot of new articles get deleted as copyright violations.

 

My suspicion is that much of our tolerance of vandals is wasted effort. We would save a lot of time and not lose any good editors by moving from the current five strikes and you are out approach to vandals to one of blocking any new editor after their first edit if that edit was blatant vandalism. Copyvio by contrast is something where we could probably retain more of the editors by trying different approaches to explaining why their edits had to be rejected and how they could contribute.


Regards

 

Jonathan Cardy

 


On 25 Sep 2014, at 00:19, Aaron Halfaker <[hidden email]> wrote:

Sure!  You'll find the hand-coded set of users here within the next half hour (cron job copies datasets over).  

 

Categories: 

  1. Vandals - Purposefully malicious, out to cause harm
  2. Bad-faith - Trying to be funny, not here to help or harm
  3. Good-faith - Trying to be productive, but failing
  4. Golden - Successfully contributing productively

-Aaron

 

On Wed, Sep 24, 2014 at 1:15 PM, James Salsman <[hidden email]> wrote:

Hi Aaron,

 

available for correlation with the number of new articles each user created?


Aaron Halfaker <[hidden email]> wrote:
>...

> I propose a project where we work together to generate 

> a summary so that I can call it "work." I've started a stub here:


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

 

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editor engagement?

Scott Hale
In reply to this post by Luca de Alfaro-5
On Fri, Sep 26, 2014 at 5:14 AM, Luca de Alfaro <[hidden email]> wrote:
Better merging would be welcome.  But also less aggressive editing/policing. 

When I edit openstreetmap I have a better overall experience: the edits may or may not go live immediately, but I don't have the impression that there is someone aggressively vetting/refining my edits while I am still doing them.  I feel welcome there. 

To make Wikipedia more welcoming, we could do a few things. 

We could allow users to save drafts.  In this way, people could work for a while at their own pace, and then publish the changes.  Currently, saving is the only way to avoid risking losing changes, but it has the very undesired effect of inviting editors/vetters to the page before one is really done. 

We could also allow a time window (even 30 minutes) before edits went live after one is done editing (using above Ajax mechanism to track when editor open), experienced editors would not need to swoop in quite so fast on the work of new users, and the whole editing atmosphere would be more relaxed and welcoming. 

The fact is that the Wikipedia editor, with its lack of ability to save drafts, poor merging, and swooping editors, feels incredibly outdated and unwelcoming - downright aggressive - to anyone used to WordPress / Google Docs / Blogger / ...

Luca

 

The technology exists to do this---[[:en:Wikipedia:Flagged_revisions]]. The challenge is that many existing users don't want flagged revisions on by default.

And that is the fundamental flaw with this whole email thread. The question needing to be answered isn't "what increases new user retention". The real question is "what increases new user retention and is acceptable to the most active/helpful existing users". The second question is much harder than the first.





_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: FW: What works for increasing editor engagement?

Luca de Alfaro-5
Flagged revisions is different though, as it requires "trusted" editors to flag things as approved.  I am simply advocating the ability to save drafts visible only to oneself before "publishing" a change.  WordPress, Blogger, etc have it.  And so newcomers could edit to their heart content, without triggering the interest of editors and the consequent conflicts, then save their changes.

Luca



On Thu, Sep 25, 2014 at 5:15 PM, Scott Hale <[hidden email]> wrote:
On Fri, Sep 26, 2014 at 5:14 AM, Luca de Alfaro <[hidden email]> wrote:
Better merging would be welcome.  But also less aggressive editing/policing. 

When I edit openstreetmap I have a better overall experience: the edits may or may not go live immediately, but I don't have the impression that there is someone aggressively vetting/refining my edits while I am still doing them.  I feel welcome there. 

To make Wikipedia more welcoming, we could do a few things. 

We could allow users to save drafts.  In this way, people could work for a while at their own pace, and then publish the changes.  Currently, saving is the only way to avoid risking losing changes, but it has the very undesired effect of inviting editors/vetters to the page before one is really done. 

We could also allow a time window (even 30 minutes) before edits went live after one is done editing (using above Ajax mechanism to track when editor open), experienced editors would not need to swoop in quite so fast on the work of new users, and the whole editing atmosphere would be more relaxed and welcoming. 

The fact is that the Wikipedia editor, with its lack of ability to save drafts, poor merging, and swooping editors, feels incredibly outdated and unwelcoming - downright aggressive - to anyone used to WordPress / Google Docs / Blogger / ...

Luca

 

The technology exists to do this---[[:en:Wikipedia:Flagged_revisions]]. The challenge is that many existing users don't want flagged revisions on by default.

And that is the fundamental flaw with this whole email thread. The question needing to be answered isn't "what increases new user retention". The real question is "what increases new user retention and is acceptable to the most active/helpful existing users". The second question is much harder than the first.






_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
12