Research Project on Real-Time Vandal Detection during Editing

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Research Project on Real-Time Vandal Detection during Editing

Martin Potthast-2
Hi everyone,

we [1] would like to announce a research project with the goal of studying
whether user interactions recorded at the time of editing are suitable to
predict vandalism in real time.

Should vandal editing behavior be sufficiently different from normal
editing behavior, this would allow for a number of interesting real-time
prevention techniques. For example:
- withholding confidently suspicious edits for review before publishing
them,
- a popup asking "I am not a vandal" (as in Google's "I am not a robot") to
analyze vandal reactions,
- a popup with a chat box to personally engage vandals, e.g., to help them
find other ways of stress relief or to understand them better,
- or at the very least: a new signal to improve traditional vandalism
detectors.

We have set up a laboratory environment to study editor behavior in a
realistic setting using a private mirror of Wikipedia. No editing
whatsoever is conducted on the real Wikipedia as part of our experiments,
and all test subjects of our user studies are made aware of the
experimental nature of their editing. We plan on making use of
crowdsourcing as a means to attain scale and diversity.

If you wish to participate in this study as a test subject yourself, please
get in touch. The more diversity, the more insightful the results will be.
We are also happy to collaborate and to answer all questions that may arise
in relation to the project. For example, our setup and tooling may turn out
to be useful to study other user behavior-related things without having to
actually deploy experiments within the live MediaWiki.

Best,
Martin

PS: The AICaptcha project seems most closely related. @Vinitha and Gergő:
If you wish, we can set up a Skype meeting to talk about a avenues for
collaboration.


[1] A group of students and researchers from Bauhaus-Universität Weimar (
www.webis.de) and Leipzig University (www.temir.org); project PI: Martin
Potthast.
_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Research Project on Real-Time Vandal Detection during Editing

Leila Zia
Hi Martin,

Thanks for the update. I'm very interested to learn more once you have
more to share.

I'm not sure if you're aware of the research on sockpuppet detection.
It's a different problem than what you describe here, but I would not
be surprised if learnings from each of these projects can help
another. You can keep in touch with the sockpuppet detection at
https://meta.wikimedia.org/wiki/Research:Sockpuppet_detection_in_Wikimedia_projects

Also, if you will have early results, feel free to submit them as part
of the March 11 deadline (http://wikiworkshop.org/2018/#dates) for
Wiki Workshop. It would be great to have a chance to discuss this
research more in person if you or your team will end up being in Lyon
for TWC2019.

Best,
Leila


On Sat, Jan 20, 2018 at 2:39 AM, Martin Potthast
<[hidden email]> wrote:

> Hi everyone,
>
> we [1] would like to announce a research project with the goal of studying
> whether user interactions recorded at the time of editing are suitable to
> predict vandalism in real time.
>
> Should vandal editing behavior be sufficiently different from normal
> editing behavior, this would allow for a number of interesting real-time
> prevention techniques. For example:
> - withholding confidently suspicious edits for review before publishing
> them,
> - a popup asking "I am not a vandal" (as in Google's "I am not a robot") to
> analyze vandal reactions,
> - a popup with a chat box to personally engage vandals, e.g., to help them
> find other ways of stress relief or to understand them better,
> - or at the very least: a new signal to improve traditional vandalism
> detectors.
>
> We have set up a laboratory environment to study editor behavior in a
> realistic setting using a private mirror of Wikipedia. No editing
> whatsoever is conducted on the real Wikipedia as part of our experiments,
> and all test subjects of our user studies are made aware of the
> experimental nature of their editing. We plan on making use of
> crowdsourcing as a means to attain scale and diversity.
>
> If you wish to participate in this study as a test subject yourself, please
> get in touch. The more diversity, the more insightful the results will be.
> We are also happy to collaborate and to answer all questions that may arise
> in relation to the project. For example, our setup and tooling may turn out
> to be useful to study other user behavior-related things without having to
> actually deploy experiments within the live MediaWiki.
>
> Best,
> Martin
>
> PS: The AICaptcha project seems most closely related. @Vinitha and Gergő:
> If you wish, we can set up a Skype meeting to talk about a avenues for
> collaboration.
>
>
> [1] A group of students and researchers from Bauhaus-Universität Weimar (
> www.webis.de) and Leipzig University (www.temir.org); project PI: Martin
> Potthast.
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Research Project on Real-Time Vandal Detection during Editing

Martin Potthast
Hi Leila,

thanks for the pointers! We've already been in touch with Vinitha, Gergő,
and Adam, talking about potential points of connecting the two projects. I
will look into the Sockpuppet detection project. About the workshop, I'm
not sure if we can manage to write up something by the time of the deadline.

Martin

On Mon, Feb 12, 2018 at 8:41 PM, Leila Zia <[hidden email]> wrote:

> Hi Martin,
>
> Thanks for the update. I'm very interested to learn more once you have
> more to share.
>
> I'm not sure if you're aware of the research on sockpuppet detection.
> It's a different problem than what you describe here, but I would not
> be surprised if learnings from each of these projects can help
> another. You can keep in touch with the sockpuppet detection at
> https://meta.wikimedia.org/wiki/Research:Sockpuppet_
> detection_in_Wikimedia_projects
>
> Also, if you will have early results, feel free to submit them as part
> of the March 11 deadline (http://wikiworkshop.org/2018/#dates) for
> Wiki Workshop. It would be great to have a chance to discuss this
> research more in person if you or your team will end up being in Lyon
> for TWC2019.
>
> Best,
> Leila
>
>
> On Sat, Jan 20, 2018 at 2:39 AM, Martin Potthast
> <[hidden email]> wrote:
> > Hi everyone,
> >
> > we [1] would like to announce a research project with the goal of
> studying
> > whether user interactions recorded at the time of editing are suitable to
> > predict vandalism in real time.
> >
> > Should vandal editing behavior be sufficiently different from normal
> > editing behavior, this would allow for a number of interesting real-time
> > prevention techniques. For example:
> > - withholding confidently suspicious edits for review before publishing
> > them,
> > - a popup asking "I am not a vandal" (as in Google's "I am not a robot")
> to
> > analyze vandal reactions,
> > - a popup with a chat box to personally engage vandals, e.g., to help
> them
> > find other ways of stress relief or to understand them better,
> > - or at the very least: a new signal to improve traditional vandalism
> > detectors.
> >
> > We have set up a laboratory environment to study editor behavior in a
> > realistic setting using a private mirror of Wikipedia. No editing
> > whatsoever is conducted on the real Wikipedia as part of our experiments,
> > and all test subjects of our user studies are made aware of the
> > experimental nature of their editing. We plan on making use of
> > crowdsourcing as a means to attain scale and diversity.
> >
> > If you wish to participate in this study as a test subject yourself,
> please
> > get in touch. The more diversity, the more insightful the results will
> be.
> > We are also happy to collaborate and to answer all questions that may
> arise
> > in relation to the project. For example, our setup and tooling may turn
> out
> > to be useful to study other user behavior-related things without having
> to
> > actually deploy experiments within the live MediaWiki.
> >
> > Best,
> > Martin
> >
> > PS: The AICaptcha project seems most closely related. @Vinitha and Gergő:
> > If you wish, we can set up a Skype meeting to talk about a avenues for
> > collaboration.
> >
> >
> > [1] A group of students and researchers from Bauhaus-Universität Weimar (
> > www.webis.de) and Leipzig University (www.temir.org); project PI: Martin
> > Potthast.
> > _______________________________________________
> > Wiki-research-l mailing list
> > [hidden email]
> > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>


--
Dr. Martin Potthast
Bauhaus-Universität Weimar
Digital Bauhaus Lab
Bauhausstr. 9a
99423 Weimar
Germany

+49 3643 58 3567
+49 171 809 1945

www.potthast.net
_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Research Project on Real-Time Vandal Detection during Editing

Leila Zia
On Feb 15, 2018 03:20, "Martin Potthast" <[hidden email]>
wrote:

Hi Leila,

thanks for the pointers! We've already been in touch with Vinitha, Gergő,
and Adam, talking about potential points of connecting the two projects.


Great. :)

I
will look into the Sockpuppet detection project. About the workshop, I'm
not sure if we can manage to write up something by the time of the deadline.


​You are not stranger to the workshop ;) but just to emphasize: the
submissions can be about ideas, work-in-progress, or completed research,
anywhere from 2-8 pages long. No pressure, of course. :)

Srijan is telling me it's very likely he will be there in the workshop (in
the conference for sure). If you end up joining us, we can have some f2f
chat about this as well, which is always nice. :)

Leila​



Martin

On Mon, Feb 12, 2018 at 8:41 PM, Leila Zia <[hidden email]> wrote:

> Hi Martin,
>
> Thanks for the update. I'm very interested to learn more once you have
> more to share.
>
> I'm not sure if you're aware of the research on sockpuppet detection.
> It's a different problem than what you describe here, but I would not
> be surprised if learnings from each of these projects can help
> another. You can keep in touch with the sockpuppet detection at
> https://meta.wikimedia.org/wiki/Research:Sockpuppet_
> detection_in_Wikimedia_projects
>
> Also, if you will have early results, feel free to submit them as part
> of the March 11 deadline (http://wikiworkshop.org/2018/#dates) for
> Wiki Workshop. It would be great to have a chance to discuss this
> research more in person if you or your team will end up being in Lyon
> for TWC2019.
>
> Best,
> Leila
>
>
> On Sat, Jan 20, 2018 at 2:39 AM, Martin Potthast
> <[hidden email]> wrote:
> > Hi everyone,
> >
> > we [1] would like to announce a research project with the goal of
> studying
> > whether user interactions recorded at the time of editing are suitable
to

> > predict vandalism in real time.
> >
> > Should vandal editing behavior be sufficiently different from normal
> > editing behavior, this would allow for a number of interesting real-time
> > prevention techniques. For example:
> > - withholding confidently suspicious edits for review before publishing
> > them,
> > - a popup asking "I am not a vandal" (as in Google's "I am not a robot")
> to
> > analyze vandal reactions,
> > - a popup with a chat box to personally engage vandals, e.g., to help
> them
> > find other ways of stress relief or to understand them better,
> > - or at the very least: a new signal to improve traditional vandalism
> > detectors.
> >
> > We have set up a laboratory environment to study editor behavior in a
> > realistic setting using a private mirror of Wikipedia. No editing
> > whatsoever is conducted on the real Wikipedia as part of our
experiments,

> > and all test subjects of our user studies are made aware of the
> > experimental nature of their editing. We plan on making use of
> > crowdsourcing as a means to attain scale and diversity.
> >
> > If you wish to participate in this study as a test subject yourself,
> please
> > get in touch. The more diversity, the more insightful the results will
> be.
> > We are also happy to collaborate and to answer all questions that may
> arise
> > in relation to the project. For example, our setup and tooling may turn
> out
> > to be useful to study other user behavior-related things without having
> to
> > actually deploy experiments within the live MediaWiki.
> >
> > Best,
> > Martin
> >
> > PS: The AICaptcha project seems most closely related. @Vinitha and
Gergő:
> > If you wish, we can set up a Skype meeting to talk about a avenues for
> > collaboration.
> >
> >
> > [1] A group of students and researchers from Bauhaus-Universität Weimar
(

> > www.webis.de) and Leipzig University (www.temir.org); project PI: Martin
> > Potthast.
> > _______________________________________________
> > Wiki-research-l mailing list
> > [hidden email]
> > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>


--
Dr. Martin Potthast
Bauhaus-Universität Weimar
Digital Bauhaus Lab
Bauhausstr. 9a
99423 Weimar
Germany

+49 3643 58 3567
+49 171 809 1945

www.potthast.net
_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l