Ethnography of Wikipedia - need help designing an ethical protocol

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Ethnography of Wikipedia - need help designing an ethical protocol

R.Stuart Geiger
Greetings fellow wiki-researchers,

I'm currently a Wikipedian and a graduate student at Georgetown
University's Communication, Culture, and Technology program in
Washington, D.C.  Some of you might remember me from this summer's
Wikimania, where I presented "Conceptions and Misconceptions Academics
Hold About Wikipedia: An Ethnography of Academics for the Wikipedian
Community." Now, as I head into the final stage of my degree, I plan
to perform the reverse study: an ethnography of Wikipedia, although
hopefully for both academics and Wikipedians.

The main reason I am writing this rather lengthy message (which is
also going to other places after some feedback here) is to inform the
community about my research and hopefully gain some feedback about my
specific protocols and techniques. Most importantly, I have to get the
permission of my university's Institutional Review Board before I
begin this research, and they require that I perform a good amount of
ethical "due diligence" with the community beforehand. First, however,
let me explain what I plan to do, which will lead into issues of
informed consent, privacy, vulnerable subjects (children), and other
thorny topics that require a response from members of the community.

Participant-observation guides my methodology. This involves entering
the community as an editor and working with all of you in the course
of doing what it is that Wikipedians do, as well as asking many
questions along the way. In one sense, I will do what I have been
doing over the past four years as an editor here: editing articles,
participating in discussions and debates, and other
encyclopedia-building tasks.  However, an essential component that
distinguishes this research is the interactive and
intentionally-ignorant questioning of current community practices and
beliefs as they happen.  Particularly in disputes in which I will and
will not be involved, I plan on asking Wikipedians to explain events,
outcomes, and justifications that may seem trivial or commonsensical.
The objective is to arrive at a better understanding of the way in
which the community and the project operates on multiple levels –
which my previous research indicates is grossly misrepresented in
contemporary academic and popular culture.

The main issue with such a study is informed consent, which means
making sure that any "interventions" I make while researching do no
harm to human subjects.  If a researcher is using surveys, interviews,
or clinical trials, informed consent is usually secured with a signed
form or click-through page that generally states the participant knows
he/she is the subject of research and agrees to have their actions
made public in certain ways.  If I were only doing surveys or
interviews, then this wouldn't be an issue; nor would it matter if I
were simply observing Wikipedia but not contributing.  The issue
arises when I become an active participant in Wikipedia as a
researcher representing my university for the purpose of collecting
and publishing data.

For obvious reasons, it is incredibly difficult if each time I
entered, for example, a deletion debate, I had to get the formal
consent of everyone involved before participating.  I am told that
because of the public nature of Wikipedia's on-wiki communication, I
can get the informed consent requirement waived – if and only if I can
show an alternative way of establishing informed consent that reflects
current community practices and norms, as agreed-upon by community
leaders or representatives.  Now, the traditional anthropological
strategy would require me to go to Jimmy Wales or the Board and
negotiate with him/them about the various protocols.  However, I think
there is a better way specific to Wikipedia, and that is creating a
page in the Wikipedia namespace where we work out what protocols any
generic ethnographic researcher ought to follow.  This way, any other
ethnographers don't have to re-invent the wheel.

So this is where all of you come in, I trust.  I've created a page at
Wikipedia:Ethically researching Wikipedia. It the following tentative
guidelines/protocols that I – or any ethnographer – would follow in
order to make sure that on-wiki interventions inform participants of
my research and protect everyone involved:

1.I will recognize that as an ethnographer, I am a guest of the
Wikipedian community and the Wikimedia Foundation. As such, I will
respect any decisions made by the community, the Arbitration
Committee, or the Wikimedia Foundation regarding the way in which I
participate in the project and collect data about my experiences.

2.I will fully disclose myself as a researcher of Wikipedia on my
account's userpage and user talk page. Here, I will explain who I am,
what I am doing and why, my research protocols, ways to opt-out of
research, and University administrators or faculty members who can be
contacted if concerns arise with my research.

3.I will have a signature that shows my status as a researcher of
Wikipedia to let editors know that I am interacting with them in such
a role. This will include a link to the above research description and
my talk page. Feor example:[[User:Staeiou|Staeiou]]
<sup>[[User:Staeiou#My Research|I'm researching
Wikipedia]]</sup><sub>[[User_talk:Staeiou|Questions, concerns,
comments?]]</sub>. I will sign every contribution I make to talk or
process pages.

4.When collecting data and publishing results, I can refer to th
specific actions of editors or quote them using their username. I can
also publish information they have made public on userpage, their
edit/log history, and the results of various programs that analyse
publicly available data like Interiot's edit counter.

5.I will let editors opt-out of my research. Any editor will be free
to tell me that he or she does not wish to be a subject in my
research. If this happens, I will not communicate with him or her
further, and I will exclude from my research any existing data
specifically based on my interactions with him or her.

6.If my research leads me to communicate with Wikipedians off-wiki –
whether via e-mail, chat, in person, or other medium outside of the
public wikispace – I will use established interview-based research
protocols to establish informed consent. This means that those who
communicate with me off-wiki will be initially informed of my research
project and asked to digitally consent to such communication being
used for research purposes. I will work to mutually establish the
privacy of data collected in each situation: if the conversation can
be quoted, paraphrased, or alluded to; if the author can be attributed
by name or username; or if the entire conversation is off-the-record.

7.I will work to minimize risks to subjects by focusing on topics
directly or indirectly related to Wikipedia, encyclopedia-building,
and the community. To protect subjects, I will not discuss personally
sensitive topics, such as editors' past or current illegal behavior,
sexual behavior, medical or psychological care, and drug or alcohol
use. If editors express these or other personally sensitive topics, I
will not include them in my research.

If anyone has any modifications or additions, please let me know, or
better yet, change it yourself!  I've marked it with {{proposed}}, and
I'd like for it to get some sort of review or consensus.

Thanks for reading this long message.  I am very excited to finally be
working on this research project, and hope to hear from some of you
soon.
R. Stuart Geiger (Staeiou)

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Ethnography of Wikipedia - need help designing an ethical protocol

Matthew Flaschen
Stuart Geiger wrote:

> 6.If my research leads me to communicate with Wikipedians off-wiki –
> whether via e-mail, chat, in person, or other medium outside of the
> public wikispace – I will use established interview-based research
> protocols to establish informed consent. This means that those who
> communicate with me off-wiki will be initially informed of my research
> project and asked to digitally consent to such communication being
> used for research purposes. I will work to mutually establish the
> privacy of data collected in each situation: if the conversation can
> be quoted, paraphrased, or alluded to; if the author can be attributed
> by name or username; or if the entire conversation is off-the-record.

One thing I would add here that's important.  You may already know this,
but you didn't specifically mention it.  If you learn something about a
subject/Wikipedia editor *off-wiki*, don't disclose it on-wiki unless
unless the subject explicitly gives you permission.  People are very
sensitive about this.

It seems like your protocols are generally well-thought and reasonable.
 I wish you the best with your research.

Matthew Flaschen

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Ethnography of Wikipedia - need help designing an ethical protocol

Mayo FM-2
Hello!

I hope everybody is fine!

Thank you Stuart Geiger for sharing your thoughts. I very much sympathies with your ethical concern.

In my view something which characterize Wikipedia is the plurality of motivations in the people participating in the community. There is people participating from different types of affiliations. Some do it as individuals and for fun, others becouse their work requires to edit Wikipedia, others as part of a learning plan in a students program or for self-learning experiences... and others do it becouse developing a research from an Academic Institution. To participate in Wikipedia you don't need to say who you are or why you do it. In this regards,  in general terms in my view you are not violating any community protocol if you do not present your self as a researcher each time. I think it is  an issue of "common-sense" in which occasions to make present that you are a researchers (by the way I like very much your idea of including it at your signature) and in which ones there is not the need to force the situation to make it present.

Apart, I am trying to elaborate a question which is related with the ethical concern, but more broad: How to develop a research which could contribute to Wikipedia critical self-awareness and enrich evolving? And I am considering two aspects on this. On the one hand, finding ways for spreading the results of the research in ways that could arrive to the community and benefit it. On the other hand, how to design the research in terms of questions addressed and methodologies used that the research process would contribute to Wikipedia goals in it self.
I would appreciate very much any advise or reflection on these questions.

Of course, both questions (the ethical concern and the applicability of the research) are not exclude one to the other.

Thanks in advance. In solidarity! Mayo 

Mayo Fuster Morell
Research: http://www.onlinecreation.info
School of information - UC Berkeley (Visiting researcher until Dec 2008)
California mobile phone: 01-510-2064743
E-mail: [hidden email]
Skype: mayoneti

European University Institute - Phd Candidate
Mobile (Italian): 0039-3345440747
Mobile (Catalan): 0034-659605957
E-mail: [hidden email]



2008/11/14 Matthew Flaschen <[hidden email]>
Stuart Geiger wrote:
> 6.If my research leads me to communicate with Wikipedians off-wiki –
> whether via e-mail, chat, in person, or other medium outside of the
> public wikispace – I will use established interview-based research
> protocols to establish informed consent. This means that those who
> communicate with me off-wiki will be initially informed of my research
> project and asked to digitally consent to such communication being
> used for research purposes. I will work to mutually establish the
> privacy of data collected in each situation: if the conversation can
> be quoted, paraphrased, or alluded to; if the author can be attributed
> by name or username; or if the entire conversation is off-the-record.

One thing I would add here that's important.  You may already know this,
but you didn't specifically mention it.  If you learn something about a
subject/Wikipedia editor *off-wiki*, don't disclose it on-wiki unless
unless the subject explicitly gives you permission.  People are very
sensitive about this.

It seems like your protocols are generally well-thought and reasonable.
 I wish you the best with your research.

Matthew Flaschen

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Ethnography of Wikipedia - need help designing an ethical protocol

Joseph Reagle
In reply to this post by R.Stuart Geiger
On Friday 14 November 2008, Stuart Geiger wrote:
> specific protocols and techniques. Most importantly, I have to get the
> permission of my university's Institutional Review Board before I
> begin this research, and they require that I perform a good amount of
> ethical "due diligence" with the community beforehand.

As did I. I basically divided the problem in two categories: pre-existing public communications, and non-public communications (emails/discussions), as I explain in my "Exempt Justification Statement":

[[
This research is concerned with the development and understanding of the collaborative culture of the Wikipedia, an online encyclopedia. I will rely upon public communications which I will cite as publications. "At risk" populations are not relevant or necessary to my research though it is possible they are present in the population at large even if not perceptible as such. (Wikipedia can be edited by anyone and has been edited by thousands.) Any nonpublic communication, such as clarifications from or conversations with sources, if used, will be attributed according to the source's preferred identity (their online identity, which may or may not be personally identifiable, or a made-up identity of my choosing). No citation on my part of existing public documents can expose a source to liability or harm; citation of non-public information is very unlikely to expose sources to harm and will be governed by consent. Consequently I am requesting exemption under 4 (citation of existing documents) and 2a (non-sensitive nor harmful "interviews" (non-public communications)).  
]]

For any non-public statements, in terms of risks and procedures I wrote:

[[
For any non-public communications, the only risk to the source is that someone could find out what he or she said to me. To avoid this risk, I will keep conversation private and confidential unless he or she gives me permission to use the conversation (see below).
...
Any non-public communications will consist of oral or e-mail conversations between the source and myself. I do not expect an oral conversation to last more than an hour or email to exceed more than three or four rounds, but this is at the source's discretion. If I am not present to administer this consent form in person I will refer the source to an online copy prior to our discussion and ask for an e-mail agreement. No incentives are provided.
]]

It further helped that I was not focussing on "sensitive" issues with "at risk" populations.

You can see my discussion of this in the Methods/Ethics section(3.2) of my dissertation proposal, which addressed "questions to ask when undertaking Internet research" as compiled by the AoIR Ethics Working Committee (Ess et al. 2002).
  http://reagle.org/joseph/2006/disp/proposal.html#heading26
and my on-line consent form is still available:
  reagle.org/joseph/2006/disp/9-consent-form.html

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Ethnography of Wikipedia - need help designing an ethical protocol

Piotr Konieczny-2
In reply to this post by R.Stuart Geiger
Stuart Geiger wrote:

[snip]

A very good idea; I have posted comments on the talk page and made some
edits to the proposal
(http://en.wikipedia.org/wiki/Wikipedia:Ethically_researching_Wikipedia).
I suggest we try to continue the discussion there.


--
Piotr Konieczny

"The problem about Wikipedia is, that it just works in reality, not in
theory."

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Ethnography of Wikipedia - need help designing an ethical protocol

Andrea Forte
Although ethical research is a global concern, note that the specific
challenges associated with adhering to IRB policies regarding informed
consent, retention of data, etc, is a concern of American academics.
This is a great conversation to have, but when talking about
guidelines for all Wikipedia research, remember that researchers in
industry labs and from countries other than the US generally do not
receive federal funds from the United States government, are not bound
by the same laws, and may have other frameworks for thinking about the
ethics of online research.


On Sun, Nov 16, 2008 at 5:56 PM, Piotr Konieczny <[hidden email]> wrote:

> Stuart Geiger wrote:
>
> [snip]
>
> A very good idea; I have posted comments on the talk page and made some
> edits to the proposal
> (http://en.wikipedia.org/wiki/Wikipedia:Ethically_researching_Wikipedia).
> I suggest we try to continue the discussion there.
>
>
> --
> Piotr Konieczny
>
> "The problem about Wikipedia is, that it just works in reality, not in
> theory."
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l