Re: [Wiki-research-l] Motivations to Contribute to Wikipedia

[James Salsman-2]

Dario Taraborelli <[hidden email]> wrote:
>
>... due to the lack of a formal policy, the RCom has never been
> in a position to grant any kind of "definitive approval" to recruit
> participants....

I appreciate that clarification, but it strictly contradicts this edit from
11 days ago:

http://meta.wikimedia.org/w/index.php?title=Research:Subject_recruitment&diff=3546474&oldid=2703471

about which Dario said, "I appreciate the documentation on the
review procedure" at
http://meta.wikimedia.org/wiki/Research_talk:Subject_recruitment

I think there are some very serious ethical issues here.  Requiring
Research Committee approval to contact editors or users was
explicitly rejected by the Research Committee:

http://meta.wikimedia.org/wiki/Research:Committee/Meetings/Meeting_2010-09-18/Log

As far as I can tell, the Research Committee has not discussed the
topic since.

I wonder what the community thinks of this new requirement.

Sincerely,
James Salsman

_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

[James Salsman-2]

Lane,

Thanks for your message:

> James: I made the edit stating the research should get approval,
> and I did that by jumping into the game and just making the edit
> based on what I read in discussion boards. I did not consider it
> to be a new requirement....

For the benefit of those who haven't clicked on the link, you edited
[[meta:Research:Subject recruitment]] to read, at the top:

"If you are doing research which involves contacting Wikimedia project
editors or users then you must first notify the Wikimedia Research
Committee by describing your project. After your project gets approval
then you may begin."

How could that not be seen as a requirement?  Do you think there is a
way to phrase it so that it would not be seen as a requirement?

Certainly this is not your fault.  As you read, Dario Taraborelli
stated on February 15, "this is a policy that we're enforcing ...
approval is required"
http://meta.wikimedia.org/w/index.php?title=Research_talk%3AFAQ&diff=3441309&oldid=3440848

And after you made that edit, Dario thanked you for it, saying, "I
appreciate the documentation on the review procedure" -- even though
the Research Committee had explicitly rejected an approval policy
requirement in September 2010, has not discussed it since, and neither
the community or the Foundation has ever endorsed any of the earlier
policy proposals.

I would not be so upset about this if I hadn't been repeatedly accused
of misconduct in failing to obtain RCom approval.

Given the ease and lack of remorse with which Dr. Taraborelli, Mr.
Walling, and Mr. Beaudette have all repeatedly lied about me while
accusing me of misconduct, I have lost all confidence in the ability
of Foundation staff to adhere to basic ethics. I intend to continue to
raise this issue until it is addressed sufficiently.

Sincerely,
James Salsman

_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

[Dario Taraborelli-3]

James,

I think I have replied consistently to your requests, both on wiki and by mail, stressing that this is the de facto standard procedure that was introduced with the creation of the RCom, pending a formal (as in voted) policy, and that the expectation is for whoever runs a survey or subject recruitment campaign to comply with this procedure. I appreciate that it implies a bit of bureaucracy but it's the best solution we can offer to help the community understand who runs a study and what for and help the researcher/investigator meet some basic requirements.

Dario

On Mar 19, 2012, at 10:06 AM, James Salsman wrote:

> Lane,
>
> Thanks for your message:
>
>> James: I made the edit stating the research should get approval,
>> and I did that by jumping into the game and just making the edit
>> based on what I read in discussion boards. I did not consider it
>> to be a new requirement....
>
> For the benefit of those who haven't clicked on the link, you edited
> [[meta:Research:Subject recruitment]] to read, at the top:
>
> "If you are doing research which involves contacting Wikimedia project
> editors or users then you must first notify the Wikimedia Research
> Committee by describing your project. After your project gets approval
> then you may begin."
>
> How could that not be seen as a requirement?  Do you think there is a
> way to phrase it so that it would not be seen as a requirement?
>
> Certainly this is not your fault.  As you read, Dario Taraborelli
> stated on February 15, "this is a policy that we're enforcing ...
> approval is required"
> http://meta.wikimedia.org/w/index.php?title=Research_talk%3AFAQ&diff=3441309&oldid=3440848
>
> And after you made that edit, Dario thanked you for it, saying, "I
> appreciate the documentation on the review procedure" -- even though
> the Research Committee had explicitly rejected an approval policy
> requirement in September 2010, has not discussed it since, and neither
> the community or the Foundation has ever endorsed any of the earlier
> policy proposals.
>
> I would not be so upset about this if I hadn't been repeatedly accused
> of misconduct in failing to obtain RCom approval.
>
> Given the ease and lack of remorse with which Dr. Taraborelli, Mr.
> Walling, and Mr. Beaudette have all repeatedly lied about me while
> accusing me of misconduct, I have lost all confidence in the ability
> of Foundation staff to adhere to basic ethics. I intend to continue to
> raise this issue until it is addressed sufficiently.
>
> Sincerely,
> James Salsman
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

[Dario Taraborelli-3]

Hi Lane,

your proposed workflow is a good description of how I would like the SR procedure to function in an ideal world. I am not myself at the forefront of SR discussions, but I'd definitely like to see a more streamlined process and a better way of signaling to participants which projects are flagged as reviewed and which aren't. Part of the discussion that we had during the last RCom meeting of the RCom was precisely focused on this issue [1].

If you want to contribute to the SR discussion, I strongly recommend you post your proposal on this page [2] so it can be seen and discussed by others. It would also probably make sense to move the entire SR discussion to a dedicated list as I suspect many wiki-research-l subscribers are not interested in following this thread. I'll also forward this to the RCom members who have been involved in SR as they will be able to make a better judgment than mine on these matters


Dario

[1] http://etherpad.wikimedia.org/RComDec2011
[2] http://meta.wikimedia.org/wiki/Research_talk:Committee/Areas_of_interest/Subject_recruitment

> Is such a flagging system already in place? If not, shall we start one?
>
> This is what I imagine is what we have consensus to do - is this how it is supposed to work?
> Researcher jumps on Wikipedia unannounced and starts recruiting for surveys
> Some Wikipedian tells the researcher to submit their project for review
> Researcher goes to landing page and completes a form for their proposal
> The proposal is posted publicly
> Any volunteer can check the proposal to see if all fields are completed
> Volunteers tag the form as being completed or incomplete - no quality review
> Completed forms eventually get reviewed by RCom according to criteria which are currently undefined
> Approved projects get a template to stick on their project page.
> Researchers must show their research page to all research recruitment candidates, who would be able to see the completed form, the flagging by a volunteer, and the approval by RCom. The approval template would also link to more information about research on Wikipedia.
> Research subjects would only be able to agree to participate in research by following instructions at the bottom of the research description form, so they would see default notices like "unflagged" or "unreviewed" if no one has checked it.

_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

[James Salsman-2]

Dario Taraborelli <[hidden email]> wrote:

> James, I think I have replied consistently to your requests, both on wiki
> and by mail....

Anyone can judge for themselves whether this is true by looking at
http://meta.wikimedia.org/wiki/Research_talk:FAQ

There you claimed that research approval was a mandatory policy, and
much more recently you thanked a third party for an edit which clearly
implies that it is strictly mandatory. But in September 2010 you
agreed with the rest of the RCom that research subject recruitment
approval should not be mandatory in favor of published guidelines
instead. And when called on the inconsistency, you wrote that approval
is not in fact mandatory. Both can not be true.

Instead of apologizing for your lie with which you attempted to impugn
my integrity, you have been trying to cover it up with rhetoric.

Is that behavior considered acceptable at the Wikimedia Foundation?

Sincerely,
James Salsman

_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

[Yaroslav M. Blanter]

Hi James,

actually, I am pretty sure we did discuss the procedure which requires
endorsement (we did not call it approval), either at the extraordinary
meeting in December (related to the survey banner story) or in the RCom
mailing list in the thread related to the same story. And this is indeed
practice established a long time ago, when we realized that uncontrolled
solicitation to fill in research forms leads to the situation when only a
small share of population is interested in filling the forms in, and they
are very quickly exhausted. The discussions are documented (though,
admittedly, they may be not documented in an optimal way).

I also strongly suggest you do not call other people liars unless you have
very strong proofs (which you do not). If you continue, I will have to als
the moderators to restore civility on the lists. Thanks for your
understanding.

Cheers
Yaroslav

On Mon, 19 Mar 2012 12:11:07 -0600, James Salsman <[hidden email]>
wrote:
> Dario Taraborelli <[hidden email]> wrote:
>
>> James, I think I have replied consistently to your requests, both on
wiki

>> and by mail....
>
> Anyone can judge for themselves whether this is true by looking at
> http://meta.wikimedia.org/wiki/Research_talk:FAQ
>
> There you claimed that research approval was a mandatory policy, and
> much more recently you thanked a third party for an edit which clearly
> implies that it is strictly mandatory. But in September 2010 you
> agreed with the rest of the RCom that research subject recruitment
> approval should not be mandatory in favor of published guidelines
> instead. And when called on the inconsistency, you wrote that approval
> is not in fact mandatory. Both can not be true.
>
> Instead of apologizing for your lie with which you attempted to impugn
> my integrity, you have been trying to cover it up with rhetoric.
>
> Is that behavior considered acceptable at the Wikimedia Foundation?
>
> Sincerely,
> James Salsman
>
> _______________________________________________
> foundation-l mailing list
> [hidden email]
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

[James Salsman-2]

Yaroslav M. Blanter <[hidden email]> wrote:

> actually, I am pretty sure we did discuss the procedure which requires
> endorsement (we did not call it approval), either at the extraordinary
> meeting in December (related to the survey banner story) or in the RCom
> mailing list in the thread related to the same story....

The discussion in the RCom mailing list archives is fairly summarized by
http://lists.wikimedia.org/pipermail/rcom-l/2011-December/000506.html
"I rather suspect that there isn't consensus on this committee to restrict
researchers in their requesting community members to complete research
questionnaires."

The subsequent etherpad minutes for the December 22, 2011 meeting,
discuss a "future subject recruitment policy" which the participants
state they do not understand how to craft.

> do not call other people liars unless you have very strong proofs....

Is there any way to interpret these two statements such that one of
them is not a lie?

"this is a policy that we're enforcing ... approval is required"
http://meta.wikimedia.org/w/index.php?title=Research_talk%3AFAQ&diff=3441309&oldid=3440848

"due to the lack of a formal policy, the RCom has never been in a
position to grant any kind of "definitive approval" to recruit
participants"
http://lists.wikimedia.org/pipermail/wiki-research-l/2012-March/001896.html

The first of those two statements was made in an effort to accuse me
of misconduct. I stand by my statements, and I am certain that I have
acted ethically.

It would be best if this issue were addressed as a mistake on the part
of those who have contradicted themselves.

Sincerely,
James Salsman

_______________________________________________
foundation-l mailing list
[hidden email]
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l