Reflection and research: User surveys

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Reflection and research: User surveys

erikzachte
SJ:

> With a brief discussion about preserving privacy in aggregate data,
> randomizing test and control samples, and a tweak to allow web forms
> on pages that are aware of your wikipedia userid, we could have a
> simple projects-wide survey completed within a month.  Let's make this
> a priority and make such a thing happen -- then figure out how to
> optimize future iterations.
>
> The latest discussions on meta are here:
> http://meta.wikimedia.org/wiki/General_User_Survey
>

SJ, great to hear you welcome the survey. After Wikimania 2005 the project
fell idle,
because I had too many other WM obligations and a not so good winter
healthwise.
Wikimania 2006 gave the project new elan and now someone else will code it.

Status:
Technical design has started but needs some more work:
I'll make a mockup input script for the form generator.
http://meta.wikimedia.org/wiki/General_User_Survey/Implementation
Programming will start in a reasonable time frame, see Kevin's earlier post.

I'm not so sure this takes only one month :(

Major issues:
1 Authentication
  Best after single login is active, in a few weeks time?
2 Anonimisation of results
  May need some more thinking, this is sensitive matter
  We had a heated debate about this in Frankfurt, we'll probably get into
this further
  when we have a proof of concept, and more people show up to give feedback.
3 Translation issues
  A Mediawiki wide survey needs to be held in many languages to reduce bias
where opinions are asked.
4 Results should be script-processable, e.g. no free format feedback.
  Thus all answers should be on a numeric scale or predefined
  (e.g. country numbers instead of country names in all esoteric languages
that no script can handle)

Because of 3 a survey form needs to be built dynamically.
No English/German/Japanese/etc texts intermingled with PHP script.
That would be a maintenance nightmare.

Depending on how much time the programmer can spend on the project, we could
probably show an alpha version about 4 weeks after he starts.
Then start major discussion on final questions (this will work better when
people see a alpha version to play with),
and finally freeze questions and invite translators.

I would be happy if we did a major survey in November/December.

Please don't ask for quick hacks. I know all this sounds like an invitation
for some self-proclaimed code magician to make something barely functional
in a weekend, pronounce the job done and then leave the 'dirty details'
(usually 80% of what needs to be done) for others to clean up. I'd rather
see to it that the first version is usable and a good platform for future
reuse and extension.

Erik Zachte :)



_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://mail.wikipedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Reflection and research: User surveys

SJ-5
On 9/21/06, Erik Zachte <[hidden email]> wrote:

> SJ:
> > With a brief discussion about preserving privacy in aggregate data,
> > randomizing test and control samples, and a tweak to allow web forms
> > on pages that are aware of your wikipedia userid, we could have a
> > simple projects-wide survey completed within a month.  Let's make this
> > a priority and make such a thing happen -- then figure out how to
> > optimize future iterations.
> >
> > The latest discussions on meta are here:
> > http://meta.wikimedia.org/wiki/General_User_Survey
> >
>
> SJ, great to hear you welcome the survey. After Wikimania 2005 the project
> fell idle,
> because I had too many other WM obligations and a not so good winter
> healthwise.
> Wikimania 2006 gave the project new elan and now someone else will code it.
>
> Status:
> Technical design has started but needs some more work:
> I'll make a mockup input script for the form generator.
> http://meta.wikimedia.org/wiki/General_User_Survey/Implementation
> Programming will start in a reasonable time frame, see Kevin's earlier post.
>
> I'm not so sure this takes only one month :(

This depends on how many people are working on it.   Have you heard
further from Elan Pavlov?

> 3 Translation issues
>   A Mediawiki wide survey needs to be held in many languages to reduce bias
> where opinions are asked.

> Because of 3 a survey form needs to be built dynamically.
> No English/German/Japanese/etc texts intermingled with PHP script.
> That would be a maintenance nightmare.

Quite right.  Perhaps the initial survey can be a single page and
concise, minimizing the cost of this step  (It would still be an
enjoyable 40-language project)


> Depending on how much time the programmer can spend on the project, we could
> probably show an alpha version about 4 weeks after he starts.

Could this be coded by more than one person?

> Please don't ask for quick hacks. I know all this sounds like an invitation
> for some self-proclaimed code magician to make something barely functional
> in a weekend, pronounce the job done and then leave the 'dirty details'
> (usually 80% of what needs to be done) for others to clean up. I'd rather
> see to it that the first version is usable and a good platform for future
> reuse and extension.

:-)  I'm thinking of 'quick' in terms of a very simple interface --
not something people need to see and play with to be able to guess at
useful qustions! -- and a short set of basic Q's (as above, to
minimize translation as well).   A second pass not with more code
functionality, but with more attention to the layout of the form,
distribution of questions, promotion of the effort to minimize bias,
&c.

SJ
_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://mail.wikipedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: Reflection and research: User surveys

Kevin Gamble-2
I will start passing some of these things on to the programmer so he is at
least beginning to give it some serious thought. I am sure he would
welcome others to the effort.

As I mentioned to Erik (and others) this person is booked solid through
the end of September, and as with most coding efforts it's running late.
I'm guessing that serious work may be delayed a couple of weeks into
October.

Kevin


> On 9/21/06, Erik Zachte <[hidden email]> wrote:
>> SJ:
>> > With a brief discussion about preserving privacy in aggregate data,
>> > randomizing test and control samples, and a tweak to allow web forms
>> > on pages that are aware of your wikipedia userid, we could have a
>> > simple projects-wide survey completed within a month.  Let's make this
>> > a priority and make such a thing happen -- then figure out how to
>> > optimize future iterations.
>> >
>> > The latest discussions on meta are here:
>> > http://meta.wikimedia.org/wiki/General_User_Survey
>> >
>>
>> SJ, great to hear you welcome the survey. After Wikimania 2005 the
>> project
>> fell idle,
>> because I had too many other WM obligations and a not so good winter
>> healthwise.
>> Wikimania 2006 gave the project new elan and now someone else will code
>> it.
>>
>> Status:
>> Technical design has started but needs some more work:
>> I'll make a mockup input script for the form generator.
>> http://meta.wikimedia.org/wiki/General_User_Survey/Implementation
>> Programming will start in a reasonable time frame, see Kevin's earlier
>> post.
>>
>> I'm not so sure this takes only one month :(
>
> This depends on how many people are working on it.   Have you heard
> further from Elan Pavlov?
>
>> 3 Translation issues
>>   A Mediawiki wide survey needs to be held in many languages to reduce
>> bias
>> where opinions are asked.
>
>> Because of 3 a survey form needs to be built dynamically.
>> No English/German/Japanese/etc texts intermingled with PHP script.
>> That would be a maintenance nightmare.
>
> Quite right.  Perhaps the initial survey can be a single page and
> concise, minimizing the cost of this step  (It would still be an
> enjoyable 40-language project)
>
>
>> Depending on how much time the programmer can spend on the project, we
>> could
>> probably show an alpha version about 4 weeks after he starts.
>
> Could this be coded by more than one person?
>
>> Please don't ask for quick hacks. I know all this sounds like an
>> invitation
>> for some self-proclaimed code magician to make something barely
>> functional
>> in a weekend, pronounce the job done and then leave the 'dirty details'
>> (usually 80% of what needs to be done) for others to clean up. I'd
>> rather
>> see to it that the first version is usable and a good platform for
>> future
>> reuse and extension.
>
> :-)  I'm thinking of 'quick' in terms of a very simple interface --
> not something people need to see and play with to be able to guess at
> useful qustions! -- and a short set of basic Q's (as above, to
> minimize translation as well).   A second pass not with more code
> functionality, but with more attention to the layout of the form,
> distribution of questions, promotion of the effort to minimize bias,
> &c.
>
> SJ
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> http://mail.wikipedia.org/mailman/listinfo/wiki-research-l
>


_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://mail.wikipedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

General User Survey - proposal '$'

Piotr Konieczny-2
Hello everyone,

Maybe I am beating a dead horse, but I'd really like to see GUS
(http://meta.wikimedia.org/wiki/GUS) done. Now, I am somewhat tired with
waiting for the promised help - 6 months delay (or 18...) in my books
means something is not going to get done. But I have a solution: instead
of waiting for some developer to get interested in our case and write
the script(s) for us, couldn't we apply to Wikimedia Fundation for some
small grant to hire somebody and write whaterver we cannot write
ourselves? It seems that we are now dealing with millions of $ in our
budget, surely a few hundreds (if we even need so much) can be spared
for such an important issue as finalizing GUS-related works?

--
Piotr Konieczny aka Prokonsul Piotrus

"Be seeing you, Commander."
        -- Bester to Sinclair in Babylon 5:"Mind War"

_______________________________________________
Wiki-research-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/wiki-research-l