unique visitors

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

unique visitors

phoebe ayers-3
Hi all,

Can someone help me with my failing memory and remind me what the
current state of affairs is re: unique visitors -- we're not counting
them anymore? We are counting them but not via comscore? Something
else?

Just putting together a talk and wanted the latest numbers.

thanks,
-- Phoebe


--
* I use this address for lists; send personal messages to phoebe.ayers
<at> gmail.com *

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: unique visitors

Dan Andreescu
Hi Phoebe,

We can't use the comScore numbers any more because they were not tracking mobile devices.  Since a lot of our traffic is now coming from mobile devices, comScore numbers were showing a sharp decline, but this was not a reflection of reality.  We've been working on a way to measure uniques that's sensitive to privacy, and we're very close to release that data, we're just working on the blog post now.  One of the drawbacks is that we can't report on a single total number across all our projects.  We'll have an announcement shortly but until then I'll see if we can share the numbers with you privately.  Since we can't compute one total number, would you give us a list of wikis that you would like numbers for?

Dan

On Thu, Mar 17, 2016 at 12:52 PM, phoebe ayers <[hidden email]> wrote:
Hi all,

Can someone help me with my failing memory and remind me what the
current state of affairs is re: unique visitors -- we're not counting
them anymore? We are counting them but not via comscore? Something
else?

Just putting together a talk and wanted the latest numbers.

thanks,
-- Phoebe


--
* I use this address for lists; send personal messages to phoebe.ayers
<at> gmail.com *

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: unique visitors

Kevin Leduc
In reply to this post by phoebe ayers-3
FYI there was an official announcement about dropping the comScore numbers here: https://meta.wikimedia.org/wiki/ComScore/Announcement


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: unique visitors

phoebe ayers-3
In reply to this post by Dan Andreescu
On Thu, Mar 17, 2016 at 1:04 PM, Dan Andreescu <[hidden email]> wrote:
> Hi Phoebe,
>
> We can't use the comScore numbers any more because they were not tracking
> mobile devices.  Since a lot of our traffic is now coming from mobile
> devices, comScore numbers were showing a sharp decline, but this was not a
> reflection of reality.

Right, I thought this was the case but didn't know the current state
of affairs. (Missed the announcement from Kevin).

We've been working on a way to measure uniques
> that's sensitive to privacy, and we're very close to release that data,
> we're just working on the blog post now.

Awesome, that is great to hear! Congratulations!

One of the drawbacks is that we
> can't report on a single total number across all our projects.

Hmm. That's unfortunate for understanding reach -- if nothing else,
the idea that "half a billion people access Wikipedia" (eg from
earlier comscore reports) was a PR-friendly way of giving an idea of
the scale of our readership. But I can see why it would be tricky to
measure. Since this is the research list: I suspect there's still lots
to be done in understanding just how multilingual people use different
language editions of Wikipedia, too.

We'll have
> an announcement shortly but until then I'll see if we can share the numbers
> with you privately.  Since we can't compute one total number, would you give
> us a list of wikis that you would like numbers for?

I was thinking about total numbers, but for this particular project
maybe English, Spanish & Arabic -- I'm interested in global languages.
Thank you!

Phoebe

>
> Dan
>
> On Thu, Mar 17, 2016 at 12:52 PM, phoebe ayers <[hidden email]>
> wrote:
>>
>> Hi all,
>>
>> Can someone help me with my failing memory and remind me what the
>> current state of affairs is re: unique visitors -- we're not counting
>> them anymore? We are counting them but not via comscore? Something
>> else?
>>
>> Just putting together a talk and wanted the latest numbers.
>>
>> thanks,
>> -- Phoebe
>>
>>
>> --
>> * I use this address for lists; send personal messages to phoebe.ayers
>> <at> gmail.com *
>>
>> _______________________________________________
>> Wiki-research-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>



--
* I use this address for lists; send personal messages to phoebe.ayers
<at> gmail.com *

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: unique visitors

Leila Zia
​Hey Phoebe,​

On Thu, Mar 17, 2016 at 12:40 PM, phoebe ayers <[hidden email]> wrote:
On Thu, Mar 17, 2016 at 1:04 PM, Dan Andreescu <[hidden email]> wrote:

One of the drawbacks is that we
> can't report on a single total number across all our projects.

Hmm. That's unfortunate for understanding reach -- if nothing else,
the idea that "half a billion people access Wikipedia" (eg from
earlier comscore reports) was a PR-friendly way of giving an idea of
the scale of our readership. But I can see why it would be tricky to
measure. Since this is the research list: I suspect there's still lots
to be done in understanding just how multilingual people use different
language editions of Wikipedia, too.

​I'm not sure how ComScore estimated unique visitors/people, and whether they estimated unique users/people or unique devices. I'd like to point out that the current approach for counting uniques that Dan has referred to will tell you the unique device count, and not unique user/people count. There may be big differences in those numbers, for example, I access Wikipedia with at least two devices everyday and I know I'm not unique. ;) Erik Zachte and I have had some very early discussions about how to go from the unique device number to unique user number, but we are nowhere close to any solution. In the mean time, I suggest replacing unique users with unique devices in your communications.

Good luck with your presentation. :)

Best,
Leila​

 

Phoebe

>
> Dan
>
> On Thu, Mar 17, 2016 at 12:52 PM, phoebe ayers <[hidden email]>
> wrote:
>>
>> Hi all,
>>
>> Can someone help me with my failing memory and remind me what the
>> current state of affairs is re: unique visitors -- we're not counting
>> them anymore? We are counting them but not via comscore? Something
>> else?
>>
>> Just putting together a talk and wanted the latest numbers.
>>
>> thanks,
>> -- Phoebe
>>
>>
>> --
>> * I use this address for lists; send personal messages to phoebe.ayers
>> <at> gmail.com *
>>
>> _______________________________________________
>> Wiki-research-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>



--
* I use this address for lists; send personal messages to phoebe.ayers
<at> gmail.com *

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: unique visitors

Andrew Gray-3
In reply to this post by phoebe ayers-3
On 17 March 2016 at 19:40, phoebe ayers <[hidden email]> wrote:

>> One of the drawbacks is that we
>> can't report on a single total number across all our projects.
>
> Hmm. That's unfortunate for understanding reach -- if nothing else,
> the idea that "half a billion people access Wikipedia" (eg from
> earlier comscore reports) was a PR-friendly way of giving an idea of
> the scale of our readership. But I can see why it would be tricky to
> measure. Since this is the research list: I suspect there's still lots
> to be done in understanding just how multilingual people use different
> language editions of Wikipedia, too.

Building on this question a little: with the information we currently
have, is it actively *wrong* for us to keep using the "half a billion"
figure as a very rough first-order estimate? (Like Phoebe, I think I
keep trotting it out when giving talks). Do the new figures give us
reason to think it's substantially higher or lower than that, or even
not meaningfully answerable?

--
- Andrew Gray
  [hidden email]

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: unique visitors

Toby Negrin
Hi Andrew, Phoebe -- here's what the Communication department here is comfortable with:

Wikipedia and the other projects operated by the Wikimedia Foundation receive hundreds of millions of unique visitors per month

 The numbers of unique devices for enwiki are far greater than 500mm. But of course, people use more than one device and/or shared computers. There are datasets out there about average number of devices per person so we could potentially use this as a scaling factor to get a higher level of confidence but IMO the mapping from device to actual human is always going to be dicey.

For comparison, I worked at Yahoo for a long time and generally understand their tech stack -- in their 2014 annual report, Yahoo speaks to "more than 1 billion MAUs".[1] From my experience, I really don't know how they could measure this with any certainty without estimation or other statistical techniques because they have the same measurement issues that we do. Only Facebook or other sites where personalization is necessary for the site to work can report on reach without some sort of qualification.

-Toby




On Fri, Mar 18, 2016 at 4:09 AM, Andrew Gray <[hidden email]> wrote:
On 17 March 2016 at 19:40, phoebe ayers <[hidden email]> wrote:

>> One of the drawbacks is that we
>> can't report on a single total number across all our projects.
>
> Hmm. That's unfortunate for understanding reach -- if nothing else,
> the idea that "half a billion people access Wikipedia" (eg from
> earlier comscore reports) was a PR-friendly way of giving an idea of
> the scale of our readership. But I can see why it would be tricky to
> measure. Since this is the research list: I suspect there's still lots
> to be done in understanding just how multilingual people use different
> language editions of Wikipedia, too.

Building on this question a little: with the information we currently
have, is it actively *wrong* for us to keep using the "half a billion"
figure as a very rough first-order estimate? (Like Phoebe, I think I
keep trotting it out when giving talks). Do the new figures give us
reason to think it's substantially higher or lower than that, or even
not meaningfully answerable?

--
- Andrew Gray
  [hidden email]

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: unique visitors

phoebe ayers-3
On Fri, Mar 18, 2016 at 9:45 AM, Toby Negrin <[hidden email]> wrote:
> Hi Andrew, Phoebe -- here's what the Communication department here is
> comfortable with:
>
>> Wikipedia and the other projects operated by the Wikimedia Foundation
>> receive hundreds of millions of unique visitors per month
>

Thanks Toby. I'll start using that and add it to my presentation toolkit.

>  The numbers of unique devices for enwiki are far greater than 500mm. But of
> course, people use more than one device and/or shared computers. There are
> datasets out there about average number of devices per person so we could
> potentially use this as a scaling factor to get a higher level of confidence
> but IMO the mapping from device to actual human is always going to be dicey.

Sounds like a research project! Like Andrew, for communication
purposes I'm less interested in exactitude than I am in
order-of-magnitude. (The kinds of things I use these numbers for:
comparisons against the online population, against the population
reached by libraries, etc. -- all of which is deeply qualified, of
course.)

Semi-off topic thoughts about "reach" as a metric:

I wonder if there's a qualitative project somewhere in here about
*types* of use -- e.g. if I'm using WP on my phone & my work pc is
that really equivalent use? Perhaps I am using them for different
kinds of information seeking, e.g. looking up terms related to work vs
looking up info on movie stars -- does this different kind of use
matter for how we construct and present information, or count "use"?

Can we build testable hypotheses about use patterns & needs for people
who do straight device swapping (phone to tablet to pc, for the same
purposes) versus people who have devices for different purposes (i.e.
work v. personal) versus people who share devices? (Obviously, all
this goes well beyond just Wikipedia use).

I also think there's something in here about levels of access related
to language, which relates to multilingual use of Wikipedia. Someone
who speaks a language served by a Wikipedia with 100K articles can not
access Wikipedia to the same depth or level that a person who can use
English Wikipedia with 5M articles can.

In other words, though we talk about reach, not all reach is the same.
The depth to which Wikipedia reaches me -- someone with unlimited data
on multiple devices and 24/7 device access for all purposes, who reads
English well and a couple other languages poorly -- is way different
from the depth which someone with part-time access on a mobile phone
who speaks an underserved language is reached by our projects. This
may be pretty obvious, but I hadn't thought about the implications for
claiming "we reach x millions" before.

-- phoebe


> For comparison, I worked at Yahoo for a long time and generally understand
> their tech stack -- in their 2014 annual report, Yahoo speaks to "more than
> 1 billion MAUs".[1] From my experience, I really don't know how they could
> measure this with any certainty without estimation or other statistical
> techniques because they have the same measurement issues that we do. Only
> Facebook or other sites where personalization is necessary for the site to
> work can report on reach without some sort of qualification.
>
> -Toby
>
> [1]
> http://static.tumblr.com/7drgjla/386nnw4n9/yahoo_inc._2014_annual_report.pdf
>
>
>
> On Fri, Mar 18, 2016 at 4:09 AM, Andrew Gray <[hidden email]>
> wrote:
>>
>> On 17 March 2016 at 19:40, phoebe ayers <[hidden email]> wrote:
>>
>> >> One of the drawbacks is that we
>> >> can't report on a single total number across all our projects.
>> >
>> > Hmm. That's unfortunate for understanding reach -- if nothing else,
>> > the idea that "half a billion people access Wikipedia" (eg from
>> > earlier comscore reports) was a PR-friendly way of giving an idea of
>> > the scale of our readership. But I can see why it would be tricky to
>> > measure. Since this is the research list: I suspect there's still lots
>> > to be done in understanding just how multilingual people use different
>> > language editions of Wikipedia, too.
>>
>> Building on this question a little: with the information we currently
>> have, is it actively *wrong* for us to keep using the "half a billion"
>> figure as a very rough first-order estimate? (Like Phoebe, I think I
>> keep trotting it out when giving talks). Do the new figures give us
>> reason to think it's substantially higher or lower than that, or even
>> not meaningfully answerable?
>>
>> --
>> - Andrew Gray
>>   [hidden email]
>>
>> _______________________________________________
>> Wiki-research-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>



--
* I use this address for lists; send personal messages to phoebe.ayers
<at> gmail.com *

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: unique visitors

Toby Negrin
Hi Phoebe --

Allow me to direct you to the eminent statistician George Box's aphorism: "All models are wrong but some are useful"[1] :)

Right now the Reading team models our success on three axes:

Reach: how many people read our content
Engagement: how much do they read
Retention: how often do they come back

Just as metrics for games measure "fun", we think of ours as measuring "learning" but there's a lot of nuance left on the table, particularly in the areas that you highlight above.

We're slowly building up our quantitive and qualitative understanding of our Readers. Leila and team have been doing some really interesting work on Reader intention and we've made some serious strides in our qualitative research as well. Every new project and exploration adds another dimension of understanding but clearly we need to balance complexity and effort with expressive value.

-Toby


On Fri, Mar 18, 2016 at 7:45 AM, phoebe ayers <[hidden email]> wrote:
On Fri, Mar 18, 2016 at 9:45 AM, Toby Negrin <[hidden email]> wrote:
> Hi Andrew, Phoebe -- here's what the Communication department here is
> comfortable with:
>
>> Wikipedia and the other projects operated by the Wikimedia Foundation
>> receive hundreds of millions of unique visitors per month
>

Thanks Toby. I'll start using that and add it to my presentation toolkit.

>  The numbers of unique devices for enwiki are far greater than 500mm. But of
> course, people use more than one device and/or shared computers. There are
> datasets out there about average number of devices per person so we could
> potentially use this as a scaling factor to get a higher level of confidence
> but IMO the mapping from device to actual human is always going to be dicey.

Sounds like a research project! Like Andrew, for communication
purposes I'm less interested in exactitude than I am in
order-of-magnitude. (The kinds of things I use these numbers for:
comparisons against the online population, against the population
reached by libraries, etc. -- all of which is deeply qualified, of
course.)

Semi-off topic thoughts about "reach" as a metric:

I wonder if there's a qualitative project somewhere in here about
*types* of use -- e.g. if I'm using WP on my phone & my work pc is
that really equivalent use? Perhaps I am using them for different
kinds of information seeking, e.g. looking up terms related to work vs
looking up info on movie stars -- does this different kind of use
matter for how we construct and present information, or count "use"?

Can we build testable hypotheses about use patterns & needs for people
who do straight device swapping (phone to tablet to pc, for the same
purposes) versus people who have devices for different purposes (i.e.
work v. personal) versus people who share devices? (Obviously, all
this goes well beyond just Wikipedia use).

I also think there's something in here about levels of access related
to language, which relates to multilingual use of Wikipedia. Someone
who speaks a language served by a Wikipedia with 100K articles can not
access Wikipedia to the same depth or level that a person who can use
English Wikipedia with 5M articles can.

In other words, though we talk about reach, not all reach is the same.
The depth to which Wikipedia reaches me -- someone with unlimited data
on multiple devices and 24/7 device access for all purposes, who reads
English well and a couple other languages poorly -- is way different
from the depth which someone with part-time access on a mobile phone
who speaks an underserved language is reached by our projects. This
may be pretty obvious, but I hadn't thought about the implications for
claiming "we reach x millions" before.

-- phoebe


> For comparison, I worked at Yahoo for a long time and generally understand
> their tech stack -- in their 2014 annual report, Yahoo speaks to "more than
> 1 billion MAUs".[1] From my experience, I really don't know how they could
> measure this with any certainty without estimation or other statistical
> techniques because they have the same measurement issues that we do. Only
> Facebook or other sites where personalization is necessary for the site to
> work can report on reach without some sort of qualification.
>
> -Toby
>
> [1]
> http://static.tumblr.com/7drgjla/386nnw4n9/yahoo_inc._2014_annual_report.pdf
>
>
>
> On Fri, Mar 18, 2016 at 4:09 AM, Andrew Gray <[hidden email]>
> wrote:
>>
>> On 17 March 2016 at 19:40, phoebe ayers <[hidden email]> wrote:
>>
>> >> One of the drawbacks is that we
>> >> can't report on a single total number across all our projects.
>> >
>> > Hmm. That's unfortunate for understanding reach -- if nothing else,
>> > the idea that "half a billion people access Wikipedia" (eg from
>> > earlier comscore reports) was a PR-friendly way of giving an idea of
>> > the scale of our readership. But I can see why it would be tricky to
>> > measure. Since this is the research list: I suspect there's still lots
>> > to be done in understanding just how multilingual people use different
>> > language editions of Wikipedia, too.
>>
>> Building on this question a little: with the information we currently
>> have, is it actively *wrong* for us to keep using the "half a billion"
>> figure as a very rough first-order estimate? (Like Phoebe, I think I
>> keep trotting it out when giving talks). Do the new figures give us
>> reason to think it's substantially higher or lower than that, or even
>> not meaningfully answerable?
>>
>> --
>> - Andrew Gray
>>   [hidden email]
>>
>> _______________________________________________
>> Wiki-research-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>



--
* I use this address for lists; send personal messages to phoebe.ayers
<at> gmail.com *

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: unique visitors

Leila Zia

On Fri, Mar 18, 2016 at 8:44 AM, Toby Negrin <[hidden email]> wrote:

We're slowly building up our quantitive and qualitative understanding of our Readers. Leila and team have been doing some really interesting work on Reader intention and we've made some serious strides in our qualitative research as well.

​Ph​oebe, the page Toby mentioned is well documented and we are updating it as we learn more. If you are interested to get a quick sense of what we know about reader motivations, depth of information they are seeking, and their prior knowledge of the topic before reading an article, please look at the analysis section in https://meta.wikimedia.org/wiki/Research:Characterizing_Wikipedia_Reader_Behaviour/S3-English. Those results are based on a survey we ran in November 2015. We have run another larger survey in March 2016, and we are currently analyzing that survey. It will take months for us to finish this work, but we are hoping to make significant progress in this area and answer some basic but fundamental questions we don't have answers for today.

Of course, ping if you want to chat more about this research. :)

Best,
L

 

On Fri, Mar 18, 2016 at 7:45 AM, phoebe ayers <[hidden email]> wrote:
On Fri, Mar 18, 2016 at 9:45 AM, Toby Negrin <[hidden email]> wrote:
> Hi Andrew, Phoebe -- here's what the Communication department here is
> comfortable with:
>
>> Wikipedia and the other projects operated by the Wikimedia Foundation
>> receive hundreds of millions of unique visitors per month
>

Thanks Toby. I'll start using that and add it to my presentation toolkit.

>  The numbers of unique devices for enwiki are far greater than 500mm. But of
> course, people use more than one device and/or shared computers. There are
> datasets out there about average number of devices per person so we could
> potentially use this as a scaling factor to get a higher level of confidence
> but IMO the mapping from device to actual human is always going to be dicey.

Sounds like a research project! Like Andrew, for communication
purposes I'm less interested in exactitude than I am in
order-of-magnitude. (The kinds of things I use these numbers for:
comparisons against the online population, against the population
reached by libraries, etc. -- all of which is deeply qualified, of
course.)

Semi-off topic thoughts about "reach" as a metric:

I wonder if there's a qualitative project somewhere in here about
*types* of use -- e.g. if I'm using WP on my phone & my work pc is
that really equivalent use? Perhaps I am using them for different
kinds of information seeking, e.g. looking up terms related to work vs
looking up info on movie stars -- does this different kind of use
matter for how we construct and present information, or count "use"?

Can we build testable hypotheses about use patterns & needs for people
who do straight device swapping (phone to tablet to pc, for the same
purposes) versus people who have devices for different purposes (i.e.
work v. personal) versus people who share devices? (Obviously, all
this goes well beyond just Wikipedia use).

I also think there's something in here about levels of access related
to language, which relates to multilingual use of Wikipedia. Someone
who speaks a language served by a Wikipedia with 100K articles can not
access Wikipedia to the same depth or level that a person who can use
English Wikipedia with 5M articles can.

In other words, though we talk about reach, not all reach is the same.
The depth to which Wikipedia reaches me -- someone with unlimited data
on multiple devices and 24/7 device access for all purposes, who reads
English well and a couple other languages poorly -- is way different
from the depth which someone with part-time access on a mobile phone
who speaks an underserved language is reached by our projects. This
may be pretty obvious, but I hadn't thought about the implications for
claiming "we reach x millions" before.

-- phoebe


> For comparison, I worked at Yahoo for a long time and generally understand
> their tech stack -- in their 2014 annual report, Yahoo speaks to "more than
> 1 billion MAUs".[1] From my experience, I really don't know how they could
> measure this with any certainty without estimation or other statistical
> techniques because they have the same measurement issues that we do. Only
> Facebook or other sites where personalization is necessary for the site to
> work can report on reach without some sort of qualification.
>
> -Toby
>
> [1]
> http://static.tumblr.com/7drgjla/386nnw4n9/yahoo_inc._2014_annual_report.pdf
>
>
>
> On Fri, Mar 18, 2016 at 4:09 AM, Andrew Gray <[hidden email]>
> wrote:
>>
>> On 17 March 2016 at 19:40, phoebe ayers <[hidden email]> wrote:
>>
>> >> One of the drawbacks is that we
>> >> can't report on a single total number across all our projects.
>> >
>> > Hmm. That's unfortunate for understanding reach -- if nothing else,
>> > the idea that "half a billion people access Wikipedia" (eg from
>> > earlier comscore reports) was a PR-friendly way of giving an idea of
>> > the scale of our readership. But I can see why it would be tricky to
>> > measure. Since this is the research list: I suspect there's still lots
>> > to be done in understanding just how multilingual people use different
>> > language editions of Wikipedia, too.
>>
>> Building on this question a little: with the information we currently
>> have, is it actively *wrong* for us to keep using the "half a billion"
>> figure as a very rough first-order estimate? (Like Phoebe, I think I
>> keep trotting it out when giving talks). Do the new figures give us
>> reason to think it's substantially higher or lower than that, or even
>> not meaningfully answerable?
>>
>> --
>> - Andrew Gray
>>   [hidden email]
>>
>> _______________________________________________
>> Wiki-research-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>



--
* I use this address for lists; send personal messages to phoebe.ayers
<at> gmail.com *

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l



_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: unique visitors

Leila Zia
I'm sorry, I forgot to mention this: The link I shared is what we know about English Wikipedia. We have also run 2 surveys in Spanish and Persian Wikipedias that were ground-work for running a similar survey as in S3-English in those languages. It is important for us to understand if reader characteristics are different in other languages, and if so, we need to be able to characterize how they are different than English.

L

Leila Zia
Research Scientist
Wikimedia Foundation

On Fri, Mar 18, 2016 at 9:21 AM, Leila Zia <[hidden email]> wrote:

On Fri, Mar 18, 2016 at 8:44 AM, Toby Negrin <[hidden email]> wrote:

We're slowly building up our quantitive and qualitative understanding of our Readers. Leila and team have been doing some really interesting work on Reader intention and we've made some serious strides in our qualitative research as well.

​Ph​oebe, the page Toby mentioned is well documented and we are updating it as we learn more. If you are interested to get a quick sense of what we know about reader motivations, depth of information they are seeking, and their prior knowledge of the topic before reading an article, please look at the analysis section in https://meta.wikimedia.org/wiki/Research:Characterizing_Wikipedia_Reader_Behaviour/S3-English. Those results are based on a survey we ran in November 2015. We have run another larger survey in March 2016, and we are currently analyzing that survey. It will take months for us to finish this work, but we are hoping to make significant progress in this area and answer some basic but fundamental questions we don't have answers for today.

Of course, ping if you want to chat more about this research. :)

Best,
L

 

On Fri, Mar 18, 2016 at 7:45 AM, phoebe ayers <[hidden email]> wrote:
On Fri, Mar 18, 2016 at 9:45 AM, Toby Negrin <[hidden email]> wrote:
> Hi Andrew, Phoebe -- here's what the Communication department here is
> comfortable with:
>
>> Wikipedia and the other projects operated by the Wikimedia Foundation
>> receive hundreds of millions of unique visitors per month
>

Thanks Toby. I'll start using that and add it to my presentation toolkit.

>  The numbers of unique devices for enwiki are far greater than 500mm. But of
> course, people use more than one device and/or shared computers. There are
> datasets out there about average number of devices per person so we could
> potentially use this as a scaling factor to get a higher level of confidence
> but IMO the mapping from device to actual human is always going to be dicey.

Sounds like a research project! Like Andrew, for communication
purposes I'm less interested in exactitude than I am in
order-of-magnitude. (The kinds of things I use these numbers for:
comparisons against the online population, against the population
reached by libraries, etc. -- all of which is deeply qualified, of
course.)

Semi-off topic thoughts about "reach" as a metric:

I wonder if there's a qualitative project somewhere in here about
*types* of use -- e.g. if I'm using WP on my phone & my work pc is
that really equivalent use? Perhaps I am using them for different
kinds of information seeking, e.g. looking up terms related to work vs
looking up info on movie stars -- does this different kind of use
matter for how we construct and present information, or count "use"?

Can we build testable hypotheses about use patterns & needs for people
who do straight device swapping (phone to tablet to pc, for the same
purposes) versus people who have devices for different purposes (i.e.
work v. personal) versus people who share devices? (Obviously, all
this goes well beyond just Wikipedia use).

I also think there's something in here about levels of access related
to language, which relates to multilingual use of Wikipedia. Someone
who speaks a language served by a Wikipedia with 100K articles can not
access Wikipedia to the same depth or level that a person who can use
English Wikipedia with 5M articles can.

In other words, though we talk about reach, not all reach is the same.
The depth to which Wikipedia reaches me -- someone with unlimited data
on multiple devices and 24/7 device access for all purposes, who reads
English well and a couple other languages poorly -- is way different
from the depth which someone with part-time access on a mobile phone
who speaks an underserved language is reached by our projects. This
may be pretty obvious, but I hadn't thought about the implications for
claiming "we reach x millions" before.

-- phoebe


> For comparison, I worked at Yahoo for a long time and generally understand
> their tech stack -- in their 2014 annual report, Yahoo speaks to "more than
> 1 billion MAUs".[1] From my experience, I really don't know how they could
> measure this with any certainty without estimation or other statistical
> techniques because they have the same measurement issues that we do. Only
> Facebook or other sites where personalization is necessary for the site to
> work can report on reach without some sort of qualification.
>
> -Toby
>
> [1]
> http://static.tumblr.com/7drgjla/386nnw4n9/yahoo_inc._2014_annual_report.pdf
>
>
>
> On Fri, Mar 18, 2016 at 4:09 AM, Andrew Gray <[hidden email]>
> wrote:
>>
>> On 17 March 2016 at 19:40, phoebe ayers <[hidden email]> wrote:
>>
>> >> One of the drawbacks is that we
>> >> can't report on a single total number across all our projects.
>> >
>> > Hmm. That's unfortunate for understanding reach -- if nothing else,
>> > the idea that "half a billion people access Wikipedia" (eg from
>> > earlier comscore reports) was a PR-friendly way of giving an idea of
>> > the scale of our readership. But I can see why it would be tricky to
>> > measure. Since this is the research list: I suspect there's still lots
>> > to be done in understanding just how multilingual people use different
>> > language editions of Wikipedia, too.
>>
>> Building on this question a little: with the information we currently
>> have, is it actively *wrong* for us to keep using the "half a billion"
>> figure as a very rough first-order estimate? (Like Phoebe, I think I
>> keep trotting it out when giving talks). Do the new figures give us
>> reason to think it's substantially higher or lower than that, or even
>> not meaningfully answerable?
>>
>> --
>> - Andrew Gray
>>   [hidden email]
>>
>> _______________________________________________
>> Wiki-research-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>
>
> _______________________________________________
> Wiki-research-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>



--
* I use this address for lists; send personal messages to phoebe.ayers
<at> gmail.com *

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l




_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Reply | Threaded
Open this post in threaded view
|

Re: unique visitors

Mark J. Nelson
In reply to this post by phoebe ayers-3
phoebe ayers <[hidden email]> writes:

> I wonder if there's a qualitative project somewhere in here about
> *types* of use -- e.g. if I'm using WP on my phone & my work pc is
> that really equivalent use? Perhaps I am using them for different
> kinds of information seeking, e.g. looking up terms related to work vs
> looking up info on movie stars -- does this different kind of use
> matter for how we construct and present information, or count "use"?

Beyond the issue of devices, I think this is important in part because
the raw traffic counts (and reach numbers and similar) paint a very
specific story of what Wikimedia is doing and is successful at. (And
what you measure influences what you tend to optimize for.)

Specifically, a small slice of content, mainly English Wikipedia
articles on pop culture, recent news events, and U.S. politics,
contribute a disproportionate share of views. (A weekly top-25 list for
enwiki is at https://en.wikipedia.org/wiki/Wikipedia:Top_25_Report ). So
if you're measuring aggregate numbers, you're measuring mainly that
specific type of content. If the goal is really simply to reach as many
people as possible, have high page views and unique visitor counts,
etc., then this subset of articles is really the only important part of
Wikimedia's mission--- articles on, say, mathematics, don't contribute
anything to moving the needle if that's the metric.

-Mark

_______________________________________________
Wiki-research-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l