Search engine indexing of userspace - has something changed?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Search engine indexing of userspace - has something changed?

Dan Garry
Hello!

In this thread
<https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(proposals)#Userpage_drafts_shown_in_search_engines>,
there was a discussion about indexing of user space by search engines. In a
nutshell, user space pages are not subject to content policies so that
users can write drafts freely, and having those pages indexed by search
engines like Google is viewed as problematic since those pages can seem
fairly official.

I seem to recall that it was not the default in the past that user pages
were indexed by search engines. I'm trying to figure out if there's some
other cause for this that's happened recently, because I'd prefer to avoid
piling hacks on and not address the root issue.

Does anyone know of anything that's changed recently that might've changed
the way that search engines index user space?

Thanks,
Dan

--
Dan Garry
Product Manager, Discovery
Wikimedia Foundation
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Search engine indexing of userspace - has something changed?

Jon Robson
Looks like a bug

Looking closely this meta tag is present in user pages:
<meta name="robots" content="noindex,follow">

but not present in that particular sub page [1].
I haven't had time to investigate further but please raise a phabricator task.

[1] https://en.m.wikipedia.org/wiki/User:Nobleeagle/India_as_an_emerging_superpower

On Mon, Jul 6, 2015 at 10:32 AM, Dan Garry <[hidden email]> wrote:

> Hello!
>
> In this thread
> <https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(proposals)#Userpage_drafts_shown_in_search_engines>,
> there was a discussion about indexing of user space by search engines. In a
> nutshell, user space pages are not subject to content policies so that
> users can write drafts freely, and having those pages indexed by search
> engines like Google is viewed as problematic since those pages can seem
> fairly official.
>
> I seem to recall that it was not the default in the past that user pages
> were indexed by search engines. I'm trying to figure out if there's some
> other cause for this that's happened recently, because I'd prefer to avoid
> piling hacks on and not address the root issue.
>
> Does anyone know of anything that's changed recently that might've changed
> the way that search engines index user space?
>
> Thanks,
> Dan
>
> --
> Dan Garry
> Product Manager, Discovery
> Wikimedia Foundation
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l



--
Jon Robson
* http://jonrobson.me.uk
* https://www.facebook.com/jonrobson
* @rakugojon

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Search engine indexing of userspace - has something changed?

Steven Walling
FWIW I have also seen many cases of userspace drafts being indexed. Perhaps
something to do with the fact that they are always subpages?

On Mon, Jul 6, 2015 at 10:35 AM Jon Robson <[hidden email]> wrote:

> Looks like a bug
>
> Looking closely this meta tag is present in user pages:
> <meta name="robots" content="noindex,follow">
>
> but not present in that particular sub page [1].
> I haven't had time to investigate further but please raise a phabricator
> task.
>
> [1]
> https://en.m.wikipedia.org/wiki/User:Nobleeagle/India_as_an_emerging_superpower
>
> On Mon, Jul 6, 2015 at 10:32 AM, Dan Garry <[hidden email]> wrote:
> > Hello!
> >
> > In this thread
> > <
> https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(proposals)#Userpage_drafts_shown_in_search_engines
> >,
> > there was a discussion about indexing of user space by search engines.
> In a
> > nutshell, user space pages are not subject to content policies so that
> > users can write drafts freely, and having those pages indexed by search
> > engines like Google is viewed as problematic since those pages can seem
> > fairly official.
> >
> > I seem to recall that it was not the default in the past that user pages
> > were indexed by search engines. I'm trying to figure out if there's some
> > other cause for this that's happened recently, because I'd prefer to
> avoid
> > piling hacks on and not address the root issue.
> >
> > Does anyone know of anything that's changed recently that might've
> changed
> > the way that search engines index user space?
> >
> > Thanks,
> > Dan
> >
> > --
> > Dan Garry
> > Product Manager, Discovery
> > Wikimedia Foundation
> > _______________________________________________
> > Wikitech-l mailing list
> > [hidden email]
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
>
>
> --
> Jon Robson
> * http://jonrobson.me.uk
> * https://www.facebook.com/jonrobson
> * @rakugojon
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Search engine indexing of userspace - has something changed?

Mormegil
In reply to this post by Jon Robson
Hi

On Mon, Jul 6, 2015 at 7:35 PM, Jon Robson <[hidden email]> wrote:

> Looking closely this meta tag is present in user pages:
> <meta name="robots" content="noindex,follow">
>
> but not present in that particular sub page [1].
> I haven't had time to investigate further but please raise a phabricator
> task.
>
> [1]
> https://en.m.wikipedia.org/wiki/User:Nobleeagle/India_as_an_emerging_superpower
>

You are not looking into userspace. The user page has been moved into the
mainspace a few days ago, and has been a redirect since. [2] The mobile
view seems not to indicate the fact you were redirected at all.

-- [[cs:User:Mormegil | Petr Kadlec]]

[2]
https://en.wikipedia.org/w/index.php?title=User:Nobleeagle/India_as_an_emerging_superpower&action=history
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l