Interwiki-Bots

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Interwiki-Bots

DaB.-2
Hello all,

while I was under the shower today, I got the following weird idea: We have
several interwiki-bots on the toolserver, that do more or less the same, but
by different ts-users. Some bots run old versions of the software, some stop
for unknown reason and were never restarted (properly because the ts-user
left) and some just work how they should. Often I read in the wikimedia-
projects (most time in my homewiki dewp of corse), that there are problems
with a bot (because it add a wrong interwiki-link again) and the users don't
know how to contact the bot-owner and what they should do.

So my idea: Amalgamate all interwiki-bots (in the same programming-language of
corse) into 1 multi-maintainer-project. The advantages would be, that we would
use lesser resources, the bot-maintainer could work together, they could use a
database together, it would be easier for wikimedia-project-user to contact us
(in jira for example or with a mailinglist), if something is wrong, it would
be easier to contact the bot-software-maintainer and so on.

Any thoughts about that? Good idea or a "you had too much hot water in the
shower"-idea?

Sincerly,
DaB.

--
wp-blog.de

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

Conrad Irwin

>
> So my idea: Amalgamate all interwiki-bots (in the same programming-language of
> corse) into 1 multi-maintainer-project. The advantages would be, that we would
> use lesser resources, the bot-maintainer could work together, they could use a
> database together, it would be easier for wikimedia-project-user to contact us
> (in jira for example or with a mailinglist), if something is wrong, it would
> be easier to contact the bot-software-maintainer and so on.
>
> Any thoughts about that? Good idea or a "you had too much hot water in the
> shower"-idea?
>

This has already happened for the main namespace on Wiktionary, see
http://en.wiktionary.org/wiki/User:Interwicket. Obviously it's easier
for that bot because wiktionary interwiki links are always between
identical titles.

I think it is very sensible to have "the toolserver" be responsible for
all interwiki linking on projects (including other namespaces on
wiktionary). Doing so would allow a shared database of "correct" links
to be built, particularly for articles that are known to be problematic
(hey, there could even be a web interface for reporting errors).

Whether the toolserver should be responsible for making "all" interwiki
edits is more debatable, but it would certainly be useful to create a
more central place for this to happen.

Conrad

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

Mormegil
In reply to this post by DaB.-2
2010/1/7 DaB. <[hidden email]>:
> Any thoughts about that? Good idea or a "you had too much hot water in the
> shower"-idea?

I think it is an “obvious idea”—with which I mean “great idea”. I
would _love_ to see something like that; or, at the very least,
something going in that direction (i.e. reduction of the number of
bots). The multiplication of interwiki bots is a complete waste of
everyone’s time and space, IMHO.

-- [[cs:User:Mormegil | Petr Kadlec]]

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

Victor Vasiliev
In reply to this post by DaB.-2
Keep in mind that running a multi-maintainer bot would be problematic
since local policies of many wikis (at least ru: and en:) prohibit
sharing single account between different users.

--vvv


_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

Maarten Dammers
Hi Victor,

Victor Vasiliev schreef:
> Keep in mind that running a multi-maintainer bot would be problematic
> since local policies of many wikis (at least ru: and en:) prohibit
> sharing single account between different users.
>  
I don't that rule applies to bots.
Take for example CommonsDelinker. It is run from a multi-maintainer
account and runs at about every Wikimedia wiki.

Maarten


_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

mike.lifeguard
In reply to this post by Victor Vasiliev
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10-01-07 11:23 AM, Victor Vasiliev wrote:
> Keep in mind that running a multi-maintainer bot would be problematic
> since local policies of many wikis (at least ru: and en:) prohibit
> sharing single account between different users.
>
> --vvv

If that policy prevents multi-maintainer bots, then the policy is wrong
and must be ignored. Mocking whoever came up with it would be optional.

- -Mike
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAktGImYACgkQst0AR/DaKHsm5wCdECluh+NfNrarIcdjttUe06FT
cp0AnA5EFKTZUgBZyJEhUtv6wNg4iqwC
=LM54
-----END PGP SIGNATURE-----

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

Marco Schuster-2
In reply to this post by Mormegil
On Thu, Jan 7, 2010 at 4:22 PM, Petr Kadlec <[hidden email]> wrote:
2010/1/7 DaB. <[hidden email]>:
> Any thoughts about that? Good idea or a "you had too much hot water in the
> shower"-idea?

I think it is an “obvious idea”—with which I mean “great idea”. I
would _love_ to see something like that; or, at the very least,
something going in that direction (i.e. reduction of the number of
bots). The multiplication of interwiki bots is a complete waste of
everyone’s time and space, IMHO.
Optimally, they should not exist at all and interwiki linking handled by MW itself.
Until this happens, the idea of one multi-maintainer bot doing *ALL* IW linking would be great.

Marco


--
VMSoft GbR
Nabburger Str. 15
81737 München
Geschäftsführer: Marco Schuster, Volker Hemmert
http://vmsoft-gbr.de

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

Michael Andersen-3
In reply to this post by DaB.-2
I think it would be a good idea to replace the fully automated iw-bots with a multi-maintainer-bot.

You can also do iw "manually" where you (as human) choose with iw-link to accept. Therefore it should also be possible to have bots you can run/operate "manually".

I think there are many things that are done at the different projects where it would be possible to share bots. But let's start with iw and see how it works.

So if it was because of the hot water please also take a hot shower tomorrow ;-)

:-)
MGA73

-----Oprindelig meddelelse-----
Fra: [hidden email] [mailto:[hidden email]] På vegne af DaB.
Sendt: 7. januar 2010 16:14
Til: [hidden email]
Emne: [Toolserver-l] Interwiki-Bots

Hello all,

while I was under the shower today, I got the following weird idea: We have several interwiki-bots on the toolserver, that do more or less the same, but by different ts-users. Some bots run old versions of the software, some stop for unknown reason and were never restarted (properly because the ts-user
left) and some just work how they should. Often I read in the wikimedia- projects (most time in my homewiki dewp of corse), that there are problems with a bot (because it add a wrong interwiki-link again) and the users don't know how to contact the bot-owner and what they should do.

So my idea: Amalgamate all interwiki-bots (in the same programming-language of
corse) into 1 multi-maintainer-project. The advantages would be, that we would use lesser resources, the bot-maintainer could work together, they could use a database together, it would be easier for wikimedia-project-user to contact us (in jira for example or with a mailinglist), if something is wrong, it would be easier to contact the bot-software-maintainer and so on.

Any thoughts about that? Good idea or a "you had too much hot water in the shower"-idea?

Sincerly,
DaB.

--
wp-blog.de


_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

DaB.-2
In reply to this post by DaB.-2
Hello all,

ok after I got no "stupid idea!"-responses from yours, I will create a
Multimaintainer-project for the start. The problem: How should I name it?
Interwikibot? Have we still interwiki-bots that dont't use the python-
framework? Should I better name it python-interwikibot or a complete other
name? Suggestions?

Sincerly,
DaB.

--
wp-blog.de

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

seth-35
Hi!

DaB. wrote:
> [...] I will create a Multimaintainer-project for the start. The
> problem: How should I name it? [...] Suggestions?

I would simply call it:

   klaus

prosit
seth

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

Nakor Wikipedia
In reply to this post by DaB.-2
On 01/09/2010 02:57 PM, DaB. wrote:
> Suggestions?

what about putting TS in it's name to point out it runs from the toolserver?

  hanks,

N.

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

DaB.-2
Hello,
Am Samstag 09 Januar 2010 23:18:13 schrieb Nakor:
> what about putting TS in it's name to point out it runs from the
>  toolserver?

I not spoke of the name of the bot, but of the name of the multi-maintainer-
project. For the bot-name I would suggest ts-interwikibot or something (or we
could also use a allready existing botname of a ts-user which join the
multimaintaincer-project).

Sincerly,
DaB.


--
wp-blog.de

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

Platonides
DaB. wrote:

> Hello,
> Am Samstag 09 Januar 2010 23:18:13 schrieb Nakor:
>> what about putting TS in it's name to point out it runs from the
>>  toolserver?
>
> I not spoke of the name of the bot, but of the name of the multi-maintainer-
> project. For the bot-name I would suggest ts-interwikibot or something (or we
> could also use a allready existing botname of a ts-user which join the
> multimaintaincer-project).
>
> Sincerly,
> DaB.

The project name should be the same as the bot, since both should be one.

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

KALAN-4
The name should be as simple as possible. We already have two compound
nouns (inter+wiki, tool+server), and if we try to construct a
meta-compound noun out of them, it starts to horrify everything.

My suggestion: Interbot.

This clear, sense-making and recognizable name is free on all
Wikimedia projects and is now registered to DaB.’s email.

— Kalan

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

Nicolas Dumazet
Great idea.

There is are also two interesting points which have not been mentioned.

First, having a single bot avoid "interwiki bot wars" on pages. It is
not so uncommon to have several bots editing back and forth on the
same page, simply because they dont share the same settings/run the
same software.

And second, a more personal/controversial remark:
At pywikipedia, and more generally in the bot communities on each
projects, we have regularly young power-users showing up, and trying
out bots, because, well... mostly e-fame I would say, and the
excitement of playing with a new toy.
Depending on the bot policies, those rather Python-unskilled people
can get bot flags on some wikis, and end up running tools they dont
understand and that they dont update. That hurts, in many contexts.
Perhaps you'll remember the WikiDreamer case, which had me running
around all projects. And periodically, we have other troubles, with
users not able to communicate with the projects they run their bot on,
etc, etc...


That would give us a good opportunity to clear those issues, and to
improve bot images in general by having a single entity doing those
changes.


2010/1/10 Kalan <[hidden email]>:
> The name should be as simple as possible. We already have two compound
> nouns (inter+wiki, tool+server), and if we try to construct a
> meta-compound noun out of them, it starts to horrify everything.
>
> My suggestion: Interbot.
>
> This clear, sense-making and recognizable name is free on all
> Wikimedia projects and is now registered to DaB.’s email.

I disagree. Interbot sounds very unclear to me.
But we digress: I dont see why we need to find a user name now.

Also, I dont see why we should use a single SUL account crosswiki.
Think international: it is a nice opportunity to have a bot name that
would be localized for each project. Make sure that any beginner
contributor can understand what the bot has done by giving it a local
name that he can understand.

That would give us very little extra work (just a single page listing
all usernames with a few contribs links for each wiki).



I am also very excited by the technical promises of that proposition.
* I am pretty sure that one process only would not be enough to ensure
a decent-enough refresh rate. It will also require to cycle through
the projects to make sure that all projects are treated.
* Also, the only way the have to "slice" interwiki work is to select a
portion of the source's wiki Special:Allpages that will be treated by
the bot: it might not be good enough if several processes are running
in the same time, because they will end up duplicating work
* Centralizing all work gives us several nice performance/logging opportunities


Regards,

--
Nicolas Dumazet — NicDumZ

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

Andre Engels
I'm a bit more wary about these changes. I do not currently have an
account at the toolserver; I used to have one, but I had no idea what
to do with it. Does this mean that in the future if I want to do some
interwiki botting work I will have to do it from the toolserver? How
is that supposed to be an improvement? At the moment, if I want to run
a bot, I open a screen and run it. Am I supposed to get some kind of
remote desktop connection in the future?


--
André Engels, [hidden email]

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

DaB.-2
Hello,
At Wednesday 13 January 2010 21:26:13 DaB. wrote:
> Does this mean that in the future if I want to do some
> interwiki botting work I will have to do it from the toolserver?

that's the decision of the wikimedia-projects. We can't decide for them. We
can only decide, that there will be only 1 interwiki-bot on the toolserver in
future - interwiki-bots outside the TS is not our business.

Sincerly,
DaB.

--
wp-blog.de

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

Daniel Kinzler
In reply to this post by Andre Engels
Andre Engels schrieb:
>At the moment, if I want to run
> a bot, I open a screen and run it. Am I supposed to get some kind of
> remote desktop connection in the future?

As DaB sais: we can't impose any rules on bot use on the individual projects,
and don't want to. It just seems silly to run several interwiki-bots on the
toolserver, instead of cooperating to run one.

But as to your question about "some kind of remote desktop connection": close.
you get a remote console. it's called SSH. that'S how must people use the
toolserver. what did you think?

-- daniel


_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

Andre Engels
On Wed, Jan 13, 2010 at 9:53 PM, Daniel Kinzler <[hidden email]> wrote:
> Andre Engels schrieb:
>>At the moment, if I want to run
>> a bot, I open a screen and run it. Am I supposed to get some kind of
>> remote desktop connection in the future?
>
> As DaB sais: we can't impose any rules on bot use on the individual projects,
> and don't want to.

Ok, then I'll just shut up...

> It just seems silly to run several interwiki-bots on the
> toolserver, instead of cooperating to run one.

Still, there's the matter of what it means to 'cooperate to run one'.
Would that mean that there's only a single interwiki bot process
running? That can only work if it is fast enough to go through _all_
pages on _all_ languages in a reasonable time. And even then you have
only replaced the autonomous bots. Running with hints would be hard to
get in; interactively running the bot would not be possible at all
under such a scheme. Or does it just mean using a single username? In
that case I don't see the advantage. Or is it that you want to be sure
to use a single codebase? Keeping bots up-to-date is a good idea, but
the problem it solves is not a very big one, in my opinion.

I think better results could be reached by putting the emphasis on the
"cooperating" part rather than the "running one" part. Get some
database or such where the bots notify when they have either updated a
page or found that it did not need updating, then have the bots
(except when running with hints or such) request this database before
doing interwiki on a page and skip it if its last notification was
less than so-and-so-much time ago (for example one week).

--
André Engels, [hidden email]

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Interwiki-Bots

Platonides
Andre Engels wrote:
>> It just seems silly to run several interwiki-bots on the
>> toolserver, instead of cooperating to run one.
>
> Still, there's the matter of what it means to 'cooperate to run one'.
> Would that mean that there's only a single interwiki bot process
> running? That can only work if it is fast enough to go through _all_
> pages on _all_ languages in a reasonable time.

There may be several instances, but they would be the same bot.

> And even then you have
> only replaced the autonomous bots. Running with hints would be hard to
> get in; interactively running the bot would not be possible at all
> under such a scheme.
That's a valid concern.


> I think better results could be reached by putting the emphasis on the
> "cooperating" part rather than the "running one" part. Get some
> database or such where the bots notify when they have either updated a
> page or found that it did not need updating, then have the bots
> (except when running with hints or such) request this database before
> doing interwiki on a page and skip it if its last notification was
> less than so-and-so-much time ago (for example one week).

Several cooperating instances of the same bot running from the
toolserver can do a lot of magic sharing information and using the
tables replicated at the toolserver.

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
12