Hello, and the short term plans

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Hello, and the short term plans

Marc-Andre
<futurama>Good news, everyone!</futurama>

As many of you know, I officially started my duties today as the WMF
Operations Engineer attached to the Tool Labs.  I intend to make a point
of informing all of you of recent news, what I'm working on, and where
I'm headed at regular intervals (probably weekly).

First, a bit of news:  I have had confirmation this weekend that the DB
replication made available to Tool Labs users will, in fact, allow the
creation of databases alongside the project ones.  This means that one
of the use cases that seemed the most troublesome in the transition
(joins between the WMF databases and tool-specific ones) will be fully
supported.

We are making good strides in documenting the *impressive* inventory of
tools that run on toolserver and their requirements (thanks, Silke!).  
The list-in-progress can be found at [1].  If you see missing or
incorrect information, please feel free to adjust it -- the more
precisely we know the requirements, the faster we can see about meeting
them.

I've started documenting my preleminary design for the shiny new Tool
Labs infrastructure at [2].  This is a living document, and will see a
great deal of revision before it's over (and will serve as the seed for
the documentation).  I will shortly create a new Labs project where that
architecture is deployed in preproduction so we can shake out the
kinks.  The existing projects, "bots" and "webtools" will be left active
for the forseeable future until (a) the new architecture has proven
itself and (b) every user has sucessfuly moved their tools to it.

I'm planning on having the new project be fully operational for new
tools by the time the Amsterdam Hackathon takes place at the end of May
at the very least.

For the next week, I'll be mostly in information-gathering mode, as well
as refining the design and requirements of the Tool Labs.  Feel free to
poke me for information (or /with/ information) by email or on IRC
(where I am user 'Coren' and idle on #wikimedia-labs and
#wikimedia-toolserve at the very least)

-- Marc A. Pelletier

[1] http://www.mediawiki.org/wiki/Toolserver/List_of_Tools
[2] http://www.mediawiki.org/wiki/Wikimedia_Labs/Tool_Labs/Design



_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Hello, and the short term plans

Maarten Dammers
Hi Marc,

Why start from scratch again? Why not just migrate the current
Toolserver setup to the WMF cloud infrastructure?

Maarten

Op 25-2-2013 18:52, Marc A. Pelletier schreef:

> <futurama>Good news, everyone!</futurama>
>
> As many of you know, I officially started my duties today as the WMF
> Operations Engineer attached to the Tool Labs.  I intend to make a
> point of informing all of you of recent news, what I'm working on, and
> where I'm headed at regular intervals (probably weekly).
>
> First, a bit of news:  I have had confirmation this weekend that the
> DB replication made available to Tool Labs users will, in fact, allow
> the creation of databases alongside the project ones. This means that
> one of the use cases that seemed the most troublesome in the
> transition (joins between the WMF databases and tool-specific ones)
> will be fully supported.
>
> We are making good strides in documenting the *impressive* inventory
> of tools that run on toolserver and their requirements (thanks,
> Silke!).  The list-in-progress can be found at [1].  If you see
> missing or incorrect information, please feel free to adjust it -- the
> more precisely we know the requirements, the faster we can see about
> meeting them.
>
> I've started documenting my preleminary design for the shiny new Tool
> Labs infrastructure at [2].  This is a living document, and will see a
> great deal of revision before it's over (and will serve as the seed
> for the documentation).  I will shortly create a new Labs project
> where that architecture is deployed in preproduction so we can shake
> out the kinks.  The existing projects, "bots" and "webtools" will be
> left active for the forseeable future until (a) the new architecture
> has proven itself and (b) every user has sucessfuly moved their tools
> to it.
>
> I'm planning on having the new project be fully operational for new
> tools by the time the Amsterdam Hackathon takes place at the end of
> May at the very least.
>
> For the next week, I'll be mostly in information-gathering mode, as
> well as refining the design and requirements of the Tool Labs. Feel
> free to poke me for information (or /with/ information) by email or on
> IRC (where I am user 'Coren' and idle on #wikimedia-labs and
> #wikimedia-toolserve at the very least)
>
> -- Marc A. Pelletier
>
> [1] http://www.mediawiki.org/wiki/Toolserver/List_of_Tools
> [2] http://www.mediawiki.org/wiki/Wikimedia_Labs/Tool_Labs/Design
>
>
>
> _______________________________________________
> Toolserver-l mailing list ([hidden email])
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette



_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Hello, and the short term plans

Marc-Andre
On 02/25/2013 05:30 PM, Maarten Dammers wrote:
> Why start from scratch again? Why not just migrate the current
> Toolserver setup to the WMF cloud infrastructure?

There are a number of obstacles to this, not least of which is the usage
of Solaris and other non-Free tools in the infrastructure.

But, more to the point, we have the opportunity to start with a clean
slate and plan ahead; something which River and Daniel never had the
chance to do.  This allows us to reexamine the context today and provide
for a more robust environment from its inception -- which is not
feasible on an environment which is in use and is depended upon by
dozens of maintainers.

All of that said, much of the dev-visible architecture is designed to be
as close to the toolserver's as possible to ease transition. I don't
believe there are any gratuitous changes; the few differences to be
found are meant to bring immediate tangible benefits to the tool
maintainers.

-- Marc


_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Hello, and the short term plans

Magnus Manske-2
In reply to this post by Maarten Dammers
It appears they are not starting "from scratch", but to set up an environment as familiar as possible for the tools, within the inevitable changes that come with such a migration. "The fewer changes needed to existing tools the better" from the Design page is a hint in that direction ;-)


On Mon, Feb 25, 2013 at 10:30 PM, Maarten Dammers <[hidden email]> wrote:
Hi Marc,

Why start from scratch again? Why not just migrate the current Toolserver setup to the WMF cloud infrastructure?

Maarten

Op 25-2-2013 18:52, Marc A. Pelletier schreef:
<futurama>Good news, everyone!</futurama>

As many of you know, I officially started my duties today as the WMF Operations Engineer attached to the Tool Labs.  I intend to make a point of informing all of you of recent news, what I'm working on, and where I'm headed at regular intervals (probably weekly).

First, a bit of news:  I have had confirmation this weekend that the DB replication made available to Tool Labs users will, in fact, allow the creation of databases alongside the project ones. This means that one of the use cases that seemed the most troublesome in the transition (joins between the WMF databases and tool-specific ones) will be fully supported.

We are making good strides in documenting the *impressive* inventory of tools that run on toolserver and their requirements (thanks, Silke!).  The list-in-progress can be found at [1].  If you see missing or incorrect information, please feel free to adjust it -- the more precisely we know the requirements, the faster we can see about meeting them.

I've started documenting my preleminary design for the shiny new Tool Labs infrastructure at [2].  This is a living document, and will see a great deal of revision before it's over (and will serve as the seed for the documentation).  I will shortly create a new Labs project where that architecture is deployed in preproduction so we can shake out the kinks.  The existing projects, "bots" and "webtools" will be left active for the forseeable future until (a) the new architecture has proven itself and (b) every user has sucessfuly moved their tools to it.

I'm planning on having the new project be fully operational for new tools by the time the Amsterdam Hackathon takes place at the end of May at the very least.

For the next week, I'll be mostly in information-gathering mode, as well as refining the design and requirements of the Tool Labs. Feel free to poke me for information (or /with/ information) by email or on IRC (where I am user 'Coren' and idle on #wikimedia-labs and #wikimedia-toolserve at the very least)

-- Marc A. Pelletier

[1] http://www.mediawiki.org/wiki/Toolserver/List_of_Tools
[2] http://www.mediawiki.org/wiki/Wikimedia_Labs/Tool_Labs/Design



_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette



_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette


_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette