Tool Labs update and news, oh my!

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Tool Labs update and news, oh my!

Marc-Andre
Greetings, programs!

This week's update will be brief, if optimistic:

On the news front, there have been performance and reliability issues with Gluster than are being worked on by Ryan Lane. He is experimenting with an upcoming network driver and tweaking with automount timing to improve the situation.  Along with the database replication, the shared storage is one of the key component of the infrastructure that are going to receive the most attention in the short term.

There is now a new project (named, predictably enough, "tools") that is the intended destination of the new architecture for the Tool Labs.  We have a functionnal webservice environment, as well as a workable compute cluster to support work and long running processes.  Already, a few brave souls have stepped forward to test that new environment; others are welcome to peek in or join the project with the usual "beta" caveats.

I'm not ready to open the gates entirely yet, since none of the management is currently automatized (that's a big part of my TODO for the week); but tools which have no dependencies on access to databases should already be able to be experimentally moved to the new project.

This week, I plan to concentrate on the first draft of an interface through which tool maintainers can manage their assets on the project, and do a first documentation pass regarding how to write and/or move a tool on the new project.

-- Marc

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Tool Labs update and news, oh my!

MZMcBride-2
Marc A. Pelletier wrote:
>There is now a new project (named, predictably enough, "tools") that is
>the intended destination of the new architecture for the Tool Labs.  We
>have a functional webservice environment, as well as a workable compute
>cluster to support work and long running processes.  Already, a few
>brave souls have stepped forward to test that new environment; others
>are welcome to peek in or join the project with the usual "beta" caveats.

I meant to ask this last week and it may be documented somewhere already
(if so, just reply with a link), but I'm having a little trouble wrapping
my head around Toolserver --> Tool Labs right now.

The Toolserver is a pool of shared resources, with a few roots, and a
structure that's largely single user. That is, each Toolserver user gets
his or her own directory inside /home and then works in that area.

Wikimedia Labs, as I understand it, is generally a pool of shared
resources, but the idea is to create virtual machines that each
individually have roots and only a few users per virtual machine (i.e.,
per project).

My confusion is how this "tools" project within Wikimedia Labs will be
structured.

MZMcBride



_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Tool Labs update and news, oh my!

Platonides
On 04/03/13 19:30, MZMcBride wrote:

> The Toolserver is a pool of shared resources, with a few roots, and a
> structure that's largely single user. That is, each Toolserver user gets
> his or her own directory inside /home and then works in that area.
>
> Wikimedia Labs, as I understand it, is generally a pool of shared
> resources, but the idea is to create virtual machines that each
> individually have roots and only a few users per virtual machine (i.e.,
> per project).
>
> My confusion is how this "tools" project within Wikimedia Labs will be
> structured.
>
> MZMcBride

A few machines where you install the different tools. So you wouldn't
need to manage a complete VM, or install apache for your web app. You
would place the tool in the appropiate dir and it _should_ just work.
Quite similar to how toolserver worked, but more tool-centric (as if you
created a MMP per tool, but easier).

Disclaimer: This my own conception, I may have understood everything
wrong, etc.

_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Reply | Threaded
Open this post in threaded view
|

Re: Tool Labs update and news, oh my!

Marc-Andre
On 03/04/2013 01:58 PM, Platonides wrote:
> A few machines where you install the different tools. So you wouldn't
> need to manage a complete VM, or install apache for your web app. You
> would place the tool in the appropiate dir and it_should_  just work.
> Quite similar to how toolserver worked, but more tool-centric (as if you
> created a MMP per tool, but easier).

That's a very good nutshell.

The basic outline is thus:  there is a compute grid and a webserver
cloud (and, in the future, databases and other common services) that are
managed at the project levels by the roots.  Those are made available to
all the tools.

Each "tool" is, essentially, storage, a webspace, a set of permissions,
a user, and a group.  Members of the group are the tool's maintainers
and have complete control over it.  They can queue jobs or continuous
tasks, and put what they need in their webspace to function.  They'll
also have permission over future resources (like databases, etc).

-- Marc


_______________________________________________
Toolserver-l mailing list ([hidden email])
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette