Moving Mathoid to production cluster

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Moving Mathoid to production cluster

Moritz Schubotz-2
Hi,

during the last year the math extension achieved a goal defined back
in 2003. Support of MathML. In addition there is SVG support for
MathML disabled browsers. (See http://arxiv.org/abs/1404.6179 for the
details)
I would like to give Wikipedia users a chance to test this new long
awaited feature.
Therefore we would need a mathoid instance that is accessible from the
production cluster. Greg Grossmeier already created the required table
in the database. (Sorry for the "friction" connected with this
process)
Currently the MathJax team is working on a phantom.js less method to
render texvc to mathml and svg. Some days ago I have tested that it,
and it works quite well. I would appreciate a discussion with ops that
to figure out how this can be can go to production. The original idea
was to use jenkins to build the mathoid debian package. Even though
the debian package builds without any issues in the launchpad ppa repo
jenkins can not build the package. If there is a reference project
that uses jenkins to build debian packages that go to production this
would really help to figure out what is different for mathoid and why
the package building does not work even though it works on launchpad.

Best
Physikerwelt

PS: I was informed that there is a related RT that I can not access
https://rt.wikimedia.org/Ticket/Display.html?id=6077

--
Mit freundlichen Grüßen
Moritz Schubotz

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Moving Mathoid to production cluster

Kartik Mistry
On Mon, Jul 7, 2014 at 10:07 AM, Moritz Schubotz <[hidden email]> wrote:

> Currently the MathJax team is working on a phantom.js less method to
> render texvc to mathml and svg. Some days ago I have tested that it,
> and it works quite well. I would appreciate a discussion with ops that
> to figure out how this can be can go to production. The original idea
> was to use jenkins to build the mathoid debian package. Even though
> the debian package builds without any issues in the launchpad ppa repo
> jenkins can not build the package. If there is a reference project
> that uses jenkins to build debian packages that go to production this
> would really help to figure out what is different for mathoid and why
> the package building does not work even though it works on launchpad.

Congrats to MathJax team!

For Debian package, you can take a look at ContentTranslation's [1]
server component - cxserver deployment repository [2] as example.

[1] https://www.mediawiki.org/wiki/Content_translation
[2] https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/services/cxserver/deploy

If you need any help in Debian part, feel free to ping me on/offlist/IRC.

--
Kartik Mistry/કાર્તિક મિસ્ત્રી | IRC: kart_
{kartikm, 0x1f1f}.wordpress.com

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Moving Mathoid to production cluster

Chris McMahon
In reply to this post by Moritz Schubotz-2
I would really like to see this follow the standard deploy scheme:
 implement it in beta labs; then enable it for mediawiki.org and test2wiki;
then enable it on production cluster nodes.
-Chris


On Mon, Jul 7, 2014 at 3:07 AM, Moritz Schubotz <[hidden email]>
wrote:

> Hi,
>
> during the last year the math extension achieved a goal defined back
> in 2003. Support of MathML. In addition there is SVG support for
> MathML disabled browsers. (See http://arxiv.org/abs/1404.6179 for the
> details)
> I would like to give Wikipedia users a chance to test this new long
> awaited feature.
> Therefore we would need a mathoid instance that is accessible from the
> production cluster. Greg Grossmeier already created the required table
> in the database. (Sorry for the "friction" connected with this
> process)
> Currently the MathJax team is working on a phantom.js less method to
> render texvc to mathml and svg. Some days ago I have tested that it,
> and it works quite well. I would appreciate a discussion with ops that
> to figure out how this can be can go to production. The original idea
> was to use jenkins to build the mathoid debian package. Even though
> the debian package builds without any issues in the launchpad ppa repo
> jenkins can not build the package. If there is a reference project
> that uses jenkins to build debian packages that go to production this
> would really help to figure out what is different for mathoid and why
> the package building does not work even though it works on launchpad.
>
> Best
> Physikerwelt
>
> PS: I was informed that there is a related RT that I can not access
> https://rt.wikimedia.org/Ticket/Display.html?id=6077
>
> --
> Mit freundlichen Grüßen
> Moritz Schubotz
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Moving Mathoid to production cluster

Moritz Schubotz-2
Hi Chris,

me too.
How can I implement in in beta labs?

Best
Moritz

On Mon, Jul 14, 2014 at 6:35 PM, Chris McMahon <[hidden email]> wrote:

> I would really like to see this follow the standard deploy scheme:
>  implement it in beta labs; then enable it for mediawiki.org and test2wiki;
> then enable it on production cluster nodes.
> -Chris
>
>
> On Mon, Jul 7, 2014 at 3:07 AM, Moritz Schubotz <[hidden email]>
> wrote:
>
>> Hi,
>>
>> during the last year the math extension achieved a goal defined back
>> in 2003. Support of MathML. In addition there is SVG support for
>> MathML disabled browsers. (See http://arxiv.org/abs/1404.6179 for the
>> details)
>> I would like to give Wikipedia users a chance to test this new long
>> awaited feature.
>> Therefore we would need a mathoid instance that is accessible from the
>> production cluster. Greg Grossmeier already created the required table
>> in the database. (Sorry for the "friction" connected with this
>> process)
>> Currently the MathJax team is working on a phantom.js less method to
>> render texvc to mathml and svg. Some days ago I have tested that it,
>> and it works quite well. I would appreciate a discussion with ops that
>> to figure out how this can be can go to production. The original idea
>> was to use jenkins to build the mathoid debian package. Even though
>> the debian package builds without any issues in the launchpad ppa repo
>> jenkins can not build the package. If there is a reference project
>> that uses jenkins to build debian packages that go to production this
>> would really help to figure out what is different for mathoid and why
>> the package building does not work even though it works on launchpad.
>>
>> Best
>> Physikerwelt
>>
>> PS: I was informed that there is a related RT that I can not access
>> https://rt.wikimedia.org/Ticket/Display.html?id=6077
>>
>> --
>> Mit freundlichen Grüßen
>> Moritz Schubotz
>>
>> _______________________________________________
>> Wikitech-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l



--
Mit freundlichen Grüßen
Moritz Schubotz

  Telefon (Büro):  +49 30 314 22784
  Telefon (Privat):+49 30 488 27330
  E-Mail: [hidden email]
  Web: http://www.physikerwelt.de
  Skype: Schubi87
  ICQ: 200302764
  Msn: [hidden email]

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Moving Mathoid to production cluster

Chris McMahon
On Tue, Jul 15, 2014 at 1:10 AM, Moritz Schubotz <[hidden email]>
wrote:

> Hi Chris,
>
> me too.
> How can I implement in in beta labs?
>

I'd say to start by filing a bugzilla ticket for Wikimedia
Labs/deployment-prep.
Then it is a matter of registering the proper extensions and config in
puppet.
Would mathoid need a dedicated host?


>
> Best
> Moritz
>
> On Mon, Jul 14, 2014 at 6:35 PM, Chris McMahon <[hidden email]>
> wrote:
> > I would really like to see this follow the standard deploy scheme:
> >  implement it in beta labs; then enable it for mediawiki.org and
> test2wiki;
> > then enable it on production cluster nodes.
> > -Chris
> >
> >
> > On Mon, Jul 7, 2014 at 3:07 AM, Moritz Schubotz <[hidden email]>
> > wrote:
> >
> >> Hi,
> >>
> >> during the last year the math extension achieved a goal defined back
> >> in 2003. Support of MathML. In addition there is SVG support for
> >> MathML disabled browsers. (See http://arxiv.org/abs/1404.6179 for the
> >> details)
> >> I would like to give Wikipedia users a chance to test this new long
> >> awaited feature.
> >> Therefore we would need a mathoid instance that is accessible from the
> >> production cluster. Greg Grossmeier already created the required table
> >> in the database. (Sorry for the "friction" connected with this
> >> process)
> >> Currently the MathJax team is working on a phantom.js less method to
> >> render texvc to mathml and svg. Some days ago I have tested that it,
> >> and it works quite well. I would appreciate a discussion with ops that
> >> to figure out how this can be can go to production. The original idea
> >> was to use jenkins to build the mathoid debian package. Even though
> >> the debian package builds without any issues in the launchpad ppa repo
> >> jenkins can not build the package. If there is a reference project
> >> that uses jenkins to build debian packages that go to production this
> >> would really help to figure out what is different for mathoid and why
> >> the package building does not work even though it works on launchpad.
> >>
> >> Best
> >> Physikerwelt
> >>
> >> PS: I was informed that there is a related RT that I can not access
> >> https://rt.wikimedia.org/Ticket/Display.html?id=6077
> >>
> >> --
> >> Mit freundlichen Grüßen
> >> Moritz Schubotz
> >>
> >> _______________________________________________
> >> Wikitech-l mailing list
> >> [hidden email]
> >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > _______________________________________________
> > Wikitech-l mailing list
> > [hidden email]
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
>
>
> --
> Mit freundlichen Grüßen
> Moritz Schubotz
>
>   Telefon (Büro):  +49 30 314 22784
>   Telefon (Privat):+49 30 488 27330
>   E-Mail: [hidden email]
>   Web: http://www.physikerwelt.de
>   Skype: Schubi87
>   ICQ: 200302764
>   Msn: [hidden email]
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Moving Mathoid to production cluster

Moritz Schubotz-2
Hi,

the discussion stopped with the question:
How to test the new rendering on betalabs?
So, does who knows the answer?

Best
Moritz

On Tue, Jul 15, 2014 at 12:19 PM, Moritz Schubotz <[hidden email]> wrote:

> Hi Chris,
>
> By default the Math extension users the dedicated host
> mathoid.testme.wmflabs.org this  host is accessible from the beta cluster.
> A bug related to the beta cluster is available here
>
> https://bugzilla.wikimedia.org/show_bug.cgi?id=66516
>
> But I have no clue how to change the config for betalabs.
>
> Best
>
> Moritz
> Am 15.07.2014 17:06 schrieb "Chris McMahon" <[hidden email]>:
>
>> On Tue, Jul 15, 2014 at 1:10 AM, Moritz Schubotz <[hidden email]>
>> wrote:
>>
>> > Hi Chris,
>> >
>> > me too.
>> > How can I implement in in beta labs?
>> >
>>
>> I'd say to start by filing a bugzilla ticket for Wikimedia
>> Labs/deployment-prep.
>> Then it is a matter of registering the proper extensions and config in
>> puppet.
>> Would mathoid need a dedicated host?
>>
>>
>> >
>> > Best
>> > Moritz
>> >
>> > On Mon, Jul 14, 2014 at 6:35 PM, Chris McMahon <[hidden email]>
>> > wrote:
>> > > I would really like to see this follow the standard deploy scheme:
>> > >  implement it in beta labs; then enable it for mediawiki.org and
>> > test2wiki;
>> > > then enable it on production cluster nodes.
>> > > -Chris
>> > >
>> > >
>> > > On Mon, Jul 7, 2014 at 3:07 AM, Moritz Schubotz
>> > > <[hidden email]>
>> > > wrote:
>> > >
>> > >> Hi,
>> > >>
>> > >> during the last year the math extension achieved a goal defined back
>> > >> in 2003. Support of MathML. In addition there is SVG support for
>> > >> MathML disabled browsers. (See http://arxiv.org/abs/1404.6179 for the
>> > >> details)
>> > >> I would like to give Wikipedia users a chance to test this new long
>> > >> awaited feature.
>> > >> Therefore we would need a mathoid instance that is accessible from
>> > >> the
>> > >> production cluster. Greg Grossmeier already created the required
>> > >> table
>> > >> in the database. (Sorry for the "friction" connected with this
>> > >> process)
>> > >> Currently the MathJax team is working on a phantom.js less method to
>> > >> render texvc to mathml and svg. Some days ago I have tested that it,
>> > >> and it works quite well. I would appreciate a discussion with ops
>> > >> that
>> > >> to figure out how this can be can go to production. The original idea
>> > >> was to use jenkins to build the mathoid debian package. Even though
>> > >> the debian package builds without any issues in the launchpad ppa
>> > >> repo
>> > >> jenkins can not build the package. If there is a reference project
>> > >> that uses jenkins to build debian packages that go to production this
>> > >> would really help to figure out what is different for mathoid and why
>> > >> the package building does not work even though it works on launchpad.
>> > >>
>> > >> Best
>> > >> Physikerwelt
>> > >>
>> > >> PS: I was informed that there is a related RT that I can not access
>> > >> https://rt.wikimedia.org/Ticket/Display.html?id=6077
>> > >>
>> > >> --
>> > >> Mit freundlichen Grüßen
>> > >> Moritz Schubotz
>> > >>
>> > >> _______________________________________________
>> > >> Wikitech-l mailing list
>> > >> [hidden email]
>> > >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>> > > _______________________________________________
>> > > Wikitech-l mailing list
>> > > [hidden email]
>> > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>> >
>> >
>> >
>> > --
>> > Mit freundlichen Grüßen
>> > Moritz Schubotz
>> >
>> >   Telefon (Büro):  +49 30 314 22784
>> >   Telefon (Privat):+49 30 488 27330
>> >   E-Mail: [hidden email]
>> >   Web: http://www.physikerwelt.de
>> >   Skype: Schubi87
>> >   ICQ: 200302764
>> >   Msn: [hidden email]
>> >
>> > _______________________________________________
>> > Wikitech-l mailing list
>> > [hidden email]
>> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>> >
>> _______________________________________________
>> Wikitech-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Moving Mathoid to production cluster

Bryan Davis
On Wed, Jul 23, 2014 at 3:42 PM, Moritz Schubotz <[hidden email]> wrote:
> Hi,
>
> the discussion stopped with the question:
> How to test the new rendering on betalabs?
> So, does who knows the answer?

Beta uses the same puppet configuration as production to provision
hosts. You will need to get all the puppet configuration (and
associated packaging) setup in operations/puppet.git that is needed to
install and configure your application in production. Then we can
provision a new VM in the deployment-prep project (beta) and apply the
puppet role(s) needed to provision the application. I suppose there
probably is some configuration to be done in
operations/mediawiki-config.git as well to tell the beta wikis how to
talk to mathoid.

Bryan
--
Bryan Davis              Wikimedia Foundation    <[hidden email]>
[[m:User:BDavis_(WMF)]]  Sr Software Engineer            Boise, ID USA
irc: bd808                                        v:415.839.6885 x6855

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Moving Mathoid to production cluster

Moritz Schubotz-2
Hi Bryan,

thank you very much for your contribution to the Math extension and
the associated vagrant role.
I have tried to write a patch for operations/puppet
https://gerrit.wikimedia.org/r/#/c/148836/
but that's far away from finished.
A lot of configuration and server assignment has to be done.
I searched for the string parsoid in the config files and found a lot
of entries, that seem to specify how parsoid is deployed to individual
servers.
It thought it makes little sense to copy that.
I had the impression that prior to this change
http://git.wikimedia.org/blobdiff/operations%2Fmediawiki-config/173851ef628359d96fb196172a01cfcfd1da82e5/wmf-config%2FCommonSettings.php
it was possible to enable the MathML rendering mode on betalabs.
The comment
"These variables point to non-WMF servers by default."
is not correct.
$wgMathMathMLUrl = "http://mathoid.testme.wmflabs.org" is hosted in
the eqiad cluster.
But it showed that there things on the beta cluster are different from vagrant
https://bugzilla.wikimedia.org/show_bug.cgi?id=66516
describes that the MathML tags are filtered out somehow.

Best
Moritz

On Wed, Jul 23, 2014 at 7:43 PM, Bryan Davis <[hidden email]> wrote:

> On Wed, Jul 23, 2014 at 3:42 PM, Moritz Schubotz <[hidden email]> wrote:
>> Hi,
>>
>> the discussion stopped with the question:
>> How to test the new rendering on betalabs?
>> So, does who knows the answer?
>
> Beta uses the same puppet configuration as production to provision
> hosts. You will need to get all the puppet configuration (and
> associated packaging) setup in operations/puppet.git that is needed to
> install and configure your application in production. Then we can
> provision a new VM in the deployment-prep project (beta) and apply the
> puppet role(s) needed to provision the application. I suppose there
> probably is some configuration to be done in
> operations/mediawiki-config.git as well to tell the beta wikis how to
> talk to mathoid.
>
> Bryan
> --
> Bryan Davis              Wikimedia Foundation    <[hidden email]>
> [[m:User:BDavis_(WMF)]]  Sr Software Engineer            Boise, ID USA
> irc: bd808                                        v:415.839.6885 x6855
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Moving Mathoid to production cluster

Bryan Davis
On Thu, Jul 24, 2014 at 7:00 AM, Moritz Schubotz <[hidden email]> wrote:
> Hi Bryan,
>
> thank you very much for your contribution to the Math extension and
> the associated vagrant role.

:) You're welcome. Thanks for being dedicated enough to this problem
to work on it as a volunteer.

> I have tried to write a patch for operations/puppet
> https://gerrit.wikimedia.org/r/#/c/148836/
> but that's far away from finished.
> A lot of configuration and server assignment has to be done.

It looks like your patch is getting some feedback this morning. \o/

> I searched for the string parsoid in the config files and found a lot
> of entries, that seem to specify how parsoid is deployed to individual
> servers.
> It thought it makes little sense to copy that.

I'm not sure what sort of config Mathoid needs. In general, cluster
config uses feature flags like $wmgUseParsoid that are set in
InitialiseSettings.php. Then the feature flags are used in
CommonSettings.php to guard setting "LocalSettings.php" config values
for wikis where the feature flag is enabled. Both prod and beta use
the settings from this file. CommonSettings-labs.php can be used to
change settings from the defaults used in production to match beta
specific needs.

> I had the impression that prior to this change
> http://git.wikimedia.org/blobdiff/operations%2Fmediawiki-config/173851ef628359d96fb196172a01cfcfd1da82e5/wmf-config%2FCommonSettings.php
> it was possible to enable the MathML rendering mode on betalabs.
> The comment
> "These variables point to non-WMF servers by default."
> is not correct.
> $wgMathMathMLUrl = "http://mathoid.testme.wmflabs.org" is hosted in
> the eqiad cluster.

For the purposes of production configuration, something hosted in
wmflabs.org should be treated the same as something hosted at nsa.gov.
We don't want production to talk to labs ever. First there are privacy
concerns and second Labs will melt if you turn even a tiny fraction of
enwiki's traffic towards it.

> But it showed that there things on the beta cluster are different from vagrant
> https://bugzilla.wikimedia.org/show_bug.cgi?id=66516
> describes that the MathML tags are filtered out somehow.

I get MathML rendering of the formula on
http://deployment.wikimedia.beta.wmflabs.org/wiki/User:Anomie/Sandbox,
but I'm not sure what that proves or disproves in relation to your bug
reports.

Bryan
--
Bryan Davis              Wikimedia Foundation    <[hidden email]>
[[m:User:BDavis_(WMF)]]  Sr Software Engineer            Boise, ID USA
irc: bd808                                        v:415.839.6885 x6855

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Moving Mathoid to production cluster

Moritz Schubotz-2
Hi Bryan,

On Thu, Jul 24, 2014 at 1:06 PM, Bryan Davis <[hidden email]> wrote:

> On Thu, Jul 24, 2014 at 7:00 AM, Moritz Schubotz <[hidden email]> wrote:
>> Hi Bryan,
>>
>> thank you very much for your contribution to the Math extension and
>> the associated vagrant role.
>
> :) You're welcome. Thanks for being dedicated enough to this problem
> to work on it as a volunteer.
>
>> I have tried to write a patch for operations/puppet
>> https://gerrit.wikimedia.org/r/#/c/148836/
>> but that's far away from finished.
>> A lot of configuration and server assignment has to be done.
>
> It looks like your patch is getting some feedback this morning. \o/
>
>> I searched for the string parsoid in the config files and found a lot
>> of entries, that seem to specify how parsoid is deployed to individual
>> servers.
>> It thought it makes little sense to copy that.
>
> I'm not sure what sort of config Mathoid needs. In general, cluster
> config uses feature flags like $wmgUseParsoid that are set in
> InitialiseSettings.php. Then the feature flags are used in
> CommonSettings.php to guard setting "LocalSettings.php" config values
> for wikis where the feature flag is enabled. Both prod and beta use
> the settings from this file. CommonSettings-labs.php can be used to
> change settings from the defaults used in production to match beta
> specific needs.
>
There are two components. The mathoid server, that does not need any
settings at the moment, and the Math extension that has some settings.
In my comment above I was not talking of the PHP settings for mathoid,
but for settings the define on which production instances parsoid is
installed,
and how monitoring is done etc.

>> I had the impression that prior to this change
>> http://git.wikimedia.org/blobdiff/operations%2Fmediawiki-config/173851ef628359d96fb196172a01cfcfd1da82e5/wmf-config%2FCommonSettings.php
>> it was possible to enable the MathML rendering mode on betalabs.
>> The comment
>> "These variables point to non-WMF servers by default."
>> is not correct.
>> $wgMathMathMLUrl = "http://mathoid.testme.wmflabs.org" is hosted in
>> the eqiad cluster.
>
> For the purposes of production configuration, something hosted in
> wmflabs.org should be treated the same as something hosted at nsa.gov.
> We don't want production to talk to labs ever. First there are privacy
> concerns and second Labs will melt if you turn even a tiny fraction of
> enwiki's traffic towards it.
Got it.
>
>> But it showed that there things on the beta cluster are different from vagrant
>> https://bugzilla.wikimedia.org/show_bug.cgi?id=66516
>> describes that the MathML tags are filtered out somehow.
>
> I get MathML rendering of the formula on
> http://deployment.wikimedia.beta.wmflabs.org/wiki/User:Anomie/Sandbox,
> but I'm not sure what that proves or disproves in relation to your bug
> reports.
The question is where did you test that. Locally get MathML as well.
But when I tested on the beta labs cluster (with the old config that
used "http://mathoid.testme.wmflabs.org") most of the MathML elements
were filtered out.
That's the bug report. I guess the reason for that is that there is a
different configuration on the beta cluster or another extension that
interferes with the math extension in an inadequate way.
Moritz

>
> Bryan
> --
> Bryan Davis              Wikimedia Foundation    <[hidden email]>
> [[m:User:BDavis_(WMF)]]  Sr Software Engineer            Boise, ID USA
> irc: bd808                                        v:415.839.6885 x6855
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Moving Mathoid to production cluster

Moritz Schubotz-2
Hi all,

I have developed the Mathoid deployment puppet script with great
support of Mwalker and Bryan Davis as far as I could get.
https://gerrit.wikimedia.org/r/#/c/148836/ aims to deploy mathoid on
beta in a first step.
Is there anything I can do to speed up the process beside waiting for
code review?

Best
Physikerwelt

On Fri, Jul 25, 2014 at 12:53 PM, Moritz Schubotz
<[hidden email]> wrote:

> Hi Bryan,
>
> On Thu, Jul 24, 2014 at 1:06 PM, Bryan Davis <[hidden email]> wrote:
>> On Thu, Jul 24, 2014 at 7:00 AM, Moritz Schubotz <[hidden email]> wrote:
>>> Hi Bryan,
>>>
>>> thank you very much for your contribution to the Math extension and
>>> the associated vagrant role.
>>
>> :) You're welcome. Thanks for being dedicated enough to this problem
>> to work on it as a volunteer.
>>
>>> I have tried to write a patch for operations/puppet
>>> https://gerrit.wikimedia.org/r/#/c/148836/
>>> but that's far away from finished.
>>> A lot of configuration and server assignment has to be done.
>>
>> It looks like your patch is getting some feedback this morning. \o/
>>
>>> I searched for the string parsoid in the config files and found a lot
>>> of entries, that seem to specify how parsoid is deployed to individual
>>> servers.
>>> It thought it makes little sense to copy that.
>>
>> I'm not sure what sort of config Mathoid needs. In general, cluster
>> config uses feature flags like $wmgUseParsoid that are set in
>> InitialiseSettings.php. Then the feature flags are used in
>> CommonSettings.php to guard setting "LocalSettings.php" config values
>> for wikis where the feature flag is enabled. Both prod and beta use
>> the settings from this file. CommonSettings-labs.php can be used to
>> change settings from the defaults used in production to match beta
>> specific needs.
>>
> There are two components. The mathoid server, that does not need any
> settings at the moment, and the Math extension that has some settings.
> In my comment above I was not talking of the PHP settings for mathoid,
> but for settings the define on which production instances parsoid is
> installed,
> and how monitoring is done etc.
>>> I had the impression that prior to this change
>>> http://git.wikimedia.org/blobdiff/operations%2Fmediawiki-config/173851ef628359d96fb196172a01cfcfd1da82e5/wmf-config%2FCommonSettings.php
>>> it was possible to enable the MathML rendering mode on betalabs.
>>> The comment
>>> "These variables point to non-WMF servers by default."
>>> is not correct.
>>> $wgMathMathMLUrl = "http://mathoid.testme.wmflabs.org" is hosted in
>>> the eqiad cluster.
>>
>> For the purposes of production configuration, something hosted in
>> wmflabs.org should be treated the same as something hosted at nsa.gov.
>> We don't want production to talk to labs ever. First there are privacy
>> concerns and second Labs will melt if you turn even a tiny fraction of
>> enwiki's traffic towards it.
> Got it.
>>
>>> But it showed that there things on the beta cluster are different from vagrant
>>> https://bugzilla.wikimedia.org/show_bug.cgi?id=66516
>>> describes that the MathML tags are filtered out somehow.
>>
>> I get MathML rendering of the formula on
>> http://deployment.wikimedia.beta.wmflabs.org/wiki/User:Anomie/Sandbox,
>> but I'm not sure what that proves or disproves in relation to your bug
>> reports.
> The question is where did you test that. Locally get MathML as well.
> But when I tested on the beta labs cluster (with the old config that
> used "http://mathoid.testme.wmflabs.org") most of the MathML elements
> were filtered out.
> That's the bug report. I guess the reason for that is that there is a
> different configuration on the beta cluster or another extension that
> interferes with the math extension in an inadequate way.
> Moritz
>
>>
>> Bryan
>> --
>> Bryan Davis              Wikimedia Foundation    <[hidden email]>
>> [[m:User:BDavis_(WMF)]]  Sr Software Engineer            Boise, ID USA
>> irc: bd808                                        v:415.839.6885 x6855
>>
>> _______________________________________________
>> Wikitech-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Moving Mathoid to production cluster

Moritz Schubotz-2
Hi,

finally the change got merged. Now help is needed for the initial
deployment to betacluster.
See:
  https://bugzilla.wikimedia.org/show_bug.cgi?id=69298
The goal is to check out files from
  https://gerrit.wikimedia.org/r/mediawiki/services/mathoid
to the
  /srv/deployment/mathoid/mathoid
On the mathoid server.

Best
Physikerwelt

On Tue, Jul 29, 2014 at 11:59 AM, Moritz Schubotz
<[hidden email]> wrote:

> Hi all,
>
> I have developed the Mathoid deployment puppet script with great
> support of Mwalker and Bryan Davis as far as I could get.
> https://gerrit.wikimedia.org/r/#/c/148836/ aims to deploy mathoid on
> beta in a first step.
> Is there anything I can do to speed up the process beside waiting for
> code review?
>
> Best
> Physikerwelt
>
> On Fri, Jul 25, 2014 at 12:53 PM, Moritz Schubotz
> <[hidden email]> wrote:
>> Hi Bryan,
>>
>> On Thu, Jul 24, 2014 at 1:06 PM, Bryan Davis <[hidden email]> wrote:
>>> On Thu, Jul 24, 2014 at 7:00 AM, Moritz Schubotz <[hidden email]> wrote:
>>>> Hi Bryan,
>>>>
>>>> thank you very much for your contribution to the Math extension and
>>>> the associated vagrant role.
>>>
>>> :) You're welcome. Thanks for being dedicated enough to this problem
>>> to work on it as a volunteer.
>>>
>>>> I have tried to write a patch for operations/puppet
>>>> https://gerrit.wikimedia.org/r/#/c/148836/
>>>> but that's far away from finished.
>>>> A lot of configuration and server assignment has to be done.
>>>
>>> It looks like your patch is getting some feedback this morning. \o/
>>>
>>>> I searched for the string parsoid in the config files and found a lot
>>>> of entries, that seem to specify how parsoid is deployed to individual
>>>> servers.
>>>> It thought it makes little sense to copy that.
>>>
>>> I'm not sure what sort of config Mathoid needs. In general, cluster
>>> config uses feature flags like $wmgUseParsoid that are set in
>>> InitialiseSettings.php. Then the feature flags are used in
>>> CommonSettings.php to guard setting "LocalSettings.php" config values
>>> for wikis where the feature flag is enabled. Both prod and beta use
>>> the settings from this file. CommonSettings-labs.php can be used to
>>> change settings from the defaults used in production to match beta
>>> specific needs.
>>>
>> There are two components. The mathoid server, that does not need any
>> settings at the moment, and the Math extension that has some settings.
>> In my comment above I was not talking of the PHP settings for mathoid,
>> but for settings the define on which production instances parsoid is
>> installed,
>> and how monitoring is done etc.
>>>> I had the impression that prior to this change
>>>> http://git.wikimedia.org/blobdiff/operations%2Fmediawiki-config/173851ef628359d96fb196172a01cfcfd1da82e5/wmf-config%2FCommonSettings.php
>>>> it was possible to enable the MathML rendering mode on betalabs.
>>>> The comment
>>>> "These variables point to non-WMF servers by default."
>>>> is not correct.
>>>> $wgMathMathMLUrl = "http://mathoid.testme.wmflabs.org" is hosted in
>>>> the eqiad cluster.
>>>
>>> For the purposes of production configuration, something hosted in
>>> wmflabs.org should be treated the same as something hosted at nsa.gov.
>>> We don't want production to talk to labs ever. First there are privacy
>>> concerns and second Labs will melt if you turn even a tiny fraction of
>>> enwiki's traffic towards it.
>> Got it.
>>>
>>>> But it showed that there things on the beta cluster are different from vagrant
>>>> https://bugzilla.wikimedia.org/show_bug.cgi?id=66516
>>>> describes that the MathML tags are filtered out somehow.
>>>
>>> I get MathML rendering of the formula on
>>> http://deployment.wikimedia.beta.wmflabs.org/wiki/User:Anomie/Sandbox,
>>> but I'm not sure what that proves or disproves in relation to your bug
>>> reports.
>> The question is where did you test that. Locally get MathML as well.
>> But when I tested on the beta labs cluster (with the old config that
>> used "http://mathoid.testme.wmflabs.org") most of the MathML elements
>> were filtered out.
>> That's the bug report. I guess the reason for that is that there is a
>> different configuration on the beta cluster or another extension that
>> interferes with the math extension in an inadequate way.
>> Moritz
>>
>>>
>>> Bryan
>>> --
>>> Bryan Davis              Wikimedia Foundation    <[hidden email]>
>>> [[m:User:BDavis_(WMF)]]  Sr Software Engineer            Boise, ID USA
>>> irc: bd808                                        v:415.839.6885 x6855
>>>
>>> _______________________________________________
>>> Wikitech-l mailing list
>>> [hidden email]
>>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l



--
Mit freundlichen Grüßen
Moritz Schubotz

  Telefon (Büro):  +49 30 314 22784
  Telefon (Privat):+49 30 488 27330
  E-Mail: [hidden email]
  Web: http://www.physikerwelt.de
  Skype: Schubi87
  ICQ: 200302764
  Msn: [hidden email]

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l