Managing external dependencies for MediaWiki core

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Managing external dependencies for MediaWiki core

Daniel Kinzler
This is about whether it's OK for MediaWiki core to depend on other PHP
libraries, and how to manage such dependencies.

Background: A while back, I proposed a simple class for assertions to be added
to core[1]. It was then suggested[2] that this could be placed in a separate
component, which could then be re-used by others via composer. Since the
assertions are very little code and nicely self-contained, this should be easy
to do.

However, if we want to use these in MediaWiki core, core would now depend on the
assertion component. That means that either MediaWiki would require installation
via Composer, or we have to bundle the library in some other way.

What's the best practice for this kind of thing? Shall we just make the
assertion repo an git submodule, and then pull and bundle it when making tarball
releases? Should we switch the generation of tarballs to using composer? Or
should we require composer based installs in the future? Are there other options?

Cheers,
Daniel

[1] https://www.mediawiki.org/wiki/Requests_for_comment/Assert
[2]
https://www.mediawiki.org/wiki/Talk:Requests_for_comment/Assert#Use_outside_of_MediaWiki

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Managing external dependencies for MediaWiki core

Antoine Musso-3
Le 25/07/2014 11:08, Daniel Kinzler a écrit :

> This is about whether it's OK for MediaWiki core to depend on other PHP
> libraries, and how to manage such dependencies.
>
> Background: A while back, I proposed a simple class for assertions to be added
> to core[1]. It was then suggested[2] that this could be placed in a separate
> component, which could then be re-used by others via composer. Since the
> assertions are very little code and nicely self-contained, this should be easy
> to do.
>
> However, if we want to use these in MediaWiki core, core would now depend on the
> assertion component. That means that either MediaWiki would require installation
> via Composer, or we have to bundle the library in some other way.
>
> What's the best practice for this kind of thing? Shall we just make the
> assertion repo an git submodule, and then pull and bundle it when making tarball
> releases? Should we switch the generation of tarballs to using composer? Or
> should we require composer based installs in the future? Are there other options?

Hello Daniel,

The idea is to rely on the git repository mediawiki/core/vendor (that is
going to be renamed as mediawiki/vendor).   It contains a composer.json
file and is supposed to contain the download code.

On production we would just clone mediawiki/vendor with the same branch
that mediawiki/core is using.  Much like we are already doing with
extensions.

For continuous integration purposes, that is a bit more challenging.  I
have worked on a client that let us clone multiple repositories and make
sure they all use the branch of the patch being tested with graceful
fallback to master.

So if one push a change to mediawiki/vendor 1.24wmf25 , the script would
clone mediawiki/core and attempt to checkout 1.24wmf25 then run the
tests with the resulting code.

I still have to figure out a bunch of details such as how to clone so
many repositories on Jenkins slaves for each job we have.  But that is
being worked on for sure.

cheers,

--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Managing external dependencies for MediaWiki core

Bryan Davis
On Fri, Jul 25, 2014 at 4:45 AM, Antoine Musso <[hidden email]> wrote:

> Le 25/07/2014 11:08, Daniel Kinzler a écrit :
>> This is about whether it's OK for MediaWiki core to depend on other PHP
>> libraries, and how to manage such dependencies.
>>
>> Background: A while back, I proposed a simple class for assertions to be added
>> to core[1]. It was then suggested[2] that this could be placed in a separate
>> component, which could then be re-used by others via composer. Since the
>> assertions are very little code and nicely self-contained, this should be easy
>> to do.
>>
>> However, if we want to use these in MediaWiki core, core would now depend on the
>> assertion component. That means that either MediaWiki would require installation
>> via Composer, or we have to bundle the library in some other way.
>>
>> What's the best practice for this kind of thing? Shall we just make the
>> assertion repo an git submodule, and then pull and bundle it when making tarball
>> releases? Should we switch the generation of tarballs to using composer? Or
>> should we require composer based installs in the future? Are there other options?
>
> Hello Daniel,
>
> The idea is to rely on the git repository mediawiki/core/vendor (that is
> going to be renamed as mediawiki/vendor).   It contains a composer.json
> file and is supposed to contain the download code.
>
> On production we would just clone mediawiki/vendor with the same branch
> that mediawiki/core is using.  Much like we are already doing with
> extensions.
>
> For continuous integration purposes, that is a bit more challenging.  I
> have worked on a client that let us clone multiple repositories and make
> sure they all use the branch of the patch being tested with graceful
> fallback to master.
>
> So if one push a change to mediawiki/vendor 1.24wmf25 , the script would
> clone mediawiki/core and attempt to checkout 1.24wmf25 then run the
> tests with the resulting code.
>
> I still have to figure out a bunch of details such as how to clone so
> many repositories on Jenkins slaves for each job we have.  But that is
> being worked on for sure.

What a just-in-time question and answer! Antoine is describing the
implementation of the RFC for how the WMF will manage deploying
Composer managed dependencies [0] which was just approved this week by
the Architecture team. I have an outstanding todo from the approval
meeting to get the existing gerrit repo renamed (well actually to
create a new repo and migrate the existing content over). I hope to
submit the patches necessary for this small change next week.

Look over the RFC and see if it answers most of your questions and
please ping me for clarifications on things that are missing or
confusing.

[0]: https://www.mediawiki.org/wiki/Requests_for_comment/Composer_managed_libraries_for_use_on_WMF_cluster

--
Bryan Davis              Wikimedia Foundation    <[hidden email]>
[[m:User:BDavis_(WMF)]]  Sr Software Engineer            Boise, ID USA
irc: bd808                                        v:415.839.6885 x6855

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l