MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

classic Classic list List threaded Threaded
50 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Chad
On Mon, Jul 22, 2013 at 11:24 AM, Jeroen De Dauw <[hidden email]>wrote:

> The solution proposed by Hashar in some other thread is to have a
> "MediaWiki installation" package, which just contains a composer.json file
> with core in it, where people can add dependencies, and then install.
> Before that will work, we'll need to get rid of "global scope assumptions"
> in all core files. For instance, WebStart.php has globals in it, accessed
> without specifying they are globals. This will break when included via
> Composer, as it will not do so in global scope.
>
>
Easier said than done. I've been fighting this fight for years and we're
only about 3 globals closer to doing so. I'm totally in favor of getting
rid of global scope usage, but let's please not trivialize the amount of
work it will take to get there.

-Chad
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Antoine Musso-3
In reply to this post by Ryan Lane-2
Le 22/07/13 19:31, Ryan Lane a écrit :

>
>> > Another reason for having separate components like the WikibaseModel or
>> > Diff is that they are not MediaWiki extensions, but pure PHP libraries.
>> > Any PHP script can reuse them. Since the WikibaseModel is not trivial, this
>> > should help with the writing of bots and scripts dealing with Wikidata
>> > data.
>> > How should we handle such components?
>> > Should they be moved to Wikibase and we require every script to depend on
>> > the whole of Wikibase, and thus MediaWiki?
>> >
>> >
> For pure PHP libraries, they could be distributed like pure PHP libraries
> usually are. They can be packaged for multiple distros and be available via
> apt/yum/composer (or pear). Having them as MediaWiki extensions is somewhat
> awkward.

Maybe we can move the standalone libraries out of mediawiki/extensions/
to some new namespace such as php/ . Then the standalone libraries are
also extensions :-(




<rant>
Please no pear. That is a dumb system that needs to disappear.  You cant
even upload GPL code there, they enforce their own style and require to
use PEAR glue code such as status code and autoloader.
</rant>

--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

C. Scott Ananian
In reply to this post by Jeroen De Dauw-2
The npm peerDependency option is explicitly for extensions/plugins that
depend on other extensions. Ie, a jQuery plugin that requires another
jQuery plugin to also be loaded (a peer).  This is the inter-extension
dependency problem that was being discussed, as I understand it. (I could
be wrong of course.)
  --scott
On Jul 22, 2013 2:25 PM, "Jeroen De Dauw" <[hidden email]> wrote:

> Hey,
>
> Regardless, the question is: can composer help?  It appears that it can:
> >   https://www.mediawiki.org/wiki/Composer
> >
>
> Yeah, and we've already done most of the steps to make this work for
> extensions in core. If you got a MW install, you can already install
> Wikibase (together with all its dependencies), though its going to leave
> you with a modified composer.json.
>
> The solution proposed by Hashar in some other thread is to have a
> "MediaWiki installation" package, which just contains a composer.json file
> with core in it, where people can add dependencies, and then install.
> Before that will work, we'll need to get rid of "global scope assumptions"
> in all core files. For instance, WebStart.php has globals in it, accessed
> without specifying they are globals. This will break when included via
> Composer, as it will not do so in global scope.
>
> This global scope problem is also present in pretty much all MediaWiki
> extensions, except for Wikibase and its libraries, where it was fixed in
> part to not break when used via Composer. This means we will not be able to
> do things in core and have extensions automatically work. (Unless you do
> some kind of hack where you promote all variables in the current scope to
> global scope, though I suspect we do not want to do this :D).
>
>  --scott (who will attempt to ignore future non-technical posts on this
> > thread)
> >
>
> That is a good explicit rule. Thanks, I'll follow it as well.
>
> Cheers
>
> --
> Jeroen De Dauw
> http://www.bn2vs.com
> Don't panic. Don't be evil. ~=[,,_,,]:3
> --
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Antoine Musso-3
In reply to this post by Derric Atzrott
Le 22/07/13 15:20, Derric Atzrott a écrit :
> To throw another viewpoint into the mix.  If we require composer, we
> require users to learn to use composer.  Some like myself have never
> used it, and while it’s a skill I should probably learn that will
> save me considerable time, it may be that not all will find being
> forced to learn a new piece of software so great.
>
> Of course I could be missing something here.

That is indeed a valid argument. Given we migrated our community from
subversion to git, I am confident enough that using composer will be
very easy to the community.  An almost complete tutorial is a few
paragraph long, there are ton of resources around and more and more
upstream are migrating to composer :-)

So yeah composer adds one more brick, but that solve so many issues that
I think it is worth the tiny time investment.


  mkdir project; cd project
  composer require foo/bar
  require 'vendor/autoload.php';

Done :-]

--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Denny Vrandečić
2013/7/22 Antoine Musso <[hidden email]>

> Given we migrated our community from
> subversion to git, I am confident enough that using composer will be
> very easy to the community.


:D
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Denny Vrandečić
In reply to this post by Ryan Lane-2
2013/7/22 Ryan Lane <[hidden email]>

> For pure PHP libraries, they could be distributed like pure PHP libraries
> usually are. They can be packaged for multiple distros and be available via
> apt/yum/composer (or pear). Having them as MediaWiki extensions is somewhat
> awkward.
>
>
Yes, agree on that.




> How does splitting extensions apart make it easier to check this during CI?
> Is there some automated way to do this when they are split, but not when
> they are together?
>
>
Yes.
Assume you have class Ext which relies on class Core, but class Core should
not rely on class Ext, because they are on different architectural levels.
If Ext and Core are together, your CI will load them both and you won't
notice if you access Ext from Core. No error is thrown.
If Core is not together with Ext, then unit-testing Core will not load Ext.
A call to Core will fail, and your CI will discover the error.
The split helps keeping the architectural integrity of the code, and avoids
it being



> My concern was increasing complexity for admins with no effort towards
> decreasing it. Composer may indeed be an answer to some of the issues. From
> an admin's perspective using composer shouldn't be much harder than what
> they are currently expected to do and it would simplify the extension
> process.
>
>
Yes, I think so too. Composer could be a viable option, and within Wikidata
we are quite happy with it. We would recommend adoption by the wider
community.
If the wider community chooses another solution for the dependency problem,
we will adapt Wikibase to it too.

Cheers,
Denny
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Tyler Romeo
On Mon, Jul 22, 2013 at 3:23 PM, Denny Vrandečić <
[hidden email]> wrote:

> Assume you have class Ext which relies on class Core, but class Core should
> not rely on class Ext, because they are on different architectural levels.
> If Ext and Core are together, your CI will load them both and you won't
> notice if you access Ext from Core. No error is thrown.
> If Core is not together with Ext, then unit-testing Core will not load Ext.
> A call to Core will fail, and your CI will discover the error.
> The split helps keeping the architectural integrity of the code, and avoids
> it being
>

Architectural integrity of code is a design-level issue. Continuous
integration is a programming and quality assurance-level issue. They have
nothing to do with each other, and you can maintain architectural integrity
just fine without having to split your singular product into multiple
products. If that were true, than the MW core would be split across fifty
different repositories. (If Makefiles can compile different parts of a
product independently, then so can we.)

*-- *
*Tyler Romeo*
Stevens Institute of Technology, Class of 2016
Major in Computer Science
www.whizkidztech.com | [hidden email]
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Denny Vrandečić
2013/7/22 Tyler Romeo <[hidden email]>

> Architectural integrity of code is a design-level issue. Continuous
> integration is a programming and quality assurance-level issue. They have
> nothing to do with each other, and you can maintain architectural integrity
> just fine without having to split your singular product into multiple
> products.
>

I would disagree with you regarding your statement that architectural
integrity and quality assurance have nothing to do with each other. I hope
I do not have to explain - but if I do, feel free to ask.

I agree with you regarding your statement that you can maintain
architectural integrity without checking it automatically. You can also
make sure that code style is not violated manually. Or that code works
without unit tests and by testing manually. But considering that reviewing
resources are scarce, I prefer to test as much automatically as possibly by
CI and relieve the reviewer from considering e.g. the dependency
architecture of your classes during a review. I do not see the advantage of
doing that manually.

I think that also core would benefit if architectural constraints could be
enforced by CI.

> If that were true, than the MW core would be split across fifty
> different repositories. (If Makefiles can compile different parts of a
> product independently, then so can we.)

I fail to understand what you mean here, sorry.
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Jeroen De Dauw-2
In reply to this post by Chad
Hey,

> The solution proposed by Hashar in some other thread is to have a
> > "MediaWiki installation" package, which just contains a composer.json
> file
> > with core in it, where people can add dependencies, and then install.
> > Before that will work, we'll need to get rid of "global scope
> assumptions"
> > in all core files. For instance, WebStart.php has globals in it, accessed
> > without specifying they are globals. This will break when included via
> > Composer, as it will not do so in global scope.
> >
> >
> Easier said than done. I've been fighting this fight for years and we're
> only about 3 globals closer to doing so. I'm totally in favor of getting
> rid of global scope usage, but let's please not trivialize the amount of
> work it will take to get there.
>

> Easier said than done.

Certainly. Else it would probably already be done :)

> I've been fighting this fight for years and we're only about 3 globals
closer to doing so.

I suspect you are not understanding me correctly after reading that. I'm
talking about referring to global variables as if they where local
variables (thus without the "global" keyword) which happens to work if your
code is being executed from global scope, but breaks when it is included
from any other scope. Getting rid of all globals would be an humongous task
and one that is simply not feasible in a short period of time. Luckily we
do not need to do this to have core work with Composer.

> but let's please not trivialize the amount of work it will take to get
there.

I certainly did not mean to imply doing this is trivial, as I agree with
you it is a lot of work. That is why I'm listing this as one of the main
obstacles.

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Antoine Musso-3
In reply to this post by Antoine Musso-3
Le 19/07/13 23:26, Antoine Musso a écrit :
>  mkdir mysite
>  cd mysite
>  composer require mediawiki/core
>  composer require wikibase/wikibase
>  # which installs data-values/data-values ask/ask as well
<snip>
> And a vendor directory containing all dependencies.  At the root of your
> directory you would write some glue maybe looking like:
>
>  <?php
>  require('vendor/autoload.php');
>  require('vendor/mediawiki/core/index.php');

The glue need to give a hint to WebStart to properly initialize $IP


 require('vendor/autoload.php');
 putenv( 'MW_INSTALL_PATH=' . __DIR__ . '/vendor/mediawiki/core' );
 require('vendor/mediawiki/core/index.php');

:)

--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
123