MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

classic Classic list List threaded Threaded
50 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Antoine Musso-3
Le 21/07/13 22:58, Ryan Lane a écrit :
>  When I attempt to
> upgrade MediaWiki I currently have to write down all of the extensions, and
> ensure all of them are compatible with MediaWiki. With some subsets, I need
> to ensure they are compatible with each other (like SMW, SF, SRF). Now I'm
> going to need to do that and track the compatibility between extensions and
> dependency extensions. I'm actually going to have to write an upgrade
> matrix to upgrade, and that's not OK.

Why would you want to manually keep track of the dependencies when a
tool such as composer can handle it for you?

--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Antoine Musso-3
In reply to this post by Denny Vrandečić
Le 22/07/13 13:29, Denny Vrandečić a écrit :

>> > And installing yoursite would be something like:
>> >
>> >  mkdir mysite
>> >  cd mysite
>> >  composer require mediawiki/core
>> >  composer require wikibase/wikibase
>> >  # which installs data-values/data-values ask/ask as well
>> >
>> >
> Just curious, why is "composer require mediawiki/core" needed?

wikibase/wikibase does not list mediawiki/core as a dependency :)


 $ composer show wikibase/wikibase
 ...
 requires:
 php >=5.3.0
 data-values/data-values *
 wikibase/data-model *
 diff/diff >=0.6
 data-values/data-types *


--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Jeroen De Dauw-2
Hey,

> wikibase/wikibase does not list mediawiki/core as a dependency :)

Indeed. Right now this just allows you to install Wikibase into an existing
MW install. Before we can go all the way, we first need to be able to do a
MediaWiki (+extensions) install, which is something still under discussion.

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Denny Vrandečić
Ah, OK, understood. Thanks.


2013/7/22 Jeroen De Dauw <[hidden email]>

> Hey,
>
> > wikibase/wikibase does not list mediawiki/core as a dependency :)
>
> Indeed. Right now this just allows you to install Wikibase into an existing
> MW install. Before we can go all the way, we first need to be able to do a
> MediaWiki (+extensions) install, which is something still under discussion.
>
> Cheers
>
> --
> Jeroen De Dauw
> http://www.bn2vs.com
> Don't panic. Don't be evil. ~=[,,_,,]:3
> --
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



--
Project director Wikidata
Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin
Tel. +49-30-219 158 26-0 | http://wikimedia.de

Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Max Semenik
In reply to this post by Jeroen De Dauw-2
On 22.07.2013, 16:11 Jeroen wrote:

> Hey,

>> Just curious, why is "composer require mediawiki/core" needed?


> Because Hashar is talking about having a "MediaWiki installation" package,
> which effectively turns into "My MediaWiki installation" when you start
> using it. This package specifies all the things you want to install (in its
> composer.json). This will always include core, and also list the extensions
> you want on your particular install.

> The reason for having this seperate package is to get around the problem I
> described earlier in this thread - if you just have core, and you install
> other things into it by modifying the composer.josn of core, you'll end up
> with a modified non-optional file in the core repo, which causes hassle
> when you want to update.

> Of course if we got with such a "MediaWiki installation" package, I'd be
> natural to put mediawiki-core already in the composer.json file, so people
> would not actually need to run this command.

So it's a Composer's limitation. It woud've been awesome if it was
possible to do something like

if exists "optional.json":
   install "optional.json"

:)

--
Best regards,
  Max Semenik ([[User:MaxSem]])


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Derric Atzrott
In reply to this post by Antoine Musso-3
>>  When I attempt to
>> upgrade MediaWiki I currently have to write down all of the extensions, and
>> ensure all of them are compatible with MediaWiki. With some subsets, I need
>> to ensure they are compatible with each other (like SMW, SF, SRF). Now I'm
>> going to need to do that and track the compatibility between extensions and
>> dependency extensions. I'm actually going to have to write an upgrade
>> matrix to upgrade, and that's not OK.
>
>Why would you want to manually keep track of the dependencies when a
>tool such as composer can handle it for you?

To throw another viewpoint into the mix.  If we require composer, we require users to learn to use composer.  Some like myself have never used it, and while it’s a skill I should probably learn that will save me considerable time, it may be that not all will find being forced to learn a new piece of software so great.

Of course I could be missing something here.

Thank you,
Derric Atzrott


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Denny Vrandečić
In reply to this post by Jeroen De Dauw-2
Hi,

please, everyone calm down and honesty try harder to assume faith and
respect the capabilities of each other. Respect includes to avoid
terminology like "to bitch", "to sneak in", "stupid" when describing each
other's actions.

One good way is when you are angry about an email, step back, wait a day,
calm down, and answer only then. Since this matter is important, but not
urgent, there is nothing that requires quick responses. No one's going to
make a decision for all of us right away, so there is no need to reply
quickly and escalate.


I assume Ryan didn't mean to single out the Wikidata development team.
Other teams have done this as well -- the Translate extension depends on
ULS, CodeEditor depends on WikiEditor, Semantic Result Formats depends on
Semantic MediaWiki etc. I assume Ryan just choose Wikibase because it
exemplifies the symptoms so well. Ryan, please correct me if my assumptions
are wrong.

The reminder of this mail has two parts, first it tries to explain the
motivation of why the Wikidata dev team did things the way we did, and
second, it asks this list to please resolve the underlying actual issue.

First:

The original message by Ryan lists the number of dependencies for Wikibase.
He lists, e.g. Scribunto as an extension.
The question is, how to avoid this dependency?
Should we move Scribunto to core?
Should we turn Scribunto and Wikibase into one extension?

The same for Babel and ULS, listed as optional extensions.

Another extension mentioned is Diff. Diff provides generic functionality
that can be used by different extensions.
It seems that in this case the suggestion that Ryan makes is to move it to
core.
So shall we move functionality that more than one extension depend on
generally to core?

One of the reasons for DataValues, Ask and some others being in their own
extension is that they already are or are planned very soonish to be reused
in SMW.
Since it is suggested that generic functionality should be moved to core,
would that include functionality shared by Wikibase and SMW?
Or how else would we manage that shared code?

Another reason for having separate components like the WikibaseModel or
Diff is that they are not MediaWiki extensions, but pure PHP libraries.
Any PHP script can reuse them. Since the WikibaseModel is not trivial, this
should help with the writing of bots and scripts dealing with Wikidata data.
How should we handle such components?
Should they be moved to Wikibase and we require every script to depend on
the whole of Wikibase, and thus MediaWiki?

If you add everything needed by Wikibase into a single extension, how do
you ensure that no unnecessary dependencies creep in?
Is there a code-analyzer that can run as part of Jenkins that check that
the architecture is not being violated, and that parts of the code to not
introduce dependencies on other parts where they should not?
Separating such components allows us to check this part of the architecture
during CI, which is indeed extremely helpful.


I would indeed be very much interested in better solutions for these
questions than we currently have.

As Ryan said in his thread-opening email, "For legitimate library-like
extensions I have no constructive alternative, but there must be some sane
alternative to this."
A lot of the issues would be resolved if we had this constructive
alternative. The solution will likely also help to deal with the other
dependencies.
I hope it is understandable that I do not consider the time of the Wikidata
development well spent to replace our architecture with something else,
before we have agreed on what this something else should be.


Second:

I would be interested in answers to the above questions.
But maybe we really should concentrate on getting the actual question
resolved, which has been discussed on this list several times without
consensus, those that would allows us to answer the above questions
trivially:
how should we deal with modularity, extensions, components, etc. in
MediaWiki?
I hope the answer is not "throw everything in core or into monolithical
extensions which do not have dependencies among each other", but let's see
what the discussion will bring. Once we have this answer, we can implement
the results. Until then I am not sure whether I found it productive to
single the Wikidata team out in the way we are doing things.

I do not think the Wikidata development team is anticipating and
predetermining answers to those questions, but in order for us to continue
develop we had to make some decisions for us. But we don't tell anyone else
how to do their job, and we try to stick to current consensus on how to do
things. And as soon as we have consensus, we will adopt. We did this
previously a few times, and I don't see why we should not do it again.

Cheers,
Denny





2013/7/22 Jeroen De Dauw <[hidden email]>

> Hey,
>
> > Validator was rejected from being included in core
>
> I'm happy it did. The code was quite poor at that time (two years back?).
> And it still is not a hallmark of great design, though certainly not below
> average MW quality at this point. Regardless of that, I now think it is a
> bad idea to have it in core and would not petition to have it there in the
> future.
>
> > but now basically every extension you maintain uses it.
>
> Can you please look at objective reality first before making claims about
> how you would like the world to be so you can bitch about it? Not only are
> you overly simplifying things, you get simple facts plain wrong.
>
> Cheers
>
> --
> Jeroen De Dauw
> http://www.bn2vs.com
> Don't panic. Don't be evil. ~=[,,_,,]:3
> --
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



--
Project director Wikidata
Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin
Tel. +49-30-219 158 26-0 | http://wikimedia.de

Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

C. Scott Ananian
It seems that a package manager is a reasonable way to manage a growing set
of dependencies.
Composer seems to me (without knowing too much about it) to be a reasonable
package manager.

I've heard the objection expressed that this will require new users of
wikibase/etc to "learn composer".  Presumably the addition of a
composer.json file to the repo won't in itself prevent users from using the
released tarballs and ignoring composer altogether, so the only folks who
have to work with composer are those developing/admining a mediawiki
derivative with hairy dependencies, such that learning composer pays for
itself.  Correct me if I misunderstand.

Can someone outline (for discussion's sake) the other objections to
composer?  Otherwise, this discussion does not seem too knotty:  someone
should submit a patch to "composer-ify" mediawiki/core and the various
extensions, the release process should include uploading versions to
https://packagist.org/ (optional, since composer can apparently install
directly from WMF git), and then people should use it for a while before
deciding if something else is needed (which may be just patches to
composer).
  --scott, trying to drain the drama out of this discussion and understand
the technical issues.

ps. are patches needed to composer to make this process more
lightweight/require less modification to mediawiki/core?  I note that
mediawiki seems to have support in
https://github.com/composer/installersalready.

pps. I think the multiple entry points as a technical issue with the
installer?  By from my reading, there's no reason why you couldn't write a
separate 'mediawiki-composer' shell package that included just stub
index.php, api.php, etc files which loaded the appropriate entry point from
inside vendor/mediawiki or whatever.  So (in my understanding) the changes
required to mediawiki/core are minimal.  Please correct me if I'm wrong
(hopefully with a detailed list of the changes actually required).

--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Mark A. Hershberger-2
In reply to this post by Derric Atzrott
On 07/22/2013 09:20 AM, Derric Atzrott wrote:
> If we require composer, we require users to learn to use composer.
> Some like myself have never used it, and while it’s a skill I should
> probably learn that will save me considerable time, it may be that
> not all will find being forced to learn a new piece of software so great.
>
> Of course I could be missing something here.

Composer is becoming widely used in the PHP universe.  If your only
focus of development is MediaWiki, then, yes, this is a pain.

However, the use of composer could open up whole new possibilities.  For
the old Perl hands, composer is a lot like cpan.  There are a lot of
ugly things in there, but it doesn't take too much to find some really
useful re-usable components.

And composer would only be a requirement for developers, not end users.
 There is no reason that we can't still distribute tarballs that a
Sysadmin who just maintains a wiki or a wiki farm today would install in
the same way he currently does.

Mark.

--
http://hexmode.com/

Love alone reveals the true shape of the universe.
     -- "Everywhere Present", Stephen Freeman

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Chad
On Mon, Jul 22, 2013 at 8:36 AM, Mark A. Hershberger <[hidden email]>wrote:

> On 07/22/2013 09:20 AM, Derric Atzrott wrote:
> > If we require composer, we require users to learn to use composer.
> > Some like myself have never used it, and while it’s a skill I should
> > probably learn that will save me considerable time, it may be that
> > not all will find being forced to learn a new piece of software so great.
> >
> > Of course I could be missing something here.
>
> Composer is becoming widely used in the PHP universe.  If your only
> focus of development is MediaWiki, then, yes, this is a pain.
>
> However, the use of composer could open up whole new possibilities.  For
> the old Perl hands, composer is a lot like cpan.  There are a lot of
> ugly things in there, but it doesn't take too much to find some really
> useful re-usable components.
>
> And composer would only be a requirement for developers, not end users.
>  There is no reason that we can't still distribute tarballs that a
> Sysadmin who just maintains a wiki or a wiki farm today would install in
> the same way he currently does.
>

Telling me it's like cpan just brings back awful awful memories...

And a developer, please don't *require* me to use Composer. Don't want
it, don't need it.

-Chad
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Mark A. Hershberger-2
On 07/22/2013 11:43 AM, Chad wrote:
> Telling me it's like cpan just brings back awful awful memories...

I apologize.  I can't say my experience with cpan was all roses, but it
seems that my experience was better than yours. ;)

--
http://hexmode.com/

Love alone reveals the true shape of the universe.
     -- "Everywhere Present", Stephen Freeman

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

C. Scott Ananian
On Mon, Jul 22, 2013 at 12:00 PM, Mark A. Hershberger <[hidden email]>wrote:

> On 07/22/2013 11:43 AM, Chad wrote:
> > Telling me it's like cpan just brings back awful awful memories...
>
> I apologize.  I can't say my experience with cpan was all roses, but it
> seems that my experience was better than yours. ;)
>

The cool kids say "it's like npm" now.
 --scott

--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Chad
You're not convincing me ;-)

-Chad
On Jul 22, 2013 9:06 AM, "C. Scott Ananian" <[hidden email]> wrote:

> On Mon, Jul 22, 2013 at 12:00 PM, Mark A. Hershberger <[hidden email]
> >wrote:
>
> > On 07/22/2013 11:43 AM, Chad wrote:
> > > Telling me it's like cpan just brings back awful awful memories...
> >
> > I apologize.  I can't say my experience with cpan was all roses, but it
> > seems that my experience was better than yours. ;)
> >
>
> The cool kids say "it's like npm" now.
>  --scott
>
> --
> (http://cscott.net)
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Tyler Romeo
In reply to this post by C. Scott Ananian
Just to clear some things up, composer is *not* a package manager. It is
actually pretty terrible at being a package manager, mainly because it's
not supposed to be one. The purpose of composer is solely dependency
management.

Because of that, using it as a package manager requires using some hackish
techniques as mentioned above, where you require the MW core as a library
(even though it isn't one). That's the main reason I'd consider not using
composer.

*-- *
*Tyler Romeo*
Stevens Institute of Technology, Class of 2016
Major in Computer Science
www.whizkidztech.com | [hidden email]


On Mon, Jul 22, 2013 at 12:06 PM, C. Scott Ananian
<[hidden email]>wrote:

> On Mon, Jul 22, 2013 at 12:00 PM, Mark A. Hershberger <[hidden email]
> >wrote:
>
> > On 07/22/2013 11:43 AM, Chad wrote:
> > > Telling me it's like cpan just brings back awful awful memories...
> >
> > I apologize.  I can't say my experience with cpan was all roses, but it
> > seems that my experience was better than yours. ;)
> >
>
> The cool kids say "it's like npm" now.
>  --scott
>
> --
> (http://cscott.net)
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

C. Scott Ananian
On Mon, Jul 22, 2013 at 12:15 PM, Tyler Romeo <[hidden email]> wrote:

> Just to clear some things up, composer is *not* a package manager. It is
> actually pretty terrible at being a package manager, mainly because it's
> not supposed to be one. The purpose of composer is solely dependency
> management.
>

Arguably the problem which needs to be solved here is dependency management.

Regardless, the question is: can composer help?  It appears that it can:
  https://www.mediawiki.org/wiki/Composer

I'm interested in having a technical discussion of next steps.  What can we
do to make this work even better?
 --scott (who will attempt to ignore future non-technical posts on this
thread)

--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Tyler Romeo
On Mon, Jul 22, 2013 at 12:53 PM, C. Scott Ananian
<[hidden email]>wrote:

> Arguably the problem which needs to be solved here is dependency
> management.
>
> Regardless, the question is: can composer help?  It appears that it can:
>   https://www.mediawiki.org/wiki/Composer
>
> I'm interested in having a technical discussion of next steps.  What can we
> do to make this work even better?
>  --scott (who will attempt to ignore future non-technical posts on this
> thread)
>

Partially. The issue is that extensions are both packages and dependencies.
Scribunto is itself an independent package that a wiki can install, but at
the same time it can be a dependency for other extensions. That's where it
gets tricky...

*-- *
*Tyler Romeo*
Stevens Institute of Technology, Class of 2016
Major in Computer Science
www.whizkidztech.com | [hidden email]
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

C. Scott Ananian
On Mon, Jul 22, 2013 at 12:55 PM, Tyler Romeo <[hidden email]> wrote:

> Partially. The issue is that extensions are both packages and dependencies.
> Scribunto is itself an independent package that a wiki can install, but at
> the same time it can be a dependency for other extensions. That's where it
> gets tricky...
>

npm has a newly-added 'peerDependencies' feature.  It looks like composer
could use a similar feature.  Anyone want to work on a composer patch?
 --scott

--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Ryan Lane-2
In reply to this post by Denny Vrandečić
On Mon, Jul 22, 2013 at 6:25 AM, Denny Vrandečić <
[hidden email]> wrote:

> I assume Ryan didn't mean to single out the Wikidata development team.
> Other teams have done this as well -- the Translate extension depends on
> ULS, CodeEditor depends on WikiEditor, Semantic Result Formats depends on
> Semantic MediaWiki etc. I assume Ryan just choose Wikibase because it
> exemplifies the symptoms so well. Ryan, please correct me if my assumptions
> are wrong.
>
>
Indeed, this is a general problem, but Wikibase does this quite a bit more
than other extensions have in the past. It's been a growing concern of mine
as an admin for quite a while now.


> The reminder of this mail has two parts, first it tries to explain the
> motivation of why the Wikidata dev team did things the way we did, and
> second, it asks this list to please resolve the underlying actual issue.
>
> First:
>
> The original message by Ryan lists the number of dependencies for Wikibase.
> He lists, e.g. Scribunto as an extension.
> The question is, how to avoid this dependency?
> Should we move Scribunto to core?
> Should we turn Scribunto and Wikibase into one extension?
>
> The same for Babel and ULS, listed as optional extensions.
>
> Another extension mentioned is Diff. Diff provides generic functionality
> that can be used by different extensions.
> It seems that in this case the suggestion that Ryan makes is to move it to
> core.
> So shall we move functionality that more than one extension depend on
> generally to core?
>
> One of the reasons for DataValues, Ask and some others being in their own
> extension is that they already are or are planned very soonish to be reused
> in SMW.
> Since it is suggested that generic functionality should be moved to core,
> would that include functionality shared by Wikibase and SMW?
> Or how else would we manage that shared code?
>
>
This is the hardest scenario of the group. It's likely best that they are
in separate extensions. This is still a complication, but is more an issue
with MediaWiki's lack of extension management than it is with the
modularity of the extensions themselves.


> Another reason for having separate components like the WikibaseModel or
> Diff is that they are not MediaWiki extensions, but pure PHP libraries.
> Any PHP script can reuse them. Since the WikibaseModel is not trivial, this
> should help with the writing of bots and scripts dealing with Wikidata
> data.
> How should we handle such components?
> Should they be moved to Wikibase and we require every script to depend on
> the whole of Wikibase, and thus MediaWiki?
>
>
For pure PHP libraries, they could be distributed like pure PHP libraries
usually are. They can be packaged for multiple distros and be available via
apt/yum/composer (or pear). Having them as MediaWiki extensions is somewhat
awkward.


> If you add everything needed by Wikibase into a single extension, how do
> you ensure that no unnecessary dependencies creep in?
> Is there a code-analyzer that can run as part of Jenkins that check that
> the architecture is not being violated, and that parts of the code to not
> introduce dependencies on other parts where they should not?
> Separating such components allows us to check this part of the architecture
> during CI, which is indeed extremely helpful.
>
>
How does splitting extensions apart make it easier to check this during CI?
Is there some automated way to do this when they are split, but not when
they are together?


>
> I would indeed be very much interested in better solutions for these
> questions than we currently have.
>
> As Ryan said in his thread-opening email, "For legitimate library-like
> extensions I have no constructive alternative, but there must be some sane
> alternative to this."
> A lot of the issues would be resolved if we had this constructive
> alternative. The solution will likely also help to deal with the other
> dependencies.
> I hope it is understandable that I do not consider the time of the Wikidata
> development well spent to replace our architecture with something else,
> before we have agreed on what this something else should be.
>
>
Yes, that's surely a good position to take :).


> Second:
>
> I would be interested in answers to the above questions.
> But maybe we really should concentrate on getting the actual question
> resolved, which has been discussed on this list several times without
> consensus, those that would allows us to answer the above questions
> trivially:
> how should we deal with modularity, extensions, components, etc. in
> MediaWiki?
> I hope the answer is not "throw everything in core or into monolithical
> extensions which do not have dependencies among each other", but let's see
> what the discussion will bring. Once we have this answer, we can implement
> the results. Until then I am not sure whether I found it productive to
> single the Wikidata team out in the way we are doing things.
>
>
My concern was increasing complexity for admins with no effort towards
decreasing it. Composer may indeed be an answer to some of the issues. From
an admin's perspective using composer shouldn't be much harder than what
they are currently expected to do and it would simplify the extension
process.

- Ryan
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Jeroen De Dauw-2
In reply to this post by C. Scott Ananian
Hey,

npm has a newly-added 'peerDependencies' feature.  It looks like composer
> could use a similar feature.  Anyone want to work on a composer patch?
>

I might be misunderstanding what this peerDependencies does, though if not,
then it is different then what is needed for the scenario described below:

> Partially. The issue is that extensions are both packages and
dependencies.
> Scribunto is itself an independent package that a wiki can install, but at
> the same time it can be a dependency for other extensions. That's where it
> gets tricky...

If you want to install Scribuntu, then add it to the requires section of
composer.json. If you want the extension that needs scrubuntu, then add the
extension to the requires section of composer.json. If you want both, add
both. Then run composer install, and you are done. The resolving of
dependencies Composer does eliminates trickiness one would run into when
doing manual installation.

npm has a newly-added 'peerDependencies' feature.  It looks like composer
> could use a similar feature.  Anyone want to work on a composer patch?
>

As I understand this peerDependencies thing, it is designed to be able to
deal with cycles in your dependency graph. That might be nice to have in
such a tool, though I'd argue not the most important feature. If you got
cycles in your dependency graph, you got a serious problem with your
dependencies, which you should fix. See also: Acyclic Dependencies Principle

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: MediaWiki extensions as core-like libraries: MediaWiki's fun new landmine for admins

Jeroen De Dauw-2
In reply to this post by C. Scott Ananian
Hey,

Regardless, the question is: can composer help?  It appears that it can:
>   https://www.mediawiki.org/wiki/Composer
>

Yeah, and we've already done most of the steps to make this work for
extensions in core. If you got a MW install, you can already install
Wikibase (together with all its dependencies), though its going to leave
you with a modified composer.json.

The solution proposed by Hashar in some other thread is to have a
"MediaWiki installation" package, which just contains a composer.json file
with core in it, where people can add dependencies, and then install.
Before that will work, we'll need to get rid of "global scope assumptions"
in all core files. For instance, WebStart.php has globals in it, accessed
without specifying they are globals. This will break when included via
Composer, as it will not do so in global scope.

This global scope problem is also present in pretty much all MediaWiki
extensions, except for Wikibase and its libraries, where it was fixed in
part to not break when used via Composer. This means we will not be able to
do things in core and have extensions automatically work. (Unless you do
some kind of hack where you promote all variables in the current scope to
global scope, though I suspect we do not want to do this :D).

 --scott (who will attempt to ignore future non-technical posts on this
> thread)
>

That is a good explicit rule. Thanks, I'll follow it as well.

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
123