Some questions about #switch

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Some questions about #switch

Alex Brollo
I'm using more and more #switch into templates, it's surprising how many
issues it can solve, and how much large arrays it can manage.  My questions
are:
1. Is there a reasonable upper limit for the number of #switch options?
2. Is there a difference in server load between a long list of #switch
options and a "tree" of options t.i. something as a nested #switch?
3. Is there any drawback into a large use of #swtch-based templates?

I presume that my questions are far from original, please give me a link to
previous talks or help/doc pages if any.

Alex
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Some questions about #switch

Roan Kattouw-2
On Mon, Jan 30, 2012 at 3:11 PM, Alex Brollo <[hidden email]> wrote:
> I'm using more and more #switch into templates, it's surprising how many
> issues it can solve, and how much large arrays it can manage.  My questions
> are:
> 1. Is there a reasonable upper limit for the number of #switch options?
> 2. Is there a difference in server load between a long list of #switch
> options and a "tree" of options t.i. something as a nested #switch?
> 3. Is there any drawback into a large use of #swtch-based templates?
>
I can at least tell you that the answer to #1 is probably no, and the
answer to #3 is yes. There were some server issues last week that were
blamed on ocwiki's use of templates with #switch statements with
thousands of options. For instance,
https://oc.wikipedia.org/w/index.php?title=Mod%C3%A8l:Altmejcom&action=edit
has 36,611 cases, because it's essentially used as a database. That's
the kind of thing that takes forever to parse.

Roan

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Some questions about #switch

Helder .
What about ptwikibooks' usage of
https://pt.wikibooks.org/wiki/Template:Lista_de_capítulos/Posterior?action=edit
https://pt.wikibooks.org/wiki/Template:Lista_de_capítulos/Anterior?action=edit
https://pt.wikibooks.org/wiki/Template:Lista_de_capítulos/Imprimir?action=edit
https://pt.wikibooks.org/wiki/Template:Lista_de_capítulos/Coleção?action=edit
etc...?
Is that acceptable?
Those templates were created as a workaround for features which are (still)
missing on non-Wikipedia projects, such as these:
https://bugzilla.wikimedia.org/showdependencytree.cgi?id=15071&hide_resolved=1

It would be really great if on this GSoC 2012,
https://www.mediawiki.org/wiki/Summer_of_Code_2012
, some of the projects were focused on improving Wikibooks/Wikisources...
I hope someone gets interested.


On Mon, Jan 30, 2012 at 15:24, Roan Kattouw <[hidden email]> wrote:

> On Mon, Jan 30, 2012 at 3:11 PM, Alex Brollo <[hidden email]>
> wrote:
> > I'm using more and more #switch into templates, it's surprising how many
> > issues it can solve, and how much large arrays it can manage.  My
> questions
> > are:
> > 1. Is there a reasonable upper limit for the number of #switch options?
> > 2. Is there a difference in server load between a long list of #switch
> > options and a "tree" of options t.i. something as a nested #switch?
> > 3. Is there any drawback into a large use of #swtch-based templates?
> >
> I can at least tell you that the answer to #1 is probably no, and the
> answer to #3 is yes. There were some server issues last week that were
> blamed on ocwiki's use of templates with #switch statements with
> thousands of options. For instance,
> https://oc.wikipedia.org/w/index.php?title=Mod%C3%A8l:Altmejcom&action=edit
> has 36,611 cases, because it's essentially used as a database. That's
> the kind of thing that takes forever to parse.
>
> Roan
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Some questions about #switch

Roan Kattouw-2
On Mon, Jan 30, 2012 at 8:13 PM, Helder <[hidden email]> wrote:
> What about ptwikibooks' usage of
> https://pt.wikibooks.org/wiki/Template:Lista_de_capítulos/Posterior?action=edit
It looks like that finds the current page name in a list of pages,
then returns the next one in the list. I can see how that makes sense
for a navigation template, I guess. But does this really need to
support up to a thousand (!) parameters? Aren't these pages typically
numbered?

Roan

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Some questions about #switch

Helder .
On Mon, Jan 30, 2012 at 17:20, Roan Kattouw <[hidden email]> wrote:

> On Mon, Jan 30, 2012 at 8:13 PM, Helder <[hidden email]> wrote:
> > What about ptwikibooks' usage of
> >
> https://pt.wikibooks.org/wiki/Template:Lista_de_capítulos/Posterior?action=edit
> It looks like that finds the current page name in a list of pages,
> then returns the next one in the list. I can see how that makes sense
> for a navigation template, I guess.


That is correct. This template is used to generate links to the "next"
chapter of the books. See e.g. the blue bars on top and bottom of this
chapter:
https://pt.wikibooks.org/wiki/GIMP/Caixa_de_ferramentas


> But does this really need to support up to a thousand (!) parameters?


Well, I remember at least two books which have more than 500 chapters:
https://pt.wikibooks.org/wiki/Template:Lista_de_cap%C3%ADtulos/Log%C3%ADstica
https://pt.wikibooks.org/wiki/Template:Lista_de_cap%C3%ADtulos/Guia_do_Linux
and our cookbook has ~2000 chapters:
https://pt.wikibooks.org/wiki/Template:Lista_de_cap%C3%ADtulos/Livro_de_receitas


> Aren't these pages typically numbered?

Not really.
The chapter names are usually "descriptive names" (see examples in the
lists above), not just numbers.

Best regards,
Helder
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Some questions about #switch

Tim Starling-2
In reply to this post by Roan Kattouw-2
On 31/01/12 04:24, Roan Kattouw wrote:

> On Mon, Jan 30, 2012 at 3:11 PM, Alex Brollo <[hidden email]> wrote:
>> I'm using more and more #switch into templates, it's surprising how many
>> issues it can solve, and how much large arrays it can manage.  My questions
>> are:
>> 1. Is there a reasonable upper limit for the number of #switch options?
>> 2. Is there a difference in server load between a long list of #switch
>> options and a "tree" of options t.i. something as a nested #switch?
>> 3. Is there any drawback into a large use of #swtch-based templates?
>>
> I can at least tell you that the answer to #1 is probably no, and the
> answer to #3 is yes. There were some server issues last week that were
> blamed on ocwiki's use of templates with #switch statements with
> thousands of options. For instance,
> https://oc.wikipedia.org/w/index.php?title=Mod%C3%A8l:Altmejcom&action=edit
> has 36,611 cases, because it's essentially used as a database. That's
> the kind of thing that takes forever to parse.

Actually it's relatively fast to parse, but it uses hundreds of
megabytes of unaccounted memory and can even send the servers into swap.

I'm considering introducing a limit on #switch cases of 2000 or so per
article, to address this issue. No doubt many templates will break,
but it's important to protect our servers, and we've always
discouraged this kind of #switch application.

-- Tim Starling


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Some questions about #switch

Steven Walling
On Mon, Jan 30, 2012 at 3:23 PM, Tim Starling <[hidden email]>wrote:

> I'm considering introducing a limit on #switch cases of 2000 or so per
> article, to address this issue. No doubt many templates will break,
> but it's important to protect our servers, and we've always
> discouraged this kind of #switch application.
>

As a Wikipedian, I would be very happy to see a limitation imposed like
this. I think the silent majority of editors who want to see pages load and
save quickly/reliably outweighs the need for unlimited use of a very
complicated parser functions structure.

Steven
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Some questions about #switch

Nickanc Wikipedia
In reply to this post by Tim Starling-2
> I'm considering introducing a limit on #switch cases of 2000 or so per
> article, to address this issue. No doubt many templates will break,
> but it's important to protect our servers, and we've always
> discouraged this kind of #switch application.
>

In my opinion, we shall first check and list which templates will
break, suggesting to fix them as soon as possible. Because of so many
cases distincted, a 2000 cases #switch should be a overused template
inside its wiki. Then introducing the limit.

Nickanc
[[m:User:Nickanc]]

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Some questions about #switch

Happy Melon-2
On 31 January 2012 14:05, Nickanc Wikipedia <[hidden email]> wrote:

> > I'm considering introducing a limit on #switch cases of 2000 or so per
> > article, to address this issue. No doubt many templates will break,
> > but it's important to protect our servers, and we've always
> > discouraged this kind of #switch application.
> >
>
> In my opinion, we shall first check and list which templates will
> break, suggesting to fix them as soon as possible. Because of so many
> cases distincted, a 2000 cases #switch should be a overused template
> inside its wiki. Then introducing the limit.
>
> Nickanc
> [[m:User:Nickanc]]
>
>
Could this be implemented through a parser tracking category?  IIRC the
preprocessor generates other tracking cats when it hits its existing
limits, can it easily add a category *without* affecting the rendering?

--HM
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Some questions about #switch

b-jorsch
In reply to this post by Tim Starling-2
On Tue, Jan 31, 2012 at 10:23:39AM +1100, Tim Starling wrote:
>
> I'm considering introducing a limit on #switch cases of 2000 or so per
> article, to address this issue. No doubt many templates will break,
> but it's important to protect our servers, and we've always
> discouraged this kind of #switch application.

Is the problem too many switch cases on the page, or just too many
switch cases in any one switch? Does that proposed limit mean that 100
uses of a template with 20 #switch cases will break, too? Will the limit
count {{#switch:{{{1}}}|a|b|c=foo}} as 1 or 3 cases?

And will the obvious on-wiki fix of using nested #ifs just make things worse?

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Some questions about #switch

Aaron Schulz
In reply to this post by Steven Walling
+1. SERIOUSLY. This always comes to mind when these issues come up. Editors and readers (on cache miss) shouldn't have to suffer through this. We shouldn't forget the 99% :)
Reply | Threaded
Open this post in threaded view
|

Re: Some questions about #switch

MZMcBride-2
In reply to this post by Steven Walling
Steven Walling wrote:

> On Mon, Jan 30, 2012 at 3:23 PM, Tim Starling <[hidden email]>wrote:
>> I'm considering introducing a limit on #switch cases of 2000 or so per
>> article, to address this issue. No doubt many templates will break,
>> but it's important to protect our servers, and we've always
>> discouraged this kind of #switch application.
>
> As a Wikipedian, I would be very happy to see a limitation imposed like
> this. I think the silent majority of editors who want to see pages load and
> save quickly/reliably outweighs the need for unlimited use of a very
> complicated parser functions structure.

This is probably going to read like an attack, but it isn't one.

I continue to view the existence of "expensive parser functions" as a
failure. I don't believe that having users focus on such details is useful
or necessary and I think it largely distracts from any wiki's primary
mission. While these types of functions make a page render/load take longer
(and this unquestionably needs to be addressed), it's more than a vocal and
active minority who favor templates that can do automatic math conversions
and unit labeling, string parsing for automatic page title italics and the
like, and more. In some ways, if these templates weren't so popular, there'd
be no issue. But they're popular for a reason (even if some are only popular
among hardcore template nerds, the result is the same...).

As I recall, it wasn't an overwhelming amount of time that was needed to fix
#ifexist to batch its queries rather than imposing a limit on it (it was
#ifexist that brought the idea of expensive parser functions into existence,
as I remember it... I'm too lazy to hunt down revs). Implementing the latter
was a more interesting coding challenge, I think, and there was a thought
that the methodology used could lay the framework for other parser functions
to be limited going forward (which inevitably happened).

However, I believe this was the wrong approach then and continues to be the
wrong approach now. The time spent here developing ways to limit #switch
rather than fixing the underlying issue (lack of a proper way to do advanced
string manipulation, conversion, etc. in templates) seems like a waste of
time to me, particularly with a better way (Lua) now possibly on the actual
horizon.

And, for what it's worth, Aaron's comment about the 99% is kind of bullshit.
The 99% aren't editing and are (instead) hitting cache layers. They're
completely unaffected by the slow rendering time. If the 99% were getting
the slow rendering times, this issue would have been solved years ago. It's
because it only hits a subset of users (and a subset of articles) that it's
lingered for as long as it has.

MZMcBride



_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Some questions about #switch

Antoine Musso-3
In reply to this post by Roan Kattouw-2
Roan Kattouw wrote:
> https://oc.wikipedia.org/w/index.php?title=Mod%C3%A8l:Altmejcom&action=edit
> has 36,611 cases, because it's essentially used as a database.

Looks like a basic "key => value" store which seems to be a valid usage.

Could that use case be fulfilled by semantic mediawiki?
Should we create an extension that would let users create key => value
stores?


--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Some questions about #switch

Victor Vasiliev
On Wed, Feb 1, 2012 at 7:02 PM, Antoine Musso <[hidden email]> wrote:
> Looks like a basic "key => value" store which seems to be a valid usage.
>
> Could that use case be fulfilled by semantic mediawiki?
> Should we create an extension that would let users create key => value
> stores?
>

I assume that proper scripting language would be a solution, if not
proper, than at least partial.

--vvv

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Some questions about #switch

Mormegil
In reply to this post by Antoine Musso-3
On 1 February 2012 16:02, Antoine Musso <[hidden email]> wrote:
> Should we create an extension that would let users create key => value
> stores?

We already have that… See e.g. [[w:Category:Political party colour templates]].

-- [[cs:User:Mormegil | Petr Kadlec]]

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Some questions about #switch

Gabriel Wicke-2
On 02/01/2012 04:38 PM, Petr Kadlec wrote:
> On 1 February 2012 16:02, Antoine Musso <[hidden email]> wrote:
>> Should we create an extension that would let users create key => value
>> stores?
>
> We already have that… See e.g. [[w:Category:Political party colour templates]].

Or UI messages in the MediaWiki namespace, or.. the wiki in general ;)

Gabriel


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Some questions about #switch

Aaron Schulz
In reply to this post by MZMcBride-2
By 99%, I meant 99% of users don't care (much or at all) about the things these crazy templates tend to offer, they just want to read the article. I remember Domas complaining about this last hackathon when...fixing...ocwiki.

Any article that uses slow templates that take forever to render is hard to edit (since there is always a fresh parse). Making such pages hard to edit or slow to view for users who are logged in and might have a few custom preferences or otherwise have a generic cache miss is pretty disappointing. It also is discouraging to new editors trying to change the page. Maybe it's a result of the "don't care about performance" policy taken to extremes.

I've been pushing for Lua for months (rather than delay on JS vs Lua), and I'm glad it's gotten steam again. Hopefully it will make these problems moot. I'm also glad to see the increased focus on new editors in general (from the features team).
Reply | Threaded
Open this post in threaded view
|

Re: Some questions about #switch

Alex Brollo
Thanks for contributions.
Really I was going to use large switches, both as associative arrays and as
sets. I hoped, that algorithm was based simply on string search into the
code (I presume, it is possible, and I know how efficient is plain string
search in any decent language) but I guess, from your replies, that I was
wrong. :-(

I'll use strictly the 2000 items limit; I presume that - in occasional
cases - I will be forced to split larger switches into different
containers, after some string manipulation with padleft trick.

Well, it's hyronical  that wiki is based on a powerful database, while user
interface forces contributors to use any kind of dirty trick to simulate
basic, extremely simple database-like jobs ..... :-(

Alex
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l