[MediaWiki-l] Questions regarding MW Cache

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[MediaWiki-l] Questions regarding MW Cache

Huji Lee
I have a number of related questions regarding MediaWiki's caching
mechanism. I would be grateful if one could answer these please. The
assumption here is that a wiki is configured using the exact same caching
methods that Wikimedia uses.

Please feel free to provide links to https://www.mediawiki.org as well as
https://noc.wikimedia.org when specifying configuration variables in your
answers as I am sure this would benefit future readers of this thread as
well.

Q1: Page [[foo]] contains template {{bar}}. I edit that template, but the
page is not immediately updated. If I don't manually purge the cache of
that page, how long would it take for the page cache to be updated? Which
variable sets that time? And what is the value of that variable currently
at Wikimedia wikis?

Q2: Page [[foo]] contains template {{bar}}, and that template provides a
link to page [[norf]]. I edit the template such that it instead links to
page [[qux]]. However, when I go to Special:WhatLinksHere/norf I still see
[[foo]] listed. Even if I try ?action=purge on [[foo]], that behavior
doesn't change. The only way to change it is either to edit [[foo]] and
change its content, or to simply go to the edit page of [[foo]] and save
without changes.

Q2.A: If I don't do that null edit, how long would it take for the page
cache to be updated? Which variable sets that time? And what is the value
of that variable currently at Wikimedia wikis?

Q2.B: Why is it that ?action=purge doesn't update the pagelinks table?

Q2.C: Why is it that saving a page without any edits triggers any update
mechanism? (I would assume it'd be less costly overall to say "if diff is
null, don't do anything at all"; this question is more philosophical: why
would we not like the quoted behavior?

Q2.D: Does the Special:WhatLinksHere results come from a cache? In other
words, if I query the database directly right after I change the template
(as mentioned in Q2) but before I do any null edits, would it still show
the old page link from [[foo]] to [[norf]]? Or would it show the correct
link from [[foo]] to [[qux]] while Special:WhatLinksHere is still showing
the old link?

Q2.E: Is it possible to induce the same "resolution" through API? That is,
is it possible to try to save a page without any changes to its content via
API, and thereby forcefully update the page links entries for that page?

Q3: Assume that 100 pages contain template {{bar}}, and that template
provides a link to page [[norf]]. I edit the template such that it instead
links to page [[qux]]. Would that add any jobs to the job queue? If yes,
how many? Should I expect 100 new jobs to be added, because 100 page
cache's need updating?

Thanks in advance,

Huji
_______________________________________________
MediaWiki-l mailing list
To unsubscribe, go to:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding MW Cache

John Doe-27
When editing a template the resulting link updates go thru the Job Queue
https://www.mediawiki.org/wiki/Manual:Job_queue which has a lot of
variables which determine the speed that jobs get executed.

When you perform a null edit the page is reparsed, resulting in updates to
the link related tables (iwlink, external_links, page_links ect) which is
what the whatlinks here results. Normally the job queue forces a reparse as
needed.

a standard purge does not update the table, however a forcelinkupdate added
to the purge URL will achieve the same thing see
https://www.mediawiki.org/wiki/Manual:Parameters_to_index.php#Actions

As for job queue numbers, dont try and understand it, its a confusing mess.
A single job might affect many pages, might or might not spawn more jobs
and several other variable actions

On Sun, May 24, 2015 at 8:35 AM, Huji Lee <[hidden email]> wrote:

> I have a number of related questions regarding MediaWiki's caching
> mechanism. I would be grateful if one could answer these please. The
> assumption here is that a wiki is configured using the exact same caching
> methods that Wikimedia uses.
>
> Please feel free to provide links to https://www.mediawiki.org as well as
> https://noc.wikimedia.org when specifying configuration variables in your
> answers as I am sure this would benefit future readers of this thread as
> well.
>
> Q1: Page [[foo]] contains template {{bar}}. I edit that template, but the
> page is not immediately updated. If I don't manually purge the cache of
> that page, how long would it take for the page cache to be updated? Which
> variable sets that time? And what is the value of that variable currently
> at Wikimedia wikis?
>
> Q2: Page [[foo]] contains template {{bar}}, and that template provides a
> link to page [[norf]]. I edit the template such that it instead links to
> page [[qux]]. However, when I go to Special:WhatLinksHere/norf I still see
> [[foo]] listed. Even if I try ?action=purge on [[foo]], that behavior
> doesn't change. The only way to change it is either to edit [[foo]] and
> change its content, or to simply go to the edit page of [[foo]] and save
> without changes.
>
> Q2.A: If I don't do that null edit, how long would it take for the page
> cache to be updated? Which variable sets that time? And what is the value
> of that variable currently at Wikimedia wikis?
>
> Q2.B: Why is it that ?action=purge doesn't update the pagelinks table?
>
> Q2.C: Why is it that saving a page without any edits triggers any update
> mechanism? (I would assume it'd be less costly overall to say "if diff is
> null, don't do anything at all"; this question is more philosophical: why
> would we not like the quoted behavior?
>
> Q2.D: Does the Special:WhatLinksHere results come from a cache? In other
> words, if I query the database directly right after I change the template
> (as mentioned in Q2) but before I do any null edits, would it still show
> the old page link from [[foo]] to [[norf]]? Or would it show the correct
> link from [[foo]] to [[qux]] while Special:WhatLinksHere is still showing
> the old link?
>
> Q2.E: Is it possible to induce the same "resolution" through API? That is,
> is it possible to try to save a page without any changes to its content via
> API, and thereby forcefully update the page links entries for that page?
>
> Q3: Assume that 100 pages contain template {{bar}}, and that template
> provides a link to page [[norf]]. I edit the template such that it instead
> links to page [[qux]]. Would that add any jobs to the job queue? If yes,
> how many? Should I expect 100 new jobs to be added, because 100 page
> cache's need updating?
>
> Thanks in advance,
>
> Huji
> _______________________________________________
> MediaWiki-l mailing list
> To unsubscribe, go to:
> https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
>
_______________________________________________
MediaWiki-l mailing list
To unsubscribe, go to:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding MW Cache

Huji Lee
Great answers.

Here is a follow up question though: I edited a template used in thousands
of pages, yet the Job Queue for that wiki did NOT increase by a factor of
thousand. It simply increased from around 10 to around 30. And since then
(several days ago), the associated page links are not updated. I understand
that Job Queue is not easy to comprehend, but can you shed a light as to
why the job queue only grow that much, and why the links haven't been
updated since? (Note that job queue very quickly shrank to 10 or so again).

On Sun, May 24, 2015 at 8:47 AM, John <[hidden email]> wrote:

> When editing a template the resulting link updates go thru the Job Queue
> https://www.mediawiki.org/wiki/Manual:Job_queue which has a lot of
> variables which determine the speed that jobs get executed.
>
> When you perform a null edit the page is reparsed, resulting in updates to
> the link related tables (iwlink, external_links, page_links ect) which is
> what the whatlinks here results. Normally the job queue forces a reparse as
> needed.
>
> a standard purge does not update the table, however a forcelinkupdate added
> to the purge URL will achieve the same thing see
> https://www.mediawiki.org/wiki/Manual:Parameters_to_index.php#Actions
>
> As for job queue numbers, dont try and understand it, its a confusing mess.
> A single job might affect many pages, might or might not spawn more jobs
> and several other variable actions
>
> On Sun, May 24, 2015 at 8:35 AM, Huji Lee <[hidden email]> wrote:
>
> > I have a number of related questions regarding MediaWiki's caching
> > mechanism. I would be grateful if one could answer these please. The
> > assumption here is that a wiki is configured using the exact same caching
> > methods that Wikimedia uses.
> >
> > Please feel free to provide links to https://www.mediawiki.org as well
> as
> > https://noc.wikimedia.org when specifying configuration variables in
> your
> > answers as I am sure this would benefit future readers of this thread as
> > well.
> >
> > Q1: Page [[foo]] contains template {{bar}}. I edit that template, but the
> > page is not immediately updated. If I don't manually purge the cache of
> > that page, how long would it take for the page cache to be updated? Which
> > variable sets that time? And what is the value of that variable currently
> > at Wikimedia wikis?
> >
> > Q2: Page [[foo]] contains template {{bar}}, and that template provides a
> > link to page [[norf]]. I edit the template such that it instead links to
> > page [[qux]]. However, when I go to Special:WhatLinksHere/norf I still
> see
> > [[foo]] listed. Even if I try ?action=purge on [[foo]], that behavior
> > doesn't change. The only way to change it is either to edit [[foo]] and
> > change its content, or to simply go to the edit page of [[foo]] and save
> > without changes.
> >
> > Q2.A: If I don't do that null edit, how long would it take for the page
> > cache to be updated? Which variable sets that time? And what is the value
> > of that variable currently at Wikimedia wikis?
> >
> > Q2.B: Why is it that ?action=purge doesn't update the pagelinks table?
> >
> > Q2.C: Why is it that saving a page without any edits triggers any update
> > mechanism? (I would assume it'd be less costly overall to say "if diff is
> > null, don't do anything at all"; this question is more philosophical: why
> > would we not like the quoted behavior?
> >
> > Q2.D: Does the Special:WhatLinksHere results come from a cache? In other
> > words, if I query the database directly right after I change the template
> > (as mentioned in Q2) but before I do any null edits, would it still show
> > the old page link from [[foo]] to [[norf]]? Or would it show the correct
> > link from [[foo]] to [[qux]] while Special:WhatLinksHere is still showing
> > the old link?
> >
> > Q2.E: Is it possible to induce the same "resolution" through API? That
> is,
> > is it possible to try to save a page without any changes to its content
> via
> > API, and thereby forcefully update the page links entries for that page?
> >
> > Q3: Assume that 100 pages contain template {{bar}}, and that template
> > provides a link to page [[norf]]. I edit the template such that it
> instead
> > links to page [[qux]]. Would that add any jobs to the job queue? If yes,
> > how many? Should I expect 100 new jobs to be added, because 100 page
> > cache's need updating?
> >
> > Thanks in advance,
> >
> > Huji
> > _______________________________________________
> > MediaWiki-l mailing list
> > To unsubscribe, go to:
> > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> >
> _______________________________________________
> MediaWiki-l mailing list
> To unsubscribe, go to:
> https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
>
_______________________________________________
MediaWiki-l mailing list
To unsubscribe, go to:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding MW Cache

John Doe-27
Editing the template adds 1? Job to the queue that then updates everything.
If needed you can run/schedule runjobs.php in the maintenance folder. I
know smaller wikis sometimes have issues with the jobs not getting executed
fast enough

On Sunday, May 24, 2015, Huji Lee <[hidden email]> wrote:

> Great answers.
>
> Here is a follow up question though: I edited a template used in thousands
> of pages, yet the Job Queue for that wiki did NOT increase by a factor of
> thousand. It simply increased from around 10 to around 30. And since then
> (several days ago), the associated page links are not updated. I understand
> that Job Queue is not easy to comprehend, but can you shed a light as to
> why the job queue only grow that much, and why the links haven't been
> updated since? (Note that job queue very quickly shrank to 10 or so again).
>
> On Sun, May 24, 2015 at 8:47 AM, John <[hidden email]
> <javascript:;>> wrote:
>
> > When editing a template the resulting link updates go thru the Job Queue
> > https://www.mediawiki.org/wiki/Manual:Job_queue which has a lot of
> > variables which determine the speed that jobs get executed.
> >
> > When you perform a null edit the page is reparsed, resulting in updates
> to
> > the link related tables (iwlink, external_links, page_links ect) which is
> > what the whatlinks here results. Normally the job queue forces a reparse
> as
> > needed.
> >
> > a standard purge does not update the table, however a forcelinkupdate
> added
> > to the purge URL will achieve the same thing see
> > https://www.mediawiki.org/wiki/Manual:Parameters_to_index.php#Actions
> >
> > As for job queue numbers, dont try and understand it, its a confusing
> mess.
> > A single job might affect many pages, might or might not spawn more jobs
> > and several other variable actions
> >
> > On Sun, May 24, 2015 at 8:35 AM, Huji Lee <[hidden email]
> <javascript:;>> wrote:
> >
> > > I have a number of related questions regarding MediaWiki's caching
> > > mechanism. I would be grateful if one could answer these please. The
> > > assumption here is that a wiki is configured using the exact same
> caching
> > > methods that Wikimedia uses.
> > >
> > > Please feel free to provide links to https://www.mediawiki.org as well
> > as
> > > https://noc.wikimedia.org when specifying configuration variables in
> > your
> > > answers as I am sure this would benefit future readers of this thread
> as
> > > well.
> > >
> > > Q1: Page [[foo]] contains template {{bar}}. I edit that template, but
> the
> > > page is not immediately updated. If I don't manually purge the cache of
> > > that page, how long would it take for the page cache to be updated?
> Which
> > > variable sets that time? And what is the value of that variable
> currently
> > > at Wikimedia wikis?
> > >
> > > Q2: Page [[foo]] contains template {{bar}}, and that template provides
> a
> > > link to page [[norf]]. I edit the template such that it instead links
> to
> > > page [[qux]]. However, when I go to Special:WhatLinksHere/norf I still
> > see
> > > [[foo]] listed. Even if I try ?action=purge on [[foo]], that behavior
> > > doesn't change. The only way to change it is either to edit [[foo]] and
> > > change its content, or to simply go to the edit page of [[foo]] and
> save
> > > without changes.
> > >
> > > Q2.A: If I don't do that null edit, how long would it take for the page
> > > cache to be updated? Which variable sets that time? And what is the
> value
> > > of that variable currently at Wikimedia wikis?
> > >
> > > Q2.B: Why is it that ?action=purge doesn't update the pagelinks table?
> > >
> > > Q2.C: Why is it that saving a page without any edits triggers any
> update
> > > mechanism? (I would assume it'd be less costly overall to say "if diff
> is
> > > null, don't do anything at all"; this question is more philosophical:
> why
> > > would we not like the quoted behavior?
> > >
> > > Q2.D: Does the Special:WhatLinksHere results come from a cache? In
> other
> > > words, if I query the database directly right after I change the
> template
> > > (as mentioned in Q2) but before I do any null edits, would it still
> show
> > > the old page link from [[foo]] to [[norf]]? Or would it show the
> correct
> > > link from [[foo]] to [[qux]] while Special:WhatLinksHere is still
> showing
> > > the old link?
> > >
> > > Q2.E: Is it possible to induce the same "resolution" through API? That
> > is,
> > > is it possible to try to save a page without any changes to its content
> > via
> > > API, and thereby forcefully update the page links entries for that
> page?
> > >
> > > Q3: Assume that 100 pages contain template {{bar}}, and that template
> > > provides a link to page [[norf]]. I edit the template such that it
> > instead
> > > links to page [[qux]]. Would that add any jobs to the job queue? If
> yes,
> > > how many? Should I expect 100 new jobs to be added, because 100 page
> > > cache's need updating?
> > >
> > > Thanks in advance,
> > >
> > > Huji
> > > _______________________________________________
> > > MediaWiki-l mailing list
> > > To unsubscribe, go to:
> > > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> > >
> > _______________________________________________
> > MediaWiki-l mailing list
> > To unsubscribe, go to:
> > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> >
> _______________________________________________
> MediaWiki-l mailing list
> To unsubscribe, go to:
> https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
>
_______________________________________________
MediaWiki-l mailing list
To unsubscribe, go to:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding MW Cache

Huji Lee
Ok, got it.

Is there anyway to get a sense of how fast/slow jobs are being executed on
a wiki? For instance, on English Wikipedia, I can see how many jobs are in
the query using the API, but can I get a sense of the average time it gets
for a job to run?

On Sun, May 24, 2015 at 9:22 AM, John <[hidden email]> wrote:

> Editing the template adds 1? Job to the queue that then updates everything.
> If needed you can run/schedule runjobs.php in the maintenance folder. I
> know smaller wikis sometimes have issues with the jobs not getting executed
> fast enough
>
> On Sunday, May 24, 2015, Huji Lee <[hidden email]> wrote:
>
> > Great answers.
> >
> > Here is a follow up question though: I edited a template used in
> thousands
> > of pages, yet the Job Queue for that wiki did NOT increase by a factor of
> > thousand. It simply increased from around 10 to around 30. And since then
> > (several days ago), the associated page links are not updated. I
> understand
> > that Job Queue is not easy to comprehend, but can you shed a light as to
> > why the job queue only grow that much, and why the links haven't been
> > updated since? (Note that job queue very quickly shrank to 10 or so
> again).
> >
> > On Sun, May 24, 2015 at 8:47 AM, John <[hidden email]
> > <javascript:;>> wrote:
> >
> > > When editing a template the resulting link updates go thru the Job
> Queue
> > > https://www.mediawiki.org/wiki/Manual:Job_queue which has a lot of
> > > variables which determine the speed that jobs get executed.
> > >
> > > When you perform a null edit the page is reparsed, resulting in updates
> > to
> > > the link related tables (iwlink, external_links, page_links ect) which
> is
> > > what the whatlinks here results. Normally the job queue forces a
> reparse
> > as
> > > needed.
> > >
> > > a standard purge does not update the table, however a forcelinkupdate
> > added
> > > to the purge URL will achieve the same thing see
> > > https://www.mediawiki.org/wiki/Manual:Parameters_to_index.php#Actions
> > >
> > > As for job queue numbers, dont try and understand it, its a confusing
> > mess.
> > > A single job might affect many pages, might or might not spawn more
> jobs
> > > and several other variable actions
> > >
> > > On Sun, May 24, 2015 at 8:35 AM, Huji Lee <[hidden email]
> > <javascript:;>> wrote:
> > >
> > > > I have a number of related questions regarding MediaWiki's caching
> > > > mechanism. I would be grateful if one could answer these please. The
> > > > assumption here is that a wiki is configured using the exact same
> > caching
> > > > methods that Wikimedia uses.
> > > >
> > > > Please feel free to provide links to https://www.mediawiki.org as
> well
> > > as
> > > > https://noc.wikimedia.org when specifying configuration variables in
> > > your
> > > > answers as I am sure this would benefit future readers of this thread
> > as
> > > > well.
> > > >
> > > > Q1: Page [[foo]] contains template {{bar}}. I edit that template, but
> > the
> > > > page is not immediately updated. If I don't manually purge the cache
> of
> > > > that page, how long would it take for the page cache to be updated?
> > Which
> > > > variable sets that time? And what is the value of that variable
> > currently
> > > > at Wikimedia wikis?
> > > >
> > > > Q2: Page [[foo]] contains template {{bar}}, and that template
> provides
> > a
> > > > link to page [[norf]]. I edit the template such that it instead links
> > to
> > > > page [[qux]]. However, when I go to Special:WhatLinksHere/norf I
> still
> > > see
> > > > [[foo]] listed. Even if I try ?action=purge on [[foo]], that behavior
> > > > doesn't change. The only way to change it is either to edit [[foo]]
> and
> > > > change its content, or to simply go to the edit page of [[foo]] and
> > save
> > > > without changes.
> > > >
> > > > Q2.A: If I don't do that null edit, how long would it take for the
> page
> > > > cache to be updated? Which variable sets that time? And what is the
> > value
> > > > of that variable currently at Wikimedia wikis?
> > > >
> > > > Q2.B: Why is it that ?action=purge doesn't update the pagelinks
> table?
> > > >
> > > > Q2.C: Why is it that saving a page without any edits triggers any
> > update
> > > > mechanism? (I would assume it'd be less costly overall to say "if
> diff
> > is
> > > > null, don't do anything at all"; this question is more philosophical:
> > why
> > > > would we not like the quoted behavior?
> > > >
> > > > Q2.D: Does the Special:WhatLinksHere results come from a cache? In
> > other
> > > > words, if I query the database directly right after I change the
> > template
> > > > (as mentioned in Q2) but before I do any null edits, would it still
> > show
> > > > the old page link from [[foo]] to [[norf]]? Or would it show the
> > correct
> > > > link from [[foo]] to [[qux]] while Special:WhatLinksHere is still
> > showing
> > > > the old link?
> > > >
> > > > Q2.E: Is it possible to induce the same "resolution" through API?
> That
> > > is,
> > > > is it possible to try to save a page without any changes to its
> content
> > > via
> > > > API, and thereby forcefully update the page links entries for that
> > page?
> > > >
> > > > Q3: Assume that 100 pages contain template {{bar}}, and that template
> > > > provides a link to page [[norf]]. I edit the template such that it
> > > instead
> > > > links to page [[qux]]. Would that add any jobs to the job queue? If
> > yes,
> > > > how many? Should I expect 100 new jobs to be added, because 100 page
> > > > cache's need updating?
> > > >
> > > > Thanks in advance,
> > > >
> > > > Huji
> > > > _______________________________________________
> > > > MediaWiki-l mailing list
> > > > To unsubscribe, go to:
> > > > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> > > >
> > > _______________________________________________
> > > MediaWiki-l mailing list
> > > To unsubscribe, go to:
> > > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> > >
> > _______________________________________________
> > MediaWiki-l mailing list
> > To unsubscribe, go to:
> > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> >
> _______________________________________________
> MediaWiki-l mailing list
> To unsubscribe, go to:
> https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
>
_______________________________________________
MediaWiki-l mailing list
To unsubscribe, go to:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding MW Cache

John Doe-27
If you have database access you can look at job timestamp  to get an idea

On Sunday, May 24, 2015, Huji Lee <[hidden email]> wrote:

> Ok, got it.
>
> Is there anyway to get a sense of how fast/slow jobs are being executed on
> a wiki? For instance, on English Wikipedia, I can see how many jobs are in
> the query using the API, but can I get a sense of the average time it gets
> for a job to run?
>
> On Sun, May 24, 2015 at 9:22 AM, John <[hidden email]
> <javascript:;>> wrote:
>
> > Editing the template adds 1? Job to the queue that then updates
> everything.
> > If needed you can run/schedule runjobs.php in the maintenance folder. I
> > know smaller wikis sometimes have issues with the jobs not getting
> executed
> > fast enough
> >
> > On Sunday, May 24, 2015, Huji Lee <[hidden email] <javascript:;>>
> wrote:
> >
> > > Great answers.
> > >
> > > Here is a follow up question though: I edited a template used in
> > thousands
> > > of pages, yet the Job Queue for that wiki did NOT increase by a factor
> of
> > > thousand. It simply increased from around 10 to around 30. And since
> then
> > > (several days ago), the associated page links are not updated. I
> > understand
> > > that Job Queue is not easy to comprehend, but can you shed a light as
> to
> > > why the job queue only grow that much, and why the links haven't been
> > > updated since? (Note that job queue very quickly shrank to 10 or so
> > again).
> > >
> > > On Sun, May 24, 2015 at 8:47 AM, John <[hidden email]
> <javascript:;>
> > > <javascript:;>> wrote:
> > >
> > > > When editing a template the resulting link updates go thru the Job
> > Queue
> > > > https://www.mediawiki.org/wiki/Manual:Job_queue which has a lot of
> > > > variables which determine the speed that jobs get executed.
> > > >
> > > > When you perform a null edit the page is reparsed, resulting in
> updates
> > > to
> > > > the link related tables (iwlink, external_links, page_links ect)
> which
> > is
> > > > what the whatlinks here results. Normally the job queue forces a
> > reparse
> > > as
> > > > needed.
> > > >
> > > > a standard purge does not update the table, however a forcelinkupdate
> > > added
> > > > to the purge URL will achieve the same thing see
> > > >
> https://www.mediawiki.org/wiki/Manual:Parameters_to_index.php#Actions
> > > >
> > > > As for job queue numbers, dont try and understand it, its a confusing
> > > mess.
> > > > A single job might affect many pages, might or might not spawn more
> > jobs
> > > > and several other variable actions
> > > >
> > > > On Sun, May 24, 2015 at 8:35 AM, Huji Lee <[hidden email]
> <javascript:;>
> > > <javascript:;>> wrote:
> > > >
> > > > > I have a number of related questions regarding MediaWiki's caching
> > > > > mechanism. I would be grateful if one could answer these please.
> The
> > > > > assumption here is that a wiki is configured using the exact same
> > > caching
> > > > > methods that Wikimedia uses.
> > > > >
> > > > > Please feel free to provide links to https://www.mediawiki.org as
> > well
> > > > as
> > > > > https://noc.wikimedia.org when specifying configuration variables
> in
> > > > your
> > > > > answers as I am sure this would benefit future readers of this
> thread
> > > as
> > > > > well.
> > > > >
> > > > > Q1: Page [[foo]] contains template {{bar}}. I edit that template,
> but
> > > the
> > > > > page is not immediately updated. If I don't manually purge the
> cache
> > of
> > > > > that page, how long would it take for the page cache to be updated?
> > > Which
> > > > > variable sets that time? And what is the value of that variable
> > > currently
> > > > > at Wikimedia wikis?
> > > > >
> > > > > Q2: Page [[foo]] contains template {{bar}}, and that template
> > provides
> > > a
> > > > > link to page [[norf]]. I edit the template such that it instead
> links
> > > to
> > > > > page [[qux]]. However, when I go to Special:WhatLinksHere/norf I
> > still
> > > > see
> > > > > [[foo]] listed. Even if I try ?action=purge on [[foo]], that
> behavior
> > > > > doesn't change. The only way to change it is either to edit [[foo]]
> > and
> > > > > change its content, or to simply go to the edit page of [[foo]] and
> > > save
> > > > > without changes.
> > > > >
> > > > > Q2.A: If I don't do that null edit, how long would it take for the
> > page
> > > > > cache to be updated? Which variable sets that time? And what is the
> > > value
> > > > > of that variable currently at Wikimedia wikis?
> > > > >
> > > > > Q2.B: Why is it that ?action=purge doesn't update the pagelinks
> > table?
> > > > >
> > > > > Q2.C: Why is it that saving a page without any edits triggers any
> > > update
> > > > > mechanism? (I would assume it'd be less costly overall to say "if
> > diff
> > > is
> > > > > null, don't do anything at all"; this question is more
> philosophical:
> > > why
> > > > > would we not like the quoted behavior?
> > > > >
> > > > > Q2.D: Does the Special:WhatLinksHere results come from a cache? In
> > > other
> > > > > words, if I query the database directly right after I change the
> > > template
> > > > > (as mentioned in Q2) but before I do any null edits, would it still
> > > show
> > > > > the old page link from [[foo]] to [[norf]]? Or would it show the
> > > correct
> > > > > link from [[foo]] to [[qux]] while Special:WhatLinksHere is still
> > > showing
> > > > > the old link?
> > > > >
> > > > > Q2.E: Is it possible to induce the same "resolution" through API?
> > That
> > > > is,
> > > > > is it possible to try to save a page without any changes to its
> > content
> > > > via
> > > > > API, and thereby forcefully update the page links entries for that
> > > page?
> > > > >
> > > > > Q3: Assume that 100 pages contain template {{bar}}, and that
> template
> > > > > provides a link to page [[norf]]. I edit the template such that it
> > > > instead
> > > > > links to page [[qux]]. Would that add any jobs to the job queue? If
> > > yes,
> > > > > how many? Should I expect 100 new jobs to be added, because 100
> page
> > > > > cache's need updating?
> > > > >
> > > > > Thanks in advance,
> > > > >
> > > > > Huji
> > > > > _______________________________________________
> > > > > MediaWiki-l mailing list
> > > > > To unsubscribe, go to:
> > > > > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> > > > >
> > > > _______________________________________________
> > > > MediaWiki-l mailing list
> > > > To unsubscribe, go to:
> > > > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> > > >
> > > _______________________________________________
> > > MediaWiki-l mailing list
> > > To unsubscribe, go to:
> > > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> > >
> > _______________________________________________
> > MediaWiki-l mailing list
> > To unsubscribe, go to:
> > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> >
> _______________________________________________
> MediaWiki-l mailing list
> To unsubscribe, go to:
> https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
>
_______________________________________________
MediaWiki-l mailing list
To unsubscribe, go to:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding MW Cache

Huji Lee
Which table would I look at?

Also, Manu:Job queue
<https://www.mediawiki.org/wiki/Manual:Job_queue#Updating_links_tables_when_a_template_changes>
says "When a template changes, MediaWiki adds a job to the job queue for
each article transcluding that template" which is contradicting what we
already agreed on. And also making me wonder why 1000+ jobs were not
created when I edited a template with that many transclusions.

On Sun, May 24, 2015 at 1:56 PM, John <[hidden email]> wrote:

> If you have database access you can look at job timestamp  to get an idea
>
> On Sunday, May 24, 2015, Huji Lee <[hidden email]> wrote:
>
> > Ok, got it.
> >
> > Is there anyway to get a sense of how fast/slow jobs are being executed
> on
> > a wiki? For instance, on English Wikipedia, I can see how many jobs are
> in
> > the query using the API, but can I get a sense of the average time it
> gets
> > for a job to run?
> >
> > On Sun, May 24, 2015 at 9:22 AM, John <[hidden email]
> > <javascript:;>> wrote:
> >
> > > Editing the template adds 1? Job to the queue that then updates
> > everything.
> > > If needed you can run/schedule runjobs.php in the maintenance folder. I
> > > know smaller wikis sometimes have issues with the jobs not getting
> > executed
> > > fast enough
> > >
> > > On Sunday, May 24, 2015, Huji Lee <[hidden email] <javascript:;>>
> > wrote:
> > >
> > > > Great answers.
> > > >
> > > > Here is a follow up question though: I edited a template used in
> > > thousands
> > > > of pages, yet the Job Queue for that wiki did NOT increase by a
> factor
> > of
> > > > thousand. It simply increased from around 10 to around 30. And since
> > then
> > > > (several days ago), the associated page links are not updated. I
> > > understand
> > > > that Job Queue is not easy to comprehend, but can you shed a light as
> > to
> > > > why the job queue only grow that much, and why the links haven't been
> > > > updated since? (Note that job queue very quickly shrank to 10 or so
> > > again).
> > > >
> > > > On Sun, May 24, 2015 at 8:47 AM, John <[hidden email]
> > <javascript:;>
> > > > <javascript:;>> wrote:
> > > >
> > > > > When editing a template the resulting link updates go thru the Job
> > > Queue
> > > > > https://www.mediawiki.org/wiki/Manual:Job_queue which has a lot of
> > > > > variables which determine the speed that jobs get executed.
> > > > >
> > > > > When you perform a null edit the page is reparsed, resulting in
> > updates
> > > > to
> > > > > the link related tables (iwlink, external_links, page_links ect)
> > which
> > > is
> > > > > what the whatlinks here results. Normally the job queue forces a
> > > reparse
> > > > as
> > > > > needed.
> > > > >
> > > > > a standard purge does not update the table, however a
> forcelinkupdate
> > > > added
> > > > > to the purge URL will achieve the same thing see
> > > > >
> > https://www.mediawiki.org/wiki/Manual:Parameters_to_index.php#Actions
> > > > >
> > > > > As for job queue numbers, dont try and understand it, its a
> confusing
> > > > mess.
> > > > > A single job might affect many pages, might or might not spawn more
> > > jobs
> > > > > and several other variable actions
> > > > >
> > > > > On Sun, May 24, 2015 at 8:35 AM, Huji Lee <[hidden email]
> > <javascript:;>
> > > > <javascript:;>> wrote:
> > > > >
> > > > > > I have a number of related questions regarding MediaWiki's
> caching
> > > > > > mechanism. I would be grateful if one could answer these please.
> > The
> > > > > > assumption here is that a wiki is configured using the exact same
> > > > caching
> > > > > > methods that Wikimedia uses.
> > > > > >
> > > > > > Please feel free to provide links to https://www.mediawiki.org
> as
> > > well
> > > > > as
> > > > > > https://noc.wikimedia.org when specifying configuration
> variables
> > in
> > > > > your
> > > > > > answers as I am sure this would benefit future readers of this
> > thread
> > > > as
> > > > > > well.
> > > > > >
> > > > > > Q1: Page [[foo]] contains template {{bar}}. I edit that template,
> > but
> > > > the
> > > > > > page is not immediately updated. If I don't manually purge the
> > cache
> > > of
> > > > > > that page, how long would it take for the page cache to be
> updated?
> > > > Which
> > > > > > variable sets that time? And what is the value of that variable
> > > > currently
> > > > > > at Wikimedia wikis?
> > > > > >
> > > > > > Q2: Page [[foo]] contains template {{bar}}, and that template
> > > provides
> > > > a
> > > > > > link to page [[norf]]. I edit the template such that it instead
> > links
> > > > to
> > > > > > page [[qux]]. However, when I go to Special:WhatLinksHere/norf I
> > > still
> > > > > see
> > > > > > [[foo]] listed. Even if I try ?action=purge on [[foo]], that
> > behavior
> > > > > > doesn't change. The only way to change it is either to edit
> [[foo]]
> > > and
> > > > > > change its content, or to simply go to the edit page of [[foo]]
> and
> > > > save
> > > > > > without changes.
> > > > > >
> > > > > > Q2.A: If I don't do that null edit, how long would it take for
> the
> > > page
> > > > > > cache to be updated? Which variable sets that time? And what is
> the
> > > > value
> > > > > > of that variable currently at Wikimedia wikis?
> > > > > >
> > > > > > Q2.B: Why is it that ?action=purge doesn't update the pagelinks
> > > table?
> > > > > >
> > > > > > Q2.C: Why is it that saving a page without any edits triggers any
> > > > update
> > > > > > mechanism? (I would assume it'd be less costly overall to say "if
> > > diff
> > > > is
> > > > > > null, don't do anything at all"; this question is more
> > philosophical:
> > > > why
> > > > > > would we not like the quoted behavior?
> > > > > >
> > > > > > Q2.D: Does the Special:WhatLinksHere results come from a cache?
> In
> > > > other
> > > > > > words, if I query the database directly right after I change the
> > > > template
> > > > > > (as mentioned in Q2) but before I do any null edits, would it
> still
> > > > show
> > > > > > the old page link from [[foo]] to [[norf]]? Or would it show the
> > > > correct
> > > > > > link from [[foo]] to [[qux]] while Special:WhatLinksHere is still
> > > > showing
> > > > > > the old link?
> > > > > >
> > > > > > Q2.E: Is it possible to induce the same "resolution" through API?
> > > That
> > > > > is,
> > > > > > is it possible to try to save a page without any changes to its
> > > content
> > > > > via
> > > > > > API, and thereby forcefully update the page links entries for
> that
> > > > page?
> > > > > >
> > > > > > Q3: Assume that 100 pages contain template {{bar}}, and that
> > template
> > > > > > provides a link to page [[norf]]. I edit the template such that
> it
> > > > > instead
> > > > > > links to page [[qux]]. Would that add any jobs to the job queue?
> If
> > > > yes,
> > > > > > how many? Should I expect 100 new jobs to be added, because 100
> > page
> > > > > > cache's need updating?
> > > > > >
> > > > > > Thanks in advance,
> > > > > >
> > > > > > Huji
> > > > > > _______________________________________________
> > > > > > MediaWiki-l mailing list
> > > > > > To unsubscribe, go to:
> > > > > > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> > > > > >
> > > > > _______________________________________________
> > > > > MediaWiki-l mailing list
> > > > > To unsubscribe, go to:
> > > > > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> > > > >
> > > > _______________________________________________
> > > > MediaWiki-l mailing list
> > > > To unsubscribe, go to:
> > > > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> > > >
> > > _______________________________________________
> > > MediaWiki-l mailing list
> > > To unsubscribe, go to:
> > > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> > >
> > _______________________________________________
> > MediaWiki-l mailing list
> > To unsubscribe, go to:
> > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> >
> _______________________________________________
> MediaWiki-l mailing list
> To unsubscribe, go to:
> https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
>
_______________________________________________
MediaWiki-l mailing list
To unsubscribe, go to:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l