[MediaWiki-l] Objectcache table - records expiry time 2038

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[MediaWiki-l] Objectcache table - records expiry time 2038

John Horne
Hello,

We are running an old version of mediawiki (1.24.2), and have noticed recently
that overnight we are seeing that the disk utilisation is reaching nearly 100%
for almost an hour. During the night, at different times, we run a database
backup (mysqldump) and a database check (mysqlcheck). It turns out that these
are causing the disk usage problem.

In particular the database table 'objectcache' has nearly 500,000 entries.
Whilst the table has an expiry date/time field for the records, all but about
10 of the records have the expiry date set to the year 2038!

Looking at the key names for some of the records they seem to be of the form:
'SM:resourceloader:filter:minify-css:7:066ff155efe7761cc6472668436b9554'.

My questions are:

1) Is it right that 'minify-css' records should have a very long expiry date
(such that they are basically going to accumulate over time)?

2) The mediawiki docs say that it is possible to purge the table as required
since the records will be recreated. Will this have any effect on someone who
is using the wiki at the time the table is purged?


Current plans are to modify our backup script to exclude the table, but include
its 'structure' into the backup file. (Kudos to AmitP's answer at
https://stackoverflow.com/questions/13593148/mysql-dump-exclude-some-table-data
). In addition, our database check script will perform a table purge beforehand
if it is safe to do so.
(Plans also include updating the mediawiki version!)



Thanks,

John.

--
John Horne | Senior Operations Analyst | Technology and Information Services
University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK
________________________________
[http://www.plymouth.ac.uk/images/email_footer.gif]<http://www.plymouth.ac.uk/worldclass>

This email and any files with it are confidential and intended solely for the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, Plymouth University accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. Plymouth University does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.
_______________________________________________
MediaWiki-l mailing list
To unsubscribe, go to:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
Reply | Threaded
Open this post in threaded view
|

Re: Objectcache table - records expiry time 2038

Brian Wolff
Generally speaking objectcache is used as a last resort form of caching if
you dont have apcu, memcached, redis, or something else configured.

Truncating the contents of the table will make the next couple of requests
to mediawiki slower as some of the cache entries will need to be rebuilt.
But it probably wont be horrible as long as you dont do it too often. Id
reccomend experimenting during off peak hours so you can see what the
performance impact is.

In the long run, I would reccomend installing apcu so that cache stuff is
stored there instead of the db.

--
Brian

On Tuesday, February 6, 2018, John Horne <[hidden email]> wrote:
> Hello,
>
> We are running an old version of mediawiki (1.24.2), and have noticed
recently
> that overnight we are seeing that the disk utilisation is reaching nearly
100%
> for almost an hour. During the night, at different times, we run a
database
> backup (mysqldump) and a database check (mysqlcheck). It turns out that
these
> are causing the disk usage problem.
>
> In particular the database table 'objectcache' has nearly 500,000 entries.
> Whilst the table has an expiry date/time field for the records, all but
about
> 10 of the records have the expiry date set to the year 2038!
>
> Looking at the key names for some of the records they seem to be of the
form:
> 'SM:resourceloader:filter:minify-css:7:066ff155efe7761cc6472668436b9554'.
>
> My questions are:
>
> 1) Is it right that 'minify-css' records should have a very long expiry
date
> (such that they are basically going to accumulate over time)?
>
> 2) The mediawiki docs say that it is possible to purge the table as
required
> since the records will be recreated. Will this have any effect on someone
who
> is using the wiki at the time the table is purged?
>
>
> Current plans are to modify our backup script to exclude the table, but
include
> its 'structure' into the backup file. (Kudos to AmitP's answer at
>
https://stackoverflow.com/questions/13593148/mysql-dump-exclude-some-table-data
> ). In addition, our database check script will perform a table purge
beforehand

> if it is safe to do so.
> (Plans also include updating the mediawiki version!)
>
>
>
> Thanks,
>
> John.
>
> --
> John Horne | Senior Operations Analyst | Technology and Information
Services
> University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK
> ________________________________
> [http://www.plymouth.ac.uk/images/email_footer.gif]<
http://www.plymouth.ac.uk/worldclass>
>
> This email and any files with it are confidential and intended solely for
the use of the recipient to whom it is addressed. If you are not the
intended recipient then copying, distribution or other use of the
information contained is strictly prohibited and you should not rely on it.
If you have received this email in error please let the sender know
immediately and delete it from your system(s). Internet emails are not
necessarily secure. While we take every care, Plymouth University accepts
no responsibility for viruses and it is your responsibility to scan emails
and their attachments. Plymouth University does not accept responsibility
for any changes made after it was sent. Nothing in this email or its
attachments constitutes an order for goods or services unless accompanied
by an official order form.
> _______________________________________________
> MediaWiki-l mailing list
> To unsubscribe, go to:
> https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
>
_______________________________________________
MediaWiki-l mailing list
To unsubscribe, go to:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
Reply | Threaded
Open this post in threaded view
|

Re: Objectcache table - records expiry time 2038

John Horne
On Tue, 2018-02-06 at 15:33 -0500, Brian Wolff wrote:
> Generally speaking objectcache is used as a last resort form of caching if
> you dont have apcu, memcached, redis, or something else configured.
>
...
>
> In the long run, I would reccomend installing apcu so that cache stuff is
> stored there instead of the db.
>
Hello,

Thanks for the reply.

At the moment we have the main cache type set to CACHE_NONE, but php does have
apc enabled (not sure why we didn't use it). We are running a relatively small
wiki, but I think we may install memcached (and increase the server memory a
bit). We run memcached on some other, non-wiki, servers with no problem.

I note that with MW 1.27+ there seems to be an issue using apc as the main
cache and with session caching. For that reason using memcache may be easier at
the moment, and not cause problems when we upgrade mediawiki.



John.

--
John Horne | Senior Operations Analyst | Technology and Information Services
University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK
________________________________
[http://www.plymouth.ac.uk/images/email_footer.gif]<http://www.plymouth.ac.uk/worldclass>

This email and any files with it are confidential and intended solely for the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, Plymouth University accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. Plymouth University does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.
_______________________________________________
MediaWiki-l mailing list
To unsubscribe, go to:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
Reply | Threaded
Open this post in threaded view
|

Re: Objectcache table - records expiry time 2038

Brian Wolff
On Wednesday, February 7, 2018, John Horne <[hidden email]>
wrote:
> On Tue, 2018-02-06 at 15:33 -0500, Brian Wolff wrote:
>> Generally speaking objectcache is used as a last resort form of caching
if

>> you dont have apcu, memcached, redis, or something else configured.
>>
> ...
>>
>> In the long run, I would reccomend installing apcu so that cache stuff is
>> stored there instead of the db.
>>
> Hello,
>
> Thanks for the reply.
>
> At the moment we have the main cache type set to CACHE_NONE, but php does
have
> apc enabled (not sure why we didn't use it). We are running a relatively
small
> wiki, but I think we may install memcached (and increase the server
memory a
> bit). We run memcached on some other, non-wiki, servers with no problem.
>
> I note that with MW 1.27+ there seems to be an issue using apc as the main
> cache and with session caching. For that reason using memcache may be
easier at
> the moment, and not cause problems when we upgrade mediawiki.
>
>

Fwiw, i dont think theres actually any bugs with session handling at apcu
in 1.27 - just that if apcu isnt working properly (e.g. apcu.shm_size too
small or any other situation where apcu doesnt actually store values long
enough) then sessions wont work (the other caching stuff wont work either
but sessions being broken is much more noticable than site being "slow").
But memcached is probably the best caching approach so your plan sounds
like a good idea.

One side effect of killing objectcache i didnt think of before is it will
log everyone out.

--
brian

> John.
>
> --
> John Horne | Senior Operations Analyst | Technology and Information
Services
> University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK
> ________________________________
> [http://www.plymouth.ac.uk/images/email_footer.gif]<
http://www.plymouth.ac.uk/worldclass>
>
> This email and any files with it are confidential and intended solely for
the use of the recipient to whom it is addressed. If you are not the
intended recipient then copying, distribution or other use of the
information contained is strictly prohibited and you should not rely on it.
If you have received this email in error please let the sender know
immediately and delete it from your system(s). Internet emails are not
necessarily secure. While we take every care, Plymouth University accepts
no responsibility for viruses and it is your responsibility to scan emails
and their attachments. Plymouth University does not accept responsibility
for any changes made after it was sent. Nothing in this email or its
attachments constitutes an order for goods or services unless accompanied
by an official order form.
> _______________________________________________
> MediaWiki-l mailing list
> To unsubscribe, go to:
> https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
>
_______________________________________________
MediaWiki-l mailing list
To unsubscribe, go to:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
Reply | Threaded
Open this post in threaded view
|

Re: Objectcache table - records expiry time 2038

Chad
In reply to this post by Brian Wolff
On Tue, Feb 6, 2018 at 12:34 PM Brian Wolff <[hidden email]> wrote:

> Generally speaking objectcache is used as a last resort form of caching if
> you dont have apcu, memcached, redis, or something else configured.
>
>
I wonder if swapping it for a MEMORY table would be beneficial? Then it'd
behave much more like the other object cache options (apcu.shm_size is
max_heap_table_size, doesn't survive restarts, etc...)

Filed a task to follow-up: https://phabricator.wikimedia.org/T186752

-Chad
_______________________________________________
MediaWiki-l mailing list
To unsubscribe, go to:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l