Deprecation logging in production

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Deprecation logging in production

Erik Bernhardson
Noticed earlier today, because some code that had been deprecated since
1.21 was removed and starting causing fatals, that we don't log deprecation
notices in production. This has now been fixed.

Please check logstash for `channel:deprecated` and see if anything you are
reposinsible for is using hard-deprecated methods and fix them.

Some culprits which have each generated hundreds of messages in the ~5
minutes it's been turned on:

Use of SpecialRecentChangesQuery hook (used in
FlaggedRevsUIHooks::modifyRecentChangesQuery) was deprecated in MediaWiki
1.23. [Called from SpecialRecentChanges::runMainQueryHook in
/srv/mediawiki/php-1.29.0-wmf.8/includes/specials/SpecialRecentchanges.ph

Use of wfSetupSession was deprecated in MediaWiki 1.27. [Called from
CollectionSession::startSession in
/srv/mediawiki/php-1.29.0-wmf.8/extensions/Collection/Collection.session.php
at line 38]

Use of SpecialWatchlistQuery hook (used in
FlaggedRevsUIHooks::modifyChangesListQuery) was deprecated in MediaWiki
1.23. [Called from SpecialWatchlist::runMainQueryHook in
/srv/mediawiki/php-1.29.0-wmf.8/includes/specials/SpecialWatchlist.php at
line 309]

Use of Revision::getText was deprecated in MediaWiki 1.21. [Called from
FlaggedRevision::getRevText in
/srv/mediawiki/php-1.29.0-wmf.8/extensions/FlaggedRevs/backend/FlaggedRevision.php
at line 480]

Use of SpecialRecentChangesQuery hook (used in
FlaggedRevsUIHooks::modifyRecentChangesQuery) was deprecated in MediaWiki
1.23. [Called from SpecialRecentChanges::runMainQueryHook in
/srv/mediawiki/php-1.29.0-wmf.9/includes/specials/SpecialRecentchanges.ph
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Deprecation logging in production

Max Semenik
Yay, this is really cool!

Current warnings sorted by popularity:

maxsem@fluorine:/a/mw-log$ cat deprecated.log | grep -oP '(?<=INFO: ).*' |
sort | uniq -c | sort -rn
  20598 Use of SpecialRecentChangesQuery hook (used in
FlaggedRevsUIHooks::modifyRecentChangesQuery) was deprecated in MediaWiki
1.23. [Called from SpecialRecentChanges::runMainQueryHook in
/srv/mediawiki/php-1.29.0-wmf.8/includes/specials/SpecialRecentchanges.php
at line 287]
   2705 Use of wfSetupSession was deprecated in MediaWiki 1.27. [Called
from CollectionSession::startSession in
/srv/mediawiki/php-1.29.0-wmf.8/extensions/Collection/Collection.session.php
at line 38]
   2271 Use of SpecialWatchlistQuery hook (used in
FlaggedRevsUIHooks::modifyChangesListQuery) was deprecated in MediaWiki
1.23. [Called from SpecialWatchlist::runMainQueryHook in
/srv/mediawiki/php-1.29.0-wmf.8/includes/specials/SpecialWatchlist.php at
line 309]
   2094 Use of Revision::getText was deprecated in MediaWiki 1.21. [Called
from FlaggedRevision::getRevText in
/srv/mediawiki/php-1.29.0-wmf.8/extensions/FlaggedRevs/backend/FlaggedRevision.php
at line 480]
   1821 Use of SpecialRecentChangesQuery hook (used in
FlaggedRevsUIHooks::modifyRecentChangesQuery) was deprecated in MediaWiki
1.23. [Called from SpecialRecentChanges::runMainQueryHook in
/srv/mediawiki/php-1.29.0-wmf.9/includes/specials/SpecialRecentchanges.php
at line 287]
    189 Use of SyntaxHighlight_GeSHi::prepare is deprecated. [Called from
ScribuntoContent::fillParserOutput in
/srv/mediawiki/php-1.29.0-wmf.8/extensions/Scribunto/common/ScribuntoContent.php
at line 128]
    189 Use of SyntaxHighlight_GeSHi::buildHeadItem is deprecated. [Called
from ScribuntoContent::fillParserOutput in
/srv/mediawiki/php-1.29.0-wmf.8/extensions/Scribunto/common/ScribuntoContent.php
at line 141]
    132 Use of SyntaxHighlight_GeSHi::prepare is deprecated. [Called from
ScribuntoContent::fillParserOutput in
/srv/mediawiki/php-1.29.0-wmf.9/extensions/Scribunto/common/ScribuntoContent.php
at line 128]
    132 Use of SyntaxHighlight_GeSHi::buildHeadItem is deprecated. [Called
from ScribuntoContent::fillParserOutput in
/srv/mediawiki/php-1.29.0-wmf.9/extensions/Scribunto/common/ScribuntoContent.php
at line 141]
     92 Use of ExtractThumbParameters was deprecated in MediaWiki 1.22.
[Called from wfStreamThumb in /srv/mediawiki/php-1.29.0-wmf.9/thumb.php at
line 174]
     70 Use of Revision::getRawUser was deprecated in MediaWiki 1.25.
[Called from FlaggedRevsHooks::incrementReverts in
/srv/mediawiki/php-1.29.0-wmf.8/extensions/FlaggedRevs/backend/FlaggedRevs.hooks.php
at line 702]
     30 Use of wfSetupSession was deprecated in MediaWiki 1.27. [Called
from CollectionSession::startSession in
/srv/mediawiki/php-1.29.0-wmf.9/extensions/Collection/Collection.session.php
at line 38]
     20 Use of wfSetupSession was deprecated in MediaWiki 1.27. [Called
from AbuseFilter::executeFilterActions in
/srv/mediawiki/php-1.29.0-wmf.8/extensions/AbuseFilter/includes/AbuseFilter.class.php
at line 796]
     18 Use of Revision::getRawUserText was deprecated in MediaWiki 1.25.
[Called from FlaggablePageView::addToDiffView in
/srv/mediawiki/php-1.29.0-wmf.8/extensions/FlaggedRevs/frontend/FlaggablePageView.php
at line 1498]
     13 Use of SpecialWatchlistQuery hook (used in
FlaggedRevsUIHooks::modifyChangesListQuery) was deprecated in MediaWiki
1.23. [Called from SpecialWatchlist::runMainQueryHook in
/srv/mediawiki/php-1.29.0-wmf.9/includes/specials/SpecialWatchlist.php at
line 309]
     11 Use of ExtractThumbParameters was deprecated in MediaWiki 1.22.
[Called from wfStreamThumb in /srv/mediawiki/php-1.29.0-wmf.8/thumb.php at
line 174]
      4 Use of Linker::link with parameter $query as string (should be
array) was deprecated in MediaWiki 1.20. [Called from Linker::linkKnown in
/srv/mediawiki/php-1.29.0-wmf.8/includes/Linker.php at line 163]
      3 Use of wfSetupSession was deprecated in MediaWiki 1.27. [Called
from wfAjaxPostCollection in
/srv/mediawiki/php-1.29.0-wmf.8/extensions/Collection/Collection.php at
line 268]
      2 Use of wfSetupSession was deprecated in MediaWiki 1.27. [Called
from AbuseFilter::executeFilterActions in
/srv/mediawiki/php-1.29.0-wmf.9/extensions/AbuseFilter/includes/AbuseFilter.class.php
at line 796]

On Wed, Jan 25, 2017 at 4:18 PM, Erik Bernhardson <
[hidden email]> wrote:

> Noticed earlier today, because some code that had been deprecated since
> 1.21 was removed and starting causing fatals, that we don't log deprecation
> notices in production. This has now been fixed.
>
> Please check logstash for `channel:deprecated` and see if anything you are
> reposinsible for is using hard-deprecated methods and fix them.
>
> Some culprits which have each generated hundreds of messages in the ~5
> minutes it's been turned on:
>
> Use of SpecialRecentChangesQuery hook (used in
> FlaggedRevsUIHooks::modifyRecentChangesQuery) was deprecated in MediaWiki
> 1.23. [Called from SpecialRecentChanges::runMainQueryHook in
> /srv/mediawiki/php-1.29.0-wmf.8/includes/specials/SpecialRecentchanges.ph
>
> Use of wfSetupSession was deprecated in MediaWiki 1.27. [Called from
> CollectionSession::startSession in
> /srv/mediawiki/php-1.29.0-wmf.8/extensions/Collection/
> Collection.session.php
> at line 38]
>
> Use of SpecialWatchlistQuery hook (used in
> FlaggedRevsUIHooks::modifyChangesListQuery) was deprecated in MediaWiki
> 1.23. [Called from SpecialWatchlist::runMainQueryHook in
> /srv/mediawiki/php-1.29.0-wmf.8/includes/specials/SpecialWatchlist.php at
> line 309]
>
> Use of Revision::getText was deprecated in MediaWiki 1.21. [Called from
> FlaggedRevision::getRevText in
> /srv/mediawiki/php-1.29.0-wmf.8/extensions/FlaggedRevs/
> backend/FlaggedRevision.php
> at line 480]
>
> Use of SpecialRecentChangesQuery hook (used in
> FlaggedRevsUIHooks::modifyRecentChangesQuery) was deprecated in MediaWiki
> 1.23. [Called from SpecialRecentChanges::runMainQueryHook in
> /srv/mediawiki/php-1.29.0-wmf.9/includes/specials/SpecialRecentchanges.ph
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l




--
Best regards,
Max Semenik ([[User:MaxSem]])
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Deprecation logging in production

Gergo Tisza
In reply to this post by Erik Bernhardson
Nice, thanks!​

Here is a simple dashboard:
https://logstash.wikimedia.org/app/kibana#/dashboard/mediawiki-deprecated
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Deprecation logging in production

Brad Jorsch (Anomie)
In reply to this post by Max Semenik
On Wed, Jan 25, 2017 at 7:43 PM, Max Semenik <[hidden email]> wrote:

>     189 Use of SyntaxHighlight_GeSHi::prepare is deprecated. [Called from
> ScribuntoContent::fillParserOutput in
> /srv/mediawiki/php-1.29.0-wmf.8/extensions/Scribunto/common/
> ScribuntoContent.php
> at line 128]
>     189 Use of SyntaxHighlight_GeSHi::buildHeadItem is deprecated. [Called
> from ScribuntoContent::fillParserOutput in
> /srv/mediawiki/php-1.29.0-wmf.8/extensions/Scribunto/common/
> ScribuntoContent.php
> at line 141]
>     132 Use of SyntaxHighlight_GeSHi::prepare is deprecated. [Called from
> ScribuntoContent::fillParserOutput in
> /srv/mediawiki/php-1.29.0-wmf.9/extensions/Scribunto/common/
> ScribuntoContent.php
> at line 128]
>     132 Use of SyntaxHighlight_GeSHi::buildHeadItem is deprecated. [Called
> from ScribuntoContent::fillParserOutput in
> /srv/mediawiki/php-1.29.0-wmf.9/extensions/Scribunto/common/
> ScribuntoContent.php
> at line 141]
>

https://gerrit.wikimedia.org/r/#/c/245581/ is blocked on
https://gerrit.wikimedia.org/r/#/c/245580/, which seems to be waiting for
an update to do it in a different way.

   2705 Use of wfSetupSession was deprecated in MediaWiki 1.27. [Called

> from CollectionSession::startSession in
> /srv/mediawiki/php-1.29.0-wmf.8/extensions/Collection/
> Collection.session.php
> at line 38]
>      30 Use of wfSetupSession was deprecated in MediaWiki 1.27. [Called
> from CollectionSession::startSession in
> /srv/mediawiki/php-1.29.0-wmf.9/extensions/Collection/
> Collection.session.php
> at line 38]
>      20 Use of wfSetupSession was deprecated in MediaWiki 1.27. [Called
> from AbuseFilter::executeFilterActions in
> /srv/mediawiki/php-1.29.0-wmf.8/extensions/AbuseFilter/
> includes/AbuseFilter.class.php
> at line 796]
>       3 Use of wfSetupSession was deprecated in MediaWiki 1.27. [Called
> from wfAjaxPostCollection in
> /srv/mediawiki/php-1.29.0-wmf.8/extensions/Collection/Collection.php at
> line 268]
>       2 Use of wfSetupSession was deprecated in MediaWiki 1.27. [Called
> from AbuseFilter::executeFilterActions in
> /srv/mediawiki/php-1.29.0-wmf.9/extensions/AbuseFilter/
> includes/AbuseFilter.class.php
> at line 796]
>

https://phabricator.wikimedia.org/T124371 has details about how someone
could go about cleaning these up.

--
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Deprecation logging in production

Antoine Musso-3
In reply to this post by Gergo Tisza
On 26/01/17 01:48, Gergo Tisza wrote:
>
> Here is a simple dashboard:
> https://logstash.wikimedia.org/app/kibana#/dashboard/mediawiki-deprecated

Thanks quite helpful.

The equivalent for beta which is publicly available:
https://logstash-beta.wmflabs.org/app/kibana#/dashboard/mediawiki-deprecated


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Deprecation logging in production

Antoine Musso-3
In reply to this post by Erik Bernhardson
On 26/01/17 01:18, Erik Bernhardson wrote:

> Noticed earlier today, because some code that had been deprecated since
> 1.21 was removed and starting causing fatals, that we don't log deprecation
> notices in production. This has now been fixed.
>
> Please check logstash for `channel:deprecated` and see if anything you are
> reposinsible for is using hard-deprecated methods and fix them.
>
> Some culprits which have each generated hundreds of messages in the ~5
> minutes it's been turned on:
>
> Use of SpecialRecentChangesQuery hook (used in
> FlaggedRevsUIHooks::modifyRecentChangesQuery) was deprecated in MediaWiki
> 1.23. [Called from SpecialRecentChanges::runMainQueryHook in
> /srv/mediawiki/php-1.29.0-wmf.8/includes/specials/SpecialRecentchanges.ph
<snip>

Thanks for the logging channel, that is going to be quite helpful. To
anyone spotting them, please fill corresponding tasks against the
appropriate extension/skin and add in #wikimedia-log-errors to the task.


There are a few things that puzzle me though:

We have CI running with $wgDevelopmentWarnings = true; which fails the
test whenever wfDeprecated() is invoked.  But I guess lot of extension
code lack tests for those hooks.

We have a strict deprecation policy which ask for extensions to be
updated when a code is @deprecated.  And removing code/hook must only be
made when we have verified that all extensions we host have been migrated.
https://www.mediawiki.org/wiki/Deprecation

Then mistakes happen :(

--
Antoine "hashar" Musso





_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Deprecation logging in production

Daniel Kinzler-2
Am 26.01.2017 um 10:36 schrieb Antoine Musso:
> We have a strict deprecation policy which ask for extensions to be updated when
> a code is @deprecated.  And removing code/hook must only be made when we have
> verified that all extensions we host have been migrated.
> https://www.mediawiki.org/wiki/Deprecation

Not all extensions. The relevant bits of the (new, as of two weeks ago!) policy:

* Developers SHOULD update any extension or skin bundled with the MediaWiki
tarball when soft deprecating, and MAY update popular extensions

* If they don't submit patches, developers MUST file bugs about bundled
extensions/skins using deprecated functions so their maintainers can work on
updating them.

* Extensions and skins bundled with the MediaWiki tarball MUST NOT trigger hard
deprecation warnings and MUST be updated to use the new code.


But that's just for bundeled extensions, not deployed extensions. There is this:


* As one of the principles of MediaWiki, developers should ensure any removals
will not cause issues in the Wikimedia setup and extensions deployed there. If
they do, developers should expect to be reverted by Wikimedia system administrators.

But that's only removals, not deprecation.

--
Daniel Kinzler
Principal Platform Engineer

Wikimedia Deutschland
Gesellschaft zur Förderung Freien Wissens e.V.

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Deprecation logging in production

Chad
On Thu, Jan 26, 2017 at 10:19 AM Daniel Kinzler <[hidden email]>
wrote:

> * Extensions and skins bundled with the MediaWiki tarball MUST NOT trigger
> hard
> deprecation warnings and MUST be updated to use the new code.
>
>
> But that's just for bundeled extensions, not deployed extensions. There is
> this:
>
>
> * As one of the principles of MediaWiki, developers should ensure any
> removals
> will not cause issues in the Wikimedia setup and extensions deployed
> there. If
> they do, developers should expect to be reverted by Wikimedia system
> administrators.
>
>
As the person to changed the tarball criteria to MUST NOT, I would
be totally in favor of extending that to deployed extensions.

Removing code that's still used in production is embarrassing/sad
and offenders should be ashamed ;-)

-Chad
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l