[Train] 1.33.0-wmf.24 status update

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[Train] 1.33.0-wmf.24 status update

Dan Duvall
The 1.33.0-wmf.24 version of MediaWiki is blocked[0].

The new version is deployed to group0[1], but can proceed no further until
these issues are resolved:

* Citoid should only usurp "<ref" / ctrl+shift+k shortcuts if full
configured - https://phabricator.wikimedia.org/T219510
* RefreshLinksJob::runForTitle: transaction round 'RefreshLinksJob::run'
already started on commons - https://phabricator.wikimedia.org/T220037

The first issue was deemed noncritical for group1 but the second caused a
large spike in DBTransactionError rates. The second issue already has a
proposed fix but had not yet merged at the closing of the train deployment
window.

If both issues are resolved by tomorrow's train window, group1 and group2
will be promoted in succession at that time (barring further issues).

Thank you for your help resolving these issues!

Your humble train toiler,
Dan

[0]. https://phabricator.wikimedia.org/T206678
[1]. <https://tools.wmflabs.org/versions/>

--
Dan Duvall
Software Engineer, Release Engineering
Wikimedia Foundation <http://wikimediafoundation.org>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: [Train] 1.33.0-wmf.24 status update

Dan Duvall
The 1.33.0-wmf.24 version of MediaWiki remains blocked[0], holding at
group0 pending resolution of the following issue:

* RefreshLinksJob::runForTitle: transaction round 'RefreshLinksJob::run'
already started on commons - https://phabricator.wikimedia.org/T220037

Errors related to the issue were mitigated somewhat by a backported patch
to the EventBus extension. However, the 90% drop in DBTransactionError
rates still left us at a hefty ~ 800 errors per minute and the root cause
is yet unknown.

Thanks to everyone who is helping to resolve this week's issues!

Just a note that since we are well outside today's train deployment window
and we refrain from Friday deploys, any fixes for the outstanding issue can
be deployed for group0 but promotion of 1.33.0-wmf.24 will be held until
Monday.

Kindly,
Dan


On Wed, Apr 3, 2019 at 2:29 PM Dan Duvall <[hidden email]> wrote:

> The 1.33.0-wmf.24 version of MediaWiki is blocked[0].
>
> The new version is deployed to group0[1], but can proceed no further until
> these issues are resolved:
>
> * Citoid should only usurp "<ref" / ctrl+shift+k shortcuts if full
> configured - https://phabricator.wikimedia.org/T219510
> * RefreshLinksJob::runForTitle: transaction round 'RefreshLinksJob::run'
> already started on commons - https://phabricator.wikimedia.org/T220037
>
> The first issue was deemed noncritical for group1 but the second caused a
> large spike in DBTransactionError rates. The second issue already has a
> proposed fix but had not yet merged at the closing of the train deployment
> window.
>
> If both issues are resolved by tomorrow's train window, group1 and group2
> will be promoted in succession at that time (barring further issues).
>
> Thank you for your help resolving these issues!
>
> Your humble train toiler,
> Dan
>
> [0]. https://phabricator.wikimedia.org/T206678
> [1]. <https://tools.wmflabs.org/versions/>
>
> --
> Dan Duvall
> Software Engineer, Release Engineering
> Wikimedia Foundation <http://wikimediafoundation.org>
>


--
Dan Duvall
Software Engineer, Release Engineering
Wikimedia Foundation <http://wikimediafoundation.org>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: [Train] 1.33.0-wmf.24 status update

Dan Duvall
The 1.33.0-wmf.24 version of MediaWiki was successfully promoted to all
wikis today. Thanks to everyone who helped fix outstanding issues from last
week!

Kindly,
Dan

On Thu, Apr 4, 2019 at 2:36 PM Dan Duvall <[hidden email]> wrote:

> The 1.33.0-wmf.24 version of MediaWiki remains blocked[0], holding at
> group0 pending resolution of the following issue:
>
> * RefreshLinksJob::runForTitle: transaction round 'RefreshLinksJob::run'
> already started on commons - https://phabricator.wikimedia.org/T220037
>
> Errors related to the issue were mitigated somewhat by a backported patch
> to the EventBus extension. However, the 90% drop in DBTransactionError
> rates still left us at a hefty ~ 800 errors per minute and the root cause
> is yet unknown.
>
> Thanks to everyone who is helping to resolve this week's issues!
>
> Just a note that since we are well outside today's train deployment window
> and we refrain from Friday deploys, any fixes for the outstanding issue can
> be deployed for group0 but promotion of 1.33.0-wmf.24 will be held until
> Monday.
>
> Kindly,
> Dan
>
>
> On Wed, Apr 3, 2019 at 2:29 PM Dan Duvall <[hidden email]> wrote:
>
>> The 1.33.0-wmf.24 version of MediaWiki is blocked[0].
>>
>> The new version is deployed to group0[1], but can proceed no further
>> until these issues are resolved:
>>
>> * Citoid should only usurp "<ref" / ctrl+shift+k shortcuts if full
>> configured - https://phabricator.wikimedia.org/T219510
>> * RefreshLinksJob::runForTitle: transaction round 'RefreshLinksJob::run'
>> already started on commons - https://phabricator.wikimedia.org/T220037
>>
>> The first issue was deemed noncritical for group1 but the second caused a
>> large spike in DBTransactionError rates. The second issue already has a
>> proposed fix but had not yet merged at the closing of the train deployment
>> window.
>>
>> If both issues are resolved by tomorrow's train window, group1 and group2
>> will be promoted in succession at that time (barring further issues).
>>
>> Thank you for your help resolving these issues!
>>
>> Your humble train toiler,
>> Dan
>>
>> [0]. https://phabricator.wikimedia.org/T206678
>> [1]. <https://tools.wmflabs.org/versions/>
>>
>> --
>> Dan Duvall
>> Software Engineer, Release Engineering
>> Wikimedia Foundation <http://wikimediafoundation.org>
>>
>
>
> --
> Dan Duvall
> Software Engineer, Release Engineering
> Wikimedia Foundation <http://wikimediafoundation.org>
>


--
Dan Duvall
Software Engineer, Release Engineering
Wikimedia Foundation <http://wikimediafoundation.org>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l