Holding 1.33.0-wmf.4

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Holding 1.33.0-wmf.4

Antoine Musso-3
Hello,

I am holding deployment of 1.33.0-wmf.4 to the Wikimedia infrastructure.
Some code path attempts to retrieve a memcached entry using an empty
string as a key. It is most probably going to cause major issues if we
keep pushing the train.

The task is:
https://phabricator.wikimedia.org/T209429

memcached error: A BAD KEY WAS PROVIDED/CHARACTERS OUT OF RANGE


It originates from MemcachedPeclBagOStuff::checkResult() which logs an
error albeit without any context.  Probably adding an exception context
would give us a stacktrace and thus find the faulting code.


--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Holding 1.33.0-wmf.4

Antoine Musso-3
On 14/11/2018 15:29, Antoine Musso wrote:

>
> I am holding deployment of 1.33.0-wmf.4 to the Wikimedia infrastructure.
> Some code path attempts to retrieve a memcached entry using an empty
> string as a key. It is most probably going to cause major issues if we
> keep pushing the train.
>
> The task is:
> https://phabricator.wikimedia.org/T209429
>
> memcached error: A BAD KEY WAS PROVIDED/CHARACTERS OUT OF RANGE
>
>
> It originates from MemcachedPeclBagOStuff::checkResult() which logs an
> error albeit without any context.  Probably adding an exception context
> would give us a stacktrace and thus find the faulting code.

Hello,

I started a patch to add a stacktrace to the log context, but eventually
had a personal emergency.  Luckily Brad Jorsch crafted a patch that we
have deployed, he found the root cause and I have deployed the fix.

The task is resolved. I have resumed with the train and 1.33.0-wmf.4 is
now deployed on group 1 as expected.

If there is anything looking suspicious on group0 or group1 wikis please
fill sub tasks to the blocker task:
https://phabricator.wikimedia.org/T206658

--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l