s3 primary db master failover on Thursday 10th April 05:00AM UTC

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

s3 primary db master failover on Thursday 10th April 05:00AM UTC

Manuel Arostegui
Hello,

Due to hardware malfunction on the current s3 database primary master
(T219115) we need to failover it to a different host.

We are going to do this on Thursday 10th at 05:00 AM UTC, and we have
requested a 30 minutes window (T220080) as we have to go read only for all
the wikis living on s3 (P7994).

Impact: Writes will be blocked, reads will remain unaffected.
Time: 05:00AM UTC - 05:30 AM UTC (we do not expect to use the full 30
minutes window).

Communication will happen at #wikimedia-operations
If you are around at that time and want to help with the monitoring, please
join us!

Thanks
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: s3 primary db master failover on Thursday 10th April 05:00AM UTC

RhinosF1 Wikipedia
Thanks for the information,
I've seen you've added user notice to the Phab task but as far as I'm aware
that needs to be added asap so it can be translated so I've added it with
https://meta.wikimedia.org/w/index.php?title=Tech/News/2019/15&diff=18991805&oldid=18990102

On Fri, 5 Apr 2019 at 07:23, Manuel Arostegui <[hidden email]>
wrote:

> Hello,
>
> Due to hardware malfunction on the current s3 database primary master
> (T219115) we need to failover it to a different host.
>
> We are going to do this on Thursday 10th at 05:00 AM UTC, and we have
> requested a 30 minutes window (T220080) as we have to go read only for all
> the wikis living on s3 (P7994).
>
> Impact: Writes will be blocked, reads will remain unaffected.
> Time: 05:00AM UTC - 05:30 AM UTC (we do not expect to use the full 30
> minutes window).
>
> Communication will happen at #wikimedia-operations
> If you are around at that time and want to help with the monitoring, please
> join us!
>
> Thanks
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: s3 primary db master failover on Thursday 10th April 05:00AM UTC

Manuel Arostegui
Correction, this is the Thursday 11th of April not the 10th.
Sorry for the confusion!

On Fri, Apr 5, 2019 at 8:43 AM RhinosF1 Wikipedia <[hidden email]>
wrote:

> Thanks for the information,
> I've seen you've added user notice to the Phab task but as far as I'm aware
> that needs to be added asap so it can be translated so I've added it with
>
> https://meta.wikimedia.org/w/index.php?title=Tech/News/2019/15&diff=18991805&oldid=18990102
>
> On Fri, 5 Apr 2019 at 07:23, Manuel Arostegui <[hidden email]>
> wrote:
>
> > Hello,
> >
> > Due to hardware malfunction on the current s3 database primary master
> > (T219115) we need to failover it to a different host.
> >
> > We are going to do this on Thursday 10th at 05:00 AM UTC, and we have
> > requested a 30 minutes window (T220080) as we have to go read only for
> all
> > the wikis living on s3 (P7994).
> >
> > Impact: Writes will be blocked, reads will remain unaffected.
> > Time: 05:00AM UTC - 05:30 AM UTC (we do not expect to use the full 30
> > minutes window).
> >
> > Communication will happen at #wikimedia-operations
> > If you are around at that time and want to help with the monitoring,
> please
> > join us!
> >
> > Thanks
> > _______________________________________________
> > Wikitech-l mailing list
> > [hidden email]
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: s3 primary db master failover on Thursday 10th April 05:00AM UTC

Manuel Arostegui
Hello,

Reminder: this is happening in 30 minutes.

Manuel.

On Fri, Apr 5, 2019 at 8:57 AM Manuel Arostegui <[hidden email]>
wrote:

> Correction, this is the Thursday 11th of April not the 10th.
> Sorry for the confusion!
>
> On Fri, Apr 5, 2019 at 8:43 AM RhinosF1 Wikipedia <[hidden email]>
> wrote:
>
>> Thanks for the information,
>> I've seen you've added user notice to the Phab task but as far as I'm
>> aware
>> that needs to be added asap so it can be translated so I've added it with
>>
>> https://meta.wikimedia.org/w/index.php?title=Tech/News/2019/15&diff=18991805&oldid=18990102
>>
>> On Fri, 5 Apr 2019 at 07:23, Manuel Arostegui <[hidden email]>
>> wrote:
>>
>> > Hello,
>> >
>> > Due to hardware malfunction on the current s3 database primary master
>> > (T219115) we need to failover it to a different host.
>> >
>> > We are going to do this on Thursday 10th at 05:00 AM UTC, and we have
>> > requested a 30 minutes window (T220080) as we have to go read only for
>> all
>> > the wikis living on s3 (P7994).
>> >
>> > Impact: Writes will be blocked, reads will remain unaffected.
>> > Time: 05:00AM UTC - 05:30 AM UTC (we do not expect to use the full 30
>> > minutes window).
>> >
>> > Communication will happen at #wikimedia-operations
>> > If you are around at that time and want to help with the monitoring,
>> please
>> > join us!
>> >
>> > Thanks
>> > _______________________________________________
>> > Wikitech-l mailing list
>> > [hidden email]
>> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>> _______________________________________________
>> Wikitech-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: s3 primary db master failover on Thursday 10th April 05:00AM UTC

Manuel Arostegui
Hello,

This was done successfully.

Times in UTC:

Read only starts: 05:01:05
Read only stops: 05:03:55

Total read only time: 02:50 minutes

Cheers

On Thu, Apr 11, 2019 at 6:31 AM Manuel Arostegui <[hidden email]>
wrote:

> Hello,
>
> Reminder: this is happening in 30 minutes.
>
> Manuel.
>
> On Fri, Apr 5, 2019 at 8:57 AM Manuel Arostegui <[hidden email]>
> wrote:
>
>> Correction, this is the Thursday 11th of April not the 10th.
>> Sorry for the confusion!
>>
>> On Fri, Apr 5, 2019 at 8:43 AM RhinosF1 Wikipedia <[hidden email]>
>> wrote:
>>
>>> Thanks for the information,
>>> I've seen you've added user notice to the Phab task but as far as I'm
>>> aware
>>> that needs to be added asap so it can be translated so I've added it with
>>>
>>> https://meta.wikimedia.org/w/index.php?title=Tech/News/2019/15&diff=18991805&oldid=18990102
>>>
>>> On Fri, 5 Apr 2019 at 07:23, Manuel Arostegui <[hidden email]>
>>> wrote:
>>>
>>> > Hello,
>>> >
>>> > Due to hardware malfunction on the current s3 database primary master
>>> > (T219115) we need to failover it to a different host.
>>> >
>>> > We are going to do this on Thursday 10th at 05:00 AM UTC, and we have
>>> > requested a 30 minutes window (T220080) as we have to go read only for
>>> all
>>> > the wikis living on s3 (P7994).
>>> >
>>> > Impact: Writes will be blocked, reads will remain unaffected.
>>> > Time: 05:00AM UTC - 05:30 AM UTC (we do not expect to use the full 30
>>> > minutes window).
>>> >
>>> > Communication will happen at #wikimedia-operations
>>> > If you are around at that time and want to help with the monitoring,
>>> please
>>> > join us!
>>> >
>>> > Thanks
>>> > _______________________________________________
>>> > Wikitech-l mailing list
>>> > [hidden email]
>>> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>> _______________________________________________
>>> Wikitech-l mailing list
>>> [hidden email]
>>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>
>>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l