First step for MCR merged: Deprecate and gut the Revision class

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

First step for MCR merged: Deprecate and gut the Revision class

Daniel Kinzler-2
Hello all!

Addshore last night merged the patch[1] that is the first major step towards
Multi-Content-Revisions[2]: it completely guts the Revision class and turns it
into a thin proxy for the new RevisionStore service. The new code is now live
on beta.

This is our second attempt: The first one, on December 18th, thoroughly
corrupted the beta database. It took us some time and a lot of help from Aaron
and especially Roan to figure out what was happening. A detailed post-mortem by
Roan can be found at [3].

Anyway - this stage of MCR development introduces the new multi-revision capable
interface for revision storage (and blob storage) [4]. It does not yet introduce
the new database schema, that will be the next step [5][6]. While doing the
refactoring, I tried to keep the structure of the existing code mostly intact,
just moving functionality out of Revision into the new classes, most importantly
RevisionRecord, RevisionStore, and BlobStore.

Beware that with the next deployment (due January 2nd) the live sites will start
using the new code. Please keep an eye out for any strangeness regarding
revision handling. Adam greatly improved test coverage of the relevant code
(thanks Adam!), but it's always possible that we missed some edge case, maybe
something about archived revisions that were partially migrated from on old
schema or something similarly fun.

Exiting times!

Cheers
Daniel


[1] https://gerrit.wikimedia.org/r/#/c/399174/
[2] https://www.mediawiki.org/wiki/Requests_for_comment/Multi-Content_Revisions
[3] https://phabricator.wikimedia.org/T183252#3853749
[4] https://phabricator.wikimedia.org/T174025
[5] https://phabricator.wikimedia.org/T174024
[6] https://phabricator.wikimedia.org/T174030


--
Daniel Kinzler
Principal Platform Engineer

Wikimedia Deutschland
Gesellschaft zur Förderung Freien Wissens e.V.


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: First step for MCR merged: Deprecate and gut the Revision class

Toby Negrin
Congratulations Daniel and everyone who worked on this -- it's a big
milestone for the structured data and the MediaWiki in general!

The post-mortem is an epic read -- another round of thanks to everyone who
pitched in.

-Toby

On Fri, Dec 22, 2017 at 2:26 AM, Daniel Kinzler <[hidden email]
> wrote:

> Hello all!
>
> Addshore last night merged the patch[1] that is the first major step
> towards
> Multi-Content-Revisions[2]: it completely guts the Revision class and
> turns it
> into a thin proxy for the new RevisionStore service. The new code is now
> live
> on beta.
>
> This is our second attempt: The first one, on December 18th, thoroughly
> corrupted the beta database. It took us some time and a lot of help from
> Aaron
> and especially Roan to figure out what was happening. A detailed
> post-mortem by
> Roan can be found at [3].
>
> Anyway - this stage of MCR development introduces the new multi-revision
> capable
> interface for revision storage (and blob storage) [4]. It does not yet
> introduce
> the new database schema, that will be the next step [5][6]. While doing the
> refactoring, I tried to keep the structure of the existing code mostly
> intact,
> just moving functionality out of Revision into the new classes, most
> importantly
> RevisionRecord, RevisionStore, and BlobStore.
>
> Beware that with the next deployment (due January 2nd) the live sites will
> start
> using the new code. Please keep an eye out for any strangeness regarding
> revision handling. Adam greatly improved test coverage of the relevant code
> (thanks Adam!), but it's always possible that we missed some edge case,
> maybe
> something about archived revisions that were partially migrated from on old
> schema or something similarly fun.
>
> Exiting times!
>
> Cheers
> Daniel
>
>
> [1] https://gerrit.wikimedia.org/r/#/c/399174/
> [2] https://www.mediawiki.org/wiki/Requests_for_comment/
> Multi-Content_Revisions
> [3] https://phabricator.wikimedia.org/T183252#3853749
> [4] https://phabricator.wikimedia.org/T174025
> [5] https://phabricator.wikimedia.org/T174024
> [6] https://phabricator.wikimedia.org/T174030
>
>
> --
> Daniel Kinzler
> Principal Platform Engineer
>
> Wikimedia Deutschland
> Gesellschaft zur Förderung Freien Wissens e.V.
>
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: First step for MCR merged: Deprecate and gut the Revision class

Chad
In reply to this post by Daniel Kinzler-2
Considering the code just landed last night and a good number of us are
going to be gone for vacation--is rolling this out with the Jan 2nd deploy
a little aggressive? I'd love to see this sit on beta (with eyes on it) for
a little longer. Or a way to deploy to testwiki etc independent of major
sites?

The first deploy after a holiday break is already pretty exciting, and
rolling this out feels like something that could use a dedicated window.

(Otherwise, kudos to the MCR team for hitting this milestone)

-Chad

On Fri, Dec 22, 2017 at 2:27 AM Daniel Kinzler <[hidden email]>
wrote:

> Hello all!
>
> Addshore last night merged the patch[1] that is the first major step
> towards
> Multi-Content-Revisions[2]: it completely guts the Revision class and
> turns it
> into a thin proxy for the new RevisionStore service. The new code is now
> live
> on beta.
>
> This is our second attempt: The first one, on December 18th, thoroughly
> corrupted the beta database. It took us some time and a lot of help from
> Aaron
> and especially Roan to figure out what was happening. A detailed
> post-mortem by
> Roan can be found at [3].
>
> Anyway - this stage of MCR development introduces the new multi-revision
> capable
> interface for revision storage (and blob storage) [4]. It does not yet
> introduce
> the new database schema, that will be the next step [5][6]. While doing the
> refactoring, I tried to keep the structure of the existing code mostly
> intact,
> just moving functionality out of Revision into the new classes, most
> importantly
> RevisionRecord, RevisionStore, and BlobStore.
>
> Beware that with the next deployment (due January 2nd) the live sites will
> start
> using the new code. Please keep an eye out for any strangeness regarding
> revision handling. Adam greatly improved test coverage of the relevant code
> (thanks Adam!), but it's always possible that we missed some edge case,
> maybe
> something about archived revisions that were partially migrated from on old
> schema or something similarly fun.
>
> Exiting times!
>
> Cheers
> Daniel
>
>
> [1] https://gerrit.wikimedia.org/r/#/c/399174/
> [2]
> https://www.mediawiki.org/wiki/Requests_for_comment/Multi-Content_Revisions
> [3] https://phabricator.wikimedia.org/T183252#3853749
> [4] https://phabricator.wikimedia.org/T174025
> [5] https://phabricator.wikimedia.org/T174024
> [6] https://phabricator.wikimedia.org/T174030
>
>
> --
> Daniel Kinzler
> Principal Platform Engineer
>
> Wikimedia Deutschland
> Gesellschaft zur Förderung Freien Wissens e.V.
>
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: First step for MCR merged: Deprecate and gut the Revision class

C. Scott Ananian
Having just read through T183252, I feel Roan deserves a big hand for his
"I take a walk and become Sherlock Holmes" detective work and "I'm just
like Indiana Jones, except with code not tombs and bugs not snakes" code
archaeology.

I love working with smart folks.
  --scott

On Fri, Dec 22, 2017 at 11:37 AM, Chad <[hidden email]> wrote:

> Considering the code just landed last night and a good number of us are
> going to be gone for vacation--is rolling this out with the Jan 2nd deploy
> a little aggressive? I'd love to see this sit on beta (with eyes on it) for
> a little longer. Or a way to deploy to testwiki etc independent of major
> sites?
>
> The first deploy after a holiday break is already pretty exciting, and
> rolling this out feels like something that could use a dedicated window.
>
> (Otherwise, kudos to the MCR team for hitting this milestone)
>
> -Chad
>
> On Fri, Dec 22, 2017 at 2:27 AM Daniel Kinzler <
> [hidden email]>
> wrote:
>
> > Hello all!
> >
> > Addshore last night merged the patch[1] that is the first major step
> > towards
> > Multi-Content-Revisions[2]: it completely guts the Revision class and
> > turns it
> > into a thin proxy for the new RevisionStore service. The new code is now
> > live
> > on beta.
> >
> > This is our second attempt: The first one, on December 18th, thoroughly
> > corrupted the beta database. It took us some time and a lot of help from
> > Aaron
> > and especially Roan to figure out what was happening. A detailed
> > post-mortem by
> > Roan can be found at [3].
> >
> > Anyway - this stage of MCR development introduces the new multi-revision
> > capable
> > interface for revision storage (and blob storage) [4]. It does not yet
> > introduce
> > the new database schema, that will be the next step [5][6]. While doing
> the
> > refactoring, I tried to keep the structure of the existing code mostly
> > intact,
> > just moving functionality out of Revision into the new classes, most
> > importantly
> > RevisionRecord, RevisionStore, and BlobStore.
> >
> > Beware that with the next deployment (due January 2nd) the live sites
> will
> > start
> > using the new code. Please keep an eye out for any strangeness regarding
> > revision handling. Adam greatly improved test coverage of the relevant
> code
> > (thanks Adam!), but it's always possible that we missed some edge case,
> > maybe
> > something about archived revisions that were partially migrated from on
> old
> > schema or something similarly fun.
> >
> > Exiting times!
> >
> > Cheers
> > Daniel
> >
> >
> > [1] https://gerrit.wikimedia.org/r/#/c/399174/
> > [2]
> > https://www.mediawiki.org/wiki/Requests_for_comment/
> Multi-Content_Revisions
> > [3] https://phabricator.wikimedia.org/T183252#3853749
> > [4] https://phabricator.wikimedia.org/T174025
> > [5] https://phabricator.wikimedia.org/T174024
> > [6] https://phabricator.wikimedia.org/T174030
> >
> >
> > --
> > Daniel Kinzler
> > Principal Platform Engineer
> >
> > Wikimedia Deutschland
> > Gesellschaft zur Förderung Freien Wissens e.V.
> >
> >
> > _______________________________________________
> > Wikitech-l mailing list
> > [hidden email]
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: First step for MCR merged: Deprecate and gut the Revision class

Subramanya Sastry
+1 to what Chad said reg deploy and what Toby, Chad & Scott said with
their kudos and appreciation :)  -Subbu.


On 12/22/2017 11:00 AM, C. Scott Ananian wrote:

> Having just read through T183252, I feel Roan deserves a big hand for his
> "I take a walk and become Sherlock Holmes" detective work and "I'm just
> like Indiana Jones, except with code not tombs and bugs not snakes" code
> archaeology.
>
> I love working with smart folks.
>    --scott
>
> On Fri, Dec 22, 2017 at 11:37 AM, Chad <[hidden email]> wrote:
>
>> Considering the code just landed last night and a good number of us are
>> going to be gone for vacation--is rolling this out with the Jan 2nd deploy
>> a little aggressive? I'd love to see this sit on beta (with eyes on it) for
>> a little longer. Or a way to deploy to testwiki etc independent of major
>> sites?
>>
>> The first deploy after a holiday break is already pretty exciting, and
>> rolling this out feels like something that could use a dedicated window.
>>
>> (Otherwise, kudos to the MCR team for hitting this milestone)
>>
>> -Chad
>>
>> On Fri, Dec 22, 2017 at 2:27 AM Daniel Kinzler <
>> [hidden email]>
>> wrote:
>>
>>> Hello all!
>>>
>>> Addshore last night merged the patch[1] that is the first major step
>>> towards
>>> Multi-Content-Revisions[2]: it completely guts the Revision class and
>>> turns it
>>> into a thin proxy for the new RevisionStore service. The new code is now
>>> live
>>> on beta.
>>>
>>> This is our second attempt: The first one, on December 18th, thoroughly
>>> corrupted the beta database. It took us some time and a lot of help from
>>> Aaron
>>> and especially Roan to figure out what was happening. A detailed
>>> post-mortem by
>>> Roan can be found at [3].
>>>
>>> Anyway - this stage of MCR development introduces the new multi-revision
>>> capable
>>> interface for revision storage (and blob storage) [4]. It does not yet
>>> introduce
>>> the new database schema, that will be the next step [5][6]. While doing
>> the
>>> refactoring, I tried to keep the structure of the existing code mostly
>>> intact,
>>> just moving functionality out of Revision into the new classes, most
>>> importantly
>>> RevisionRecord, RevisionStore, and BlobStore.
>>>
>>> Beware that with the next deployment (due January 2nd) the live sites
>> will
>>> start
>>> using the new code. Please keep an eye out for any strangeness regarding
>>> revision handling. Adam greatly improved test coverage of the relevant
>> code
>>> (thanks Adam!), but it's always possible that we missed some edge case,
>>> maybe
>>> something about archived revisions that were partially migrated from on
>> old
>>> schema or something similarly fun.
>>>
>>> Exiting times!
>>>
>>> Cheers
>>> Daniel
>>>
>>>
>>> [1] https://gerrit.wikimedia.org/r/#/c/399174/
>>> [2]
>>> https://www.mediawiki.org/wiki/Requests_for_comment/
>> Multi-Content_Revisions
>>> [3] https://phabricator.wikimedia.org/T183252#3853749
>>> [4] https://phabricator.wikimedia.org/T174025
>>> [5] https://phabricator.wikimedia.org/T174024
>>> [6] https://phabricator.wikimedia.org/T174030
>>>
>>>
>>> --
>>> Daniel Kinzler
>>> Principal Platform Engineer
>>>
>>> Wikimedia Deutschland
>>> Gesellschaft zur Förderung Freien Wissens e.V.
>>>
>>>
>>> _______________________________________________
>>> Wikitech-l mailing list
>>> [hidden email]
>>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>> _______________________________________________
>> Wikitech-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>
>
>


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: First step for MCR merged: Deprecate and gut the Revision class

Marko Obrovac
+1 too for Chad's concerns, especially knowing that there is still a bug
opened that breaks Revision::newFromId()~[1], which is used throughout the
EventBus system to create events.

That said, really big congrats are in order for the MCR project. It will
undoubtedly move Mediawiki and the Wikimedia projects in a good direction
and open in up to future use cases that we probably cannot phantom in this
point in time!


Cheers,
Marko

[1] https://phabricator.wikimedia.org/T183505

Marko Obrovac, PhD
Senior Services Engineer
Wikimedia Foundation

On 22 December 2017 at 18:13, Subramanya Sastry <[hidden email]>
wrote:

> +1 to what Chad said reg deploy and what Toby, Chad & Scott said with
> their kudos and appreciation :)  -Subbu.
>
>
>
> On 12/22/2017 11:00 AM, C. Scott Ananian wrote:
>
>> Having just read through T183252, I feel Roan deserves a big hand for his
>> "I take a walk and become Sherlock Holmes" detective work and "I'm just
>> like Indiana Jones, except with code not tombs and bugs not snakes" code
>> archaeology.
>>
>> I love working with smart folks.
>>    --scott
>>
>> On Fri, Dec 22, 2017 at 11:37 AM, Chad <[hidden email]> wrote:
>>
>> Considering the code just landed last night and a good number of us are
>>> going to be gone for vacation--is rolling this out with the Jan 2nd
>>> deploy
>>> a little aggressive? I'd love to see this sit on beta (with eyes on it)
>>> for
>>> a little longer. Or a way to deploy to testwiki etc independent of major
>>> sites?
>>>
>>> The first deploy after a holiday break is already pretty exciting, and
>>> rolling this out feels like something that could use a dedicated window.
>>>
>>> (Otherwise, kudos to the MCR team for hitting this milestone)
>>>
>>> -Chad
>>>
>>> On Fri, Dec 22, 2017 at 2:27 AM Daniel Kinzler <
>>> [hidden email]>
>>> wrote:
>>>
>>> Hello all!
>>>>
>>>> Addshore last night merged the patch[1] that is the first major step
>>>> towards
>>>> Multi-Content-Revisions[2]: it completely guts the Revision class and
>>>> turns it
>>>> into a thin proxy for the new RevisionStore service. The new code is now
>>>> live
>>>> on beta.
>>>>
>>>> This is our second attempt: The first one, on December 18th, thoroughly
>>>> corrupted the beta database. It took us some time and a lot of help from
>>>> Aaron
>>>> and especially Roan to figure out what was happening. A detailed
>>>> post-mortem by
>>>> Roan can be found at [3].
>>>>
>>>> Anyway - this stage of MCR development introduces the new multi-revision
>>>> capable
>>>> interface for revision storage (and blob storage) [4]. It does not yet
>>>> introduce
>>>> the new database schema, that will be the next step [5][6]. While doing
>>>>
>>> the
>>>
>>>> refactoring, I tried to keep the structure of the existing code mostly
>>>> intact,
>>>> just moving functionality out of Revision into the new classes, most
>>>> importantly
>>>> RevisionRecord, RevisionStore, and BlobStore.
>>>>
>>>> Beware that with the next deployment (due January 2nd) the live sites
>>>>
>>> will
>>>
>>>> start
>>>> using the new code. Please keep an eye out for any strangeness regarding
>>>> revision handling. Adam greatly improved test coverage of the relevant
>>>>
>>> code
>>>
>>>> (thanks Adam!), but it's always possible that we missed some edge case,
>>>> maybe
>>>> something about archived revisions that were partially migrated from on
>>>>
>>> old
>>>
>>>> schema or something similarly fun.
>>>>
>>>> Exiting times!
>>>>
>>>> Cheers
>>>> Daniel
>>>>
>>>>
>>>> [1] https://gerrit.wikimedia.org/r/#/c/399174/
>>>> [2]
>>>> https://www.mediawiki.org/wiki/Requests_for_comment/
>>>>
>>> Multi-Content_Revisions
>>>
>>>> [3] https://phabricator.wikimedia.org/T183252#3853749
>>>> [4] https://phabricator.wikimedia.org/T174025
>>>> [5] https://phabricator.wikimedia.org/T174024
>>>> [6] https://phabricator.wikimedia.org/T174030
>>>>
>>>>
>>>> --
>>>> Daniel Kinzler
>>>> Principal Platform Engineer
>>>>
>>>> Wikimedia Deutschland
>>>> Gesellschaft zur Förderung Freien Wissens e.V.
>>>>
>>>>
>>>> _______________________________________________
>>>> Wikitech-l mailing list
>>>> [hidden email]
>>>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>>>
>>> _______________________________________________
>>> Wikitech-l mailing list
>>> [hidden email]
>>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>>
>>>
>>
>>
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: First step for MCR merged: Deprecate and gut the Revision class

Ramsey Isler
In reply to this post by Daniel Kinzler-2
Fantastic news! Great work handling this behemoth of a technical challenge.

On Fri, Dec 22, 2017 at 2:26 AM, Daniel Kinzler <[hidden email]
> wrote:

> Hello all!
>
> Addshore last night merged the patch[1] that is the first major step
> towards
> Multi-Content-Revisions[2]: it completely guts the Revision class and
> turns it
> into a thin proxy for the new RevisionStore service. The new code is now
> live
> on beta.
>
> This is our second attempt: The first one, on December 18th, thoroughly
> corrupted the beta database. It took us some time and a lot of help from
> Aaron
> and especially Roan to figure out what was happening. A detailed
> post-mortem by
> Roan can be found at [3].
>
> Anyway - this stage of MCR development introduces the new multi-revision
> capable
> interface for revision storage (and blob storage) [4]. It does not yet
> introduce
> the new database schema, that will be the next step [5][6]. While doing the
> refactoring, I tried to keep the structure of the existing code mostly
> intact,
> just moving functionality out of Revision into the new classes, most
> importantly
> RevisionRecord, RevisionStore, and BlobStore.
>
> Beware that with the next deployment (due January 2nd) the live sites will
> start
> using the new code. Please keep an eye out for any strangeness regarding
> revision handling. Adam greatly improved test coverage of the relevant code
> (thanks Adam!), but it's always possible that we missed some edge case,
> maybe
> something about archived revisions that were partially migrated from on old
> schema or something similarly fun.
>
> Exiting times!
>
> Cheers
> Daniel
>
>
> [1] https://gerrit.wikimedia.org/r/#/c/399174/
> [2] https://www.mediawiki.org/wiki/Requests_for_comment/
> Multi-Content_Revisions
> [3] https://phabricator.wikimedia.org/T183252#3853749
> [4] https://phabricator.wikimedia.org/T174025
> [5] https://phabricator.wikimedia.org/T174024
> [6] https://phabricator.wikimedia.org/T174030
>
>
> --
> Daniel Kinzler
> Principal Platform Engineer
>
> Wikimedia Deutschland
> Gesellschaft zur Förderung Freien Wissens e.V.
>
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: First step for MCR merged: Deprecate and gut the Revision class

addshorewiki
So the plan going forward will be to create a feature flag for the MCR
Revision gutting.
I'll crack on with that this evening.

If that turns out to be too messy then we can revert the MCR patches for
the next wmf branch.
I'm currently keeping track of this @
https://wikitech.wikimedia.org/wiki/User:Addshore/MCR_Revert

On 22 December 2017 at 18:39, Ramsey Isler <[hidden email]> wrote:

> Fantastic news! Great work handling this behemoth of a technical challenge.
>
> On Fri, Dec 22, 2017 at 2:26 AM, Daniel Kinzler <
> [hidden email]> wrote:
>
>> Hello all!
>>
>> Addshore last night merged the patch[1] that is the first major step
>> towards
>> Multi-Content-Revisions[2]: it completely guts the Revision class and
>> turns it
>> into a thin proxy for the new RevisionStore service. The new code is now
>> live
>> on beta.
>>
>> This is our second attempt: The first one, on December 18th, thoroughly
>> corrupted the beta database. It took us some time and a lot of help from
>> Aaron
>> and especially Roan to figure out what was happening. A detailed
>> post-mortem by
>> Roan can be found at [3].
>>
>> Anyway - this stage of MCR development introduces the new multi-revision
>> capable
>> interface for revision storage (and blob storage) [4]. It does not yet
>> introduce
>> the new database schema, that will be the next step [5][6]. While doing
>> the
>> refactoring, I tried to keep the structure of the existing code mostly
>> intact,
>> just moving functionality out of Revision into the new classes, most
>> importantly
>> RevisionRecord, RevisionStore, and BlobStore.
>>
>> Beware that with the next deployment (due January 2nd) the live sites
>> will start
>> using the new code. Please keep an eye out for any strangeness regarding
>> revision handling. Adam greatly improved test coverage of the relevant
>> code
>> (thanks Adam!), but it's always possible that we missed some edge case,
>> maybe
>> something about archived revisions that were partially migrated from on
>> old
>> schema or something similarly fun.
>>
>> Exiting times!
>>
>> Cheers
>> Daniel
>>
>>
>> [1] https://gerrit.wikimedia.org/r/#/c/399174/
>> [2] https://www.mediawiki.org/wiki/Requests_for_comment/Multi-
>> Content_Revisions
>> [3] https://phabricator.wikimedia.org/T183252#3853749
>> [4] https://phabricator.wikimedia.org/T174025
>> [5] https://phabricator.wikimedia.org/T174024
>> [6] https://phabricator.wikimedia.org/T174030
>>
>>
>> --
>> Daniel Kinzler
>> Principal Platform Engineer
>>
>> Wikimedia Deutschland
>> Gesellschaft zur Förderung Freien Wissens e.V.
>>
>>
>> _______________________________________________
>> Wikitech-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
>
>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: First step for MCR merged: Deprecate and gut the Revision class

Roan Kattouw-2
In reply to this post by C. Scott Ananian
Thanks!

Context for those who don't want to read a couple hundred lines of IRC
logs: I was cooped up in the house all day and noticed it was about to get
dark, so I really did take a walk (relatively abruptly) after dealing with
the worst of the issue. During this walk I thought things over and realized
what the explanation for the early symptoms of this bug was, and I
explained it to Adam on IRC when I got back.

On Fri, Dec 22, 2017 at 6:00 PM, C. Scott Ananian <[hidden email]>
wrote:

> Having just read through T183252, I feel Roan deserves a big hand for his
> "I take a walk and become Sherlock Holmes" detective work and "I'm just
> like Indiana Jones, except with code not tombs and bugs not snakes" code
> archaeology.
>
> I love working with smart folks.
>   --scott
>
> On Fri, Dec 22, 2017 at 11:37 AM, Chad <[hidden email]> wrote:
>
> > Considering the code just landed last night and a good number of us are
> > going to be gone for vacation--is rolling this out with the Jan 2nd
> deploy
> > a little aggressive? I'd love to see this sit on beta (with eyes on it)
> for
> > a little longer. Or a way to deploy to testwiki etc independent of major
> > sites?
> >
> > The first deploy after a holiday break is already pretty exciting, and
> > rolling this out feels like something that could use a dedicated window.
> >
> > (Otherwise, kudos to the MCR team for hitting this milestone)
> >
> > -Chad
> >
> > On Fri, Dec 22, 2017 at 2:27 AM Daniel Kinzler <
> > [hidden email]>
> > wrote:
> >
> > > Hello all!
> > >
> > > Addshore last night merged the patch[1] that is the first major step
> > > towards
> > > Multi-Content-Revisions[2]: it completely guts the Revision class and
> > > turns it
> > > into a thin proxy for the new RevisionStore service. The new code is
> now
> > > live
> > > on beta.
> > >
> > > This is our second attempt: The first one, on December 18th, thoroughly
> > > corrupted the beta database. It took us some time and a lot of help
> from
> > > Aaron
> > > and especially Roan to figure out what was happening. A detailed
> > > post-mortem by
> > > Roan can be found at [3].
> > >
> > > Anyway - this stage of MCR development introduces the new
> multi-revision
> > > capable
> > > interface for revision storage (and blob storage) [4]. It does not yet
> > > introduce
> > > the new database schema, that will be the next step [5][6]. While doing
> > the
> > > refactoring, I tried to keep the structure of the existing code mostly
> > > intact,
> > > just moving functionality out of Revision into the new classes, most
> > > importantly
> > > RevisionRecord, RevisionStore, and BlobStore.
> > >
> > > Beware that with the next deployment (due January 2nd) the live sites
> > will
> > > start
> > > using the new code. Please keep an eye out for any strangeness
> regarding
> > > revision handling. Adam greatly improved test coverage of the relevant
> > code
> > > (thanks Adam!), but it's always possible that we missed some edge case,
> > > maybe
> > > something about archived revisions that were partially migrated from on
> > old
> > > schema or something similarly fun.
> > >
> > > Exiting times!
> > >
> > > Cheers
> > > Daniel
> > >
> > >
> > > [1] https://gerrit.wikimedia.org/r/#/c/399174/
> > > [2]
> > > https://www.mediawiki.org/wiki/Requests_for_comment/
> > Multi-Content_Revisions
> > > [3] https://phabricator.wikimedia.org/T183252#3853749
> > > [4] https://phabricator.wikimedia.org/T174025
> > > [5] https://phabricator.wikimedia.org/T174024
> > > [6] https://phabricator.wikimedia.org/T174030
> > >
> > >
> > > --
> > > Daniel Kinzler
> > > Principal Platform Engineer
> > >
> > > Wikimedia Deutschland
> > > Gesellschaft zur Förderung Freien Wissens e.V.
> > >
> > >
> > > _______________________________________________
> > > Wikitech-l mailing list
> > > [hidden email]
> > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > _______________________________________________
> > Wikitech-l mailing list
> > [hidden email]
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
>
>
>
> --
> (http://cscott.net)
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: First step for MCR merged: Deprecate and gut the Revision class

Daniel Kinzler-2
In reply to this post by Chad
Am 22.12.2017 um 17:37 schrieb Chad:
> Considering the code just landed last night and a good number of us are
> going to be gone for vacation--is rolling this out with the Jan 2nd deploy
> a little aggressive? I'd love to see this sit on beta (with eyes on it) for
> a little longer. Or a way to deploy to testwiki etc independent of major
> sites?

Hi Chad!

The ideas was exactly to have this on beta as long as possible - like two
(originally three) weeks over christmas. I suppose the problem is "with eyes on
it"...

> The first deploy after a holiday break is already pretty exciting, and
> rolling this out feels like something that could use a dedicated window.

So, maybe we could deploy only to testwiki and friends on January 2nd, and only
do a full deployment the week after?

To be honest, I was not considering such a step for a change that is "just" a
refactoring. I was reserving that for the actual schema change we plan for MCR.
But perhaps that was a mistake.

-- daniel

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: First step for MCR merged: Deprecate and gut the Revision class

Brion Vibber-4
In reply to this post by Daniel Kinzler-2
Woohoo! The pieces fall in place... :)

-- brion

On Dec 22, 2017 2:27 AM, "Daniel Kinzler" <[hidden email]>
wrote:

> Hello all!
>
> Addshore last night merged the patch[1] that is the first major step
> towards
> Multi-Content-Revisions[2]: it completely guts the Revision class and
> turns it
> into a thin proxy for the new RevisionStore service. The new code is now
> live
> on beta.
>
> This is our second attempt: The first one, on December 18th, thoroughly
> corrupted the beta database. It took us some time and a lot of help from
> Aaron
> and especially Roan to figure out what was happening. A detailed
> post-mortem by
> Roan can be found at [3].
>
> Anyway - this stage of MCR development introduces the new multi-revision
> capable
> interface for revision storage (and blob storage) [4]. It does not yet
> introduce
> the new database schema, that will be the next step [5][6]. While doing the
> refactoring, I tried to keep the structure of the existing code mostly
> intact,
> just moving functionality out of Revision into the new classes, most
> importantly
> RevisionRecord, RevisionStore, and BlobStore.
>
> Beware that with the next deployment (due January 2nd) the live sites will
> start
> using the new code. Please keep an eye out for any strangeness regarding
> revision handling. Adam greatly improved test coverage of the relevant code
> (thanks Adam!), but it's always possible that we missed some edge case,
> maybe
> something about archived revisions that were partially migrated from on old
> schema or something similarly fun.
>
> Exiting times!
>
> Cheers
> Daniel
>
>
> [1] https://gerrit.wikimedia.org/r/#/c/399174/
> [2] https://www.mediawiki.org/wiki/Requests_for_comment/
> Multi-Content_Revisions
> [3] https://phabricator.wikimedia.org/T183252#3853749
> [4] https://phabricator.wikimedia.org/T174025
> [5] https://phabricator.wikimedia.org/T174024
> [6] https://phabricator.wikimedia.org/T174030
>
>
> --
> Daniel Kinzler
> Principal Platform Engineer
>
> Wikimedia Deutschland
> Gesellschaft zur Förderung Freien Wissens e.V.
>
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: First step for MCR merged: Deprecate and gut the Revision class

C. Scott Ananian
In reply to this post by addshorewiki
I think a simple revert would be simplest.  Adding a feature flag adds new
possibilities of overlooked bugs, especially since this is "just" a
refactoring and so *in theory* shouldn't be changing anything.

Maybe we could just cherry-pick a revert onto the Jan 2 branch, rather than
revert on master and then un-revert after the branch.  It shouldn't be
interpreted as any comment on code quality or "readiness", just a
reflection of the fact that the first deploy after the holiday will
inevitable cause *some* breakage, and in our eggnog-induced stupor it would
be nice if we could just rule out this refactor as contributing to whatever
the breakage turns out to be.  (That's why a feature flag doesn't seem
helpful to me; it would still be lines of code to comb through.  Better
either a clean revert or just deploy the thing as is.)
 --scott

On Fri, Dec 22, 2017 at 2:01 PM, Addshore <[hidden email]> wrote:

> So the plan going forward will be to create a feature flag for the MCR
> Revision gutting.
> I'll crack on with that this evening.
>
> If that turns out to be too messy then we can revert the MCR patches for
> the next wmf branch.
> I'm currently keeping track of this @
> https://wikitech.wikimedia.org/wiki/User:Addshore/MCR_Revert
>
> On 22 December 2017 at 18:39, Ramsey Isler <[hidden email]> wrote:
>
> > Fantastic news! Great work handling this behemoth of a technical
> challenge.
> >
> > On Fri, Dec 22, 2017 at 2:26 AM, Daniel Kinzler <
> > [hidden email]> wrote:
> >
> >> Hello all!
> >>
> >> Addshore last night merged the patch[1] that is the first major step
> >> towards
> >> Multi-Content-Revisions[2]: it completely guts the Revision class and
> >> turns it
> >> into a thin proxy for the new RevisionStore service. The new code is now
> >> live
> >> on beta.
> >>
> >> This is our second attempt: The first one, on December 18th, thoroughly
> >> corrupted the beta database. It took us some time and a lot of help from
> >> Aaron
> >> and especially Roan to figure out what was happening. A detailed
> >> post-mortem by
> >> Roan can be found at [3].
> >>
> >> Anyway - this stage of MCR development introduces the new multi-revision
> >> capable
> >> interface for revision storage (and blob storage) [4]. It does not yet
> >> introduce
> >> the new database schema, that will be the next step [5][6]. While doing
> >> the
> >> refactoring, I tried to keep the structure of the existing code mostly
> >> intact,
> >> just moving functionality out of Revision into the new classes, most
> >> importantly
> >> RevisionRecord, RevisionStore, and BlobStore.
> >>
> >> Beware that with the next deployment (due January 2nd) the live sites
> >> will start
> >> using the new code. Please keep an eye out for any strangeness regarding
> >> revision handling. Adam greatly improved test coverage of the relevant
> >> code
> >> (thanks Adam!), but it's always possible that we missed some edge case,
> >> maybe
> >> something about archived revisions that were partially migrated from on
> >> old
> >> schema or something similarly fun.
> >>
> >> Exiting times!
> >>
> >> Cheers
> >> Daniel
> >>
> >>
> >> [1] https://gerrit.wikimedia.org/r/#/c/399174/
> >> [2] https://www.mediawiki.org/wiki/Requests_for_comment/Multi-
> >> Content_Revisions
> >> [3] https://phabricator.wikimedia.org/T183252#3853749
> >> [4] https://phabricator.wikimedia.org/T174025
> >> [5] https://phabricator.wikimedia.org/T174024
> >> [6] https://phabricator.wikimedia.org/T174030
> >>
> >>
> >> --
> >> Daniel Kinzler
> >> Principal Platform Engineer
> >>
> >> Wikimedia Deutschland
> >> Gesellschaft zur Förderung Freien Wissens e.V.
> >>
> >>
> >> _______________________________________________
> >> Wikitech-l mailing list
> >> [hidden email]
> >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
> >
> >
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: First step for MCR merged: Deprecate and gut the Revision class

Brion Vibber-4
On Dec 22, 2017 3:03 PM, "C. Scott Ananian" <[hidden email]> wrote:

I think a simple revert would be simplest.  Adding a feature flag adds new
possibilities of overlooked bugs, especially since this is "just" a
refactoring and so *in theory* shouldn't be changing anything.


Agreed, it's likely to introduce more bugs with a second code path that
way...


Maybe we could just cherry-pick a revert onto the Jan 2 branch, rather than
revert on master and then un-revert after the branch.  It shouldn't be
interpreted as any comment on code quality or "readiness", just a
reflection of the fact that the first deploy after the holiday will
inevitable cause *some* breakage, and in our eggnog-induced stupor it would
be nice if we could just rule out this refactor as contributing to whatever
the breakage turns out to be.  (That's why a feature flag doesn't seem
helpful to me; it would still be lines of code to comb through.  Better
either a clean revert or just deploy the thing as is.)


+1

-- brion

 --scott

On Fri, Dec 22, 2017 at 2:01 PM, Addshore <[hidden email]> wrote:

> So the plan going forward will be to create a feature flag for the MCR
> Revision gutting.
> I'll crack on with that this evening.
>
> If that turns out to be too messy then we can revert the MCR patches for
> the next wmf branch.
> I'm currently keeping track of this @
> https://wikitech.wikimedia.org/wiki/User:Addshore/MCR_Revert
>
> On 22 December 2017 at 18:39, Ramsey Isler <[hidden email]> wrote:
>
> > Fantastic news! Great work handling this behemoth of a technical
> challenge.
> >
> > On Fri, Dec 22, 2017 at 2:26 AM, Daniel Kinzler <
> > [hidden email]> wrote:
> >
> >> Hello all!
> >>
> >> Addshore last night merged the patch[1] that is the first major step
> >> towards
> >> Multi-Content-Revisions[2]: it completely guts the Revision class and
> >> turns it
> >> into a thin proxy for the new RevisionStore service. The new code is
now
> >> live
> >> on beta.
> >>
> >> This is our second attempt: The first one, on December 18th, thoroughly
> >> corrupted the beta database. It took us some time and a lot of help
from
> >> Aaron
> >> and especially Roan to figure out what was happening. A detailed
> >> post-mortem by
> >> Roan can be found at [3].
> >>
> >> Anyway - this stage of MCR development introduces the new
multi-revision

> >> capable
> >> interface for revision storage (and blob storage) [4]. It does not yet
> >> introduce
> >> the new database schema, that will be the next step [5][6]. While doing
> >> the
> >> refactoring, I tried to keep the structure of the existing code mostly
> >> intact,
> >> just moving functionality out of Revision into the new classes, most
> >> importantly
> >> RevisionRecord, RevisionStore, and BlobStore.
> >>
> >> Beware that with the next deployment (due January 2nd) the live sites
> >> will start
> >> using the new code. Please keep an eye out for any strangeness
regarding

> >> revision handling. Adam greatly improved test coverage of the relevant
> >> code
> >> (thanks Adam!), but it's always possible that we missed some edge case,
> >> maybe
> >> something about archived revisions that were partially migrated from on
> >> old
> >> schema or something similarly fun.
> >>
> >> Exiting times!
> >>
> >> Cheers
> >> Daniel
> >>
> >>
> >> [1] https://gerrit.wikimedia.org/r/#/c/399174/
> >> [2] https://www.mediawiki.org/wiki/Requests_for_comment/Multi-
> >> Content_Revisions
> >> [3] https://phabricator.wikimedia.org/T183252#3853749
> >> [4] https://phabricator.wikimedia.org/T174025
> >> [5] https://phabricator.wikimedia.org/T174024
> >> [6] https://phabricator.wikimedia.org/T174030
> >>
> >>
> >> --
> >> Daniel Kinzler
> >> Principal Platform Engineer
> >>
> >> Wikimedia Deutschland
> >> Gesellschaft zur Förderung Freien Wissens e.V.
> >>
> >>
> >> _______________________________________________
> >> Wikitech-l mailing list
> >> [hidden email]
> >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
> >
> >
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: First step for MCR merged: Deprecate and gut the Revision class

Daniel Kinzler-2
In reply to this post by C. Scott Ananian
Am 23.12.2017 um 00:03 schrieb C. Scott Ananian:
> I think a simple revert would be simplest.  Adding a feature flag adds new
> possibilities of overlooked bugs, especially since this is "just" a
> refactoring and so *in theory* shouldn't be changing anything.
>
> Maybe we could just cherry-pick a revert onto the Jan 2 branch, rather than
> revert on master and then un-revert after the branch.  

A revert is certainly an option, I tried to keep this as isolated as possible.
Reverting just on the branch would allow us to keep testing on beta without
disruption, and without having to go back and forth con core code, causing merge
conflicts.

But there is another option to consider: Only deploy to testwiki (and friends)
on Jan 2, and not to production wikis. This would give us a week to look at this
in a production-like environment, on top of the time on beta, before it really
goes live.

-- daniel

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: First step for MCR merged: Deprecate and gut the Revision class

addshorewiki
So, the patches that would need to be reverted can be found on wikitech
https://wikitech.wikimedia.org/wiki/User:Addshore/MCR_Revert

I have also created a patch with a switch wrapping the refactoring
https://gerrit.wikimedia.org/r/#/c/399881/

I'm going to continue testing this code on beta over the christmas period
patching any holes that I find as I do.

On 23 December 2017 at 00:14, Daniel Kinzler <[hidden email]>
wrote:

> Am 23.12.2017 um 00:03 schrieb C. Scott Ananian:
> > I think a simple revert would be simplest.  Adding a feature flag adds
> new
> > possibilities of overlooked bugs, especially since this is "just" a
> > refactoring and so *in theory* shouldn't be changing anything.
> >
> > Maybe we could just cherry-pick a revert onto the Jan 2 branch, rather
> than
> > revert on master and then un-revert after the branch.
>
> A revert is certainly an option, I tried to keep this as isolated as
> possible.
> Reverting just on the branch would allow us to keep testing on beta without
> disruption, and without having to go back and forth con core code, causing
> merge
> conflicts.
>
> But there is another option to consider: Only deploy to testwiki (and
> friends)
> on Jan 2, and not to production wikis. This would give us a week to look
> at this
> in a production-like environment, on top of the time on beta, before it
> really
> goes live.
>
> -- daniel
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: First step for MCR merged: Deprecate and gut the Revision class

C. Scott Ananian
It seems like this patch broke running `php tests/parser/parserTests.php`
directly?
See: https://phabricator.wikimedia.org/T184122
  --scott

On Sat, Dec 23, 2017 at 4:29 AM, Addshore <[hidden email]> wrote:

> So, the patches that would need to be reverted can be found on wikitech
> https://wikitech.wikimedia.org/wiki/User:Addshore/MCR_Revert
>
> I have also created a patch with a switch wrapping the refactoring
> https://gerrit.wikimedia.org/r/#/c/399881/
>
> I'm going to continue testing this code on beta over the christmas period
> patching any holes that I find as I do.
>
> On 23 December 2017 at 00:14, Daniel Kinzler <[hidden email]>
> wrote:
>
>> Am 23.12.2017 um 00:03 schrieb C. Scott Ananian:
>> > I think a simple revert would be simplest.  Adding a feature flag adds
>> new
>> > possibilities of overlooked bugs, especially since this is "just" a
>> > refactoring and so *in theory* shouldn't be changing anything.
>> >
>> > Maybe we could just cherry-pick a revert onto the Jan 2 branch, rather
>> than
>> > revert on master and then un-revert after the branch.
>>
>> A revert is certainly an option, I tried to keep this as isolated as
>> possible.
>> Reverting just on the branch would allow us to keep testing on beta
>> without
>> disruption, and without having to go back and forth con core code,
>> causing merge
>> conflicts.
>>
>> But there is another option to consider: Only deploy to testwiki (and
>> friends)
>> on Jan 2, and not to production wikis. This would give us a week to look
>> at this
>> in a production-like environment, on top of the time on beta, before it
>> really
>> goes live.
>>
>> -- daniel
>>
>> _______________________________________________
>> Wikitech-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>
>
>


--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l