New feature deployed: Finding text changes in moved paragraphs in diffs

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

New feature deployed: Finding text changes in moved paragraphs in diffs

Birgit Müller
Hi all,


over the past 1,5 years WMDE’s Technical Wishes team was working on a major
improvement of wikidiff2 [1] - the engine behind the wikitext diff view.
The major outcome of our improvements got deployed to most wikis just now:
It is now easier to find moved paragraphs and text changes inside them. [2]


Why we worked on this:

The idea for the project arose from the wishlist of the German-speaking
communities: Show text changes in moved paragraphs. Before we worked on
this, moved paragraphs were shown in the diff as a deleted paragraph and
then as an added paragraph. This meant that you had to parse visually
through the paragraphs to find any potential changes - a fact that vandals
liked to make use of. [3]


Desktop version, mobile version

The desktop version of the feature already got deployed to Mediawiki.org
and deWP in December 2017. After further improvements of the diff
algorithm, the change could now get deployed to most wikis. The languages
unified Han, Thai and Japanese that don’t use spaces as delimiters will
follow soon.
The mobile version is in the making and will follow in the next weeks. The
backend work has been done by the WMDE team, the WMF’s mobile team is
taking care of the design work for the user interface of the mobile version.

Improving the algorithm of wikidiff2: More than a new feature

The work that has been done goes further than the original wish: First of
all, we not only highlighted text changes within a moved paragraph, but
also made it easier to find moved paragraphs in the first place: Clickable
arrows allow to follow the move of the paragraph.
Second, we generally improved the diff algorithm: For example, we could
find an old bug in the diff engine: Sometimes two paragraphs which are not
related at all were considered a changed paragraph. The bug fix was already
deployed in October 2017. [4]

If you are interested in learning more about the work that has been done,
we're going to publish the lessons learned by the dev team within the next
days (so stay tuned :-) !.


Thank you!

We would like to thank everyone who supported us in the work around
wikidiff2: People from different wikis looked into diffs and gave us
valuable feedback. Moritz Mühlenhoff supported us by updating the library,
and deploying it. Tim Starling and Max Semenik supported us with code
review and consultation during the Vienna Hackathon. The Desktop and Mobile
Web team is going to add a user friendly design to the mobile version. Your
advice, feedback and support is much appreciated :-)


Further feedback is very welcome!

Birgit (for the technical wishes team)


[1] Extension manual: https://www.mediawiki.org/wiki/Extension:Wikidiff2

[2] Deployment ticket: https://phabricator.wikimedia.org/T195375

[3] Main project page on meta:
https://meta.wikimedia.org/wiki/WMDE_Technical_Wishes/Show_text_changes_when_moving_text_chunks

[4] Patch on gerrit for the bug fix:
https://gerrit.wikimedia.org/r/#/c/356582/


--
Birgit Müller
Community Communications Manager
Software Development and Engineering




Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
http://wikimedia.de

Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
Wissens frei teilhaben kann. Helfen Sie uns dabei!
http://spenden.wikimedia.de/

Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: New feature deployed: Finding text changes in moved paragraphs in diffs

Victoria Coleman
Very excited to see this get deployed widely. Very timely actually! Congrats to the team!

Best,

Victoria

> On May 30, 2018, at 6:34 AM, Birgit Müller <[hidden email]> wrote:
>
> Hi all,
>
>
> over the past 1,5 years WMDE’s Technical Wishes team was working on a major
> improvement of wikidiff2 [1] - the engine behind the wikitext diff view.
> The major outcome of our improvements got deployed to most wikis just now:
> It is now easier to find moved paragraphs and text changes inside them. [2]
>
>
> Why we worked on this:
>
> The idea for the project arose from the wishlist of the German-speaking
> communities: Show text changes in moved paragraphs. Before we worked on
> this, moved paragraphs were shown in the diff as a deleted paragraph and
> then as an added paragraph. This meant that you had to parse visually
> through the paragraphs to find any potential changes - a fact that vandals
> liked to make use of. [3]
>
>
> Desktop version, mobile version
>
> The desktop version of the feature already got deployed to Mediawiki.org
> and deWP in December 2017. After further improvements of the diff
> algorithm, the change could now get deployed to most wikis. The languages
> unified Han, Thai and Japanese that don’t use spaces as delimiters will
> follow soon.
> The mobile version is in the making and will follow in the next weeks. The
> backend work has been done by the WMDE team, the WMF’s mobile team is
> taking care of the design work for the user interface of the mobile version.
>
> Improving the algorithm of wikidiff2: More than a new feature
>
> The work that has been done goes further than the original wish: First of
> all, we not only highlighted text changes within a moved paragraph, but
> also made it easier to find moved paragraphs in the first place: Clickable
> arrows allow to follow the move of the paragraph.
> Second, we generally improved the diff algorithm: For example, we could
> find an old bug in the diff engine: Sometimes two paragraphs which are not
> related at all were considered a changed paragraph. The bug fix was already
> deployed in October 2017. [4]
>
> If you are interested in learning more about the work that has been done,
> we're going to publish the lessons learned by the dev team within the next
> days (so stay tuned :-) !.
>
>
> Thank you!
>
> We would like to thank everyone who supported us in the work around
> wikidiff2: People from different wikis looked into diffs and gave us
> valuable feedback. Moritz Mühlenhoff supported us by updating the library,
> and deploying it. Tim Starling and Max Semenik supported us with code
> review and consultation during the Vienna Hackathon. The Desktop and Mobile
> Web team is going to add a user friendly design to the mobile version. Your
> advice, feedback and support is much appreciated :-)
>
>
> Further feedback is very welcome!
>
> Birgit (for the technical wishes team)
>
>
> [1] Extension manual: https://www.mediawiki.org/wiki/Extension:Wikidiff2
>
> [2] Deployment ticket: https://phabricator.wikimedia.org/T195375
>
> [3] Main project page on meta:
> https://meta.wikimedia.org/wiki/WMDE_Technical_Wishes/Show_text_changes_when_moving_text_chunks
>
> [4] Patch on gerrit for the bug fix:
> https://gerrit.wikimedia.org/r/#/c/356582/
>
>
> --
> Birgit Müller
> Community Communications Manager
> Software Development and Engineering
>
>
>
>
> Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
> Tel. (030) 219 158 26-0
> http://wikimedia.de
>
> Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
> Wissens frei teilhaben kann. Helfen Sie uns dabei!
> http://spenden.wikimedia.de/
>
> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
> Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
> der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
> Körperschaften I Berlin, Steuernummer 27/681/51985.
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: New feature deployed: Finding text changes in moved paragraphs in diffs

Birgit Müller
In reply to this post by Birgit Müller
Hey all,

here comes the link to the promised write-up of the work we did around
wikidiff2:

https://www.mediawiki.org/wiki/Extension:Wikidiff2/How_we_improved_Wikidiff2

Thoughts & questions are very welcome on the talk page there!

Also thanks a lot for the encouraging feedback here and elsewhere :-)

Birgit



2018-05-30 15:34 GMT+02:00 Birgit Müller <[hidden email]>:

> Hi all,
>
>
> over the past 1,5 years WMDE’s Technical Wishes team was working on a
> major improvement of wikidiff2 [1] - the engine behind the wikitext diff
> view. The major outcome of our improvements got deployed to most wikis
> just now: It is now easier to find moved paragraphs and text changes inside
> them. [2]
>
>
> Why we worked on this:
>
> The idea for the project arose from the wishlist of the German-speaking
> communities: Show text changes in moved paragraphs. Before we worked on
> this, moved paragraphs were shown in the diff as a deleted paragraph and
> then as an added paragraph. This meant that you had to parse visually
> through the paragraphs to find any potential changes - a fact that vandals
> liked to make use of. [3]
>
>
> Desktop version, mobile version
>
> The desktop version of the feature already got deployed to Mediawiki.org
> and deWP in December 2017. After further improvements of the diff
> algorithm, the change could now get deployed to most wikis. The languages
> unified Han, Thai and Japanese that don’t use spaces as delimiters will
> follow soon.
> The mobile version is in the making and will follow in the next weeks. The
> backend work has been done by the WMDE team, the WMF’s mobile team is
> taking care of the design work for the user interface of the mobile version.
>
> Improving the algorithm of wikidiff2: More than a new feature
>
> The work that has been done goes further than the original wish: First of
> all, we not only highlighted text changes within a moved paragraph, but
> also made it easier to find moved paragraphs in the first place: Clickable
> arrows allow to follow the move of the paragraph.
> Second, we generally improved the diff algorithm: For example, we could
> find an old bug in the diff engine: Sometimes two paragraphs which are not
> related at all were considered a changed paragraph. The bug fix was
> already deployed in October 2017. [4]
>
> If you are interested in learning more about the work that has been done,
> we're going to publish the lessons learned by the dev team within the
> next days (so stay tuned :-) !.
>
>
> Thank you!
>
> We would like to thank everyone who supported us in the work around
> wikidiff2: People from different wikis looked into diffs and gave us
> valuable feedback. Moritz Mühlenhoff supported us by updating the library,
> and deploying it. Tim Starling and Max Semenik supported us with code
> review and consultation during the Vienna Hackathon. The Desktop and Mobile
> Web team is going to add a user friendly design to the mobile version. Your
> advice, feedback and support is much appreciated :-)
>
>
> Further feedback is very welcome!
>
> Birgit (for the technical wishes team)
>
>
> [1] Extension manual: https://www.mediawiki.org/wiki/Extension:Wikidiff2
>
> [2] Deployment ticket: https://phabricator.wikimedia.org/T195375
>
> [3] Main project page on meta: https://meta.wikimedia.org/
> wiki/WMDE_Technical_Wishes/Show_text_changes_when_moving_text_chunks
>
> [4] Patch on gerrit for the bug fix: https://gerrit.wikimedia.org/
> r/#/c/356582/
>
>
> --
> Birgit Müller
> Community Communications Manager
> Software Development and Engineering
>
>
>
>
> Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
> Tel. (030) 219 158 26-0
> http://wikimedia.de
>
> Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
> Wissens frei teilhaben kann. Helfen Sie uns dabei!
> http://spenden.wikimedia.de/
>
> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
> Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
> der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
> Körperschaften I Berlin, Steuernummer 27/681/51985.
>



--
Birgit Müller
Community Communications Manager
Software Development and Engineering




Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
http://wikimedia.de

Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
Wissens frei teilhaben kann. Helfen Sie uns dabei!
http://spenden.wikimedia.de/

Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l