2018-01-10 Scrum of Scrums meeting notes

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

2018-01-10 Scrum of Scrums meeting notes

Grace Gellerman
https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-01-10

== Callouts ==
* Search PF would like somebody from performance/MediaWiki team to look at:
https://phabricator.wikimedia.org/T183101we’ve got some fails on
LinksUpdate which blocked search index update which makes Wikidata items
invisible in search (~50 items)
  ** [Perf] We've been looking at it, but we haven't identified a cause for
this issue yet.
  ** [SP] I guess we will just watch out for it and hope it was an one-time
thing.
* [Perf] Trying to compile a list of everything running in Prod that
*requires* Zend PHP5.  Currently only aware of the XML dump job (has syntax
errors in output if run under HHVM).  Anything else, please reach out to
Ian (email: imarlier@, irc: marlier)
* Reminder: Whitelist your EventLogging schema if you want data older than
90 days to be retained, talk to the Analytics team if you're not sure what
this means

== Audiences ==
=== Readers ===
==== iOS native app ====
* Blocked by: none
* Blocking: none
* Updates:
**Continuing work on 5.8 - synced reading lists

==== Android native app ====
* Blocked by: Patch to unblock DELETE endpoints in varnish:
https://gerrit.wikimedia.org/r/402433 DONE!
* Blocking: none
* Updates:
** Completed work to integrate with the new Reading List service!
Continuing to test and measure performance.

==== Reading Infrastructure ====
* Blocked by: ops on https://gerrit.wikimedia.org/r/#/c/402433/ <= this is
already merged since 2018-01-09
* Blocking: none
* Updates:
** working on request batching in reading list service
** continuing work on WP0 piracy
** Summary endpoint switchover from TextExtracts to PCS implementation
planned for Wednesday
** wrapping up media and references endpoints
** metadata and CSS endpoints to follow

==== Web ====
* Wrapping up a rework of the mobile web beta/settings page.
* Nothing new to report with regards to challenges/deliverables

==== Maps ====
* Blocked by: Services team
* Blocking: None
* Updates
** Everyone is back from vacation
** Several issues waiting on service team

==== Multimedia ====
* Nothing to report

=== Contributors ===

==== Parsing ====
* Sole update: Approaching additional wikis for replacing Tidy on those
wikis on Jan 31st.

==== Global Collaboration ====
* Blocked: Security on https://phabricator.wikimedia.org/T144467
* Blocking: Ops on Flow dumps. Matt's back from sabbatical but has been
sick off and on over the last week

=== Community Tech ===
* Investigating new year's proposals
* Will file for security/performance/DBA review for GlobalPreferences today

==== UI Standardization ====
** OOUI v0.25.0 released
https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md
*** 6 breaking and 5 deprecating changes, and 16 UI related improvements,
among those
**** Drop the `constructive` flag entirely
**** Remove `BookletLayout#getClosestPage`
Also we significantly reduced SVG file size for our interface elements by
up to 21% with fine-tuned markup optimizations.
* Ongoing:
** OOUI & based products:
*** icons: Unify, refine and align to WikimediaUI Style Guide
https://phabricator.wikimedia.org/T177432 – first patches in:
https://gerrit.wikimedia.org/r/#/c/402757/
** More optimizations/size savings for RL embedded SVGs as data URIs


== Technology ==

== Technical Operations ==
* Blocking:
** None
* Blocked:
** Global Collaboration on Flow. https://phabricator.wikimedia.org/T172025
* Updates
** s5 => s8 migration for wikidata completed successfully. Feel free to
reach out to DBAs for followup questions and/or details
** meltdown/spectre week, ops busy with the mitigations, do expect some
slower response times
** This is a big hardware refresh quarter. Work is already decided and
prioritized, but please do provide input when/if requested ASAP. Thanks

=== Release Engineering ===
* Blocking
** None?
* Blocked
** ops: zuul package update (blocks gerrit upgrade)
** ops: node-tunnel-agent package update (blocks moving node testing to
docker in ci)
* Updates
** 2 weeks of normal MediaWiki deploys (this and next) followed by 2 weeks
of no MediaWiki but SWATs as needed (DevSummit/All Hands followed by RelEng
team offsite)
** Currently building nightlies of Mediawiki on the new “releases” (aka
non-CI) Jenkins host. Working with Security on best way to handle security
patches (which is the goal, to ensure security patches stay cleanly
applicable).

=== Performance ===
* Blocking: None
* Blocked: None
* Updates
** As noted above, we've been investigating LinksUpdate failures, haven't
been able to identify a cause at this point, haven't been able to reproduce.
** Fixes to Thumbor implementation happening as issues are reported (no
major ones at this point)
** Published the first in a series of blog posts detailing how we measure
performance:
https://phabricator.wikimedia.org/phame/post/view/83/measuring_wikipedia_page_load_times/
** Trying to get a list of everything running in Prod that *requires* PHP5
(Zend) -- only thing that I know of for sure is the XML dump job.

=== Security ===
* No updates
* Sam and Brian are traveling this week
* Darian's last day is January 15th

=== Fundraising Tech ===
* Blocking: None
* Blocked: None
* Updates:
** Upstreaming stuff to Civi
** Making our Amazon Pay SDK fork support TCP proxy
** Re-starting work on new API for our main credit card processor
** Stats projects:
*** Andrew Green's druid banner impressions lib:
https://github.com/AndrewGreen/centralnotice_analytics
*** Jack Gleeson's stats collector:
https://github.com/jackgleeson/stats-collector
*** More consideration of how the get banner stats to payments in realtime
(talking with Analytics)
*** Also need to talk with analytics about preserving aggregated banner
stats for December

=== Scoring Platform ===
* Blocking: no
* Blocked: no
* Updates
** We added Extension:ORES to in-house code coverage reporting :D
*** https://doc.wikimedia.org/cover-extensions/ORES/index.html
*** https://scrutinizer-ci.com/g/wikimedia/mediawiki-extensions-ORES/

=== Analytics ===
* https://superset.wikimedia.org  You can now create dashboards on top of
druid data, no automatic user creation. Created users for PMs and analysts,
let us know if you want a user. Perf to be seen.
* Clickstream data now public. Research community has access to the paths
readers follow through the site (aggreggated counts, not per user)
* Eventlogging purging on master box almost done.
* Working on wikistats UI bugs and small features to make project selection
more friendly, once the UI tweaks are done we will leave this work on
backburner until next year.
* Work on new set of APIs to return pageviews per country continues.
Privacy study of this data:
https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews/Pageviews_by_country
* Meeting with Asaf for our next big project: geowiki (editors per country,
quite sensitive data)


** Mostly quiet (holidays)

=== Search Platform ===
* Blocked by: none
* Blocking: none
* Updates:
* Improving LTR training mechanisms (
https://phabricator.wikimedia.org/T184547)
* Working on fixes for completion suggester & redirects namespaces (
https://phabricator.wikimedia.org/T115756)
* Investigating ElasticSearch phonetic search (
https://phabricator.wikimedia.org/T182708)
* Working on refactoring search profiles to make them more config-like
https://phabricator.wikimedia.org/T183279
* Processing feedback for Wikidata fulltext & prefix search rankings
* Running test for machine-learning ranking on Hebrew wiki (
https://phabricator.wikimedia.org/T182616)
* Discussing publishing data from WDQS query logs (
https://phabricator.wikimedia.org/T143819)

== Wikidata ==
* Gaining speed after the holidays.
* Working on Lexeme continues, now working on diffs:
https://phabricator.wikimedia.org/T182424
* Working on Quality Constraint integration continues, 1.0 rollout planned
for Q1.
* Preparing for Developer Summit.
* Planning collaboration with Multimedia (MediaInfo) as well as Search for
Q1.

== MediaWiki Platform==
* Blocked by: Wikibase on https://phabricator.wikimedia.org/T183019
* Blocking: N/A
* Updates:
* Tim: Spectre/Meltdown review
* Multi-Content Revisions:
** Actor table: 380669 needs code review
** Working on https://phabricator.wikimedia.org/T36925 and
https://phabricator.wikimedia.org/T182678
** Comment table:
***Schema changes are almost done. Then we can start trying to use it.
** cleanupUsersWithNoId:
*** enwiki finished
*** Hopefully dewiki and wikidatawiki can be run this week
* ExternalStore de-PHP-serialization: Patch done before vacation. But Jaime
raised some interesting discussion on
https://phabricator.wikimedia.org/T183419 that may determine how we go
about updating ExternalStore
* Code coverage:
** Added a bunch of @covers tags in core, we also validate them pre-merge
now (see giant spike in
https://tools.wmflabs.org/coverage/mediawiki/chart.png )
** Implemented coverage reports for extensions (
https://phabricator.wikimedia.org/T71685,
https://doc.wikimedia.org/cover-extensions/ ), still in a beta period, will
announce as ready for usage end of this or next week once
https://phabricator.wikimedia.org/T182749 is implemented and documented
** Brainstorming/working on proof of concept for pre-merge coverage reports
that don't take hours to generate
* Audiences Technology Working Group planning
* Developer Summit planning

== SoS meeting ==
* ggellerman has ongoing conflicts, looking for another facilitator every
other week
* Check in on goals here, talk about cross-team dependencies in this
meeting, rather than waiting until the end of each quarter.
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l