Update on Discovery Projects

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Update on Discovery Projects

Deb Tankersley
Hi everybody,

(With apologies for cross-posting...)

You may have seen the recent communication [1
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/June_2017_changes>]
about the product and tech tune-up which went live the week of June 5th,
2017. In that communication, we promised an update on the future of
Discovery projects and we will talk about those in this email.

The Discovery team structure has now changed, but the new teams will still
work together to complete the goals as listed in the draft annual plan.[2]
A summary of their anticipated work, as we finalize these changes, is
below. We plan on doing a check-in at the end of the calendar year to see
how our goals are progressing with the new smaller and separated team
structure.

Here is a list of the various projects under the Discovery umbrella, along
with the goals that they will be working on:

Search Backend

Improve search capabilities:

   -

   Implement ‘learning to rank’ [3] and other advanced machine learning
   methodologies
   -

   Improve support for languages using new analyzers
   -

   Maintain and expand power user search functionality

Search Frontend

Improve user interface of the search results page with new functionality:

   -

   Implement explore similar [4]
   <https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_Improvements/Testing#A.2FB_test:_Add_.27explore_similar.27_pages_and_categories_for_search_results>
   -

   Update the completion suggester box [5]
   <https://www.mediawiki.org/wiki/Extension:CirrusSearch/CompletionSuggester>
   -

   Investigate the usage of a Wiktionary widget for English Wikipedia [6]

Wikidata Query Service

Expand and scale:

   -

   Improve ability to support power features on-wiki for readers
   -

   Improve full text search functionality
   -

   Implement SPARQL federation support

Portal

Create and implement automated language statistics and translation updates
for Wikipedia.org

Analysis

Provide in-depth analytics support:

   -

   Perform experimental design, data collection, and data analysis
   -

   Perform ad-hoc analyses of Discovery-domain data
   -

   Maintain and augment the Discovery Dashboards,[7] which allow the teams
   to track their KPIs and other metrics

Maps

Map support:

   -

   Implement new map style
   -

   Increase frequency of OSM data replication
   -

   As needed, assist with individual language Wikipedia’s implementation of
   mapframe [8] <https://www.mediawiki.org/wiki/Maps/how_to:_embedded_maps>

Note: There is a possibility that we can do more with maps in the coming
year; we are currently evaluating strategic, partnership, and resourcing
options.

Structured Data on Commons

Extend structured data search on Commons, as part of the structured data
grant [9] via:

   -

   Research and implement advanced search capabilities
   -

   Implement new elements, filters, relationships

Graphs and Tabular Data on Commons

We will be re-evaluating this functionality against other Commons
initiatives such as the structured data grant. As with maps, we will
provide updates when we know more.

We are still working out all the details with the new team structure and
there might be some turbulence; let us know if there are any concerns and
we will do our best to answer them.


Best regards,

Deborah Tankersley, Product Manager, Discovery

Erika Bjune, Engineering Manager, Search Platform

Jon Katz, Reading Product Lead

Toby Negrin, Interim Vice President of Product

Victoria Coleman, Chief Technology Officer


[1] https://www.mediawiki.org/wiki/Wikimedia_Engineering/June_2017_changes

[2]
https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2017-2018/Draft

[3] https://en.wikipedia.org/wiki/Learning_to_rank

[4]
https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_Improvements/Testing#A.2FB_test:_Add_.27explore_similar.27_pages_and_categories_for_search_results

[5]
https://www.mediawiki.org/wiki/Extension:CirrusSearch/CompletionSuggester

[6]
https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_Improvements/Testing#A.2FB_test:_Add_Wiktionary_widget_to_search_results_page

[7] https://discovery.wmflabs.org/

[8] https://www.mediawiki.org/wiki/Maps/how_to:_embedded_maps

[9] https://commons.wikimedia.org/wiki/Commons:Structured_data
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Update on Discovery Projects

Ryan Kaldari-2
Thanks Deb! One question: Does "As needed, assist with individual language
Wikipedia’s implementation of mapframe" mean just help support the
implementation on the wikis where mapframe is already deployed, or does it
mean potentially deploying mapframe to additional wikis?

On Tue, Jun 13, 2017 at 2:25 PM, Deborah Tankersley <
[hidden email]> wrote:

> Hi everybody,
>
> (With apologies for cross-posting...)
>
> You may have seen the recent communication [1
> <https://www.mediawiki.org/wiki/Wikimedia_Engineering/June_2017_changes>]
> about the product and tech tune-up which went live the week of June 5th,
> 2017. In that communication, we promised an update on the future of
> Discovery projects and we will talk about those in this email.
>
> The Discovery team structure has now changed, but the new teams will still
> work together to complete the goals as listed in the draft annual plan.[2]
> A summary of their anticipated work, as we finalize these changes, is
> below. We plan on doing a check-in at the end of the calendar year to see
> how our goals are progressing with the new smaller and separated team
> structure.
>
> Here is a list of the various projects under the Discovery umbrella, along
> with the goals that they will be working on:
>
> Search Backend
>
> Improve search capabilities:
>
>    -
>
>    Implement ‘learning to rank’ [3] and other advanced machine learning
>    methodologies
>    -
>
>    Improve support for languages using new analyzers
>    -
>
>    Maintain and expand power user search functionality
>
> Search Frontend
>
> Improve user interface of the search results page with new functionality:
>
>    -
>
>    Implement explore similar [4]
>    <https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_
> Improvements/Testing#A.2FB_test:_Add_.27explore_similar.
> 27_pages_and_categories_for_search_results>
>    -
>
>    Update the completion suggester box [5]
>    <https://www.mediawiki.org/wiki/Extension:CirrusSearch/
> CompletionSuggester>
>    -
>
>    Investigate the usage of a Wiktionary widget for English Wikipedia [6]
>
> Wikidata Query Service
>
> Expand and scale:
>
>    -
>
>    Improve ability to support power features on-wiki for readers
>    -
>
>    Improve full text search functionality
>    -
>
>    Implement SPARQL federation support
>
> Portal
>
> Create and implement automated language statistics and translation updates
> for Wikipedia.org
>
> Analysis
>
> Provide in-depth analytics support:
>
>    -
>
>    Perform experimental design, data collection, and data analysis
>    -
>
>    Perform ad-hoc analyses of Discovery-domain data
>    -
>
>    Maintain and augment the Discovery Dashboards,[7] which allow the teams
>    to track their KPIs and other metrics
>
> Maps
>
> Map support:
>
>    -
>
>    Implement new map style
>    -
>
>    Increase frequency of OSM data replication
>    -
>
>    As needed, assist with individual language Wikipedia’s implementation of
>    mapframe [8] <https://www.mediawiki.org/wiki/Maps/how_to:_embedded_maps
> >
>
> Note: There is a possibility that we can do more with maps in the coming
> year; we are currently evaluating strategic, partnership, and resourcing
> options.
>
> Structured Data on Commons
>
> Extend structured data search on Commons, as part of the structured data
> grant [9] via:
>
>    -
>
>    Research and implement advanced search capabilities
>    -
>
>    Implement new elements, filters, relationships
>
> Graphs and Tabular Data on Commons
>
> We will be re-evaluating this functionality against other Commons
> initiatives such as the structured data grant. As with maps, we will
> provide updates when we know more.
>
> We are still working out all the details with the new team structure and
> there might be some turbulence; let us know if there are any concerns and
> we will do our best to answer them.
>
>
> Best regards,
>
> Deborah Tankersley, Product Manager, Discovery
>
> Erika Bjune, Engineering Manager, Search Platform
>
> Jon Katz, Reading Product Lead
>
> Toby Negrin, Interim Vice President of Product
>
> Victoria Coleman, Chief Technology Officer
>
>
> [1] https://www.mediawiki.org/wiki/Wikimedia_Engineering/June_2017_changes
>
> [2]
> https://meta.wikimedia.org/wiki/Wikimedia_Foundation_
> Annual_Plan/2017-2018/Draft
>
> [3] https://en.wikipedia.org/wiki/Learning_to_rank
>
> [4]
> https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_
> Improvements/Testing#A.2FB_test:_Add_.27explore_similar.
> 27_pages_and_categories_for_search_results
>
> [5]
> https://www.mediawiki.org/wiki/Extension:CirrusSearch/CompletionSuggester
>
> [6]
> https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_
> Improvements/Testing#A.2FB_test:_Add_Wiktionary_widget_
> to_search_results_page
>
> [7] https://discovery.wmflabs.org/
>
> [8] https://www.mediawiki.org/wiki/Maps/how_to:_embedded_maps
>
> [9] https://commons.wikimedia.org/wiki/Commons:Structured_data
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Update on Discovery Projects

Deb Tankersley
Hi Ryan,

One question: Does "As needed, assist with individual language
> Wikipedia’s implementation of mapframe" mean just help support the
> implementation on the wikis where mapframe is already deployed, or does it
> mean potentially deploying mapframe to additional wikis?


​We don't have the technical resources to deploy mapframe to any additional
wikis at this time. If a wiki wants to deploy mapframe on their site, they
can do it on their own—similar to what Swedish Wikipedia accomplished.[1]

If severe bugs occur, we'll try our best to get them fixed; such as maps
aren't rendering from the tile servers. We just don't have the technical
resources to do any minor bug fixes, feature requests or any new
deployments right now.

[1] https://phabricator.wikimedia.org/T161032


Thanks,

Deb

--
deb tankersley
irc: debt
Product Manager, Discovery
Wikimedia Foundation

On Tue, Jun 13, 2017 at 4:00 PM, Ryan Kaldari <[hidden email]>
wrote:

> Thanks Deb! One question: Does "As needed, assist with individual language
> Wikipedia’s implementation of mapframe" mean just help support the
> implementation on the wikis where mapframe is already deployed, or does it
> mean potentially deploying mapframe to additional wikis?
>
> On Tue, Jun 13, 2017 at 2:25 PM, Deborah Tankersley <
> [hidden email]> wrote:
>
> > Hi everybody,
> >
> > (With apologies for cross-posting...)
> >
> > You may have seen the recent communication [1
> > <https://www.mediawiki.org/wiki/Wikimedia_Engineering/June_2017_changes
> >]
> > about the product and tech tune-up which went live the week of June 5th,
> > 2017. In that communication, we promised an update on the future of
> > Discovery projects and we will talk about those in this email.
> >
> > The Discovery team structure has now changed, but the new teams will
> still
> > work together to complete the goals as listed in the draft annual
> plan.[2]
> > A summary of their anticipated work, as we finalize these changes, is
> > below. We plan on doing a check-in at the end of the calendar year to see
> > how our goals are progressing with the new smaller and separated team
> > structure.
> >
> > Here is a list of the various projects under the Discovery umbrella,
> along
> > with the goals that they will be working on:
> >
> > Search Backend
> >
> > Improve search capabilities:
> >
> >    -
> >
> >    Implement ‘learning to rank’ [3] and other advanced machine learning
> >    methodologies
> >    -
> >
> >    Improve support for languages using new analyzers
> >    -
> >
> >    Maintain and expand power user search functionality
> >
> > Search Frontend
> >
> > Improve user interface of the search results page with new functionality:
> >
> >    -
> >
> >    Implement explore similar [4]
> >    <https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_
> > Improvements/Testing#A.2FB_test:_Add_.27explore_similar.
> > 27_pages_and_categories_for_search_results>
> >    -
> >
> >    Update the completion suggester box [5]
> >    <https://www.mediawiki.org/wiki/Extension:CirrusSearch/
> > CompletionSuggester>
> >    -
> >
> >    Investigate the usage of a Wiktionary widget for English Wikipedia [6]
> >
> > Wikidata Query Service
> >
> > Expand and scale:
> >
> >    -
> >
> >    Improve ability to support power features on-wiki for readers
> >    -
> >
> >    Improve full text search functionality
> >    -
> >
> >    Implement SPARQL federation support
> >
> > Portal
> >
> > Create and implement automated language statistics and translation
> updates
> > for Wikipedia.org
> >
> > Analysis
> >
> > Provide in-depth analytics support:
> >
> >    -
> >
> >    Perform experimental design, data collection, and data analysis
> >    -
> >
> >    Perform ad-hoc analyses of Discovery-domain data
> >    -
> >
> >    Maintain and augment the Discovery Dashboards,[7] which allow the
> teams
> >    to track their KPIs and other metrics
> >
> > Maps
> >
> > Map support:
> >
> >    -
> >
> >    Implement new map style
> >    -
> >
> >    Increase frequency of OSM data replication
> >    -
> >
> >    As needed, assist with individual language Wikipedia’s implementation
> of
> >    mapframe [8] <https://www.mediawiki.org/wiki/Maps/how_to:_embedded_
> maps
> > >
> >
> > Note: There is a possibility that we can do more with maps in the coming
> > year; we are currently evaluating strategic, partnership, and resourcing
> > options.
> >
> > Structured Data on Commons
> >
> > Extend structured data search on Commons, as part of the structured data
> > grant [9] via:
> >
> >    -
> >
> >    Research and implement advanced search capabilities
> >    -
> >
> >    Implement new elements, filters, relationships
> >
> > Graphs and Tabular Data on Commons
> >
> > We will be re-evaluating this functionality against other Commons
> > initiatives such as the structured data grant. As with maps, we will
> > provide updates when we know more.
> >
> > We are still working out all the details with the new team structure and
> > there might be some turbulence; let us know if there are any concerns and
> > we will do our best to answer them.
> >
> >
> > Best regards,
> >
> > Deborah Tankersley, Product Manager, Discovery
> >
> > Erika Bjune, Engineering Manager, Search Platform
> >
> > Jon Katz, Reading Product Lead
> >
> > Toby Negrin, Interim Vice President of Product
> >
> > Victoria Coleman, Chief Technology Officer
> >
> >
> > [1] https://www.mediawiki.org/wiki/Wikimedia_Engineering/
> June_2017_changes
> >
> > [2]
> > https://meta.wikimedia.org/wiki/Wikimedia_Foundation_
> > Annual_Plan/2017-2018/Draft
> >
> > [3] https://en.wikipedia.org/wiki/Learning_to_rank
> >
> > [4]
> > https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_
> > Improvements/Testing#A.2FB_test:_Add_.27explore_similar.
> > 27_pages_and_categories_for_search_results
> >
> > [5]
> > https://www.mediawiki.org/wiki/Extension:CirrusSearch/
> CompletionSuggester
> >
> > [6]
> > https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_
> > Improvements/Testing#A.2FB_test:_Add_Wiktionary_widget_
> > to_search_results_page
> >
> > [7] https://discovery.wmflabs.org/
> >
> > [8] https://www.mediawiki.org/wiki/Maps/how_to:_embedded_maps
> >
> > [9] https://commons.wikimedia.org/wiki/Commons:Structured_data
> > _______________________________________________
> > Wikitech-l mailing list
> > [hidden email]
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Update on Discovery Projects

Federico Leva (Nemo)
Thanks. It's good that mapframe can follow the usual
#Wikimedia-Site-requests process! Maybe worth writing down on
[[mw:Extension:Kartographer]], or at least a code comment above
wgKartographerEnableMapFrame, so that users know they can ask.

Nemo

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Update on Discovery Projects

Deb Tankersley
>
> Maybe worth writing down on [[mw:Extension:Kartographer]]


​Thanks for the suggestion; I've added a note here:
https://www.mediawiki.org/wiki/Help:Extension:Kartographer#Discovery_Maps_Update
.

Cheers,

Deb



--
deb tankersley
irc: debt
Product Manager, Discovery
Wikimedia Foundation

On Wed, Jun 14, 2017 at 4:41 AM, Federico Leva (Nemo) <[hidden email]>
wrote:

> Thanks. It's good that mapframe can follow the usual
> #Wikimedia-Site-requests process! Maybe worth writing down on
> [[mw:Extension:Kartographer]], or at least a code comment above
> wgKartographerEnableMapFrame, so that users know they can ask.
>
> Nemo
>
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l