Question to WMF: Backlog on bugs

classic Classic list List threaded Threaded
87 messages Options
12345
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

Pine W
Hopefully I can say this in a way that is mutually acceptable.

For me this discussion is difficult but enlightening. I think that this is
the most difficult Wikimedia discussion in which I have been a participant
so far this year. I don't know what agreements will emerge from this
thread, if any. But the enlightenment is useful, and I hope that other
people also feel that they are learning something new or are being heard on
issues that are important to them.

I am thinking that ideally we would all be in a room together to have this
discussion. Perhaps a second best choice would be an online meeting
regarding the subject of technical debt. We could schedule a meeting via
Doodle. It's okay if people don't want to participate, but I'd like to
suggest an online meeting as an option.

Pine
( https://meta.wikimedia.org/wiki/User:Pine )
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

Strainu
In reply to this post by Amir Sarabadani-2
În joi, 14 mar. 2019 la 01:02, Amir Sarabadani <[hidden email]> a scris:

>
> On Wed, Mar 13, 2019 at 11:02 PM Strainu <[hidden email]> wrote:
>
> > - ContentTranslation v1 (obsolete now, has been unmaintained for 2
> > years while in production)
> > - UploadWizard (2 with high priority, 40 with normal, a few dozens
> > low, hundreds more untriaged): this is the project that got us out of
> > the "overloading the lang parameter for customizing the uploader" era,
> > the project that is used by millions of people every year, including
> > during every photo contest
> >
> There's something called code stewardship [0] and there is a process called
> code stewardship review for projects that are under-, un- or unclear
> maintained [1] which basically a piece of code either gets undeployed from
> WMF infra or we find maintainer(s) to fix the bugs. You can find the list
> of current and past reviews in [2].
>
> If you think a project doesn't have enough maintainer, you can start the
> review process. If there's an active maintainer [3] but they are not fixing
> bugs, most importantly critical bugs, you can raise the issue probably here
> but with **concrete examples.**

I'll rant about UW in a separate thread, right now I just want to
mention that [3] presents 3 possible maintainers for it, **none of
which did any work on UW in the last 6 months** (and presumably much
longer) according to Phab timelines. I know documentation is hard, but
this feels a lot like a wild goose chase.

Strainu

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

John Erling Blad
In reply to this post by Andre Klapper-2
Blame games does not fix faulty processes. You fix a sinkhole by
figuring out where the water comes from and where it goes.

Why do we have bugs that isn't handled for years? Why is it easier to
get a new feature than fixing an old bug?

Google had a problem with unfixed bugs, and they started identifying
the involved developers each time the build was broken. That is pretty
harsh, but what if devs somehow was named when their bugs were
mentioned? What if there were some kind of public statistic? How would
the devs react to being identified with a bug? Would they fix the bug,
or just be mad about it? Devs at some of Googles teams got mad, but in
the end the code were fixed. Take a look at "GTAC 2013 Keynote:
Evolution from Quality Assurance to Test Engineering" [1]

What if we could show information from the bugs in Phabricator in a
"tracked" template at other wiki-projects, identifying the team
responsible and perhaps even the dev assigned to the bug? Imagine the
creds the dev would get when the bug is fixed! Because it is easy to
loose track of pages with "tracked" templates we need some other means
to show this information, and our "public monitor" could be a special
page with the same information.

We say we don't want voting over bugs, but by saying that we refuse
getting stats over how many users a specific bug hits, and because of
that we don't get sufficient information (metrics) to make decisions
about specific bugs. Some bugs (or missing features) although changes
how users are doing specific things, how do we handle that?

What if users could give a "this hits me too" from a "tracked"
template. That would give a very simple metric on how important it is
to fix a problem. To make this visible to the wiki-communities the
special page could be sorted on this metric. Of course the devs would
have completely different priorities, but this page would list the
wiki-communities priorities.

It would be a kind of blame game, but it would also give the devs an
opportunity to get sainthood by fixing annoying bugs.

[1] https://www.youtube.com/watch?v=nyOHJ4GR4iU from 32:20

On Wed, Mar 13, 2019 at 11:49 PM Andre Klapper <[hidden email]> wrote:

>
> On Wed, 2019-03-13 at 21:01 +0100, John Erling Blad wrote:
> > This is like an enormous sinkhole, with people standing on the edge,
> > warning about the sinkhole. All around people are saying "we must do
> > something"! Still the sinkhole slowly grows larger and larger. People
> > placing warning signs "Sinkhole ahead". Others notifying neighbors
> > about the growing sinkhole. But nobody does anything about the
> > sinkhole itself.
>
> And repeating the same thing over and over again while repeatedly
> ignoring requests to be more specific won't help either...
>
> andre
> --
> Andre Klapper | Bugwrangler / Developer Advocate
> https://blogs.gnome.org/aklapper/
>
>
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

Andre Klapper-2
On Thu, 2019-03-14 at 12:35 +0100, John Erling Blad wrote:
> Blame games does not fix faulty processes.

Hmm, why is this thread called "Question to WMF" instead of "Question
to developers"?

> Why do we have bugs that isn't handled for years?

Basically: Because you did not fix these bugs. Longer version:
https://www.mediawiki.org/wiki/Bug_management/Development_prioritization

> Why is it easier to get a new feature than fixing an old bug?

{{Citation needed}}.
If that was the case: Because your priority was to write code for a new
feature instead of fixing an old bug. Longer version:
https://www.mediawiki.org/wiki/Bug_management/Development_prioritization

> Google had a problem with unfixed bugs, and they started identifying
> the involved developers each time the build was broken. That is pretty
> harsh, but what if devs somehow was named when their bugs were
> mentioned? What if there were some kind of public statistic? How would
> the devs react to being identified with a bug? Would they fix the bug,
> or just be mad about it? Devs at some of Googles teams got mad, but in
> the end the code were fixed. Take a look at "GTAC 2013 Keynote:
> Evolution from Quality Assurance to Test Engineering" [1]

Not really - I see 60000 open bug reports in Chromium, for example:
https://bugs.chromium.org/p/chromium/issues/list
(Only if you want to imply that only "Google" was responsible for
fixing all bugs in that free and open source project, of course.)

> What if we could show information from the bugs in Phabricator in a
> "tracked" template at other wiki-projects, identifying the team
> responsible and perhaps even the dev assigned to the bug? Imagine the
> creds the dev would get when the bug is fixed! Because it is easy to
> loose track of pages with "tracked" templates we need some other means
> to show this information, and our "public monitor" could be a special
> page with the same information.

Feel free to extend https://www.mediawiki.org/wiki/Template:Tracked

> We say we don't want voting over bugs, but by saying that we refuse
> getting stats over how many users a specific bug hits, and because of
> that we don't get sufficient information (metrics) to make decisions
> about specific bugs.

I disagree. Different people see different priorities. Longer version:
https://www.mediawiki.org/wiki/Bug_management/Development_prioritization

> What if users could give a "this hits me too" from a "tracked"
> template. That would give a very simple metric on how important it is
> to fix a problem.

It does not, because software development is not a popularity contest:
https://www.mediawiki.org/wiki/Bug_management/Development_prioritization
Voting would create expectations that nobody will fulfill.

Cheers,
andre
--
Andre Klapper | Bugwrangler / Developer Advocate
https://blogs.gnome.org/aklapper/



_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

Andre Klapper-2
In reply to this post by John Erling Blad
On Tue, 2019-03-12 at 00:29 +0100, John Erling Blad wrote:
> It seems like some projects simply put everything coming from external
> sources into deep freezer or add "need volunteer". If they respond at
> all. In some cases it could be that the projects are defunc.

What's the expectation based on that there should always be a response?
If a bug report has all info needed to allow someone to reproduce and
work on it, anyone is free to pick it up and work on it if anyone is
interested in working on it. No further response needed.

Cheers,
andre
--
Andre Klapper | Bugwrangler / Developer Advocate
https://blogs.gnome.org/aklapper/



_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

John Erling Blad
In reply to this post by Andre Klapper-2
Sorry, but I try to point out that the process is broken and give a
few examples on how to fix the process.

On Thu, Mar 14, 2019 at 1:20 PM Andre Klapper <[hidden email]> wrote:

>
> On Thu, 2019-03-14 at 12:35 +0100, John Erling Blad wrote:
> > Blame games does not fix faulty processes.
>
> Hmm, why is this thread called "Question to WMF" instead of "Question
> to developers"?
>
> > Why do we have bugs that isn't handled for years?
>
> Basically: Because you did not fix these bugs. Longer version:
> https://www.mediawiki.org/wiki/Bug_management/Development_prioritization
>
> > Why is it easier to get a new feature than fixing an old bug?
>
> {{Citation needed}}.
> If that was the case: Because your priority was to write code for a new
> feature instead of fixing an old bug. Longer version:
> https://www.mediawiki.org/wiki/Bug_management/Development_prioritization
>
> > Google had a problem with unfixed bugs, and they started identifying
> > the involved developers each time the build was broken. That is pretty
> > harsh, but what if devs somehow was named when their bugs were
> > mentioned? What if there were some kind of public statistic? How would
> > the devs react to being identified with a bug? Would they fix the bug,
> > or just be mad about it? Devs at some of Googles teams got mad, but in
> > the end the code were fixed. Take a look at "GTAC 2013 Keynote:
> > Evolution from Quality Assurance to Test Engineering" [1]
>
> Not really - I see 60000 open bug reports in Chromium, for example:
> https://bugs.chromium.org/p/chromium/issues/list
> (Only if you want to imply that only "Google" was responsible for
> fixing all bugs in that free and open source project, of course.)
>
> > What if we could show information from the bugs in Phabricator in a
> > "tracked" template at other wiki-projects, identifying the team
> > responsible and perhaps even the dev assigned to the bug? Imagine the
> > creds the dev would get when the bug is fixed! Because it is easy to
> > loose track of pages with "tracked" templates we need some other means
> > to show this information, and our "public monitor" could be a special
> > page with the same information.
>
> Feel free to extend https://www.mediawiki.org/wiki/Template:Tracked
>
> > We say we don't want voting over bugs, but by saying that we refuse
> > getting stats over how many users a specific bug hits, and because of
> > that we don't get sufficient information (metrics) to make decisions
> > about specific bugs.
>
> I disagree. Different people see different priorities. Longer version:
> https://www.mediawiki.org/wiki/Bug_management/Development_prioritization
>
> > What if users could give a "this hits me too" from a "tracked"
> > template. That would give a very simple metric on how important it is
> > to fix a problem.
>
> It does not, because software development is not a popularity contest:
> https://www.mediawiki.org/wiki/Bug_management/Development_prioritization
> Voting would create expectations that nobody will fulfill.
>
> Cheers,
> andre
> --
> Andre Klapper | Bugwrangler / Developer Advocate
> https://blogs.gnome.org/aklapper/
>
>
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

John Erling Blad
In reply to this post by Andre Klapper-2
Yes, there should always be a response to all bugs. Without a response
the impression in the reporting wiki-community would be "nobody cares
about our bug reports".

Someone in the community finds a bug, and it is posted and discussed
in the community. Then another one writes a report in a task at
Phabricator, but nothing further happen.  A couple of months later the
first one ask again about the bug, but does not get a satisfactory
answer, and gets angry. This usually happen in cycles of a few months
to a year. We must somehow break those cycles, they are bad and
disruptive and creates a "us and them" attitude.

Users from the wiki-communities don't visit Phabricator to see all
those small administrative tasks, they see the notes from the official
and unofficial tech ambassadors, and they see the changes in the
"tracked" templates. The templates are only changed when the bugs are
closed for whatever reason, which could take years. Creating
additional manual interventions does not work, the process must be
simpler and more efficient.

On Thu, Mar 14, 2019 at 1:23 PM Andre Klapper <[hidden email]> wrote:

>
> On Tue, 2019-03-12 at 00:29 +0100, John Erling Blad wrote:
> > It seems like some projects simply put everything coming from external
> > sources into deep freezer or add "need volunteer". If they respond at
> > all. In some cases it could be that the projects are defunc.
>
> What's the expectation based on that there should always be a response?
> If a bug report has all info needed to allow someone to reproduce and
> work on it, anyone is free to pick it up and work on it if anyone is
> interested in working on it. No further response needed.
>
> Cheers,
> andre
> --
> Andre Klapper | Bugwrangler / Developer Advocate
> https://blogs.gnome.org/aklapper/
>
>
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

David Barratt
Is the Wikimedia Foundation responsible for people's emotions?

On Thu, Mar 14, 2019 at 10:51 AM John Erling Blad <[hidden email]> wrote:

> Yes, there should always be a response to all bugs. Without a response
> the impression in the reporting wiki-community would be "nobody cares
> about our bug reports".
>
> Someone in the community finds a bug, and it is posted and discussed
> in the community. Then another one writes a report in a task at
> Phabricator, but nothing further happen.  A couple of months later the
> first one ask again about the bug, but does not get a satisfactory
> answer, and gets angry. This usually happen in cycles of a few months
> to a year. We must somehow break those cycles, they are bad and
> disruptive and creates a "us and them" attitude.
>
> Users from the wiki-communities don't visit Phabricator to see all
> those small administrative tasks, they see the notes from the official
> and unofficial tech ambassadors, and they see the changes in the
> "tracked" templates. The templates are only changed when the bugs are
> closed for whatever reason, which could take years. Creating
> additional manual interventions does not work, the process must be
> simpler and more efficient.
>
> On Thu, Mar 14, 2019 at 1:23 PM Andre Klapper <[hidden email]>
> wrote:
> >
> > On Tue, 2019-03-12 at 00:29 +0100, John Erling Blad wrote:
> > > It seems like some projects simply put everything coming from external
> > > sources into deep freezer or add "need volunteer". If they respond at
> > > all. In some cases it could be that the projects are defunc.
> >
> > What's the expectation based on that there should always be a response?
> > If a bug report has all info needed to allow someone to reproduce and
> > work on it, anyone is free to pick it up and work on it if anyone is
> > interested in working on it. No further response needed.
> >
> > Cheers,
> > andre
> > --
> > Andre Klapper | Bugwrangler / Developer Advocate
> > https://blogs.gnome.org/aklapper/
> >
> >
> >
> > _______________________________________________
> > Wikitech-l mailing list
> > [hidden email]
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

Wikipedia Developers mailing list

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, March 14, 2019 3:56 PM, David Barratt <[hidden email]> wrote:

> Is the Wikimedia Foundation responsible for people's emotions?

Yes. Frustration, mostly. It is not entirely unexpected that this message originates from @wikimedia.org.

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

Andre Klapper-2
In reply to this post by John Erling Blad
On Thu, 2019-03-14 at 15:50 +0100, John Erling Blad wrote:

> Yes, there should always be a response to all bugs. Without a response
> the impression in the reporting wiki-community would be "nobody cares
> about our bug reports".
>
> Someone in the community finds a bug, and it is posted and discussed
> in the community. Then another one writes a report in a task at
> Phabricator, but nothing further happen.  A couple of months later the
> first one ask again about the bug, but does not get a satisfactory
> answer, and gets angry. This usually happen in cycles of a few months
> to a year. We must somehow break those cycles, they are bad and
> disruptive and creates a "us and them" attitude.

I've seen it a few times on wiki village pumps or wiki article talk
pages that someone points out something and then nobody else replied
(or "nobody cared", as you call it).
And then people "get angry" as you call it.
 
Do you manage to reply to all and each post in your local wiki
community, or how do you deal with this problem?

andre
--
Andre Klapper | Bugwrangler / Developer Advocate
https://blogs.gnome.org/aklapper/



_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

Gergő Tisza
In reply to this post by John Erling Blad
On Thu, Mar 14, 2019 at 4:36 AM John Erling Blad <[hidden email]> wrote:

> Google had a problem with unfixed bugs, and they started identifying
> the involved developers each time the build was broken. That is pretty
> harsh, but what if devs somehow was named when their bugs were
> mentioned? What if there were some kind of public statistic? How would
> the devs react to being identified with a bug? Would they fix the bug,
> or just be mad about it? Devs at some of Googles teams got mad, but in
> the end the code were fixed. Take a look at "GTAC 2013 Keynote:
> Evolution from Quality Assurance to Test Engineering" [1]
>

Sorry to be direct but you seem to have little understanding of what you
are talking about. You are equivocating different things and shifting
goalposts every time you comment on this thread. You are jumping between
various positions involving "a large bug backlog is bad", "important bugs
are not getting prioritized accordingly", "the WMF should scale its
services down so it has the capacity to respond to every request" (ie. fire
some developers, hire more community liasions), and now you are talking
about broken builds. Every time someone challenges your claims, you just
switch to talking about another one. This is frustrating and a waste of
other people's time. Please try to pin down what you are trying to propose
before making the proposal.

For those unfamiliar with development processes, a broken build means the
application is not starting at all, which means other developers cannot
test their own work, which means the entire development process grinds to a
halt. That is a huge hit to productivity so software organizations usually
try hard to avoid it, even though it's an internal issue with no user
impact (well, other than the organization shipping less features / fixes in
the next release because developer time was spent less effectively).
The closest equivalent we have for that is continuous integration tests
broken by merged code (although that's less severe since it usually doesn't
stop the application from working). While I'm sure the handling of those
could be improved (incidentally, that's just happening, see
https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/CI_Futures_WG
), it has nothing to do with the original topic of this thread, since it is
happening in the development cycle and not visible to users.

About backlogs in general, Chromium is probably the biggest
open-source Google repo; that has currently 940K tickets, 60K of which are
open, and another 50K have been auto-archived after a year of inactivity.
(As others have pointed out, having a huge backlog and ruthlessly closing
tasks that do not get on the roadmap are the only two realistic options,
and the latter does have its advantages, no one here seems to be in favor
of it.) We have 220K tasks in total, 40K of which are open, so that's in
the same ballpark (not that that comparing open task ratios is particularly
meaningful  - but it hopefully shows that Google's handling of the
completely unrelated build breaking issue does not magically make them have
zero bugs).

What if we could show information from the bugs in Phabricator in a
> "tracked" template at other wiki-projects, identifying the team
> responsible and perhaps even the dev assigned to the bug?


Users who are interested in that information would be spared the enormous
effort of pressing a button on the mouse, I guess?


> We say we don't want voting over bugs, but by saying that we refuse
> getting stats over how many users a specific bug hits, and because of
> that we don't get sufficient information (metrics) to make decisions
> about specific bugs. Some bugs (or missing features) although changes
> how users are doing specific things, how do we handle that?
>

How many people vote on a bug and how many people are hit by a bug are two
entirely different things, and most of the time it's hard to measure the
latter. Voting will be dominated by power users who are more engaged with
the development process, users who understand English, users who come from
a larger wiki community and can canvass better, etc. (And Phabricator
doesn't support voting anyway.)


> What if users could give a "this hits me too" from a "tracked"
> template. That would give a very simple metric on how important it is
> to fix a problem. To make this visible to the wiki-communities the
> special page could be sorted on this metric. Of course the devs would
> have completely different priorities, but this page would list the
> wiki-communities priorities.
>

Having a page which lists the priorities of wiki communities (more
realistically, one specific wiki community) would be very useful, IMO. As
others have pointed out, the reason no such list exists is that people are
spending their time complaining here, instead of building lists on their
wiki. (At which point they would quickly find out that actually getting a
consensus on priorities is a lot harder than complaining about why
everything is not being worked on at the same time.)
Various people have done various priority lists in the past, some of which
have been fairly successful in directing attention. It's definitely worth a
shot doing the same for the community you intend to represent. Voting is
not a serious replacement for that, though.
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

Gergő Tisza
In reply to this post by Strainu
On Wed, Mar 13, 2019 at 3:02 PM Strainu <[hidden email]> wrote:

> The main problem I see with the community wishlist is that it's a
> process beside the normal process, not part of it. The dedicated team
> takes 10 bugs and other developers another ~10. I think we would be
> much better off if each team at the WMF would also take the top ranked
> bug on their turf and solve it and bump the priority of all the other
> bugs by one (e.g. low->medium). One bug per year per team means at
> least 18 bugs (at least if [1] is up to date) or something similar.
>

Community Tech is seven people and they do ten wishlist requests a year.
(Granted, they do other things too, but the wishlist is their main focus.)
So you are proposing to reallocate on average 1-2 months per year for every
team to work on wishlist wishes. That's about two million dollars of donor
money. How confident are you that the wishlist is actually a good way of
estimating the impact of tasks, outside of the narrow field where editors
have personal experience (ie. editing tools)?

What a wonderful world that would be! Unfortunately, all too often I
> feel that objective measures (such as "+1" on bugs, duplicates etc.)
> have no effect on prioritization.
>

Leaving aside how objective those measures are, in my when the task is
related to a product owned by a team, they are aware and take it into
account. (Which does not necessarily mean they agree, of course.) A lot of
production components don't really have an owner, though. (Or only do to
the extent that there is someone who can be pulled away from their normal
job if the thing catches fire.) That's just the reality of running the
website we have with the capacity we have - the alternative would be
undeploying things or not starting new projects for a long time. The
Wikimedia movement happens to be in the middle of its strategic planning
process, so if you want to argue for either of these things, this is a good
time to do it. (Not a good place, though.)

- UploadWizard (2 with high priority, 40 with normal, a few dozens
> low, hundreds more untriaged): this is the project that got us out of
> the "overloading the lang parameter for customizing the uploader" era,
> the project that is used by millions of people every year, including
> during every photo contest
>

UploadWizard is not in active development currently. If you want to argue
that the Multimedia team should be reassigned to work on it (and drop the
Structured Data on Commons project), or some other rearrangement should be
made, that's a specific proposal that can be meaningfully discussed.
(Probably not here, though - that's a matter of movement strategy, not a
technical decision. So maybe better suited to wikimedia-l.)
Saying that some project should be picked up, without saying what should be
dropped to make space, is an easy thing to do. Not all that useful though.

(As an aside, I'd love to see more people self-organize to get more say in
how priorities are decided. If you look at the discussion pages on WMF
annual plans, movement strategy and so on, they do not give the impression
of a community that's seriously interested in its own future.)
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

Andre Klapper-2
In reply to this post by John Erling Blad
On Wed, 2019-03-13 at 01:22 +0100, John Erling Blad wrote:
> What frustrates me the most are
>
> - bugs found by the editor community, that has obvious simple fixes,
> which isn't acted upon for several years
> - new features that isn't fully tested, and you have to answer in the
> community about stuff you rather want to throw out
> - new features and changes that are advertised but never implemented

Could you please provide a specific example (link) for the last item?

> The first one is perhaps the one most easily fixed. I believe WMF
> could set up either an official bug squad

In my understanding a bug squad does not write patches but triages
tickets. Maybe you meant a patchsquad or Gerrit wrangler (in case you
refer to *existing* simple fixes, which is not clear from your words).
Not sure why there is call to some authority ("WMF") here.

> or use bug bounties to speed up fixing of bugs. I tend to believe bug
> bounties works best, but it would be really nice to know that bugs
> are handled in an orderly fashion by a bug squad.

See https://phabricator.wikimedia.org/T88265#1870218 for risks and
(dis)advantages of bug bounties.
Note that throwing more 'external' developers onto code does not
necessarily "speed up fixing of bugs". Often to the contrary.

> When introducing new features make a help page at Meta or Mediawiki,
> and link to the page from the feature.

Looking at the beta features section at
https://meta.wikimedia.org/wiki/Special:Preferences#mw-prefsection-betafeatures
all beta features have "Discussion" and "Information" links.
What you are suggesting already happens.

> On that page make a visible link "Don't panic!" and link to the issue
> tracker. Don't expect the users to figure out which extension
> provides the specific feature, they don't have a clue.
> For all important issues make an estimated fix time, and if no one
> works on the issue say so.

When nobody works on an issue, Phabricator's "Assigned To" field
usually shows "None". What you are suggesting already happens.

Cookie licking can happen though (means: someone assigns a ticket to
themselves and then does not work on it). It's up to each assignee to
occasionally check which tasks are (still) assigned to them:
https://phabricator.wikimedia.org/maniphest/query/5INV_avtCreJ/#R

> Don't assume the users understand fancy wording about "need
> volunteer". Need volunteer for what? Making coffee?
>
> Some features are described in Phabricator, which is fine, but some
> of
> them has extensive cookie licking which could give someone the
> impression that you actually will implement the feature.

Could you please provide a specific example (link) for this?

> That often leads to users asking about the feature, and when it will
> arrive at their project. When it does not arrive users gets upset. If
> you are working on something, say it, but also be very clear if
> something has gone into you personal freezer.

Cheers,
andre
--
Andre Klapper | Bugwrangler / Developer Advocate
https://blogs.gnome.org/aklapper/



_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

Stas Malyshev
In reply to this post by John Erling Blad
Hi!

> Yes, there should always be a response to all bugs. Without a response
> the impression in the reporting wiki-community would be "nobody cares
> about our bug reports".

Would a canned "thank you for your feedback, please stay on the line,
your call is very important to us" response make anybody feel better?

The reality of a project with huge userbase and limited resources is
that there are more bugs that can be addressed seriously and
substantially, not with a canned response that does not solve the issue,
than there's developer resource. It doesn't mean "nobody cares about the
bug reports" - it means some bug reports will be cared for first and
some later (and possibly some, unfortunately, never). This set of
priorities can be influenced by alerting developer's attention about
specific bugs needing addressing, and by existing prioritisation
processes, which very much include community input, but the harsh
reality of having a lot of bugs dictates that giving serious non-canned
attention leading to satisfactory outcome to 100% of them is IMHO not
realistic.

We could of course institute the policy of "every bug should have a
comment from a developer within X time" - but unless X is very large, I
think it will be unsatisfactory, since getting "yes, it's a very
important bug, thanks for submitting it" comment without the bug being
fixed is IMHO no better than getting no comment at all.
--
Stas Malyshev
[hidden email]

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

Strainu
In reply to this post by Gergő Tisza
În joi, 14 mar. 2019 la 22:23, Gergő Tisza <[hidden email]> a scris:
> About backlogs in general, Chromium is probably the biggest
> open-source Google repo; that has currently 940K tickets, 60K of which are
> open, and another 50K have been auto-archived after a year of inactivity.
> (As others have pointed out, having a huge backlog and ruthlessly closing
> tasks that do not get on the roadmap are the only two realistic options,
> and the latter does have its advantages, no one here seems to be in favor
> of it.) We have 220K tasks in total, 40K of which are open, so that's in
> the same ballpark

That's an overstatement: 18% (not counting bugs closed as declined) is
almost double to 11%. If you're going this route, we're doing much
worse than Chromium.

>
> On Wed, Mar 13, 2019 at 3:02 PM Strainu <[hidden email]> wrote:
>
> > The main problem I see with the community wishlist is that it's a
> > process beside the normal process, not part of it. The dedicated team
> > takes 10 bugs and other developers another ~10. I think we would be
> > much better off if each team at the WMF would also take the top ranked
> > bug on their turf and solve it and bump the priority of all the other
> > bugs by one (e.g. low->medium). One bug per year per team means at
> > least 18 bugs (at least if [1] is up to date) or something similar.
> >
>
> Community Tech is seven people and they do ten wishlist requests a year.
> (Granted, they do other things too, but the wishlist is their main focus.)
> So you are proposing to reallocate on average 1-2 months per year for every
> team to work on wishlist wishes. That's about two million dollars of donor
> money. How confident are you that the wishlist is actually a good way of
> estimating the impact of tasks, outside of the narrow field where editors
> have personal experience (ie. editing tools)?

I'm 99.9% sure the wishlist is relevant in at least half the
categories (Admins&patrollers, Bots&Gadgets, Editing, Notifications,
Programs&Events, Watchlists, Wikidata, Wikisource, Wiktionary) and
very likely (80%) also for Anti-harassment, Categories and Maps.

I'm not sure how you arrived at the $2M figure (even 36 months of dev
time - 18 teams, 2 man-months/team - only add up to ~$400K, unless
Glasdoor is waaay off on the salaries there [2]), but presumably going
down on the list will also surface bugs and not only features, which
will take less time to solve. Investing an additional 1% of the
revenue into this seems reasonable to me.

[2] https://www.glassdoor.com/Salary/Wikimedia-Foundation-Salaries-E38331.htm

>
> UploadWizard is not in active development currently.

I did not claim (or asked) that it was. What I said is that it is an
important part of the infrastructure and that it should be maintained
properly. I also said I will try to come up with a more detailed
critique later on and see if it has any result.

Strainu

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

Pine W
In reply to this post by Stas Malyshev
Stas made a point that I was considering too, although from a different
perspective.

One of the issues may be a difference between end user expectations and
what the resources are available to fulfill those expectations.

Communications requires time and mental bandwidth, both of which are
limited resources for everyone, including WMF staff and end users. Also,
there are financial considerations regarding how people use their time.

From the end user perspective, reporting a bug and then having nothing
happen, or getting an initial reply but later seeing that a bug appears to
stall for months or years, may be frustrating depending on the nature of
the bug and the patience of the user. I think that communicating with users
regarding when bugs will likely be fixed would be helpful. I think that
some of that happens already, but there's more that can be done. There are
probably ways to automate some of these communications to a degree.

On the larger scale, I don't know whether it's possible to get a good large
scale understanding of all of the open tasks in Phabricator. I speculate
that teams might be able to create semi-automated reports regarding their
own teams' tasks. To get a larger view of the situation in Phabricator
might require combining the unique outputs of the reports from individual
teams. By having a big picture view of the situation I hope that we could
improve our collective situational awareness regarding tasks, including
open feature requests and technical debt. Also, by creating snapshots of
the results of the same type of combined report over a period of months or
years, maybe we could get a sense of how technical debt is changing over
time.

To summarize: I am thinking that a two pronged approach would be good, one
regarding communications regarding the status of individual bugs, and one
regarding a big picture analysis of technical debt.

I realize that there would be costs of time and money for both of those
approaches. Automation can help with both.

My guess is that managing thousands of bugs in an continuous development
environment is challenging in the best of circumstances. I am somewhat
familiar with the end user experience and financial considerations (both of
which motivated me to participate in this thread), but I'm not an expert in
software engineering for a product that is on the scale of a top 10 website.

What do others think regarding these proposals?

Thanks for the good discussion.

Pine
( https://meta.wikimedia.org/wiki/User:Pine )
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

Andre Klapper-2
On Sat, 2019-03-16 at 04:52 +0000, Pine W wrote:

> From the end user perspective, reporting a bug and then having nothing
> happen, or getting an initial reply but later seeing that a bug appears to
> stall for months or years, may be frustrating depending on the nature of
> the bug and the patience of the user. I think that communicating with users
> regarding when bugs will likely be fixed would be helpful. I think that
> some of that happens already, but there's more that can be done. There are
> probably ways to automate some of these communications to a degree.
>
> On the larger scale, I don't know whether it's possible to get a good large
> scale understanding of all of the open tasks in Phabricator.

What does "understanding" imply; which understanding do you lack? It's
hard to help understand without knowing what's specifically missing. :)

> I speculate that teams might be able to create semi-automated reports
> regarding their own teams' tasks.

Which specific problem do you think would be improved by reports?

Not sure if it's what you're after: There are Burndown charts. Example:
https://phabricator.wikimedia.org/maniphest/report/burn/?project=PHID-PROJ-kfrrtvyn66ou2iq4y4ai
or in Phlogiston. For more information, see
https://www.mediawiki.org/wiki/Phabricator/Project_management#Scrum_in_Phabricator

> To get a larger view of the situation in Phabricator might require
> combining the unique outputs of the reports from individual teams.

Can you explain why a "larger view" would solve a problem and which
problem that is? It's probably not 'some bugs will never get fixed'.
Maybe it's 'some tickets don't get a reply'. Maybe it is 'some tickets
should get prioritized differently'. Or maybe something else.
These are different things...

> By having a big picture view of the situation I hope that we could
> improve our collective situational awareness regarding tasks,
> including open feature requests and technical debt. Also, by creating
> snapshots of the results of the same type of combined report over a
> period of months or years, maybe we could get a sense of how
> technical debt is changing over time.

For the records, that would require excluding feature requests from any
statistics. Plus definitions and understanding of tech debt differ.
Plus tech debt can also be "TODO" code comments and many other things.

> To summarize: I am thinking that a two pronged approach would be good, one
> regarding communications regarding the status of individual bugs

If something happens on an individual ticket, someone communicates that
in that ticket. Examples: People move some tasks on workboards, people
assign some tasks, people add some comments. What exactly is missing?
Do you have an example that's not abstract high-level, please? :)

> and one regarding a big picture analysis of technical debt.
>
> I realize that there would be costs of time and money for both of those
> approaches. Automation can help with both.

It's still unclear to me which actual underlying problem(s) you'd like
to get solved. The message seems to mix several things ('no reply to
some tasks', 'some tasks might get replies but not fixed', 'some tasks
should be prioritized differently' etc) but maybe I misunderstand.

andre
--
Andre Klapper | Bugwrangler / Developer Advocate
https://blogs.gnome.org/aklapper/



_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

David Barratt
In reply to this post by Strainu
Perhaps a better example would be the Drupal community who has a total of
~1,071,600 issues and ~282,350 of those are open
https://www.drupal.org/project/issues and they have several organizations
https://www.drupal.org/organizations working on the software.

I do not understand how a large backlog is a problem. It is not an
indication of anything.


On Fri, Mar 15, 2019 at 12:25 PM Strainu <[hidden email]> wrote:

> În joi, 14 mar. 2019 la 22:23, Gergő Tisza <[hidden email]> a scris:
> > About backlogs in general, Chromium is probably the biggest
> > open-source Google repo; that has currently 940K tickets, 60K of which
> are
> > open, and another 50K have been auto-archived after a year of inactivity.
> > (As others have pointed out, having a huge backlog and ruthlessly closing
> > tasks that do not get on the roadmap are the only two realistic options,
> > and the latter does have its advantages, no one here seems to be in favor
> > of it.) We have 220K tasks in total, 40K of which are open, so that's in
> > the same ballpark
>
> That's an overstatement: 18% (not counting bugs closed as declined) is
> almost double to 11%. If you're going this route, we're doing much
> worse than Chromium.
>
> >
> > On Wed, Mar 13, 2019 at 3:02 PM Strainu <[hidden email]> wrote:
> >
> > > The main problem I see with the community wishlist is that it's a
> > > process beside the normal process, not part of it. The dedicated team
> > > takes 10 bugs and other developers another ~10. I think we would be
> > > much better off if each team at the WMF would also take the top ranked
> > > bug on their turf and solve it and bump the priority of all the other
> > > bugs by one (e.g. low->medium). One bug per year per team means at
> > > least 18 bugs (at least if [1] is up to date) or something similar.
> > >
> >
> > Community Tech is seven people and they do ten wishlist requests a year.
> > (Granted, they do other things too, but the wishlist is their main
> focus.)
> > So you are proposing to reallocate on average 1-2 months per year for
> every
> > team to work on wishlist wishes. That's about two million dollars of
> donor
> > money. How confident are you that the wishlist is actually a good way of
> > estimating the impact of tasks, outside of the narrow field where editors
> > have personal experience (ie. editing tools)?
>
> I'm 99.9% sure the wishlist is relevant in at least half the
> categories (Admins&patrollers, Bots&Gadgets, Editing, Notifications,
> Programs&Events, Watchlists, Wikidata, Wikisource, Wiktionary) and
> very likely (80%) also for Anti-harassment, Categories and Maps.
>
> I'm not sure how you arrived at the $2M figure (even 36 months of dev
> time - 18 teams, 2 man-months/team - only add up to ~$400K, unless
> Glasdoor is waaay off on the salaries there [2]), but presumably going
> down on the list will also surface bugs and not only features, which
> will take less time to solve. Investing an additional 1% of the
> revenue into this seems reasonable to me.
>
> [2]
> https://www.glassdoor.com/Salary/Wikimedia-Foundation-Salaries-E38331.htm
>
> >
> > UploadWizard is not in active development currently.
>
> I did not claim (or asked) that it was. What I said is that it is an
> important part of the infrastructure and that it should be maintained
> properly. I also said I will try to come up with a more detailed
> critique later on and see if it has any result.
>
> Strainu
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

Strainu
În sâm., 16 mar. 2019 la 15:55, David Barratt <[hidden email]> a scris:
>
> Perhaps a better example would be the Drupal community who has a total of
> ~1,071,600 issues and ~282,350 of those are open
> https://www.drupal.org/project/issues and they have several organizations
> https://www.drupal.org/organizations working on the software.

Maybe, maybe not - I'm not familiar with Drupal development, but
precisely because of the fragmented contributions, chances are some
bugs fall between teams. As discussed previously in the thread, MW
development is much more centralized, so better coordination is to be
expected.

That being said, their org stats are pretty awsome, is there any way
to obtain similar stats from Phabricator/Gerrit (at least by email
domain if nothing else)?

>
> I do not understand how a large backlog is a problem. It is not an
> indication of anything.

A large backlog by itself is not alarming. A growing one for
components deployed to WMF sites is. It indicates insufficient
attention is given to ongoing maintenance of projects after they are
no longer "actively developed", which in turn creates resentment with
the reporters.

I've checked the burnup graphs Andre referred to for some of the
extensions with high editor visibility (UW, VE, CX) and they all have
a similar pattern - huge increase in the first ~12 months after being
widely deployed, then a much reduced, but visible, growing rate, with
some sharp decreases which correspond to a peak in activity (new team
culling the backlog? volunteer developer solving a few bugs?). I tried
to compare that with the overall pattern, but Phabricator timed out -
if somebody could obtain and publish the overall burnup rate data
somewhere, that would be great.

I guess the question is what's an acceptable backlog growing rate (key
secondary question: for who?) and if it is different between projects.
I don't know how to respond to that.

> On Fri, Mar 15, 2019 at 12:25 PM Strainu <[hidden email]> wrote:
>
> > În joi, 14 mar. 2019 la 22:23, Gergő Tisza <[hidden email]> a scris:
> > > About backlogs in general, Chromium is probably the biggest
> > > open-source Google repo; that has currently 940K tickets, 60K of which
> > are
> > > open, and another 50K have been auto-archived after a year of inactivity.
> > > (As others have pointed out, having a huge backlog and ruthlessly closing
> > > tasks that do not get on the roadmap are the only two realistic options,
> > > and the latter does have its advantages, no one here seems to be in favor
> > > of it.) We have 220K tasks in total, 40K of which are open, so that's in
> > > the same ballpark
> >
> > That's an overstatement: 18% (not counting bugs closed as declined) is
> > almost double to 11%. If you're going this route, we're doing much
> > worse than Chromium.
> >
> > >
> > > On Wed, Mar 13, 2019 at 3:02 PM Strainu <[hidden email]> wrote:
> > >
> > > > The main problem I see with the community wishlist is that it's a
> > > > process beside the normal process, not part of it. The dedicated team
> > > > takes 10 bugs and other developers another ~10. I think we would be
> > > > much better off if each team at the WMF would also take the top ranked
> > > > bug on their turf and solve it and bump the priority of all the other
> > > > bugs by one (e.g. low->medium). One bug per year per team means at
> > > > least 18 bugs (at least if [1] is up to date) or something similar.
> > > >
> > >
> > > Community Tech is seven people and they do ten wishlist requests a year.
> > > (Granted, they do other things too, but the wishlist is their main
> > focus.)
> > > So you are proposing to reallocate on average 1-2 months per year for
> > every
> > > team to work on wishlist wishes. That's about two million dollars of
> > donor
> > > money. How confident are you that the wishlist is actually a good way of
> > > estimating the impact of tasks, outside of the narrow field where editors
> > > have personal experience (ie. editing tools)?
> >
> > I'm 99.9% sure the wishlist is relevant in at least half the
> > categories (Admins&patrollers, Bots&Gadgets, Editing, Notifications,
> > Programs&Events, Watchlists, Wikidata, Wikisource, Wiktionary) and
> > very likely (80%) also for Anti-harassment, Categories and Maps.
> >
> > I'm not sure how you arrived at the $2M figure (even 36 months of dev
> > time - 18 teams, 2 man-months/team - only add up to ~$400K, unless
> > Glasdoor is waaay off on the salaries there [2]), but presumably going
> > down on the list will also surface bugs and not only features, which
> > will take less time to solve. Investing an additional 1% of the
> > revenue into this seems reasonable to me.
> >
> > [2]
> > https://www.glassdoor.com/Salary/Wikimedia-Foundation-Salaries-E38331.htm
> >
> > >
> > > UploadWizard is not in active development currently.
> >
> > I did not claim (or asked) that it was. What I said is that it is an
> > important part of the infrastructure and that it should be maintained
> > properly. I also said I will try to come up with a more detailed
> > critique later on and see if it has any result.
> >
> > Strainu
> >
> > _______________________________________________
> > Wikitech-l mailing list
> > [hidden email]
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Question to WMF: Backlog on bugs

Thomas Eugene Bishop
In reply to this post by Andre Klapper-2
> On Mar 13, 2019, at 6:48 PM, Andre Klapper <[hidden email] <mailto:[hidden email]>> wrote:
>
> On Wed, 2019-03-13 at 21:01 +0100, John Erling Blad wrote:
>> ... But nobody does anything about the
>> sinkhole itself.
>
> And repeating the same thing over and over again while repeatedly
> ignoring requests to be more specific won't help either...

Here’s a specific example, created in 2015:

        https://phabricator.wikimedia.org/T116145 <https://phabricator.wikimedia.org/T116145>


A bug fix was provided years ago but never accepted or rejected. It’s the first and last MediaWiki bug ever assigned to me. I’ve just unassigned myself.

In cases like this, remarks like “Because you did not fix these bugs” and “... anyone is free to pick it up and work on it ... No further response needed” miss the point. When a bug fix is provided, but nobody with authority to accept or reject it ever does so, that’s a failure on the part of those who have authority, not on the part of those who are able and willing to fix bugs. Sure, volunteers are “free” to waste their time!

You need to use and share your authority more effectively, to “be bold” with accepting and rejecting bug fixes. Authorize more people to accept or reject bug fixes. Assign each proposed bug fix to one such person, starting with the oldest bugs. Then hold those people accountable. You don’t lack volunteers, you lack volunteers with authority.

Best wishes,

Tom

Wenlin Institute, Inc. SPC (a Social Purpose Corporation)
文林研究所社会目的公司
Software for Learning Chinese
E-mail: [hidden email] <mailto:[hidden email]>     Web: http://www.wenlin.com <http://www.wenlin.com/>
Telephone: 1-877-4-WENLIN (1-877-493-6546)


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
12345