Wikimedia's GitHub org help

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Wikimedia's GitHub org help

Tyler Cipriani
Hi all!

If you've never created a repo or fork on the Wikimedia GitHub
organization you can skip this email.

I know that some repos are developed on our GitHub org for reasons.
What is developed on our GitHub org? How many things are actively
being developed on GitHub org? I have no idea :)

I recently realized that there's not a great way to figure this
out[0], but I've been able to narrow the scope a bit. Now I have a
list of repos that are (a) in our GitHub org and (b) not in our Gerrit
that I could use some help sorting through[1].

== Help, please ==

* Look through repos on The List™[1]

If your repos are on the list, for each of your repos either:

* Archive or Delete it if it's no longer maintained or empty/useless,
respectively (and remove them from the list on mw.org)[2]

Or:

* put a "{{tick}}" in the "Active" column on the list on mw.org

== Why==

In a more perfect future we could add the "mirror"[3] tag to repos on
GitHub that are mirrored from Gerrit (with a link to their canonical
repo locations; for example, gnome-deskop has this[4] and I'm very
jealous).

Hopefully, this will help folks wanting to contribute -- either a
Wikimedia GitHub repo is a mirror (in which case there's a link to
Gerrit in the description) or it's actively being developed on GitHub.

<3
-- Tyler

[0]: <https://phabricator.wikimedia.org/T237470#6407509>
[1]: <https://www.mediawiki.org/wiki/Gerrit/GitHub#Projects_on_GitHub>
[2]: <https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/archiving-a-github-repository>
[3]: <https://docs.github.com/en/github/getting-started-with-github/finding-ways-to-contribute-to-open-source-on-github#open-source-projects-with-mirrors-on-github>
[4]: <https://github.com/GNOME/gnome-desktop>

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikimedia's GitHub org help

addshorewiki
Is the main source of mirrors gerrit?
If so could we not write a script looking for .gitreview files and looking
at the URL in there?
I imagine there is also some API for marking things as mirrored? (or is it
more manual than that?)

Another thought would be adding some .wmgithub file with structured info
about repos that are on github.
Then rather than maintaining a manual list that is likely to get out of
date we could write a thin UI infront of the data in these files and the
GitHub API?

On Mon, 24 Aug 2020 at 23:47, Tyler Cipriani <[hidden email]>
wrote:

> Hi all!
>
> If you've never created a repo or fork on the Wikimedia GitHub
> organization you can skip this email.
>
> I know that some repos are developed on our GitHub org for reasons.
> What is developed on our GitHub org? How many things are actively
> being developed on GitHub org? I have no idea :)
>
> I recently realized that there's not a great way to figure this
> out[0], but I've been able to narrow the scope a bit. Now I have a
> list of repos that are (a) in our GitHub org and (b) not in our Gerrit
> that I could use some help sorting through[1].
>
> == Help, please ==
>
> * Look through repos on The List™[1]
>
> If your repos are on the list, for each of your repos either:
>
> * Archive or Delete it if it's no longer maintained or empty/useless,
> respectively (and remove them from the list on mw.org)[2]
>
> Or:
>
> * put a "{{tick}}" in the "Active" column on the list on mw.org
>
> == Why==
>
> In a more perfect future we could add the "mirror"[3] tag to repos on
> GitHub that are mirrored from Gerrit (with a link to their canonical
> repo locations; for example, gnome-deskop has this[4] and I'm very
> jealous).
>
> Hopefully, this will help folks wanting to contribute -- either a
> Wikimedia GitHub repo is a mirror (in which case there's a link to
> Gerrit in the description) or it's actively being developed on GitHub.
>
> <3
> -- Tyler
>
> [0]: <https://phabricator.wikimedia.org/T237470#6407509>
> [1]: <https://www.mediawiki.org/wiki/Gerrit/GitHub#Projects_on_GitHub>
> [2]: <
> https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/archiving-a-github-repository
> >
> [3]: <
> https://docs.github.com/en/github/getting-started-with-github/finding-ways-to-contribute-to-open-source-on-github#open-source-projects-with-mirrors-on-github
> >
> [4]: <https://github.com/GNOME/gnome-desktop>
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikimedia's GitHub org help

Tyler Cipriani
On Tue, Aug 25, 2020 at 1:17 AM Addshore <[hidden email]> wrote:
> Is the main source of mirrors gerrit?

Yes, as of yesterday there are 2,493 Gerrit repos, 2,175 GitHub repos,
and 296 of those GitHub repos have no corresponding Gerrit repo from
which they are mirrored. The remaining 1,879 repos in github are
mirrored from gerrit.

> If so could we not write a script looking for .gitreview files and looking
> at the URL in there?

You mean to find the repo from which it originated? That's possible;
it's got some caveats. For example, there are ".gitreview" files on
github pointing to non-existent gerrit repos[0]. These just have to be
cleaned up manually, I think.

If it's useful, I wrote a handful of messy shell scripts (as is my
wont) that invoke the github api to come up with the list of 296 repos
that are on github but have no corresponding gerrit repo[1].

> I imagine there is also some API for marking things as mirrored? (or is it
> more manual than that?)

I talked to GitHub support about getting the "mirrored" tag for our
repos; it's totally manual and has to go through folks at GitHub
support is what I was told[2] :(

> Another thought would be adding some .wmgithub file with structured info
> about repos that are on github.
> Then rather than maintaining a manual list that is likely to get out of
> date we could write a thin UI infront of the data in these files and the
> GitHub API?

Making a UI/tool that monitors github repo creation seems like a good
idea rather than this list. My hope is that after some overdue manual
cleanup our github org will be clean enough to be able to make
inferences based on heuristics without having to add exogenous
metadata.

-- Tyler

[0]: <https://github.com/wikimedia/mediawiki-extensions-AddMetaAndTitle/blob/master/.gitreview>
[1]: <https://github.com/thcipriani/wikimedia-github-projects>
[2]: <https://phabricator.wikimedia.org/T237470#6406876>

>
> On Mon, 24 Aug 2020 at 23:47, Tyler Cipriani <[hidden email]>
> wrote:
>
> > Hi all!
> >
> > If you've never created a repo or fork on the Wikimedia GitHub
> > organization you can skip this email.
> >
> > I know that some repos are developed on our GitHub org for reasons.
> > What is developed on our GitHub org? How many things are actively
> > being developed on GitHub org? I have no idea :)
> >
> > I recently realized that there's not a great way to figure this
> > out[0], but I've been able to narrow the scope a bit. Now I have a
> > list of repos that are (a) in our GitHub org and (b) not in our Gerrit
> > that I could use some help sorting through[1].
> >
> > == Help, please ==
> >
> > * Look through repos on The List™[1]
> >
> > If your repos are on the list, for each of your repos either:
> >
> > * Archive or Delete it if it's no longer maintained or empty/useless,
> > respectively (and remove them from the list on mw.org)[2]
> >
> > Or:
> >
> > * put a "{{tick}}" in the "Active" column on the list on mw.org
> >
> > == Why==
> >
> > In a more perfect future we could add the "mirror"[3] tag to repos on
> > GitHub that are mirrored from Gerrit (with a link to their canonical
> > repo locations; for example, gnome-deskop has this[4] and I'm very
> > jealous).
> >
> > Hopefully, this will help folks wanting to contribute -- either a
> > Wikimedia GitHub repo is a mirror (in which case there's a link to
> > Gerrit in the description) or it's actively being developed on GitHub.
> >
> > <3
> > -- Tyler
> >
> > [0]: <https://phabricator.wikimedia.org/T237470#6407509>
> > [1]: <https://www.mediawiki.org/wiki/Gerrit/GitHub#Projects_on_GitHub>
> > [2]: <
> > https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/archiving-a-github-repository
> > >
> > [3]: <
> > https://docs.github.com/en/github/getting-started-with-github/finding-ways-to-contribute-to-open-source-on-github#open-source-projects-with-mirrors-on-github
> > >
> > [4]: <https://github.com/GNOME/gnome-desktop>
> >
> > _______________________________________________
> > Wikitech-l mailing list
> > [hidden email]
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikimedia's GitHub org help

Tyler Cipriani
In reply to this post by Tyler Cipriani
Hi all!

Thank you for reviewing my list of the Wikimedia org's GitHub projects!

 A *lot* of repos were archived or deleted. Additionally, I archived
all projects that contained nothing but a ".gitreview" without a
corresponding Gerrit repository.

As a result, we've gone from 296 repositories exclusively on GitHub to
154 repositories exclusively on GitHub.

I believe this removed a lot of cruft and now all 2,038 repos on
Wikimedia's GitHub org are either:
* A mirror of a Gerrit repo (1,881 repos)
* A mirror of a Differential repo (3 repos)
* A project developed on GitHub or a fork of another GitHub project (154 repos)

I've updated the list of projects[0] that are exclusive to GitHub
(GitHub fork or a project developed on GItHub) if you'd like to take a
look.

Thanks again for all your help!

<3
-- Tyler

[0]. <https://www.mediawiki.org/wiki/Gerrit/GitHub#Projects_on_GitHub>

On Mon, Aug 24, 2020 at 4:46 PM Tyler Cipriani <[hidden email]> wrote:

>
> Hi all!
>
> If you've never created a repo or fork on the Wikimedia GitHub
> organization you can skip this email.
>
> I know that some repos are developed on our GitHub org for reasons.
> What is developed on our GitHub org? How many things are actively
> being developed on GitHub org? I have no idea :)
>
> I recently realized that there's not a great way to figure this
> out[0], but I've been able to narrow the scope a bit. Now I have a
> list of repos that are (a) in our GitHub org and (b) not in our Gerrit
> that I could use some help sorting through[1].
>
> == Help, please ==
>
> * Look through repos on The List™[1]
>
> If your repos are on the list, for each of your repos either:
>
> * Archive or Delete it if it's no longer maintained or empty/useless,
> respectively (and remove them from the list on mw.org)[2]
>
> Or:
>
> * put a "{{tick}}" in the "Active" column on the list on mw.org
>
> == Why==
>
> In a more perfect future we could add the "mirror"[3] tag to repos on
> GitHub that are mirrored from Gerrit (with a link to their canonical
> repo locations; for example, gnome-deskop has this[4] and I'm very
> jealous).
>
> Hopefully, this will help folks wanting to contribute -- either a
> Wikimedia GitHub repo is a mirror (in which case there's a link to
> Gerrit in the description) or it's actively being developed on GitHub.
>
> <3
> -- Tyler
>
> [0]: <https://phabricator.wikimedia.org/T237470#6407509>
> [1]: <https://www.mediawiki.org/wiki/Gerrit/GitHub#Projects_on_GitHub>
> [2]: <https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/archiving-a-github-repository>
> [3]: <https://docs.github.com/en/github/getting-started-with-github/finding-ways-to-contribute-to-open-source-on-github#open-source-projects-with-mirrors-on-github>
> [4]: <https://github.com/GNOME/gnome-desktop>

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikimedia's GitHub org help

Greg Rundlett (freephile)
Thanks for doing this Tyler!

Greg Rundlett
https://eQuality-Tech.com
https://freephile.org


On Tue, Sep 1, 2020 at 11:56 AM Tyler Cipriani <[hidden email]>
wrote:

> Hi all!
>
> Thank you for reviewing my list of the Wikimedia org's GitHub projects!
>
>  A *lot* of repos were archived or deleted. Additionally, I archived
> all projects that contained nothing but a ".gitreview" without a
> corresponding Gerrit repository.
>
> As a result, we've gone from 296 repositories exclusively on GitHub to
> 154 repositories exclusively on GitHub.
>
> I believe this removed a lot of cruft and now all 2,038 repos on
> Wikimedia's GitHub org are either:
> * A mirror of a Gerrit repo (1,881 repos)
> * A mirror of a Differential repo (3 repos)
> * A project developed on GitHub or a fork of another GitHub project (154
> repos)
>
> I've updated the list of projects[0] that are exclusive to GitHub
> (GitHub fork or a project developed on GItHub) if you'd like to take a
> look.
>
> Thanks again for all your help!
>
> <3
> -- Tyler
>
> [0]. <https://www.mediawiki.org/wiki/Gerrit/GitHub#Projects_on_GitHub>
>
> On Mon, Aug 24, 2020 at 4:46 PM Tyler Cipriani <[hidden email]>
> wrote:
> >
> > Hi all!
> >
> > If you've never created a repo or fork on the Wikimedia GitHub
> > organization you can skip this email.
> >
> > I know that some repos are developed on our GitHub org for reasons.
> > What is developed on our GitHub org? How many things are actively
> > being developed on GitHub org? I have no idea :)
> >
> > I recently realized that there's not a great way to figure this
> > out[0], but I've been able to narrow the scope a bit. Now I have a
> > list of repos that are (a) in our GitHub org and (b) not in our Gerrit
> > that I could use some help sorting through[1].
> >
> > == Help, please ==
> >
> > * Look through repos on The List™[1]
> >
> > If your repos are on the list, for each of your repos either:
> >
> > * Archive or Delete it if it's no longer maintained or empty/useless,
> > respectively (and remove them from the list on mw.org)[2]
> >
> > Or:
> >
> > * put a "{{tick}}" in the "Active" column on the list on mw.org
> >
> > == Why==
> >
> > In a more perfect future we could add the "mirror"[3] tag to repos on
> > GitHub that are mirrored from Gerrit (with a link to their canonical
> > repo locations; for example, gnome-deskop has this[4] and I'm very
> > jealous).
> >
> > Hopefully, this will help folks wanting to contribute -- either a
> > Wikimedia GitHub repo is a mirror (in which case there's a link to
> > Gerrit in the description) or it's actively being developed on GitHub.
> >
> > <3
> > -- Tyler
> >
> > [0]: <https://phabricator.wikimedia.org/T237470#6407509>
> > [1]: <https://www.mediawiki.org/wiki/Gerrit/GitHub#Projects_on_GitHub>
> > [2]: <
> https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/archiving-a-github-repository
> >
> > [3]: <
> https://docs.github.com/en/github/getting-started-with-github/finding-ways-to-contribute-to-open-source-on-github#open-source-projects-with-mirrors-on-github
> >
> > [4]: <https://github.com/GNOME/gnome-desktop>
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikimedia's GitHub org help

Isaac Johnson
Thanks Tyler for doing this work! Is there an easy way (if not, no big
deal) to also see the list of repos that were archived/deleted just to make
sure lack of response didn't mean something disappeared that would have
been useful to keep active?

Best,
Isaac

On Tue, Sep 1, 2020, 14:05 Greg Rundlett (freephile) <[hidden email]>
wrote:

> Thanks for doing this Tyler!
>
> Greg Rundlett
> https://eQuality-Tech.com
> https://freephile.org
>
>
> On Tue, Sep 1, 2020 at 11:56 AM Tyler Cipriani <[hidden email]>
> wrote:
>
> > Hi all!
> >
> > Thank you for reviewing my list of the Wikimedia org's GitHub projects!
> >
> >  A *lot* of repos were archived or deleted. Additionally, I archived
> > all projects that contained nothing but a ".gitreview" without a
> > corresponding Gerrit repository.
> >
> > As a result, we've gone from 296 repositories exclusively on GitHub to
> > 154 repositories exclusively on GitHub.
> >
> > I believe this removed a lot of cruft and now all 2,038 repos on
> > Wikimedia's GitHub org are either:
> > * A mirror of a Gerrit repo (1,881 repos)
> > * A mirror of a Differential repo (3 repos)
> > * A project developed on GitHub or a fork of another GitHub project (154
> > repos)
> >
> > I've updated the list of projects[0] that are exclusive to GitHub
> > (GitHub fork or a project developed on GItHub) if you'd like to take a
> > look.
> >
> > Thanks again for all your help!
> >
> > <3
> > -- Tyler
> >
> > [0]. <https://www.mediawiki.org/wiki/Gerrit/GitHub#Projects_on_GitHub>
> >
> > On Mon, Aug 24, 2020 at 4:46 PM Tyler Cipriani <[hidden email]>
> > wrote:
> > >
> > > Hi all!
> > >
> > > If you've never created a repo or fork on the Wikimedia GitHub
> > > organization you can skip this email.
> > >
> > > I know that some repos are developed on our GitHub org for reasons.
> > > What is developed on our GitHub org? How many things are actively
> > > being developed on GitHub org? I have no idea :)
> > >
> > > I recently realized that there's not a great way to figure this
> > > out[0], but I've been able to narrow the scope a bit. Now I have a
> > > list of repos that are (a) in our GitHub org and (b) not in our Gerrit
> > > that I could use some help sorting through[1].
> > >
> > > == Help, please ==
> > >
> > > * Look through repos on The List™[1]
> > >
> > > If your repos are on the list, for each of your repos either:
> > >
> > > * Archive or Delete it if it's no longer maintained or empty/useless,
> > > respectively (and remove them from the list on mw.org)[2]
> > >
> > > Or:
> > >
> > > * put a "{{tick}}" in the "Active" column on the list on mw.org
> > >
> > > == Why==
> > >
> > > In a more perfect future we could add the "mirror"[3] tag to repos on
> > > GitHub that are mirrored from Gerrit (with a link to their canonical
> > > repo locations; for example, gnome-deskop has this[4] and I'm very
> > > jealous).
> > >
> > > Hopefully, this will help folks wanting to contribute -- either a
> > > Wikimedia GitHub repo is a mirror (in which case there's a link to
> > > Gerrit in the description) or it's actively being developed on GitHub.
> > >
> > > <3
> > > -- Tyler
> > >
> > > [0]: <https://phabricator.wikimedia.org/T237470#6407509>
> > > [1]: <https://www.mediawiki.org/wiki/Gerrit/GitHub#Projects_on_GitHub>
> > > [2]: <
> >
> https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/archiving-a-github-repository
> > >
> > > [3]: <
> >
> https://docs.github.com/en/github/getting-started-with-github/finding-ways-to-contribute-to-open-source-on-github#open-source-projects-with-mirrors-on-github
> > >
> > > [4]: <https://github.com/GNOME/gnome-desktop>
> >
> > _______________________________________________
> > Wikitech-l mailing list
> > [hidden email]
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikimedia's GitHub org help

Tyler Cipriani
On Tue, Sep 1, 2020 at 12:11 PM Isaac Johnson <[hidden email]> wrote:
> Thanks Tyler for doing this work! Is there an easy way (if not, no big
> deal) to also see the list of repos that were archived/deleted just to make
> sure lack of response didn't mean something disappeared that would have
> been useful to keep active?

Sure, the list of repos removed or archived (plus a repo that was
added and a repo that was renamed during this process :)) can be found
in this diff:
https://github.com/thcipriani/wikimedia-github-projects/compare/13aa797..7cdb83b

I don't have a way to differentiate between repos that were archived
vs deleted in that list.

Thanks!
-- Tyler

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Wikimedia's GitHub org help

Isaac Johnson
Ahh perfect, of course. Thanks (no projects were ones I would have saved so
my concerns were alleviated)!

On Tue, Sep 1, 2020 at 3:47 PM Tyler Cipriani <[hidden email]>
wrote:

> On Tue, Sep 1, 2020 at 12:11 PM Isaac Johnson <[hidden email]> wrote:
> > Thanks Tyler for doing this work! Is there an easy way (if not, no big
> > deal) to also see the list of repos that were archived/deleted just to
> make
> > sure lack of response didn't mean something disappeared that would have
> > been useful to keep active?
>
> Sure, the list of repos removed or archived (plus a repo that was
> added and a repo that was renamed during this process :)) can be found
> in this diff:
>
> https://github.com/thcipriani/wikimedia-github-projects/compare/13aa797..7cdb83b
>
> I don't have a way to differentiate between repos that were archived
> vs deleted in that list.
>
> Thanks!
> -- Tyler
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>


--
Isaac Johnson (he/him/his) -- Research Scientist -- Wikimedia Foundation
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l