npm/composer entry points in release branches

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

npm/composer entry points in release branches

Antoine Musso-3
Hello,

Jenkins jobs start failing on some repositories release branches because
they are lacking the CI [entry points].  For example if the 'npm' job
has been enabled, the master branch has a 'package.json' but the release
branches do not.  That causes CI to reject backports.

I would like us to update our repositories REL1_24 and REL1_25 branches
with package.json and composer.json whenever they are present in master.


The commands being run would need be tweaked but at the very minimal
each branches should check:

* the i18n files using grunt-banana-checker (in package.json)
* php lint (jakub-onderka/php-parallel-lint) (in composer.json)


Repositories that haven't switched yet are not impacted nor are the
branches already having the package.json/composer.json files.




[entry points]
https://www.mediawiki.org/wiki/Continuous_integration/Entry_points

--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: npm/composer entry points in release branches

Legoktm
Hi,

On 09/07/2015 01:20 PM, Antoine Musso wrote:
> Jenkins jobs start failing on some repositories release branches because
> they are lacking the CI [entry points].  For example if the 'npm' job
> has been enabled, the master branch has a 'package.json' but the release
> branches do not.  That causes CI to reject backports.

Another idea I've been thinking about is making the "npm" and
"composer-test" jobs pass if no package.json/composer.json is present.
In addition to fixing the release branch issue, it also would allow us
to make CI/zuul more self-service. We could enable those jobs for all
repos, and people could add npm/composer tests and immediately get
feedback and start using them, without having to wait for someone to
update the zuul config.

> The commands being run would need be tweaked but at the very minimal
> each branches should check:
>
> * the i18n files using grunt-banana-checker (in package.json)
> * php lint (jakub-onderka/php-parallel-lint) (in composer.json)

phplint should be doable, but not all branches may pass banana-checker.
Many repositories still don't pass, and many more have only recently
begun to pass (see blockers on [1]).

[1] https://phabricator.wikimedia.org/T94547

-- Legoktm

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: npm/composer entry points in release branches

Ricordisamoa
Il 08/09/2015 10:26, Legoktm ha scritto:

> Hi,
>
> On 09/07/2015 01:20 PM, Antoine Musso wrote:
>> Jenkins jobs start failing on some repositories release branches because
>> they are lacking the CI [entry points].  For example if the 'npm' job
>> has been enabled, the master branch has a 'package.json' but the release
>> branches do not.  That causes CI to reject backports.
> Another idea I've been thinking about is making the "npm" and
> "composer-test" jobs pass if no package.json/composer.json is present.
> In addition to fixing the release branch issue, it also would allow us
> to make CI/zuul more self-service. We could enable those jobs for all
> repos, and people could add npm/composer tests and immediately get
> feedback and start using them, without having to wait for someone to
> update the zuul config.

Yes! Also tox.ini please.

>
>> The commands being run would need be tweaked but at the very minimal
>> each branches should check:
>>
>> * the i18n files using grunt-banana-checker (in package.json)
>> * php lint (jakub-onderka/php-parallel-lint) (in composer.json)
> phplint should be doable, but not all branches may pass banana-checker.
> Many repositories still don't pass, and many more have only recently
> begun to pass (see blockers on [1]).
>
> [1] https://phabricator.wikimedia.org/T94547
>
> -- Legoktm
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: npm/composer entry points in release branches

Antoine Musso-3
In reply to this post by Legoktm
Le 08/09/2015 10:26, Legoktm a écrit :
> Another idea I've been thinking about is making the "npm" and
> "composer-test" jobs pass if no package.json/composer.json is present.
> In addition to fixing the release branch issue, it also would allow us
> to make CI/zuul more self-service. We could enable those jobs for all
> repos, and people could add npm/composer tests and immediately get
> feedback and start using them, without having to wait for someone to
> update the zuul config.

Hello,

Yeah that is a good idea and we talked about it during last CI checkin.
We can just early exit 0 whenever the file is missing, a concern is that
with the CI isolation scaling that will consumes an instance for
basically nothing :-/

--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: npm/composer entry points in release branches

Antoine Musso-3
In reply to this post by Legoktm
Le 08/09/2015 10:26, Legoktm a écrit :

>> > The commands being run would need be tweaked but at the very minimal
>> > each branches should check:
>> >
>> > * the i18n files using grunt-banana-checker (in package.json)
>> > * php lint (jakub-onderka/php-parallel-lint) (in composer.json)
> phplint should be doable, but not all branches may pass banana-checker.
> Many repositories still don't pass, and many more have only recently
> begun to pass (see blockers on [1]).
>
> [1] https://phabricator.wikimedia.org/T94547

How I thought all i18n files passed already. So seems we have to hold on
this a bit, or do a mass run and only enable the banan checker for
repo/branche that pass.

I thought about adding a basic JavaScript syntax checker for repo/branch
that do not have jshint yet.  But haven't found an easy to use npm package.


--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: npm/composer entry points in release branches

Vivek Ghaisas
I Googled for npm packages that check JS syntax and came across a package
called syntax-checker [1] which checks several languages. I looked at the
source and found the lines where it checks the syntax of each language [2].
For languages other than JS, it just uses the interpreter/compiler with a
flag passed to it. However, for JS, it passes the file to UglifyJS [3] and
redirects its output to /dev/null. Presumably, UglifyJS complains when it
sees invalid JS.

Probably not the best way to do it, but certainly a solution.

[1] https://github.com/xianhuazhou/syntax-checker/
[2]
https://github.com/xianhuazhou/syntax-checker/blob/c68c562d8/lib/syntax-checker.js#L18-L28
[3] https://github.com/mishoo/UglifyJS2

--
Vivek Ghaisas
polybuildr
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: npm/composer entry points in release branches

Vivek Ghaisas
Another solution is to use a JS parser, such as Acorn [1] to parse the JS.
It can be used as a command line tool (passing the --silent) option to it,
which makes it only return the exit code which we can use to check whether
the syntax was valid or not.

The bigger question, probably, is that neither this not my previous
solution prints out the errors in the syntax, so they're not very useful to
locate errors.

A relevant discussion happened on an issue filed on the JSHint GitHub repo
[2] which asked for a syntax check only mode. It was declared a wont-fix
[3]. However, we still might be able to disable most of the rules and try
for a minimal configuration, which might just get us what we want.

[1] https://github.com/marijnh/acorn#command-line-interface
[2] https://github.com/jshint/jshint/issues/808
[3] https://github.com/jshint/jshint/issues/808#issuecomment-20763023

Vivek Ghaisas
(polybuildr)

On Tue, Sep 8, 2015 at 2:47 PM, Vivek Ghaisas <[hidden email]> wrote:

> I Googled for npm packages that check JS syntax and came across a package
> called syntax-checker [1] which checks several languages. I looked at the
> source and found the lines where it checks the syntax of each language [2].
> For languages other than JS, it just uses the interpreter/compiler with a
> flag passed to it. However, for JS, it passes the file to UglifyJS [3] and
> redirects its output to /dev/null. Presumably, UglifyJS complains when it
> sees invalid JS.
>
> Probably not the best way to do it, but certainly a solution.
>
> [1] https://github.com/xianhuazhou/syntax-checker/
> [2]
> https://github.com/xianhuazhou/syntax-checker/blob/c68c562d8/lib/syntax-checker.js#L18-L28
> [3] https://github.com/mishoo/UglifyJS2
>
> --
> Vivek Ghaisas
> polybuildr
>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l