[testing] TitleBlacklist now tested under Jenkins

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

[testing] TitleBlacklist now tested under Jenkins

Antoine Musso-3
Hello,

After much hassle with Jenkins, ant and PHPUnit, I finally managed to
run an extension PHPUnit test suite under Jenkins.

The winner is TitleBlacklist for which results are available via:
  https://integration.mediawiki.org/ci/job/Ext-TitleBlacklist/

The Jenkins job is setup to report the build status back in Gerrit.


If you see any issue, please fill in a bug under Testing infrastructure.

TODO: Job does not run PHP lint yet.

--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: [testing] TitleBlacklist now tested under Jenkins

Tyler Romeo
Are there any plans to move more extensions (or at least those with unit
tests) under Jenkins? Because I know one of my extension
(TwoFactorAuthentication) has unit tests.

*--*
*Tyler Romeo*
Stevens Institute of Technology, Class of 2015
Major in Computer Science
www.whizkidztech.com | [hidden email]



On Wed, Aug 29, 2012 at 10:18 AM, Antoine Musso <[hidden email]> wrote:

> Hello,
>
> After much hassle with Jenkins, ant and PHPUnit, I finally managed to
> run an extension PHPUnit test suite under Jenkins.
>
> The winner is TitleBlacklist for which results are available via:
>   https://integration.mediawiki.org/ci/job/Ext-TitleBlacklist/
>
> The Jenkins job is setup to report the build status back in Gerrit.
>
>
> If you see any issue, please fill in a bug under Testing infrastructure.
>
> TODO: Job does not run PHP lint yet.
>
> --
> Antoine "hashar" Musso
>
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: [testing] TitleBlacklist now tested under Jenkins

Chad
In reply to this post by Antoine Musso-3
On Wed, Aug 29, 2012 at 10:18 AM, Antoine Musso <[hidden email]> wrote:

> Hello,
>
> After much hassle with Jenkins, ant and PHPUnit, I finally managed to
> run an extension PHPUnit test suite under Jenkins.
>
> The winner is TitleBlacklist for which results are available via:
>   https://integration.mediawiki.org/ci/job/Ext-TitleBlacklist/
>
> The Jenkins job is setup to report the build status back in Gerrit.
>
>
> If you see any issue, please fill in a bug under Testing infrastructure.
>

Question: why does the config for non-extension tests attempt
to load extensions? -Parser and -Misc both seem to be failing
due to a broken inclusion of Wikibase.

Core tests should be run without any extensions.

-Chad

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: [testing] TitleBlacklist now tested under Jenkins

Antoine Musso-3
In reply to this post by Tyler Romeo
Le 29/08/12 16:24, Tyler Romeo a écrit :
> Are there any plans to move more extensions (or at least those with unit
> tests) under Jenkins? Because I know one of my extension
> (TwoFactorAuthentication) has unit tests.

Ultimately, all extensions hosted on WMF git server will be integrated
in Jenkins. If you get PHPUnit test, I will add a job for it as soon as
the current jobs are stable enough.

--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: [testing] TitleBlacklist now tested under Jenkins

Tyler Romeo
>
> Ultimately, all extensions hosted on WMF git server will be integrated
> in Jenkins. If you get PHPUnit test, I will add a job for it as soon as
> the current jobs are stable enough.


Awesome!

*--*
*Tyler Romeo*
Stevens Institute of Technology, Class of 2015
Major in Computer Science
www.whizkidztech.com | [hidden email]



On Wed, Aug 29, 2012 at 10:47 AM, Antoine Musso <[hidden email]> wrote:

> Le 29/08/12 16:24, Tyler Romeo a écrit :
> > Are there any plans to move more extensions (or at least those with unit
> > tests) under Jenkins? Because I know one of my extension
> > (TwoFactorAuthentication) has unit tests.
>
> Ultimately, all extensions hosted on WMF git server will be integrated
> in Jenkins. If you get PHPUnit test, I will add a job for it as soon as
> the current jobs are stable enough.
>
> --
> Antoine "hashar" Musso
>
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: [testing] TitleBlacklist now tested under Jenkins

Max Semenik
In reply to this post by Chad
On 29.08.2012, 18:27 Chad wrote:

> Question: why does the config for non-extension tests attempt
> to load extensions? -Parser and -Misc both seem to be failing
> due to a broken inclusion of Wikibase.

> Core tests should be run without any extensions.

Ideally, core tests should include as many extension tests as possible
to prevent core from silently breaking extensions.

--
Best regards,
  Max Semenik ([[User:MaxSem]])


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: [testing] TitleBlacklist now tested under Jenkins

Chad
On Wed, Aug 29, 2012 at 10:53 AM, Max Semenik <[hidden email]> wrote:

> On 29.08.2012, 18:27 Chad wrote:
>
>> Question: why does the config for non-extension tests attempt
>> to load extensions? -Parser and -Misc both seem to be failing
>> due to a broken inclusion of Wikibase.
>
>> Core tests should be run without any extensions.
>
> Ideally, core tests should include as many extension tests as possible
> to prevent core from silently breaking extensions.
>

That's why we write tests for extensions. You could very easily
write two extensions that produce conflicting output--core should
not break because of this.

-Chad

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: [testing] TitleBlacklist now tested under Jenkins

Jeroen De Dauw-2
In reply to this post by Antoine Musso-3
Hey,

Ultimately, all extensions hosted on WMF git server will be integrated
> in Jenkins. If you get PHPUnit test, I will add a job for it as soon as
> the current jobs are stable enough.
>

Thanks for your work on this! :)

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil.
--
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: [testing] TitleBlacklist now tested under Jenkins

Chad
On Wed, Aug 29, 2012 at 11:01 AM, Jeroen De Dauw <[hidden email]> wrote:
> Hey,
>
> Ultimately, all extensions hosted on WMF git server will be integrated
>> in Jenkins. If you get PHPUnit test, I will add a job for it as soon as
>> the current jobs are stable enough.
>>
>
> Thanks for your work on this! :)
>

Indeed! I should've said so in my first e-mail.

-Chad

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: [testing] TitleBlacklist now tested under Jenkins

Derric Atzrott
In reply to this post by Chad
>>> Core tests should be run without any extensions.
>>
>> Ideally, core tests should include as many extension tests as possible
>> to prevent core from silently breaking extensions.
>>
>
>That's why we write tests for extensions. You could very easily
>write two extensions that produce conflicting output--core should
>not break because of this.

I think we have three different things going on here.

* Unit tests for extensions to prevent core changes from breaking extensions
* Unit tests for core (with extensions enabled) to prevent core changes from
  breaking extensions.
* Unit tests for core (with extensions enabled) to prevent conflicting
  extensions from breaking the core.


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: [testing] TitleBlacklist now tested under Jenkins

Mark Holmquist-2
In reply to this post by Antoine Musso-3
> Ultimately, all extensions hosted on WMF git server will be integrated
> in Jenkins. If you get PHPUnit test, I will add a job for it as soon as
> the current jobs are stable enough.

At the risk of sounding unoriginal, _zomg this is amazing_.

Thanks a bunch, Antoine!

--
Mark Holmquist
Contractor, Wikimedia Foundation
[hidden email]
http://marktraceur.info

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: [testing] TitleBlacklist now tested under Jenkins

Antoine Musso-3
In reply to this post by Chad
Le 29/08/12 16:27, Chad a écrit :
> Question: why does the config for non-extension tests attempt
> to load extensions? -Parser and -Misc both seem to be failing
> due to a broken inclusion of Wikibase.

The -Parser and -Misc jobs are triggered by both the MediaWiki core job
and the one testing the Wikidata branch.  I originally thought it was a
good idea to a job dedicated to a PHPUnit group, I will end up creating
a job dedicated to testing the Wikidata branch.

> Core tests should be run without any extensions.

Fully agree. We can later create a job to test core + the extension
deployed on the wmf and another one for a Semantic MediaWiki setup.

--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: [testing] TitleBlacklist now tested under Jenkins

Oren Bochman
I've tried to do this for translate ext last week - so here are a couple of
questions:
1. is successfully runnig the test a requirement to successfully score on
gerrit? (i.e. how is gerrit integrated)
2. does the extension need to include php unit?

On Thu, Aug 30, 2012 at 9:43 AM, Antoine Musso <[hidden email]> wrote:

> Le 29/08/12 16:27, Chad a écrit :
> > Question: why does the config for non-extension tests attempt
> > to load extensions? -Parser and -Misc both seem to be failing
> > due to a broken inclusion of Wikibase.
>
> The -Parser and -Misc jobs are triggered by both the MediaWiki core job
> and the one testing the Wikidata branch.  I originally thought it was a
> good idea to a job dedicated to a PHPUnit group, I will end up creating
> a job dedicated to testing the Wikidata branch.
>
> > Core tests should be run without any extensions.
>
> Fully agree. We can later create a job to test core + the extension
> deployed on the wmf and another one for a Semantic MediaWiki setup.
>
> --
> Antoine "hashar" Musso
>
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



--

Oren Bochman

Office tel. 061 4921492
Mobile +36 30 866 6706
skype id: orenbochman
e-mail: [hidden email]
site http://www.riverport.hu
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: [testing] TitleBlacklist now tested under Jenkins

Antoine Musso-3
Le 30/08/12 10:11, Oren Bochman a écrit :
> 1. is successfully runnig the test a requirement to successfully score on
> gerrit? (i.e. how is gerrit integrated)

When a change is submitted in Gerrit a notification is sent to Jenkins.
If a job is setup for the repository, it will be triggered. For
extension tests the process is:
 - copy a snapshot of the current master version of MediaWiki
 - clone latest master version of the extension
 - apply the change
 - load the extension entry point (Foobar/Foobar.php by convention)
 - Run PHPUnit against the 'extensions' directory
 - Test results are aggregate and publish under Jenkins
 - If all tests are successful, Jenkins mark the change in Gerrit as
Verified +2. Else it marks it as Verified -2

> 2. does the extension need to include php unit?

The PHPUnit software is installed on the continuous integration server.
The only thing you need in the extension is to have your tests in a PHP
file ending with 'Test.php'.

The TitleBlacklist is an example.

cheers,

--
Antoine "hashar" Musso


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: [testing] TitleBlacklist now tested under Jenkins

Roan Kattouw-2
In reply to this post by Oren Bochman
On Thu, Aug 30, 2012 at 1:11 AM, Oren Bochman <[hidden email]> wrote:
> I've tried to do this for translate ext last week - so here are a couple of
> questions:
> 1. is successfully runnig the test a requirement to successfully score on
> gerrit? (i.e. how is gerrit integrated)
Jenkins sets a Verified +1 or -1 on the change, depending on whether
the tests succeed or not, just like for core. A V+1 is required to
merge a change. So to merge any change into an extension, the tests
(that is, the core tests and any extension tests) have to pass. But
the extension isn't required to have tests, it's just that you can't
submit changes that break the core tests or break any tests that are
present in the extension. Existing extensions without any tests or
awareness of phpunit will continue to work just fine, as long as they
don't try to do crazy stuff that breaks a test in core.

> 2. does the extension need to include php unit?
>
No, all of the phpunit wrapper stuff is in MediaWiki core. Adding
tests to extensions is simple, see
https://www.mediawiki.org/wiki/Manual:PHP_unit_testing/Writing_unit_tests_for_extensions
.

Roan

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: [testing] TitleBlacklist now tested under Jenkins

Jon Robson
Excellent!
Can't wait to get MobileFrontend using this.

Great work Antoine.

On Thu, Aug 30, 2012 at 12:59 PM, Roan Kattouw <[hidden email]> wrote:

> On Thu, Aug 30, 2012 at 1:11 AM, Oren Bochman <[hidden email]> wrote:
>> I've tried to do this for translate ext last week - so here are a couple of
>> questions:
>> 1. is successfully runnig the test a requirement to successfully score on
>> gerrit? (i.e. how is gerrit integrated)
> Jenkins sets a Verified +1 or -1 on the change, depending on whether
> the tests succeed or not, just like for core. A V+1 is required to
> merge a change. So to merge any change into an extension, the tests
> (that is, the core tests and any extension tests) have to pass. But
> the extension isn't required to have tests, it's just that you can't
> submit changes that break the core tests or break any tests that are
> present in the extension. Existing extensions without any tests or
> awareness of phpunit will continue to work just fine, as long as they
> don't try to do crazy stuff that breaks a test in core.
>
>> 2. does the extension need to include php unit?
>>
> No, all of the phpunit wrapper stuff is in MediaWiki core. Adding
> tests to extensions is simple, see
> https://www.mediawiki.org/wiki/Manual:PHP_unit_testing/Writing_unit_tests_for_extensions
> .
>
> Roan
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l



--
Jon Robson
http://jonrobson.me.uk
@rakugojon

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l