Client side testing

Vincent Privat vincent.privat at gmail.com
Tue Nov 7 21:11:51 UTC 2017


OK I tried to setup the JDK coverage, and this is NOT easy.
Right now I'm blocked by a bug in the Jenkins Jacoco plugin:
https://issues.jenkins-ci.org/browse/JENKINS-36168
Need to solve it first.
If I find how to setup JDK coverage for an Ant project, it will deserve
serious documentation. I will probably update JOSM wiki, but maybe it could
land somewhere on
https://wiki.openjdk.java.net/display/quality/Quality+Outreach for other
projects?

2017-11-06 23:51 GMT+01:00 Alexandre (Shura) Iline <
alexandre.iline at oracle.com>:

>
> On Nov 6, 2017, at 2:48 PM, Vincent Privat <vincent.privat at gmail.com>
> wrote:
>
> Sure! Currently coverage is restricted to JOSM classes, for obvious
> performance reasons, as we run a full build after each commit.
> But I can add a parameter to change the monitored packages in our EA
> builds.
> Is it OK if I add java, javax, sun and com.sun ? Or do you prefer to
> restrict only to packages from java.desktop module?
>
>
> Thank you!
>
> Frankly, java.desktop is what I am more interested myself right now.
>
> java, javax, sun and com.sun would also cover what I am asking for, of
> course.
>
> Shura
>
>
>
>
> 2017-11-06 19:31 GMT+01:00 Alexandre (Shura) Iline <
> alexandre.iline at oracle.com>:
>
>> This is great that you are touching the new builds, thank you, Vincent.
>>
>> I am not familiar with JOSM code, but the app is great! I have played
>> with it in the past.
>>
>> Do you think it would be possible, in addition to what you are doing, to
>> collect code coverage for the  JDK code? Specifically for the now existing
>> java.desktop module. I do not know much of JaCoCo - we are using JCov, as
>> you can imagine - so I do not know how easy it is. If it is easy, can you
>> flip the switch?
>>
>> Shura
>> > On Nov 3, 2017, at 8:33 AM, Vincent Privat <vincent.privat at gmail.com>
>> wrote:
>> >
>> > Hello Rory,
>> > Thank you :)
>> >
>> > Here is how we test various OpenJDK builds:
>> > 1.1: we use only EA builds from http://jdk.java.net/
>> > 2.1.1: no
>> > 2.1.2/2.1.3: we run all unit and integration tests of JOSM
>> > 2.1.4: yes, using jacoco
>> > 2.2: all tests are automated
>> > 2.3: it depends. During OpenJDK9 cycle we did not run a test series for
>> each weekly build, but rather once every two or three builds, unless we
>> were waiting for a specific correction from OpenJDK or another tool. At the
>> end of development, we were not waiting for any correction from OpenJDK but
>> were eagerly waiting for the updates of ASM and everything that as blocked
>> by the non-availability of ASM (in particular various Maven plugins)
>> > 2.4: Yes, they are visible here:
>> > - JOSM core: https://josm.openstreetmap.de/
>> jenkins/job/Java-EarlyAccess-JOSM/
>> > - JOSM community plugins: https://josm.openstreetmap.de/
>> jenkins/job/Java-EarlyAccess-JOSM-Plugins/
>> >
>> > I just have initiated JDK10 tests today, and we have 53 failures caused
>> by Byte Buddy not recognizing Java version 10:
>> >
>> > java.lang.IllegalArgumentException: Unknown Java version: 10
>> >       at nl.jqno.equalsverifier.internal.lib.bytebuddy.ClassFileVersi
>> on.ofJavaVersion(ClassFileVersion.java:135)
>> >
>> >
>> > I'm sure Rafael is aware of it. I will see how to update our tools
>> (equalsverifier here).
>> >
>> > Cheers,
>> > Vincent
>> >
>> > 2017-11-03 14:05 GMT+01:00 Rory O'Donnell <rory.odonnell at oracle.com>:
>> >
>> > Hi Vincent,
>> >
>> > I noticed that JOSM have been commended in the last two Quality reports
>> [1] [2] for the number and
>> > indeed the quality of bugs you have found and logged against various
>> early access releases.
>> > Thanks again for your contribution to the quality of our releases!
>> >
>> > The big question is how do you do it ?
>> >
>> > We would love to hear more details about your test environment, what
>> tests you run etc. Below is a
>> > series of questions that come to mind, but there may be other details
>> that you would like to share.
>> >       • Builds
>> >               • do you build your own OpenJDK builds, use Oracle's or
>> other EA builds ?
>> >       • Tests
>> >               • What tests do you run
>> >                       • do you run the JCK ?
>> >                       • open regression tests ?
>> >                       • other tests ?
>> >                       • do you measure code coverage ?
>> >               • Are the tests Automated or Manual ?
>> >               • How frequently to do execute the tests
>> >               • Are the results visible via an external link
>> > Rgds, Rory
>> >
>> > [1] https://wiki.openjdk.java.net/display/quality/Quality+Outrea
>> ch+report+September+2017
>> > [2] https://wiki.openjdk.java.net/display/quality/Quality+Outrea
>> ch+report+March+2017
>> > --
>> > Rgds,Rory O'Donnell
>> > Quality Engineering Manager
>> > Oracle EMEA , Dublin, Ireland
>> >
>> >
>>
>>
>
>


More information about the josm-dev mailing list