Client side testing

Tue Nov 7 21:25:30 UTC 2017

If someone's paid at Oracle to work on Jenkins plugins, this bug is a good
candidate :)
(given the Hudson/Jenkins history I don't think so, but well, sometimes we
have good surprises, as I can see Jenkins is used to power
https://ci.adoptopenjdk.net/)

2017-11-07 22:18 GMT+01:00 Alexandre (Shura) Iline <
alexandre.iline at oracle.com>:

> The issue is pretty generic and also pretty old. :(
>
> Thank you anyway.
>
> Let me know if there is anything we could help with on our side.
>
> Shuta
>
>
>
> On Nov 7, 2017, at 1:11 PM, Vincent Privat <vincent.privat at gmail.com>
> wrote:
>
> OK I tried to setup the JDK coverage, and this is NOT easy.
> Right now I'm blocked by a bug in the Jenkins Jacoco plugin:
> https://issues.jenkins-ci.org/browse/JENKINS-36168
> Need to solve it first.
> If I find how to setup JDK coverage for an Ant project, it will deserve
> serious documentation. I will probably update JOSM wiki, but maybe it could
> land somewhere on https://wiki.openjdk.java.net/display/quality/Quality+
> Outreach for other projects?
>
> 2017-11-06 23:51 GMT+01:00 Alexandre (Shura) Iline <
> alexandre.iline at oracle.com>:
>
>>
>> On Nov 6, 2017, at 2:48 PM, Vincent Privat <vincent.privat at gmail.com>
>> wrote:
>>
>> Sure! Currently coverage is restricted to JOSM classes, for obvious
>> performance reasons, as we run a full build after each commit.
>> But I can add a parameter to change the monitored packages in our EA
>> builds.
>> Is it OK if I add java, javax, sun and com.sun ? Or do you prefer to
>> restrict only to packages from java.desktop module?
>>
>>
>> Thank you!
>>
>> Frankly, java.desktop is what I am more interested myself right now.
>>
>> java, javax, sun and com.sun would also cover what I am asking for, of
>> course.
>>
>> Shura
>>
>>
>>
>>
>> 2017-11-06 19:31 GMT+01:00 Alexandre (Shura) Iline <
>> alexandre.iline at oracle.com>:
>>
>>> This is great that you are touching the new builds, thank you, Vincent.
>>>
>>> I am not familiar with JOSM code, but the app is great! I have played
>>> with it in the past.
>>>
>>> Do you think it would be possible, in addition to what you are doing, to
>>> collect code coverage for the  JDK code? Specifically for the now existing
>>> java.desktop module. I do not know much of JaCoCo - we are using JCov, as
>>> you can imagine - so I do not know how easy it is. If it is easy, can you
>>> flip the switch?
>>>
>>> Shura
>>> > On Nov 3, 2017, at 8:33 AM, Vincent Privat <vincent.privat at gmail.com>
>>> wrote:
>>> >
>>> > Hello Rory,
>>> > Thank you :)
>>> >
>>> > Here is how we test various OpenJDK builds:
>>> > 1.1: we use only EA builds from http://jdk.java.net/
>>> > 2.1.1: no
>>> > 2.1.2/2.1.3: we run all unit and integration tests of JOSM
>>> > 2.1.4: yes, using jacoco
>>> > 2.2: all tests are automated
>>> > 2.3: it depends. During OpenJDK9 cycle we did not run a test series
>>> for each weekly build, but rather once every two or three builds, unless we
>>> were waiting for a specific correction from OpenJDK or another tool. At the
>>> end of development, we were not waiting for any correction from OpenJDK but
>>> were eagerly waiting for the updates of ASM and everything that as blocked
>>> by the non-availability of ASM (in particular various Maven plugins)
>>> > 2.4: Yes, they are visible here:
>>> > - JOSM core: https://josm.openstreetmap.de/
>>> jenkins/job/Java-EarlyAccess-JOSM/
>>> > - JOSM community plugins: https://josm.openstreetmap.de/
>>> jenkins/job/Java-EarlyAccess-JOSM-Plugins/
>>> >
>>> > I just have initiated JDK10 tests today, and we have 53 failures
>>> caused by Byte Buddy not recognizing Java version 10:
>>> >
>>> > java.lang.IllegalArgumentException: Unknown Java version: 10
>>> >       at nl.jqno.equalsverifier.internal.lib.bytebuddy.ClassFileVersi
>>> on.ofJavaVersion(ClassFileVersion.java:135)
>>> >
>>> >
>>> > I'm sure Rafael is aware of it. I will see how to update our tools
>>> (equalsverifier here).
>>> >
>>> > Cheers,
>>> > Vincent
>>> >
>>> > 2017-11-03 14:05 GMT+01:00 Rory O'Donnell <rory.odonnell at oracle.com>:
>>> >
>>> > Hi Vincent,
>>> >
>>> > I noticed that JOSM have been commended in the last two Quality
>>> reports [1] [2] for the number and
>>> > indeed the quality of bugs you have found and logged against various
>>> early access releases.
>>> > Thanks again for your contribution to the quality of our releases!
>>> >
>>> > The big question is how do you do it ?
>>> >
>>> > We would love to hear more details about your test environment, what
>>> tests you run etc. Below is a
>>> > series of questions that come to mind, but there may be other details
>>> that you would like to share.
>>> >       • Builds
>>> >               • do you build your own OpenJDK builds, use Oracle's or
>>> other EA builds ?
>>> >       • Tests
>>> >               • What tests do you run
>>> >                       • do you run the JCK ?
>>> >                       • open regression tests ?
>>> >                       • other tests ?
>>> >                       • do you measure code coverage ?
>>> >               • Are the tests Automated or Manual ?
>>> >               • How frequently to do execute the tests
>>> >               • Are the results visible via an external link
>>> > Rgds, Rory
>>> >
>>> > [1] https://wiki.openjdk.java.net/display/quality/Quality+Outrea
>>> ch+report+September+2017
>>> > [2] https://wiki.openjdk.java.net/display/quality/Quality+Outrea
>>> ch+report+March+2017
>>> > --
>>> > Rgds,Rory O'Donnell
>>> > Quality Engineering Manager
>>> > Oracle EMEA , Dublin, Ireland
>>> >
>>> >
>>>
>>>
>>
>>
>
>