Thursday, October 05, 2023

Using JAXB in custom Ant tasks on recent Java versions

Apache Ant 1.10.14 was released a few weeks ago https://lists.apache.org/thread/9vhk51nkw9wjxzm7xk2q9xm6s803prmr. Like noted in that announcement, apart from the regular bug fixes, this release also has an important change which allows it to be used in the recently released Java 21 https://inside.java/2023/09/19/the-arrival-of-java-21/. One major goal of Ant project is to make sure that it can be used to build projects using latest versions of Java. As such, the Ant project team keeps a watch on any changes in the Java releases that could affect Ant and does necessary changes in Ant and releases them in its 1.10.x release series.

The announcement of 1.10.14 and this changelog https://github.com/apache/ant/blob/rel/1.10.14/WHATSNEW contains the details of what’s exactly changed in this release, so I won’t go into those details again. In this post I’ll however go through one interesting issue that was brought to my notice by more than one projects, when using recent versions of Java. The issue specifically relates to custom Ant tasks that use JAXB. However, in general, it applies to some of the APIs that have been removed from recent versions of Java (details in https://openjdk.org/jeps/320)

JAXB as noted in the reference doc https://docs.oracle.com/javase/tutorial/jaxb/intro/arch.html is an API and implementation for XML binding. Up until Java 11, JAXB API and implementation classes were shipped as part of the Java runtime. What it meant was that any application code, like custom developed Ant tasks, could just use the JAXB APIs in their code without having to explicitly specific any external library dependency. In Java 11, JAXB along with few other modules was removed from the JDK. The release notes of JDK 11 lists this change https://www.oracle.com/java/technologies/javase/11-relnote-issues.html#JDK-8190378. Additionally, JEP-320 https://openjdk.org/jeps/320 has all the details related to this removal. When using JAXB APIs,  the usage of these modules from the JDK was transparent to the application. So although the application may not explicitly have referred to these module names, it was still reliant on them because they were providing public APIs which the application had references to. Effectively, if a project was using some Ant task which used JAXB, then those projects when they switch to any Java version >= 11 will now start seeing issues due to missing compile and runtime dependency on JAXB. Let’s now consider a simple custom Ant task and see what kind of errors it might encounter. But if you are just interested in the shorter answer and some sample code to fix these classloading issues with JAXB, then please check the end of this article, starting here. For the complete details, please read on.

We will use a trivial custom Ant task implemented by a class called org.myapp.HelloTask, which looks like:


package org.myapp;

import org.apache.tools.ant.Task;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBException;

public class HelloTask extends Task {

    private String message;

    public void setMessage(final String message) {
        this.message = message;
    }

    @Override
    public void execute() {
        try {
            final JAXBContext context = JAXBContext.newInstance(DataContainer.class);
            System.out.println("Created JAXB context " + context);
        } catch (JAXBException e) {
            throw new RuntimeException(e);
        }

        System.out.println(this.message);
    }

    private static class DataContainer {
        private String data;
    }
}

The HelloTask is just here for demonstration and in practice doesn’t provide any real value. This task allows for a message attribute to be specified. Furthermore, in its execute() method it creates a javax.xml.bind.JAXBContext for the application specific DataContainer class and just prints that context. Additionally, it also prints the message that was set when the task is launched. Let’s assume that this task has been compiled and packaged as a jar file and made available to the build process. Now consider this trivial build.xml file which declares this task and then launches it:


<project default="invoke-task">
    <property name="build.dir" value="build"/>
    <property name="jar.name" value="hellotask.jar"/>

    <taskdef name="hello" classname="org.myapp.HelloTask">
        <classpath>
            <pathelement location="${build.dir}/${jar.name}"/>
        </classpath>    
    </taskdef>  

    <target name="invoke-task" description="invokes the HelloTask">
        <hello message="hello world"/>
    </target>       

</project>

There’s not much in this build.xml. All it does is, use a taskdef to define the HelloTask with the classpath containing (just the) jar file containing the HelloTask. Then in the invoke-task target we launch this hello task by passing it a message.

Let’s now run this build against Java 8:

export JAVA_HOME=<path-to-JDK-8>
ant invoke-task

When you run this (on Java 8) you should see the output similar to:


invoke-task:
    [hello] Created JAXB context jar:file:/<path-to-jdk-8>/jre/lib/rt.jar!/com/sun/xml/internal/bind/v2/runtime/JAXBContextImpl.class Build-Id: ...
    [hello] Classes known to this context:
    [hello]   [B
    [hello]   boolean
    [hello]   byte
    [hello]   char
    [hello]   com.sun.xml.internal.bind.api.CompositeStructure
    [hello]   double
    [hello]   float
    [hello]   int
...
    [hello]   long
    [hello]   org.myapp.HelloTask$DataContainer
    [hello]   short
    [hello]   void
    [hello] 
    [hello] hello world

You’ll see that the JAXB usage in the task was successful and the build completed successfully and we didn’t have to configure any classpath to include any JAXB jar files. As noted previously, this is because the Java 8 runtime ships with the relevant JAXB API and implementation classes.

Now, without changing any code in the task or in the build.xml, let’s just switch to a recent Java version. Let’s say Java 17 and run the build:

export JAVA_HOME=<path-to-JDK-17>
ant invoke-task

When you do this, you will now see:

BUILD FAILED
build.xml:5: taskdef A class needed by class org.myapp.HelloTask cannot be found: javax/xml/bind/JAXBException
 using the classloader AntClassLoader[build/hellotask.jar]

You’ll notice that the build now fails and the error message states that the class javax/xml/bind/JAXBException cannot be found in the classpath which as defined in the build.xml only included the hellotask.jar (which just has the org.myapp.HelloTask). As noted previously, this is because the JAXB API and implementation is no longer shipped in the JDK. Applications, like this project, are expected to now include the JAXB API and implementation jars in the application classpath. JEP-320 https://openjdk.org/jeps/320 lists the potential Maven co-ordinates to find such jar(s). In case of JAXB it suggests the JAXB reference implementation available in Maven central repo com.sun.xml.bind:jaxb-ri as a potential candidate. Do note that, just like several other Java EE APIs and implementations, JAXB too has several vendors which implement the JAXB API. A “reference implementation” as the name states is meant to demonstrate the implementation of the specified API. There can, and are, several other vendor implementations for such APIs. It’s upto the applications to choose the ones that they desire to use. In this demonstration, we will use the 2.3.8 version of com.sun.xml.bind:jaxb-ri dependency (available at https://repo.maven.apache.org/maven2/com/sun/xml/bind/jaxb-ri/2.3.8/). There’s no specific reason for my choice of this specific version - it’s only for demo.

Now that we have this dependency made available, let’s include it in the classpath of the taskdef of HelloTask and our build.xml (snippet) will now look like:

...
    <taskdef name="hello" classname="org.myapp.HelloTask">
        <classpath>
            <pathelement location="${build.dir}/${jar.name}"/>
            <!-- JAXB dependencies -->
            <pathelement location="${lib.dir}/jakarta.activation.jar"/>
            <pathelement location="${lib.dir}/jakarta.xml.bind-api.jar"/>
            <pathelement location="${lib.dir}/jaxb-impl.jar"/>
        </classpath>    
    </taskdef>  
...

You’ll notice that the classpath of the taskdef now includes the JAXB related dependency jars. Now let’s rerun the build on Java 17, like previously:

export JAVA_HOME=<path-to-JDK-17>
ant invoke-task

When you run this, you will now see something that starts like this:


BUILD FAILED
build.xml:17: java.lang.RuntimeException: javax.xml.bind.JAXBException: Implementation of JAXB-API has not been found on module path or classpath.
 - with linked exception:
[java.lang.ClassNotFoundException: com.sun.xml.bind.v2.ContextFactory]
    at org.myapp.HelloTask.execute(Unknown Source)
    

So the build still fails, but unlike previously where it had failed when trying to define the HelloTask itself, this time it fails in the execute() implementation of the HelloTask.

So why does it fail even with the JAXB jars in the classpath. This has to do with JAXB (and several other Java EE APIs), which rely on thread context classloader. Several of these APIs, including this call to:

final JAXBContext context = JAXBContext.newInstance(DataContainer.class);

relies on thread context classloader to find the JAXB related classes and resources. It expects the thread context classloader, whichever it is, to be able to load these JAXB classes. Thread context classloaders are specific to the thread that is currently executing the code. So let’s quickly see which classloaders are in play in the execute() method of the HelloTask. To see that, let’s add some trivial debug messages in the code, whose snippet will now look like:


    @Override
    public void execute() {
        System.out.println(HelloTask.class + " was loaded by classloader: " + HelloTask.class.getClassLoader());
        final ClassLoader tccl = Thread.currentThread().getContextClassLoader();
        System.out.println("Context classloader of current thread is " + tccl);
        if (tccl instanceof java.net.URLClassLoader) {
            // let's additionally print the classpath of the URLClassLoader
            final java.net.URL[] classpath = ((java.net.URLClassLoader) tccl).getURLs();
            System.out.println("Context classloader's classpath is " + java.util.Arrays.toString(classpath));
        }
        try {
            final JAXBContext context = JAXBContext.newInstance(DataContainer.class);
            System.out.println("Created JAXB context " + context);
        } catch (JAXBException e) {
            throw new RuntimeException(e);
        }

        System.out.println(this.message);
    }

So what we have done here is that added a few System.out.println messages which prints the classloader which loaded the HelloTask and also prints the current thread’s context classloader. Additionally if the context classloader is a java.net.URLClassLoader, we even print the classpath used by the URLClassLoader. The goal of these debug messages is to see what classloaders are in play when using that JAXB API. Let’s rerun the build again on Java 17 - it’s still expected to fail like previously, but this time we should see these debug messages:

export JAVA_HOME=<path-to-JDK-17>
ant invoke-task

This fails with the same exception stacktrace as previously, but this time you should also see, something like:


[hello] class org.myapp.HelloTask was loaded by classloader: AntClassLoader[build/hellotask.jar:lib/jakarta.activation.jar:lib/jakarta.xml.bind-api.jar:lib/jaxb-impl.jar]
[hello] Context classloader of current thread is java.net.URLClassLoader@6d6f6e28
[hello] Context classloader's classpath is [file:/apache-ant-1.10.14/lib/ant-commons-net.jar, file:/apache-ant-1.10.14/lib/ant-xz.jar, file:/apache-ant-1.10.14/lib/ant-junit4.jar, file:/apache-ant-1.10.14/lib/ant-jai.jar, file:/apache-ant-1.10.14/lib/ant-apache-resolver.jar, file:/apache-ant-1.10.14/lib/ant-jdepend.jar, file:/apache-ant-1.10.14/lib/ant-apache-regexp.jar, file:/apache-ant-1.10.14/lib/ant-apache-log4j.jar, file:/apache-ant-1.10.14/lib/ant-javamail.jar, file:/apache-ant-1.10.14/lib/ant-apache-bcel.jar, file:/apache-ant-1.10.14/lib/ant.jar, file:/apache-ant-1.10.14/lib/ant-netrexx.jar, file:/apache-ant-1.10.14/lib/ant-swing.jar, file:/apache-ant-1.10.14/lib/ant-jsch.jar, file:/apache-ant-1.10.14/lib/ant-junitlauncher.jar, file:/apache-ant-1.10.14/lib/ant-jakartamail.jar, file:/apache-ant-1.10.14/lib/ant-junit.jar, file:/apache-ant-1.10.14/lib/ant-imageio.jar, file:/apache-ant-1.10.14/lib/ant-launcher.jar, file:/apache-ant-1.10.14/lib/ant-antlr.jar, file:/apache-ant-1.10.14/lib/ant-testutil.jar, file:/apache-ant-1.10.14/lib/ant-apache-oro.jar, file:/apache-ant-1.10.14/lib/ant-jmf.jar, file:/apache-ant-1.10.14/lib/ant-apache-xalan2.jar, file:/apache-ant-1.10.14/lib/ant-apache-bsf.jar, file:/apache-ant-1.10.14/lib/ant-commons-logging.jar]

You’ll see that the HelloTask was loaded using an instance of AntClassLoader (which is internal implementation detail of the Ant project) and this classloader has the relevant JAXB jars in its classpath (as seen in the message above). You’ll also notice in the log message that the thread’s context classloader is an instance of URLClassLoader:

[hello] Context classloader of current thread is java.net.URLClassLoader@6d6f6e28

and this instance of URLClassLoader has a classpath which has only Ant specific jars and nothing related to JAXB jars. Now when the specific call to JAXBContext.newInstance(...) gets made it ends up using the URLClassLoader (since it is the thread context classloader) which doesn’t have JAXB jars. Effectively, you end up seeing the classloading failures and the build fails.

So how do we fix this. The important bit here is that the thread context classloader should be the one which has the JAXB classes available, so that it can load them. Java’s java.lang.Thread class allows the context classloader to be changed/switched. In fact, in the Java EE ecosystem, frameworks, servers and other implementations (typically not the application code), switch the thread’s context classloader to a “relevant” classloader at the “right place” and then switch it back to the old context classloader when the operation completes. We will need a similar implementation here in the custom task’s execute() method. Here’s what the snippet will now look like (we no longer need the debug logging, so that’s now been removed):

@Override
    public void execute() {
        // get the current context classloader
        final ClassLoader tccl = Thread.currentThread().getContextClassLoader();
        try {
            // change the thread context classloader to this task's classloader
            // before using the JAXB API
            Thread.currentThread().setContextClassLoader(HelloTask.class.getClassLoader());
            final JAXBContext context = JAXBContext.newInstance(DataContainer.class);
            System.out.println("Created JAXB context " + context);
        } catch (JAXBException e) {
            throw new RuntimeException(e);
        } finally {
            // restore back the old context classloader
            Thread.currentThread().setContextClassLoader(tccl);
        }

        System.out.println(this.message);
    }

Notice that we get hold of the current context classloader and then before calling JAXBContext.newInstance(...) we change the context classloader to the HelloTask’s classloader. The HelloTask’s classloader, as we have seen so far, has the necessary JAXB jars (since we defined it in the classpath of the taskdef in the build.xml) from which it should be able to load the JAXB classes. Finally, and very importantly, in a finally block we restore the context classloader to whatever it was previously - that way rest of the code isn’t impacted by switching the thread context classloader. Let’s now build the project again on Java 17:

export JAVA_HOME=<path-to-JDK-17>
ant invoke-task

When you now run this, you should see:

invoke-task:
    [hello] Created JAXB context jar:file:/lib/jaxb-impl.jar!/com/sun/xml/bind/v2/runtime/JAXBContextImpl.class Build-Id: 2.3.8
    [hello] Classes known to this context:
    [hello]   [B
    [hello]   boolean
    [hello]   byte
    [hello]   char
    [hello]   com.sun.xml.bind.api.CompositeStructure
    [hello]   double
    [hello]   float
    [hello]   int
    ...
    [hello]   org.myapp.HelloTask$DataContainer
    [hello]   short
    [hello]   void
    [hello] 
    [hello] hello world

So the build now succeeds.

To summarize, recent versions of JDK have removed certain APIs from the JDK. Some of such APIs are available as external libraries which can be configured in the application classpath. In context of Ant, if a task that is shipped by Ant makes use of such APIs, then the Ant release note and the manual of that task will make a note of such change and will also note what needs to be done to get that task functional. In other cases, where custom tasks are involved and depend on APIs that are no longer part of the JDK, on most occasions they will need to update their build files to include those dependencies in their taskdef’s classpath. In some additional cases, like this very specific JAXBContext API usage, they might even have to do changes to the task’s code to use the right classloader. Do note that I decided to use this specific API of JAXB only to demonstrate the classloader change that would be needed in the task. Not all tasks that use JAXB would need this change in the task’s code - the build.xml classpath changes are expected. Also note that switching of classloaders shouldn’t be done blindly as it can cause other classloading issues. It should only be done when the specific API call in question has semantics which specify the use of a context classloader.

Some of you would be wondering if Ant itself should be doing a change where it sets the “right” thread context classloader before invoking the tasks. It wouldn’t be right for Ant to be doing such a change - depending on what the custom task does, there could be different answers to what is the “right” thread context classloader to use and for what duration. Answers to either of those questions are task specific and as such should be handled/implemented in the (custom) tasks.

Sunday, May 16, 2021

Special characters in proxy configuration of docker daemon and systemd

 
This post is about some of the struggles I've had while dealing with configuring HTTP(s) proxy for docker daemon process. I note it here for myself as well as others who might run into this issue in future.

People familiar with Docker will know that it has a process (called docker daemon) which is responsible for handling requests from a docker client - like the docker CLI or language specific libraries like Python's docker package. This docker daemon process is responsible for handling these requests (for example for a docker image pull) and serving those requests. In that interaction, the docker daemon will communicate with docker registries either hosted internally or available over the Internet. If your system(s) where you have the docker daemon running have a proxy installed, then you will have to configure your docker daemon process to use those proxies.

The Docker documentation has a section which explains how to do that https://docs.docker.com/config/daemon/systemd/#httphttps-proxy. It's pretty straightforward as you see it there. You just configure the HTTP_PROXY and HTTPS_PROXY environment variables to point to the proxy URL.

Now imagine your proxy requires a username/password authentication. Configuring this detail too is pretty straightforward - you configure the HTTP_PROXY and HTTPS_PROXY environment variable just like before, but also include the username and password in the URL. For example, if your proxy host was 192.168.10.12 and the user name was "john" and password was "doe", you would configure the environment variable as follows:

Environment="HTTP_PROXY=http://john:doe@192.168.10.12:80"

in that configuration file noted in the Docker documentation.

Things are fine and straightforward so far. However, it gets interesting when the user name (or password) has a special character involved. For example, imagine the username is a Windows style user name where you specify the domain and the user name separated by a "\" (backslash) character. So imagine "john" being in the "dev" domain, so the username for authentication is "dev\john". Using this value literally as follows will not work:

Environment="HTTP_PROXY=http://dev\john:doe@192.168.10.12:80"

The value provided in HTTP_PROXY is used by docker daemon process and it expects it to be URL encoded. What that means is characters like the "\" (backslash) need to be handled specifically and are expected to be URL encoded. So the URL encoded version of that character is %5C. One would expect that setting that environment variable to include this encoded value would work. So:

Environment="HTTP_PROXY=http://dev%5Cjohn:doe@192.168.10.12:80"

But no - docker daemon's interaction with the docker registries, which is one of the places where the proxy gets used, kept failing. It wasn't clear why this was failing, however, looking at the logs that get generated in /var/log/messages, this message stood out:

Jan 1 04:00:12 localhost systemd[1]: /etc/systemd/system/docker.service.d/http-proxy.conf:2: Failed to resolve unit specifiers in HTTP_PROXY=http://dev%5Cjohn:doe@192.168.10.12:80, ignoring: Invalid slot

So clearly the value was being picked up but was being considered invalid and ignored. Thus leading to issues while dealing with the proxy on that system. It wasn't clear why it was considering that value invalid. After all, we had already URL encoded that special character.

It took a bit of time before I could spot a hint in that log message. If you look at that log message closely, you will notice that the log message is being logged by "systemd" not by the docker daemon. This is a sign that "systemd" is the one which is considering this value invalid. So how's systemd involved in this? In context of this issue, systemd is the process which manages "services" on your operating system. The docker daemon is just another such service. The file(s) we have been using to configure the HTTP_PROXY environment variable are infact configuration files that systemd parses and manages for the individual services. So clearly, systemd doesn't like this specified value for the HTTP_PROXY environment variable.

Reading through the systemd documentation, it became clear why it was having a problem with this value. Turns out for systemd configuration files (like the one we have here), % happens to be a special character and represents something called as a "specifier". So clearly, as noted by that warn message, it's trying to use %5 as a specifier and fails to understand what specifier it is. So we need to add another layer of escaping (although for a different tool - this time systemd) and we have to escape the % character to tell systemd not to interpret it as a specifier. To do that we use the % character, so it now becomes %%5C (as noted in https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Specifiers %% means use % literal).

So that now means, the final value for our HTTP_PROXY environment variable, as configured in the systemd service's configuration file is:

Environment="HTTP_PROXY=http://dev%%5Cjohn:doe@192.168.10.12:80"

(notice the use of %%)

Once this was done and the docker daemon restarted, things started working fine and the docker daemon was able to interact with the docker registries.

It's always tricky to get tools/libraries working when special characters are involved. As seen here, it gets even more trickier when multiple tools are involved in using the same value and each one has a different set of special characters.

I hope this post helps those who run into similar issues with docker daemon or systemd configurations in general.

Apache Ant 1.10.10 released - Better test result summary from junitlauncher task

Apache Ant 1.10.10 got released around a month back. Among the usual bug fixes, we added a new enhancement for the "junitlauncher" task.

For those of you who haven't used or know about "junitlauncher" task, it's a new task we introduced a few years back to allow projects using Ant, to be able to use the new JUnit5 testing framework. The previous (and still supported) "junit" task is meant to be used only if you want to continue using just JUnit4. If you plan to use JUnit5 (which also supports JUnit4 style testcases), then you will have to use the "junitlauncher" task.

This "junitlauncher" task has been around for a few years now and some users have reported that its "printSummary" feature isn't of much use. People familiar with the "junit" task will know that when a test gets run, the task prints an instantaneous summary like:

org.myapp.foo.bar.SimpleTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec

This is useful to see a quick summary of the tests being run.

The "junitlauncher" has a "printSummary" attribute which until Ant 1.10.10 version used to print a summary after all the tests had been executed. Furthermore, the printed summary was a summary that the JUnit5 framework generates by default, something like:

[junitlauncher]
[junitlauncher] Test run finished after 5103 ms
[junitlauncher] [         2 containers found      ]
[junitlauncher] [         0 containers skipped    ]
[junitlauncher] [         2 containers started    ]
[junitlauncher] [         0 containers aborted    ]
[junitlauncher] [         2 containers successful ]
[junitlauncher] [         0 containers failed     ]
[junitlauncher] [         1 tests found           ]
[junitlauncher] [         0 tests skipped         ]
[junitlauncher] [         1 tests started         ]
[junitlauncher] [         0 tests aborted         ]
[junitlauncher] [         1 tests successful      ]
[junitlauncher] [         0 tests failed          ]

As you can see, summary of this form isn't really useful. So some of the Ant users requested (https://bz.apache.org/bugzilla/show_bug.cgi?id=64836) this to be improved to provide a summary which resembled what we have with the "junit" task.

This Ant 1.10.10 release now consists that enhancement. When you use "printSummary=true" on the "junitlauncher" task, it will now print a more useful and immediate summary like the "junit" task does:

Running org.myapp.foo.bar.SimpleTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec


As usual, the release is available for download at the Ant downloads page https://ant.apache.org/bindownload.cgi. Please give this a try and if you have any suggestion or feedback on this release, please get in touch with us on our mailing lists https://ant.apache.org/mail.html or our issue tracker https://ant.apache.org/bugs.html.