Technical Exasol

Next level unit-testing with JUnit5

02 Apr 2019 | Share

In our Java-based Open Source repositories we at Exasol use JUnit as our testing framework. Like most organizations we used JUnit4 for a long time. But for all new projects we have switched to JUnit5. We’re also migrating existing tests in established projects across, every time we make changes.

What makes JUnit5 better than JUnit4?

Long running software projects tend to experience a lot of inertia when faced with major updates of the underlying frameworks. As a software architect this is one of the many reasons why I recommend keeping the number of frameworks used to the necessary minimum, so that you can quickly migrate and keep your products lean.

In the particular case of JUnit the benefits from the version upgrade to 5 could not be greater.

Encapsulation

In JUnit4 all test classes and methods needed to be public. With JUnit5, package-scope becomes the default, thus providing better encapsulation.

package com.exasol.adapter.capablities;

import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
// ...

class CapabilitiesTest {
    @BeforeEach
    void beforeEach() {
        // ...
    }

    @Test
    void testCreateEmptyCapabilities() {
         // ...
    }
}

Support for grouped tests

Remember how it was a very bad idea in JUnit4 to have more than one assert per test?

The reason for this was that after the first failed assert, the test bailed out. So effectively all asserts after that couldn’t give you any information. Sometimes you need to check multiple things in one test case to validate consistency. And you were forced to write your own asserts that threw the assertion error in one place only and needed to document the individual sub-tests.

In contrast JUnit5 supports groups of tests out-of-the-box. The new assertAll(…) method lets you run a series of lambdas each containing its own assertion. JUnit5 nicely documents the results of each individual sub-tests and guarantees to execute all of them even if one ore more failed.

// ...
import static org.hamcrest.MatcherAssert.assertThat;
// ...

class CapabilitiesTest {
   //...
 
   @Test
    void testCreateEmptyCapabilities() {
        final Capabilities capabilities = this.builder.build();
        assertAll(() -> assertEmptyMainCapabilities(capabilities), //
              () -> assertEmptyLiteralCapabilities(capabilities), //
              () -> assertEmptyPredicateCapabilities(capabilities), //
              () -> assertEmptyScalarFunctionCapabilities(capabilities), //
              () -> assertEmptyAggregateFunctionCapatilities(capabilities));
    }

    // ...
}

If that isn’t hierarchical enough for you, have a look at nested tests.

Changing the display name of a test

This is a small but handy feature: if you want to, you can change the name that will be displayed for each test case in the test logs. This is especially useful if you want to integrate information in the test logs that allow correlation with a test specification.

In the example below we add bot, a human-readable description and a machine-parseable identifier like the ones used in OpenFastTrace. This way you can later scan the test report and see if all test cases mentioned in your specification ran.

@DisplayName("Get capabilities [utest~get-capabilites~3]")
void testGetCapabilities() {
    // ...      
}

A uniform extension mechanism

JUnit4 has two competing extension methods: Rules and Runners. Depending on what you want to do, you need to pick the right mechanism. And to make matters worse, a test can only be run by a single runner. So combining a MockitoJUnitRunner and a Paremeterized runner is simply not possible even if that would be really useful.

If you wanted to build your own extensions with JUnit4, you needed to invest time to learn both of the above mentioned extension methods.

JUnit5 on the other hand has a one-size-fits-all uniform extension mechanism aptly called “Extensions“.

And the best thing about it is that it is very easy to understand — so writing your own extensions isn’t a big deal. It only took me a handful of days to learn about JUnit5 and build the “junit5-system-extensions“, which allow you to capture and check STDOUT and STDERR as well as intercepting and checking system exits.

At the core of this mechanism are the so called “Test Lifecycle Callbacks“. They let you hook your own code into certain phases of the test preparation, execution and clean up.

Here is an example from the ExitGuard extension that I wrote. I removed some details to point out the extension mechanism more clearly

public final class ExitGuard
        implements TestInstancePostProcessor, BeforeTestExecutionCallback,
                   AfterTestExecutionCallback, AfterAllCallback
{
    private static final String PREVIOUS_SECURITY_MANAGER_KEY = "PREV_SECMAN";
    private static final String EXIT_GUARD_SECURITY_MANAGER_KEY = "EXIT_SECMAN";

    @Override
    public void postProcessTestInstance(final Object testInstance,
                                        final ExtensionContext context)
    {
        saveCurrentSecurityManager(context);
        installExitGuardSecurityManager(context);
    }


    // ... removed for compactness

    @Override
    public void beforeTestExecution(final ExtensionContext context)
            throws Exception
    {
        getExitGuardSecurityManager(context).trapExit(true);
    }

    @Override
    public void afterTestExecution(final ExtensionContext context)
            throws Exception
    {
        getExitGuardSecurityManager(context).trapExit(false);
    }

    @Override
    public void afterAll(final ExtensionContext context)
            throws Exception
    {
        final SecurityManager previousManager =
                (SecurityManager) context.getStore(getNamespace())
                    .get(PREVIOUS_SECURITY_MANAGER_KEY);
        System.setSecurityManager(previousManager);
    }
}

The class implements interfaces for lifecycle phase callbacks. Concentrate on the public methods and you see how easy it is to extend the behavior of the test preparation and execution. In this example we extend four of the six phases. In most cases one or two will be sufficient though.

Coexistence with Hamcrest matchers

While the assertion methods in JUnit5  greatly improved compared to JUnit4, Hamcrest assertions are still better. Luckily you can use them together with JUnit5 even if JUnit5 dropped the assertThat(...) method that is necessary for invoking a Hamcrest matcher.

Just use the assertThat(...) method that is shipped with newer versions of Hamcrest instead.

//...
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.contains;
import static org.hamcrest.Matchers.containsInAnyOrder;
//...

assertThat(capabilitiesWithExclusion.getMainCapabilities(),
      contains(MainCapability.AGGREGATE_GROUP_BY_EXPRESSION));
//...

JUnit5 wins, hands down

JUnit5 is a major step forward in unit testing Java software. You benefit from improved encapsulation, better assertions and a uniform and easy to implement extension model. Exasol uses JUnit5 in all new projects and migrates existing ones step-by-step. Start learning JUnit5 today and enjoy cleaner code and better test reports.

10 trends impacting data analytics

Now that we’re well and truly in the age of data, what’s coming next? 

Free

Whitepaper