JMockit An automated testing toolkit for Java

About the JMockit Testing Toolkit

JMockit is open source software licensed under the MIT License. It includes APIs for mocking, faking, and integration testing, and a code coverage tool. The library is meant to be used together with a testing framework such as JUnit or TestNG.

The toolkit was created mainly as an attempt to overcome certain limitations found in "conventional" mocking tools, which prevented the creation of unit tests for code designed according to well-established OO practices. Another goal was to provide simpler and more succinct APIs for writing developer tests. In addition, and differently from other testing tools which specifically target the use of mocks, JMockit also includes advanced support for out-of-container integration testing, and a sophisticated code coverage tool.

How it works

Internally, the Java SE 6+ bytecode instrumentation feature (the java.lang.instrument package) is used extensively. For the reading and in-memory generation of modified bytecode arrays, the ASM library is used.

Mocking features mostly use the JVM's ability to redefine classes that have already been loaded, as exposed through the Instrumentation#redefineClasses method. This method uses the same mechanism behind the "hotswap" feature commonly used while debugging Java code, which allows a new version of a class to take the place of its current version in a running JVM.

Whenever a class to be mocked is found in the course of a test run, it has its current bytecode definition temporarily modified, so that the regular behavior of methods and constructors can be replaced with mock behavior, as specified in a test. As soon as the test is completed, the class has its bytecode restored, and its original behavior is resumed. This process repeats itself for the next test, whenever needed as specified in test code through the mocking API.

Another particularity of the mocking API is the special use of fields inside of user-defined Expectations/Verifications subclasses, for recording/verifying expectations. Rather than actual field reads/writes, uses of fields such as "anyString" and "result" get transparently rewritten, at the bytecode level, as method calls (to methods in a specific internal class). This is performed by a ClassFileTransformer implementation which JMockit puts in place when it gets initialized.

Conventional tools for mock objects

The JMockit approach is an alternative to the conventional use of "mock objects" as provided by tools such as EasyMock, jMock, and Mockito.

Such tools are based on the dynamic generation of implementation classes (through java.lang.reflect.Proxy, when given an interface to be mocked) and subclasses (through CGLIB, when given a non-final class to be mocked). This particular mocking implementation technique, in which a subclass with overriding mock methods is created, implies some penalties: final classes, constructors, and non-overridable methods simply cannot be mocked. Most importantly, however, these limitations mean that any dependency that a class under test may have on other classes must be controlled by the tests, so that mock instances can be passed to the clients of those dependencies; that is, dependencies cannot simply be instantiated with the new operator in the client class.

Code Coverage

In addition to mocking APIs, the JMockit toolkit includes its own code coverage tool, JMockit Coverage. This tool aims to provide the following benefits.

  1. Bytecode modification performed only at runtime, therefore avoiding the creation of undesirable files. No extra source or class files are created, and no coverage data file is generated unless explicitly requested. The only files created or modified are those that the user really wants as the desired output.
  2. Running tests with JMockit Coverage does not require the use of any particular command line script, Ant task, Maven plugin, or IDE-specific plugin. The tool applies the idea of convention over configuration, trying to make it as easy as possible for the developer to run tests with code coverage: by simply adding one jar file to the classpath, and optionally specifying certain initialization parameters to the JVM (system properties settable with "-D" or its Maven/Gradle equivalent).
  3. No need to specify which classes should be considered for coverage and which should not (by default, as they can be explicitly specified if needed). All code under test will automatically be analyzed for coverage. Specifically, the tool will gather coverage data for all production code executed by the test suite, excluding classes defined inside jar files (on the assumption that such code belongs to libraries used by the code under test).
  4. An HTML report and/or a serialized output file can be easily generated at the end of each test run. The first requires no extra configuration as it is the default form of coverage output; the second only requires setting the "coverage-output" system property. For the generation of the HTML report, Java source files are automatically searched in all "src" directories under the working directory.
  5. In code coverage reports, each line of production code can be accurately linked to the lines in test code which caused their execution, for each individual execution of each line of code. (However, this feature is inactive by default, because of the higher runtime cost and the larger resulting HTML report.)
  6. Besides a line coverage metric, a path coverage metric is also made available. Both metrics are calculated and shown at the source file, package, and global levels. For path coverage, each possible path through a method or constructor can be interactively displayed in the HTML report.
  7. Intra-line coverage is provided, with the appropriate red/green coloring in the HTML report for each conditionally executed line segment.

In short, the two main differentiating factors for JMockit Coverage are that 1) it generates the coverage report as easily as possible and without depending on special conditions or requiring specific plugins for Java IDEs or build tools; and 2) it provides newer and more sophisticated coverage metrics, such as path coverage and true line coverage.