Tuesday, January 17, 2012

Emma Plugin for Jenkins: Easy Code Coverage Reports


Overview

Data lovers, rejoice: Jenkins and Emma can help satisfy your urge to quantify your tests' code coverage. Behold, the Emma Plugin for Jenkins. For the un-initiated, code coverage is a measure of how well a test suite exercises a given code base. These tools will report what percentage of your packages, classes, methods or lines of code are covered by your automated tests. While good code coverage is not necessarily an indicator of quality, poor code coverage (also called "test coverage") often correlates with frequent regressions and a fragile code base (results may vary).

Emma is a free code coverage tool written in Java, for Java. Emma generates code coverage reports based on automated (or even manual!) tests of instrumented Java code. The Maven plugin for Emma will automate the code instrumentation, run the tests and generate the reports for you, so that you only have to type "mvn emma:emma" at the command line to get a coverage report. There are even Ant tasks for Emma, in case your code base is stuck in 2001. Likewise, a Gradle plugin is also available. (Hail, programmer of the future, does the world end in 2012?).

It is important to understand that Jenkins does not really know how to run Emma for you (a common misconception). The Emma Plugin for Jenkins is a reporting plugin. This means it runs after the build and tests are complete, in order to process the Emma output into a form which Jenkins understands. You are responsible for invoking Emma for your build using one of the approaches outlined above.

Stable Release Version

The current stable release is 1.26.

Requirements For Use

  • Jenkins 1.398 or newer.
  • A build which generates the Emma report using one of the approaches above.

Instructions

You can install the Emma plugin by going to your Update Center (click "Manage Jenkins", then "Manage Plugins") and select the Emma Plugin:

Then click the Install button.
Next, edit your job. In the Post-build Actions section, check the Record Emma coverage report.In the provided text box, enter the path-like pattern to tell Jenkins where to look for the reports. (for instance "*/coverage.xml"). If you leave it blank, Jenkins will search your entire workspace for the coverage.xml generated by Emma (which may take a while if your workspace is large!).


Once you run your build, you'll get to see a graph on your job's index page which plots your code coverage over time, and you'll see it change over time with each build, as you add code and tests.

This graph, like all standard Jenkins graphs, lets you click on the graph to drill down to a specific build. When you do, you'll see the Emma report for that build:


You can then drill down to individual classes, but coverage is only reported on the method level. Ideally, the plugin would let you see the source file and highlight individual lines to indicate the coverage of specific lines, but this plugin does not support that yet. (Sounds like a great feature for someone to contribute to this plugin!)

Alternatives

There are a few alternatives to Emma. Cobertura is another open source code coverage tool for Java, which also (surprise!) has a Jenkins plugin. Clover is a proprietary coverage engine from the fine folks at Atlassian (Jenkins plugin. Finally, Sonar also provides a view into Emma code coverage results and helps you track coverage over time, along with lots of other juicy metrics about your code. Even better, you can get a Sonar instance integrated with your DEV@Cloud Jenkins instance, making it even easier to get started tracking your code coverage with Jenkins and Emma.

Relevant Documentation

Ryan Campbell
Developer

CloudBees
www.cloudbees.com
 
Follow CloudBees:
Facebook Twitter

1 comment:

  1. Emma is pretty much deprecated and the team developing it is now concentrating on its successor, JaCoCo.

    There is already an issue to support JaCoCo: https://issues.jenkins-ci.org/browse/JENKINS-10835

    ReplyDelete