Specification By Example, amongst other things, allows to establish trust between product owners and dev teams. Integrating Concordion with Nuxeo test runner establishes living documentation which demonstrates the specifications and test results to all teams.
Introduction
The Nuxeo Platform already features a rich test framework, thanks to JUnit runner and the modularity of the platform. In any agile project, one of the key factors of success is the trust between product owners and developers. However, it’s hard to show JUnit’s test results in a demo! This often results in QA teams redoing some tests manually that have already been automated.
In a recent project, I had to migrate data from a legacy system into the Nuxeo Platform. It required implementing a data mapping and some data transformation rules. Since the business model was quite big and the amount of data was very large, we had to ensure that the business requirements were tested. In order to establish trust between the teams, we decided to implement what is called Specification By Example (SBE).
Specification By Example
An SBE requires the product owner to produce executable specifications. In this blog, we will rely on a simple example of specification: we want to migrate a list of books contained in a simple CSV file into a Nuxeo Library. In order to search our books, our library must be able to store a Book and its properties. A book has the following properties: * a title * an autor * a publication date * an ISBN * a description For example: When the file books_samples_1.csv is imported, then the number of Books in the system is 4. This specification describes what is expected from a business point of view and also gives some examples of expected values. The whole point of what is called a “Living Documentation” is to make that specification document executable.
Concordion to the Rescue
Concordion is a Java library that allows to bind an HTML document to methods in a Fixture - a Java class that does various binding work. When you execute that specification, it generates an HTML output of what has been tested, with errors in red and ‘test ok’ in green. So let’s take the first example and see how we can execute it.
<small>When the file <b concordion:set="#importFile">books_samples_1.csv</b>
is <b concordion:execute="importCSV(#importFile)">imported</b>, then the
number of <b concordion:set="#doctype">Book</b>
in the system is <b concordion:assertEquals="getDocCount(#doctype)">12</b>.</small>
In the HTML, we’ve inserted various tags that can set variables, and another one that executes a method. The last one is an assertion on the number of documents that have been retrieved. The methods referenced in the HTML just call the ones that are in the Fixture class.
<small>@RunWith(ConcordionFeaturesRunner.class)
@Features(BookFeature.class)
public class BookFixture {
@Inject
CoreSession session;
public void importCSV(String fileName) {
File file = new File(getClass().getClassLoader().getResource(fileName).toURI());
BookAdapter.importCSV(session, file);
}
public int getDocCount(String docType) {
String query = "SELECT * FROM " + docType;
return session.query(query).size();
}
}</small>
This test uses a special runner that is a mix of the Nuxeo Feature runner and the ConcordionRunner. It allows to run Concordion tests with all the features of the Nuxeo Junit test runner. If we run that test, here is the result we have:
After adding the doc type to Nuxeo and implementing the needed methods, the count of document turns green. This is the basis for Concordion tests: expose some specifications and examples with words and execute that test.
Testing Multiple Values
In our example, we would also like to pick up some books based on their ISBN and then test the values that have been imported. Concordion features a nice way to test list of values by using HTML tables. For instance this table can be tested:
Just by using specific Concordion attributes on the HTML table tags, we can describe our tests:
<small><table concordion:execute="#book = findBookByISBN(#isbn)">
<tr>
<th concordion:set="#isbn">ISBN</th>
<th concordion:assertEquals="#book.title">Title</th>
<th concordion:assertEquals="#book.author">Author</th>
<th concordion:assertEquals="#book.publicationYear">Publication Year</th>
<th concordion:assertEquals="#book.description">Description</th>
<th concordion:assertTrue="isCreatedNow(#book)">Creation date</th>
<th concordion:assertTrue="isModifiedNow(#book)">Modification
date</th>
</tr></small>
This means that, at each line, we will set the ISBN value to the value of the first column, then execute the findBookByISBN() method and finally evaluate the assertions on each column.
In our sample, this gives the following results where we can see that we have an error in our expectations: a Gollum bug has been inserted and it replaced all occurrences of “the Ring” to “my Precious”. We just have to fix that awful Gollum to get the whole table and the specification in green.
Going Further
By writing the first executable specification, we’ve introduced the concept of Specification By Example. Multiple specifications will cover your whole business model and make your product owner confident in what developers are delivering.
The power of HTML links allows you to link specifications to each other in order to avoid redundancy.
Concordion is just a tool whose advantage is that it’s very quick to setup in a Maven/Java environment and doesn’t need any server. But it does have a few drawbacks:
- Specifications are written in HTML so the knowledge of HTML is mandatory to write them. If your product owner is an HTML expert, it’s great!
- The tool to run those tests is a developer tool so it’s sometimes hard for a product owner to test new cases quickly unless he knows how to use Git and Maven :(
Some other tools don’t have those problems and often rely on a DSL or a wiki with a Test button which allows to quickly edit the specs and run them on new sample cases:
- FitNesse : embeds its own wiki server and works with decision table
- Cucumber : DSL based specifications
Finally, writing these type of tests correctly is not straightforward and needs practice. There is a bad smells section on Concordion which is very instructive. You can also read the Specification By Example from Gojko Adzic. The source code for our example is on github: https://github.com/dmetzler/nuxeo-book-concordion
Check it out and happy specifications!