Tag Archives: TDD

Unit testing JAXB marshalling and XJC-generated classes

JAXB – the Java Architecture for Xml Binding provides a simple way of mapping XML to POJOs. It give the ability of painlessly marshalling and unmarshalling objects to and from XML.

JAXB’s usefulness is enhanced by the ‘xjc’ tool that is included in the SDK, which converts an XML schema to a set of Java classes.

A portion of the XML schema (which itself is generated from an XML file):

  <xs:complexType name="MetaType">
      <xs:attribute type="xs:string" name="Name" use="optional"/>
      <xs:attribute type="xs:string" name="Scheme" use="optional"/>
      <xs:attribute type="xs:string" name="Value" use="optional"/>
  </xs:complexType>

Because the XML schema I’m working with has been auto-generated from sample XMLs and not hand-written (and fairly complex!), I’d like to ensure that the XML coming out of the marshalling is what I expect.

The following JUnit 4 test creates and populates the object, then verifies that the object is marshalled to XML properly:

public class MetaTypeTest {
    private final String _xmlHeader = "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>";

    @Test
    public void shouldMarshalAllAttributes() throws Exception {
        final MetaType type = new MetaType();
        type.setName("MetaName");
        type.setScheme("MetaScheme");
        type.setValue("MetaValue");

        // Can't be certain @XmlRootElement annotation has been generated, so wrap obj in JAXBElement
        final JAXBElement element = new JAXBElement(new QName("Meta"), MetaType.class, type);

        // Marshal to output stream
        JAXBContext context = JAXBContext.newInstance(MetaType.class);
        final ByteArrayOutputStream outStream = new ByteArrayOutputStream();
        context.createMarshaller().marshal(element, outStream);

        final String xmlContent = "";
        Assert.assertEquals(_xmlHeader + xmlContent, outStream.toString());
    }
}

More..

Advertisements

Building a JStack Parser

Background

Recently, I was briefed on a situation where a main public facing java web application was going down fairly regularly, for what appeared to be an out of memory error.

After an investigation by a previous developer turned up no concrete leads, they suggested using JStack to get a profile of what the threads in the application were holding locks that could be causing the crash.

Now I’ve never worked with JStack before, but the purpose seems fairly straightforward. Trouble is, with ~500 threads in the output, analysing each one to find a common cause would be a pretty intensive task.

I set about searching for some kind of output analyser for JStack, but with no luck, I decided to write my own to give me a summary of the results.

Requirements

At least for my application, there were a lot of similar-looking stacktraces, so it made sense to group and summarise the results.

The JStack output I was provided with looked something like this:

Attaching to process ID 18526, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 1.5.0_14-b03
Thread t@6872: (state = BLOCKED)
- java.lang.Thread.sleep(long) @bci=-1766132960 (Interpreted frame)
- java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
- sun.net.www.http.KeepAliveCache.run() @bci=3, line=149 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)
Thread t@6871: (state = BLOCKED)
- java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise)
- java.util.TimerThread.mainLoop() @bci=201, line=509 (Compiled frame)
- java.util.TimerThread.run() @bci=1, line=462 (Interpreted frame)

What I initially wanted from my parser was to:

  • See the total number of thread states
  • Summary information for each stacktrace.

So what I need to do is extract each thread and its corresponding stacktrace to be able to process and report on it.

The Code

Firstly the main checks for a file parameter, then passes to the application instance which will check for its existence before calling a parser.

public class ParseJStack {
    public ParseJStack(final String filename) {
        File jstackFile = new File(filename);
        if (!jstackFile.exists()) {
            System.out.println("File does not exist. Exiting.");
            return;
        }
        //new Parser(jstackFile).process();
    }

    public static void main(String[] args) {
        if (args.length < 1) {
            System.out.println("Program requires a filename");
            System.exit(1);
        } else {
            new ParseJStack(args[0]);
        }
    }
}

For now, the call to the parser is commented out, as I’m yet to create it or determine its return type.

Test-driven

Now that I’m thinking about main functionality of the application, I’ll write testcases (JUnit 4) around the container classes JStackMeta and JStackEntry which are the thread detail and stacktrace objects respectively, and will be populated by the parser engine

JStackEntry will be a fairly simple container for a List of stack calls, and a String for its header text. From the looks of it the thread headers all follow the same pattern, so I will expect to be able to extract the state (and the id if we need to later) with a simple regex pattern.

public class JStackEntryTest {
    @Test
    public void shouldGetContentsAsNotNullWhenNullParameter() throws Exception {
        final JStackEntry stackEntry = new JStackEntry("");
        Assert.assertNotNull(stackEntry.getContents());
    }

    @Test
    public void shouldGetContents() throws Exception {
        final JStackEntry stackEntry = new JStackEntry("");
        stackEntry.append("Test Content");

        Assert.assertEquals(12, stackEntry.getContents().length());
        Assert.assertEquals("Test Content", stackEntry.getContents().toString());
    }

    @Test
    public void shouldGetEntryState() throws Exception {
        final JStackEntry stackEntry = new JStackEntry("Thread t@6872: (state = BLOCKED)");
        stackEntry.append("Test Content");

        Assert.assertEquals("BLOCKED", stackEntry.getState());
    }

    @Test
    public void shouldGetUnknownStateWhenHeaderEmpty() throws Exception {
        final JStackEntry stackEntry = new JStackEntry("");
        stackEntry.append("Test Content");

        Assert.assertEquals("UNKNOWN", stackEntry.getState());
    }
}

JStackMeta will hold the collection of JStackEntry objects, and the non-entry related meta (e.g. process id, java version info that appears at the top of the file).

public class JStackMetaTest {
    @Test
    public void testConstructorInit() throws Exception {
        Assert.assertNotNull(new JStackMeta().getHeader());
        Assert.assertNotNull(new JStackMeta().getEntries());
    }

    @Test
    public void testAppend() throws Exception {
        final JStackMeta stackMeta = new JStackMeta();
        stackMeta.append("Test Meta");
        Assert.assertEquals(9, stackMeta.getHeader().length());
    }

    @Test
    public void testAddEntry() throws Exception {
        final JStackMeta stackMeta = new JStackMeta();
        final JStackEntry stackEntry = new JStackEntry("");
        stackEntry.append("Test Entry");
        stackMeta.addEntry(stackEntry);
        Assert.assertEquals(1, stackMeta.getEntries().size());
    }
}

With those two classes set, we move back to the JStackParser which populates them. It’ll take the File handle provided by the application object, read in the contents and store them a JStackMeta and its JStackEntry objects. To test this class I created a test.jstack file (which is the same content as the example at the top of this post).

public class ParserTest {
    @Test
    public void testProcess() throws Exception {
        final URL resource = getClass().getResource("test.jstack");
        final File file = new File(resource.getFile());

        final JStackMeta stackMeta = new Parser(file).process();

        Assert.assertEquals(132, stackMeta.getHeader().length());
        Assert.assertEquals(2, stackMeta.getEntries().size());
    }
}

The actual parsing function looks like this (note that I’ve decided that it is important to keep the newlines in the stack trace to simplify the output):

    /**
     * Process the JStack output file and extract the data into a {@link JStackMeta} object.
     *
     * @return The {@link JStackMeta} object representing the JStack output.
     */
    public JStackMeta process() {
        _meta = new JStackMeta();
        try {
            FileInputStream inputStream;
            inputStream = new FileInputStream(_file);

            BufferedReader in = new BufferedReader(new InputStreamReader(inputStream));

            boolean finishedHeader = false;
            JStackEntry currentEntry = new JStackEntry("");

            String line;
            while ((line = in.readLine()) != null)
            {
                // Skip blanks
                if ("".equals(line.trim())) {
                    continue;
                }
                line += "\n";

                // Check if we're done with the header lines
                if (!finishedHeader && line.startsWith("Thread")) {
                    finishedHeader = true;
                }

                if (!finishedHeader) {
                    _meta.append(line);
                    continue;
                }

                if (line.startsWith("Thread")) {
                    currentEntry = new JStackEntry(line);
                    _meta.addEntry(currentEntry);
                } else {
                    currentEntry.append(line);
                }
            }

            in.close();
        } catch (FileNotFoundException e) {
            System.out.println("ERROR: File was not found");
        } catch (IOException e) {
            System.out.println("ERROR: A problem occurred");
            e.printStackTrace();
        }

        return _meta;
    }

Reporting

The last part of the application is a simple analysis of the JStackMeta object and reporting the statistics. As per the requirements, I want two parts to this, the totals for the different states and the single-line summary with counts.

For the purposes of a quick solution, I’m writing the output to the console. I’ll make a Report interface, just in case I get inspired and write some other implementation to report to a text or HTML file..

I’ll use two maps to keep track of the totals – one map for the status counts and one for the line summary totals. To keep the second part looking organised, I’ll use a TreeMap so that the entries remain sorted.

public interface Report {
    void buildReport(JStackMeta stackMeta);
}
public class ConsoleReport implements Report {
    public void buildReport(final JStackMeta stackMeta) {
        Map<String, Integer> stateCountMap = new HashMap<String, Integer>();
        Map<String, Integer> messageCountMap = new TreeMap<String, Integer>();  // Sorted

        // Report on results
        for (int i = 0; i < stackMeta.getEntries().size(); i++) {
            JStackEntry entry = stackMeta.getEntries().get(i);
            final String state = entry.getState();

            final Integer count;
            if (stateCountMap.containsKey(state)) {
                count = stateCountMap.get(state) + 1;
            } else {
                count = 1;
            }
            stateCountMap.put(state, count);

            final StringBuilder contents = entry.getContents();
            final String strStackEnd;
            if (contents.length() != 0) {
                strStackEnd = "(" + state + ") " + contents.substring(0, contents.indexOf("\n"));
            } else {
                strStackEnd = "(" + state + ") " + "[No stacktrace]";
            }

            final Integer countMessage;
            if (messageCountMap.containsKey(strStackEnd)) {
                countMessage = messageCountMap.get(strStackEnd) + 1;
            } else {
                countMessage = 1;
            }
            messageCountMap.put(strStackEnd, countMessage);
        }

        // State counts
        for (Map.Entry<String, Integer> entry : stateCountMap.entrySet()) {
            System.out.println(entry.getValue() + " threads at " + entry.getKey());
        }

        System.out.println("\n");

        // Message counts
        for (Map.Entry<String, Integer> entry : messageCountMap.entrySet()) {
            System.out.println(entry.getValue() + "\tthreads at " + entry.getKey());
        }
    }
}

(I may have gotten lazy here and neglected some unit test coverage- but with the report being written to console its fairly easy to eyeball bugs, right??)

With the Parser and report classes implemented, I can update the code in ParseJStack with a call to the parser and report:

...
        final JStackMeta stackMeta = new Parser(jstackFile).process();

        final Report report = new ConsoleReport();
        report.buildReport(stackMeta);
...

Finally

The end result of this analysis is being able to see an overview of the JStack results, which may give some indication of where to start looking for problems with the application.

39 threads at IN_NATIVE
503 threads at BLOCKED

1	threads at (BLOCKED)  - com.sun.appserv.util.cache.BaseCache.incrementMissCount() @bci=6, line=864 (Compiled frame; information may be imprecise)
3	threads at (BLOCKED)  - java.io.ByteArrayOutputStream.write(byte[], int, int) @bci=71, line=95 (Interpreted frame)
2	threads at (BLOCKED)  - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Compiled frame; information may be imprecise)
1	threads at (BLOCKED)  - java.lang.Object.hashCode() @bci=0 (Compiled frame; information may be imprecise)
4	threads at (BLOCKED)  - java.lang.Object.wait(long) @bci=-1749045981 (Interpreted frame)
195	threads at (BLOCKED)  - java.lang.Object.wait(long) @bci=-1749046076 (Interpreted frame)
    ...

As you can see we’re interested in seeing the counts for the end frames of the threads, and the state that they are in at this point.

This will at least give an opportunity to where the majority of the threads are ending up, and from this we might be able to gain some insight on what these threads are waiting on (e.g. by looking up stack traces via the byte code index in the original JStack log)

Next Steps

  • It may be of use to include a little bit more of the stacktrace in the comparison and output – I noticed that some taces would end on the frame with the same byte code index, but their full stacks would vary slightly.
  • As mentioned, it might be useful to write the report out to a file, such as a text or HTML file.

References

Building a simple project using behavior-driven development with JBehave

IMPORTANT!
This post has moved to http://blog.brasskazoo.com!

In a previous post introducing BDD I outlined a simple bus ticket application. Since then I’ve been looking at JBehave, and thought I’d try building a simple version of the bus tickets project, using JBehave to support behavior-driven development of the application.

About JBehave

JBehave is a testing framework that takes the BDD concepts and applies it to JUnit tests, so you can effectively test your acceptance criteria directly using automated unit tests.

It uses BDD acceptance criteria as steps in a test case, and each Given/When/Then statement is interpreted and executed as a method. The acceptance criteria is simply stored in a text file which is read by JBehave when the test case is run.

Setting up JBehave

Download the distribution jars from http://jbehave.org (at the time of writing, version 2.3.2 is the latest), and extract it to a convenient location!

Set up references to the jars in your project (you need to reference at least hamcrest, junit-dep and jbehave-core for this example to work).

New project: ‘bus-tix’

For the purposes of this example, I’m creating a project I’ll call ‘bus-tix’, with production and test code packages.

The scenarios

Lets take one of the scenarios from my previous post (slightly modified):

Scenario: Inserting initial coins

Given that the application is operating
When a coin is inserted
Then ensure the time indicator displays the purchased time

I will use this as the behavioural spec for the initial work on the application. From the description of the scenario, we can instantly tell that we’re going to need a coin handler and time indicator as part of the application.

Now before I put this into a text file, I must point out that the default configuration of JBehave requires that the scenario files are named to match the class names of the test case – so if we had the test case InsertingInitialCoins, then the scenario text file must be named inserting_initial_coins (with no extension). I don’t particularly like this, so as you’ll see I modify the constructor so that I can call my scenario ‘inserting_initial_coins.scenario‘, to make it clear the purpose of the file.

We’ll need to create a test case, which extends org.jbehave.scenario.Scenario, and a ‘step’ class which extends  org.jbehave.scenario.steps.Steps. The step class will contain all the corresponding steps for the scenario file(s).

Scenario: Inserting initial coins

Given that the application is operating
 When a coin is inserted
 Then ensure the time indicator displays the purchased time

Create the test case class InsertingInitialCoins

package com.brass.bustix;

import org.jbehave.scenario.PropertyBasedConfiguration;
import org.jbehave.scenario.Scenario;
import org.jbehave.scenario.parser.ClasspathScenarioDefiner;
import org.jbehave.scenario.parser.PatternScenarioParser;
import org.jbehave.scenario.parser.ScenarioDefiner;
import org.jbehave.scenario.parser.UnderscoredCamelCaseResolver;

public class InsertingInitialCoins extends Scenario {
    public InsertingInitialCoins() {
        this(Thread.currentThread().getContextClassLoader());
    }

    public InsertingInitialCoins(final ClassLoader classLoader) {
        super(new PropertyBasedConfiguration() {
            public ScenarioDefiner forDefiningScenarios() {
                return new ClasspathScenarioDefiner(
                    new UnderscoredCamelCaseResolver(".scenario"),
                    new PatternScenarioParser(this), classLoader);
                }
            }, new BustixSteps());
        }
    }

The default constructor calls the custom scenario file loader, so that we can append ‘.scenario’ to your scenario files. If I were creating several of these test scenarios, I would pull all that up to an abstract class to avoid re-writing those constructors.

Steps class (empty for now):

package com.brass.bustix;

import static org.jbehave.Ensure.ensureThat;
import org.jbehave.scenario.annotations.Given;
import org.jbehave.scenario.annotations.Then;
import org.jbehave.scenario.annotations.When;
import org.jbehave.scenario.steps.Steps;

public class BustixSteps extends Steps {
    // TODO – fill in steps
}

Make sure that the .scenario file is copied to your compile directory! Otherwise you will get the error like:

org.jbehave.scenario.errors.ScenarioNotFoundException: Scenario com/brass/bustix/inserting_initial_coins.scenario could not be found by classloader sun.misc.Launcher$AppClassLoader@133056f

When you run the test for the first time, you will see this output:

Scenario: Inserting initial coins
Given that the application is operating (PENDING)
When a coin is inserted (PENDING)
Then ensure the time indicator displays the purchased time (PENDING)

Perhaps confusingly, the test has passed despite the steps not being executed – PENDING means that the step hasn’t been executed because no corresponding method has been found.

Defining Steps

JBehave uses annotations to mark a method as a particular step. The annotation is the first word of the step we’re using, with the rest of the text as a parameter to that annotation. So for our first step ‘Given that the application is operating’ we reformat it as an @Given annotation:

@Given(“that the application is operating”)

And then JBehave knows that the function following that annotation is what needs to be executed for that step.

So now in BustixSteps we add:

@Given(“that the application is operating”)
public void startApplication() {
    ensureThat(false);
}

The assertion statement ensures that the test will fail – an empty test will pass (as it usually does with JUnit). The ‘ensureThat’ function comes from JBehave’s wrapping of hamcrest matchers – an interesting alternative to Junit’s assert statements. I won’t go into them right now.

Rerunning the test, we get the failure we want!

Scenario: Inserting initial coins
Given that the application is operating (FAILED)
 When a coin is inserted (PENDING)
 Then ensure the time indicator displays the purchased time (PENDING)

java.lang.AssertionError:
Expected: is <true>
got: <false>
...

Lets fill in the rest of the steps for this scenario while we’re here:

package com.brass.bustix;

import static org.jbehave.Ensure.ensureThat;
import org.jbehave.scenario.annotations.Given;
import org.jbehave.scenario.annotations.Then;
import org.jbehave.scenario.annotations.When;
import org.jbehave.scenario.steps.Steps;

public class BustixSteps extends Steps {

    @Given("that the application is operating")
        public void startApplication() {
        ensureThat(false);
    }

    @When("a coin is inserted")
    public void insertCoin() {
        ensureThat(false);
    }

    @Then("ensure the time indicator displays the purchased time")
    public void testThatTimeIsDisplayed() {
        ensureThat(false);
    }
}

At this stage, we’ve got our steps outlined for our basic scenario, and we can begin the test-driven  development of the application!

Note: when you have a scenario step that begins with an ‘And’, it is still considered as one of given/when/then, e.g.

Given that the application is operating
  And the time indicator displays nothing

..are both @Given annotations.

Next: check out the hamcrest matchers to make sure you’re getting the full potential from your asserts!

References

A brief introduction to Behaviour-Driven Development

IMPORTANT!
This blog has moved to http://blog.brasskazoo.com!

Behavior-Driven Development (BDD) is a methodology developed by agile developer Dan North in 2006.

It was created on top of an existing methodology named Test-Driven Development (TDD), a fairly widely known and discussed technique. Put simply, TDD specifies that simple test cases should be written first, and the developer then writes the smallest amount of code possible to make the unit tests pass.

Whilst TDD is firmly in the realm of the developer, BDD attempts to bridge the gap between developers, testers, business analysts, and other stakeholders; it is closer to technical specifications than to simply unit tested code.

By developing a consistent vocabulary across groups, we work towards eliminating miscommunication and ambiguity. And by putting the emphasis on behavior, we are more closely working with requirement specifications and the business value of the product’s function.

Stories

Like many agile processes, BDD employs the concept of ‘user stories’ in the form of a narrative, but with a specific format to them.

Story: [Title]

As a [role]
I want [feature]
So that [benefit]

This format clearly identifies the actor, the system feature and the business value or benefit of the story.

Each story also has acceptance criteria, in a format similar to this:

Scenario 1: [Title] 

Given [context]
  And [some more context]
  ...
 When [event]
 Then ensure [outcome]
 And ensure [another outcome]
 ...

The use of the word ensure identifies outcomes that are the responsibility of the scenario. Also using the word should in the scenario indicates the desired outcome could be affected by or reliant on another part of the system, and implicitly suggests a challenge to the assertion – “Should it? Should it really?”

Test Cases

Now we get to the core test-driven aspect – writing tests to cover each scenario. North encourages starting thinking of tests in a sentence starting with ‘should’ – as in “test (that the system) should [do something]” which converts to a test case titled ‘testShouldDoSomething’, making intention of each test clear, and we should be able to relate it to the acceptance criteria.

Now for an example…

In this simple example we have a ticket machine on a bus, where a trip duration is purchased by a passenger. The machine has a coin slot, an LCD to display the purchased time, a ‘print’ button, and a ticket printer (it does not have a coin return!).

Story: Purchasing a bus ticket

As a bus passenger
I want to purchase a bus ticket
So that I can board the bus

Acceptance Criteria

Scenario 1: Inserting initial coins
Given that the ticket machine is operating
 When coins are inserted
 Then ensure the purchased travel time is displayed on the LCD screen

Scenario 2: Inserting additional coins
Given that the ticket machine is operating
 And coins have been inserted
 When coins are inserted
 Then ensure the incremented travel time is displayed on the LCD screen 

Scenario 3: Printing the ticket
Given that the ticket machine is operating
 And coins have been inserted
 When the print button is pressed
 Then ensure ticket is printed with the purchased travel time
 And ensure that the LCD screen is cleared

We can see from the narrative the full extent of the story: The user, the system feature, and the value of the feature.

The acceptance criteria can then be transformed into unit tests:


// Scenario 1
public void testShouldSetTimeOnFirstCoinInsert() {...}

// Scenario 2
public void testShouldIncrementTimeOnCoinInsert() {...}

// Scenario 3
public void testShouldPrintTicketWithPurchasedTime() {...}
public void testShouldClearScreenAfterPrint() {...}
public void testShouldFailWhenPrintWithNoPurchase() {...}

Each test case verifies an outcome of the scenario, and for scenario 3 also verifies an error condition (i.e. when the print button is pressed and there has been no coins inserted).

With the test cases supporting the acceptance criteria, we’re affectingly applying test-driven development practices to business value, visible by stakeholders outside of the development team. Considering the difficulty many of us may have convincing non-technical people (and management) of the benefits of code quality strategies, to be able to actively engage them at higher levels with story narratives and plain English acceptance criteria is quite beneficial.

Consider these points

Summarised from Dan North’s blog

  • The story title should describe an activity
  • The story narrative should include a role, a feature and a benefit. – “As a [role] I want [feature] so that [benefit]
  • The scenario title should describe what’s different
  • The scenario should be expressed in terms of Givens, Events and Outcomes
  • The givens should define all of, and no more than, the required context
  • The event should describe the feature
  • The story should be small enough to fit in an iteration

References

[1] http://dannorth.net/introducing-bdd
[2] http://en.wikipedia.org/wiki/Test_Driven_Development
[3] http://dannorth.net/whats-in-a-story
http://behaviour-driven.org/Introduction
http://stackoverflow.com/questions/2509/what-are-the-primary-differences-between-tdd-and-bdd#2548
http://en.wikipedia.org/wiki/Behavior_Driven_Development