Tag Archives: programming

Monitoring OSCache statistics with JMX

Given an existing web application that uses OSCache, we can pretty easily add support for JMX monitoring.

A couple of things first:

  • We are using Sun’s Glassfish application server
  • The application is not yet spring-enabled
  • The project uses maven for dependancy management

Since we are using Glassfish, we already have an MBean server and connector, so we don’t have to worry about configuring those parts of the stack. (An MBean is a Managed JavaBean)

As for Spring support, we are restricting its introduction to a small aspect of the application (for now!), so we are minimizing the impact on the rest of the application.

What we need to do:

  • Modify oscache.properties and add an event listener
  • Add a dependancy on spring-web
  • Add a spring context for the JMX exporter
  • Modify web.xml to load the spring context

OSCache event listener

Firstly, we need to set a listener for OSCache that will publish the statistics we are interested in.

In oscache.properties, there is a section for event listeners. We want to implement a statistical listener:

cache.event.listeners=com.opensymphony.oscache.extra.StatisticListenerImpl

Spring dependancies

If you’re using maven, add a dependency on spring-web. This has dependancies on spring-core, spring-context and spring-beans (At the time of writing, 3.0.2.RELEASE was the latest version in the maven repositories).

Otherwise download these manually and chuck them in your WEB-INF/lib.

Spring Context

I made a separate spring context file (cacheContext.xml) for the cache MBean exporter:

    <beans xmlns="http://www.springframework.org/schema/beans"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">

        <!-- OSCache stats listener bean -->
        <bean id="statisticListener" class="com.opensymphony.oscache.extra.StatisticListenerImpl"/>

        <!-- Export the OSCache stats beans -->
        <bean id="exporter" class="org.springframework.jmx.export.MBeanExporter">
            <property name="beans">
                <map>
                    <entry key="oscache-bean:name=Statistics" value="statisticListener"/>
                </map>
            </property>
        </bean>
    </beans>

The map in the exporter links the statistics bean to a key that will be visible in JConsole.

Note: The OpenSymphony link in the references below gives you code to expose an arbitrary JMX connector and port. This is unnecessary since we’re accessing via the application server.

Adding the Spring context loader

The web.xml then needs to be updated to be told to load the new cacheContext.xml.

    <context-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>/WEB-INF/cacheContext.xml</param-value>
    </context-param>

	...

    <listener>
        <listener-class>
            org.springframework.web.context.ContextLoaderListener
        </listener-class>
    </listener>

JMX Clients

JDK 1.5 includes two JMX clients that you can use.

jconsole is the legacy monitoring application, which has been superseded by the fancy UI of jvisualvm.

Either application will allow you to monitor the statistics bean, however in jvisualvm you will need to install the plugin VisualVM-MBeans under Tools > Plugins. Meanwhile jconsole has an MBeans tab by default.

When your application is running, use one of these utilities to connect to the application server (either directly or via localhost:8686 for glassfish). Under the oscache-bean > Statistics, you should be able to see counts generated by cache attributes. Double-clicking the numbers will pop up a graph, and right-clicking will allow you to export the recorded values.

Graph of OSCache statistics

References

  1. http://www.opensymphony.com/oscache/wiki/JMX%20Monitoring.html
  2. http://java.sun.com/docs/books/tutorial/jmx/mbeans/standard.html
  3. http://java.sun.com/developer/technicalArticles/J2SE/jmx.html
Advertisements

Loading custom-named BDD scenario files for JBehave

Now that I’m using JBehave in a commercial project, I’ve rewritten the loading of the scenario files in such a way that I can call my tests something like com.example.login.InvalidLoginScenario and have the corresponding scenario file under {project}/src/test/resources/invalid_login.scenario.

Previously…

The standard JBehave scenario file is loaded with UnderscoredCamelCaseResolver, which converts the classname from camel-case to underscore-seperated classname. A resource path is constructed from the package plus the underscored filename to locate the file – e.g. {src.test}/com/example/login/invalid_login_scenario.

Previously (with inspiration), I modified the testcase to override the default Configuration object, which allowed the loading of the scenario file with an extension – so it would now look for {src.test}/com/example/login/invalid_login_scenario.scenario.

Goal

To make the creation and maintenance of the JBehave scenarios and testcases easier, I decided on some standards:

  • JBehave testcase classes should be suffixed with ..Scenario, to clearly communicate their purpose.
  • Scenario filenames should map to their corresponding test classes.
  • Scenario files should reside in the same location under resources/.
  • Scenario files should have the extension .scenario, to improve readability.

But instead of having a file named invalid_login_scenario.scenario, I want the test class InvalidLoginScenario to map to the file invalid_login.scenario. All this was basically possible with existing JBehave classes, when configured the correct way (and certain functions overridden).

The Source

import org.jbehave.scenario.PropertyBasedConfiguration;
import org.jbehave.scenario.RunnableScenario;
import org.jbehave.scenario.errors.PendingErrorStrategy;
import org.jbehave.scenario.parser.ClasspathScenarioDefiner;
import org.jbehave.scenario.parser.PatternScenarioParser;
import org.jbehave.scenario.parser.ScenarioDefiner;
import org.jbehave.scenario.parser.UnderscoredCamelCaseResolver;

/**
 * Customisation of standard JBehave {@link PropertyBasedConfiguration} to allow clearer naming of scenario files:
 * <ul>
 * <li> Usage of *.scenario file extension</li>
 * <li> Strip 'Scenario' off test class names</li>
 * <li> Load scenario files from classpath root</li>
 * </ul>
 * So a test class named <code>InvalidUsernameScenario</code> would be attempting to resolve the resource path <code>/invalid_username.scenario</code>.
 *
 * This configuration also fails on 'pending' (unimplemented) steps.
 */
public final class ScenarioConfiguration extends PropertyBasedConfiguration {
    private final ClassLoader _classLoader;

    public ScenarioConfiguration(final ClassLoader classLoader) {
        _classLoader = classLoader;
    }

    @Override
    public ScenarioDefiner forDefiningScenarios() {
        final ResourceNameResolver filenameResolver = new ResourceNameResolver(".scenario");
        filenameResolver.removeFromClassname("Scenario");
        return new ClasspathScenarioDefiner(filenameResolver, new PatternScenarioParser(this.keywords()), _classLoader);
    }

    @Override
    public PendingErrorStrategy forPendingSteps() {
        return PendingErrorStrategy.FAILING;
    }

    /**
     * Override {@link UnderscoredCamelCaseResolver} to load resources from classpath root. This means we can collect
     * scenario files in a single resource directory instead of in packages.
     */
    class ResourceNameResolver extends UnderscoredCamelCaseResolver {
        public ResourceNameResolver(final String extension) {
            super(extension);
        }

        @java.lang.Override
        protected String resolveDirectoryName(final Class<? extends RunnableScenario> scenarioClass) {
            return "";
        }
    }
}

The function forDefiningScenarios() is the important part – it sets the resolver to use the .scenario extension, but also strips out ‘Scenario’ from the class name.

Also, to force the resolver to look at the classpath root, the resolveDirectoryName() function is overridden to return an empty string.

The testcases using this Configuration object would call:

public class InvalidLoginScenario extends Scenario {
    public InvalidLoginScenario() {
        super(new ScenarioConfiguration(InvalidLoginScenario.class.getClassLoader()), new LoginScenarioSteps());
    }
}

References

Building a JStack Parser

Background

Recently, I was briefed on a situation where a main public facing java web application was going down fairly regularly, for what appeared to be an out of memory error.

After an investigation by a previous developer turned up no concrete leads, they suggested using JStack to get a profile of what the threads in the application were holding locks that could be causing the crash.

Now I’ve never worked with JStack before, but the purpose seems fairly straightforward. Trouble is, with ~500 threads in the output, analysing each one to find a common cause would be a pretty intensive task.

I set about searching for some kind of output analyser for JStack, but with no luck, I decided to write my own to give me a summary of the results.

Requirements

At least for my application, there were a lot of similar-looking stacktraces, so it made sense to group and summarise the results.

The JStack output I was provided with looked something like this:

Attaching to process ID 18526, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 1.5.0_14-b03
Thread t@6872: (state = BLOCKED)
- java.lang.Thread.sleep(long) @bci=-1766132960 (Interpreted frame)
- java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
- sun.net.www.http.KeepAliveCache.run() @bci=3, line=149 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)
Thread t@6871: (state = BLOCKED)
- java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise)
- java.util.TimerThread.mainLoop() @bci=201, line=509 (Compiled frame)
- java.util.TimerThread.run() @bci=1, line=462 (Interpreted frame)

What I initially wanted from my parser was to:

  • See the total number of thread states
  • Summary information for each stacktrace.

So what I need to do is extract each thread and its corresponding stacktrace to be able to process and report on it.

The Code

Firstly the main checks for a file parameter, then passes to the application instance which will check for its existence before calling a parser.

public class ParseJStack {
    public ParseJStack(final String filename) {
        File jstackFile = new File(filename);
        if (!jstackFile.exists()) {
            System.out.println("File does not exist. Exiting.");
            return;
        }
        //new Parser(jstackFile).process();
    }

    public static void main(String[] args) {
        if (args.length < 1) {
            System.out.println("Program requires a filename");
            System.exit(1);
        } else {
            new ParseJStack(args[0]);
        }
    }
}

For now, the call to the parser is commented out, as I’m yet to create it or determine its return type.

Test-driven

Now that I’m thinking about main functionality of the application, I’ll write testcases (JUnit 4) around the container classes JStackMeta and JStackEntry which are the thread detail and stacktrace objects respectively, and will be populated by the parser engine

JStackEntry will be a fairly simple container for a List of stack calls, and a String for its header text. From the looks of it the thread headers all follow the same pattern, so I will expect to be able to extract the state (and the id if we need to later) with a simple regex pattern.

public class JStackEntryTest {
    @Test
    public void shouldGetContentsAsNotNullWhenNullParameter() throws Exception {
        final JStackEntry stackEntry = new JStackEntry("");
        Assert.assertNotNull(stackEntry.getContents());
    }

    @Test
    public void shouldGetContents() throws Exception {
        final JStackEntry stackEntry = new JStackEntry("");
        stackEntry.append("Test Content");

        Assert.assertEquals(12, stackEntry.getContents().length());
        Assert.assertEquals("Test Content", stackEntry.getContents().toString());
    }

    @Test
    public void shouldGetEntryState() throws Exception {
        final JStackEntry stackEntry = new JStackEntry("Thread t@6872: (state = BLOCKED)");
        stackEntry.append("Test Content");

        Assert.assertEquals("BLOCKED", stackEntry.getState());
    }

    @Test
    public void shouldGetUnknownStateWhenHeaderEmpty() throws Exception {
        final JStackEntry stackEntry = new JStackEntry("");
        stackEntry.append("Test Content");

        Assert.assertEquals("UNKNOWN", stackEntry.getState());
    }
}

JStackMeta will hold the collection of JStackEntry objects, and the non-entry related meta (e.g. process id, java version info that appears at the top of the file).

public class JStackMetaTest {
    @Test
    public void testConstructorInit() throws Exception {
        Assert.assertNotNull(new JStackMeta().getHeader());
        Assert.assertNotNull(new JStackMeta().getEntries());
    }

    @Test
    public void testAppend() throws Exception {
        final JStackMeta stackMeta = new JStackMeta();
        stackMeta.append("Test Meta");
        Assert.assertEquals(9, stackMeta.getHeader().length());
    }

    @Test
    public void testAddEntry() throws Exception {
        final JStackMeta stackMeta = new JStackMeta();
        final JStackEntry stackEntry = new JStackEntry("");
        stackEntry.append("Test Entry");
        stackMeta.addEntry(stackEntry);
        Assert.assertEquals(1, stackMeta.getEntries().size());
    }
}

With those two classes set, we move back to the JStackParser which populates them. It’ll take the File handle provided by the application object, read in the contents and store them a JStackMeta and its JStackEntry objects. To test this class I created a test.jstack file (which is the same content as the example at the top of this post).

public class ParserTest {
    @Test
    public void testProcess() throws Exception {
        final URL resource = getClass().getResource("test.jstack");
        final File file = new File(resource.getFile());

        final JStackMeta stackMeta = new Parser(file).process();

        Assert.assertEquals(132, stackMeta.getHeader().length());
        Assert.assertEquals(2, stackMeta.getEntries().size());
    }
}

The actual parsing function looks like this (note that I’ve decided that it is important to keep the newlines in the stack trace to simplify the output):

    /**
     * Process the JStack output file and extract the data into a {@link JStackMeta} object.
     *
     * @return The {@link JStackMeta} object representing the JStack output.
     */
    public JStackMeta process() {
        _meta = new JStackMeta();
        try {
            FileInputStream inputStream;
            inputStream = new FileInputStream(_file);

            BufferedReader in = new BufferedReader(new InputStreamReader(inputStream));

            boolean finishedHeader = false;
            JStackEntry currentEntry = new JStackEntry("");

            String line;
            while ((line = in.readLine()) != null)
            {
                // Skip blanks
                if ("".equals(line.trim())) {
                    continue;
                }
                line += "\n";

                // Check if we're done with the header lines
                if (!finishedHeader && line.startsWith("Thread")) {
                    finishedHeader = true;
                }

                if (!finishedHeader) {
                    _meta.append(line);
                    continue;
                }

                if (line.startsWith("Thread")) {
                    currentEntry = new JStackEntry(line);
                    _meta.addEntry(currentEntry);
                } else {
                    currentEntry.append(line);
                }
            }

            in.close();
        } catch (FileNotFoundException e) {
            System.out.println("ERROR: File was not found");
        } catch (IOException e) {
            System.out.println("ERROR: A problem occurred");
            e.printStackTrace();
        }

        return _meta;
    }

Reporting

The last part of the application is a simple analysis of the JStackMeta object and reporting the statistics. As per the requirements, I want two parts to this, the totals for the different states and the single-line summary with counts.

For the purposes of a quick solution, I’m writing the output to the console. I’ll make a Report interface, just in case I get inspired and write some other implementation to report to a text or HTML file..

I’ll use two maps to keep track of the totals – one map for the status counts and one for the line summary totals. To keep the second part looking organised, I’ll use a TreeMap so that the entries remain sorted.

public interface Report {
    void buildReport(JStackMeta stackMeta);
}
public class ConsoleReport implements Report {
    public void buildReport(final JStackMeta stackMeta) {
        Map<String, Integer> stateCountMap = new HashMap<String, Integer>();
        Map<String, Integer> messageCountMap = new TreeMap<String, Integer>();  // Sorted

        // Report on results
        for (int i = 0; i < stackMeta.getEntries().size(); i++) {
            JStackEntry entry = stackMeta.getEntries().get(i);
            final String state = entry.getState();

            final Integer count;
            if (stateCountMap.containsKey(state)) {
                count = stateCountMap.get(state) + 1;
            } else {
                count = 1;
            }
            stateCountMap.put(state, count);

            final StringBuilder contents = entry.getContents();
            final String strStackEnd;
            if (contents.length() != 0) {
                strStackEnd = "(" + state + ") " + contents.substring(0, contents.indexOf("\n"));
            } else {
                strStackEnd = "(" + state + ") " + "[No stacktrace]";
            }

            final Integer countMessage;
            if (messageCountMap.containsKey(strStackEnd)) {
                countMessage = messageCountMap.get(strStackEnd) + 1;
            } else {
                countMessage = 1;
            }
            messageCountMap.put(strStackEnd, countMessage);
        }

        // State counts
        for (Map.Entry<String, Integer> entry : stateCountMap.entrySet()) {
            System.out.println(entry.getValue() + " threads at " + entry.getKey());
        }

        System.out.println("\n");

        // Message counts
        for (Map.Entry<String, Integer> entry : messageCountMap.entrySet()) {
            System.out.println(entry.getValue() + "\tthreads at " + entry.getKey());
        }
    }
}

(I may have gotten lazy here and neglected some unit test coverage- but with the report being written to console its fairly easy to eyeball bugs, right??)

With the Parser and report classes implemented, I can update the code in ParseJStack with a call to the parser and report:

...
        final JStackMeta stackMeta = new Parser(jstackFile).process();

        final Report report = new ConsoleReport();
        report.buildReport(stackMeta);
...

Finally

The end result of this analysis is being able to see an overview of the JStack results, which may give some indication of where to start looking for problems with the application.

39 threads at IN_NATIVE
503 threads at BLOCKED

1	threads at (BLOCKED)  - com.sun.appserv.util.cache.BaseCache.incrementMissCount() @bci=6, line=864 (Compiled frame; information may be imprecise)
3	threads at (BLOCKED)  - java.io.ByteArrayOutputStream.write(byte[], int, int) @bci=71, line=95 (Interpreted frame)
2	threads at (BLOCKED)  - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Compiled frame; information may be imprecise)
1	threads at (BLOCKED)  - java.lang.Object.hashCode() @bci=0 (Compiled frame; information may be imprecise)
4	threads at (BLOCKED)  - java.lang.Object.wait(long) @bci=-1749045981 (Interpreted frame)
195	threads at (BLOCKED)  - java.lang.Object.wait(long) @bci=-1749046076 (Interpreted frame)
    ...

As you can see we’re interested in seeing the counts for the end frames of the threads, and the state that they are in at this point.

This will at least give an opportunity to where the majority of the threads are ending up, and from this we might be able to gain some insight on what these threads are waiting on (e.g. by looking up stack traces via the byte code index in the original JStack log)

Next Steps

  • It may be of use to include a little bit more of the stacktrace in the comparison and output – I noticed that some taces would end on the frame with the same byte code index, but their full stacks would vary slightly.
  • As mentioned, it might be useful to write the report out to a file, such as a text or HTML file.

References

How to load properties files into Spring and expose to the Java classes

IMPORTANT!
This blog has moved to http://blog.brasskazoo.com!

Loading properties files in the spring configuration

The issue:

Our webapp is using a spring and hibernate backend, and both spring configuration and Java classes need to access database connection properties.

Currently there is an application.properties file and a spring applicationContext.xml, both which contain the same information for the database connection.

Ideally, there should be only one configuration file to define the properties, which is used by both spring and Java.

Spring has the ability to load properties files as part of the application configuration, for use internally. The properties can be referenced within spring configuration files using ant-style placeholders, e.g. ${app.var}.

To solve our issue though we’ll also need to provide a populated bean that can be used by our production classes.

Here’s the properties file (application.properties):

appl.name=My Web Application
appl.home=/Users/webapp/application/

# Database properties
db.driver=org.hsqldb.jdbcDriver
db.name=v8max.db
db.url=jdbc:hsqldb:file://${appl.home}/database/${db.name}
db.user=SA
db.pass=

Spring loads the properties file using a bean of the type org.springframework.beans.factory.config.PropertyPlaceholderConfigurer, and then we can reference the properties directly in the spring configuration file.

Note that we can also use placeholders within the properties file, as in the db.url above – the PropertyPlaceholderConfigurer will resolve them too!

    <!-- Load in application properties reference -->
    <bean id="applicationProperties" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="location" value="classpath:application.properties"/>
    </bean>

    <bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource">
        <property name="driverClassName" value="${db.driver}"/>
        <property name="url" value="${db.url}"/>
        <property name="username" value="${db.user}"/>
        <property name="password" value="${db.pass}"/>
    </bean>

So now we’ve removed the need to have literals in the spring config!

Exposing the spring properties bean in java

To allow our Java classes to access the properties from the same object as spring, we’ll need to extend the PropertyPlaceholderConfigurer so that we can provide a more convenient method for retrieving the properties (there is no direct method of retrieving properties!).

We can extend the spring provided class to allow us to reuse spring’s property resolver in our Java classes:

public class PropertiesUtil extends PropertyPlaceholderConfigurer {
   private static Map propertiesMap;

   @Override
   protected void processProperties(ConfigurableListableBeanFactory beanFactory,
             Properties props) throws BeansException {
        super.processProperties(beanFactory, props);

        propertiesMap = new HashMap<String, String>();
        for (Object key : props.keySet()) {
            String keyStr = key.toString();
            propertiesMap.put(keyStr, parseStringValue(props.getProperty(keyStr),
                props, new HashSet()));
        }
    }

    public static String getProperty(String name) {
        return propertiesMap.get(name);
    }
}

If we now update the applicationProperties bean to use the PropertiesUtil class, we can use the static getProperty method to access the resolved properties via the same object as the spring configuration bean.

Of course, we could run into problems if a class tries to use PropertiesUtil before the spring context has been initialised. For example if you’re registering ServletContextListeners in your web.xml before configuring spring, you’ll get a NullPointerException if one of those classes tries to use PropertiesUtil. For that reason I’ve had to declare the spring context before any context listeners.

    <context-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>
            /WEB-INF/applicationContext.xml
        </param-value>
    </context-param>

    <listener>
        <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
    </listener>

    <!-- Other listeners -->

References

  1. http://www.jdocs.com/spring/1.2.8/org/springframework/beans/factory/config/PropertyPlaceholderConfigurer.html
  2. http://j2eecookbook.blogspot.com/2007/07/accessing-properties-loaded-via-spring.html

A cyclical dependancy anti-pattern: Identity Theft

IMPORTANT!
This blog has moved to http://blog.brasskazoo.com!

Anti-patterns:

In software engineering, an anti-pattern (or antipattern) is a design pattern that appears obvious but is ineffective or far from optimal in practice.[1]

Or as I like to think of it, the type of code that makes you want to rip your eyes out, like in that movie Event Horizon

Identity Theft

Here’s a pattern that I have seen many times in a previous codebase, which I believe suffers from some design flaws:

FooListing is a class that retrieves and maintains a list of Foo objects.

Description

FooListing is a repository-type object that holds a list of Foo objects, which it populates from the database using its populate() function.

The Foo object also has a populate() method, but this method instantiates a FooListing, and uses the FooListing populate function with its key – so now the FooListing contains the desired Foo object.

Our original Foo object now contains a reference to the Foo object that it wants to be.  So it does what anyone would do, steals its identity and hides the body.

Foo.populate() looks something like this:

public void populate() {
  FooListing listing = new FooListing()
  // Set up FooListing
  FooListing.setSearchKey(id);
  if (listing.size > 0) {
      Foo foo = listing.get(0);
      this.setBar(foo.getBar);
      // continue to copy over properties
      // ...
  } else {
      LOG.warn("Foo not found for key: " + id);
  }
}

Why this is bad:

Firstly, apart from the questional morality, it is duplicated code. FooListing already has the capability to create and populate a Foo object from the database. Two locations for this code means twice the maintenance if something changes, (more than) twice the possibility of bugs, etc.

Foo and FooListing have become tightly coupled and dependant on each other under this design – there is a cyclic dependancy which is a code smell, and may cause headaches when writing unit tests.

There is also a waste of resoures creating the uneccesary FooListing and Foo objects inside of Foo.populate(), at least some of that could be avoided by client code accessing instances of Foo via FooListing.populate

It also doesn’t make sense that a data access object like Foo should be concerned with information about how it is created. Foo should act more like a bean, or an immutable class.

Another dangerous aspect of this design is the implication that client code accessing Foo cannot be certain that Foo has been instantiated correctly. If the client code has a faulty key for Foo that does not exist in the database, when it creates new Foo(key) there is no way to know that Foo.populate() has failed to find the correct value, and instead they are left with a faulty Foo instance which was not what they requested.

Solution:

The best solution for this isolated pattern is to completely remove (or deprecate) the Foo.populate() method, and replace calls to it with FooListing instances.

If FooListing fails to find a matching Foo, the client code should realise this when FooListing returns them a null object. The client code can handle and recover from this case in context.

Implementing a getFirstResult() function in FooListing could be beneficial if there are many cases where the code with otherwise be calling get(0).

We could also simplify the calling code so that retrieving a result is a one-line operation – i.e. get() calls populate() if the list has not already been populated.

public final class FooListing {
  private List<Foo> _listing;
  private int _id;

  public FooListing {
  }

  public FooListing(int id) {
    _searchId = id;
  }

  public void populate() {
    _listing = new ArrayList<Foo>();
    // Query the database and add to _listing
  }

  public Foo get(int index) {
    if (_listing == null) {
      populate();
    }
    if (_listing.isEmpty() || index >= _listing.size()) {
      return null;
    } else {
      return _listing.get(index);
    }
  }

  public Foo getFirstResult() {
    return this.get(0);
  }

  public List<Foo> getList() {
    return _listing;
  }

  public void setSearchId(int id) {
    _id = id;
  }
}

And the client code would only need to call

Foo foo = new FooListing(id).getFirstResult();

References:

1. http://en.wikipedia.org/wiki/Anti-patterns

A simple intro to git on Windows

I’ve heard a fair amout about git in the last year, various people singing its praises for long enough that I figure its worth checking out for use with my projects.

Some history

Git was developed by Linus Tovalds in 2005, as the source control for Linux kernel development. He was inspired by the workflow provided by another proprietry tool called BitKeeper, which up until that point had been free for the Linux developers to use.

Why use it?

Basically my understanding is that the major benefits of git are (by no means an exhaustive list):

  • It is a distributed VCS, so developers can push and pull changes from each other. There is no concept of a central repository, except by convention (i.e. a team decides there is one).
  • It facilitates managing local branches, housing local repositories and working copies, so that developers can perform ‘offline’ commits to their own repositories, and update remote repositories when it is convinient.
  • Merging is done intellegently, by delving into the history of both working copy and merge target, and applying each change to the state the target was in at the time the change was made.
  • It is very quick to branch and switch between branches

Two opportunities arose recently which has given me reason to try it out:

1) A presentation by a collegue on his use of git-svn (albeit on a mac, when most developers are using Windows)

The presentation was more like an info session, not a suggestion that we should all be using it too. It was interesting to see how well it worked for his workflow though, and actually seeing it demonstrated was much more productive than another less experienced co-worker who constantly complains that git is the solution to our process problems.

2) A side project, collaboration with a friend on a project.

Git is being used as the SCM with ssh access to a remote repo, maven builds on a TeamCity build server.

git on Windows

So my home machine at the moment is a Windows box (mostly because of digital audio applications I use), I investigated my options with a Windows git client.

  • git via cygwin
    • The official git implementation for Windows, via cygwin’s command-line interface.
    • Apparently is noticibly slower than native implementations on unix, due to cygwin’s fork implementation
    • Some are concerned about the disk and CPU overhead of running the emulator on their system.
  • mSysGit
    • The official alternative to cygwin, uses its own bash shell and/or git-gui
  • Tortise Git
    • If this is anything like Tortoise SVN, this would be perfect for a development environment. Might be good to take this one back to the folks at work (they’re all used Tortoise SVN)
  • Git#
    • A .NET native implementation for Windows, GUI
  • IDE based – IntelliJ 8 and Eclipse both have support via plugins
    • Probably the easiest since I’m using IntelliJ, but I like having an alternative to opening up an IDE to do some version control maintenance if neccesary.

I prefer a command line, so I’ve stuck with what I know and installed cygwin with the packages git, gitk, vim and openssh which should be enough to get started.

Basic workflow with a remote repository

Basic git workflow

Basic git workflow

A basic workflow using a remote repository as your starting point looks something like this. A remote repository may exist outside your computer, somewhere over the internet perhaps… On your machine however, you have 3 components:

  • Local repository – The clean copy of all files in the project.
  • Index (or staging area) – A holding area for the changes you want to send to your repository.
  • Working copy

Git manages the local repo and index in hidden files in the project directory, so all you see is the files you’re working on – the working copy.

Commands

Create your local project by cloning the remote repository (I’m using ssh):

git clone ssh://user@server/path/to/project

This will create the local ‘clean’ repo, index, working copy, and initialise the various git files.

After creating or modifying a file, add it to the staging area so that it is available to commit:

git add filename.ext

A commit will commit from your index to your local repo. A push will then push those changes to the remote repository:

git commit -m 'Some message'
git push

Note: A shortcut is to use the ‘-a’ flag on commit, it will automatically stage and commit all modified files so you can skip the ‘add’ command.

Alternatively, to rollback a change you made (this will revert to last available version from the staging area):

git checkout filename.ext

To update from the remote repository:

git pull

There you have it, seems pretty straight forward to get started. And those with experience with other VCS/SCM software should have no problem adapting.

References

Check some of these links out for more info:

http://git-scm.com/documentation
http://en.wikipedia.org/wiki/Git_(software)
http://stackoverflow.com/questions/783906/git-under-windows-msys-or-cygwin
http://www.cforcoding.com/2009/09/windows-git-tutorial-cygwin-ssh-and.html

A brief introduction to Behaviour-Driven Development

IMPORTANT!
This blog has moved to http://blog.brasskazoo.com!

Behavior-Driven Development (BDD) is a methodology developed by agile developer Dan North in 2006.

It was created on top of an existing methodology named Test-Driven Development (TDD), a fairly widely known and discussed technique. Put simply, TDD specifies that simple test cases should be written first, and the developer then writes the smallest amount of code possible to make the unit tests pass.

Whilst TDD is firmly in the realm of the developer, BDD attempts to bridge the gap between developers, testers, business analysts, and other stakeholders; it is closer to technical specifications than to simply unit tested code.

By developing a consistent vocabulary across groups, we work towards eliminating miscommunication and ambiguity. And by putting the emphasis on behavior, we are more closely working with requirement specifications and the business value of the product’s function.

Stories

Like many agile processes, BDD employs the concept of ‘user stories’ in the form of a narrative, but with a specific format to them.

Story: [Title]

As a [role]
I want [feature]
So that [benefit]

This format clearly identifies the actor, the system feature and the business value or benefit of the story.

Each story also has acceptance criteria, in a format similar to this:

Scenario 1: [Title] 

Given [context]
  And [some more context]
  ...
 When [event]
 Then ensure [outcome]
 And ensure [another outcome]
 ...

The use of the word ensure identifies outcomes that are the responsibility of the scenario. Also using the word should in the scenario indicates the desired outcome could be affected by or reliant on another part of the system, and implicitly suggests a challenge to the assertion – “Should it? Should it really?”

Test Cases

Now we get to the core test-driven aspect – writing tests to cover each scenario. North encourages starting thinking of tests in a sentence starting with ‘should’ – as in “test (that the system) should [do something]” which converts to a test case titled ‘testShouldDoSomething’, making intention of each test clear, and we should be able to relate it to the acceptance criteria.

Now for an example…

In this simple example we have a ticket machine on a bus, where a trip duration is purchased by a passenger. The machine has a coin slot, an LCD to display the purchased time, a ‘print’ button, and a ticket printer (it does not have a coin return!).

Story: Purchasing a bus ticket

As a bus passenger
I want to purchase a bus ticket
So that I can board the bus

Acceptance Criteria

Scenario 1: Inserting initial coins
Given that the ticket machine is operating
 When coins are inserted
 Then ensure the purchased travel time is displayed on the LCD screen

Scenario 2: Inserting additional coins
Given that the ticket machine is operating
 And coins have been inserted
 When coins are inserted
 Then ensure the incremented travel time is displayed on the LCD screen 

Scenario 3: Printing the ticket
Given that the ticket machine is operating
 And coins have been inserted
 When the print button is pressed
 Then ensure ticket is printed with the purchased travel time
 And ensure that the LCD screen is cleared

We can see from the narrative the full extent of the story: The user, the system feature, and the value of the feature.

The acceptance criteria can then be transformed into unit tests:


// Scenario 1
public void testShouldSetTimeOnFirstCoinInsert() {...}

// Scenario 2
public void testShouldIncrementTimeOnCoinInsert() {...}

// Scenario 3
public void testShouldPrintTicketWithPurchasedTime() {...}
public void testShouldClearScreenAfterPrint() {...}
public void testShouldFailWhenPrintWithNoPurchase() {...}

Each test case verifies an outcome of the scenario, and for scenario 3 also verifies an error condition (i.e. when the print button is pressed and there has been no coins inserted).

With the test cases supporting the acceptance criteria, we’re affectingly applying test-driven development practices to business value, visible by stakeholders outside of the development team. Considering the difficulty many of us may have convincing non-technical people (and management) of the benefits of code quality strategies, to be able to actively engage them at higher levels with story narratives and plain English acceptance criteria is quite beneficial.

Consider these points

Summarised from Dan North’s blog

  • The story title should describe an activity
  • The story narrative should include a role, a feature and a benefit. – “As a [role] I want [feature] so that [benefit]
  • The scenario title should describe what’s different
  • The scenario should be expressed in terms of Givens, Events and Outcomes
  • The givens should define all of, and no more than, the required context
  • The event should describe the feature
  • The story should be small enough to fit in an iteration

References

[1] http://dannorth.net/introducing-bdd
[2] http://en.wikipedia.org/wiki/Test_Driven_Development
[3] http://dannorth.net/whats-in-a-story
http://behaviour-driven.org/Introduction
http://stackoverflow.com/questions/2509/what-are-the-primary-differences-between-tdd-and-bdd#2548
http://en.wikipedia.org/wiki/Behavior_Driven_Development