Continuous Deployment: Deploying to Glassfish with Maven and TeamCity

IMPORTANT!
This blog has moved to http://blog.brasskazoo.com!

A later stage of the continuous integration process – continuous deployment. The sooner we can deploy a tested and verified piece of software, the better!

Here I’m describing an automated deployment process that uses Maven to deploy to a Glassfish application server. TeamCity facilitates the build and test stages, with an additional deployment of the packaged web application. Using the glassfish plugin for maven 2, we can integrate the application server deployment into the continuous integration cycle, and provide a constantly up-to-date development/test environment.

Glassfish

I’ll create a new domain from scratch for the maven apps, using the default port values (i.e. admin port 4848, http port 8080), but setting the admin password and master password.

In setting up the glassfish domain we generate a password file so that we are not storing any passwords in plain text – such as in the pom or settings.xml

cd ${glassfish.home}/bin
./asadmin create-domain --savemasterpassword=true my-apps

the –savemasterpassword switch generates an encrypted ‘master-password’ file in the domains/my-apps directory.

Maven

Maven profiles makes it easy to have machine-specific variables so that moving to other platforms in the future is straightforward.

On the on the host machine I’ve created the file ~/.m2/settings.xml.

<?xml version="1.0" encoding="UTF-8"?>
xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <profiles>
    <profile>
      <id>glassfish-context</id>
      <properties>
        <local.glassfish.home>/Users/brass/bin/glassfishv3</local.glassfish.home>
        <local.glassfish.user>admin</local.glassfish.user>
        <local.glassfish.domain>my-apps</local.glassfish.domain>
        <local.glassfish.passfile>
${local.glassfish.home}/glassfish/domains/${local.glassfish.domain}/master-password
        </local.glassfish.passfile>
      </properties>
    </profile>
  </profiles>

  <activeProfiles>
    <activeProfile>glassfish-context</activeProfile>
  </activeProfiles>
</settings>

This gives us the parameters for the glassfish instance that we will use in our pom.

Update: I found that while the above works for v3.1, my dev machine’s glassfish v3.0.1 needed to reference glassfish home one directory deeper:

<local.glassfish.home>/Users/brass/bin/glassfishv3/glassfish
</local.glassfish.home>
...
<local.glassfish.passfile>
${local.glassfish.home}/domains/${local.glassfish.domain}/master-password
</local.glassfish.passfile>

In the project pom.xml, we define a profile for glassfish deployment:

<profile>
  <id>glassfish-deploy</id>
  <pluginRepositories>
    <pluginRepository>
      <id>maven.java.net</id>
        <name>Java.net Maven2 Repository</name>
        <url>http://download.java.net/maven/2</url>
      </pluginRepository>
    </pluginRepositories>
    <build>
      <plugins>
      <plugin>
        <groupId>org.glassfish.maven.plugin</groupId>
        <artifactId>maven-glassfish-plugin</artifactId>
        <version>2.1</version>
        <configuration>
          <glassfishDirectory>${local.glassfish.home}</glassfishDirectory>
          <user>${local.glassfish.user}</user>
          <passwordFile>${local.glassfish.passfile}</passwordFile>
          <autoCreate>true</autoCreate>
          <debug>true</debug>
          <echo>false</echo>
          <terse>true</terse>
          <domain>
            <name>${local.glassfish.domain}</name>
            <adminPort>4848</adminPort>
            <httpPort>8080</httpPort>
            <httpsPort>8443</httpsPort>
            <iiopPort>3700</iiopPort>
            <jmsPort>7676</jmsPort>
            <reuse>false</reuse>
          </domain>
          <components>
            <component>
              <name>${project.artifactId}</name>
              <artifact>
${project.build.directory}/${project.build.finalName}.war
              </artifact>
            </component>
          </components>
        </configuration>
      </plugin>
    </plugins>
  </build>
</profile>

The repository for the glassfish plugin repository is specified within the profile since its not part of the larger project in this case.

As you see the variables from the local settings.xml are used for the glassfish config.

TeamCity

The TeamCity setup needs to include two things in the maven2 runner config:

• Glassfish goals
• Profile parameters

The glassfish goals that are used should be able to start the domain if it isn’t running, and replace the application.

The ‘redeploy’ goal would allow a hot-swap deployment, if for example we were running other applications on the domain.

See the plugin page (http://maven-glassfish-plugin.java.net/) for more info.

Next

So now this process provides us with continuous deployment – a commit will be built, tested and deployed automatically, allowing changes to the software to be seen and used almost immediately!

Unit Testing XML – Evaluating Diffs

I am trying to test code that merges two XML files. In the unit test that I am attempting to implement, I want to compare the difference between the merge result and one of the XML files (the larger of the two).

This is a description of the file contents:

• The right-hand XML file has 13 elements underneath the root, while the left-hand file has 4.
• Two elements in both files are equivalent, so the left-hand version is discarded.

What I’m expecting is that the two remaining elements in the left hand file are merged into the resultant XML, so that in reference to the right-hand file, the merged content has two additional elements underneath the root.

XMLUnit

I’ve used XMLUnit previously to compare generated XML with expected output. In this case however, I am more concerned with evaluating the differences between the source and resultant XML.

XMLUnit has a Diff object – org.custommonkey.xmlunit.Diff – which I realised, after some investigation, doesn’t quite offer a diff in the traditional UNIX diff/patch command sense. It evaluates a document in terms of being identical, similar or different, and holds a message describing the first difference encountered.

This has some use of course, my testcase could look something like this:

...
Diff myDiff = new Diff(originalXml, mergedXml);
assertFalse("Expected differences in XML", myDiff.identical());

The message contained in myDiff here is:

[different] Expected number of child nodes '13' but was '17' - comparing  at /Group[1] to  at /Group[1]

I’m not quite contented with that as a robust unit test. I want to ensure that the two files have specific differences. When I think of comparing two files with a diff, I’m picturing a visual diff:

..and the concept of a patch – that the set of +/- lines differences are collected and made available for inspection/verification.

XMLUnit does have some alternatives that, while not exactly what I’m looking for, I could use and are worth discussing:

DetailedDiff

DetailedDiff is an extension of Diff, which will give me a list of all the Differences in the comparison.

final DetailedDiff diff = new DetailedDiff(myDiff);
assertTrue("Expected a difference in child nodes", 
    diff.getAllDifferences()
        .contains(DifferenceConstants.CHILD_NODE_NOT_FOUND));

..will assert that the comparison has resulted in a mismatch in the number of children between the two XMLs (Javadoc). A few of those assertions could describe the expected differences between the XMLs.

Counting Nodes

CountingNodeTester is another alternative that allows us to assert the total number of elements contained in the XML:

CountingNodeTester countingNodeTester = 
    new CountingNodeTester(38);
assertNodeTestPasses(mergedXml, 
    countingNodeTester, Node.ELEMENT_NODE);

Or alternatively, comparing the counts of the two XMLs (7 additional nodes):

final int countOriginal = 31;
final int countMerged = countOriginal + 7;
CountingNodeTester countingNodeOriginal = 
    new CountingNodeTester(countOriginal);
CountingNodeTester countingNodeMerged = 
    new CountingNodeTester(countMerged);
assertNodeTestPasses(originalXml, 
    countingNodeOriginal, Node.ELEMENT_NODE);
assertNodeTestPasses(mergedXml, 
    countingNodeMerged, Node.ELEMENT_NODE);

XPath

Finally we could also use XPath evaluations to assert the existence or lack of certain structures:

assertXpathNotExists("/Group/PageContainer[6]", 
   mergedXml);
...
assertXpathExists("/Group/PageContainer/External-Group/File[@Location='/Sites/centre/dcita/site.xml']", 
    mergedXml);
assertXpathEvaluatesTo("/Sites/centre/dcita/site.xml", 
    "/Group/PageContainer/External-Group/File/@Location", mergedXml);

So while XMLUnit gives us a pretty good toolset for XML comparisons, I’m still wondering if there’s a more diff-oriented tool I could use.

java-diff-utils

I found java-diff-utils, which looks like it could be a good option for handling diffs the way I’m imagining. Lets have a go!

The sample code on the website shows us the basic usage:

// Compute diff. Get the Patch object. Patch is the container for computed deltas.
Patch patch = DiffUtils.diff(original, revised);

..where original and revised are List objects.

The Patch object gives us a list of Deltas, containing the ‘original’ and ‘revised’ segments. Perfect!

This matches what we have in the image (disregarding the empty elements formatted differently)

Assert.assertEquals(1, patch.getDeltas().size());

The Delta itself contains a list of text lines, so we could potentially verify the list of strings manually.

System.out.println(patch.getDelta(0).getRevised().getLines());

..outputs something like:

[<PageContainer>
, <External-Group>
, <File Location="/Sites/centre/dcita/site.xml">, </File>
, </External-Group>
, </PageContainer>
, <PageContainer>
, <Page Title="">
, <File Location="/Content/centre/dcita/index.xml">, </File>
, <Description>Migrated from previous CMS1 Homepage, </Description>
, </Page>
, </PageContainer>
    ]

A cleaner scenario could involve saving the expected delta text to a file (/src/test/resources/merged_xml.diff), and comparing the file contents to the lines in the actual patch delta

final String target = resourceToString("/merged_xml.diff");
final String actual = 
    listToString(patch.getDelta(0).getRevised().getLines());
Assert.assertEquals(target, actual);

This needs a couple of helper functions to load the diff file into a String, and convert the delta List into a String also.

    private String resourceToString(String filename) {
        StringBuffer lines = new StringBuffer();
        String line = "";
        try {
            BufferedReader in = new BufferedReader(
                new FileReader(getClass().getResource(filename).getPath()));
            while ((line = in.readLine()) != null) {
                lines.append(line);
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
        return lines.toString();
    }

    private String listToString(List list) {
        StringBuffer buff = new StringBuffer(list.size());
        for (Object o : list) {
            buff.append(((String) o).trim());
        }
        return buff.toString();
    }

Finally…

So in the end, my test case ends with combining the above diff utils snippets:

        ...
        Patch patch = DiffUtils.diff(original, revised);
        Assert.assertEquals(1, patch.getDeltas().size());

        final String target = resourceToString("/merged_xml.diff");
        final String actual = 
            listToString(patch.getDelta(0).getRevised().getLines());
        Assert.assertEquals(target, actual);

A cyclical dependancy anti-pattern: Identity Theft

IMPORTANT!
This blog has moved to http://blog.brasskazoo.com!

Anti-patterns:

In software engineering, an anti-pattern (or antipattern) is a design pattern that appears obvious but is ineffective or far from optimal in practice.[1]

Or as I like to think of it, the type of code that makes you want to rip your eyes out, like in that movie Event Horizon…

Identity Theft

Here’s a pattern that I have seen many times in a previous codebase, which I believe suffers from some design flaws:

FooListing is a class that retrieves and maintains a list of Foo objects.

Description

FooListing is a repository-type object that holds a list of Foo objects, which it populates from the database using its populate() function.

The Foo object also has a populate() method, but this method instantiates a FooListing, and uses the FooListing populate function with its key – so now the FooListing contains the desired Foo object.

Our original Foo object now contains a reference to the Foo object that it wants to be.  So it does what anyone would do, steals its identity and hides the body.

Foo.populate() looks something like this:

public void populate() {
  FooListing listing = new FooListing()
  // Set up FooListing
  FooListing.setSearchKey(id);
  if (listing.size > 0) {
      Foo foo = listing.get(0);
      this.setBar(foo.getBar);
      // continue to copy over properties
      // ...
  } else {
      LOG.warn("Foo not found for key: " + id);
  }
}

Why this is bad:

Firstly, apart from the questional morality, it is duplicated code. FooListing already has the capability to create and populate a Foo object from the database. Two locations for this code means twice the maintenance if something changes, (more than) twice the possibility of bugs, etc.

Foo and FooListing have become tightly coupled and dependant on each other under this design – there is a cyclic dependancy which is a code smell, and may cause headaches when writing unit tests.

There is also a waste of resoures creating the uneccesary FooListing and Foo objects inside of Foo.populate(), at least some of that could be avoided by client code accessing instances of Foo via FooListing.populate

It also doesn’t make sense that a data access object like Foo should be concerned with information about how it is created. Foo should act more like a bean, or an immutable class.

Another dangerous aspect of this design is the implication that client code accessing Foo cannot be certain that Foo has been instantiated correctly. If the client code has a faulty key for Foo that does not exist in the database, when it creates new Foo(key) there is no way to know that Foo.populate() has failed to find the correct value, and instead they are left with a faulty Foo instance which was not what they requested.

Solution:

The best solution for this isolated pattern is to completely remove (or deprecate) the Foo.populate() method, and replace calls to it with FooListing instances.

If FooListing fails to find a matching Foo, the client code should realise this when FooListing returns them a null object. The client code can handle and recover from this case in context.

Implementing a getFirstResult() function in FooListing could be beneficial if there are many cases where the code with otherwise be calling get(0).

We could also simplify the calling code so that retrieving a result is a one-line operation – i.e. get() calls populate() if the list has not already been populated.

public final class FooListing {
  private List<Foo> _listing;
  private int _id;

  public FooListing {
  }

  public FooListing(int id) {
    _searchId = id;
  }

  public void populate() {
    _listing = new ArrayList<Foo>();
    // Query the database and add to _listing
  }

  public Foo get(int index) {
    if (_listing == null) {
      populate();
    }
    if (_listing.isEmpty() || index >= _listing.size()) {
      return null;
    } else {
      return _listing.get(index);
    }
  }

  public Foo getFirstResult() {
    return this.get(0);
  }

  public List<Foo> getList() {
    return _listing;
  }

  public void setSearchId(int id) {
    _id = id;
  }
}

And the client code would only need to call

Foo foo = new FooListing(id).getFirstResult();

References:
1. http://en.wikipedia.org/wiki/Anti-patterns
							

Building a simple project using behavior-driven development with JBehave

IMPORTANT!
This post has moved to http://blog.brasskazoo.com!

In a previous post introducing BDD I outlined a simple bus ticket application. Since then I’ve been looking at JBehave, and thought I’d try building a simple version of the bus tickets project, using JBehave to support behavior-driven development of the application.

About JBehave

JBehave is a testing framework that takes the BDD concepts and applies it to JUnit tests, so you can effectively test your acceptance criteria directly using automated unit tests.

It uses BDD acceptance criteria as steps in a test case, and each Given/When/Then statement is interpreted and executed as a method. The acceptance criteria is simply stored in a text file which is read by JBehave when the test case is run.

Setting up JBehave

Download the distribution jars from http://jbehave.org (at the time of writing, version 2.3.2 is the latest), and extract it to a convenient location!

Set up references to the jars in your project (you need to reference at least hamcrest, junit-dep and jbehave-core for this example to work).

New project: ‘bus-tix’

For the purposes of this example, I’m creating a project I’ll call ‘bus-tix’, with production and test code packages.

The scenarios

Lets take one of the scenarios from my previous post (slightly modified):

Scenario: Inserting initial coins

Given that the application is operating
When a coin is inserted
Then ensure the time indicator displays the purchased time

I will use this as the behavioural spec for the initial work on the application. From the description of the scenario, we can instantly tell that we’re going to need a coin handler and time indicator as part of the application.

Now before I put this into a text file, I must point out that the default configuration of JBehave requires that the scenario files are named to match the class names of the test case – so if we had the test case InsertingInitialCoins, then the scenario text file must be named inserting_initial_coins (with no extension). I don’t particularly like this, so as you’ll see I modify the constructor so that I can call my scenario ‘inserting_initial_coins.scenario‘, to make it clear the purpose of the file.

We’ll need to create a test case, which extends org.jbehave.scenario.Scenario, and a ‘step’ class which extends  org.jbehave.scenario.steps.Steps. The step class will contain all the corresponding steps for the scenario file(s).

Scenario: Inserting initial coins

Given that the application is operating
 When a coin is inserted
 Then ensure the time indicator displays the purchased time

Create the test case class InsertingInitialCoins

package com.brass.bustix;

import org.jbehave.scenario.PropertyBasedConfiguration;
import org.jbehave.scenario.Scenario;
import org.jbehave.scenario.parser.ClasspathScenarioDefiner;
import org.jbehave.scenario.parser.PatternScenarioParser;
import org.jbehave.scenario.parser.ScenarioDefiner;
import org.jbehave.scenario.parser.UnderscoredCamelCaseResolver;

public class InsertingInitialCoins extends Scenario {
    public InsertingInitialCoins() {
        this(Thread.currentThread().getContextClassLoader());
    }

    public InsertingInitialCoins(final ClassLoader classLoader) {
        super(new PropertyBasedConfiguration() {
            public ScenarioDefiner forDefiningScenarios() {
                return new ClasspathScenarioDefiner(
                    new UnderscoredCamelCaseResolver(".scenario"),
                    new PatternScenarioParser(this), classLoader);
                }
            }, new BustixSteps());
        }
    }

The default constructor calls the custom scenario file loader, so that we can append ‘.scenario’ to your scenario files. If I were creating several of these test scenarios, I would pull all that up to an abstract class to avoid re-writing those constructors.

Steps class (empty for now):

package com.brass.bustix;

import static org.jbehave.Ensure.ensureThat;
import org.jbehave.scenario.annotations.Given;
import org.jbehave.scenario.annotations.Then;
import org.jbehave.scenario.annotations.When;
import org.jbehave.scenario.steps.Steps;

public class BustixSteps extends Steps {
    // TODO – fill in steps
}

Make sure that the .scenario file is copied to your compile directory! Otherwise you will get the error like:

org.jbehave.scenario.errors.ScenarioNotFoundException: Scenario com/brass/bustix/inserting_initial_coins.scenario could not be found by classloader sun.misc.Launcher$AppClassLoader@133056f

When you run the test for the first time, you will see this output:

Scenario: Inserting initial coins
Given that the application is operating (PENDING)
When a coin is inserted (PENDING)
Then ensure the time indicator displays the purchased time (PENDING)

Perhaps confusingly, the test has passed despite the steps not being executed – PENDING means that the step hasn’t been executed because no corresponding method has been found.

Defining Steps

JBehave uses annotations to mark a method as a particular step. The annotation is the first word of the step we’re using, with the rest of the text as a parameter to that annotation. So for our first step ‘Given that the application is operating’ we reformat it as an @Given annotation:

@Given(“that the application is operating”)

And then JBehave knows that the function following that annotation is what needs to be executed for that step.

So now in BustixSteps we add:

@Given(“that the application is operating”)
public void startApplication() {
    ensureThat(false);
}

The assertion statement ensures that the test will fail – an empty test will pass (as it usually does with JUnit). The ‘ensureThat’ function comes from JBehave’s wrapping of hamcrest matchers – an interesting alternative to Junit’s assert statements. I won’t go into them right now.

Rerunning the test, we get the failure we want!

Scenario: Inserting initial coins
Given that the application is operating (FAILED)
 When a coin is inserted (PENDING)
 Then ensure the time indicator displays the purchased time (PENDING)

java.lang.AssertionError:
Expected: is <true>
got: <false>
...

Lets fill in the rest of the steps for this scenario while we’re here:

package com.brass.bustix;

import static org.jbehave.Ensure.ensureThat;
import org.jbehave.scenario.annotations.Given;
import org.jbehave.scenario.annotations.Then;
import org.jbehave.scenario.annotations.When;
import org.jbehave.scenario.steps.Steps;

public class BustixSteps extends Steps {

    @Given("that the application is operating")
        public void startApplication() {
        ensureThat(false);
    }

    @When("a coin is inserted")
    public void insertCoin() {
        ensureThat(false);
    }

    @Then("ensure the time indicator displays the purchased time")
    public void testThatTimeIsDisplayed() {
        ensureThat(false);
    }
}

At this stage, we’ve got our steps outlined for our basic scenario, and we can begin the test-driven  development of the application!

Note: when you have a scenario step that begins with an ‘And’, it is still considered as one of given/when/then, e.g.

Given that the application is operating
  And the time indicator displays nothing

..are both @Given annotations.

Next: check out the hamcrest matchers to make sure you’re getting the full potential from your asserts!

References

• A brief introduction to Behaviour-Driven Development
• JBehave two minute tutorial
• Getting started with JBehave in 8 steps
• Hamcrest matchers

A brief introduction to Behaviour-Driven Development