Continuous Deployment: Deploying to Glassfish with Maven and TeamCity

IMPORTANT!
This blog has moved to http://blog.brasskazoo.com!

A later stage of the continuous integration process – continuous deployment. The sooner we can deploy a tested and verified piece of software, the better!

Here I’m describing an automated deployment process that uses Maven to deploy to a Glassfish application server. TeamCity facilitates the build and test stages, with an additional deployment of the packaged web application. Using the glassfish plugin for maven 2, we can integrate the application server deployment into the continuous integration cycle, and provide a constantly up-to-date development/test environment.

Glassfish

I’ll create a new domain from scratch for the maven apps, using the default port values (i.e. admin port 4848, http port 8080), but setting the admin password and master password.

In setting up the glassfish domain we generate a password file so that we are not storing any passwords in plain text – such as in the pom or settings.xml

cd ${glassfish.home}/bin
./asadmin create-domain --savemasterpassword=true my-apps

the –savemasterpassword switch generates an encrypted ‘master-password’ file in the domains/my-apps directory.

Maven

Maven profiles makes it easy to have machine-specific variables so that moving to other platforms in the future is straightforward.

On the on the host machine I’ve created the file ~/.m2/settings.xml.

<?xml version="1.0" encoding="UTF-8"?>
xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <profiles>
    <profile>
      <id>glassfish-context</id>
      <properties>
        <local.glassfish.home>/Users/brass/bin/glassfishv3</local.glassfish.home>
        <local.glassfish.user>admin</local.glassfish.user>
        <local.glassfish.domain>my-apps</local.glassfish.domain>
        <local.glassfish.passfile>
${local.glassfish.home}/glassfish/domains/${local.glassfish.domain}/master-password
        </local.glassfish.passfile>
      </properties>
    </profile>
  </profiles>

  <activeProfiles>
    <activeProfile>glassfish-context</activeProfile>
  </activeProfiles>
</settings>

This gives us the parameters for the glassfish instance that we will use in our pom.

Update: I found that while the above works for v3.1, my dev machine’s glassfish v3.0.1 needed to reference glassfish home one directory deeper:

<local.glassfish.home>/Users/brass/bin/glassfishv3/glassfish
</local.glassfish.home>
...
<local.glassfish.passfile>
${local.glassfish.home}/domains/${local.glassfish.domain}/master-password
</local.glassfish.passfile>

In the project pom.xml, we define a profile for glassfish deployment:

<profile>
  <id>glassfish-deploy</id>
  <pluginRepositories>
    <pluginRepository>
      <id>maven.java.net</id>
        <name>Java.net Maven2 Repository</name>
        <url>http://download.java.net/maven/2</url>
      </pluginRepository>
    </pluginRepositories>
    <build>
      <plugins>
      <plugin>
        <groupId>org.glassfish.maven.plugin</groupId>
        <artifactId>maven-glassfish-plugin</artifactId>
        <version>2.1</version>
        <configuration>
          <glassfishDirectory>${local.glassfish.home}</glassfishDirectory>
          <user>${local.glassfish.user}</user>
          <passwordFile>${local.glassfish.passfile}</passwordFile>
          <autoCreate>true</autoCreate>
          <debug>true</debug>
          <echo>false</echo>
          <terse>true</terse>
          <domain>
            <name>${local.glassfish.domain}</name>
            <adminPort>4848</adminPort>
            <httpPort>8080</httpPort>
            <httpsPort>8443</httpsPort>
            <iiopPort>3700</iiopPort>
            <jmsPort>7676</jmsPort>
            <reuse>false</reuse>
          </domain>
          <components>
            <component>
              <name>${project.artifactId}</name>
              <artifact>
${project.build.directory}/${project.build.finalName}.war
              </artifact>
            </component>
          </components>
        </configuration>
      </plugin>
    </plugins>
  </build>
</profile>

The repository for the glassfish plugin repository is specified within the profile since its not part of the larger project in this case.

As you see the variables from the local settings.xml are used for the glassfish config.

TeamCity

The TeamCity setup needs to include two things in the maven2 runner config:

• Glassfish goals
• Profile parameters

The glassfish goals that are used should be able to start the domain if it isn’t running, and replace the application.

The ‘redeploy’ goal would allow a hot-swap deployment, if for example we were running other applications on the domain.

See the plugin page (http://maven-glassfish-plugin.java.net/) for more info.

Next

So now this process provides us with continuous deployment – a commit will be built, tested and deployed automatically, allowing changes to the software to be seen and used almost immediately!

Monitoring OSCache statistics with JMX

Given an existing web application that uses OSCache, we can pretty easily add support for JMX monitoring.

A couple of things first:

• We are using Sun’s Glassfish application server
• The application is not yet spring-enabled
• The project uses maven for dependancy management

Since we are using Glassfish, we already have an MBean server and connector, so we don’t have to worry about configuring those parts of the stack. (An MBean is a Managed JavaBean)

As for Spring support, we are restricting its introduction to a small aspect of the application (for now!), so we are minimizing the impact on the rest of the application.

What we need to do:

• Modify oscache.properties and add an event listener
• Add a dependancy on spring-web
• Add a spring context for the JMX exporter
• Modify web.xml to load the spring context

OSCache event listener

Firstly, we need to set a listener for OSCache that will publish the statistics we are interested in.

In oscache.properties, there is a section for event listeners. We want to implement a statistical listener:

cache.event.listeners=com.opensymphony.oscache.extra.StatisticListenerImpl

Spring dependancies

If you’re using maven, add a dependency on spring-web. This has dependancies on spring-core, spring-context and spring-beans (At the time of writing, 3.0.2.RELEASE was the latest version in the maven repositories).

Otherwise download these manually and chuck them in your WEB-INF/lib.

Spring Context

I made a separate spring context file (cacheContext.xml) for the cache MBean exporter:

    <beans xmlns="http://www.springframework.org/schema/beans"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">

        <!-- OSCache stats listener bean -->
        <bean id="statisticListener" class="com.opensymphony.oscache.extra.StatisticListenerImpl"/>

        <!-- Export the OSCache stats beans -->
        <bean id="exporter" class="org.springframework.jmx.export.MBeanExporter">
            <property name="beans">
                <map>
                    <entry key="oscache-bean:name=Statistics" value="statisticListener"/>
                </map>
            </property>
        </bean>
    </beans>

The map in the exporter links the statistics bean to a key that will be visible in JConsole.

Note: The OpenSymphony link in the references below gives you code to expose an arbitrary JMX connector and port. This is unnecessary since we’re accessing via the application server.

Adding the Spring context loader

The web.xml then needs to be updated to be told to load the new cacheContext.xml.

    <context-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>/WEB-INF/cacheContext.xml</param-value>
    </context-param>

	...

    <listener>
        <listener-class>
            org.springframework.web.context.ContextLoaderListener
        </listener-class>
    </listener>

JMX Clients

JDK 1.5 includes two JMX clients that you can use.

jconsole is the legacy monitoring application, which has been superseded by the fancy UI of jvisualvm.

Either application will allow you to monitor the statistics bean, however in jvisualvm you will need to install the plugin VisualVM-MBeans under Tools > Plugins. Meanwhile jconsole has an MBeans tab by default.

When your application is running, use one of these utilities to connect to the application server (either directly or via localhost:8686 for glassfish). Under the oscache-bean > Statistics, you should be able to see counts generated by cache attributes. Double-clicking the numbers will pop up a graph, and right-clicking will allow you to export the recorded values.

References

1. http://www.opensymphony.com/oscache/wiki/JMX%20Monitoring.html
2. http://java.sun.com/docs/books/tutorial/jmx/mbeans/standard.html
3. http://java.sun.com/developer/technicalArticles/J2SE/jmx.html