Monthly Archives: June 2008

Functional testing with Maven, Cargo and Selenium

Posted on by Carlos Sanchez
23

Setting up automated functional integration tests is not too hard if you have the right tools. It can take you a bit of time to setup but in the long run you’ll benefit from reduced QA times, reduced risks, a more confident development team, the ability to do safe refactorings, and many more advantages.

I’m going to explain how Maven, Selenium, Cargo and JBoss 4.2 can be setup to run automatically in a continuous integration server such as Continuum customizing the server configuration as needed and deploying any webapp automatically. Every time the webapp is changed the CI server will execute the tests against the latest version ensuring you are always in a safe state.

The biggest difference with other tutorials I’ve found is that most of them cover just Jetty and are not updated to the latest versions of libraries and tools, so here it is my contribution.

Architecture

  • A new project is setup with dependencies to the war project to be tested. Also required a dependency to selenium java client.
  • Cargo will download and install the application server (JBoss)
  • We will copy any required configuration and libraries (ie. jdbc driver)
  • Cargo will start the application server
  • The Selenium server is started
  • Surefire executes the junit tests that interact with the selenium server and test the running app
  • Cargo will stop the app server

We use profiles to enable different combination of browser/application server. By default cargo uses jetty.

Config Profiles
JBoss 4.2 and Firefox (default) -Pjboss42x,firefox
JBoss 4.2 and Internet Explorer -Pjboss42x,iexplore
Jetty and Firefox -Pfirefox
Jetty and Internet Explorer -Piexplore

The POM

Dependencies

<dependencies>
    <dependency>
      <groupId>com.acme</groupId>
      <artifactId>mywebapp</artifactId>
      <version>${project.version}</version>
      <type>war</type>
    </dependency>
    <!-- the jdbc driver we need to copy to the appserver -->
    <dependency>
      <groupId>mysql</groupId>
      <artifactId>mysql-connector-java</artifactId>
    </dependency>
    <dependency>
      <groupId>org.openqa.selenium.client-drivers</groupId>
      <artifactId>selenium-java-client-driver</artifactId>
      <version>1.0-SNAPSHOT</version> <!-- required for firefox 3 else use 1.0-beta-1 -->
      <scope>test</scope>
    </dependency>
  </dependencies>

Properties used in several places

Ports, where to uncompress the application server,…

<properties>
    <cargo.install.directory>${project.build.directory}/installs</cargo.install.directory>
    <selenium.port>14444</selenium.port>
    <servlet.port>18880</servlet.port>
    <selenium.background>true</selenium.background>
  </properties>

Plugin configuration

JDBC driver

Copy mysql jdbc driver to the app server lib folder

<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-dependency-plugin</artifactId>
        <executions>
          <execution>
            <id>copy-jdbc-lib</id>
            <phase>generate-resources</phase>
            <goals>
              <goal>copy-dependencies</goal>
            </goals>
            <configuration>
              <includeGroupIds>mysql</includeGroupIds>
              <outputDirectory>${lib.target}</outputDirectory>
            </configuration>
          </execution>
        </executions>
      </plugin>

Cargo

Install the application server in an early phase so we can customize it with our configuration files (see profiles). Then start before integration tests and stop afterwards. Parameters are used so different profiles can use different application servers.

      <plugin>
        <groupId>org.codehaus.cargo</groupId>
        <artifactId>cargo-maven2-plugin</artifactId>
        <executions>
          <execution>
            <id>install</id>
            <phase>generate-resources</phase>
            <goals>
              <goal>install</goal>
            </goals>
          </execution>
          <execution>
            <id>start-container</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>start</goal>
            </goals>
            <configuration>
              <wait>false</wait>
            </configuration>
          </execution>
          <execution>
            <id>stop-container</id>
            <phase>post-integration-test</phase>
            <goals>
              <goal>stop</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <container>
            <containerId>${container.name}</containerId>
            <zipUrlInstaller>
              <url>${container.url}</url>
              <installDir>${cargo.install.directory}/${container.name}</installDir>
            </zipUrlInstaller>
            <log>${project.build.directory}/logs/${container.name}.log</log>
            <output>${project.build.directory}/logs/${container.name}.out</output>
            <timeout>600000</timeout>
          </container>
          <configuration>
            <!--
            <home>${project.build.directory}/${container.name}conf</home>
            <type>existing</type>
            -->
            <properties>
              <cargo.servlet.port>${servlet.port}</cargo.servlet.port>
              <cargo.jboss.configuration>default</cargo.jboss.configuration>
              <cargo.rmi.port>1099</cargo.rmi.port>
            </properties>

            <deployables>
              <!-- application to deploy -->
              <deployable>
                <groupId>com.acme</groupId>
                <artifactId>mywebapp</artifactId>
                <type>war</type>
                <properties>
                  <context>acontext</context>
                </properties>
              </deployable>
            </deployables>
          </configuration>
        </configuration>
      </plugin>

Selenium

Make surefire skip tests during test phase and run them in the integration-test phase. Pass some properties as system properties so they are accessible from the junit test case.

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <configuration>
          <!-- Skip the normal tests, we'll run them in the integration-test phase -->
          <skip>true</skip>
          <systemProperties>
            <property>
              <name>browser</name>
              <value>${browser}</value>
            </property>
            <property>
              <name>servlet.port</name>
              <value>${servlet.port}</value>
            </property>
            <property>
              <name>selenium.port</name>
              <value>${selenium.port}</value>
            </property>
          </systemProperties>
        </configuration>
        <executions>
          <execution>
            <phase>integration-test</phase>
            <goals>
              <goal>test</goal>
            </goals>
            <configuration>
              <skip>false</skip>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>selenium-maven-plugin</artifactId>
        <!-- to run headless in a Unix server with a virtual framebuffer X server Xvfb
             you need to call first the goal selenium:xvfb ie. "mvn clean selenium:xvfb install"
             see http://mojo.codehaus.org/selenium-maven-plugin/examples/headless-with-xvfb.html -->
        <executions>
          <execution>
            <id>start-selenium</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>start-server</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <background>${selenium.background}</background>
          <port>${selenium.port}</port>
          <logOutput>true</logOutput>
        </configuration>
      </plugin>

Application server profiles

We can configure a different profile for each application server and set some specific application server configuration.

<profiles>
    <profile>
      <id>jboss42x</id>
      <activation>
        <activeByDefault>true</activeByDefault>
      </activation>
      <properties>
        <container.name>jboss42x</container.name>
        <container.url>http://internap.dl.sourceforge.net/sourceforge/jboss/jboss-4.2.1.GA.zip</container.url>
        <jboss.version>4.2.1.GA</jboss.version>
        <jboss.conf.directory>${cargo.install.directory}/${container.name}/jboss-${jboss.version}/jboss-${jboss.version}/server/default</jboss.conf.directory>
        <lib.target>${jboss.conf.directory}/deploy/lib</lib.target>
        <war.target>${jboss.conf.directory}/deploy</war.target>
      </properties>

      <dependencies>
        <dependency>
          <groupId>org.jboss</groupId>
          <artifactId>jboss</artifactId>
          <version>${jboss.version}</version>
          <type>zip</type>
          <scope>test</scope>
        </dependency>
      </dependencies>
      <build>
        <plugins>
          <!-- copy to the application server directory any customized configuration files that we need -->
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-antrun-plugin</artifactId>
            <executions>
              <execution>
                <phase>process-resources</phase>
                <configuration>
                  <tasks>
                    <copy todir="${jboss.conf.directory}" overwrite="true">
                      <fileset dir="${basedir}/src/test/${container.name}"/>
                    </copy>
                  </tasks>
                </configuration>
                <goals>
                  <goal>run</goal>
                </goals>
              </execution>
            </executions>
          </plugin>
        </plugins>
      </build>
    </profile>

Browser profiles

As with the application servers we have a profile for each browser

<profile>
      <id>firefox</id>
      <activation>
        <activeByDefault>true</activeByDefault>
      </activation>
      <properties>
        <browser>*firefox</browser>
      </properties>
    </profile>
    <profile>
      <id>iexplore</id>
      <properties>
        <browser>*iexplore</browser>
      </properties>
    </profile>
    <profile>
      <id>otherbrowser</id>
      <properties>
        <browser>*custom ${browserPath}</browser>
      </properties>
    </profile>

Enabling testing during development

Make selenium not to run in the background so we can execute tests from the IDE

    <profile>
      <id>dev</id>
      <properties>
        <selenium.background>false</selenium.background>
      </properties>
    </profile>

Repositories

Required for Selenium dependencies

<repositories>
    <repository>
      <id>openqa.org</id>
      <name>OpenQA Repository</name>
      <url>http://archiva.openqa.org/repository/releases</url>
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
      <releases>
        <enabled>true</enabled>
      </releases>
    </repository>
    <!-- for selenium 1.0-SNAPSHOT -->
    <repository>
      <id>snapshots.openqa.org</id>
      <name>OpenQA Sanpshots Repository</name>
      <url>http://archiva.openqa.org/repository/snapshots</url>
      <snapshots>
        <enabled>true</enabled>
      </snapshots>
      <releases>
        <enabled>false</enabled>
      </releases>
    </repository>
  </repositories>

Running in the build server

In an Unix server without X running you can still run Selenium tests using Xvfb (virtual framebuffer X server) by calling selenium:xvfb provided it’s properly configured.

Also you can pass the path to the browser binary if not in the PATH

mvn clean selenium:xvfb install -Dbrowser="*firefox /usr/lib64/firefox-1.5.0.12/firefox-bin"

The JUnit test

public class SeleniumHelloWorldTest
    extends TestCase
{
    private DefaultSelenium selenium;

    private String baseUrl;

    @Override
    public void setUp()
        throws Exception
    {
        super.setUp();
        String port = System.getProperty( "servlet.port" );
        baseUrl = "http://localhost:" + port;
        selenium = createSeleniumClient( baseUrl );
        selenium.start();
    }

    @Override
    public void tearDown()
        throws Exception
    {
        selenium.stop();
        super.tearDown();
    }

    protected DefaultSelenium createSeleniumClient( String url )
        throws Exception
    {
        String browser = System.getProperty( "browser" );
        String port = System.getProperty( "selenium.port" );
        return new DefaultSelenium( "localhost", Integer.parseInt( port ), browser, url );
    }

    public void testHelloWorld()
        throws Exception
    {
        selenium.open( baseUrl + "/mycontext/" );
        assertTrue( selenium.isTextPresent( "acme" ) );
    }
}

Debugging and troubleshooting (update)

You can check JBoss logs in target/logs/jboss42x.out and Selenium server logs in target/selenium/server.log

References

Other wiki entries and blogs