Functional testing with Maven, Cargo and Selenium

Setting up automated functional integration tests is not too hard if you have the right tools. It can take you a bit of time to setup but in the long run you’ll benefit from reduced QA times, reduced risks, a more confident development team, the ability to do safe refactorings, and many more advantages.

I’m going to explain how Maven, Selenium, Cargo and JBoss 4.2 can be setup to run automatically in a continuous integration server such as Continuum customizing the server configuration as needed and deploying any webapp automatically. Every time the webapp is changed the CI server will execute the tests against the latest version ensuring you are always in a safe state.

The biggest difference with other tutorials I’ve found is that most of them cover just Jetty and are not updated to the latest versions of libraries and tools, so here it is my contribution.

Architecture

  • A new project is setup with dependencies to the war project to be tested. Also required a dependency to selenium java client.
  • Cargo will download and install the application server (JBoss)
  • We will copy any required configuration and libraries (ie. jdbc driver)
  • Cargo will start the application server
  • The Selenium server is started
  • Surefire executes the junit tests that interact with the selenium server and test the running app
  • Cargo will stop the app server

We use profiles to enable different combination of browser/application server. By default cargo uses jetty.

Config Profiles
JBoss 4.2 and Firefox (default) -Pjboss42x,firefox
JBoss 4.2 and Internet Explorer -Pjboss42x,iexplore
Jetty and Firefox -Pfirefox
Jetty and Internet Explorer -Piexplore

The POM

Dependencies

<dependencies>
    <dependency>
      <groupId>com.acme</groupId>
      <artifactId>mywebapp</artifactId>
      <version>${project.version}</version>
      <type>war</type>
    </dependency>
    <!-- the jdbc driver we need to copy to the appserver -->
    <dependency>
      <groupId>mysql</groupId>
      <artifactId>mysql-connector-java</artifactId>
    </dependency>
    <dependency>
      <groupId>org.openqa.selenium.client-drivers</groupId>
      <artifactId>selenium-java-client-driver</artifactId>
      <version>1.0-SNAPSHOT</version> <!-- required for firefox 3 else use 1.0-beta-1 -->
      <scope>test</scope>
    </dependency>
  </dependencies>

Properties used in several places

Ports, where to uncompress the application server,…

<properties>
    <cargo.install.directory>${project.build.directory}/installs</cargo.install.directory>
    <selenium.port>14444</selenium.port>
    <servlet.port>18880</servlet.port>
    <selenium.background>true</selenium.background>
  </properties>

Plugin configuration

JDBC driver

Copy mysql jdbc driver to the app server lib folder

<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-dependency-plugin</artifactId>
        <executions>
          <execution>
            <id>copy-jdbc-lib</id>
            <phase>generate-resources</phase>
            <goals>
              <goal>copy-dependencies</goal>
            </goals>
            <configuration>
              <includeGroupIds>mysql</includeGroupIds>
              <outputDirectory>${lib.target}</outputDirectory>
            </configuration>
          </execution>
        </executions>
      </plugin>

Cargo

Install the application server in an early phase so we can customize it with our configuration files (see profiles). Then start before integration tests and stop afterwards. Parameters are used so different profiles can use different application servers.

      <plugin>
        <groupId>org.codehaus.cargo</groupId>
        <artifactId>cargo-maven2-plugin</artifactId>
        <executions>
          <execution>
            <id>install</id>
            <phase>generate-resources</phase>
            <goals>
              <goal>install</goal>
            </goals>
          </execution>
          <execution>
            <id>start-container</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>start</goal>
            </goals>
            <configuration>
              <wait>false</wait>
            </configuration>
          </execution>
          <execution>
            <id>stop-container</id>
            <phase>post-integration-test</phase>
            <goals>
              <goal>stop</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <container>
            <containerId>${container.name}</containerId>
            <zipUrlInstaller>
              <url>${container.url}</url>
              <installDir>${cargo.install.directory}/${container.name}</installDir>
            </zipUrlInstaller>
            <log>${project.build.directory}/logs/${container.name}.log</log>
            <output>${project.build.directory}/logs/${container.name}.out</output>
            <timeout>600000</timeout>
          </container>
          <configuration>
            <!--
            <home>${project.build.directory}/${container.name}conf</home>
            <type>existing</type>
            -->
            <properties>
              <cargo.servlet.port>${servlet.port}</cargo.servlet.port>
              <cargo.jboss.configuration>default</cargo.jboss.configuration>
              <cargo.rmi.port>1099</cargo.rmi.port>
            </properties>

            <deployables>
              <!-- application to deploy -->
              <deployable>
                <groupId>com.acme</groupId>
                <artifactId>mywebapp</artifactId>
                <type>war</type>
                <properties>
                  <context>acontext</context>
                </properties>
              </deployable>
            </deployables>
          </configuration>
        </configuration>
      </plugin>

Selenium

Make surefire skip tests during test phase and run them in the integration-test phase. Pass some properties as system properties so they are accessible from the junit test case.

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <configuration>
          <!-- Skip the normal tests, we'll run them in the integration-test phase -->
          <skip>true</skip>
          <systemProperties>
            <property>
              <name>browser</name>
              <value>${browser}</value>
            </property>
            <property>
              <name>servlet.port</name>
              <value>${servlet.port}</value>
            </property>
            <property>
              <name>selenium.port</name>
              <value>${selenium.port}</value>
            </property>
          </systemProperties>
        </configuration>
        <executions>
          <execution>
            <phase>integration-test</phase>
            <goals>
              <goal>test</goal>
            </goals>
            <configuration>
              <skip>false</skip>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>selenium-maven-plugin</artifactId>
        <!-- to run headless in a Unix server with a virtual framebuffer X server Xvfb
             you need to call first the goal selenium:xvfb ie. "mvn clean selenium:xvfb install"
             see http://mojo.codehaus.org/selenium-maven-plugin/examples/headless-with-xvfb.html -->
        <executions>
          <execution>
            <id>start-selenium</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>start-server</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <background>${selenium.background}</background>
          <port>${selenium.port}</port>
          <logOutput>true</logOutput>
        </configuration>
      </plugin>

Application server profiles

We can configure a different profile for each application server and set some specific application server configuration.

<profiles>
    <profile>
      <id>jboss42x</id>
      <activation>
        <activeByDefault>true</activeByDefault>
      </activation>
      <properties>
        <container.name>jboss42x</container.name>
        <container.url>http://internap.dl.sourceforge.net/sourceforge/jboss/jboss-4.2.1.GA.zip</container.url>
        <jboss.version>4.2.1.GA</jboss.version>
        <jboss.conf.directory>${cargo.install.directory}/${container.name}/jboss-${jboss.version}/jboss-${jboss.version}/server/default</jboss.conf.directory>
        <lib.target>${jboss.conf.directory}/deploy/lib</lib.target>
        <war.target>${jboss.conf.directory}/deploy</war.target>
      </properties>

      <dependencies>
        <dependency>
          <groupId>org.jboss</groupId>
          <artifactId>jboss</artifactId>
          <version>${jboss.version}</version>
          <type>zip</type>
          <scope>test</scope>
        </dependency>
      </dependencies>
      <build>
        <plugins>
          <!-- copy to the application server directory any customized configuration files that we need -->
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-antrun-plugin</artifactId>
            <executions>
              <execution>
                <phase>process-resources</phase>
                <configuration>
                  <tasks>
                    <copy todir="${jboss.conf.directory}" overwrite="true">
                      <fileset dir="${basedir}/src/test/${container.name}"/>
                    </copy>
                  </tasks>
                </configuration>
                <goals>
                  <goal>run</goal>
                </goals>
              </execution>
            </executions>
          </plugin>
        </plugins>
      </build>
    </profile>

Browser profiles

As with the application servers we have a profile for each browser

<profile>
      <id>firefox</id>
      <activation>
        <activeByDefault>true</activeByDefault>
      </activation>
      <properties>
        <browser>*firefox</browser>
      </properties>
    </profile>
    <profile>
      <id>iexplore</id>
      <properties>
        <browser>*iexplore</browser>
      </properties>
    </profile>
    <profile>
      <id>otherbrowser</id>
      <properties>
        <browser>*custom ${browserPath}</browser>
      </properties>
    </profile>

Enabling testing during development

Make selenium not to run in the background so we can execute tests from the IDE

    <profile>
      <id>dev</id>
      <properties>
        <selenium.background>false</selenium.background>
      </properties>
    </profile>

Repositories

Required for Selenium dependencies

<repositories>
    <repository>
      <id>openqa.org</id>
      <name>OpenQA Repository</name>
      <url>http://archiva.openqa.org/repository/releases</url>
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
      <releases>
        <enabled>true</enabled>
      </releases>
    </repository>
    <!-- for selenium 1.0-SNAPSHOT -->
    <repository>
      <id>snapshots.openqa.org</id>
      <name>OpenQA Sanpshots Repository</name>
      <url>http://archiva.openqa.org/repository/snapshots</url>
      <snapshots>
        <enabled>true</enabled>
      </snapshots>
      <releases>
        <enabled>false</enabled>
      </releases>
    </repository>
  </repositories>

Running in the build server

In an Unix server without X running you can still run Selenium tests using Xvfb (virtual framebuffer X server) by calling selenium:xvfb provided it’s properly configured.

Also you can pass the path to the browser binary if not in the PATH

mvn clean selenium:xvfb install -Dbrowser="*firefox /usr/lib64/firefox-1.5.0.12/firefox-bin"

The JUnit test

public class SeleniumHelloWorldTest
    extends TestCase
{
    private DefaultSelenium selenium;

    private String baseUrl;

    @Override
    public void setUp()
        throws Exception
    {
        super.setUp();
        String port = System.getProperty( "servlet.port" );
        baseUrl = "http://localhost:" + port;
        selenium = createSeleniumClient( baseUrl );
        selenium.start();
    }

    @Override
    public void tearDown()
        throws Exception
    {
        selenium.stop();
        super.tearDown();
    }

    protected DefaultSelenium createSeleniumClient( String url )
        throws Exception
    {
        String browser = System.getProperty( "browser" );
        String port = System.getProperty( "selenium.port" );
        return new DefaultSelenium( "localhost", Integer.parseInt( port ), browser, url );
    }

    public void testHelloWorld()
        throws Exception
    {
        selenium.open( baseUrl + "/mycontext/" );
        assertTrue( selenium.isTextPresent( "acme" ) );
    }
}

Debugging and troubleshooting (update)

You can check JBoss logs in target/logs/jboss42x.out and Selenium server logs in target/selenium/server.log

References

Other wiki entries and blogs

23 thoughts on “Functional testing with Maven, Cargo and Selenium

  1. Hi, Carlos:
    Thanks for kindly share. And could I ask you a question?
    I am wondering there is a way to run Selenium tests in background, that is to say when we run our tests, Selenium doesn’t open Firefox or IE, just run these tests in background, we only want to know whether our tests success, or just want some tasks automatically executed, and don’t want to notice them.

  2. Hi Carlos,

    Thank you for this article really well documented !

    I have a question about Firefox 3. I enabled the snapshot version for selenium-java-client-driver so that I can run it for my tests.
    However it doesn’t help, it is displayed in the console that the test is running but Firefox is never launched and the test is hanging…

    May be you have an idea about this ?

    Thanks,
    Marc.

  3. Thank you for the great example of Cargo usage. You blog is the first one where I found enough information to get this thing working like I needed. Thanks!

  4. Hi,
    Thanks for this wonderful information.
    I am able to successfully execute test cases in both html and junit test cases recorded using selenium.
    Now, as a next step of automation, was wondering if we can parameterize the values being passed to test cases. Say I have ten users and want to check if all of them can login with their userid and passwords, and want to do it with a single test case and values being picked up from some external entity, say a CSV file or something.

    Please let me know how I can proceed for this, any pointers would also be of great help.

  5. Harry, there are definitely ways to parametrize tests, but that sort of specific, customized help is what pays my bills 😉

    Please let me know if you are interested in getting some help from my company and will get in touch with you.

    Hope you understand

  6. Carlos, this is a great post. I have a question though, Is it possible to configure selenium to use a custom proxy? I’m having trouble with maven selenium plugin and firefox 3, cause it creates a new profile everytime the test is run but the proxy is not configured and my tests fails. Is there a way to fix this? Thanks!

  7. Hi. Very convenient and nicely described maven poms. Thanks a lot. It safes time. I started something like this time ago to deploy on JBoss before WS functional tests. But then I got a problem with auto installation of OC4J application server. Do you have any experience with that?

  8. I want to know how can we configure selenium sever to run on another host?
    Can i change the default localhost to any other host?
    if i can do this how selenium test would run?
    And will it be advantageous to run selenium server on different machine?

  9. Hi Carlos

    I am able to run my selenium test with maven on firefox browser.I use mvn clean verify command.

    But there is one problem.
    The firefox windows which are opened to run the selenium test,does not get closed.
    So next time when i run ‘mvn clean verify’ it says:
    “Unable to delete directory selenium”

    Then if i manually close firefox and again run the command it runs successfully.

    Please help me to solve this problem.

  10. Varun:

    you can start selenium in any host from maven with selenium-maven-plugin:start-server or start it standalone. Then change all references to localhost to the new server address. Or you can use selenium grid, see http://www.jroller.com/carlossg/entry/javaone_slides_enterprise_build_and

    newbea:

    you are probably not stopping the selenium session in your tests. In the example code you can see I stop it in the tearDown method that is automatically called by TestNG

  11. Thanx
    Actually i did not mention super.tear() in my tearDown method.After including it i have to include throws Exception and it works.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s