BDD With Spock and Selenium

Goal

To develop a web site specification-first using Spock and Selenium WebDriver.

tl;dr

The sample code for this project is available on github and will be a helpful resource for following this article. The interesting technologies showcased include Spock, Selenium WebDriver, Selenium PageObjects, Sauce OnDemand, Gradle and Grails 2.0.

The Project

Suppose our product owner would like to publish a website for teachers to schedule their classes and for students to register for those courses. At the first backlog grooming, the product owner has prioritized the stories that would constitute a minimum viable product.

User Stories

During the team's first sprint planning meeting, we sized the stories, accepted four into the sprint, and have added tasks for each. Our story board for the first sprint contains sticky notes for our user stories, the acceptance criteria and the individual tasks for each story.

As a teacher
I want to sign up
So I can add courses

Acceptance Criteria:

• User is greeted with intro screen
• User is able to register as a teacher or to login
• After registration or login, a teacher sees account info

As a student
I want to sign up
So I can take courses

Acceptance Criteria:

• User is able to register or to login as a student
• After registration or login, a student sees account info

As a teacher
I want to add courses
So students can register for them

Acceptance Criteria:

• A teacher can add a course
• Courses occur in a semester
• Courses occur in timeslots
• Courses can have prerequisites

As a student
I want to register for courses
So I can get credits toward my degree

Acceptance Criteria:

• A student can register for a course
• A student must take any pre-reqs before registering for a course
• A student cannot take 2 courses that occur in the same timeslot

Project Structure

We will create a top-level project course_registry and the two sub-projects web and specifications. We will also create a build.gradle file in the root along with settings.gradle and a build.gradle file in the specifications subproject for running automated acceptance tests (While this is not strictly necessary, it will help to keep our project boundaries clear).

  course_registry
   .
  |____build.gradle
  |____settings.gradle
  | specifications
  | |____build.gradle
  | web

course_registry/build.gradle

allprojects {
  apply plugin: 'eclipse'
  version = '1.0.0-SNAPSHOT'
  group = 'timezra.course_registry'
}

subprojects {
  sourceCompatibility = 1.6
}

course_registry/settings.gradle

include 'web', 'specifications'

course_registry/specifications/build.gradle

apply plugin: 'groovy'

repositories {
  mavenCentral()
  mavenRepo url: "http://m2repo.spockframework.org/snapshots"
}

dependencies {
  groovy group: 'org.codehaus.groovy', name: 'groovy', version: '1.8.5'
  testCompile group: 'org.spockframework', name: 'spock-core', version: '0.6-groovy-1.8-SNAPSHOT'
  testCompile group: 'org.seleniumhq.selenium', name: 'selenium-java', version: '2.16.1'
  testCompile group: 'junit', name: 'junit', version: '4.10'
}

We will create a src/test/groovy folder in the specifications project. At this point we can generate Eclipse .project and .classpath files for all the projects from the project root in order to work within an IDE.

course_registry $> gradle eclipseProject
course_registry $> gradle eclipseClasspath

The Teacher Registration Spec

We are now able to write the first specification for our acceptance criteria.

timezra/course_registry/TeacherRegistrationSpec.groovy

package timezra.course_registry

import static java.util.concurrent.TimeUnit.SECONDS
import org.openqa.selenium.WebDriver
import org.openqa.selenium.firefox.FirefoxDriver
import spock.lang.Specification

class TeacherRegistrationSpec extends Specification {

  WebDriver driver

  def setup() {
    driver = new FirefoxDriver()
    driver.manage().timeouts().implicitlyWait 10, SECONDS
  }

  def cleanup() {
    driver.quit()
  }

  def "a user is greeted with an intro screen"() {
    when:
    driver.get "http://localhost:8080/course_registry"

    then:
    driver.title == "Course Registry Home"
  }
}

We can run this spec and watch it fail.

course_registry $> gradle test --info
....
Test a user is greeted with an intro screen(timezra.course_registry.TeacherRegistrationSpec) FAILED: org.gradle.messaging.remote.internal.PlaceholderException: org.spockframework.runtime.SpockComparisonFailure: Condition not satisfied:

driver.title == "Course Registry Home"
|      |     |
|      |     false
|      |     17 differences (15% similarity)
|      |     (Pr)o(bl-)e(m) (load)i(ng--) (pag)e
|      |     (C-)o(urs)e(-) (Reg-)i(stry) (Hom)e
|      Problem loading page
org.openqa.selenium.firefox.FirefoxDriver@4532be10

Test timezra.course_registry.TeacherRegistrationSpec FAILED
Gradle Worker 1 finished executing tests.
1 test completed, 1 failure

FAILURE: Build failed with an exception.
....

Satisfying the Spec

Suppose the team decides to use Grails 2.0 to implement the specification. This is not a restriction based on our other technology choices, since gradle is a general purpose build tool and since our specification defines how a user will interact with our web project entirely through a browser. The decision is based on the convenience of the framework, community support, the plug-in ecosystem and the skills of the developers.

First, we need to create an empty web/grails-app directory to indicate to the grails bootstrap that the project will be a grails application.
Then, we will configure our web build for Grails 2.0 in a new web/build.gradle file.

web/build.gradle

buildscript {
  repositories {
    mavenCentral()
    mavenRepo url: 'https://repository.jboss.org/nexus/content/groups/public/'
    mavenRepo url: 'http://repo.grails.org/grails/repo'
  }

  dependencies {
    classpath group: 'org.grails', name: 'grails-gradle-plugin', version: '1.1.0'
  }
}

grailsVersion = '2.0.0'
apply plugin: 'grails'

dependencies {
  compile group: 'org.grails', name: 'grails-resources', version: grailsVersion
  compile group: 'org.grails', name: 'grails-crud', version: grailsVersion
  compile group: 'org.grails', name: 'grails-hibernate', version: grailsVersion
  compile group: 'org.grails', name: 'grails-plugin-datasource', version: grailsVersion
  runtime group: 'org.grails', name: 'grails-plugin-log4j', version: grailsVersion
  runtime group: 'org.grails', name: 'grails-plugin-url-mappings', version: grailsVersion
  runtime group: 'org.grails', name: 'grails-plugin-gsp', version: grailsVersion
  runtime group: 'org.grails', name: 'grails-plugin-filters', version: grailsVersion
  runtime group: 'org.grails', name: 'grails-plugin-scaffolding', version: grailsVersion
  runtime group: 'org.grails', name: 'grails-plugin-services', version: grailsVersion
  runtime group: 'org.grails', name: 'grails-plugin-servlets', version: grailsVersion
  runtime group: 'com.h2database', name: 'h2', version: '1.3.163'
  runtime group: 'net.sf.ehcache', name: 'ehcache-core', version: '2.4.6'
}

repositories {
  mavenCentral()
  mavenRepo url: 'https://repository.jboss.org/nexus/content/groups/public/'
  mavenRepo url: 'http://repo.grails.org/grails/repo'
}

From the project root we will initialize the grails project.

course_registry $> gradle grails-init
:web:grails-init
The ResolvedArtifact.getResolvedDependency() method is deprecated and will be removed in the next version of Gradle.
| Configuring classpath
| Error log4j:WARN No appenders could be found for logger (org.springframework.core.io.support.PathMatchingResourcePatternResolver).
| Error log4j:WARN Please initialize the log4j system properly.
| Error log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
| Environment set to development.....

BUILD SUCCESSFUL

Total time: 1 mins 36.328 secs

We can satisfy the spec simply by modifying application.properties with the expected web application deployment path and project version, the homepage to include the expected title, and messages.properties to contain the expected messages.

web/application.properties

app.grails.version=2.0.0
app.name=course_registry
app.servlet.version=2.5
app.version=1.0.0-SNAPSHOT

web/grails-app/views/index.gsp

<html>
  <head>
    <title><g:message code="home.title" /></title>
    <meta name="layout" content="main" />
    <style type="text/css" media="screen">
      #pageBody {
        margin-left: 280px;
        margin-right: 20px;
       }
    </style>
  </head>
  <body>
    <div id="pageBody" class="dialog">
     <p>
      <g:message code="home.welcome.message" />
     </p>
    </div>
  </body>
</html>

web/grails-app/i18n/messages.properties

....
home.title=Course Registry Home
home.welcome.message=Welcome to the course registry.

We can now add hooks to the specifications project to stop and start the web application before and after running the specifications, respectively.

specifications/build.gradle

....
test.dependsOn ':web:webStart'

gradle.taskGraph.afterTask { Task task, TaskState state ->
  if(':specifications:test' == task.path) {
    project(':web').tasks.getByPath('webStop').execute()
  }
}

These hooks depend on a specific interface in the web project, i.e., the existence of the tasks webStart and webStop. We can deploy the Grails 2.0 artifact to an embedded Jetty server through gradle to satisfy this interface.

web/build.gradle

....
apply plugin: 'jetty'

def stopPort = 8001
def safeWord = 'banana'

task webStart(dependsOn: 'grails-war') << {
  def jettyRunWar = tasks.getByPath('jettyRunWar')
  jettyRunWar.webApp = new File(projectDir, "target/course_registry-${version}.war")
  jettyRunWar.contextPath = 'course_registry'
  jettyRunWar.daemon = true
  jettyRunWar.stopPort = stopPort
  jettyRunWar.stopKey = safeWord
  jettyRunWar.execute()
}

task webStop << {
  def jettyStop = tasks.getByPath('jettyStop')
  jettyStop.stopPort = stopPort
  jettyStop.stopKey = safeWord
  jettyStop.execute()
}
....

The first specification should now be satisfied.

course_registry $> gradle test --info
....
Started Jetty Server
Gradle Worker 1 executing tests.
Test a user is greeted with an intro screen(timezra.course_registry.TeacherRegistrationSpec) PASSED
Gradle Worker 1 finished executing tests.
....

NB: The Jetty plugin must be applied before the grails plugin, or else you will see an error similar to the following:

FAILURE: Build failed with an exception.

* Where:
Build file '/path/to/course_registry/web/build.gradle' line: 38

* What went wrong:
A problem occurred evaluating project ':web'.
Cause: Cannot add task ':web:clean' as a task with that name already exists.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 7.624 secs

Adding another test for our second acceptance criterion should be straightforward now that our infrastructure is in place.

timezra/course_registry/TeacherRegistrationSpec.groovy

  ....
  def "a user can register as a teacher"() {
    when:
    driver.get "http://localhost:8080/course_registry"
    WebElement teacherLink = driver.findElement(By.id('teacher_link'))
    teacherLink.click()
    WebElement name = driver.findElement(By.id('name'))
    type name, 'John Doe'
    WebElement email = driver.findElement(By.id('email'))
    type email, "${UUID.randomUUID()}@rutgers.edu"
    WebElement password = driver.findElement(By.id('password'))
    type password, '1234567'
    WebElement create = driver.findElement(By.id('create'))
    create.click()

    then:
    driver.title == 'Show Teacher'
    driver.findElement(By.className('message')).text ==~ /Teacher \d+ created/
  }

  def type(field, text) {
    field.clear()
    field.sendKeys text
  }

We can satisfy this specification by generating a Grails Teacher domain object with fields for the name, email and password, along with a Grails controller that uses dynamic scaffolding to generate the view and actions available, and by adding a link to the scaffolded Create Teacher page on our web/grails-app/views/index.gsp. Since the point of this tutorial is not to cover basic Grails development, we do not need to go into those specifics here, but sample code can be found on the project site or in any Grails tutorial for further reference.

Page Objects

Before we get too far with automating our acceptance criteria, we should begin to look towards a more abstract representation of the testable components of our application. From the second acceptance test above, we can already see patterns emerging. For example, when we navigate to a particular page, it would be convenient to identify that we are on the correct page, perhaps by its title. It would also be convenient to represent each view in the application as its own object, and each view could encapsulate its specific WebElements. Finally, there is a small set of actions available for any WebElement on a page, and it would be helpful to build these actions into a testing DSL. Fortunately, by combining the Page Object pattern and Groovy's dynamic language features, we can achieve all these goals with a minimal amount of code.

As we are satisfying the specifications for all the acceptance criteria, a base Page Object emerges.

specifications/src/test/groovy/timezra/course_registry/pages/CourseRegistryPage.groovy

package timezra.course_registry.pages

import static org.apache.commons.lang.StringUtils.splitByCharacterTypeCamelCase

import org.openqa.selenium.WebDriver
import org.openqa.selenium.WebElement
import org.openqa.selenium.support.FindBy
import org.openqa.selenium.support.PageFactory
import org.openqa.selenium.support.ui.Select

abstract class CourseRegistryPage {

  WebDriver driver

  @FindBy(className = "message")
  WebElement message

  static <T extends CourseRegistryPage> T goTo(String address, WebDriver driver, Class<T> page) {
    driver.get address
    PageFactory.initElements driver, page
  }

  CourseRegistryPage(WebDriver driver) {
    this.driver = driver
    String title = splitByCharacterTypeCamelCase(getClass().simpleName).join ' '
    if(!title.equals(driver.title)) {
      throw new IllegalStateException("Should be on page '${title}' but was on page '${driver.title}' instead")
    }
  }

  def methodMissing(String name, args) {
    def m
    if((m = name =~ /click_(\w+)/)) {
      def webElement = this."${m[0][1]}"
      webElement.click()
      PageFactory.initElements driver, args[0]
    }
    else if((m = name =~ /type_(\w+)/)) {
      def webElement = this."${m[0][1]}"
      webElement.clear()
      webElement.sendKeys args[0]
    }
    else if((m = name =~ /choose_(\w+)/)) {
      def webElement = this."${m[0][1]}"
      def select = new Select(webElement)
      select.selectByVisibleText args[0]
    }
    else {
      throw new MissingMethodException(name, getClass(), args)
    }
  }
}

NB: The constructor verifies that the page is correct by comparing the class name to the page title; in addition, the static goTo method demonstrates how to use the PageFactory for initializing a Page Object; the methodMissing declaration also showcases Groovy's ability to call dynamic methods based on the combination of the click, type and choose actions with individual WebElement names. Such phrases become first-class elements of the testing DSL.

We can see the use of WebElement injection and dynamic methods in action with a CreateUser page that serves as the base for the CreateTeacher and CreateStudent pages.

specifications/src/test/groovy/timezra/course_registry/pages/CreateUser.groovy

package timezra.course_registry.pages

import groovy.transform.InheritConstructors

import org.openqa.selenium.WebDriver
import org.openqa.selenium.WebElement

@InheritConstructors
abstract class CreateUser extends CourseRegistryPage {
  WebElement name
  WebElement email
  WebElement password
  WebElement create

  CreateUser(WebDriver driver) {
    super(driver)
  }

  protected <T> T register(name, email, password, Class<T> nextPage) {
    type_name name
    type_email email
    type_password password
    click_create nextPage
  }
}

Other Browsers

So far, we have done all our testing with a single WebDriver, i.e., the FirefoxDriver. For simple testing, a single driver works fine, but if we wish to ensure that our website works in multiple browsers, we will need to configure our tests to use multiple WebDrivers. Fortunately, since we are using Gradle to manage our dependencies, the InternetExplorerDriver and ChromeDriver should be available to us automatically, along with the AndroidDriver and IPhoneDriver for testing our application on mobile devices. In addition, an OperaDriver is also available.

NB: Some of these drivers require that additional platform-specific software be installed, so please read the project pages for documentation on additional requirements.

Configuring our specifications to use multiple WebDrivers should just be a matter of parameterizing each Spock feature.

NB: Unlike JUnit tests which are parameterized by fixture, Spock specifications are parameterized per feature.

We will modify our TeacherRegistrationSpec to use the Spock where: block for parameterization, we will configure multiple WebDrivers, and we will share the instance driver field among the parameterized features.

timezra/course_registry/TeacherRegistrationSpec.groovy

class TeacherRegistrationSpec extends Specification {

  @Shared
  WebDriver driver

  def setup() {
    driver.manage().timeouts().implicitlyWait 10, SECONDS
  }

  def cleanup() {
    driver.quit()
  }

  def "a user is greeted with an intro screen"() {
    when:
    ....

    then:
    ....

    where:
    browser << browsers()
  }

  def "a user can register as a teacher"() {
    when:
    ....

    then:
    ....

    where:
    browser << browsers()
  }
  
  ....

  protected def browsers() {
    System.setProperty("webdriver.chrome.driver", "/path/to/chromedriver")
    def drivers = [
      new HtmlUnitDriver(),
      new FirefoxDriver(),
      new ChromeDriver()
    ]
    new Browsers(spec: this, delegate: drivers.iterator())
  }

  private static final class Browsers {
    @Delegate Iterator<WebDriver> delegate
    TeacherRegistrationSpec spec

    @Override WebDriver next() {
      spec.driver = delegate.next()
    }
  }
}

Testing In The Cloud

Now that our specifications are running through multiple browsers on a single machine, we can move a step further towards testing multiple browser versions on multiple OSes with cloud-based services such as Sauce Labs. For such a powerful service, the configuration changes to our existing application are surprisingly simple. We will modify our TeacherRegistrationSpec to use a RemoteWebDriver for the specific browser and OS combination we would like to test along with our Sauce Labs username and API key as described in the Sauce OnDemand documentation. We will also need to add a Gradle plugin to start SauceConnect from the build, just as we start our web application before running acceptance tests. Finally, we should ensure that all references to localhost in our application are changed to the IP address of the machine where we will be running our tests; otherwise, we might see the SauceConnect proxy freeze (you might have a different experience, and this freezing might just be attributable to the Gremlins in my machine).

timezra/course_registry/TeacherRegistrationSpec.groovy

class TeacherRegistrationSpec extends Specification {
  ....
  def "a user is greeted with an intro screen"() {
    when:
    driver.get "http://<your.ip.address>:8080/course_registry"
    ....
  }

  def "a user can register as a teacher"() {
    when:
    driver.get "http://<your.ip.address>:8080/course_registry/"
    ....
  }
  ....
  protected def browsers() {
    def capabilities = DesiredCapabilities.internetExplorer()
    capabilities.setCapability("version", "7")
    capabilities.setCapability("platform", Platform.XP)
    capabilities.setCapability("name", "Testing Teacher Registration in Sauce")

    def sauceDriver = new RemoteWebDriver(
        new URL(
        "http://<username>:<apiKey>@ondemand.saucelabs.com:80/wd/hub"),
        capabilities)
    def drivers = [sauceDriver]
    new Browsers(spec: this, delegate: drivers.iterator())
  }
  ....
}

course_registry/specifications/build.gradle

import java.util.concurrent.Executors
import java.util.concurrent.ExecutorService
....
repositories {
  ....
  mavenRepo url: "https://repository-saucelabs.forge.cloudbees.com/release"
}

dependencies {
  ....
  runtime group: 'com.saucelabs', name: 'sauce-connect', version: '3.0.18'
}

apply plugin: SauceConnect

sauceConfig {
  username = '<username>'
  apiKey = '<apiKey>'
}

test.dependsOn ':web:webStart', 'sauceConnect'

gradle.taskGraph.afterTask { Task task, TaskState state ->
  if(':specifications:test'.equals(task.path)) {
    tasks.getByPath('sauceDisconnect').execute()
    project(':web').tasks.getByPath('webStop').execute()
  }
}

class SauceConnect implements Plugin<Project> {
  
  ExecutorService executor = Executors.newFixedThreadPool(2)
  
  def void apply(Project project) {
    project.extensions.sauceConfig = new SauceConnectExtension()
    
    project.task('sauceConnect') << {
      
      def output = new PipedOutputStream()
      def input = new PipedInputStream(output)
      def reader = new BufferedReader(new InputStreamReader(input))
      
      executor.execute {
        println "Connecting to Sauce Labs as ${project.sauceConfig.username} with key ${project.sauceConfig.apiKey}...."
        try {
          project.javaexec {
            main = 'com.saucelabs.sauceconnect.SauceConnect'
            classpath = project.sourceSets.main.runtimeClasspath
            args = [project.sauceConfig.username, project.sauceConfig.apiKey]
            standardOutput = output
          } 
        } catch(Exception ignored) {
          // Executor has been shutdown
        }
      }
      boolean okToStart = false
      executor.execute {
        def nextLine
        try {
          while((nextLine = reader.readLine()) != null) {
            println nextLine
            if(!okToStart) {
              if(nextLine =~ /Please wait for "You may start your tests" to start your tests/) {
                continue
              }
              else if(nextLine =~ /You may start your tests/) {
                okToStart = true
              }
            }
          }
        } catch(Exception ignored) {
          // Executor has been shutdown
        }
      }
      while(!okToStart) {
        Thread.sleep 250
      }
    }
    
    project.task('sauceDisconnect') << {
      println "Disconnecting from Sauce Labs...."
      executor.shutdownNow()
    }
  }
}

class SauceConnectExtension {
  String username
  String apiKey
}

Conclusion

This post took us through the definition of user stories for a small website, the setup of a Gradle project for automating the website build and the run of its corresponding acceptance tests, the creation of our first WebDriver-based Spock specification for our acceptance criteria and the fulfillment of that spec with a Grails 2.0 application, the use of Selenium PageObjects to create abstract representations of our web pages before those web pages are even written, all the way to the creation of a Gradle plugin for running our acceptance tests on various browser/OS combinations in Sauce Labs. Each of those items in itself is worthy of a tutorial series. The combination of all these elements demonstrates with a remarkably small amount of configuration and code the type of robust and scalable ATDD infrastructure that is possible and should be at the core of any enterprise web application.