Copyright © 2011 John Ferguson Smart.

Online version published by Wakaleo Consulting.

This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States license. For more information about this license, see creativecommons.org/licenses/by-nc-nd/3.0/us/.

Java™ and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc., in the United States and other countries.

Eclipse™ is a trademark of the Eclipse Foundation, Inc., in the United States and other countries.

Apache and the Apache feather logo are trademarks of The Apache Software Foundation.

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and Wakaleo Consulting was aware of a trademark claim, the designations have been printed in caps or initial caps.

While every precaution has been taken in the preparation of this book, the publisher and authors assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein.

2. Introducing Thucydides

Thucydides (Thoo-SID-a-dees) is a tool designed to make writing automated acceptance and regression tests easier. It provides features that make it easier to organize and structure your acceptance tests, associating them with the user stories or features that they test. As the tests are executed, Thucydides generates illustrated documentation describing how the application is used based on the stories described by the tests.

Thucydides provides strong support for automated web tests based on Selenium 2, though it can also be used effectively for non-web tests.

Thucydides was a Greek historian known for his astute analysis skills who rigorously recorded events that he witnessed and participated in himself. In the same way, the Thucydides framework observes and analyzes your acceptance tests, and records a detailed account of their execution.

3. Basic concepts of Acceptance and Regression Testing

To get the most out of Thucydides, it is useful to understand some of the basic principles behind Acceptance Test Driven Development. Thucydides is commonly used for both Automated Acceptance Tests and Regression tests, and the principles discussed here apply, with minor variations, to both.

Acceptance Test Driven Development, or ATDD, is an advanced form of Test Driven Development (TDD) in which automated acceptance criteria — defined in collaboration with users — drive and focus the development process. This helps ensure that everyone understands what features are under development.

One of the important things about ATDD is the idea of "Specification by Example". Specification by Example refers to the use of relatively concrete examples to illustrate how a system should work, as opposed to more formally written specifications.

Let’s look at an example. In many projects, requirements are expressed as simple stories along the following lines:

In order to earn money to buy a new car
As a car owner
I want to sell my old car online

If we were implementing an online car sales web site that helps people achieve this goal, we would typically define a set of acceptance criteria to flesh out this story. For example, we might have the following criteria in our list of acceptance criteria:

  • Car owner can place a standard car ad online

  • Car owner can place a premium car ad online

  • Car ad should display the brand, model and year of the car

and so on.

A tester planning the tests for these acceptance criteria might draw up a test plan outlining of the way she expects to test these criteria. For the first criteria, she might start off with a high-level plan like the following:

  • Go to car ads section and choose to post a standard car ad

  • Enter car details

  • Choose publication options

  • Preview ad

  • Enter payment details

  • See ad confirmation

Each of these steps might need to be broken down into smaller steps.

  • Enter car details

    • Enter car make, model and year

    • Select options

    • Add photos

    • Enter description

These steps are often fleshed out with more concrete details:

  • Enter car details

    • Create an ad for a 2006 Mitsubishi Pajero

    • Add Air Conditioning and CD Player

    • Add three photos

    • Enter a description

For our purposes, Regression Tests can be defined as end-to-end tests that ensure that an application behaves as expected, and that it continues to behave as expected in future releases. Whereas ATDD Acceptance Tests are defined very early on in the piece, before development starts, Regression Tests involve an existing system. Other than that, the steps involved in defining and automating the tests are very similar.

Now different project stakeholders will be interested in different levels of detail. Some, such as project managers and management in general, will be interested only in which application features work, and which need to be done. Others, such as business analysts and QA, will be interested in the details of how each acceptance scenario is implemented, possible down to the screen level.

Thucydides helps you structure your automated acceptance tests into steps and sub-steps like the ones illustrated above. This tends to make the tests clearer, more flexible and easier to maintain. In addition, when the tests are executed, Thucydides produces illustrated, narrative-style reports like the one in [fig-test-report].

figs/test-report.png
Figure 1. A test report generated by Thucydides

When it comes to implementing the tests themselves, Thucydides also provides many features that make it easier, faster and cleaner to write clear, maintainable tests. This is particularly true for automated web tests using Selenium 2, but Thucydides also caters for non-web tests as well. Thucydides currently works well with JUnit and easyb - integration with other BDD frameworks is in progress.

4. Getting started with Thucydides

4.1. Creating a new Thucydides project

The easiest way to start a new Thucydides project is to use the Maven archetype. Two archetypes are currently available: one for using Thucydides with JUnit, and another if you also want to write your acceptance tests (or a part of them) using easyb. In this section, we will create a new Thucydides project using the Thucydides archetype, and go through the essential features of this project.

From the command line, you can run mvn archetype:generate and then select the net.thucydides.thucydides-easyb-archetype archetype from the proposed list of archetypes. Or you can use your favorite IDE to generate a new Maven project using an archetype.

$ mvn archetype:generate
...
Define value for property 'groupId': : com.mycompany
Define value for property 'artifactId': : webtests
Define value for property 'version':  1.0-SNAPSHOT: :
Define value for property 'package':  com.mycompany: :
Confirm properties configuration:
groupId: com.mycompany
artifactId: webtests
version: 1.0-SNAPSHOT
package: com.mycompany
 Y: :
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2:33.290s
[INFO] Finished at: Fri Oct 28 07:20:41 NZDT 2011
[INFO] Final Memory: 7M/81M
[INFO] ------------------------------------------------------------------------

This will create a simple Thucydides project, complete with a Page Object, a Step library and two test cases, one using JUnit, and one using easyb. The actual tests run against the online dictionary at Wiktionary.org. Before going any further, take the project for a spin. First, however, you will need to add the net.thucydides.maven.plugins to your plugin groups in your settings.xml file:

<settings>
   <pluginGroups>
       <pluginGroup>net.thucydides.maven.plugins</pluginGroup>
       ...
      </pluginGroups>
  ...
</settings>

This will let you invoke the Maven thucydides plugin from the command line in the short-hand form shown here. Now go into the generated project directory, run the tests and generate the reports:

$ mvn test thucydides:aggregate

This should run some web tests and generate a report in target/site/thucydides directory (open the index.html file).

If you drill down into the individual test reports, you will see an illustrated narrative for each test similar to the one shown in [fig-test-report]

Now for the details. the project directory structure is shown here:

+ src
   + main
      + java
         + com.mycompany.pages
            - HomePage.java

   + test
      + java
         + com.mycompany.pages
            + requirements
               - Application.java
            + steps
               - EndUserSteps.java
            - LookupADefinitionStoryTest.java

      + stories
         + com.mycompany
            - LookupADefinition.story

This project is designed to provide a starting point for your Thucydides acceptance tests, and to illustrate some of the basic features. The tests come in two flavors: easyb and JUnit. easyb is a Groovy-based BDD (Behaviour Driven Development) library which works well for this kind of test. The sample easyb story can be found in the LookupADefinition.story file, and looks something like this:

using "thucydides"

thucydides.uses_default_base_url "http://en.wiktionary.org/wiki/Wiktionary:Main_Page"
thucydides.uses_steps_from EndUserSteps
thucydides.tests_story SearchByKeyword

scenario "Looking up the definition of 'apple'", {
    given "the user is on the Wikionary home page", {
        end_user.is_the_home_page()
    }
    when "the end user looks up the definition of the word 'apple'", {
        end_user.looks_for "apple"
    }
    then "they should see the definition of 'apple", {
       end_user.should_see_definition_containing_words "A common, round fruit"
    }
}

A cursory glance at this story will show that it relates a user looking up the definition of the word apple. However only the “what” is expressed at this level – the details are hidden inside the test steps and, further down, inside page objects.

If you prefer pure Java tests, the JUnit equivalent can be found in the LookupADefinitionStoryTest.java file:

@Story(Application.Search.SearchByKeyword.class)
@RunWith(ThucydidesRunner.class)
public class LookupADefinitionStoryTest {

    @Managed(uniqueSession = true)
    public WebDriver webdriver;

    @ManagedPages(defaultUrl = "http://en.wiktionary.org/wiki/Wiktionary:Main_Page")
    public Pages pages;

    @Steps
    public EndUserSteps endUser;

    @Issue("#WIKI-1")
    @Test
    public void looking_up_the_definition_of_apple_should_display_the_corresponding_article() {
        endUser.is_the_home_page();
                endUser.looks_for("apple");
        endUser.should_see_definition_containing_words("A common, round fruit");

    }
}

As you can see, this is a little more technical but still very high level.

The step libraries contain the implementation of each of the steps used in the high-level tests. For complex tests, these steps can in turn call other steps. The step library used in this example can be found in EndUserSteps.java:

public class EndUserSteps extends ScenarioSteps {

        public EndUserSteps(Pages pages) {
                super(pages);
        }

    @Step
    public void searches_by_keyword(String keyword) {
        enters(keyword);
        performs_search();
    }

        @Step
        public void enters(String keyword) {
        onHomePage().enter_keywords(keyword);
        }

    @Step
    public void performs_search() {
        onHomePage().starts_search();
    }

    private HomePage onHomePage() {
        return getPages().currentPageAt(HomePage.class);
    }

    @Step
        public void should_see_article_with_title(String title) {
        assertThat(onHomePage().getTitle(), is(title));
        }

    @Step
    public void is_on_the_wikipedia_home_page() {
        onHomePage().open();
    }
}

Page Objects are a way of encapsulating the implementation details about a particular page. Selenium 2 has particularly good support for page objects, and Thucydides leverages this. The sample page object can be found in the HomePage.java class:

@DefaultUrl("http://en.wiktionary.org/wiki/Wiktionary:Main_Page")
public class SearchPage extends PageObject {

    @FindBy(name="search")
        private WebElement searchInput;

        @FindBy(name="go")
        private WebElement searchButton;

        public SearchPage(WebDriver driver) {
                super(driver);
        }

        public void enter_keywords(String keyword) {
                searchInput.sendKeys(keyword);
        }

    public void starts_search() {
        searchButton.click();
    }

    public List<String> getDefinitions() {
        WebElement definitionList = getDriver().findElement(By.tagName("ol"));
        List<WebElement> results = definitionList.findElements(By.tagName("li"));
        return convert(results, new ExtractDefinition());
    }

    class ExtractDefinition implements Converter<WebElement, String> {
        public String convert(WebElement from) {
            return from.getText();
        }
    }
}

The final piece in the puzzle is the Application.java class, which is a way of representing the structure of your requirements in Java form, so that your easyb and JUnit tests can be mapped back to the requirements they are testing:

        public class Application {
            @Feature
            public class Search {
                public class SearchByKeyword {}
                public class SearchByAnimalRelatedKeyword {}
                public class SearchByFoodRelatedKeyword {}
                public class SearchByMultipleKeywords {}
                public class SearchForQuote{}
            }

            @Feature
            public class Backend {
                public class ProcessSales {}
                public class ProcessSubscriptions {}
            }

            @Feature
            public class Contribute {
                public class AddNewArticle {}
                public class EditExistingArticle {}
            }
        }

This is what enables Thucydides to generate the aggregate reports about features and stories.

In the following sections, we will look at different aspects of writing automated tests with Thucydides in more detail.

5. Writing Acceptance Tests with Thucydides

In this section, we look at the things you need to know to write your acceptance or regression tests using Thucydides in more detail. We will also outline a general approach to writing your web-based acceptance tests that has worked well for us in the past.

  1. Define and organize the requirements or user stories you need to test

  2. Write high level pending tests for the acceptance criteria

  3. Choose a test to implement, and break it into a small (typically between 3 and 7) high-level steps

  4. Implement these steps, either by breaking them down into other steps, or by accessing Page Objects.

  5. Implement any new Page Object methods that you have discovered.

Note

These steps should not been seen as a linear or waterfall-style approach. Indeed, the process is usually quite incremental, with requirements being added to the Application class as they are required, and pending tests being used to defined tests before they are fleshed out.

5.1. Organizing your requirements

To get the most out of automated tests in Thucydides, you need to tell Thucydides which features of your application you are testing in each test. While this step is optional, it is highly recommended.

The current version of Thucydides uses a simple, three-level organization to structure acceptance tests into more manageable chunks. At the highest level, an application is broken into features, which is a high-level functionality or group of related functions. A feature contains a number of stories (corresponding to user stories, use cases, and so on). Each story is validated by a number of examples, or acceptance criteria, which are automated in the form of web tests (sometimes called scenarios). Each test, in turn, is implemented using a number of steps.

Of course this structure and these terms are merely a convenience to allow a higher-level vision of your acceptance tests. However, this sort of three-level abstraction seems to be fairly common.

In the current version of Thucydides, you define this structure within the test code, as (very light-weight) Java classes
[Future versions of Thucydides will support other ways of defining your user requirements.]
. This makes it easier to refactor and rename user stories and features within the tests, and gives a central point of reference in the test suite illustrating what features are being tested. A simple example is shown here. The Application class is simply a convenient way of placing the features and user stories in the one file. Features are marked with the @Feature annotation. User stories are declared as inner classes nested inside a @Feature class.

public class Application {

    @Feature
    public class ManageCompanies {
        public class AddNewCompany {}
        public class DeleteCompany {}
        public class ListCompanies {}
    }

    @Feature
    public class ManageCategories {
        public class AddNewCategory {}
        public class ListCategories {}
        public class DeleteCategory {}
    }

    @Feature
    public class ManageTags {
        public class DisplayTagCloud {}
    }

    @Feature
    public class ManageJobs {}

    @Feature
    public class BrowseJobs {
        public class UserLookForJobs {}
        public class UserBrowsesJobTabs {}
    }
}

6. Defining high-level tests

There are two approaches to automated acceptance criteria or regression tests with Thucydides. Both involve implementing the tests as a sequence of very high-level steps, and then fleshing out those steps by drilling down into the details, until you get to the Page Objects. The difference involves the language used to implement the high-level tests. Tools like easyb are more focused on communication with non-developers, and allow high level tests to be expressed more easily in business terms. On the other hand, developers often find it more comfortable to work directly with JUnit, so if communication with non-technical stakeholders is not a high priority, this might be a preferred option.

In the current version of Thucydides, you can write your tests using easyb (for a more BDD-style approach) or in JUnit using Java or another JVM language (Groovy is a popular choice). Other BDD tools will be supported in future versions. We will discuss both here, but you can use whatever you and your team are more comfortable with.

6.1. Defining high-level tests in easyb

Easyb (http://easyb.org) is a Groovy-based BDD tool. It makes it easy to write light-weight stories and scenarios using the classic BDD-style "given-when-then" structure, and then to implement them in Groovy. The Thucydides easyb plugin is designed to make it easy to write Thucydides tests using easyb.

6.1.1. Writing a pending easyb story

In easyb, you write tests (referred to as "scenarios") that, when using Thucydides, correspond to automated acceptance criteria. Tests are grouped into "stories" - each story has it’s own file.

Scenarios are first written as "pending". These are just high-level outlines, describing a set of acceptance criteria for a particular story in a "given-when-then" structure.

When the tests are executed, pending scenarios are skipped. However they appear in the reports, so that you know what features still need to be implemented. An example of how pending scenarios appear in a Thucydides report can be seen in [fig-story-results-pending].

figs/story-results-pending.png
Figure 2. Pending tests are shown with the calendar icon

Here is an example of a pending easyb story using Thucydides:

using "thucydides"

import net.thucydides.demos.jobboard.requirements.Application.ManageCategories.AddNewCategory

thucydides.tests_story AddNewCategory

scenario "The administrator adds a new category to the system",
{
        given "a new category needs to be added to the system"
        when "the administrator adds a new category"
        then "the system should confirm that the category has been created"
        and "the new category should be visible to job seekers"
}

scenario "The administrator adds a category with an existing code to the system",
{
        given "the administrator is on the categories list page"
        when "the user adds a new category with an existing code"
        then "an error message should be displayed"
}

Let’s examine this story piece-by-piece. First, you need to declare that you are using Thucydides. You do this by using the easyb using keyword:

using "thucydides"

This will, among other things, inject the thucydides object into your story context so that you can configure Thucydides to run your story correctly.

Next, you need to tell Thucydides what story you are testing. You do this by referencing one of the story classes you defined earlier. That’s what we are doing here:

import net.thucydides.demos.jobboard.requirements.Application.ManageCategories.AddNewCategory

thucydides.tests_story AddNewCategory

The rest of the easyb story is just a set of regular easyb pending scenarios. For the moment, there is no implementation, so they will appear as "pending" in the reports:

scenario "The administrator adds a new category to the system",
{
        given "a new category needs to be added to the system"
        when "the administrator adds a new category"
        then "the system should confirm that the category has been created"
        and "the new category should be visible to job seekers"
}

scenario "The administrator adds a category with an existing code to the system",
{
        given "the administrator is on the categories list page"
        when "the user adds a new category with an existing code"
        then "an error message should be displayed"
}

You typically declare many pending stories, preferably in collaboration with the product owner or BAs, at the start of an iteration. This lets you get a good picture of what stories need to be implemented in a given iteration, and also an idea of the relative complexity of each story.

6.1.2. Implementing the easyb stories

The next step is to implement your stories. Let’s look at an implemented version of the first of these scenarios:

using "thucydides"

import net.thucydides.demos.jobboard.requirements.Application.ManageCategories.AddNewCategory
import net.thucydides.demos.jobboard.steps.AdministratorSteps
import net.thucydides.demos.jobboard.steps.JobSeekerSteps

thucydides.uses_default_base_url "http://localhost:9000"
thucydides.uses_steps_from AdministratorSteps
thucydides.uses_steps_from JobSeekerSteps
thucydides.tests_story AddNewCategory

def cleanup_database() {
    administrator.deletes_category("Scala Developers");
}

scenario "The administrator adds a new category to the system",
{
    given "a new category needs to be added to the system",
    {
      administrator.logs_in_to_admin_page_if_first_time()
      administrator.opens_categories_list()
    }
    when "the administrator adds a new category",
    {
       administrator.selects_add_category()
       administrator.adds_new_category("Scala Developers","SCALA")
    }
    then "the system should confirm that the category has been created",
    {
        administrator.should_see_confirmation_message "The Category has been created"
    }
    and "the new category should be visible to job seekers",
    {
        job_seeker.opens_jobs_page()
        job_seeker.should_see_job_category "Scala Developers"
    }
}

Again, let’s break this down. In the first section, we import the classes we need to use:

using "thucydides"

import net.thucydides.demos.jobboard.requirements.Application.ManageCategories.AddNewCategory
import net.thucydides.demos.jobboard.steps.AdministratorSteps
import net.thucydides.demos.jobboard.steps.JobSeekerSteps

Next, we declare the default base URL to be used for the tests. Like the equivalent annotation in the JUnit tests, this is used for tests executed from within the IDE, or if no base URL is defined on the command line using the webdriver.base.url parameter.

thucydides.uses_default_base_url "http://localhost:9000"

We also need to declare the test step libraries we will be using. We do this using thucydides.uses_steps_from. This will inject an instance variable into the easyb context for each declared step library. If the step library class name ends in Steps (e.g. JobSeekerSteps), the name of the variable will be the class name less the Steps suffix, converted to lower case and underscores (e.g. "job_seeker"). We will learn more about implementing test step libraries further on.

thucydides.uses_steps_from AdministratorSteps
thucydides.uses_steps_from JobSeekerSteps
thucydides.tests_story AddNewCategory

Finally we implement the scenario. Notice, that since this is Groovy, we can declare fixture methods to help set up and tear down the test environment as required:

def cleanup_database() {
    administrator.deletes_category("Scala Developers");
}

The implementation usually just invokes step methods, as illustrated here:

scenario "The administrator adds a new category to the system",
{
    given "a new category needs to be added to the system",
    {
      administrator.logs_in_to_admin_page_if_first_time()
      administrator.opens_categories_list()
    }
    when "the administrator adds a new category",
    {
       administrator.selects_add_category()
       administrator.adds_new_category("Scala Developers","SCALA")
    }
    then "the system should confirm that the category has been created",
    {
        administrator.should_see_confirmation_message "The Category has been created"
    }
    and "the new category should be visible to job seekers",
    {
        job_seeker.opens_jobs_page()
        job_seeker.should_see_job_category "Scala Developers"
        cleanup_database()
    }
}

6.2. Defining high-level tests in JUnit

Thucydides integrates smoothly with ordinary JUnit 4 tests, using the ThucydidesRunner test runner and a few other specialized annotations. This is one of the easiest ways to start out with Thucydides, and is very well suited for regression testing, where communication and clarification with the various stakeholders is less of a requirement.

Here is an example of a Thucydides JUnit web test:

@RunWith(ThucydidesRunner.class)
@Story(UserLookForJobs.class)
public class LookForJobsStory {

    @Managed
    public WebDriver webdriver;

    @ManagedPages(defaultUrl = "http://localhost:9000")
    public Pages pages;

    @Steps
    public JobSeekerSteps job_seeker;

    @Test
    public void user_looks_for_jobs_by_key_word() {
        job_seeker.opens_jobs_page();
        job_seeker.searches_for_jobs_using("Java");
        job_seeker.should_see_message("No jobs found.");
    }

    @Test
    public void when_no_matching_job_found_should_display_error_message() {
        job_seeker.opens_jobs_page();
        job_seeker.searches_for_jobs_using("unknownJobCriteria");
        job_seeker.should_see_message("No jobs found.");
    }

    @Pending @Test
    public void tags_should_be_displayed_to_help_the_user_find_jobs() {}

    @Pending @Test
    public void the_user_can_list_all_of_the_jobs_for_a_given_tag() {}

    @Pending @Test
    public void the_user_can_see_the_total_number_of_jobs_on_offer() {}

}

Let’s examine this section-by-section. The class starts with the @RunWith annotation, to indicate that this is a Thucydides test. We also use the @Story annotation to indicate which user story (defined as nested classes of the the @Feature classes above) is being tested. This is used to generate the aggregate reports.

@RunWith(ThucydidesRunner.class)
@Story(UserLookForJobs.class)
public class LookForJobsStory {
    ...

Next, come two essential annotations for any web tests. First of all, your test case needs a public Webdriver field, annotated with the @Managed annotation. This enables Thucydides to take care of opening and closing a WebDriver driver for you, and lets Thucydides use this driver in the pages and test steps when the tests are executed:

    @Managed
    public WebDriver webdriver;

The second essential field is an instance of the Pages class, annotated with the @ManagedPages annotation. This is essentially a page factory, that Thucydides uses to provide you with instantiated page objects. The defaultUrl attribute lets you define a URL to use when your pages open, if no other base URL has been defined. This is useful for IDE testing:

    @ManagedPages(defaultUrl = "http://localhost:9000")
    public Pages pages;

Note that these two annotations are only required for web tests. If your Thucydides test does not use web tests, you can safely leave them out.

For high-level acceptance or regression tests, it is a good habit to define the high-level test as a sequence of high-level steps. It will make your tests more readable and easier to maintain if you delegate the implementation details of your test (the "how") to reusable "step" methods. We will discuss how to define these step methods later. However, the minimum you need to do is to define the class where the steps will be defined, using the @Steps annotation. This annotation tells Thucydides to listen to method calls on this object, and (for web tests) to inject the WebDriver instance and the page factory into the Steps class so that they can be used in the step methods.

    @Steps
    public JobSeekerSteps job_seeker;

6.2.1. Pending tests

Tests that contain no steps are considered to be pending. Alternatively, you can force a step to be skipped (and marked as pending) by using the @Pending annotation or the @Ignore annotation. Note that the semantics are slightly different: @Ignore indicates that you are temporarily suspending execution of a test, whereas @Pending means that the test has been specified but not yet implemented. So both these tests will be pending:

@Test
public void administrator_adds_an_existing_company_to_the_system() {}

@Pending @Test
public void administrator_adds_a_company_with_an_existing_code_to_the_system() {
    steps.login_to_admin_page();
    steps.open_companies_list();
    steps.select_add_company();
    // More to come
}

A test is also considered pending if any of the steps used in that test are pending. For a step to be pending, it needs to be annotated with the @Pending annotation.

6.2.2. Running tests in a single browser session

Normally, Thucydides opens a new browser session for each test. This helps ensure that each test is isolated and independent. However, sometimes it is useful to be able to run tests in a single browser session, in particular for performance reasons on read-only screens. You can do this by using the uniqueSession attribute in the @Managed annotation, as shown below. In this case, the browser will open at the start of the test case, and not close until all of the tests have been executed.

@RunWith(ThucydidesRunner.class)
public class OpenStaticDemoPageSample {

    @Managed(uniqueSession=true)
    public WebDriver webdriver;

    @ManagedPages(defaultUrl = "classpath:static-site/index.html")
    public Pages pages;

    @Steps
    public DemoSiteSteps steps;

    @Test
    @Title("The user opens the index page")
    public void the_user_opens_the_page() {
        steps.should_display("A visible title");
    }

    @Test
    @Title("The user selects a value")
    public void the_user_selects_a_value() {
        steps.enter_values("Label 2", true);
        steps.should_have_selected_value("2");
    }

    @Test
    @Title("The user enters different values.")
    public void the_user_opens_another_page() {
        steps.enter_values("Label 3", true);
        steps.do_something();
        steps.should_have_selected_value("3");
    }
}

If you do not need WebDriver support in your test, you can skip the @Managed and @Pages annotations, e.g.

@RunWith(ThucydidesRunner.class)
@Story(Application.Backend.ProcessSales.class)
public class WorkWithBackendTest {

    @Steps
    public BackendSteps backend;

    @Test
    public void when_processing_a_sale_transation() {
        backend.accepts_a_sale_transaction();
        backend.should_the_update_mainframe();
    }
}

6.3. Running Thucydides in different browsers

Thucydides supports all browser-based WebDriver drivers, i.e. Firefox, Internet Explorer and Chrome, as well as HTMLUnit. By default, it will use Firefox. However, you can override this option using the webdriver.driver system property. To set this from the command line, you could do the following:

$ mvn test -Dwebdriver.driver=iexplorer

If you are not using Firefox by default, it is also useful to define this variable as a property in your Maven pom.xml file, e.g.

<properties>
    <webdriver.driver>iexplorer</webdriver.driver>
</properties>

For this to work with JUnit, however, you need to pass the webdriver.driver property to JUnit. JUnit runs in a separate JVM, and will not see the system properties defined in the Maven build. To get around this, you need to pass them into JUnit explicitly using the systemPropertyVariables configuration option, e.g.

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.7.1</version>
            <configuration>
                <systemPropertyVariables>
                    <webdriver.driver>${webdriver.driver}</webdriver.driver>
                </systemPropertyVariables>
            </configuration>
        </plugin>

6.4. Forcing the use of a particular driver in a test case or test

The @Managed annotation also lets you specify what driver you want to use for a particular test case, via the driver attribute. Current supported values are “firefox”, “iexplorer”, “chrome” and “htmlunit”. The driver attribute lets you override the system-level default driver for specific requirements. For example, the following test case will run in Chrome, regardless of the webdriver.driver system property value used:

@RunWith(ThucydidesRunner.class)
@Story(Application.Search.SearchByKeyword.class)
public class SearchByFoodKeywordStoryTest {

    @Managed(uniqueSession = true, driver="chrome")
    public WebDriver webdriver;

    @ManagedPages(defaultUrl = "http://www.google.co.nz")
    public Pages pages;

    @Steps
    public EndUserSteps endUser;

    @Test
    public void searching_by_keyword_pears_should_display_the_corresponding_article() {
        endUser.is_the_google_home_page();
            endUser.enters("pears");
        endUser.starts_search();
            endUser.should_see_article_with_title_containing("Pear");
    }

    @Test
    @WithDriver("firefox")
    public void searching_by_keyword_pineapples_should_display_the_corresponding_article() {
        endUser.is_the_google_home_page();
            endUser.enters("pineapples");
        endUser.starts_search();
        endUser.should_see_article_with_title_containing("Pineapple");
    }
}

In easyb, you can use the uses_driver directive, as shown here:

using "thucydides"
...
thucydides.uses_default_base_url "http://localhost:9000"
thucydides.uses_driver chrome

...

scenario "The administrator adds a new category to the system",
{
    given "a new category needs to be added to the system",
    {
      administrator.logs_in_to_admin_page_if_first_time()
      administrator.opens_categories_list()
    }
    when "the administrator adds a new category",
    {
       administrator.selects_add_category()
       administrator.adds_new_category("Scala Developers","SCALA")
    }
    then "the system should confirm that the category has been created",
    {
        administrator.should_see_confirmation_message "The Category has been created"
    }
    and "the new category should be visible to job seekers",
    {
        job_seeker.opens_jobs_page()
        job_seeker.should_see_job_category "Scala Developers"
    }
}

In JUnit, you can also use the @WithDriver annotation to specify a driver for an individual test. This will override both the system-level driver and the @Managed annotation’s driver attribute, if provided. For example, the following test will always run in Firefox:

        @Test
        @WithDriver("firefox")
        public void searching_by_keyword_pineapples_should_display_the_corresponding_article() {
            endUser.is_the_google_home_page();
            endUser.enters("pineapples");
            endUser.starts_search();
            endUser.should_see_article_with_title_containing("Pineapple");
        }

7. Writing Acceptance Tests with JBehave

Thucydides is an open source library designed to make it easier to define, implement and report on automated acceptance criteria. Until now, Thucydides tests have been implemented using JUnit or easyb. However the most recent version of Thucydides, version 0.9.x, now lets you write your acceptance criteria using the popular JBehave framework.

7.1. JBehave and Thucydides

JBehave is an open source BDD framework originally written by Dan North, the inventor of BDD. It is strongly integrated into the JVM world, and widely used by Java development teams wanting to implement BDD practices in their projects.

In JBehave, you write automate your acceptance criteria by writing test stories and scenarios using the familiar BDD "given-when-then" notation, as shown in the following example:

Scenario: Searching by keyword and category

Given Sally wants to buy some antique stamps for her son
When she looks for ads in the 'Antiques' category containing 'stamps'
Then she should obtain a list of ads related to 'stamps' from the 'Antiques' category

Scenarios like this go in .story files: a story file is designed to contain all the scenarios (acceptence criteria) of a given user story. A story file can also have a narrative section at the top, which gives some background and context about the story being tested:

In order to find the items I am interested in faster
As a buyer
I want to be able to list all the ads with a particular keyword in the description or title.

Scenario: Searching by keyword and category

Given Sally wants to buy some antique stamps for her son
When she looks for ads in the 'Antiques' category containing 'stamps'
Then she should obtain a list of ads related to 'stamps' from the 'Antiques' category

Scenario: Searching by keyword and location

Given Sally wants to buy a puppy for her son
When she looks for ads in the Pets & Animals category containing puppy in New South Wales
Then she should obtain a list of Pets & Animals ads containing the word puppy
  from advertisers in New South Wales

You usually implement a JBehave story using classes and methods written in Java, Groovy or Scala. You implement the story steps using annotated methods to represent the steps in the text scenarios, as shown in the following example:

public class SearchSteps {
    @Given("Sally wants to buy a $gift for her son")
    public void sally_wants_to_buy_a_gift(String gift) {
        // test code
    }

    @When("When she looks for ads in the $category category containing $keyword in $region")
    public void looking_for_an_ad(String category, String keyword, String region){
        // more test code
    }
}

7.2. Working with JBehave and Thucydides

Thucydides and JBehave work well together. Thucydides uses simple conventions to make it easier to get started writing and implementing JBehave stories, and reports on both JBehave and Thucydides steps, which can be seamlessly combined in the same class, or placed in separate classes, depending on your preferences.

To get started, you will need to add the Thucydides JBehave plugin to your project. In Maven, just add the following dependencies to your pom.xml file:

<dependency>
    <groupId>net.thucydides</groupId>
    <artifactId>thucydides-core</artifactId>
    <version>0.9.2</version>
</dependency>
<dependency>
    <groupId>net.thucydides</groupId>
    <artifactId>thucydides-jbehave-plugin</artifactId>
    <version>0.9.0</version>
</dependency>

New versions come out regularly, so be sure to check the Maven Central repository (http://search.maven.org) to know the latest version numbers for each dependency.

7.3. Setting up your project and organizing your directory structure

JBehave is a highly flexible tool. The downside of this is that, out of the box, JBehave requires quite a bit of bootstrap code to get started. Thucydides tries to simplify this process by using a convention-over-configuration approach, which significantly reduces the amount of work needed to get started with your acceptance tests. In fact, you can get away with as little as an empty JUnit test case and a sensibly-organized directory structure for your JBehave stories.

7.3.1. The JUnit test runner

The JBehave tests are run via a JUnit runner. This makes it easier to run the tests both from within an IDE or as part of the build process. All you need to do is to extend the ThucydidesJUnitStories, as shown here:

package net.thucydides.showcase.jbehave;

import net.thucydides.jbehave.ThucydidesJUnitStories;

public class JBehaveTestCase extends ThucydidesJUnitStories {
    public JBehaveTestCase() {}
}

When you run this test, Thucydides will run any JBehave stories that it finds in the default directory location. By convention, it will look for a ‘stories` folder on your classpath, so `src/test/resources/stories’ is a good place to put your story files.

7.3.2. Organizing your requirements

Placing all of your JBehave stories in one directory does not scale well; it is generally better to organize them in a directory structure that groups them in some logical way. In addition, if you structure your requirements well, Thucydides will be able to provide much more meaningful reporting on the test results.

By default, Thucydides supports a simple directory-based convention for organizing your requirements. The standard structure uses three levels: capabilities, features and stories. A story is represented by a JBehave .story file so two directory levels underneath the stories directory will do the trick. An example of this structure is shown below:

+ src
  + test
    + resources
      + stories
        + grow_potatoes                     [a capability]
          + grow_organic_potatoes           [a feature]
            - plant_organic_potatoes.story  [a story]
            - dig_up_organic_potatoes.story [another story]
          + grow_sweet_potatoes             [another feature]
          ...

If you prefer another hierarchy, you can use the thucydides.capability.types system property to override the default convention. For example. if you prefer to organize your requirements in a hierachy consisting of epics, theme and stories, you could set the thucydides.capability.types property to epic,theme (the story level is represented by the .story file).

When you start a project, you will typically have a good idea of the capabilities you intent to implement, and probably some of the main features. If you simply store your .story files in the right directory structure, the Thucydides reports will reflect these requirements, even if no tests have yet been specified for them. This is an excellent way to keep track of project progress. At the start of an iteration, the reports will show all of the requirements to be implemented, even those with no tests defined or implemented yet. As the iteration progresses, more and more acceptance criteria will be implemented, until acceptance criteria have been defined and implemented for all of the requirements that need to be developed.

figs/jbehave-initial-project.png
Figure 3. A Thucyides project using JBehave can organize the stories in an appropriate directory structure

An optional but useful feature of the JBehave story format is the narrative section that can be placed at the start of a story to help provide some more context about that story and the scenarios it contains. This narrative will appear in the Thucydides reports, to help give product owners, testers and other team members more information about the background and motivations behind each story. For example, if you are working on an online classifieds website, you might want users to be able to search ads using keywords. You could describe this functionality with a textual description like this one:

Story: Search for ads by keyword
In order to find the items I am interested in faster
As a buyer
I want to be able to list all the ads with a particular keyword
in the description or title.

However to make the reports more useful still, it is a good idea to document not only the stories, but to also do the same for your higher level requirements. In Thucydides, you can do this by placing a text file called narrative.txt in each of the requirements directories you want to document (see below). These files follow the JBehave/Cucumber convention for writing narratives, with an optional title on the first line, followed by a narrative section started by the keyword Narrative:. For example, for a search feature for an online classifieds web site, you might have a description along the following lines:

Search for online ads

Narrative:
In order to increase sales of advertised articles
As a seller
I want potential buyers to be able to display only the ads for
articles that they might be interested in purchasing.

When you run these stories (without having implemented any actual tests), you will get a report containing lots of pending tests, but more interestingly, a list of the requirements that need to be implemented, even if there are no tests or stories associated with them yet. This makes it easier to plan an iteration: you will initially have a set of requirements with only a few tests, but as the iteration moves forward, you will typically see the requirements fill out with pending and passing acceptance criteria as work progresses.

figs/jbehave-requirements-report.png
Figure 4. You can see the requirements that you need to implement n the requirements report

7.3.3. Customizing the requirements module

You can also easily extend the Thucydides requirements support so that it fits in to your own system. This is a two-step process. First, you need to write an implementation of the RequirementsTagProvider interface.

package com.acme.tests

public class MyRequirementsTagProvider implements RequirementsTagProvider {
    @Override
    public List<Requirement> getRequirements() {
        // Return the full list of available requirements from your system
    }

    @Override
    public Optional<Requirement> getParentRequirementOf(TestOutcome testOutcome) {
        // Return the requirement, if any, associated with a particular test result
    }

    @Override
    public Set<TestTag> getTagsFor(TestOutcome testOutcome) {
        // Return all the requirements, and other tags, associated with a particular test result
    }
}

Next, create a text file in your src/main/resources/META-INF/serices directory called net.thucydides.core.statistics.service.TagProvider, and put the fullly qualified name of your RequirementsTagProvider implementation.

7.3.4. Story meta-data

You can use the JBehave Meta tag to provide additional information to Thucydides about the test. The @driver annotation lets you specify what WebDriver driver to use, eg.

Meta:
@driver htmlunit

Scenario: A scenario that uses selenium

Given I am on the test page
When I enter the first name <firstname>
And I enter the last name <lastname>
Then I should see <firstname> and <lastname> in the names fields
And I should be using HtmlUnit

Examples:
|firstname|lastname|
|Joe      | Blow|
|John     | Doe   |

You can also use the @issue annotation to link scenarios with issues, as illustrated here:

Meta:
@issue MYPROJ-1, MYPROJ-2

Scenario: A scenario that works
Meta:
@issues MYPROJ-3,MYPROJ-4
@issue MYPROJ-5

Given I have an implemented JBehave scenario
And the scenario works
When I run the scenario
Then I should get a successful result

You can also attribute tags to the story as a whole, or to individual scenarios:

Meta:
@tag capability:a capability

Scenario: A scenario that works
Meta:
@tags domain:a domain, iteration: iteration 1

Given I have an implemented JBehave scenario
And the scenario works
When I run the scenario
Then I should get a successful result

7.3.5. Implementing the tests

If you want your tests to actually do anything, you will also need classes in which you place your JBehave step implementations. If you place these in any package at or below the package of your main JUnit test, JBehave will find them with no extra configuration.

Thucydides makes no distinction between the JBehave-style @Given, @When and @Then annotations, and the Thucydides-style @Step annotations: both will appear in the test reports. However you need to start with the @Given, @When and @Then-annotated methods so that JBehave can find the correct methods to call for your stories. A method annotated with @Given, @When or @Then can call Thucydides @Step methods, or call page objects directly (though the extra level of abstraction provided by the @Step methods tends to make the tests more reusable and maintainable on larger projects).

A typical example is shown below. In this implementation of one of the scenarios we saw above, the high-level steps are defined using methods annotated with the JBehave @Given, @When and @Then annotations. These methods, in turn, use steps that are implemented in the BuyerSteps class, which contains a set of Thucydides @Step methods. The advantage of using this two-leveled approach is that it helps maintain a degree of separation between the definition of what is being done in a test, and how it is being implemented. This tends to make the tests easier to understand and easier to maintain.

public class SearchScenarioSteps {
    @Steps
    BuyerSteps buyer;

    @Given("Sally wants to buy a $present for her son")
    public void buyingAPresent(String present) {
        buyer.opens_home_page();
    }

    @When("she looks for ads in the $category category containing $keyword in $region")
    public void adSearchByCategoryAndKeywordInARegion(String category,String keyword,String region){
        buyer.chooses_region(region);
        buyer.chooses_category_and_keywords(category, keyword);
        buyer.performs_search();
    }

    @Then("she should obtain a list of $category ads containing the word $keyword from advertisers in $region")
    public void resultsForACategoryAndKeywordInARegion(String category,String keyword,String region){
        buyer.should_only_see_results_with_titles_containing(keyword);
        buyer.should_only_see_results_from_region(region);
        buyer.should_only_see_results_in_category(category);
    }
}

The Thucydides steps can be found in the BuyserSteps class. This class in turn uses Page Objects to interact with the actual web application, as illustrated here:

public class BuyerSteps extends ScenarioSteps {

    HomePage homePage;
    SearchResultsPage searchResultsPage;

    public BuyerSteps(Pages pages) {
        super(pages);
        homePage = getPages().get(HomePage.class);
        searchResultsPage = getPages().get(SearchResultsPage.class);
    }

    @Step
    public void opens_home_page() {
        homePage.open();
    }

    @Step
    public void chooses_region(String region) {
        homePage.chooseRegion(region);
    }

    @Step
    public void chooses_category_and_keywords(String category, String keywords) {
        homePage.chooseCategoryFromDropdown(category);
        homePage.enterKeywords(keywords);
    }

    @Step
    public void performs_search() {
        homePage.performSearch();
    }

    @Step
    public void should_only_see_results_with_titles_containing(String title) {
        searchResultsPage.allTitlesShouldContain(title);
    }
    ...
}

The Page Objects are similar to those you would find in any Thucydides project, as well as most WebDriver projects. An example is listed below:

@DefaultUrl("http://www.newsclassifieds.com.au")
public class HomePage extends PageObject {

    @CacheLookup
    @FindBy(name="adFilter.searchTerm")
    WebElement searchTerm;

    @CacheLookup
    @FindBy(css=".keywords button")
    WebElement search;

    public HomePage(WebDriver driver) {
        super(driver);
    }

    public void chooseRegion(String region) {
        findBy("#location-select .arrow").then().click();
        waitFor(500).milliseconds();
        findBy("//ul[@class='dropdown-menu']//a[.='" + region + "']").then().click();
    }

    public void chooseCategoryFromDropdown(String category) {
        getDriver().navigate().refresh();
        findBy("#category-select").then(".arrow").then().click();
        findBy("//span[@id='category-select']//a[contains(.,'" + category + "')]").then().click();
    }

    public void enterKeywords(String keywords) {
        element(searchTerm).type(keywords);
    }

    public void performSearch() {
        element(search).click();
    }
}

When these tests are executed, the JBehave steps combine with the Thucydides steps to create a narrative report of the test results:

figs/thucydides-test-report.png
Figure 5. You can see the requirements that you need to implement in the requirements report

8. Implementing Step Libraries

Once you have defined the steps you need to describe your high level tests, you need to implement these steps. In an automated web test, test steps represent the level of abstraction between your Page Objects (which are designed in terms of actions that you perform on a given page) and higher-level stories (sequences of more business-focused actions that illustrate how a given user story has been implemented). Steps can contain other steps, and are included in the Thucydides reports. Whenever a step is executed, a screenshot is stored and displayed in the report.

8.1. Creating Step Libraries

Test steps are regular java methods, annotated with the @Step annotation. You organize steps and step groups in step libraries. A step library is just a normal Java class. If you are running web tests, your step library should either have a Pages member variable, or (more simply) extend the ScenarioSteps class, e.g.

public class JobSeekerSteps extends ScenarioSteps {
    public JobSeekerSteps(Pages pages) {
        super(pages);
    }


    @Step
    public void opens_jobs_page() {
        FindAJobPage page = getPages().get(FindAJobPage.class);
        page.open();
    }

    @Step
    public void searches_for_jobs_using(String keywords) {
        FindAJobPage page = getPages().get(FindAJobPage.class);
        page.look_for_jobs_with_keywords(keywords);

    }
}

Note that step methods can take parameters. The parameters that are passed into a step method will be recorded and reported in the Thucydides reports, making this an excellent technique to make your tests more maintainable and more modular.

Steps can also call other steps, which is very useful for more complicated test scenarios. The result is the sort of nested structure you can see in [fig-test-report].

9. Defining Page Objects

If you are working with WebDriver web tests, you will be familiar with the concept of Page Objects. Page Objects are a way of isolating the implementation details of a web page inside a class, exposing only business-focused methods related to that page. They are an excellent way of making your web tests more maintainable.

In Thucydides, page objects can be just ordinary WebDriver page objects, on the condition that they have a constructor that accepts a WebDriver parameter. However, the Thucydides PageObject class provides a number of utility methods that make page objects more convenient to work with, so a Thucydides Page Object generally extends this class.

Here is a simple example:

@DefaultUrl("http://localhost:9000/somepage")
public class FindAJobPage extends PageObject {

    WebElement keywords;
    WebElement searchButton;

    public FindAJobPage(WebDriver driver) {
        super(driver);
    }

    public void look_for_jobs_with_keywords(String values) {
        typeInto(keywords, values);
        searchButton.click();
    }

    public List<String> getJobTabs() {
        List<WebElement> tabs = getDriver().findElements(By.xpath("//div[@id='tabs']//a"));
        return extract(tabs, on(WebElement.class).getText());
    }
}

The typeInto method is a shorthand that simply clears a field and enters the specified text. If you prefer a more fluent-API style, you can also do something like this:

@DefaultUrl("http://localhost:9000/somepage")
public class FindAJobPage extends PageObject {
        WebElement keywordsField;
        WebElement searchButton;

        public FindAJobPage(WebDriver driver) {
            super(driver);
        }

        public void look_for_jobs_with_keywords(String values) {
            **enter(values).into(keywordsField);**
            searchButton.click();
        }

        public List<String> getJobTabs() {
            List<WebElement> tabs = getDriver().findElements(By.xpath("//div[@id='tabs']//a"));
            return extract(tabs, on(WebElement.class).getText());
        }
}

You can use an even more fluent style of expressing the implementation steps by using methods like find, findBy and then.

For example, you can use webdriver "By" finders with element name, id, css selector or xpath selector as follows:

page.find(By.name("demo")).then(By.name("specialField")).getValue();

page.find(By.cssSelector(".foo")).getValue();

page.find(By.xpath("//th")).getValue();

You can also use findBy method and pass the css or xpath selector directly. For example,

page.findBy("#demo").then("#specialField").getValue(); //css selectors

page.findBy("//div[@id='dataTable']").getValue(); //xpath selector

9.1. Using pages in a step library

When you need to use a page object in one of your steps, you just ask for one from the Page factory, providing the class of the page object you need, e.g.

FindAJobPage page = getPages().get(FindAJobPage.class);

If you want to make sure you are one the right page, you can use the currentPageAt() method. This will check the page class for any @At annotations present in the Page Object class and, if present, check that the current URL corresponds to the URL pattern specified in the annotation. For example, when you invoke it using currentPageAt(), the following Page Object will check that the current URL is precisely http://www.apache.org.

@At("http://www.apache.org")
public class ApacheHomePage extends PageObject {
    ...
}

The @At annotation also supports wildcards and regular expressions. The following page object will match any Apache sub-domain:

@At("http://.*.apache.org")
public class AnyApachePage extends PageObject {
    ...
}

More generally, however, you are more interested in what comes after the host name. You can use the special #HOST token to match any server name. So the following Page Object will match both http://localhost:8080/app/action/login.form an http://staging.acme.com/app/action/login.form. It will also ignore parameters, so http://staging.acme.com/app/action/login.form?username=toto&password=oz will work fine too.

@At(urls={"#HOST/app/action/login.form"})
public class LoginPage extends PageObject {
   ...
}

9.2. Opening the page

A page object is usually designed to work with a particular web page. When the open() method is invoked, the browser will be opened to the default URL for the page.

The @DefaultUrl annotation indicates the URL that this test should use when run in isolation (e.g. from within your IDE). Generally, however, the host part of the default URL will be overridden by the webdriver.base.url property, as this allows you to set the base URL across the board for all of your tests, and so makes it easier to run your tests on different environments by simply changing this property value. For example, in the test class above, setting the webdriver.base.url to https://staging.mycompany.com would result in the page being opened at the URL of https://staging.mycompany.com/somepage.

You can also define named URLs that can be used to open the web page, optionally with parameters. For example, in the following code, we define a URL called open.issue, that accepts a single parameter:

@DefaultUrl("http://jira.mycompany.org")
@NamedUrls(
  {
    @NamedUrl(name = "open.issue", url = "http://jira.mycompany.org/issues/{1}")
  }
)
public class JiraIssuePage extends PageObject {
    ...
}

You could then open this page to the http://jira.mycompany.org/issues/ISSUE-1 URL as shown here:

page.open("open.issue", withParameters("ISSUE-1"));

You could also dispense entirely with the base URL in the named URL definition, and rely on the default values:

@DefaultUrl("http://jira.mycompany.org")
@NamedUrls(
  {
    @NamedUrl(name = "open.issue", url = "/issues/{1}")
  }
)
public class JiraIssuePage extends PageObject {
    ...
}

And naturally you can define more than one definition:

@NamedUrls(
  {
          @NamedUrl(name = "open.issue", url = "/issues/{1}"),
          @NamedUrl(name = "close.issue", url = "/issues/close/{1}")
  }
)

You should never try to implement the open() method yourself. In fact, it is final. If you need your page to do something upon loading, such as waiting for a dynamic element to appear, you can use the @WhenPageOpens annotation. Methods in the PageObject with this annotation will be invoked (in an unspecified order) after the URL has been opened. In this example, the open() method will not return until the dataSection web element is visible:

@DefaultUrl("http://localhost:8080/client/list")
    public class ClientList extends PageObject {

     @FindBy(id="data-section");
     WebElement dataSection;
     ...

     @WhenPageOpens
     public void waitUntilTitleAppears() {
         element(dataSection).waitUntilVisible();
     }
}

9.3. Working with web elements

9.3.1. Checking whether elements are visible

The element method of the PageObject class provides a convenient fluent API for dealing with web elements, providing some commonly-used extra features that are not provided out-of-the-box by the WebDriver API. For example, you can check that an element is visible as shown here:

public class FindAJobPage extends PageObject {

    WebElement searchButton;

    public boolean searchButtonIsVisible() {
        return element(searchButton).isVisible();
    }
    ...
}

If the button is not present on the screen, the test will wait for a short period in case it appears due to some Ajax magic. If you don’t want the test to do this, you can use the faster version:

public boolean searchButtonIsVisibleNow() {
    return element(searchButton).isCurrentlyVisible();
}

You can turn this into an assert by using the shouldBeVisible() method instead:

public void checkThatSearchButtonIsVisible() {
    element(searchButton).shouldBeVisible();
}

This method will through an assertion error if the search button is not visible to the end user.

If you are happy to expose the fact that your page has a search button to your step methods, you can make things even simpler by adding an accessor method that returns a WebElementFacade, as shown here:

public WebElementFacade searchButton() {
    return element(searchButton);
}

Then your steps will contain code like the following:

        searchPage.searchButton().shouldBeVisible();

9.3.2. Checking whether elements are enabled

You can also check whether an element is enabled or not:

element(searchButton).isEnabled() element(searchButton).shouldBeEnabled()

There are also equivalent negative methods:

element(searchButton).shouldNotBeVisible();
element(searchButton).shouldNotBeCurrentlyVisible();
element(searchButton).shouldNotBeEnabled()

You can also check for elements that are present on the page but not visible, e.g:

element(searchButton).isPresent();
element(searchButton).isNotPresent();
element(searchButton).shouldBePresent();
element(searchButton).shouldNotBePresent();

9.3.3. Manipulating select lists

There are also helper methods available for drop-down lists. Suppose you have the following dropdown on your page:

<select id="color">
    <option value="red">Red</option>
    <option value="blue">Blue</option>
    <option value="green">Green</option>
</select>

You could write a page object to manipulate this dropdown as shown here:

public class FindAJobPage extends PageObject {

        @FindBy(id="color")
        WebElement colorDropdown;

        public selectDropdownValues() {
            element(colorDropdown).selectByVisibleText("Blue");
            assertThat(element(colorDropdown).getSelectedVisibleTextValue(), is("Blue"));

            element(colorDropdown).selectByValue("blue");
            assertThat(element(colorDropdown).getSelectedValue(), is("blue"));

            page.element(colorDropdown).selectByIndex(2);
            assertThat(element(colorDropdown).getSelectedValue(), is("green"));

        }
        ...
}

9.3.4. Determining focus

You can determine whether a given field has the focus as follows:

element(firstName).hasFocus()

You can also wait for elements to appear, disappear, or become enabled or disabled:

element(button).waitUntilEnabled()
element(button).waitUntilDisabled()

or

element(field).waitUntilVisible()
element(button).waitUntilNotVisible()

9.3.5. Using direct XPath and CSS selectors

Another way to access a web element is to use an XPath or CSS expression. You can use the element method with an XPath expression to do this more simply. For example, imagine your web application needs to click on a list item containing a given post code. One way would be as shown here:

WebElement selectedSuburb = getDriver().findElement(By.xpath("//li/a[contains(.,'" + postcode + "')]"));
selectedSuburb.click();

However, a simpler option would be to do this:

element(By.xpath("//li/a[contains(.,'" + postcode + "')]")).click();

9.4. Working with Asynchronous Pages

Asynchronous pages are those whose fields or data is not all displayed when the page is loaded. Sometimes, you need to wait for certain elements to appear, or to disappear, before being able to proceed with your tests. Thucydides provides some handy methods in the PageObject base class to help with these scenarios. They are primarily designed to be used as part of your business methods in your page objects, though in the examples we will show them used as external calls on a PageObject instance for clarity.

9.4.1. Checking whether an element is visible

In WebDriver terms, there is a distinction between when an element is present on the screen (i.e. in the HTML source code), and when it is rendered (i.e. visible to the user). You may also need to check whether an element is visible on the screen. You can do this in two ways. Your first option is to use the isElementVisible method, which returns a boolean value based on whether the element is rendered (visible to the user) or not:

assertThat(indexPage.isElementVisible(By.xpath("//h2[.='A visible title']")), is(true));

or

assertThat(indexPage.isElementVisible(By.xpath("//h2[.='An invisible title']")), is(false));

Your second option is to actively assert that the element should be visible:

indexPage.shouldBeVisible(By.xpath("//h2[.='An invisible title']");

If the element does not appear immediately, you can wait for it to appear:

indexPage.waitForRenderedElements(By.xpath("//h2[.='A title that is not immediately visible']"));

An alternative to the above syntax is to use the more fluid waitFor method which takes a css or xpath selector as argument:

indexPage.waitFor("#popup"); //css selector

indexPage.waitFor("//h2[.='A title that is not immediately visible']"); //xpath selector

If you just want to check if the element is present though not necessarily visible, you can use waitForRenderedElementsToBePresent :

indexPage.waitForRenderedElementsToBePresent(By.xpath("//h2[.='A title that is not immediately visible']"));

or its more expressive flavour, waitForPresenceOf which takes a css or xpath selector as argument.

indexPage.waitForPresenceOf("#popup"); //css

indexPage.waitForPresenceOf("//h2[.='A title that is not immediately visible']"); //xpath

You can also wait for an element to disappear by using waitForRenderedElementsToDisappear or waitForAbsenceOf :

indexPage.waitForRenderedElementsToDisappear(By.xpath("//h2[.='A title that will soon disappear']"));

indexPage.waitForAbsenceOf("#popup");

indexPage.waitForAbsenceOf("//h2[.='A title that will soon disappear']");

For simplicity, you can also use the waitForTextToAppear and waitForTextToDisappear methods:

indexPage.waitForTextToDisappear("A visible bit of text");

If several possible texts may appear, you can use waitForAnyTextToAppear or waitForAllTextToAppear:

indexPage.waitForAnyTextToAppear("this might appear","or this", "or even this");

If you need to wait for one of several possible elements to appear, you can also use the waitForAnyRenderedElementOf method:

indexPage.waitForAnyRenderedElementOf(By.id("color"), By.id("taste"), By.id("sound"));

9.5. Executing Javascript

There are times when you may find it useful to execute a little Javascript directly within the browser to get the job done. You can use the evaluateJavascript() method of the PageObject class to do this. For example, you might need to evaluate an expression and use the result in your tests. The following command will evaluate the document title and return it to the calling Java code:

String result = (String) evaluateJavascript("return document.title");

Alternatively, you may just want to execute a Javascript command locally in the browser. In the following code, for example, we set the focus to the firstname input field:

        evaluateJavascript("document.getElementById('firstname').focus()");

And, if you are familiar with JQuery, you can also invoke JQuery expressions:

        evaluateJavascript("$('#firstname').focus()");

This is often a useful strategy if you need to trigger events such as mouse-overs that are not currently supported by the WebDriver API.

9.6. Uploading files

Uploading files is easy. Files to be uploaded can be either placed in a hard-coded location (bad) or stored on the classpath (better). Here is a simple example:

public class NewCompanyPage extends PageObject {
    ...
    @FindBy(id="object_logo")
    WebElement logoField;

    public NewCompanyPage(WebDriver driver) {
        super(driver);
    }

    public void loadLogoFrom(String filename) {
        upload(filename).to(logoField);
    }
}

9.7. Using Fluent Matcher expressions

When writing acceptance tests, you often find yourself expressing expectations about individual domain objects or collections of domain objects. For example, if you are testing a multi-criteria search feature, you will want to know that the application finds the records you expected. You might be able to do this in a very precise manner (for example, knowing exactly what field values you expect), or you might want to make your tests more flexible by expressing the ranges of values that would be acceptable. Thucydides provides a few features that make it easier to write acceptance tests for this sort of case.

In the rest of this section, we will study some examples based on tests for the Maven Central search site (see [maven-search-report]). This site lets you search the Maven repository for Maven artifacts, and view the details of a particular artifact.

figs/maven-search-report.png
Figure 6. The results page for the Maven Central search page

We will use some imaginary regression tests for this site to illustrate how the Thucydides matchers can be used to write more expressive tests. The first scenario we will consider is simply searching for an artifact by name, and making sure that only artifacts matching this name appear in the results list. We might express this acceptance criteria informally in the following way:

  • Give that the developer is on the search page,

  • And the developer searches for artifacts called Thucydides

  • Then the developer should see at least 16 Thucydides artifacts, each with a unique artifact Id

In JUnit, a Thucydides test for this scenario might look like the one:

...
import static net.thucydides.core.matchers.BeanMatchers.the_count;
import static net.thucydides.core.matchers.BeanMatchers.each;
import static net.thucydides.core.matchers.BeanMatchers.the;
import static org.hamcrest.Matchers.greaterThanOrEqualTo;
import static org.hamcrest.Matchers.is;
import static org.hamcrest.Matchers.startsWith;

@RunWith(ThucydidesRunner.class)
public class WhenSearchingForArtifacts {

    @Managed
    WebDriver driver;

    @ManagedPages(defaultUrl = "http://search.maven.org")
    public Pages pages;

    @Steps
    public DeveloperSteps developer;

    @Test
    public void should_find_the_right_number_of_artifacts() {
        developer.opens_the_search_page();
        developer.searches_for("Thucydides");
        developer.should_see_artifacts_where(the("GroupId", startsWith("net.thucydides")),
                                             each("ArtifactId").isDifferent(),
                                             the_count(is(greaterThanOrEqualTo(16))));

    }
}

Let’s see how the test in this class is implemented. The should_find_the_right_number_of_artifacts() test could be expressed as follows:

  1. When we open the search page

  2. And we search for artifacts containing the word Thucydides

  3. Then we should see a list of artifacts where each Group ID starts with "net.thucydides", each Artifact ID is unique, and that there are at least 16 such entries displayed.

The implementation of these steps is illustrated here:

...
import static net.thucydides.core.matchers.BeanMatcherAsserts.shouldMatch;

public class DeveloperSteps extends ScenarioSteps {

    public DeveloperSteps(Pages pages) {
        super(pages);
    }

    @Step
    public void opens_the_search_page() {
        onSearchPage().open();
    }

    @Step
    public void searches_for(String search_terms) {
        onSearchPage().enter_search_terms(search_terms);
        onSearchPage().starts_search();
    }

    @Step
    public void should_see_artifacts_where(BeanMatcher... matchers) {
        shouldMatch(onSearchResultsPage().getSearchResults(), matchers);
    }

    private SearchPage onSearchPage() {
        return getPages().get(SearchPage.class);
    }

    private SearchResultsPage onSearchResultsPage() {
        return getPages().get(SearchResultsPage.class);
    }
}

The first two steps are implemented by relatively simple methods. However the third step is more interesting. Let’s look at it more closely:

    @Step
    public void should_see_artifacts_where(BeanMatcher... matchers) {
        shouldMatch(onSearchResultsPage().getSearchResults(), matchers);
    }

Here, we are passing an arbitrary number of expressions into the method. These expressions actually matchers, instances of the BeanMatcher class. Not that you usually have to worry about that level of detail - you create these matcher expressions using a set of static methods provided in the BeanMatchers class. So you typically would pass fairly readable expressions like the("GroupId", startsWith("net.thucydides")) or each("ArtifactId").isDifferent().

The shouldMatch() method from the BeanMatcherAsserts class takes either a single Java object, or a collection of Java objects, and checks that at least some of the objects match the constraints specified by the matchers. In the context of web testing, these objects are typically POJOs provided by the Page Object to represent the domain object or objects displayed on a screen.

There are a number of different matcher expressions to choose from. The most commonly used matcher just checks the value of a field in an object. For example, suppose you are using the domain object shown here:

     public class Person {
        private final String firstName;
        private final String lastName;

        Person(String firstName, String lastName) {
            this.firstName = firstName;
            this.lastName = lastName;
        }

        public String getFirstName() {...}

        public String getLastName() {...}
    }

You could write a test to ensure that a list of Persons contained at least one person named "Bill" by using the "the" static method, as shown here:

    List<Person> persons = Arrays.asList(new Person("Bill", "Oddie"), new Person("Tim", "Brooke-Taylor"));

    shouldMatch(persons, the("firstName", is("Bill"))

The second parameter in the the() method is a Hamcrest matcher, which gives you a great deal of flexibility with your expressions. For example, you could also write the following:

    List<Person> persons = Arrays.asList(new Person("Bill", "Oddie"), new Person("Tim", "Brooke-Taylor"));

    shouldMatch(persons, the("firstName", is(not("Tim"))));
    shouldMatch(persons, the("firstName", startsWith("B")));

You can also pass in multiple conditions:

    List<Person> persons = Arrays.asList(new Person("Bill", "Oddie"), new Person("Tim", "Brooke-Taylor"));

    shouldMatch(persons, the("firstName", is("Bill"), the("lastName", is("Oddie"));

Thucydides also provides the DateMatchers class, which lets you apply Hamcrest matches to standard java Dates and JodaTime DateTimes. The following code samples illustrate how these might be used:

    DateTime january1st2010 = new DateTime(2010,01,01,12,0).toDate();
    DateTime may31st2010 = new DateTime(2010,05,31,12,0).toDate();

    the("purchaseDate", isBefore(january1st2010))
    the("purchaseDate", isAfter(january1st2010))
    the("purchaseDate", isSameAs(january1st2010))
    the("purchaseDate", isBetween(january1st2010, may31st2010))

You sometimes also need to check constraints that apply to all of the elements under consideration. The simplest of these is to check that all of the field values for a particular field are unique. You can do this using the each() method:

    shouldMatch(persons, each("lastName").isDifferent())

You can also check that the number of matching elements corresponds to what you are expecting. For example, to check that there is only one person who’s first name is Bill, you could do this:

     shouldMatch(persons, the("firstName", is("Bill"), the_count(is(1)));

You can also check the minimum and maximum values using the max() and min() methods. For example, if the Person class had a getAge() method, we could ensure that every person is over 21 and under 65 by doing the following:

     shouldMatch(persons, min("age", greaterThanOrEqualTo(21)),
                          max("age", lessThanOrEqualTo(65)));

These methods work with normal Java objects, but also with Maps. So the following code will also work:

    Map<String, String> person = new HashMap<String, String>();
    person.put("firstName", "Bill");
    person.put("lastName", "Oddie");

    List<Map<String,String>> persons = Arrays.asList(person);
    shouldMatch(persons, the("firstName", is("Bill"))

The other nice thing about this approach is that the matchers play nicely with the Thucydides reports. So when you use the BeanMatcher class as a parameter in your test steps, the conditions expressed in the step will be displayed in the test report, as shown in [fig-maven-search-report].

figs/maven-search-report.png
Figure 7. Conditional expressions are displayed in the test reports

There are two common usage patterns when building Page Objects and steps that use this sort of matcher. The first is to write a Page Object method that returns the list of domain objects (for example, Persons) displayed on the table. For example, the getSearchResults() method used in the should_see_artifacts_where() step could be implemented as follows:

    public List<Artifact> getSearchResults() {
        List<WebElement> rows = resultTable.findElements(By.xpath(".//tr[td]"));
        List<Artifact> artifacts = new ArrayList<Artifact>();
        for (WebElement row : rows) {
            List<WebElement> cells = row.findElements(By.tagName("td"));
            artifacts.add(new Artifact(cells.get(0).getText(),
                                       cells.get(1).getText(),
                                       cells.get(2).getText()));

        }
        return artifacts;
    }

The second is to access the HTML table contents directly, without explicitly modelling the data contained in the table. This approach is faster and more effective if you don’t expect to reuse the domain object in other pages. We will see how to do this next.

9.7.1. Working with HTML Tables

Since HTML tables are still widely used to represent sets of data on web applications, Thucydides comes the HtmlTable class, which provides a number of useful methods that make it easier to write Page Objects that contain tables. For example, the rowsFrom method returns the contents of an HTML table as a list of Maps, where each map contains the cell values for a row indexed by the corresponding heading, as shown here:

...
import static net.thucydides.core.pages.components.HtmlTable.rowsFrom;

public class SearchResultsPage extends PageObject {

    WebElement resultTable;

    public SearchResultsPage(WebDriver driver) {
        super(driver);
    }

    public List<Map<String, String>> getSearchResults() {
        return rowsFrom(resultTable);
    }

}

This saves a lot of typing - our getSearchResults() method now looks like this:

    public List<Map<String, String>> getSearchResults() {
        return rowsFrom(resultTable);
    }

And since the Thucydides matchers work with both Java objects and Maps, the matcher expressions will be very similar. The only difference is that the Maps returned are indexed by the text values contained in the table headings, rather than by java-friendly property names.

You can also use the HtmlTable class to select particular rows within a table to work with. For example, another test scenario for the Maven Search page involves clicking on an artifact and displaying the details for that artifact. The test for this might look something like this:

    @Test
    public void clicking_on_artifact_should_display_details_page() {
        developer.opens_the_search_page();
        developer.searches_for("Thucydides");
        developer.open_artifact_where(the("ArtifactId", is("thucydides")),
                                      the("GroupId", is("net.thucydides")));

        developer.should_see_artifact_details_where(the("artifactId", is("thucydides")),
                                                    the("groupId", is("net.thucydides")));
    }

Now the open_artifact_where() method needs to click on a particular row in the table. This step looks like this:

    @Step
    public void open_artifact_where(BeanMatcher... matchers) {
        onSearchResultsPage().clickOnFirstRowMatching(matchers);
    }

So we are effectively delegating to the Page Object, who does the real work. The corresponding Page Object method looks like this:

import static net.thucydides.core.pages.components.HtmlTable.filterRows;
...
    public void clickOnFirstRowMatching(BeanMatcher... matchers) {
        List<WebElement> matchingRows = filterRows(resultTable, matchers);
        WebElement targetRow = matchingRows.get(0);
        WebElement detailsLink = targetRow.findElement(By.xpath(".//a[contains(@href,'artifactdetails')]"));
        detailsLink.click();
    }

The interesting part here is the first line of the method, where we use the filterRows() method. This method will return a list of WebElements that match the matchers you have passed in. This method makes it fairly easy to select the rows you are interested in for special treatment.

9.8. Running several steps using the same page object

Sometimes, querying the browser can be expensive. For example, if you are testing tables with large numbers of web elements (e.g. a web element for each cell), performance can be slow, and memory usage high. Normally, Thucydides will requery the page (and create a new Page Object) each time you call Pages.get() or Pages.currentPageAt(). If you are certain that the page will not change (i.e., that you are only performing read-only operations on the page), you can use the onSamePage() method of the ScenarioSteps class to ensure that subsequent calls to Pages.get() or Pages.currentPageAt() will return the same page object:

@RunWith(ThucydidesRunner.class)
public class WhenDisplayingTableContents {

    @Managed
    public WebDriver webdriver;

    @ManagedPages(defaultUrl = "http://my.web.site/index.html")
    public Pages pages;

    @Steps
    public DemoSiteSteps steps;

    @Test
    public void the_user_opens_another_page() {
        steps.navigate_to_page_with_a_large_table();
        steps.onSamePage(DemoSiteSteps.class).check_row(1);
        steps.onSamePage(DemoSiteSteps.class).check_row(2);
        steps.onSamePage(DemoSiteSteps.class).check_row(3);
    }
}

10. Spring Integration

If you are running your acceptance tests against an embedded web server (for example, using Jetty), it can occasionally be useful to access the service layers directly for fixture or infrastructure-related code. For example, you may have a scenario where a user action must, as a side effect, record an audit log in a table in the database. To keep your test focused and simple, you may want to call the service layer directly to check the audit logs, rather than logging on as an administrator and navigating to the audit logs screen.

Spring provides excellent support for integration tests, via the SpringJUnit4ClassRunner test runner. Unfortunately, if you are using Thucydides, this is not an option, as a test cannot have two runners at the same time. Fortunately, however, there is a solution! To inject dependencies using a Spring configuration file, you just need to include the Thucydides SpringIntegration rule in your test class. You instantiate this variable as shown here:

@Rule
public SpringIntegration springIntegration = new SpringIntegration();

Then you use the @ContextConfiguration annotation to define the configuration file or files to use. The you can inject dependencies as you would with an ordinary Spring integration test, using the usual Spring annotations such as @Autowired or @Resource. For example, suppose we are using the following Spring configuration file, called ‘config.xml’:

<beans>
    <bean id="widgetService" class="net.thucydides.junit.spring.WidgetService">
        <property name="name"><value>Widgets</value></property>
        <property name="quota"><value>1</value></property>
    </bean>
    <bean id="gizmoService" class="net.thucydides.junit.spring.GizmoService">
        <property name="name"><value>Gizmos</value></property>
        <property name="widgetService"><ref bean="widgetService" /></property>
    </bean>
</beans>

We can use this configuration file to inject dependencies as shown here:

@RunWith(ThucydidesRunner.class)
@ContextConfiguration(locations = "/config.xml")
public class WhenInjectingSpringDependencies {

    @Managed
    WebDriver driver;

    @ManagedPages(defaultUrl = "http://www.google.com")
    public Pages pages;

    @Rule
    public SpringIntegration springIntegration = new SpringIntegration();

    @Autowired
    public GizmoService gizmoService;

    @Test
    public void shouldInstanciateGizmoService() {
        assertThat(gizmoService, is(not(nullValue())));
    }

    @Test
    public void shouldInstanciateNestedServices() {
        assertThat(gizmoService.getWidgetService(), is(not(nullValue())));
    }
}

Other context-related annotations such as @DirtiesContext will also work as they would in a traditional Spring Integration test. Spring will create a new ApplicationContext for each test, but it will use a single ApplicationContext for all of the methods in your test. If one of your tests modifies an object in the ApplicationContext, you may want to tell Spring so that it can reset the context for the next test. You do this using the @DirtiesContext annotation. In the following test case, for example, the tests will fail without the @DirtiesContext annotation:

@RunWith(ThucydidesRunner.class)
@ContextConfiguration(locations = "/spring/config.xml")
public class WhenWorkingWithDirtyContexts {

    @Managed
    WebDriver driver;

    @ManagedPages(defaultUrl = "http://www.google.com")
    public Pages pages;

    @Rule
    public SpringIntegration springIntegration = new SpringIntegration();

    @Autowired
    public GizmoService gizmoService;

    @Test
    @DirtiesContext
    public void shouldNotBeAffectedByTheOtherTest() {
        assertThat(gizmoService.getName(), is("Gizmos"));
        gizmoService.setName("New Gizmos");
    }

    @Test
    @DirtiesContext
    public void shouldNotBeAffectedByTheOtherTestEither() {
        assertThat(gizmoService.getName(), is("Gizmos"));
        gizmoService.setName("New Gizmos");
    }

}

11. Thucydides Reporting

To generate the full Thucydides reports, run mvn thucydides:aggregate. For this to work, you need to add the right plugins group to your settings.xml file, as shown here:

<settings>
 <pluginGroups>
   <pluginGroup>net.thucydides.maven.plugins</pluginGroup>
   ...
 </pluginGroups>
 ...
</settings>

You can run this in the same command as your tests by setting the maven.test.failure.ignore property to true: if you don’t do this, Maven will stop if any errors occur and not proceed to the report generation:

$ mvn clean verify thucydides:aggregate -Dmaven.test.failure.ignore=true

You can also integrate the Thucydides reports into the standard Maven reports. If you are using Maven 2, just add the Thucydides Maven plugin to the reporting section:

<reporting>
    <plugins>
        ...
        <plugin>
            <groupId>net.thucydides.maven.plugins</groupId>
            <artifactId>maven-thucydides-plugin</artifactId>
            <version>${thucydides.version}</version>
        </plugin>
    </plugins>
</reporting>

If you are using Maven 3, you need to add the Maven Thucydides report to the maven-site-plugin configuration as shown here:

<build>
    <plugins>
        ...
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-site-plugin</artifactId>
            <version>3.0-beta-3</version>
            <configuration>
                <reportPlugins>
                    ...
                    <plugin>
                        <groupId>net.thucydides.maven.plugins</groupId>
                        <artifactId>maven-thucydides-plugin</artifactId>
                        <version>${thucydides.version}</version>
                    </plugin>
                </reportPlugins>
            </configuration>
        </plugin>
    </plugins>
</build>

To generate this report, run the mvn site command after running mvn verify, e.g.

$ mvn clean verify site

This will produce a summary report in the generated Maven site documentation, with links to the more detailed Thucydides reports:

figs/thucydides-maven-reports.png
Figure 8. Thucydides test reports in the Maven site

12. Running Thucydides tests from the command line

You typically run Thucydides as part of the build process (either locally or on a CI server). In addition to the webdriver.driver option discussed about, you can also pass a number of parameters in as system properties to customize the way the tests are run. The full list is shown here:

  • webdriver.driver: What browser do you want your tests to run in: firefox, chrome or iexplorer.

  • webdriver.base.url: The default starting URL for the application, and base URL for relative paths.

  • webdriver.timeouts.implicitlywait: How long webdriver waits for elements to appear by default, in milliseconds.

  • thucydides.outputDirectory: Where should reports be generated.

  • thucydides.only.save.failing.screenshots : Should Thucydides only store screenshots for failing steps? This can save disk space and speed up the tests a little. It is very useful for data-driven testing.

  • thucydides.restart.browser.frequency: During data-driven tests, some browsers (Firefox in particular) may slow down over time due to memory leaks. To get around this, you can get Thucydides to start a new browser session at regular intervals when it executes data-driven tests.

  • thucycides.step.delay: Pause (in ms) between each test step.

  • untrusted.certificates: Useful if you are running Firefox tests against an HTTPS test server without a valid certificate. This will make Thucydides use a profile with the AssumeUntrustedCertificateIssuer property set.

  • thucydides.timeout: How long should the driver wait for elements not immediately visible.

  • thucydides.browser.width and thucydides.browser.height: Resize the browser to the specified dimensions, in order to take larger screenshots. This should work with Internet Explorer and Firefox, but not with Chrome.

  • thucydides.resized.image.width : Value in pixels. If set, screenshots are resized to this size. Useful to save space.

  • thucydides.issue.tracker.url: The URL used to generate links to the issue tracking system.

  • thucydides.activate.firebugs : * Activate the Firebugs and FireFinder plugins for Firefox when running the WebDriver tests. This is useful for debugging, but is not recommended when running the tests on a build server.

  • thucydides.batch.count: If batch testing is being used, this is the size of the batches being executed.

  • thucydides.batch.number :If batch testing is being used, this is the number of the batch being run on this machine.

  • thucydides.restart.browser.frequency : Restart the browser every so often during data-driven tests.

An example of using these parameters is shown here:

$ mvn test -Dwebdriver.driver=iexplorer -Dwebdriver.base.url=http://myapp.staging.acme.com

This will run the tests against the staging server using Internet Explorer.

  • webdriver.firefox.profile: The path to the directory of the profile to use when starting firefox. This defaults to webdriver creating an anonymous profile. This is useful if you want to run the web tests using your own Firefox profile. If you are not sure about how to find the path to your profile, look here: http://support.mozilla.com/en-US/kb/Profiles. For example, to run the default profile on a Mac OS X system, you would do something like this:

$ mvn test -Dwebdriver.firefox.profile=/Users/johnsmart/Library/Application\ Support/Firefox/Profiles/2owb5g1d.default

On Windows, it would be something like:

C:\Projects\myproject>mvn test -Dwebdriver.firefox.profile=C:\Users\John Smart\AppData\Roaming\Mozilla\Firefox\Profiles\mvxjy48u.default
  • thucydides.history: The directory in which build history summary data is stored for each project. Each project has it’s own sub-folder within this directory. Defaults to ~./thucydides.

If you want to set default values for some of these properties for your own development environment (e.g. to always activate the Firebugs plugin on your development machine), create a file called thucydides.properties in your home directory, and set any default values here. These values will still be overridden by any values defined in the environment variables. An example is shown here:

thucydides.activate.firebugs = true
thucydides.browser.width = 1200

13. Integrating with issue tracking systems

13.1. Basic issue tracking integration

http://my.jira.server/browse/MYPROJECT-{0}

To do this in Maven, you need to pass this system property to JUnit using the maven-surefire-plugin as shown here:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.7.1</version>
    <configuration>
        <systemPropertyVariables>
            <thucydides.issue.tracker.url>http://my.jira.server/browse/MYPROJECT-{0}</thucydides.issue.tracker.url>
        </systemPropertyVariables>
    </configuration>
</plugin>

Thucydides also provides special support for the Atlassian JIRA issue tracking tool. If you provide the jira.url system property instead of the thucydides.issue.tracker.url, you only need to provide the base URL for your JIRA instance, rather than the full path:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.7.1</version>
    <configuration>
        <systemPropertyVariables>
            <jira.url>http://my.jira.server</jira.url>
        </systemPropertyVariables>
    </configuration>
</plugin>

You need to provide the issue number. You can place this in the test title, prefixed by the # character. For easyb tests, this just means mentioning the issue number (always starting with a # character) somewhere in the scenario name. For JUnit tests, you use the @Title annotation as shown here:

@RunWith(ThucydidesRunner.class)
public class FixingAnIssueScenario {

    @Managed
    public WebDriver webdriver;

    @ManagedPages(defaultUrl = "http://www.mysite.com")
    public Pages pages;

    @Steps
    public SampleScenarioSteps steps;

    @Title("Shopping cart should let users add multiple articles - fixes issues #123")
    @Test
    public void shopping_cart_should_let_users_add_multiple_articles() {
        steps.add_item_to_cart("nuts");
        steps.add_item_to_cart("bolts");
        steps.cart_should_contain("nuts","bolts");
    }
}

Another way to specify issues in JUnit is to use the @Issue or @Issues annotations. You can use the @Issue annotation to associate an individual test with a specific issue

@Issue("#123")
@Test
public void shopping_cart_should_let_users_add_multiple_articles() {
    steps.add_item_to_cart("nuts");
    steps.add_item_to_cart("bolts");
    steps.cart_should_contain("nuts","bolts");
}

You can also place the @Issue annotation at the class level, in which case the issue will be associated with every test in the class:

@RunWith(ThucydidesRunner.class)
@Issue("#123")
public class FixingAnIssueScenario {

        @Managed
        public WebDriver webdriver;

        @ManagedPages(defaultUrl = "http://www.mysite.com")
        public Pages pages;

        @Steps
        public SampleScenarioSteps steps;

        @Test
        public void shopping_cart_should_let_users_add_multiple_articles() {
            steps.add_item_to_cart("nuts");
            steps.add_item_to_cart("bolts");
            steps.cart_should_contain("nuts","bolts");
        }

        @Test
        public void some_other_test() {
            ...
        }
}

If a test needs to be associated with several issues, you can use the @Issues annotation instead:

@Issues({"#123", "#456"})
@Test public void shopping_cart_should_let_users_add_multiple_articles() {
    steps.add_item_to_cart("nuts"); steps.add_item_to_cart("bolts");
        steps.cart_should_contain("nuts","bolts");
}

When you do this, issues will appear in the Thucydides reports with a hyperlink to the corresponding issue in your issue tracking system.

## Overriding the default reports directory

By default, Thucydides generates it’s reports in the target/site/thucydides directory. There are a couple of ways to override this, if need be. First of all, if you are overriding the default Maven output directory, you can override the <directory> element in the <build> section of your pom.xml file. However, since the Maven Surefire plugin runs the tests in a forked JVM by default, you will also need to pass in the project.build.directory property to the unit tests by using the <systemPropertyVariables> configuration element, as shown here:

     ...
    <build>
        <directory>${basedir}/build</directory>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.11</version>
                <configuration>
                    <includes>
                        <include>**/*TestScenario.java</include>
                    </includes>
                    <systemPropertyVariables>
                        <project.build.directory>${project.build.directory}</project.build.directory>
                    </systemPropertyVariables>
                </configuration>
            </plugin>
            <plugin>
                <groupId>net.thucydides.maven.plugins</groupId>
                <artifactId>maven-thucydides-plugin</artifactId>
                <version>${thucydides.version}</version>
            </plugin>
        </plugins>
    </build>

This will result in the Thucydides reports being generated in ‘build/site/thucydides` instead of `target/site/thucydides’.

If you only want to override the Thucydides output directory, you can use the thucydides.outputDirectory and thucydides.sourceDirectory properties, either in the pom.xml file, or from the commnd line. For example, the following properties will generate the Thucydides reports in the build/thucydides-reports directory:

<properties>
    <thucydides.outputDirectory>${basedir}/build/thucydides-reports</thucydides.outputDirectory>
    <thucydides.sourceDirectory>${basedir}/build/thucydides-reports</thucydides.sourceDirectory>
</properties>

14. Using Thucydides tags

Viewing test results is certainly useful, but, from a release and deployment point of view, it is just scratching the surface. Even more interesting is the ability to view test results in terms of stories, features, behaviors, scenarios, or whatever other categorizations you find useful.

Thucydides Tags provide a very flexible mechanism for categorizing and reporting on your test results, which serve as an alternative to the Story/Feature structure described earlier. You can decide on an appropriate set of tag types (such as "feature", "behavior", "epic", "scenario", "non-functional requirement", etc.), and then assign tags of the different types to your tests. To declare a tag type, you simply use a tag of the specified type - it will then automatically appear in the Thucydides reports.

You can assign a tag manually to a test using the @WithTag annotation:

    @WithTag(name="important functionality", type = "functionality")
    class SomeTestScenarioWithTags {
        @Test
        public void a_simple_test_case() {
        }

        @WithTag(name="simple story",type = "story")
        @Test
        public void should_do_this() {
        }

        @Test
        public void should_do_that() {
        }
    }

Note that tags can be assigned at the test or the class level. Tag names and types are free text -

14.1. Writing a Thucydides tags plugin

Thucydides tags are easy to integrate with other applications, such as issue tracking or agile project management systems. In this section, we look at how to write a plugin that will let Thucydides automatically assign tags to your tests, based on your specific requirements.

In Thucydides, tags are arbitrary tuples of String values (name and type), represented by the TagType class. You can create a tag using the TagTest.withName() method, as shown here:

  TestTag specialFeatureTag = TestTag.withName("special feature").andType("feature");

Any feature types you provide will be displayed as separate tabs at the top of the reports screen, and will provide all of the usual aggregation and filtering features that come with the standard reports.

To define your own tags, you need to write your own tag provider, by implementing the TagProvider interface, shown below:

  public interface TagProvider {
      Set<TestTag> getTagsFor(final TestOutcome testOutcome);
  }

The unique method of this interface, getTagsFor(), takes a TestOutcome object, and returns the set of tags associated with this test outcome. The TestOutcome class provides a large number of fields describing the test and it’s results. For example, to obtain the list of the issues specified for this test using the getIssues() method. The following code is an example of a tag provider that provides a list of tags based on the test’s associated issues (specified by the @Issue and @Issues annotations).

  import ch.lambdaj.function.convert.Converter;
  import net.thucydides.core.model.TestOutcome;
  import net.thucydides.core.model.TestTag;
  import java.util.Set;
  import static ch.lambdaj.Lambda.convert;

  public class IssueBasedTagProvider implements TagProvider {

      public IssueBasedTagProvider() {
      }

      public Set<TestTag> getTagsFor(final TestOutcome testOutcome) {

          Set<String> issues = testOutcome.getIssues();
          return Sets.newHashSet(convert(issues, toTestTags()));
      }

      private Converter<String, String> toTestTags() {
          return new Converter<Object, TestTag>() {

              @Override
              public TestTag convert(String issue) {
                  String tagName = getNameForTag(issue);
                  String tagType = getTypeForTag(issue);
                  return TestTag.withName(tagName).andType(tagType);
              }
          };
      }

      String getNameForTag(String issue) {...}
      String getTypeForTag(String issue) {...}
  }

You also need to provide a service definition in the /META-INF/services folder on the classpath, so that Thucydides can register and use your plugin. A simple way to do this is to create a Maven project with a file called net.thucydides.core.statistics.service.TagProvider in the src/resources/META-INF/sevices folder. This file is a text file containing the fully-qualified name of your tag provider, e.g.

  com.mycompany.thucydides.MyThucydidesTagProvider

Now just include the generated JAR file in your dependencies, and Thucydides will use it automatically to include your custom tags in the reports.

14.2. Bi-directional JIRA integration

A common strategy for organizations using JIRA is to represent story cards, and/or the associated acceptance criteria, as JIRA issues. It is useful to know what automated tests have been executed for a given JIRA story card, and what story is being tested for a given test.

You can add both of these features to your Thucydides project by using the thucydides-jira-plugin. First, you need to add the thucydides-jira-plugin to your Maven dependencies. The dependencies you will need (including the normal Thucydides ones) are listed here:

    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit-dep</artifactId>
        <version>4.10</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.hamcrest</groupId>
        <artifactId>hamcrest-all</artifactId>
        <version>1.1</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>net.thucydides</groupId>
        <artifactId>thucydides-junit</artifactId>
        <version>0.6.1</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>net.thucydides.easyb</groupId>
        <artifactId>thucydides-easyb-plugin</artifactId>
        <version>0.6.1</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>net.thucydides.plugins.jira</groupId>
        <artifactId>thucydides-jira-plugin</artifactId>
        <version>0.6.1</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.codehaus.groovy</groupId>
        <artifactId>groovy-all</artifactId>
        <version>1.8.5</version>
    </dependency>
    ...

Note that the JIRA workflow integration needs Groovy 1.8.5 or higher to work properly.

You will also need an slf4j implementation, e.g. ‘slf4j-log4j12′ (if you are using Log4j) or ‘logback-classic’ (if you are using LogBack) (see http://www.slf4j.org/codes.html#StaticLoggerBinder for more details). If you’re stuck, just add slf4j-simple:

    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-simple</artifactId>
        <version>1.6.1</version>
    </dependency>

In Thucydides, you can refer to a JIRA issue by placing a reference to the corresponding JIRA issue number either in the name of the test (using the @Title annotation, for example), or, more simply, by using the @Issue or @Issues annotation as shown here:

    @RunWith(ThucydidesRunner.class)
    public class SearchByKeywordStoryTest {

        @Managed(uniqueSession = true)
        public WebDriver webdriver;

        @ManagedPages(defaultUrl = "http://www.wikipedia.com")
        public Pages pages;

        @Steps
        public EndUserSteps endUser;

        @Issue("#WIKI-1")
        @Test
        public void searching_by_unambiguious_keyword_should_display_the_corresponding_article() {
            endUser.is_on_the_wikipedia_home_page();
            endUser.looks_up_cats();
            endUser.should_see_article_with_title("Cat - Wikipedia, the free encyclopedia");

        }
    }

In this example, the test will be associated with issue WIKI-1.

Alternatively, you may want to associate an issue (such as a story card) with all of the stories in a test case by placing the @Issue (or @Issues) annotation at the class level:

        @RunWith(ThucydidesRunner.class)
        @Issue("#WIKI-1")
        public class SearchByKeywordStoryTest {

            @Managed(uniqueSession = true)
            public WebDriver webdriver;

            @ManagedPages(defaultUrl = "http://www.wikipedia.com")
            public Pages pages;

            @Steps
            public EndUserSteps endUser;

            @Test
            public void searching_by_unambiguious_keyword_should_display_the_corresponding_article() {
                endUser.is_on_the_wikipedia_home_page();
                endUser.looks_up_cats();
                endUser.should_see_article_with_title("Cat - Wikipedia, the free encyclopedia");

            }
        }

Thucydides can use these annotations to integrate with the issues in JIRA. The most simple JIRA integration involves adding links to the corresponding JIRA issues in the Thucydides reports. To activate this, you simply need to provide the jira.url command line option. You do however need to pass this option to JUnit using the maven-surefire-plugin, as shown here:

  <build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.10</version>
            <configuration>
                <argLine>-Xmx1024m</argLine>
                <systemPropertyVariables>
                    <jira.url>http://jira.acme.com</jira.url>
                </systemPropertyVariables>
            </configuration>
        </plugin>
        ...

For tighter, round-trip integration you can also use thucydides-jira-plugin. This will not only include links to JIRA in the Thucydides reports, but it will also update the corresponding JIRA issues with links to the corresponding Story page in the Thucydides reports. To set this up, add the thucydides-jira-plugin dependency to your project dependencies:

    <dependency>
        <groupId>net.thucydides.plugins.jira</groupId>
        <artifactId>thucydides-jira-plugin</artifactId>
        <version>0.6.1</version>
        <scope>test</scope>
    </dependency>

You also need to provide a username and password to connect to JIRA, and the URL where your Thucydides reports will be published (for example, on your CI server). You do using by passing in the jira.username, jira.password, build.id and hucydides.public.url system parameters. The build.id parameter identifies the current test run, and helps Thucydides know whether a given JIRA issue has already been updated by another test in the current test run. Thucydides lists all of the tests for a given JIRA card, along with their results, in the JIRA comment, and optionally updates the state of the JIRA issue accordingly (see below).

  <build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.10</version>
            <configuration>
                <argLine>-Xmx1024m</argLine>
                <systemPropertyVariables>
                    <jira.url>http://jira.acme.com</jira.url>
                    <jira.username>${jira.demo.user}</jira.username>
                    <jira.password>${jira.demo.password}</jira.password>
                    <build.id>${env.BUILD_ID}</build.id>
                    <thucydides.public.url>http://localhost:9000</thucydides.public.url>
                </systemPropertyVariables>
            </configuration>
        </plugin>
        ...

Thucydides also generates aggregate reports grouping results for stories and features. To include the JIRA links in these reports as well, you need to set the jiraUrl configuration option in the maven-thucydides-plugin, as illustrated here:

    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-site-plugin</artifactId>
        <version>3.0-beta-3</version>
        <configuration>
            <reportPlugins>
                <plugin>
                    <groupId>net.thucydides.maven.plugins</groupId>
                    <artifactId>maven-thucydides-plugin</artifactId>
                    <version>@project.version@</version>
                    <configuration>
                         <jiraUrl>http://jira.acme.com</jiraUrl>
                     </configuration>
                 </plugin>
            </reportPlugins>
        </configuration>
    </plugin>

If you do not want Thucydides to update the JIRA issues for a particular run (e.g. for testing or debugging purposes), you can also set thucydides.skip.jira.updates to true, e.g.

 $ mvn verify -Dthucydides.skip.jira.updates=true

You can also configure the plugin to update the status of JIRA issues. This is deactivated by default: to use this option, you need to set the thucydides.jira.workflow.active option to ‘true’, e.g.

 $ mvn verify -Dthucydides.jira.workflow.active=true

The default configuration will work with the default JIRA workflow: open or in progress issues associated with successful tests will be resolved, and closed or resolved issues associated with failing tests will be reopened. If you are using a customized workflow, or want to modify the way the transitions work, you can write your own workflow configuration. Workflow configuration uses a simple Groovy DSL. The following is an example of the configuration file used for the default workflow:

    when 'Open', {
        'success' should: 'Resolve Issue'
    }

    when 'Reopened', {
        'success' should: 'Resolve Issue'
    }

    when 'Resolved', {
        'failure' should: 'Reopen Issue'
    }

    when 'In Progress', {
        'success' should: ['Stop Progress','Resolve Issue']
    }

    when 'Closed', {
        'failure' should: 'Reopen Issue'
    }

You can write your own configuration file and place it on the classpath of your test project (e.g. in the src/test/resources directory). Then you can override the default configuration by using the thucydides.jira.workflow property in the Maven pom.xml file or directly on the command line e.g.

 $ mvn verify -Dthucydides.jira.workflow=my-workflow.groovy

Alternatively, you can simply create a file called jira-workflow.groovy and place it somewhere on your classpath. Thucydides will then use this workflow. In both these cases, you don’t need to explicitly set the thucydides.jira.workflow.active property.

You can also integrate JIRA issues into your easyb Thucydides stories. When using the Thucydides easyb integration, you associate one or more issues with the easyb story as a whole, but not with the individual scenarios. You do this using the thucydides.tests_issue notation:

    using "thucydides"

    thucydides.uses_default_base_url "http://www.wikipedia.com"
    thucydides.uses_steps_from EndUserSteps
    thucydides.tests_story SearchByKeyword

    thucydides.tests_issue "#WIKI-2"

    scenario "Searching for cats", {
        given "the user is on the home page", {
            end_user.is_on_the_wikipedia_home_page()
        }
        when "the end user searches for 'cats'", {
            end_user.looks_up_cats()
        }
        then "they should see the corresponding article", {
           end_user.should_see_article_with_title("Cat - Wikipedia, the free encyclopedia")
        }
    }

You can also associate several issues using thucydides.tests_issues:

    thucydides.tests_issue "#WIKI-2", "#WIKI-3"

To use easyb with Thucydides, you need to add the latest version of thucydides-easyb-plugin to your dependencies if it is not already there:

    <dependency>
        <groupId>net.thucydides.easyb</groupId>
        <artifactId>thucydides-easyb-plugin</artifactId>
        <version>0.6.1</version>
        <scope>test</scope>
    </dependency>

As with JUnit, you will need to pass in the proper parameters to easyb for this to work. You will also need to be using the maven-easyb-plugin version 1.4 or higher, configured to pass in the JIRA parameters as shown here:

    <plugin>
        <groupId>org.easyb</groupId>
        <artifactId>maven-easyb-plugin</artifactId>
        <version>1.4</version>
        <executions>
            <execution>
                <goals>
                    <goal>test</goal>
                </goals>
            </execution>
        </executions>
        <configuration>
            <storyType>html</storyType>
            <storyReport>target/easyb/easyb.html</storyReport>
            <easybTestDirectory>src/test/stories</easybTestDirectory>
            <parallel>true</parallel>
            <jvmArguments>
                <jira.url>http://jira.acme.com</jira.url>
                <jira.username>${jira.demo.user}</jira.username>
                <jira.password>${jira.demo.password}</jira.password>
                <thucydides.public.url>http://localhost:9000</thucydides.public.url>
            </systemPropertyVariables>
            </jvmArguments>
        </configuration>
    </plugin>

Once this is done, Thucydides will update the relevant JIRA issues automatically whenever the tests are executed.

14.3. Increasing the size of screenshots

Sometimes the default window size is too small to display all of the application screen in the screenshots. You can increase the size of the window Thucydides opens by providing the thucydides.browser.width and thucydides.browser.height system properties. For example, to use a browser window with dimensions of 1200x1024, you could do the following:

$ mvn clean verify -Dthucydides.browser.width=1200 -Dthucydides.browser.height=1024

Typically, the width parameter is the only one you will need to specify, as the height will be determined by the contents of the browser page.

If you are running Thucydides with JUnit, you can also specify this parameter (and any of the others, for that matter) directly in your pom.xml file, in the maven-surefire-plugin configuration, e.g:

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.7.1</version>
            <configuration>
                <argLine>-Xmx1024m</argLine>
                <systemPropertyVariables>
                    <thucydides.browser.width>1200</thucydides.browser.width>
                </systemPropertyVariables>
            </configuration>
        </plugin>
        ...

When the browser width is larger than 1000px, the slideshow view in the reports will expand to show the full screenshots.

Note there are some caveats with this feature. In particular, it will not work at all with Chrome, as Chrome, by design, does not support window resizing. In addition, since WebDriver uses a real browser, so the maximum size will be limited by the physical size of the browser. This limitation applies to the browser width, as the full vertical length of the screen will still be recorded in the screenshot even if it scrolls beyond a single page.

14.3.1. Screenshots and OutOfMemoryError issues

Selenium needs memory to take screenshots, particularly if the screens are large. If Selenium runs out of memory when taking screenshots, it will log an error in the test output. In this case, configure the maven-surefire-plugin to use more memory, as illustrated here:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.7.1</version>
    <configuration>
        <argLine>-Xmx1024m</argLine>
    </configuration>
</plugin>

15. Managing state between steps

Sometimes it is useful to be able to pass information between steps. For example, you might need to check that client detailed entered on a registration appears correctly on a confirmation page later on.

You can do this by passing values from one step to another, however this tends to clutter up the steps. Another approach is to use the Thucydides test session, which is essentially a hash map where you can store variables for the duration of a single test. You can obtain this session map using the Thucydides.getCurrentSession() static method.

As illustrated here, you can p

@Step
public void notes_publication_name_and_date() {
    PublicationDatesPage page = pages().get(PublicationDatesPage.class);
    String publicationName = page.getPublicationName();
    DateTime publicationDate = page.getPublicationDate();

    Thucydides.getCurrentSession().put("publicationName", publicationName);
    Thucydides.getCurrentSession().put("publicationDate", publicationDate);
}

Then, in a step invoked later on in the test, you can check the values stored in the session:

public void checks_publication_details_on_confirmation_page() {

    ConfirmationPage page = pages().get(ConfirmationPage.class);

    String selectedPublicationName = (String) Thucydides.getCurrentSession().get("publicationName");
    DateTime selectedPublicationDate = (DateTime) Thucydides.getCurrentSession().get("publicationDate");

    assertThat(page.getPublicationDate(), is(selectedPublicationName));
    assertThat(page.getPublicationName(), is(selectedPublicationDate));

}

If no variable is found with the requested name, the test will fail. The test session is cleared at the start of each test.

16. Data-Driven Testing

16.1. Data-Driven Tests in JUnit

In JUnit 4, you can use the Parameterized test runner to perform data-driven tests. In Thucydides, you use the ThucydidesParameterizedRunner. This runner is very similar to the JUnit Parameterized test runner, except that you use the TestData annotation to provide test data, and you can use all of the other Thucydides annotations (@Managed, @ManagedPages, @Steps and so on). This test runner will also generate proper Thucydides HTML and XML reports for the executed tests.

An example of a data-driven Thucydides test is shown below. In this test, we are checking that valid ages and favorite colors are accepted by the sign-on page of an (imaginary) application. To test this, we use several combinations of ages and favorite colors, specified by the testData() method. These values are represented as instance variables in the test class, and instantiated via the constructor.

@RunWith(ThucydidesParameterizedRunner.class)
public class WhenEnteringPersonalDetails {

    @TestData
    public static Collection<Object[]> testData() {
        return Arrays.asList(new Object[][]{
                {25, "Red"},
                {40, "Blue"},
                {36, "Green"},
        });
    }

    @Managed
    public WebDriver webdriver;

    @ManagedPages(defaultUrl = "http://www.myapp.com")
    public Pages pages;

    @Steps
    public SignupSteps signup;

    private Integer age;
    private String favoriteColor;

    public WhenEnteringPersonalDetails(Integer age, String favoriteColor) {
        this.age = age;
        this.favoriteColor = favoriteColor;
    }

    @Test
    public void valid_personal_details_should_be_accepted() {
        signup.navigateToPersonalDetailsPage();
        signup.enterPersonalDetails(age, favoriteColor);
    }
}

16.2. Reporting on data-driven web tests

When you generate reporting on data-driven web tests, the reports display full test outcomes and screenshots for each set of data. An overall story report is displayed for the data-driven test, which a test case for each row of test data. The test data used for each test is displayed in the report.

16.3. Running data-driven tests in parallel

Data-driven web tests can be long, especially if you need to navigate to a particular page before testing a different field value each time. In most cases, however, this is necessary, as it is unsafe to make assumptions about the state of the web page after a previous data-driven test. One effective way to speed them up, however, is to run them in parallel. You can configure ThucydidesParameterizedRunner tests to run in parallel by using the Concurrent annotation.

@RunWith(ThucydidesParameterizedRunner.class)
@Concurrent
public class WhenEnteringPersonalDetails {...

By default, this will run your tests concurrently, by default using two threads per CPU core. If you want to fine-tune the number of threads to be used, you can specify the threads annotation property.

@RunWith(ThucydidesParameterizedRunner.class)
@Concurrent(threads="4")
public class WhenEnteringPersonalDetails {...

You can also express this as a value relative to the number of available processors. For example, to run 4 threads per CPU, you could specify the following:

@RunWith(ThucydidesParameterizedRunner.class)
@Concurrent(threads="4x")
public class WhenEnteringPersonalDetails {...

16.4. Data-driven testing using CSV files

Thucydides lets you perform data-driven testing using test data in a CSV file. You store your test data in a CSV file (by default with columns separated by commas), with the first column acting as a header:

        NAME,AGE,PLACE OF BIRTH
        Jack Smith, 30, Smithville
        Joe Brown, 40, Brownville
        Mary Williams, 20, Williamsville

Next, create a test class containing properties that match the columns in the test data. Each property should be a property by the JavaBeans definition, with a matching getter and setter. The test class will typically contain one or more tests that use these properties as parameters to the test step or Page Object methods.

The class will also contain the @UseTestDataFrom annotation to indicate where to find the CSV file (this can either be a file on the classpath or a relative or absolute file path - putting the data set on the class path (e.g. in src/test/resources) makes the tests more portable). You also use the @RunWith annotation as well as the other usual Thucydides annotations (@Managed, @ManagedPages and @Steps).

An example of such a class is shown here:

        @RunWith(ThucydidesParameterizedRunner.class)
        @UseTestDataFrom("test-data/simple-data.csv")
        public class SampleCSVDataDrivenScenario {

            private String name;
            private String age;
            private String placeOfBirth;

            public SampleCSVDataDrivenScenario() {
            }

            @Qualifier
            public String getQualifier() {
                return name;
            }

            @Managed
            public WebDriver webdriver;

            @ManagedPages(defaultUrl = "http://www.google.com")
            public Pages pages;

            @Steps
            public SampleScenarioSteps steps;

            @Test
            public void data_driven_test() {
                System.out.println(getName() + "/" + getAge() + "/" + getCity());
            }

            public String getName() {
                return name;
            }

            public void setName(String name) {
                this.name = name;
            }

            public String getAge() {
                return age;
            }

            public void setAge(String age) {
                this.age = age;
            }

            public String getPlaceOfBirth() {
                return placeOfBirth;
            }

            public void setPlaceOfBirth(String placeOfBirth) {
                this.placeOfBirth = placeOfBirth;
            }
        }

Each row of test data needs to be distinguished in the generated reports. By default, Thucydides will call the toString() method. If you provide a public method returning a String that is annotated by the @Qualifier annotation, then this method will be used to distinguish data sets. It should return a value that is unique to each data set.

The test runner will create a new instance of this class for each row of data in the CSV file, assigning the properties with corresponding values in the test data. SoWhen we run this test, we will get an output like this:

        Jack Smith/30/Smithville
        Joe Brown/40/Brownville
        Mary Williams/20/Williamsville

There are a few points to note. The columns in the CSV files are converted to camel-case property names (so "NAME" becomes name and "PLACE OF BIRTH" becomes placeOfBirth). Since we are testing web applications, all of the fields should be strings.

If some of the field values contain commas, you will need to use a different separator. You can use the separator attribute of the @UseTestDataFrom annotation to specify an alternative separator. For example, the following data uses a semi-colon separator:

    NAME;AGE;ADDRESS
    Joe Smith; 30; 10 Main Street, Smithville
    Jack Black; 40; 1 Main Street, Smithville
    Mary Williams, 20, 2 Main Street, Williamsville

To run our tests against this data, we would use a test class like the following:

        @RunWith(ThucydidesParameterizedRunner.class)
        @UseTestDataFrom(value="test-data/simple-semicolon-data.csv", separator=';')
        public class SampleCSVDataDrivenScenario {

            private String name;
            private String age;
            private String address;

            public SampleCSVDataDrivenScenario() {
            }

            @Qualifier
            public String getQualifier() {
                return name;
            }

            @Managed
            public WebDriver webdriver;

            @ManagedPages(defaultUrl = "http://www.google.com")
            public Pages pages;

            @Steps
            public SampleScenarioSteps steps;

            @Test
            public void data_driven_test() {
                System.out.println(getName() + "/" + getAge() + "/" + getAddress());
            }

            public String getName() {
                return name;
            }

            public void setName(String name) {
                this.name = name;
            }

            public String getAge() {
                return age;
            }

            public void setAge(String age) {
                this.age = age;
            }

            public String getAddress() {
                return address;
            }

            public void setAddress(String address) {
                this.address = address;
            }
        }

This will generate an output like this:

        Joe Smith/30/10 Main Street, Smithville
        Jack Black/40/1 Main Street, Smithville
        Mary Williams/20/2 Main Street, Williamsville

Excel support will be added in a future version. However if you store your test data in CSV form, it becomes easier to keep track of changes to test data in your version control system.

16.5. Using data-driven testing for individual steps

Sometimes you want to use data-driven testing at the step level, rather than at the test level. For example, you might want to navigate to a particular screen in the application, and then try many combinations of data, or loop over a sequence of steps with data from a CSV file. This avoids having to reopen the browser for each row of data.

You can do this by adding property values to your Step files. Consider the following steps file:

        public class SampleDataDrivenSteps extends ScenarioSteps {

            public SampleDataDrivenSteps(Pages pages) {
                super(pages);
            }

            private String name;
            private String age;
            private String address;

            public void setName(String name) {
                this.name = name;
            }

            public void setAge(String age) {
                this.age = age;
            }

            public void setAddress(String address) {
                this.address = address;
            }

            @StepGroup
            public void enter_new_user_details() {
                enter_name_and_age(name, age);
                enter_address(address);
            }

            @Step
            public void enter_address(String address) {
                ...
            }

            @Step
            public void enter_name_and_age(String name, String age) {
                ...
            }

                @Step
                public void navigate_to_user_accounts_page() {
                        ...
                }
        }

The enter_personal_details step group uses the step fields to run the enter_name_and_age and enter_address steps. We want to fetch this data from a CSV file, and loop through the enter_personal_details step for each row of data.

We do this using the withTestDataFrom() method of the StepData class:

        import net.thucydides.core.annotations.ManagedPages;
        import net.thucydides.core.annotations.Steps;
        import net.thucydides.core.pages.Pages;
        import net.thucydides.junit.annotations.Managed;
        import net.thucydides.junit.runners.ThucydidesRunner;
        import org.junit.Test;
        import org.junit.runner.RunWith;
        import org.openqa.selenium.WebDriver;

        import static net.thucydides.core.steps.StepData.withTestDataFrom;

        @RunWith(ThucydidesRunner.class)
        public class SamplePassingScenarioWithTestSpecificData {

            @Managed
            public WebDriver webdriver;

            @ManagedPages(defaultUrl = "http://www.google.com")
            public Pages pages;

            @Steps
            public SampleDataDrivenSteps steps;


            @Test
            public void happy_day_scenario() throws Throwable {
                        steps.navigate_to_user_accounts_page();
                withTestDataFrom("test-data/simple-data.csv").run(steps).enter_new_user_details();
            }
        }

This will call the data_driven_test_step() multiple times, each time injecting data from the test-data/simple-data.csv file into the step.

You also can use as many data files as you want, even in the same test. You can also use the same data file for more than one test step. Remember only the properties that match columns in the CSV file will be instantiated - the others will be ignored:

        @RunWith(ThucydidesRunner.class)
        public class SamplePassingScenarioWithTestSpecificData {

            @Managed
            public WebDriver webdriver;

            @ManagedPages(defaultUrl = "http://www.google.com")
            public Pages pages;

            @Steps
            public SampleDataDrivenSteps steps;

            @Steps
            public DifferentDataDrivenSteps different_steps;


            @Test
            public void happy_day_scenario() throws Throwable {
                        steps.navigate_to_user_accounts_page();

                withTestDataFrom("test-data/simple-data.csv").run(steps).enter_new_user_details();

                withTestDataFrom("test-data/some_other-data.csv").run(different_steps).enter_other_details();
            }
        }

Note that, as a shortcut, you can dispense with the setter methods and just declare the relevant fields public. So the Steps class shown above could be rewritten like this:

        public class SampleDataDrivenSteps extends ScenarioSteps {

            public SampleDataDrivenSteps(Pages pages) {
                super(pages);
            }

            public String name;
            public String age;
            public String address;

            @StepGroup
            public void enter_new_user_details() {
                enter_name_and_age(name, age);
                enter_address(address);
            }

            @Step
            public void enter_address(String address) {
                ...
            }

            @Step
            public void enter_name_and_age(String name, String age) {
                ...
            }

                @Step
                public void navigate_to_user_accounts_page() {
                        ...
                }
        }

17. Running Thucydides tests in parallel batches

Web tests make good candidates for concurrent testing, in theory at least, but the implementation can be tricky. For example, although it is easy enough to configure both JUnit and easyb to run tests in parallel, running several webdriver instances of Firefox in parallel on the same display, for example, tends to become unreliable.

The natural solution in this case is to split the web tests into smaller batches, and to run each batch on a different machine and/or on a different virtual display. When each batch has finished, the results can be retrieved and aggregated into the final test reports.

However splitting tests into batches by hand tends to be tedious and unreliable – it is easy to forget to add a new test to a batch, for example, or have unevenly-distributed batches.

The latest version of Thucydides lets you do this automatically, by splitting your test cases evenly into batches of a given size. In practice, you run a build job for each batch. You need to specify two parameters when you run each build: the total number of batches being run (thucydides.batch.count), and the number of the batch being run in this build (thucydides.batch.number).

For example, the following will divide the test cases into 3 batches (thucydides.batch.count), and only run the first test in each batch (thucydides.batch.number):

mvn verify -Dthucydides.batch.count=3 -Dthucydides.batch.number=1

This will only work with the JUnit integration. However this feature is also supported in easyb (as of easyb version 1.5), though using different parameters. When using the Thucydides easyb integration, you also need to provide the equivalent options for easyb:

mvn verify -Deasyb.batch.count=3 -Deasyb.batch.number=1

If you have both easyb and JUnit Thucydides tests, you will need to specify both options.

18. Further Reading

Articles