Three Amigos, One Cucumber and Where to Stick it in Jenkins

This article is aimed at the stalwart of software development, the Test Manager! The scenario: your boss has been on a jolly and has heard the term Cucumber whilst enjoying the free bar! Apparently, using a cucumber has helped his friend/rival steal a march on the market and it’s your job to work out how you can repeat this success using a long, green-skinned fruit! This article is aimed at giving the Test Manager enough information to understand what on earth Three Amigos would do with cucumber in an automated way. Don’t worry, we promise to avoid all food analogies or allergies, sorry!

Ok, let’s think of some common things we do during our Test Planning and Automation Phases, what usually goes wrong and how we can fix it.

The first thing we try and do is understand the Application Under Test (this is true if you are working in Agile or Waterfall or whatever), this typically involves amongst other things a workshop with the Business Analyst, The Development Team and the Testers. I make that a count of three, aha! The Three Amigos! Of course, this meeting can involve a whole host of others though the point is there are three groups of people present, or in jargon three domains. These groups are trying to come to a shared understanding of the requirements which typically results in three sets of documentation each with its’ own vocabulary. The long running specification workshop eventually wraps up, with each group relatively content that they know what they are doing and can carry out their respective tasks. The Test Manager and Team set about their business only to discover some way in to the process that there have been several misunderstandings and the tests don’t validate the customer requirements and even though it’s a no-blame culture, people want to know whose fault it is that it’s not working. Sound familiar? Wouldn’t it be nice if there was a tool that could effectively close this gap in understanding, using a shared language and at the same time give us a flying start in the automation process? Well brace yourself for Cucumber!

I made a common mistake when first looking into Cucumber – I took it to be a pure test automation tool. I missed the point, it really is a superb way of collaborating. The Three Amigos (yes it was taken from the film) work together in short meetings (hopefully no more than an hour) on a regular basis to arrive at a shared understanding of how the software should behave and capture this behaviour in scenarios. Well, that’s nothing new you say! The clever bit is the way that the Scenario is captured; Cucumber makes use of Feature files. These files are in plain English and have a very light weight structure. For example, at Redhound we have developed a product called Rover Test Intelligence and below is the actual feature file we use to test a particular scenario, without any other form of documentation can you tell what the product and the test do?

Feature: Rover Categorise Data
As a Release Manager I want to be able to categorise unexpected
differences in data between Production and UAT whilst ignoring 
irrelevant fields
Scenario: User categorises data
Given That I am logged in to the Rover Categorisation screen
When I select a difference group
And I launch the categorise process
Then I can tag the record with a difference label

Try another

Feature: Rover See Data Differences
As a Release Manager I want to be able to see differences in data 
between Production and UAT whilst ignoring irrelevant fields
Scenario: User views differences
Given That I am logged in to the Rover Categorisation screen
When I select a difference group
And I launch the see differences process
Then I can view the differences in data between the two record sets

As you can see this is, hopefully, understandable to most people reading it. And, this is important, the steps “Given, When, And & Then” can be interpreted by a computer so, that is Three Amigos, a Cucumber and a Laptop! It may not be obvious but this feature file is written in a Language called Gherkin. The Feature files can be developed in their totality outside the Three Amigos specification meeting so long as a feedback loop is in place to ensure the Amigos stay friendly.

When I say it can be interpreted by a computer there is work to do here. At this point the Test Engineers get busy; however, when you run Cucumber it takes the Feature file steps and creates a skeleton type framework and the Test Engineers then have to put the meat on the bones – no, Cucumber will not write your automation tests you still have to code them.

At Redhound we are using IntelliJ IDEA for the Integrated Development Environment, Maven for dependency management and Java as the language of choice. With this set up, when you run a Feature file for the first time, rover_see_data_differences.feature, Cucumber will helpfully generate the following:

//You can implement missing steps with the snippets below:
@Given("^That I am logged in to the Rover Categorisation screen$")
public void That_I_am_logged_in_to_the_Rover_Categorisation_screen() throws Throwable {
// Express the Regexp above with the code you wish you had
    throw new PendingException();
}

@When("^I select a difference group$")
public void I_select_a_difference_group() throws Throwable {
    // Express the Regexp above with the code you wish you had
    throw new PendingException();
}

@When("^I launch the see differences process$")
public void I_launch_the_see_differences_process() throws Throwable {
    // Express the Regexp above with the code you wish you had
    throw new PendingException();
}

@Then("^I can view the differences in data between the two record sets$")
public void I_can_view_the_differences_in_data_between_the_two_record_sets() throws Throwable {
    // Express the Regexp above with the code you wish you had
    throw new PendingException();
}

Granted, the above does look a little more technical, though you can recognise that the steps for the Feature file are now linked via a regular expression to Java code, brilliant! The generated code snippet can be cut and paste directly into Java class and the process of developing your automated tests and indeed your software can begin. Your own test code is placed in the auto generated method bodies replacing the “throw new PendingException();” statement.

The real advantage here is that there is a shared understanding of what the feature steps mean, a so called ubiquitous language; the developers can make it, the testers can break it, and the Business Analyst can see that the product is being developed in line with actual requirements and the tests are sound. This is an iterative process that goes under the guise of Behavioural Driven Development, the desired behaviour drives the development and test! Another term you may see used for the same process is “Specification by Example” (2). Though the irony is not lost in using Cucumber and Gherkin to describe the tool which in no way describes what it is or does, still, it is catchy!

Ok, pause for a breath…

To recap, Cucumber should be thought of as a collaboration tool that brings the Three Amigos together in order to define, using examples, a scenario. When Cucumber runs it generates helpful skeleton code that can be filled in by Test Engineers to create an automated acceptance test framework. The cumulative behaviours in all of the Feature files will eventually equate to the specifications for the system.

Now, how to link Cucumber in to your Continuous Integration and Deployment framework. We have discussed Continuous Integration & Deployment with Docker, Jenkins & Selenium however, it can be confusing to see just how all these bits link together…

The way we do it is to have our automated tests safely tucked away in Git Hub. We have Jenkins and Git Hub linked together using the inbuilt facility of Git Hub – Web Hooks. A change in the base code will trigger Jenkins to run the job. The source code uses Maven for dependency management, which in turn uses Profiles – this is simply a collection of tests under a Test Suite. Jenkins is configured to execute Maven Tests so that test suites can be run accordingly. (See (3) for diagram).

We can’t finish without mentioning that maximum benefits are achieved if you use Cucumber in an Agile Testing framework. You get all the benefits of short iterations, quickly finding defects, less handovers due to multi-disciplinary teams etc, etc. However, just collaborating in the Three Amigos style can assist you no end in understanding what you are supposed to be testing.

Final Summary – Cucumber can be thought of as a collaboration tool, that in conjunction with a Specification by Example process can bring enormous benefits to your automation test efforts. Cucumber won’t write your automation tests for you though it creates skeleton code from a plain English Feature file. If the Three Amigos (a Business Analyst, a Test Analyst and a Developer) work together in short bursts a common understanding and language is achieved, greatly increasing your chances of delivering successful software. We actively encourage the adoption of this approach to enable you to achieve your goals.