To give more flexibility and maintainability to user Galen came up with another simple format for defining test suites. Test suites are just a set of pages that needs to be checked with the appropriate test specs. But there is something more in there which gives a lot of power in testing.

Basic Format
Variables
Importing Test Suites
Browser Factory
Page Actions
Parameterization
Tables
Loading Properties
Disabling Tests
Test Groups
Waiting for dynamic content
Page Dump
Mutation Testing

Basic Format

Same way as in specs language the indentation is done with 1 to 8 spaces. A test starts with a name and then you define the page and page actions

Home page on a small mobile device
    http://example.com/home     320x600
        check   homepage.gspec --include "mobile,all" --exclude "nomobile"

In the example above we have defined the simplest test named "Home page on a small mobile device". For this test we also specified that we want to run it against http://example.com/home with browser resized to 320x600. Then we defined the so called Page Action in which it is specified to use homepage.gspec file and also it says which tags to use and which ones to exclude.

This is how the two tests in a same suite would look like:

Home page on a small mobile device
    http://example.com/home     320x600
        check   homepage.gspec --include "mobile,all" --exclude "nomobile"

Login page on a small mobile device
    http://example.com/login 320x600
        check   login.gspec --include "mobile,all" --exclude "nomobile"

You can also filter some sections with --section argument:

Home page on a small mobile device
    http://example.com/home     320x600
        check   homepage.gspec --section "Login for * device"

Passing JavaScript variables to spec file

You can pass some data from your tests into your spec file so that they are accessible from JavaScript statements. To do this you need to pass it via --VvariableName variableValue argument. For example:

Home page test
    http://example.com 1024x768
        check homepage.gspec --include desktop --Vusername John

Variables

It is always nice to put commonly used data in variables. e.g. we can put domain name in the variable. This can be done with special instruction: set

@@ set domain example.com

Home page on a small mobile device
    http://${domain}/home     320x600
        check   homepage.gspec --include "mobile,all" --exclude "nomobile"

You can also define multiple variables within same instruction like this:

@@ set
    domain          example.com
    mobile_size     320x600

Home page on a small mobile device
    http://${domain}/home     ${mobile_size}
        check   homepage.gspec --include "mobile,all" --exclude "nomobile"

Or you can also use variable reference when defining another variable:

@@ set
    domain          example.com
    base_url        http://${domain}/

Home page on a small mobile device
    ${base_url} 320x600
        check   homepage.gspec --include "mobile,all" --exclude "nomobile"

Import Test Suites

If you want to reuse objects and tables from other test suites, you can import them like this:

@@ import base.test

# ....

Browser Factory

There few ways you could run Galen. You can run tests in a specific browser (Firefox, Chrome, IE) or you can run tests against Selenium Grid.

Defining a local browser

Homepage in local Firefox browser
    selenium firefox http://example.com/  640x480
        check homepage.gspec

Homepage in local Chrome browser
    selenium chrome http://example.com/  640x480
        check homepage.gspec

Homepage in local Internet Explorer browser
    selenium ie http://example.com/  640x480
        check homepage.gspec

Homepage in PhantomJS browser
    selenium phantomjs http://example.com/ 640x480
        check homepage.gspec

Actually if you skip the browser name before url Galen will pick Firefox by default.

Running in Selenium Grid

If you want to run Galen tests in parallel or if you want to maintain cross-browser testing then it is a good idea to configure a Selenium Grid (follow this tutorial for setting up a Selenium Grid). In your Galen tests you can use it like this

Homepage in Selenium Grid in Chrome
    selenium grid http://your.selenium.grid.hostname:4444/wd/hub --page http://example.com --size 640x480 --browser Chrome --version "21.0" --platform "XP"
        check homepage.gspec

The following are the arguments that you can use for selenium grid config:

browser - name of a browser which you can set in your Selenium Grid
version -version a browser
platform -name of operating system
page - the url to your test website
size - the size of the browser window to which it should be resized
dc. - the desired capability prefix. Every argument that starts with dc. will be taken as desired capability for grid node

Actually the browser, version and platform are optional arguments so you can use it like this:

Homepage in Selenium Grid
    selenium grid http://your.selenium.grid.hostname:4444/wd/hub --page http://example.com --size 640x480
        check homepage.gspec

Or it would be even better to put selenium grid in the variable:

@@ Set
    run_in_grid    selenium grid http://your.selenium.grid.hostname:4444/wd/hub

Homepage in Selenium Grid
    ${run_in_grid} --page http://example.com --size 640x480
        check homepage.gspec

And here is an example of using desired capabilities

Homepage in Selenium Grid
    selenium grid http://localhost:4444/wd/hub --dc.browser android --dc.device-orientation landscape

JavaScript Browser Factory

In case you would like to take full control over instantiating the WebDriver you can use the so called jsfactory. It delegates the creation of browser to a user-defined script. This might be very handy in case you need to provide some additional settings in a profile. Lets image you need to open browser with specific language settings. Here is an example of a script mydriver.js that does that:

importClass(org.openqa.selenium.firefox.FirefoxDriver);
importClass(org.openqa.selenium.firefox.FirefoxProfile);
importClass(com.galenframework.utils.GalenUtils);
importClass(com.galenframework.browser.SeleniumBrowser);

var pageUrl = args[0];
var size = GalenUtils.readSize(args[1]);

var profile = new FirefoxProfile();
profile.setPreference("intl.accept_languages", "no,en-us,en");

var browser = new SeleniumBrowser(new FirefoxDriver(profile));
browser.load(pageUrl);
browser.changeWindowSize(size);

browser;

And here is an example how to use this script mydriver.js in the test suite:

Home page on a mobile device
    jsfactory mydriver.js http://samples.galenframework.com/tutorial1/tutorial1.html  400x480
        check homepage.gspec --include "mobile"

You should always provide a script for jsfactory. All other arguments will be copied to the args array in the script. As you see from example above the argument http://samples.galenframework.com/tutorial1/tutorial1.html is stored at 0 position in args array and argument 640x480 is at 1 position

IMPORTANT! If you use jsfactory you also have to handle opening of a url and resizing browser window. In this browser factory Galen is not doing anything so you have to manage this by yourself. Just take a look how it is implemented in the example above

In case you don’t want to implement browser window resize and loading of page url you can use page actions open and resize for jsfactory based tests.

Home page test
    jsfactory firefox.js
        open http://example.com
        resize 640x480
        check homepage.gspec

Replacing page url in reports

Once the test is done Galen will generate HTML report and in it it will use the browser factory that you defined in your spec. If you would like to change the way it appears there you can use ‘|’ symbol like in the following example

Login page test
    Login page url /login  | http://example.com/login   640x480
        check login.gspec

In the end galen will use everything before ‘|’ in HTML report. So only the Login page url /login part will be shown.

Page Actions

Page Actions: Cookies Handling

Sometimes you need to work with functionality that is hidden behind Feature switch (or Feature Toggle). That is now possible with Galen Framework. Lets imagine that for your website you use the cookie MyFeature1=allowed; path=/ to set a feature switch. Now you need to test that functionality behind the feature switch. You could do it like this.

Homepage on mobile device
    http://example.com/   400x600
        cookie "MyFeature1=allowed; path=/"
        check homepage.gspec --include "all,mobile"

What will happen is that Galen will generate a javascript for setting a new cookie and will inject it in browser. After that it will refresh the browser window so that you can test your page properly.

Page Actions: JavaScript Injection

Sometimes you need to test things that are not easy accessible. For example user profile page. Normally you need to authorize first. Or another example: there could be a hidden menu that pops up only in case you interact with something on a page. For both problems there two different solutions: javascript injection and selenium interaction

You can inject your own javascript with which you can change your page. For that you need to save a javascript in a separate file e.g. open-popup.js:

$("popup").show();

Then you should trigger this javascript injection from your Galen tests like this:

Popup on Home Page
    http://example.com/ 640x480
        inject open-popup.js
        check homepage-popup.gspec

Page Actions: Selenium Interaction

Another thing that you might need sometimes – is doing something on website before testing with Galen Specs. Lets say you need to login to get to your profile. Galen allows you to run javascript in which you can take over the browser using WebDriver.

IMPORTANT! Even though this page action also uses javascript please note its not getting injected in your web page. It is actually being executed separately and in it you can operate with your browser: click the elements, type text to text fields and even change the size of a window.

The content of some login flow in file login.js


// These two values are passed from Galen test
var userLogin = arg.login;
var userPassword = arg.password

// Here we type user login and password on our login page
driver.findElement(By.cssSelector("#login-form .login")).sendKeys(userLogin);
driver.findElement(By.cssSelector("#login-form .password")).sendKeys(userPassword);

// Submitting the login page
driver.findElement(By.cssSelector("#login-form .submit")).click();

// Waiting till user profile page is shown
function pageIsLoaded() {
    return driver.findElement(By.id("user-profile")) != null;
}


waitFor(pageIsLoaded);

function waitFor(func) {
    var timeout = 10;

    while(timeout > 0 && !func()) {
        timeout = timeout - 1;
        Thread.sleep(1000);
    }

    if (!func()) {
        throw new Error("Wait timeout");
    }
}

Once we are finished with the login.js lets call it before we can test user profile page with userprofile.gspec file

User profile page
    http://example.com/login 640x480
        run login.js '{login: "[email protected]", password: "qwerty"}'
        check userprofile.gspec

Page Actions: Open and Resize

In case you use your own factory for creating a browser and you don’t want to handle url opening in there you can use open page action

Home page test
    jsfactory firefox.js
        open http://example.com
        check homepage.gspec

You can also use resize page action for changing browser window size

Home page test
    jsfactory firefox.js
        open http://example.com
        resize 640x480
        check homepage.gspec

Parameterization

Once you start thinking on testing the responsive design for your web application you might need to look into parameterization. For instance you want to run tests against different devices or different browsers. This can be done by writing a data table and marking the test with "parameterized" special instruction

@@ parameterized
    | deviceName | tags      | size     |
    | Mobile     | mobile    | 320x600  |
    | Tablet     | tablet    | 640x480  |
    | Desktop    | desktop   | 1024x800 |
Home page on ${deviceName} device
    http://example.com/ ${size}
        check homepage.gspec --include "${tags}"

As you see we defined a data table where the first row is used as a header with names of parameters. Using the example above Galen will run test 3 times with all listed parameters. In the report you will see the tests runs like this:

Home page on Mobile device
Home page on Tablet device
Home page on Desktop device

But what if we want to parameterize the parameterized? What could that mean? Lets say you would like to run tests in all browsers and with all different sizes. This can also be done in the following way:

@@ parameterized
    | browser  |
    | firefox  |
    | chrome   |
    | ie       |
@@ parameterized
    | deviceName | tags      | size     |
    | Mobile     | mobile    | 320x600  |
    | Tablet     | tablet    | 640x480  |
    | Desktop    | desktop   | 1024x800 |
Home page on ${deviceName} device in ${browser} browser
    ${browser} http://example.com/ ${size}
        check homepage.gspec --include "${tags}"

The example above would give us 9 runs for the same test. The resulting report would give these test runs:

Home page on Mobile device in firefox browser
Home page on Tablet device in firefox browser
Home page on Desktop device in firefox browser
Home page on Mobile device in chrome browser
Home page on Tablet device in chrome browser
Home page on Desktop device in chrome browser
Home page on Mobile device in ie browser
Home page on Tablet device in ie browser
Home page on Desktop device in ie browser

Tables

Of course the parameterizations are nice but what if we are using 30 tests and we want to parameterize all of them? In this case we can share the data tables and reuse them in our parameterized tests. Take a look at the example below

@@ table devices
    | deviceName | tags      | size     |
    | Mobile     | mobile    | 320x600  |
    | Tablet     | tablet    | 640x480  |
    | Desktop    | desktop   | 1024x800 |

@@ parameterized using devices
Home page on ${deviceName} device
    http://example.com/ ${size}
        check homepage.gspec --include "${tags}"

@@ parameterized using devices
Login page on ${deviceName} device
    http://example.com/login ${size}
        check login.gspec --include "${tags}"

You can also merge tables inside parameterization block like this

@@ table deviceList_A
    | deviceName | tags      | size     |
    | Mobile     | mobile    | 320x600  |
    | Tablet     | tablet    | 640x480  |
    | Desktop    | desktop   | 1024x800 |

@@ table deviceList_B
    | deviceName | tags      | size     |
    | Mobile 2   | mobile    | 360x600  |
    | Mobile 3   | mobile    | 400x600  |
    | Mobile 4   | mobile    | 500x600  |

@@ parameterized using deviceList_A, deviceList_B
Login page on ${deviceName} device
    http://example.com/login ${size}
        check login.gspec --include "${tags}"

Loading Properties

Sometimes you might want to define some specific values in a property file and share its contents across multiple spec files. Here is how you can do it. For example we can create a custom.properties file and put the following content in it:

header.desktop.height= 80 to 90px
header.mobile.height= ~ 45px

In the test suite we can load the properties right before the check command:

Home page test on desktop
    http://example.com 1024x768
        properties custom.properties
        check homepage.gspec --include desktop

Here are the contents of homepage.gspec

@objects
    header          #header

= Header =
    header:
        @on desktop
            height ${header.desktop.height}
        @on mobile
            height ${header.mobile.height}

Disabling Tests

In case you want to temporarily disable a test just put the disabled special instruction before it.

@@ disabled
Home page
    http://example.com      640x480
        check homepage.gspec

Login page
    http://example.com/login      640x480
        check homepage.gspec

Test Groups

Since version 1.6 you can tag your tests with groups. That will allow you to group your tests in HTML report and choose which group to run in command line. To mark a specific test as a group just use groups keyword:

@@ groups mobile, homepage, temp
Home page test
    http://example.com 1024x768
        check homepage.gspec

Waiting for dynamic content

Sometimes you need to test a page that is generated dynamically and you have to wait until it is properly rendered before checking its layout. Since version 0.10.0 Galen offers a wait action. It can be used in two ways:

Strict timeout
Condition based timeout

Strict timeout

You can define timeout in milliseconds (ms), seconds (s) or minutes (m)

Home page test
    http://example.com 640x480
        wait 10s
        check homepage.gspec

Condition based timeout

For this way of waiting you need to provide a locator (css, xpath or id) for an element and a condition for it. There 4 conditions that could be used in wait action:

element appears in DOM
element is removed from DOM
element becomes visible
element gets hidden

In case the condition fails Galen will throw an exception and interrupt the test.

Example of waiting for element to appear in DOM:

Home page test
    http://example.com 640x480
        wait 1m until exist "css: div.list a"
        check homepage.gspec

Example of waiting for element to be removed from DOM:

Home page test
    http://example.com 640x480
        wait 1m until gone "xpath: //div[@class='list']//a"
        check homepage.gspec

Example of waiting for element to become visible:

Home page test
    http://example.com 640x480
        wait 1m until visible "id: login-box"
        check homepage.gspec

Example of waiting for element to become hidden:

Home page test
    http://example.com 640x480
        wait 1m until hidden "id: login-box"
        check homepage.gspec

You can also mix multiple conditions with multiple elements like this:

Home page test
    http://example.com 640x480
        wait 1m until exist "id: login-box" "css: div.list"  gone "css: #logout"  visible "css: a.signup"
        check homepage.gspec

Page Dump

Since version 1.3 Galen has a feature of creating a page dump. With a page dump you can store information about all your test objects on the page together with image samples.

Home page test
    http://example.com 640x480
        dump homepage.gspec --name "Home page" --export dumps/homepage --max-width 200 --max-height 200

Mutation Testing

Since version 2.4 galen lets you perform the so called mutation testing of your galen specs. It lets you check whether your galen specs are good enough by running “mutations” for each object and checking if it was catched by the spec. It works like the check command and uses same arguments. All you need is a working galen spec (it should always pass, otherwise mutation test will not succeed). In the galen test suite you can invoke with the following command:

Home page test
    http://example.com 640x480
        mutate homepage.gspec --offset 5

Where the --offset 5 is an offset for each individual mutation.

Comments

We have moved all the discussions to Google Groups. From this moment, if you have problems with your test code or some issues with installation, please ask your questions in https://groups.google.com/forum/#!forum/galen-framework.

Galen Test Suite Syntax