Basil Language Documentation¶

Basil (Browser Automation Script Intermediary Language) is a programming language for creating web browser automation tests.

Basil defines:

actions to be performed to get a browser into a desired state
assertions to verify the browser state
steps, tests and test suites to arrange collections of actions and assertions
page models to de-couple tests from the pages being tested
data providers to de-couple parameterised tests from the data driving them

Goals:

concise when compared to equivalent WebDriver-powered code
human-readable, understandable by non-developers
transpilable to a target language for execution

Tutorial Overview¶

A learn-as-you-go guide on writing browser automation tests in basil.

Topics

We’ll start out by covering some terminology so that we’re all clear about browser automation testing concepts.

You will then create a test and will see how through a series of test steps we can get the browser into an expected state and then verify that the browser state is as expected.

We will show you how the operations a user can perform within a browser are translated into a sequence of actions within a test and how the verification of browser state is carried out through a set of assertions.

You will learn how to define commonly-repeated test steps outside of a test and how to import a step into many tests.

Repeated references to page elements will get sorted out when we see how page models can define page properties and how to refer to these within your tests.

We will see how through the use of parameterised tests we can create tests with ‘blanks’ and replace those blanks later with specific values.

Test suites will be used to group tests in whatever manner best fits your needs.

We will finish by learning how to use data providers to separate what is being tested from the data with which it is being tested and how to use environment variables to keep secrets secret.

Get To Know the Terminology¶

Throughout this tutorial we will be talking about test steps, tests, test suites, page models, data providers, parameterised tests and other related terms.

You may already be familiar with these concepts from the testing you’ve carried out before. Or perhaps this is all new to you.

In either case, here’s a quick overview of the concepts to be covered so that we are all starting out with the right understanding.

Test Step¶

A step verifies one small, precise, exact piece of functionality. It does so by performing a sequence of actions and then evaluating a series of assertions.

An action is something that an end user can do in a browser, such as visit a URL, click on a link, enter text into a field or submit a form.

An assertion is statement of fact that we want to demonstrate to be either correct or incorrect (true or false). For example, the statement may be “The browser title is ‘Google’”.

If we visit https://google.com/ (that’s an action) we might want to then verify that this has occurred by asserting that “The browser title is ‘Google’” is true. If so, all is good. If not, we might have run into a bug.

Test¶

A test step does nothing on its own. Used within a test, a series of steps let us get to the point where we can assert if everything is as it should be.

How about testing a user sign in process? We might want to visit a website, click the “sign in” button, enter credentials into a form, submit the form and, once that is all complete, verify that we are now indeed signed in as the correct user.

That’s a series of steps we’d need to follow. And with each step we’d need to be sure that we’re heading in the right direction.

A test brings a collection of steps together to examine a full user journey.

Parameterised Test¶

A regular test combines that being tested with the data with which it is tested. Do so often enough and you notice that you’re repeating yourself for common test needs such as “visit this page and verify that the title is correct”.

A parameterised plain English test may read as “visit url and verify that the title is expected-title”.

A parameterised test does just this. You can define commonly-tested needs with blanks where some actual values might go. When the parameterised test is used, you can pass in the parameter values needed to fill in the blanks.

Test Suites¶

You are likely to have many journeys that need examining as part of the whole test effort. It is unlikely that all user journeys require full examination at all possible test opportunities.

You may have a smoke test, a set of tests covering the most critical functionality, in addition to a broader set tests that cover ever less critical components.

A test suite is a collection of tests that can be grouped according to your priorities.

Page Models¶

The tests you create are performed against specific pages. Each page has a URL and has some elements with which we want to interact.

We want to check if certain text labels are as they should be, we want to click links and buttons and we want to enter text into forms.

A page model is a way of putting all of this information in one place. You’ll probably need to refer to specific page elements in a variety of tests. A page model lets you define page elements in just one place and refer to them over and over again wherever you need.

Data Providers¶

After creating tests you’ll notice that what at first appeared to be a series of somewhat-similar tests are in fact the same test repeated many times with the only difference being the data supplied to the test.

Data providers allow you to define sets of data to be passed to a parameterised test. The parameterised test will then be run once per set of data.

A data provider helps keep the item being tested separate from what it is being tested with. Imagine you want to do a thing over here using that set of data over there. Keeping the two separate makes each easier to understand.

Creating a Test¶

Through a series of steps, a test performs actions to get the browser into a required state and evaluates assertions to verify that the state is as expected.

We will be creating a YAML file to define our test. In fact, all basil code is stored in YAML files.

Note

You will benefit from an editor (a text editor or IDE) that understands YAML (.yml) files. Your favourite editor probably already does so, but go and check if you’re not sure. Whitespace is significant and tabs are bad. Use an editor that knows how to handle YAML.

Testing Google Search¶

Let’s start somewhere nice and simple. We want to

visit https://www.google.com/
enter “example” into the search box
submit the form
examine the title of the resulting page to verify that this all worked correctly

Here’s what that looks like in basil:

# examples/test/google-search-query-literal.yml
config:
  browsers:
    - chrome
  url: https://www.google.com

"verify Google is open":
  assertions:
    - $page.url is "https://www.google.com"
    - $page.title is "Google"

"query 'example'":
  actions:
    - set $".gLFyf.gsfi" to "example"
    - click $".FPdoLc.VlcLAe input[name=btnK]"

  assertions:
    - $page.title is "example - Google Search"

Understanding the Test¶

We then have two steps (verify Google is open and query 'example') to handle carrying out actions and evaluating the browser state.

Configuring the Test¶

You can’t test a web page without first opening a browser and visiting a URL. Every test starts with a config section declaring the browser to use and the url to start at.

config:
  browsers:
    - chrome
  url: https://www.google.com

Test Steps¶

Remember from the terminology overview that a step verifies one precise piece of functionality and a test combines a series of steps to represent a full user journey.

Our test contains two steps, each having a name that we’ve chosen:

verify Google is open
query 'example'

The name can be whatever you chose. There is no right answer, but it is good practice to name a test after what an end user would be looking to achieve. Phrasing the name of a step as an action to be undertaken works well.

Verifying the Opened Page¶

The first step within a test can be whatever you choose. In practice, verifying that the correct page has been opened is generally a good idea.

"verify Google is open":
  assertions:
    - $page.url is "https://www.google.com"
    - $page.title is "Google"

Firstly we name the step (verify Google is open). This name is not part of the step itself, it is here to give meaning to the step within the context of the test.

The step itself comprises the assertions that follow the name:

# examples/step/google-assert-open-literal.yml
assertions:
  - $page.url is "https://www.google.com"
  - $page.title is "Google"

The assertions section covers the factors that we need to examine to be sure that the browser is in the state that we expect. A test step must include assertions. If a test step is not checking that things are correct, the test step isn’t really doing any meaningful testing.

In this case there are two assertions: one to verify the page URL and one to verify the content of the page <title> element.

?What

The above assertions refer to $page.url and $page.title. These are built-in parameters referencing page properties.

Querying For the Word “example”¶

We want to enter the word “example” (without the quotes) into the Google search field and to submit the search form.

"query 'example'":
  actions:
    - set $".gLFyf.gsfi" to "example"
    - click $".FPdoLc.VlcLAe input[name=btnK]"

  assertions:
    - $page.title is "example - Google Search"

This step has an actions section. This lists the steps needed: setting the input field to the required value and clicking the form submission button. Actions are optional but necessary if you want to interact with the browser in any way.

The input field and submit button are identified here with CSS selectors. Page elements can also be identified with XPath and, more robustly, with page model element references. The tutorial on page models goes into more depth.

The assertions section lists one assertion to verify that the page <title> is as expected. We could also verify that page.url is correct but in this case the page title suffices. The fewer assertions needed to verify the browser state the faster your tests will complete.

Understanding Actions and Assertions¶

The previous tutorial on tests showed how a test brings a collection of steps together to examine a full user journey.

Here’s the test we previously created for testing Google search:

# examples/test/google-search-query-literal.yml
config:
  browsers:
    - chrome
  url: https://www.google.com

"verify Google is open":
  assertions:
    - $page.url is "https://www.google.com"
    - $page.title is "Google"

"query 'example'":
  actions:
    - set $".gLFyf.gsfi" to "example"
    - click $".FPdoLc.VlcLAe input[name=btnK]"

  assertions:
    - $page.title is "example - Google Search"

In this tutorial section, we’ll look at how actions and assertions are formed and what we can do with them.

Actions¶

An action is something that an end user can do in a browser, such as click on link, enter text into a field or submit a form. Each specific thing you can do within a browser when using a web page is an action.

We can click elements, set element values or submit elements, as well as take a few more types of action.

All actions take the form of a verb (click, set, submit) followed in many cases by the element on which to apply the action and, possibly, some additional arguments (such as when setting an element value).

Each test starts with a set of actions to get the browser into an expected state.

Action Examples¶

- click $".sign-in-form .submit-button"
- click $".listed-item":1
- click $imported_page_model.elements.element_name
- click $elements.element_name

- set $"#sign-in-form .username" to "user@example.com"
- set $"#sign-in-form .username" to "\"user@example.com\"" # (literal double quotes)
- set $imported_page_model.elements.username to "user@example.com"
- set $elements.element_name to "user@example.com"
- set $elements.element_name to $data.field_name
- set $".selector" to $page.title # odd but valid
- set $"//foo" to $page.url       # also odd but valid
- set $".input1" to $".input2"    # also odd but valid

- submit $"#sign-in-form"
- submit $imported_page_model.elements.submit_button
- submit $elements.element_name

- wait 1
- wait 15
- wait $data.duration
- wait $env.DURATION

- wait-for $"#asychronously-loaded-content"
- wait-for $imported_page_model.elements.delayed_element_name
- wait-for $elements.delayed_element_name

- reload

- forward

- back

Action Verb List¶

Technical Reference: Action Verbs

Assertions¶

Actions get the browser into an expected state. Assertions then verify that the state is as expected.

In plain English, we might say “Verify that the page title is “Google”. A basil assertion for this reads as $page.title is "Google".

If an assertion is found to be incorrect (if the page <title> is not the word “Google”) your test will have failed.

Assertion Examples¶

- $".selector" is "Hello!"
- $".banner .image".src is "http://example.com/images/banner.png"
- $page.title is "Homepage"
- $page.title is "\"Homepage\"" # (literal double quotes)
- $imported_page_model.elements.sign_in_button is "Sign in"
- $elements.element_name is $page.url # odd but valid
- $elements.element_name is $data.expected_element_value
- $elements.element_name is $env.KEY

- $".heading" is-not "Error"
- $".user-container .status".data-status is-not "active"
- $page.title is-not "Homepage"
- $page.title is-not "\"Homepage\"" (literal double quotes)
- $imported_page_model.elements.sign_in_button is-not "Sign in"
- $elements.element_name is-not $page.url # odd but valid
- $elements.element_name is-not $data.expected_element_value
- $elements.element_name is-not $env.KEY

- $".heading" includes "welcome"
- $".heading".title includes "welcome"
- $page.title includes "page"
- $imported_page_model.elements.sign_in_button includes "Sign"
- $elements.element_name includes $page.url # odd but valid
- $elements.element_name includes $data.key
- $elements.element_name includes $env.KEY

- $".heading" includes "error"
- $".heading".title includes "error"
- $page.title excludes "error"
- $imported_page_model.elements.sign_in_button excludes "Log"
- $elements.element_name excludes $page.url # odd but valid
- $elements.element_name excludes $data.key
- $elements.element_name excludes $env.KEY

- $".form .profile" matches "/.*/"
- $".form":action matches "/\/login\//"
- $page.title matches "/homepage$/i"
- $imported_page_model.elements.sign_in_button matches "/^Sign/"
- $elements.element_name matches $data.regex
- $elements.element_name matches $env.REGEX # odd way to go about it but valid

- $".selector" exists
- $".profile":data-image exists
- $"#logo" exists
- $imported_page_model.elements.sign_in_button exists
- $element_name exists

- $".username-field":disabled not-exists
- $"#logo" not-exists
- $imported_page_model.elements.sign_in_button not-exists
- $element_name not-exists

Assertion Operator List¶

Technical Reference: Assertion Operations

Importing Steps Into Your Test¶

In the previous tutorial on tests we opened https://www.google.com/ and then we performed a search. The Google search form is shown on the homepage which is why we first verify that we’re at the homepage before performing a search.

Searching is not the only action we can perform on the homepage. We could have plenty of tests that take the form:

verify that we’re at the homepage
do something

By defining the verify that we’re at the homepage step independently, we can use the step within any test without needing to repeatedly redefine the step.

Defining and Importing a Step¶

This is the same as the first step from the test tutorial placed into its own file.

# examples/step/google-assert-open-literal.yml
assertions:
  - $page.url is "https://www.google.com"
  - $page.title is "Google"

Here we import and use the step within our test.

# examples/test/google-import-assert-open.yml
config:
  browsers:
    - chrome
  url: https://www.google.com

imports:
  steps:
    google_assert_open: "../step/google-assert-open-literal.yml"

"verify Google is open":
  use: google_assert_open

"query 'example'":
  actions:
    - set $".gLFyf.gsfi" to "example"
    - click $".FPdoLc.VlcLAe input[name=btnK]"

  assertions:
    - $page.title is "example - Google Search"

Imports are defined in the imports section within the steps section. We will see later that you can import more than just steps.

A step import has an import name (google_open) and an import path (step/google-assert-open-literal.yml).

The import name is something you choose. It can be anything you like. We use the import name to refer to the step later within the test An import name that is concise and which makes sense later in the context of the test is a good idea.

The import path tells us where to look, relative to the test, to find the step to be imported. Here we defined the step in a file at step/google-assert-open-literal.yml relative to our test.

Referencing an Imported Step¶

Compare our previous test that defined the step directly to how we are now using the imported step.

Previously

"verify Google is open":
  assertions:
    - $page.url is "https://www.google.com"
    - $page.title is "Google"

Now

"verify Google is open":
  use: google_assert_open

Extending an Imported Step¶

An imported step can have additional assertions applied. As actions always come before assertions and as an imported step will have assertions defined, an imported step cannot have additional actions.

# examples/test/google-import-assert-open-additional-assertions.yml
config:
  browsers:
    - chrome
  url: https://www.google.com

imports:
  steps:
    google_assert_open: "../step/google-assert-open-literal.yml"

"verify Google is open":
  use: google_assert_open

  assertions:
    - $page.title excludes "Error"

"query 'example'":
  actions:
    - set $".gLFyf.gsfi" to "example"
    - click $".FPdoLc.VlcLAe input[name=btnK]"

  assertions:
    - $page.title is "example - Google Search"

That’s starting to look quite nice, except I feel that those identifiers (such as .gLFyf.gsfi) are going to get messy if we keep having to write them out in the middle of each test that needs them.

Next? Page models!

Creating a Page Model¶

We’ve been working on a test to open https:/www.google.com and to query for the word “example”. For the querying part, we need to identify the search input field and the button to submit the search form.

"query 'example'":
  actions:
    - set $".gLFyf.gsfi" to "example"
    - click $".FPdoLc.VlcLAe input[name=btnK]"

  assertions:
    - $page.title is "example - Google Search"

The test step works for now so that’s fine, right?

Well, those selectors are a bit messy and that’s not so fine. The selectors are hard to read (.gLFyf.gsfi doesn’t really shout “search input”) and may well need to be updated in many places when the page being tested changes.

Let’s create a page model to store all of these page properties. We can refer to the page model in our tests, making the tests clearer to read. And we can make sure that if something needs updating it only needs to be changed in one place.

Creating and Using a Page Model¶

A page model is a YAML object with url and elements properties. The url property holds the page URL. The elements section maps convenience names to ways of identifying elements.

# examples/page/google.com.yml
url: "https://www.google.com"
elements:
  search_input: $".gLFyf.gsfi"
  search_button: $".FPdoLc.VlcLAe input[name=btnK]"

For the time being those selectors are a little hard on the eyes but at least we need to only update them in one place.

# examples/step/google-assert-open-literal.yml
assertions:
  - $page.url is "https://www.google.com"
  - $page.title is "Google"

# examples/test/google-import-assert-open-with-page-model.yml
config:
  browsers:
    - chrome
  url: google_com.url

imports:
  steps:
    google_assert_open: "../step/google-assert-open-literal.yml"
  pages:
    google_com: "../page/google.com.yml"

"verify Google is open":
  use: google_assert_open

"query 'example'":
  actions:
    - set $google_com.elements.search_input to "example"
    - click $google_com.elements.search_button

  assertions:
    - $page.title is "example - Google Search"

We’ve added a pages section to the imports section to reference the page model we created. We’ve chosen an import name (google_com) and provided an import path (page/google.com.yml).

Within our assertions, we use the name of the page import to reference the elements that the page model defines.

See how click $google_com.elements.search_button is a lot easier on the brain than click $".FPdoLc.VlcLAe input[name=btnK]"?

Scoping Elements Within a Page Model¶

Both the search input field and the search button are within a form. And both elements have some properties other than class names that are probably less prone to change.

We can make our page model more robust.

# examples/page/google.com-selector-scoped.yml
url: "https://www.google.com"
elements:
  # finds "[name=q]" within the context of "form[action='/search']"
  # using CSS syntax
  search_input: $"form[action='/search'] input[name=q]"

  # finds "[type=submit]" within the context of "form[action='/search']"
  # using CSS syntax
  search_button: $"form[action='/search'] input[type=submit]"

That makes my brain hurt just a little bit less. But we’re still repeating the form[action='/search'] part of each of the selectors. We can do better!

We have given the $"form[action=/search]" identifier the name search_form. We can reference this element via the name we chose by prefixing the name with a dollar sign. Within our page model, $search_form is the same as saying $"form[action=/search]".

Combine this with the >> operator for a parent-child relationship and everything is much easier to read.

# examples/page/google.com-element-scoped-reference.yml
url: "https://www.google.com"
elements:
  search_form: $"form[action=/search]"

  # finds "[name=q]" within the context of search_form
  # using basil parent-child syntax
  search_input: $search_form >> $"[name=q]"

  # finds "[type=submit]" within the context of search_form
  # using basil parent-child syntax
  search_button: $search_form >> $"[type=submit]"

Creating and Using Parameterised Tests¶

We previously saw how a commonly-used test step can be defined independently and imported into a test where needed.

In that instance, the commonly-used step verified that the page URL and page title were as expected.

# examples/step/google-assert-open-literal.yml
assertions:
  - $page.url is "https://www.google.com"
  - $page.title is "Google"

Needing to verify that the correct page is open is a common starting point for many tests.

Instead of declaring page properties to verify within the test step itself, it would be nice to leave those details aside for the time being and to provide the appropriate details when needed.

Let’s do that.

Defining Parameters in a Step¶

# examples/step/assert-page-open-parameterised.yml
assertions:
  - $page.url is $data.expected_url
  - $page.title is $data.expected_title

We’ve replace literal values (https://www.google.com and Google) with data parameters (data.expected_url and data.expected_title).

Parameters Crash Course

Parameters are prefixed with $. The prefix denotes that something is a parameter and not a literal value. Data parameters (defined by you that expect values at some point) must be prefixed with data.. The parameters reference does into detail about the different parameter types.

The parameter names we choose are used within a test that imports our parameterised test step. Pick something that makes sense outside of the context of the step.

We now have a general-purpose step for verifying that the correct page has been opened.

Given that pretty much every test must start with the opening of a page, this parameterised step can be re-used in pretty much every test. Let’s see how.

Importing and Using a Parameterised Test¶

# examples/step/assert-page-open-parameterised.yml
assertions:
  - $page.url is $data.expected_url
  - $page.title is $data.expected_title

# examples/test/google-open-parameterised.yml
config:
  browsers:
    - chrome
  url: https://www.google.com

imports:
  steps:
    assert_opened_page_step: "../step/assert-page-open-parameterised.yml"

"verify Google is open":
  use: assert_opened_page_step
  data:
    # Just a single data set needed here
    0:
      expected_url: $config.url
      expected_title: "Google"

We provide an import name and import path just as when importing any other step.

The data property of the step is a list of data sets. Each data set is a YAML object with property names matching the test parameter names and values as needed. In this case our there is just one data set.

Parameterising Querying Google¶

A great feature of a parameterised test is that it can be passed a list of many data sets and the test will run once for each data set.

The act of opening https://www.google.com doesn’t lend itself to being repeated many times with different sets of data. But querying Google does!

# examples/page/google.com-element-scoped-reference.yml
url: "https://www.google.com"
elements:
  search_form: $"form[action=/search]"

  # finds "[name=q]" within the context of search_form
  # using basil parent-child syntax
  search_input: $search_form >> $"[name=q]"

  # finds "[type=submit]" within the context of search_form
  # using basil parent-child syntax
  search_button: $search_form >> $"[type=submit]"

# examples/step/assert-page-open-parameterised.yml
assertions:
  - $page.url is $data.expected_url
  - $page.title is $data.expected_title

# examples/step/google-query-parameterised.yml
actions:
  - set $elements.search_input to $data.query_term
  - click $elements.search_button

assertions:
  - $page.title is $data.expected_title

# examples/test/google-search-parameterised.yml
config:
  browsers:
    - chrome
  url: google_com.url

imports:
  pages:
    google_com: "../page/google.com-element-scoped-reference.yml"
  steps:
    assert_opened_page_step: "../step/assert-page-open-parameterised.yml"
    query_step: "../step/google-search-query-parameterised.yml"

"verify Google is open":
  use: assert_opened_page_step
  data:
    # Just one data set needed to verify the opened page
    0:
      expected_url: $config.url
      expected_title: "Google"

"query":
  use: query_step
  data:
    # Two data sets for querying
    foo:
      search_term: "foo"
      expected_title: "foo - Google Search"

    bar:
      search_term: "bar"
      expected_title: "bar - Google Search"
  elements:
    search_input: $google_com.elements.search_input
    search_button: $google_com.elements.search_button

Our import of a the parameterised test step to verify the opened page is there as before.

We’ve also defined a parameterised test step to query Google and we’ve defined a data list with two data sets to pass to the parameterised querying test. We’ve also parameterised the page elements within the querying test step.

What if we wanted to run the test to query Google with more than just two data sets? We could keep on adding data sets within the test. And adding. And adding. Eventually we are going to run into a collection of data sets large enough that it makes our test hard to follow.

The separation of that being tested from the data with which it is being tested is often useful. We’ll solve that next when we look at data providers.

Using a Data Provider¶

We previously learned how a data set list can be passed to a parameterised test step to run a single step once per set of data.

We defined the data list set within our test. Doing so can get cumbersome over time. An overly-large collection of data can make the test hard to follow. We run the risk of too-closely coupling that which is being tested with the data with which it is being tested.

A data provider imported into a test addresses these matters.

Creating a Data Provider¶

We create a data_providers section within our imports section, choose an import a name and give the path to our data provider.

Now just pass the import name we chose as the data property of the test.

# examples/data-provider/google-search-query.yml
foo:
  search_term: "foo"
  expected_title: "foo - Google Search"

bar:
  search_term: "bar"
  expected_title: "bar - Google Search"

# examples/test/google-search-parameterised-with-data-provider.yml
config:
  browsers:
    - chrome
  url: google_com.url

imports:
  steps:
    assert_opened_page_step: "../step/assert-page-open-parameterised.yml"
    query_step: "../step/google-search-query-parameterised.yml"
  data_providers:
    imported_query_data: "../data-provider/google-search-query.yml"

"verify Google is open":
  use: assert_opened_page_step
  data:
    0:
      expected_url: $config.url
      expected_title: "Google"

"query":
  use: query_step
  data: imported_query_data

Using Test Suites¶

We have so far created a test to open https://www.google.com and to search for given terms. We could keep adding steps to this one test but doing so will eventually be impractical.

One test per user journey is more manageable and easier to maintain. This will lead to a collection of tests covering a range of pages, functionality and user journeys. Tests will vary in their level of critical importance.

A test suite brings together a collection of tests grouped in a manner that fits your needs. A given test can be present in any number of test suites.

You might want to group tests by priority, examining first the most critical user journeys as grouped in one suite and then examining afterwards some lower-priority functionality grouped in further test suites.

Let’s see how this works by creating some more tests for Google.

Additional Google Tests¶

Google has a store where you can buy devices and accessories. The page at https://about.google presents an overview of the company. Let’s create some tests for those.

# examples/test/google-store.yml
config:
  browsers:
    - chrome
  url: https://store.google.com

"verify Google Store is open":
  assertions:
    - $page.url is "https://store.google.com"
    - $page.title matches "/^Google Store/"

"top navigation elements are present":
  assertions:
    - $"div[new-product-nav]" exists
    - $"div[new-product-nav] button[data-category-id='phones']" exists
    - $"div[new-product-nav] button[data-category-id='connected_home']" exists
    - $"div[new-product-nav] button[data-category-id='tablets']" exists
    - $"div[new-product-nav] button[data-category-id='laptops']" exists
    - $"div[new-product-nav] button[data-category-id='accessories']" exists
    - $"div[new-product-nav] a[href='/collection/offers']" exists

# examples/test/about-google.yml
config:
  browsers:
    - chrome
  url: https://about.google/

"verify about.google is open":
  assertions:
    - $page.url is $config.url
    - $page.title is "About | Google"

"top navigation elements are present":
  assertions:
    - $"nav .top-nav" exists
    - $"nav .top-nav a[href='./']" exists
    - $"nav .top-nav a[href='./google-in-uk/']" exists
    - $"nav .top-nav a[href='./products/']" exists
    - $"nav .top-nav a[href='./commitments/']" exists
    - $"nav .top-nav a[href='./stories/']" exists

Creating Test Suites¶

A test suite is a YAML list. It doesn’t get much easier than this.

Create a YAML document for your test suite and within define a list of imports. The first test imported is the first to be run and so on.

# examples/test-suite/google-high-priority.yml

- "../test/google-search-query-literal.yml"
- "../test/google-store.yml"

# examples/test-suite/google-low-priority.yml

- "../test/about-google.yml"

Using Environment Variables¶

We’ve seen how to pass data to parameterised tests and we’ve seen how data providers can pass many sets of data to run a test many times.

Most of your test data will be benign, such as our test Google search terms of foo and bar.

Some test data is intended to be kept a secret. Perhaps not a nobody-must-ever-know secret, but certainly something that you don’t want defined in the middle of a test or within a data provider.

Defining secrets within your basil code will mean that those secrets will eventually end up in your code repository. Such secrets are irreversibly no longer secret.

A common good practice is to store secrets within environment variables. Let’s see how you can use environment variables within action arguments, assertion arguments and within data sets and data providers.

Creating a Test Requiring User Credentials¶

# examples/test/example-sign-in.yml
config:
  browsers:
    - chrome
  url: https://www.example.com

"open https://www.example.com":
  assertions:
    - $page.url is $config.url
    - $page.title is "Example Domain"

"sign in as user":
  actions:
    - set $".sign-in-form .user" to "user@example.com"
    - set $".sign-in-form .password" to "password123"
    - click $".sign-in-form .submit"

  assertions:
    - $page.url is "https://www.example.com/account/"
    - $page.title is "Welcome user@example.com"

Defining and Using An Environment Variable¶

Let’s pretend that some environment variables exist:

export TEST_USER_USERNAME=user@example.com
export TEST_USER_PASSWORD=password123

Referencing an environment variable is very similar to referencing a parameter in a test.

# examples/test/example-sign-in-with-environment-variables.yml
config:
  browsers:
    - chrome
  url: https://www.example.com

"open https://www.example.com":
  assertions:
    - $page.url is $config.url
    - $page.title is "Example Domain"

"sign in as user":
  actions:
    - set $".sign-in-form .user" to $env.TEST_USER_EMAIL
    - set $".sign-in-form .password" to $env.TEST_USER_PASSWORD
    - click $".sign-in-form .submit"

  assertions:
    - $page.url is "https://www.example.com/account/"
    - $page.title is "Welcome $env.TEST_USER_EMAIL"

Prefixing the environment variable name with env. marks the parameter as an environment variable: env.TEST_USER_EMAIL.

Using Environment Variables Within Data Sets and Data Providers¶

What if we have a set of test users that we want to run through the sign in test?

Here are two examples, one with the data sets defined within the test and a one with the data sets defined within a data provider.

# examples/step/example-sign-in-as-user.yml
actions:
  - set $".sign-in-form .user" to $data.username
  - set $".sign-in-form .password" to $data.password
  - click $".sign-in-form .submit"

assertions:
  - $page.url is "https://www.example.com/account/"
  - $page.title is "Welcome $data.username"

# examples/data-provider/example-sign-in.yml
user1:
  username: $env.TEST_USER_1_USERNAME
  password: $env.TEST_USER_1_PASSWORD

user2:
  username: $env.TEST_USER_2_USERNAME
  password: $env.TEST_USER_2_PASSWORD

# examples/test/example-sign-in-with-data-provider.yml
config:
  browsers:
    - chrome
  url: https://www.example.com

import:
  steps:
    sign_in_step: "../step/example-sign-in-as-user.yml"
  data_providers:
    users: "../data-provider/example-sign-in.yml"

"open https://www.example.com":
  assertions:
    - $page.url is $config.url
    - $page.title is "Example Domain"

"sign in as user (literal data)":
  use: sign_in_step
  data:
    user1:
      username: $env.TEST_USER_1_USERNAME
      password: $env.TEST_USER_1_PASSWORD

    user2:
      username: $env.TEST_USER_2_USERNAME
      password: $env.TEST_USER_2_PASSWORD

"sign in as user (imported data)":
  use: sign_in_step
  data: users

Examples Overview¶

A series of examples starting with a simple test suite and expanding it to include imported parameterised tests, a page model and some data providers.

Topics

Simple Test¶

# examples/test/google-search-query-literal.yml
config:
  browsers:
    - chrome
  url: https://www.google.com

"verify Google is open":
  assertions:
    - $page.url is "https://www.google.com"
    - $page.title is "Google"

"query 'example'":
  actions:
    - set $".gLFyf.gsfi" to "example"
    - click $".FPdoLc.VlcLAe input[name=btnK]"

  assertions:
    - $page.title is "example - Google Search"

Simple Test With Descendant Selectors¶

# examples/test/google-search-query-literal.yml
config:
  browsers:
    - chrome
  url: https://www.google.com

"verify Google is open":
  assertions:
    - $page.url is "https://www.google.com"
    - $page.title is "Google"

"query 'example'":
  actions:
    - set $"form" >> $"[name=q]" to "example"
    - click $"form" >> $"input[type=submit]"

  assertions:
    - $page.title is "example - Google Search"

Import a Step Into a Test¶

# examples/step/google-assert-open-literal.yml
assertions:
  - $page.url is "https://www.google.com"
  - $page.title is "Google"

# examples/test/google-import-assert-open.yml
config:
  browsers:
    - chrome
  url: https://www.google.com

imports:
  steps:
    google_assert_open: "../step/google-assert-open-literal.yml"

"verify Google is open":
  use: google_assert_open

"query 'example'":
  actions:
    - set $".gLFyf.gsfi" to "example"
    - click $".FPdoLc.VlcLAe input[name=btnK]"

  assertions:
    - $page.title is "example - Google Search"

Define Page Properties in a Page Model¶

# examples/page/google.com-element-scoped-literal.yml
url: "https://www.google.com"
elements:
  search_form: $"form[action=/search]"

  # finds "[name=q]" within the context of $"form[action=/search]"
  # using basil parent-child syntax
  search_input: $"form[action=/search]" >> $"[name=q]"

  # finds "[type=submit]" within the context of $"form[action=/search]"
  # using basil parent-child syntax
  search_button: $"form[action=/search]" >> $"[type=submit]"

# examples/page/google.com-element-scoped-reference.yml
url: "https://www.google.com"
elements:
  search_form: $"form[action=/search]"

  # finds "[name=q]" within the context of search_form
  # using basil parent-child syntax
  search_input: $search_form >> $"[name=q]"

  # finds "[type=submit]" within the context of search_form
  # using basil parent-child syntax
  search_button: $search_form >> $"[type=submit]"

# examples/step/google-assert-open-literal.yml
assertions:
  - $page.url is "https://www.google.com"
  - $page.title is "Google"

# examples/test/google-import-assert-open-with-page-model.yml
config:
  browsers:
    - chrome
  url: google_com.url

imports:
  steps:
    google_assert_open: "../step/google-assert-open-literal.yml"
  pages:
    google_com: "../page/google.com.yml"

"verify Google is open":
  use: google_assert_open

"query 'example'":
  actions:
    - set $google_com.elements.search_input to "example"
    - click $google_com.elements.search_button

  assertions:
    - $page.title is "example - Google Search"

Parameterise a Test¶

# examples/page/google.com-element-scoped-reference.yml
url: "https://www.google.com"
elements:
  search_form: $"form[action=/search]"

  # finds "[name=q]" within the context of search_form
  # using basil parent-child syntax
  search_input: $search_form >> $"[name=q]"

  # finds "[type=submit]" within the context of search_form
  # using basil parent-child syntax
  search_button: $search_form >> $"[type=submit]"

# examples/step/assert-page-open-parameterised.yml
assertions:
  - $page.url is $data.expected_url
  - $page.title is $data.expected_title

# examples/step/google-query-parameterised.yml
actions:
  - set $elements.search_input to $data.query_term
  - click $elements.search_button

assertions:
  - $page.title is $data.expected_title

# examples/test/google-search-parameterised.yml
config:
  browsers:
    - chrome
  url: google_com.url

imports:
  pages:
    google_com: "../page/google.com-element-scoped-reference.yml"
  steps:
    assert_opened_page_step: "../step/assert-page-open-parameterised.yml"
    query_step: "../step/google-search-query-parameterised.yml"

"verify Google is open":
  use: assert_opened_page_step
  data:
    # Just one data set needed to verify the opened page
    0:
      expected_url: $config.url
      expected_title: "Google"

"query":
  use: query_step
  data:
    # Two data sets for querying
    foo:
      search_term: "foo"
      expected_title: "foo - Google Search"

    bar:
      search_term: "bar"
      expected_title: "bar - Google Search"
  elements:
    search_input: $google_com.elements.search_input
    search_button: $google_com.elements.search_button

Test Suite¶

# examples/test/google-search-query-literal.yml
config:
  browsers:
    - chrome
  url: https://www.google.com

"verify Google is open":
  assertions:
    - $page.url is "https://www.google.com"
    - $page.title is "Google"

"query 'example'":
  actions:
    - set $".gLFyf.gsfi" to "example"
    - click $".FPdoLc.VlcLAe input[name=btnK]"

  assertions:
    - $page.title is "example - Google Search"

# examples/test/google-store.yml
config:
  browsers:
    - chrome
  url: https://store.google.com

"verify Google Store is open":
  assertions:
    - $page.url is "https://store.google.com"
    - $page.title matches "/^Google Store/"

"top navigation elements are present":
  assertions:
    - $"div[new-product-nav]" exists
    - $"div[new-product-nav] button[data-category-id='phones']" exists
    - $"div[new-product-nav] button[data-category-id='connected_home']" exists
    - $"div[new-product-nav] button[data-category-id='tablets']" exists
    - $"div[new-product-nav] button[data-category-id='laptops']" exists
    - $"div[new-product-nav] button[data-category-id='accessories']" exists
    - $"div[new-product-nav] a[href='/collection/offers']" exists

# examples/test-suite/google-high-priority.yml

- "../test/google-search-query-literal.yml"
- "../test/google-store.yml"

Define Data Externally In a Data Provider¶

# examples/page/google.com-element-scoped-reference.yml
url: "https://www.google.com"
elements:
  search_form: $"form[action=/search]"

  # finds "[name=q]" within the context of search_form
  # using basil parent-child syntax
  search_input: $search_form >> $"[name=q]"

  # finds "[type=submit]" within the context of search_form
  # using basil parent-child syntax
  search_button: $search_form >> $"[type=submit]"

# examples/step/assert-page-open-parameterised.yml
assertions:
  - $page.url is $data.expected_url
  - $page.title is $data.expected_title

# examples/step/google-query-parameterised.yml
actions:
  - set $elements.search_input to $data.query_term
  - click $elements.search_button

assertions:
  - $page.title is $data.expected_title

# examples/data-provider/google-search-query.yml
foo:
  search_term: "foo"
  expected_title: "foo - Google Search"

bar:
  search_term: "bar"
  expected_title: "bar - Google Search"

# examples/test/google-search-parameterised-with-data-provider.yml
config:
  browsers:
    - chrome
  url: google_com.url

imports:
  steps:
    assert_opened_page_step: "../step/assert-page-open-parameterised.yml"
    query_step: "../step/google-search-query-parameterised.yml"
  data_providers:
    imported_query_data: "../data-provider/google-search-query.yml"

"verify Google is open":
  use: assert_opened_page_step
  data:
    0:
      expected_url: $config.url
      expected_title: "Google"

"query":
  use: query_step
  data: imported_query_data

Using Data Within Environment Variables¶

# examples/step/example-sign-in-as-user.yml
actions:
  - set $".sign-in-form .user" to $data.username
  - set $".sign-in-form .password" to $data.password
  - click $".sign-in-form .submit"

assertions:
  - $page.url is "https://www.example.com/account/"
  - $page.title is "Welcome $data.username"

# examples/data-provider/example-sign-in.yml
user1:
  username: $env.TEST_USER_1_USERNAME
  password: $env.TEST_USER_1_PASSWORD

user2:
  username: $env.TEST_USER_2_USERNAME
  password: $env.TEST_USER_2_PASSWORD

# examples/test/example-sign-in-with-data-provider.yml
config:
  browsers:
    - chrome
  url: https://www.example.com

import:
  steps:
    sign_in_step: "../step/example-sign-in-as-user.yml"
  data_providers:
    users: "../data-provider/example-sign-in.yml"

"open https://www.example.com":
  assertions:
    - $page.url is $config.url
    - $page.title is "Example Domain"

"sign in as user (literal data)":
  use: sign_in_step
  data:
    user1:
      username: $env.TEST_USER_1_USERNAME
      password: $env.TEST_USER_1_PASSWORD

    user2:
      username: $env.TEST_USER_2_USERNAME
      password: $env.TEST_USER_2_PASSWORD

"sign in as user (imported data)":
  use: sign_in_step
  data: users

Variable Types¶

`<string>`	A string containing any characters.
`<integer>`	An integer.
`<positive-integer>`	An integer greater than zero.
`<url>`	A URL as defined in RFC 3986
`<identifier>`	An identifier.
`<regular-expression>`	A PRCE regular expression.
`<selector>`	A selector.
`<css-selector>`	A CSS selector.
`<xpath>`	A XPath expression.
`<action>`	An action.
`<assertion>`	An selector.

Browser Properties¶

Properties for the browser can be accessed through the built-in browser object and can be referenced in actions and assertions.

size¶

The size of the browser window.

Example¶

# examples/step/browser-size.yml
actions:
  - set $browser.size to "1024,768"

Page Properties¶

Properties for the current page can be accessed through the built-in page object and can be referenced in actions and assertions.

url¶

The current content of the browser address bar.

title¶

The page title (content of the <title> element).

Example¶

# examples/step/google-assert-open-literal.yml
assertions:
  - $page.url is "https://www.google.com"
  - $page.title is "Google"

Environment Variables¶

Environment variable values can be accessed through the environment variables object and can be referenced actions, assertions and data providers.

A default can be provided if the environment variable does not exist: $env.APP_SECRET|"secret".

Examples¶

`env.APP_SECRET`	Use the `APP_SECRET` environment variable value.
`env.APP_SECRET\|"secret"`	Use the `APP_SECRET` environment variable value with a default of “secret!” if not set.

Parameters¶

Parameters may be used within actions, assertions and data providers.

A parameter is prefixed with $. This denotes that something is a parameter and not a literal value.

There are five types of parameter:

built-in objects
test configuration
data parameters
element parameters
environment variables

Built-in Objects¶

Browser properties and page properties are built-in objects that can be referenced in actions and assertions.

Use $browser.{name} and $page.{name} to reference these built-in objects.

# examples/step/browser-size.yml
actions:
  - set $browser.size to "1024,768"

# examples/step/google-assert-open-literal.yml
assertions:
  - $page.url is "https://www.google.com"
  - $page.title is "Google"

Test Configuration¶

The config section of a test is referenced within the test using $config.{name}.

# examples/test/about-google-verify-opened-page.yml
config:
  browsers:
    - chrome
  url: https://about.google/

"verify about.google is open":
  assertions:
    - $page.url is $config.url
    - $page.title is "About | Google"

Data Parameters¶

Data parameters can be used in actions and assertions. Literal values are replaced with data parameters.

Data is passed to a test step from a test. Data passed to a test step is referenced within the step using $data.{name}.

# examples/step/assert-page-open-parameterised.yml
assertions:
  - $page.url is $data.expected_url
  - $page.title is $data.expected_title

# examples/test/google-open-parameterised.yml
config:
  browsers:
    - chrome
  url: https://www.google.com

imports:
  steps:
    assert_opened_page_step: "../step/assert-page-open-parameterised.yml"

"verify Google is open":
  use: assert_opened_page_step
  data:
    # Just a single data set needed here
    0:
      expected_url: $config.url
      expected_title: "Google"

Element Parameters¶

Element parameters can be used in the identifiers of actions and assertions. Identifiers are replaced with element parameters.

Elements are passed to a test step from a test. Elements passed to a test step are referenced using $elements.{name}.

# examples/step/google-query-parameterised.yml
actions:
  - set $elements.search_input to $data.query_term
  - click $elements.search_button

assertions:
  - $page.title is $data.expected_title

# examples/page/google.com.yml
url: "https://www.google.com"
elements:
  search_input: $".gLFyf.gsfi"
  search_button: $".FPdoLc.VlcLAe input[name=btnK]"

# examples/test/google-imported-steps-with-page-model.yml
config:
  browsers:
    - chrome
  url: google_com.url

imports:
  pages:
    google_com: "../page/google.com.yml"
  steps:
    google_query: "../step/google-query-parameterised.yml"

"verify Google is open":
  assertions:
    - $page.title is "Google"

"query 'example'":
  use: google_query
  data:
    0:
      query_term: "foo"
      expected_title: "foo - Google Search"

  elements:
    search_input: $google_com.elements.search_input
    search_button: $google_com.elements.search_button

Environment Variables¶

Environment variables can be treated as parameters within actions, assertions and data providers. Values can be referenced using $env.{name}.

# examples/data-provider/example-sign-in.yml
user1:
  username: $env.TEST_USER_1_USERNAME
  password: $env.TEST_USER_1_PASSWORD

user2:
  username: $env.TEST_USER_2_USERNAME
  password: $env.TEST_USER_2_PASSWORD

Syntax Notation¶

Syntax definitions follow a specific notation taking the following general form.

definition-block

-----------------

description-block

The definition-block section defines the syntax. The description-block section describes the purpose of non-literal values, giving either a type, range for values or expanded definition.

A solid line of hyphens separates the definition from the description. It is neither part of the definition nor description.

{this-variable}|{that-variable} {later-defined} [{optional}] literal

--------------------------------------------------------------------

this-variable:
    <type>, the type for the variable value (<string>, <integer>, <url>).

later-defined:
    A variable value to be defined later, most commonly if composed of multiple parts where
    each part requires its own definition or description.

optional:
    <type>, optional values are encapsulated in [square brackets].

| (pipe character):
    OR. Separates choices from which one choice is allowed.

Any characters within the definition not encapsulated in curly or square brackets and which are not the pipe character are literal. Literal characters do not denote any special purpose and to be present as-is.

Actions¶

Actions are operations that can be performed to get the browser into the desired state.

Syntax¶

An action takes the form of a verb followed optionally an argument, a keyword and an additional argument.

The verb is always required. Whether arguments and keywords are optional is dependent on the verb.

{verb} [{arguments-section}]

----------------------------

verb:
    back
    click
    forward
    reload
    set
    submit
    wait
    wait-for

arguments-section:
    dependent on the verb

Click¶

Click on an element.

click {identifier}

------------------

identifier:
    <identifier>

Arguments¶

`identifier`	<identifier>	An identifier.

Examples¶

- click $".sign-in-form .submit-button"
- click $".listed-item":1
- click $imported_page_model.elements.element_name
- click $elements.element_name

Set¶

Set the value of an element. Most applicable to form field elements.

set {identifier} to "{value}"

-----------------------------

identifier:
    <identifier>

value:
    <string>

Arguments¶

`identifier`	<identifier>	An identifier.
`value`	<string>	A string.

Examples¶

- set $"#sign-in-form .username" to "user@example.com"
- set $"#sign-in-form .username" to "\"user@example.com\"" # (literal double quotes)
- set $imported_page_model.elements.username to "user@example.com"
- set $elements.element_name to "user@example.com"
- set $elements.element_name to $data.field_name
- set $".selector" to $page.title # odd but valid
- set $"//foo" to $page.url       # also odd but valid
- set $".input1" to $".input2"    # also odd but valid

Submit¶

Submits a form.

submit {identifier}

-------------------

identifier:
    <identifier>

Arguments¶

`identifier`	<identifier>	An identifier.

Examples¶

- submit $"#sign-in-form"
- submit $imported_page_model.elements.submit_button
- submit $elements.element_name

Wait¶

Wait for a specified number of seconds.

wait {number-of-seconds}

------------------------

number-of-seconds:
    <positive-integer>

Arguments¶

`number-of-seconds`	<positive-integer>	An integer greater than zero.

Examples¶

- wait 1
- wait 15
- wait $data.duration
- wait $env.DURATION

Wait-for¶

Wait for an element to be rendered. Waits for up to 30 seconds.

wait-for {identifier}

---------------------

{identifier}:
    <identifier>

Arguments¶

`identifier`	<identifier>	An identifier.

Examples¶

- wait-for $"#asychronously-loaded-content"
- wait-for $imported_page_model.elements.delayed_element_name
- wait-for $elements.delayed_element_name

Reload¶

Reload the current page.

reload

Forward¶

Move forward one item in the current session history.

forward

Back¶

Move back one item in the current session history.

back

Example List¶

- click $".sign-in-form .submit-button"
- click $".listed-item":1
- click $imported_page_model.elements.element_name
- click $elements.element_name

- set $"#sign-in-form .username" to "user@example.com"
- set $"#sign-in-form .username" to "\"user@example.com\"" # (literal double quotes)
- set $imported_page_model.elements.username to "user@example.com"
- set $elements.element_name to "user@example.com"
- set $elements.element_name to $data.field_name
- set $".selector" to $page.title # odd but valid
- set $"//foo" to $page.url       # also odd but valid
- set $".input1" to $".input2"    # also odd but valid

- submit $"#sign-in-form"
- submit $imported_page_model.elements.submit_button
- submit $elements.element_name

- wait 1
- wait 15
- wait $data.duration
- wait $env.DURATION

- wait-for $"#asychronously-loaded-content"
- wait-for $imported_page_model.elements.delayed_element_name
- wait-for $elements.delayed_element_name

- reload

- forward

- back

Assertions¶

Assertions are used to verify that the browser is in the desired state.

Syntax¶

An assertion takes the form of an identifier followed by an operator and an optional value.

{identifier} {operator} [{value}]

-----------------------------------

identifier:
    <identifier>

operator:
    excludes
    includes
    is
    is-not
    exists
    not-exists
    matches

value:
    <string>

Operators¶

is
is-not
exists
not-exists
includes
excludes
matches

Is¶

Checks if the value being compared equals a given value.

{identifier} is "{value}"

-------------------------

identifier:
    <identifier>

{value}:
    <string>

Arguments¶

`identifier`	<identifier>	An identifier.
`value`	<string>	A string, in double quotes.

Examples¶

- $".selector" is "Hello!"
- $".banner .image".src is "http://example.com/images/banner.png"
- $page.title is "Homepage"
- $page.title is "\"Homepage\"" # (literal double quotes)
- $imported_page_model.elements.sign_in_button is "Sign in"
- $elements.element_name is $page.url # odd but valid
- $elements.element_name is $data.expected_element_value
- $elements.element_name is $env.KEY

Is-not¶

Checks if the value being compared does not equal a given value.

{identifier} is-not "{value}"

-----------------------------

identifier:
    <identifier>

{value}:
    <string>

Arguments¶

`identifier`	<identifier>	An identifier.
`value`	<string>	A string, in double quotes.

Examples¶

- $".heading" is-not "Error"
- $".user-container .status".data-status is-not "active"
- $page.title is-not "Homepage"
- $page.title is-not "\"Homepage\"" (literal double quotes)
- $imported_page_model.elements.sign_in_button is-not "Sign in"
- $elements.element_name is-not $page.url # odd but valid
- $elements.element_name is-not $data.expected_element_value
- $elements.element_name is-not $env.KEY

Exists¶

Checks if an element exists on the page.

{identifier} exists

-------------------------------

identifier:
    <identifier>

Arguments¶

`identifier`	<identifier>	An identifier.

Examples¶

- $".selector" exists
- $".profile":data-image exists
- $"#logo" exists
- $imported_page_model.elements.sign_in_button exists
- $element_name exists

Not-Exists¶

Checks if an element does not exist on the page.

{identifier} not-exists

-------------------------------

identifier:
    <identifier>

Arguments¶

`identifier`	<identifier>	An identifier.

Examples¶

- $".username-field":disabled not-exists
- $"#logo" not-exists
- $imported_page_model.elements.sign_in_button not-exists
- $element_name not-exists

Includes¶

Checks if the value being compared contains a given value.

{identifier} includes "{value}"

-------------------------------

identifier:
    <identifier>

{value}:
    <string>

Everything after the includes keyword (except the space after the keyword) is the value to be used.

Arguments¶

`identifier`	<identifier>	An identifier.
`value`	<string>	A string, in double quotes.

Examples¶

- $".heading" includes "welcome"
- $".heading".title includes "welcome"
- $page.title includes "page"
- $imported_page_model.elements.sign_in_button includes "Sign"
- $elements.element_name includes $page.url # odd but valid
- $elements.element_name includes $data.key
- $elements.element_name includes $env.KEY

Excludes¶

Checks if the value being compared does not contain a given value.

{identifier} excludes "{value}"

-------------------------------

identifier:
    <identifier>

{value}:
    <string>

Everything after the excludes keyword (except the space after the keyword) is the value to be used.

Arguments¶

`identifier`	<identifier>	An identifier.
`value`	<string>	A string, in double quotes.

Examples¶

- $".heading" includes "error"
- $".heading".title includes "error"
- $page.title excludes "error"
- $imported_page_model.elements.sign_in_button excludes "Log"
- $elements.element_name excludes $page.url # odd but valid
- $elements.element_name excludes $data.key
- $elements.element_name excludes $env.KEY

Matches¶

Checks if the value being compared matches a given regular expression.

{identifier} matches "{regex}"

------------------------------

identifier:
    <identifier>

{regex}:
    <regular-expression>

The value to be used must be within double quotes. The double quotes are not part of the value.

Arguments¶

`identifier`	<identifier>	An identifier.
`regex`	<regular-expression>	A regular expression.

Examples¶

- $".form .profile" matches "/.*/"
- $".form":action matches "/\/login\//"
- $page.title matches "/homepage$/i"
- $imported_page_model.elements.sign_in_button matches "/^Sign/"
- $elements.element_name matches $data.regex
- $elements.element_name matches $env.REGEX # odd way to go about it but valid

Example List¶

- $".selector" is "Hello!"
- $".banner .image".src is "http://example.com/images/banner.png"
- $page.title is "Homepage"
- $page.title is "\"Homepage\"" # (literal double quotes)
- $imported_page_model.elements.sign_in_button is "Sign in"
- $elements.element_name is $page.url # odd but valid
- $elements.element_name is $data.expected_element_value
- $elements.element_name is $env.KEY

- $".heading" is-not "Error"
- $".user-container .status".data-status is-not "active"
- $page.title is-not "Homepage"
- $page.title is-not "\"Homepage\"" (literal double quotes)
- $imported_page_model.elements.sign_in_button is-not "Sign in"
- $elements.element_name is-not $page.url # odd but valid
- $elements.element_name is-not $data.expected_element_value
- $elements.element_name is-not $env.KEY

- $".heading" includes "welcome"
- $".heading".title includes "welcome"
- $page.title includes "page"
- $imported_page_model.elements.sign_in_button includes "Sign"
- $elements.element_name includes $page.url # odd but valid
- $elements.element_name includes $data.key
- $elements.element_name includes $env.KEY

- $".heading" includes "error"
- $".heading".title includes "error"
- $page.title excludes "error"
- $imported_page_model.elements.sign_in_button excludes "Log"
- $elements.element_name excludes $page.url # odd but valid
- $elements.element_name excludes $data.key
- $elements.element_name excludes $env.KEY

- $".form .profile" matches "/.*/"
- $".form":action matches "/\/login\//"
- $page.title matches "/homepage$/i"
- $imported_page_model.elements.sign_in_button matches "/^Sign/"
- $elements.element_name matches $data.regex
- $elements.element_name matches $env.REGEX # odd way to go about it but valid

- $".selector" exists
- $".profile":data-image exists
- $"#logo" exists
- $imported_page_model.elements.sign_in_button exists
- $element_name exists

- $".username-field":disabled not-exists
- $"#logo" not-exists
- $imported_page_model.elements.sign_in_button not-exists
- $element_name not-exists

Identifiers¶

An identifier uniquely identifies an element within a page. It is either a selector, a page model reference or a reference to an element passed to an imported step.

Syntax¶

{selector}|{page-model-reference}|{passed-element-reference}

------------------------------------------------------------

selector:
    <selector>

page-model-reference:
    ${import-name}.elements.{element-name}

passed-element-reference:
    $elements.{element-name}

import-name:
    <string>

element-name:
    <string>

Examples¶

`$"#element-id"`	CSS selector matching against the element `id` attribute.
`$"//*[@id='element-id']"`	XPath expression matching against the element `id` attribute.
`$".listed-item":1`	CSS selector matching all elements with the `listed-item` class name, fetching the first item in the list.
`$".listed-item":3`	CSS selector matching all elements with the `listed-item` class name, fetching the third item in the list.
`$".listed-item":first`	Special position keyword `first` is equivalent to `:1`.
`$".listed-item":1`	Special position keyword `last` fetches the last matching item.
`$google_com.elements.search_button`	Page model element reference.
`$elements.element_name`	Passed element reference.

Selectors¶

A selector uniquely identifies an element, or an attribute of an element, within a page with the use of a CSS selector or XPath expression. A selector is one form of identifier.

Syntax¶

$"{selector-string}"[:{position}][.{attribute-name}]

--------------------------------

selector-string:
    <css-selector>
    <xpath>

position:
    <integer>

attribute_name:
    <string>

Examples¶

`$"#element-id"`	CSS selector matching against the element `id` attribute.
`$"#element-id".title`	CSS selector referencing the `title` attribute of the matched element.
`$"//*[@id='element-id']"`	XPath expression matching against the element `id` attribute.
`$"//*[@id='element-id']".title`	XPath expression referencing the `title` attribute of the matched element.
`$".listed-item":1`	Finding the first matched item from the start of the list.
`$".listed-item":1.title`	Finding the `title` attribute of the first matched item from the start of the list.
`$".listed-item":3`	Finding the third matched item from the start of the list.
`$".listed-item":-1`	Finding the first matched item from the end of the list.
`$".listed-item":-3`	Finding the third matched item from the end of the list.
`$".listed-item":first`	Special position keyword `first` is equivalent to `:1`.
`$".listed-item":last`	Special position keyword `last` is equivalent to `:-1`.

Descendant Selectors¶

A descendant selector uniquely identifies an element (descendant) relative to another element (ancestor). The targeted descendant need not be a direct descendant of the given ancestor.

A descendant selector may be used to target an element by means of an ancestor > descendant relationship which is otherwise not possible or impractical using a single CSS- or XPath-based selector.

Syntax¶

{selector} >> {selector}[ >> {selector}]

----------------------------------------

selector:
    <selector>

Examples¶

`$"#ancestor" >> $"#descendant"`	CSS selector descendant within the scope of a CSS selector ancestor.
`$"//[@id='ancestor']" >> $"//[@id='descendant']"`	XPath expression descendant within the scope of an XPath expression ancestor.
`$"#ancestor" >> $"//*[@id='descendant']"`	XPath expression descendant within the scope of a CSS selector ancestor.
`$"//*[@id='ancestor']" >> $"#descendant"`	CSS selector descendant within the scope of an XPath expression ancestor.
`$"#grandparent" >> $"#parent" >> $"#child"`	Targeting the child of a parent which itself is the child of a grandparent.
`$".grandparent" >> $".parent":2 >> $".child":4`	Targeting the fourth child of the second child of the grandparent.

Page Model¶

A page model holds properties of a web page, avoiding the need to repetitively define such properties at the point that they are needed.

Syntax¶

A page model is a YAML document with url and elements properties.

The url property provides the URL for the web page. The elements property is an optional collection mapping convenience names to selectors.

Element selectors may optionally be prefixed with an element name that has already been defined within the same page model. Doing so scopes the selector to the given element.

url: {url}
elements:
    {element-name-1}: [${predefined-element-name}|{selector} >> ] {selector}
    ...
    {element-name-N}: [${predefined-element-name}|{selector} >> ] {selector}

----------------------------------------------------------------

url:
    <url>

element-name-*:
    <string>

predefined-element-name:
    <string>, name of an element that has previously been defined

selector:
    <selector>

Examples¶

# examples/page/google.com.yml
url: "https://www.google.com"
elements:
  search_input: $".gLFyf.gsfi"
  search_button: $".FPdoLc.VlcLAe input[name=btnK]"

# examples/page/google.com-selector-scoped.yml
url: "https://www.google.com"
elements:
  # finds "[name=q]" within the context of "form[action='/search']"
  # using CSS syntax
  search_input: $"form[action='/search'] input[name=q]"

  # finds "[type=submit]" within the context of "form[action='/search']"
  # using CSS syntax
  search_button: $"form[action='/search'] input[type=submit]"

# examples/page/google.com-element-scoped-literal.yml
url: "https://www.google.com"
elements:
  search_form: $"form[action=/search]"

  # finds "[name=q]" within the context of $"form[action=/search]"
  # using basil parent-child syntax
  search_input: $"form[action=/search]" >> $"[name=q]"

  # finds "[type=submit]" within the context of $"form[action=/search]"
  # using basil parent-child syntax
  search_button: $"form[action=/search]" >> $"[type=submit]"

# examples/page/google.com-element-scoped-reference.yml
url: "https://www.google.com"
elements:
  search_form: $"form[action=/search]"

  # finds "[name=q]" within the context of search_form
  # using basil parent-child syntax
  search_input: $search_form >> $"[name=q]"

  # finds "[type=submit]" within the context of search_form
  # using basil parent-child syntax
  search_button: $search_form >> $"[type=submit]"

Test Steps¶

A step modifies the browser state and verifies the state using:

an optional sequence of actions to be performed to get the browser into a desired state
a set of assertions to verify the browser state

Syntax¶

A step is a YAML object with actions and assertions properties. The actions property is a list of actions. The assertions property is a list of assertions.

Action arguments and assertion values can be represented by named parameters. Values for parameters are provided by test suites.

actions:
  - {action-1}
  ...
  - {action-N}

assertions:
  - {assertion-1}
  ...
  - {assertion-N}

-----------------

action-*:
    <action>

assertion-*:
    <assertion>

Actions are optional. Assertions are non-optional. A step can omit actions.

assertions:
  - {assertion-1}
  ...
  - {assertion-N}

-----------------

assertion-*:
    <assertion>

Examples¶

Literal Values¶

# examples/step/google-assert-open-literal.yml
assertions:
  - $page.url is "https://www.google.com"
  - $page.title is "Google"

Parameterised Values¶

# examples/step/assert-page-open-parameterised.yml
assertions:
  - $page.url is $data.expected_url
  - $page.title is $data.expected_title

Parameterised Elements and Parameterised Values¶

# examples/step/google-query-parameterised.yml
actions:
  - set $elements.search_input to $data.query_term
  - click $elements.search_button

assertions:
  - $page.title is $data.expected_title

No Actions¶

# examples/step/google-homepage-assertions.yml
assertions:
  - $"#hplogo" exists
  - $"a[href*='about']" exists

Tests¶

A test is composed of a series of steps. Steps may be defined directly within a test or may be defined independently and imported into a test. A test may import steps, may import page models and may import data providers.

Syntax¶

A test is a YAML object. The most basic test specifies a browsers collection and starting url and defines steps directly.

config:
    browsers:
        - {browser-1}
        ...
        - {browser-N}
    url: {url}

{step-name-1}:
    {step}
...
{step-name-N}
    {step}

----------------------

browser-*:
    <string>

url:
    <url>

step-name-*:
    <string>

step:
    actions:
      - {action-1}
      ...
      - {action-N}

    assertions:
      - {assertion-1}
      ...
      - {assertion-N}

action-*:
    <action>

assertion-*:
    <assertion>

A step may be defined independently of the test and imported into any number of tests. Import names are user-defined. Import paths are relative to the test.

config:
    browsers:
        - {browser-1}
        ...
        - {browser-N}
    url: {url}

imports:
    steps:
        {step-import-name}: "{import-path}"

{step-name}:
    use: {step-import-name}

-------------------------------------------

step-import-name:
    <string>

import-path:
    <string>, a path to the file to import, relative to the location of the test

browser-*:
    <string>

url:
    <url>

step-name:
    <string>

A test may both define and import steps.

config:
    browsers:
        - {browser-1}
        ...
        - {browser-N}
    url: {url}

imports:
    steps:
        {step-import-name}: "{import-path}"

{step-name-1}:
    use: {step-import-name}
...
{step-name-N}
    {step}

-------------------------------------------

step-import-name:
    <string>

import-path:
    <string>, a path to the file to import, relative to the location of the test

browser-*:
    <string>

url:
    <url>

step-name-*:
    <string>

step:
    actions:
      - {action-1}
      ...
      - {action-N}

    assertions:
      - {assertion-1}
      ...
      - {assertion-N}

action-*:
    <action>

assertion-*:
    <assertion>

Imported steps that require parameters must have the parameter values passed at usage time.

Data can be supplied via the data property of the step. The data property can define one or more data sets. Each data set provides values for the parameters as required by a step.

config:
    browsers:
        - {browser-1}
        ...
        - {browser-N}
    url: {url}

imports:
    steps:
        {step-import-name}: "{import-path}"

{step-name}:
    use: {step-import-name}
    data:
        {data-set-name-1}:
            {parameter-name-1}: {parameter-value-1}
            ...
            {parameter-name-N}: {parameter-value-N}
        ...
        {data-set-name-N}:
            {parameter-name-1}: {parameter-value-1}
            ...
            {parameter-name-N}: {parameter-value-N}

---------------------------------------------------

step-import-name:
    <string>

import-path:
    <string>, a path to the file to import, relative to the location of the test

browser-*:
    <string>

url:
    <url>

step-name:
    <string>

data-set-name-*:
    <string>

parameter-name-*:
    <string>, must match the name of a parameter as required by a step

parameter-value-*:
    <string>

Data can be supplied through the import of a data provider. A data provider is a collection of data sets defined externally to the test.

config:
    browsers:
        - {browser-1}
        ...
        - {browser-N}
    url: {url}

imports:
    steps:
        {step-import-name}: "{import-path}"
    data_providers:
        {data-provider-import-name}: "{import-path}"

{step-name}:
    use: {step-import-name}
    data: {data-provider-import-name}

----------------------------------------------------

step-import-name:
    <string>

import-path:
    <string>, a path to the file to import, relative to the location of the test

browser-*:
    <string>

url:
    <url>

data-provider-import-name:
    <string>

step-name:
    <string>

Page models can be imported and their defined page properties referenced within the test.

config:
    browsers:
        - {browser-1}
        ...
        - {browser-N}
    url: {url}

imports:
    pages:
        {page-import-name}: "{import-path}"

{step-name}:
  actions:
    - open {import_name}

  assertions:
    - {page-import-name}.elements.{element-name} {operator} [{value}]

-----------------------------------------------------------------------

page-import-name:
    <string>

import-path:
    <string>, a path to the file to import, relative to the location of the test

browser-*:
    <string>

url:
    <url>

step-name:
    <string>

element-name:
    <string>, defined within the imported page model

operator:
    <string>, operator for assertion

value:
    <string>, value for assertion

Examples¶

Steps With Literal Values¶

# examples/test/google-search-query-literal.yml
config:
  browsers:
    - chrome
  url: https://www.google.com

"verify Google is open":
  assertions:
    - $page.url is "https://www.google.com"
    - $page.title is "Google"

"query 'example'":
  actions:
    - set $".gLFyf.gsfi" to "example"
    - click $".FPdoLc.VlcLAe input[name=btnK]"

  assertions:
    - $page.title is "example - Google Search"

Parameterised Steps With Inline Data¶

# examples/step/assert-page-open-parameterised.yml
assertions:
  - $page.url is $data.expected_url
  - $page.title is $data.expected_title

# examples/test/google-open-parameterised.yml
config:
  browsers:
    - chrome
  url: https://www.google.com

imports:
  steps:
    assert_opened_page_step: "../step/assert-page-open-parameterised.yml"

"verify Google is open":
  use: assert_opened_page_step
  data:
    # Just a single data set needed here
    0:
      expected_url: $config.url
      expected_title: "Google"

Parameterised Steps With Provided Data¶

# examples/step/google-search-query-parameterised.yml
actions:
  - set $".gLFyf.gsfi" to $data.search_term
  - click $".FPdoLc.VlcLAe input[name=btnK]"

assertions:
  - $page.title is $data.expected_title

# examples/data-provider/google-search-query.yml
foo:
  search_term: "foo"
  expected_title: "foo - Google Search"

bar:
  search_term: "bar"
  expected_title: "bar - Google Search"

# examples/test/google-search-query-parameterised.yml
config:
  browsers:
    - chrome
  url: https://www.google.com

imports:
  steps:
    query_step: "../step/google-search-query-parameterised.yml"
  data_providers:
    google_search_query_data: "../data-provider/google-search-query-parameterised.yml"

"verify Google is open":
  assertions:
    - $page.title is "Google"

"query":
  use: query_step
  data: google_search_query_data

Steps Imported Into the Test With Additional Assertions¶

# examples/step/google-assert-open-literal.yml
assertions:
  - $page.url is "https://www.google.com"
  - $page.title is "Google"

# examples/step/google-query-example-literal.yml
actions:
  - set $".gLFyf.gsfi" to "example"
  - click $".FPdoLc.VlcLAe input[name=btnK]"

assertions:
  - $page.title is "example - Google Search"

# examples/test/google-import-assert-open-import-query-additional-assertions.yml
config:
  browsers:
    - chrome
  url: https://www.google.com

imports:
  steps:
    google_assert_open: "../step/google-assert-open-literal.yml"
    google_query_example: "../step/google-query-example-literal.yml"

"verify Google is open":
  use: google_assert_open

  assertions:
    - $page.title excludes "error"

"query 'example'":
  use: google_query_example

Imported Page Model Used Directly Within Test¶

# examples/page/google.com.yml
url: "https://www.google.com"
elements:
  search_input: $".gLFyf.gsfi"
  search_button: $".FPdoLc.VlcLAe input[name=btnK]"

# examples/test/google-inline-tests-with-page-model.rst
config:
  browsers:
    - chrome
  url: google_com.url

imports:
  pages:
    google_com: "../page/google.com.yml"

"verify Google is open":
  assertions:
    - $page.title is "Google"

"query 'example'":
  actions:
    - set $google_com.elements.search_input to "example"
    - click $google_com.elements.search_button

  assertions:
    - $page.title is "example - Google Search"

Imported Page Model Elements Passed To Imported Step¶

# examples/page/google.com.yml
url: "https://www.google.com"
elements:
  search_input: $".gLFyf.gsfi"
  search_button: $".FPdoLc.VlcLAe input[name=btnK]"

# examples/step/google-query-parameterised.yml
actions:
  - set $elements.search_input to $data.query_term
  - click $elements.search_button

assertions:
  - $page.title is $data.expected_title

# examples/test/google-imported-steps-with-page-model.yml
config:
  browsers:
    - chrome
  url: google_com.url

imports:
  pages:
    google_com: "../page/google.com.yml"
  steps:
    google_query: "../step/google-query-parameterised.yml"

"verify Google is open":
  assertions:
    - $page.title is "Google"

"query 'example'":
  use: google_query
  data:
    0:
      query_term: "foo"
      expected_title: "foo - Google Search"

  elements:
    search_input: $google_com.elements.search_input
    search_button: $google_com.elements.search_button

Tests Suites¶

A test suite is a series of tests. Tests are defined externally and imported into a suite.

Syntax¶

A test suite is a YAML list.

- "{import-path-1}"
- ...
- "{import-path-N}"

-------------------------

import-path-*:
    <string>, a path to the file to import, relative to the location of the suite

Example¶

# examples/test/google-search-query-literal.yml
config:
  browsers:
    - chrome
  url: https://www.google.com

"verify Google is open":
  assertions:
    - $page.url is "https://www.google.com"
    - $page.title is "Google"

"query 'example'":
  actions:
    - set $".gLFyf.gsfi" to "example"
    - click $".FPdoLc.VlcLAe input[name=btnK]"

  assertions:
    - $page.title is "example - Google Search"

# examples/test-suite/google-open-and-query.yml
- "../test/google-search-query-literal.yml"

Data Providers¶

A data provider defines a data collection to be used with a parameterised test.

A data collection contains one or more data sets. Each data set has a name and provides values for the parameters as required by a test.

Syntax¶

A data provider defines a data collection. A data collection is a YAML object with user-defined named properties. Each property name is the name of a data set. Each data set maps parameter names to parameter values.

{data-set-name-1}
    {parameter-name-1}: {parameter-value-1}
    ...
    {parameter-name-N}: {parameter-value-N}

...

{data-set-name-N}
    {parameter-name-1}: {parameter-value-1}
    ...
    {parameter-name-N}: {parameter-value-N}

-------------------------------------------

data-set-name-*:
    <string>, must be unique with a data collection

parameter-name-*:
    <string>, must match the name of a parameter as required by a test

parameter-value-*:
    <string>

Examples¶

# examples/data-provider/google-search-query.yml
foo:
  search_term: "foo"
  expected_title: "foo - Google Search"

bar:
  search_term: "bar"
  expected_title: "bar - Google Search"

# examples/data-provider/example-sign-in.yml
user1:
  username: $env.TEST_USER_1_USERNAME
  password: $env.TEST_USER_1_PASSWORD

user2:
  username: $env.TEST_USER_2_USERNAME
  password: $env.TEST_USER_2_PASSWORD