selsa

Selenium, Saucelabs, Webdriver and lots of despair

selsa lets you run tests agains a local selenium or against Sauce Labs both locally or during your CI.

The objective of selsa is to do as little as possible. It provides extensive debug logs and expose its underlying modules

• selenium-standalone,
• sauce-connect-launcher
• selenium-sauce
• webdriverio

Many projects aim for the best possible developer experience by hiding as much of the underlying complexity as possible, and by combining it with test runners and dev servers. But not selsa. It puts debug-ability first.

Install

npm install --save selsa

Usage

This example uses tap, but selsa can be used with any test framework

const selsa = require('selsa')
const test = require('tap').test
 
const selsaOptions = {
  client: 'saucelabs:chrome'
}
 
test('Landing page', (t) => {
  selsa(selsaOptions, (error, api) => {
    t.tearDown(api.tearDown)
 
    api.browser
      .url('http://localhost:8000')
      .getTitle()
        .then((title) => {
          t.equals(title, 'My demo page')
          t.end()
        }))
  })
})

Testing on Travis

selsa plays nicely with Travis. If you want to test using selenium, make sure to only test in Firefox as it's the only supported browser, and add the following lines:

before_install:
  - export DISPLAY=:99.0
  - sh -e /etc/init.d/xvfb start

You can also install a more recent firefox version. I recommend v47.0.2, as I could not get 48 to work and 49 later only works with the latest gecko drive and Selenium 3, which requires Java 8, and that is not available on Travis by default. You will also need to set SELENIUM_VERSION to 2.53.1 on Travis unless you have Java 8 available.

addons:
  firefox: "47.0.2"

I would also recommend to cache the saucelabs binaries for faster build

cache:
  directories:
  - node_modules/selenium-standalone/.selenium

Set SAUCELABS_USERNAME & SAUCELABS_ACCESS_KEY as env variables, and test as many different browser configurations using the env.matrix setting, e.g.

env:
  matrix:
  - CLIENT=selenium:firefox SELENIUM_VERSION=2.53.1
  - CLIENT=saucelabs:chrome
  - CLIENT="saucelabs:internet explorer:10:Windows 8"
  - CLIENT="saucelabs:iphone:8.4:OS X 10.11"

Options

selsa(selsaOptions, (error, api) => {})

selsa can be configured by passing selsaOptions as shown above or by using ENV variables, listed below the option name.

Setting (ENV)	Default / Example
client (CLIENT) (saucelabs|selenium):browserName:browserVerion:platform, e.g. 'saucelabs:internet explorer:10:win10'	'selenium:chrome'
timeout (CLIENT_TIMEOUT) webdriver timouts	180000
selenium.hub (SELENIUM_HUB) Url to selenium hub	'http://localhost:4444/wd/hub/status'
selenium.standalone.version (SELENIUM_VERSION) Selenium version to install	see selenium-standalone defaults
saucelabs.connect.username (SAUCELABS_USERNAME) Saucelabs username for authentication	e.g. 'pat'
saucelabs.connect.accessKey (SAUCELABS_ACCESS_KEY) Saucelabs access key for authentication	e.g. 'abcd5678-1234-1234-1234-abcd5678abcd'
saucelabs.connect.retries (SAUCELABS_CONNECT_RETRIES) Amount of retries if case of connection error (excl. auth error)	10
saucelabs.connect.retryTimeout (SAUCELABS_CONNECT_RETRY_TIMEOUT) Timeout between connection retries in ms	Random between 5000 and 60000
saucelabs.webdriver.desiredCapabilities.idle-timeout (SAUCELABS_IDLE_TIMEOUT) SauceLabs Idle Test Timeout	90, allowed maximum is 1000
saucelabs.webdriver.desiredCapabilities.max-duration (SAUCELABS_MAX_DURATION) SauceLabs Maximum Test Duration	1800, allowed maximum is 10800
saucelabs.webdriver.desiredCapabilities.max-duration (SAUCELABS_COMMAND_TIMEOUT) SauceLabs Command Timeout	300, allowed maximum is 600

Learnings

Timeouts

When running your tests agains Sauce Labs, make sure to give enough timeouts. Connecting and spawning from Travis often time takes over 30s.

Unreliable builds

No matter how much we try, Selenium tests and random timeouts and other errors seem to occur very often. If you tried everything else to make your tests 100% reliable on travis, try travis-retry

Credits

The abstraction of Selenium vs Sauce Labs is partly inspired by selenium-sauce

License

Apache License