e2e-ui-tests-iauditor-webapp

End-to-End Test Automation using Cypress

Recent Build

What is Cypress?

Cypress is a next generation front end testing tool built for the modern web. We address the key pain points developers and QA engineers face when testing modern applications. Click here to learn more about Cypress.

How to Get Started?

1. To get started, first follow the setup steps for setting up Frontend Reactor here up to the lerna bootstrap step.
2. Retrieve the iAuditor account password for the test environment in 1Passowrd here.
3. You will need to set this password to an environment variable. Open or create ~/.zshrc and add the following anywhere in the file (substituting the value for the information from the previous step)

export TEST_ACCOUNT_PASSWORD=<passwordFrom1Password>

4. Restart your shell so that these lines will be sourced: exec zsh
5. In your terminal, type yarn test:e2e-ui and follow the prompts (eg. local, sandpit). -> This should open the Cypress UI. First time it may time out to verifying Cypress, if that happens try again with same command.
6. Click on the desired spec to run, eg. actions.spec.ts to run the Actions tests.
7. Navigate to your preferred product area under cypress/tests and start testing!

Running sensor end-to-end tests?

1. Retrieve s12 inlet token from 1Password here. This will be used to generate test readings against a test/virtual device.
2. You will need to set this token to an environment variable. Open or create ~/.zshrc and add the following anywhere in the file (substituting the value for the information from the previous step)

export TEST_IOT_S12_INLET_TOKEN=<tokenFrom1Password>

Running on LambdaTest

1. To run on LambdaTest you will need a LambdaTest account. You should be able to log in to automation.lambdatest.com.
2. Locate your access key details at the link above (at the top in the middle).
3. You will need to set environment variables with the details. Open or create ~/.zshrc and add the following anywhere in the file (substituting the values for the information from the previous step):

export LT_USERNAME=<yourLambdaTestUserName>
export LT_ACCESS_KEY=<yourLambdaTestAccessKey>

4. Restart your shell so that these lines will be sourced: exec zsh
5. Run yarn test:e2e-ui and choose LambdaTest when prompted.

In order to access LambdaTest portal, please login to LambdaTest using Okta SSO using this Self-Serve guide. Only use Okta SSO. Do not use Login as Google Account on LambdaTest portal otherwise you will end up creating your own free organisation in the LambdaTest.

Environment Variables

• TEST_ACCOUNT_PASSWORD - Account password to login to the test accounts in SANDPIT. You can retrieve password from QE Vault in 1Password. Security Team recommends not to run tests against PROD version of WebApp from your local environment. You should only store sandpit test accounts password in your local environment variable.
• LT_USERNAME & LT_ACCESS_KEY - Credential required to run tests on the LambdaTest from your local IDE.

To start a build from BuildKite, create a new build of this pipeline and (optionally) set the following environment variables:

• TEST_ENV - Indicates where to run tests. Expected values are: sandpit, prod, or dev environment key (actions, inspections, analytics etc).
• REGION - Indicates in which region to run tests. Expected values are: us, au or eu (au & eu regions are only compatible with 'prod' value for ENV).
• SPEC - The name of a directory under cypress/tests/ containing spec files to run (eg. "collaboration" or "sensors"). Use this to run only one team's tests. Leave unset to run all specs.

Triggering test from other pipelines

• Add this code block to pipeline.yml file to be able to trigger Cypress E2E smoke tests in other pipelines.

- name: ":cypress: E2E UI Smoke Tests (Cypress) Sandpit"
    branches: "master"
    trigger: "e2e-ui-tests-iauditor-webapp"
    label: "Running all E2E Smoke Tests"
    build:
      env:
        TEST_ENV: "sandpit"
        REGION: "us"
      branch: "master"

Generating test reports

HTML Reports

It is possible to generate an interactive HTML report from a test run.

1. Run desired test specs in a headless mode (not the local option) with yarn test:e2e-ui.
2. Once your tests have been run, do yarn test:e2e-ui:report and the HTML report will be merged and open in your browser.

ReportPortal

ReportPortal is enabled for End-2-End test execution. It means every test execution from the Master branch on Buildkite is logged into the ReportPortal.

If you are developing tests locally or want to run tests locally, add your ReportPortal token in reportportalAgentJsCypressReporterOptions.token field of cypress.json file. Refer to How to get token for more information.

Refer to ReportPortal Documentation for more information and how to access it.

Testing in Development Environment

[coming soon]

Prerequisites

1. iAuditor Test Data Generation Tool must be run on dev environment prior to executing tests. Tests rely on structural data to be present in the environment where it executes tests.
2. Set REGION=us, and ENV={devEnv}, where devEnv is a key of your development environment, such as actions, inspections.

Solution & Structure

Refer to this confluence document for detailed solution designs and decisions.

Project Structure

frontend-reactor
├── e2e-ui-tests
│   ├── cypress                                # main Cypress directory
│   │   ├── tests                              # main directory with e2e tests
│   │   │   ├── actions                        # e2e tests directory for Actions team
│   │   │   │   ├── actions-helpers.ts
│   │   │   │   └── actions.spec.ts
│   │   │   ├── analytics                      # e2e tests directory for Analytics team
│   │   │   │   ├── analytics-helpers.ts
│   │   │   │   ├── analytics.spec.ts
│   │   │   ├── collaboration                  # e2e tests directory for Collaboration team
│   │   │   │   ├── collaboration-helpers.ts
│   │   │   │   └── collaboration.spec.ts
│   │   │   ├── headsup                        # e2e tests directory for Headsup team
│   │   │   │   ├── headsup-helpers.ts
│   │   │   │   └── headsup.spec.ts
│   │   │   ├── inspections                    # e2e tests directory for Inspections team
│   │   │   │   ├── inspections-helpers.ts
│   │   │   │   └── inspections.spec.ts
│   │   │   ├── issues                         # e2e tests directory for issues team
│   │   │   │   ├── issues-helpers.ts
│   │   │   │   └── issues.spec.ts
│   │   │   ├── login                          # e2e tests directory for login team
│   │   │   │   ├── login-helpers.ts
│   │   │   │   └── login.spec.ts
│   │   │   ├── onboarding                     # e2e tests directory for onboarding team
│   │   │   │   ├── onboarding-helpers.ts
│   │   │   │   └── onboarding.spec.ts
│   │   │   ├── reporting                      # e2e tests directory for reporting team
│   │   │   │   ├── reporting-helpers.ts
│   │   │   │   └── reporting.spec.ts
│   │   │   └── sensors                        # e2e tests directory for sensors team
│   │   │       ├── sensors-helpers.ts
│   │   │       └── sensors.spec.ts
│   │   │
│   │   ├── plugins                            # plugins to extend Cypress
│   │   │   └── index.ts
│   │   ├── structural-test-data               # Structural Test Data
│   │   │   └── data.ts
│   │   └── support                            # Custom commands for Cypress
│   │       ├── commands.ts
│   │       ├── helpers.ts
│   │       ├── index.d.ts
│   │       └── index.ts

Best Practices

What is End-2-End tests?

When to Write E2E Test

• If it's a critical user journey.
• If there isn't any other way you can test.

When NOT to Write E2E Test

• If you can have test coverage in the Unit level.
• If you can have test coverage in the API/Integration level.
• If it's not a critical user journey.
• If it's negative flow.
• For business rules.

Do's

• Run shared code before each test.
• Test case descriptions should be general and informative.
• Methods name should be general and informative.
• Write selectors that are resilient to changes (Use data-anchor).
• Code should be Readable and Maintainable.

Don’ts

• Don’t add static wait time. Never.
• Don’t write dependent tests between spec files.
• Don’t use unstable selectors. If an element does not have data-anchor, pair with Frontend Engineer to add data-anchor for the element.
• Don’t ignore flaky tests.

Know How

I'm getting this weird prompt and it is hanging my test

If you are writing tests that are related to components utilising copy-to-clipboard as a dependency chances are you will face an issue where the test hangs and displays a prompt. This is due to how copy-to-clipboard fallback mechanics works.

Issue link: https://github.com/cypress-io/cypress/issues/2739

To mitigate that add at the start of your test

// Disable window prompt which is used in link creation by copy-to-clipboard library
// This prompt pauses test execution during `cypress open`
cy.window().then(win => {
  cy.stub(win, 'prompt').returns('DISABLED WINDOW PROMPT');
});

I have a shortened link in the form of sfty.io when I try to call it with cy.visit it gave me an error

When testing with shortened links or link of different origin e.g. google.com you can't actually use cy.visit() to move to that site because Cypress disallows it. In order to mitigate that we have a command cy.forceVisit(link) to allow you to visit sites of different origin.

reference: https://stackoverflow.com/questions/62053892/cypress-how-to-visit-a-url-of-a-different-origin

Adding users to an existing test organisation

User accounts used for e2e-ui-tests are in sync across all environments. It is important to keep user accounts in sync to have test script independent and runnable against any environment without any changes to the script itself. To add a new user to the existing organisation, follow below steps:

1. Add a user in the credentials in cypress/structural-test-data/data.ts. Make sure to follow the pattern for the region.

2. Add a user in the qaOrgUsers := []organisations.User array in Test Data Generation repository as per below example

    {
      Username:  "[e-mail]",
      Password:  user1.Password,
      Firstname: "Actions",
      Lastname:  "WebApp",
    },

3. Raise a pull request.