codeceptjs-resemblehelper
Helper for resemble.js, used for image comparison in tests with Playwright, Webdriver, Puppeteer, Appium, TestCafe!
codeceptjs-resemblehelper is a CodeceptJS helper which can be used to compare screenshots and make the tests fail/pass based on the tolerance allowed.
If two screenshot comparisons have difference greater then the tolerance provided, the test will fail.
NPM package: https://www.npmjs.com/package/codeceptjs-resemblehelper
To install the package, just run npm install codeceptjs-resemblehelper
.
Configuration
This helper should be added in codecept.json/codecept.conf.js
Example:
{
"helpers": {
"ResembleHelper" : {
"require": "codeceptjs-resemblehelper",
"baseFolder": "./tests/screenshots/base/",
"diffFolder": "./tests/screenshots/diff/",
"prepareBaseImage": true
}
}
}
To use the Helper, users may provide the parameters:
baseFolder
: Mandatory. This is the folder for base images, which will be used with screenshot for comparison.
diffFolder
: Mandatory. This will the folder where resemble would try to store the difference image, which can be viewed later.
prepareBaseImage
: Optional. When true
then the system replaces all of the baselines related to the test case(s) you ran. This is equivalent of setting the option prepareBaseImage: true
in all verifications of the test file.
compareWithImage
: Optional. A custom filename to compare the screenshot with. The compareWithImage
file must be located inside the baseFolder
.
Usage
These are the major functions that help in visual testing:
First one is the seeVisualDiff
which basically takes two parameters
-
baseImage
Name of the base image, this will be the image used for comparison with the screenshot image. It is mandatory to have the same image file names for base and screenshot image. -
options
options can be passed which includeprepaseBaseImage
,tolerance
andneedsSameDimension
.
/**
* Check Visual Difference for Base and Screenshot Image
* @param baseImage Name of the Base Image (Base Image path is taken from Configuration)
* @param options Options ex {prepareBaseImage: true, tolerance: 5, needsSameDimension: false} along with Resemble JS Options, read more here: https://github.com/rsmbl/Resemble.js
* @returns {Promise<void>}
*/
async seeVisualDiff(baseImage, options) {}
Second one is the seeVisualDiffForElement
which basically compares elements on the screenshot, selector for element must be provided.
It is exactly same as seeVisualDiff
function, only an additional selector
CSS|XPath|ID locators is provided
/**
* See Visual Diff for an Element on a Page
*
* @param selector Selector which has to be compared, CSS|XPath|ID
* @param baseImage Base Image for comparison
* @param options Options ex {prepareBaseImage: true, tolerance: 5} along with Resemble JS Options, read more here: https://github.com/rsmbl/Resemble.js
* @returns {Promise<void>}
*/
async seeVisualDiffForElement(selector, baseImage, options){}
Note:
seeVisualDiffForElement
only works when the page for baseImage is open in the browser, so that WebdriverIO can fetch coordinates of the provided selector.
Third one is the screenshotElement
which basically takes screenshot of the element. Selector for the element must be provided. It saves the image in the output directory as mentioned in the config folder.
I.screenshotElement("selectorForElement", "nameForImage");
Note: This method only works with puppeteer.
Finally to use the helper in your test, you can write something like this:
Feature('to verify monitoried Remote Db instances');
Scenario('Open the System Overview Dashboard', async (I, adminPage, loginPage) => {
adminPage.navigateToDashboard("OS", "System Overview");
I.saveScreenshot("Complete_Dashboard_Image.png");
adminPage.applyTimer("1m");
adminPage.viewMetric("CPU Usage");
I.saveScreenshot("Complete_Metric_Image.png");
});
Scenario('Compare CPU Usage Images', async (I) => {
// setting tolerance and prepareBaseImage in the options array
I.seeVisualDiff("Complete_Metric_Image.png", {prepareBaseImage: false, tolerance: 5});
// passing a selector, to only compare that element on both the images now
// We need to navigate to that page first, so that webdriver can fetch coordinates for the selector
adminPage.navigateToDashboard("OS", "System Overview");
I.seeVisualDiffForElement("//div[@class='panel-container']", "Complete_Dashboard_Image.png", {prepareBaseImage: false, tolerance: 3});
});
Note:
seeVisualDiff
andseeVisualDiffElement
work only when the dimensions of the screenshot as well as the base image are same so as to avoid unexpected results.
Ignored Box
You can also exclude part of the image from comparison, by specifying the excluded area in pixels from the top left.
Just declare an object and pass it in options as ignoredBox
:
const box = {
left: 0,
top: 10,
right: 0,
bottom: 10
};
I.seeVisualDiff("image.png", {prepareBaseImage: true, tolerance: 1, ignoredBox: box});
After this, that specific mentioned part will be ignored while comparison.
This works for seeVisualDiff
and seeVisualDiffForElement
.
resemble.js Output Settings
You can set further output settings used by resemble.js. Declare an object specifying them and pass it in the options as outputSettings
:
const outputSettings = {
ignoreAreasColoredWith: {r: 250, g: 250, b: 250, a: 0},
// read more here: https://github.com/rsmbl/Resemble.js
};
I.seeVisualDiff("image.png", {prepareBaseImage: true, tolerance: 1, outputSettings: outputSettings});
Refer to the resemble.js documentation for available output settings.
Skip Failure
You can avoid the test fails for a given threshold but yet generates the difference image.
Just declare an object and pass it in options as skipFailure
:
I.seeVisualDiff("image.png", {prepareBaseImage: true, tolerance: 1, skipFailure: true});
After this, the system generates the difference image but does not fail the test.
This works for seeVisualDiff
and seeVisualDiffForElement
.
Allure Reporter
Allure reports may also be generated directly from the tool. To do so, add
"plugins": {
"allure": {}
}
in the config file.
The attachments will be added to the report only when the calulated mismatch is greater than the given tolerance.
Set output
to where the generated report is to be stored. Default is the output directory of the project.
AWS Support
AWS S3 support to upload and download various images is also provided.
It can be used by adding the aws code inside "ResembleHelper"
in the "helpers"
section in config file. The final result should look like:
{
"helpers": {
"ResembleHelper" : {
"require": "codeceptjs-resemblehelper",
"baseFolder": "<location of base folder>",
"diffFolder": "<location of diff folder>",
"aws": {
"accessKeyId" : "<Your AccessKeyId>",
"secretAccessKey": "<Your secretAccessKey>",
"region": "<Region of Bucket>",
"bucketName": "<Bucket Name>"
}
}
}
}
When this option has been provided, the helper will download the base image from the S3 bucket.
This base image has to be located inside a folder named "base".
The resultant output image will be uploaded in a folder named "output" and diff image will be uploaded to a folder named "diff" in the S3 bucket.
If the prepareBaseImage
option is marked true
, then the generated base image will be uploaded to a folder named "base" in the S3 bucket.
Note: The tests may take a bit longer to run when the AWS configuration is provided as determined by the internet speed to upload/download images.
Other S3 Providers
The same configuration as above, but with endpoint field:
{
"helpers": {
"ResembleHelper" : {
"require": "codeceptjs-resemblehelper",
"baseFolder": "<location of base folder>",
"diffFolder": "<location of diff folder>",
"aws": {
"accessKeyId" : "<Your AccessKeyId>",
"secretAccessKey": "<Your secretAccessKey>",
"region": "<Region of Bucket>",
"bucketName": "<Bucket Name>",
"endpoint": "<Endpoint of Bucket>"
}
}
}
}
Compare with custom image
Usually, every screenshot needs to have the same filename as an existing image inside the baseFolder
directory. To change this behavior, you can use the compareWithImage
option and specify a different image inside the baseFolder
directory.
This is useful, if you want to compare a single screenshot against multiple base images - for example, when you want to validate that the main menu element is identical on all app pages.
I.seeVisualDiffForElement("#element", "image.png", {compareWithImage: "dashboard.png"});
I.seeVisualDiffForElement("#element", "image.png", {compareWithImage: "account.png"});
Or, in some cases there are intended visual differences for different browsers or operating systems:
const os = "win32" === process.platform ? "win" : "mac";
// Compare "image.png" either with "image-win.png" or "image-mac.png":
I.seeVisualDiff("image.png", {compareWithImage: `image-${os}.png`});
Known Issues:
Issue in Windows where the image comparison is not carried out, and therefore no Mismatch Percentage is shown. See 'loadImageData' function in resemble.js