msportalfx-mock
The Azure Portal Mocking Framework provides extension teams the ability to dynamically record arbitrary network requests while running E2E tests and replay them during subsequent runs to prevent long-running or costly operations from being performed with each test run.
The Framework spawns a NodeJS Express server and patches Portal to redirect specified network requests to the server which acts as a proxy to the target when recording and as a mock provider when replaying.
It includes several built-in target handlers for ARM, storage data plane, and static data fixtures and provides interfaces for defining additional custom target handlers.
Benefits
- Reduces load on back-end services thus improving capacity and reducing costs
- While in use within the IaaS extension team, framework eliminated on average 300k-500k network requests and 9k-13k ARM deployments per day.
- Speeds test execution
- For long-running operations like deployments, replaying them is almost immediate. So 10+ minute runs are reduced to less than a minute.
- Easily refresh test data
- Rather than having static data fixtures saved into a repository, by loading and storing mock data to a storage account, teams can easily refresh the data periodically to quickly catch breaking changes to back-end APIs.
- Invisible to tests
- Typically won't require any changes to existing tests since it just acts as a middle-man between Portal and the targets.
Installation
npm install --save-dev msportalfx-mock
Usage
Server
MockFx starts up a NodeJS Express server running on a specified port to mock a set of targets:
// Create an instance of the built-in ARM target
const armTarget = createARMTarget({
host: "management.azure.com",
loadRequests: async (context: MockFx.Context): Promise<MockFx.Request[]> => {
// Load mocked requests from storage (local file, storage account, etc.) for replay
return [];
},
storeRequests: async (context: MockFx.Context, requests: MockFx.Request[]): void => {
// Store recorded requests to storage
},
});
// Create MockFx instance
const mockFx = await MockFx.create({
port: 5000,
targets: [armTarget],
});
// Start the server
await mockFx.start();Certificates
MockFx handles creating, installing, and managing self-signed SSL certificates for the server to use:
| Issued To | Issued By | Friendly Name | Expiration | Install Location(s) |
|---|---|---|---|---|
| localhost | MockFx Server Root | MockFx Server | Every 90 days | Cert:\LocalMachine\My |
| MockFx Server Root | MockFx Server Root | Every 90 days | Cert:\LocalMachine\My Cert:\LocalMachine\CA Cert:\LocalMachine\Root |
Registering Tests
Since MockFx stands up a single server for all tests, in order to support running tests in parallel, we require 2 keys to register a test with the server:
- A unique run ID which splits up concurrent test runs. This can usually just be a random string.
- A unique test ID within a run. Usually the test name or relative filepath to the test in source control.
Every test in a run must be registered by calling either mockFx.registerTest(context) directly or over HTTPS POST using the register end-point
from MockFx.getEndpoints(...).registerTest with the test's context object as its body. If using the network API, the MockFx server must be started first with mockFx.start().
const runId = 'uniqueRunID';
const testId = 'testID';
await mockFx.registerTest({
mode: MockFx.Mode.Record,
runId,
testId,
targets: [armTarget.name],
});or
await mockFx.start();
const mockFxPort = 5000;
const runId = 'uniqueRunID';
const testId = 'testID';
const registerEndpoint = MockFx.getEndpoints(mockFxPort, runId, testId).registerTest;
const context: MockFx.Context = {
mode: MockFx.Mode.Record,
runId,
testId,
targets: [armTarget.name],
};
const body = JSON.stringify(context);
const request = https.request({
...url.parse(registerEndpoint),
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': body.length
}
});
request.write(body);
request.end();Mocking Portal Requests
To mock requests from Portal, client code must be executed in the browser in both Portal's execution context and in every extension's context.
We can use the Portal's patch query parameter to automatically inject the client plug-in into all extensions loaded in a session. The plug-in overrides the
global XMLHttpRequest.open method (which is used to specify the URL for a request), and based on the request's URL, conditionally redirects
the request to the mock server if its host matches a registered target. This method works with direct XHR requests and when using Axios or the
Portal's request methods that use jQuery since they all use XHRs under the hood. It does not support the fetch API, but that could
be easily supported in the future if the need arises.
MockFx provides an end-point that serves the client code at MockFx.getEndpoints(...).plugin. You can pass this URL into Portal's patch parameter so that whenever an
extension is loaded, the client code will automatically be executed:
const clientPluginURL = MockFx.getEndpoints(mockFxPort, runId, testId).plugin;
const portalURL = 'https://portal.azure.com/' +
'?feature.canmodifyextensions=true' +
'&feature.prewarming=false' +
'#home?patch=["' + encodeURI(clientPluginURL) + '"]';To patch Portal itself, you can get the client plug-in code directly with mockFx.getPluginCode(...) and use your web driver to execute the code in the main execution context:
const clientPluginCode = mockFx.getPluginCode(runId, testId);
webdriver.executeScript(clientPluginCode);Client Plug-in Code
(function () {
// Config generated at run-time based on the test's registration
const config = {
hosts: ["MANAGEMENT.AZURE.COM"],
runId: "uniqueRunID",
testId: "testID",
port: 5000
};
var realOpen = XMLHttpRequest.prototype.open;
XMLHttpRequest.prototype.open = function (method, url, async, user, password) {
// Parse the URL, using the document's base URI if using a relative URL
var parsedURL = new URL(url, document.baseURI);
var originalHost = parsedURL.host;
// If any of the targeted hosts contains the request's host
var shouldRedirect = config.hosts.some(function (host) { return originalHost.toUpperCase().includes(host); });
if (shouldRedirect) {
parsedURL.protocol = "https";
parsedURL.host = "localhost:" + config.port;
url = parsedURL.toString();
}
realOpen.call(this, method, url, async, user, password);
if (shouldRedirect) {
// Add required headers to correlate this request to the registered test
this.setRequestHeader("x-mockfx-run-id", config.runId);
this.setRequestHeader("x-mockfx-test-id", config.testId);
this.setRequestHeader("x-mockfx-original-host", originalHost);
this.setRequestHeader("x-mockfx-source", "portal");
}
};
})();Finalizing Tests
Once your test run is finished, finalize it to instruct MockFx to call its targets' storeRequests methods for the test to
store the recorded data and then free up the memory used by the test instance. You can call mockFx.finalizeTest(...) directly or
over HTTPS POST using the finalize end-point from MockFx.getEndpoints(...).finalizeTest with the test's finalization object as its body.
const runId = 'uniqueRunID';
const testId = 'testID';
const testPassed = true; // or false if test failed
await mockFx.finalizeTest({
runId,
testId,
shouldStore: testPassed
});or
const mockFxPort = 5000;
const runId = 'uniqueRunID';
const testId = 'testID';
const testPassed = true; // or false if test failed
const finalizationEndpoint = MockFx.getEndpoints(mockFxPort, runId, testId).finalizeTest;
const finalization: MockFx.Context = {
runId,
testId,
shouldStore: testPassed
};
const body = JSON.stringify(finalization);
const request = https.request({
...url.parse(finalizationEndpoint),
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': body.length
}
});
request.write(body);
request.end();Stopping MockFx
Once all tests have finished, stop the server to close all connections:
await mockFx.stop();Logging
You can subscribe to MockFx logs with MockFx.logger:
Note: logger is a static property on the MockFx class.
MockFx.logger.on("log", (level: MockFx.LogLevel, message: string) => {
switch (level) {
case MockFx.LogLevel.Debug:
case MockFx.LogLevel.Info:
case MockFx.LogLevel.Warn:
console.log(level, message);
}
});
MockFx.logger.on("error", (error: Error) => {
console.error(error);
});
MockFx.logger.on("telemetry", (telemetry: MockFx.Telemetry) => {
console.log(JSON.stringify(telemetry, null, 4));
});Be sure to subscribe to the logging events before performing any other commands with MockFx to catch everything.
Example
import { createARMTarget, MockFx } from "msportalfx-mock";
import { WebDriver } from "selenium-webdriver";
import * as url from 'url';
let mockFx: MockFx;
const mockFxPort = 5000;
const runId = 'uniqueRunID';
const testsToRun = ['testID'];
const armTarget = createARMTarget({
host: "management.azure.com",
loadRequests: async (context: MockFx.Context): Promise<MockFx.Request[]> => {
// Load from storage account
return [];
},
storeRequests: async (context: MockFx.Context, requests: MockFx.Request[]) => {
// Save to storage account
},
});
(async () => {
MockFx.logger.on("log", (level: MockFx.LogLevel, message: string) => {
switch (level) {
case MockFx.LogLevel.Debug:
case MockFx.LogLevel.Info:
case MockFx.LogLevel.Warn:
default:
console.log(level, message);
}
});
MockFx.logger.on("error", (error: Error) => {
console.error(error);
});
MockFx.logger.on("telemetry", (telemetry: MockFx.Telemetry) => {
console.log(JSON.stringify(telemetry, null, 4));
});
mockFx = await MockFx.create({
port: mockFxPort,
targets: [armTarget],
});
await mockFx.start();
await Promise.all(testsToRun.map((testId) => runTest(testId)));
await mockFx.stop();
})();
async function runTest(testId: string) {
await mockFx.registerTest({
mode: MockFx.Mode.Record,
runId,
testId,
targets: [armTarget.name]
});
const webdriver: WebDriver = new WebDriver();
const clientPluginCode = mockFx.getPluginCode(runId, testId);
const clientPluginURL = MockFx.getEndpoints(mockFxPort, runId, testId).plugin;
// Use webdriver to load Portal with "patch" parameter to load the client plug-in code
// into each extension's JS execution context
await webdriver.get('https://portal.azure.com/' +
'?feature.canmodifyextensions=true' +
'&feature.prewarming=false' +
'#home?patch=["' + encodeURI(clientPluginURL) + '"]'
);
// Use webdriver to load client plug-in code into Portal's JS execution context
await webdriver.executeScript(clientPluginCode);
// Run test logic
const testPassed = true; // or false
await mockFx.finalizeTest({
runId,
testId,
shouldStore: testPassed
});
}MockFx API
Note: MockFx has a direct API for registering and finalizing tests, but they're also exposed through the server to enable
registering and finalizing tests from other processes/threads than the one MockFx is running from. Rather than hard-coding the end-points,
use MockFx.getEndpoints(...) to query them to make upgrading easier.
class MockFx {
/**
* The current version of the framework. It's a good idea to use this to bin your stored mock data
* to allow for upgrading the framework when breaking changes are introduced in order to not break
* test runs using previous version.
*/
static version: string;
/**
* Log event emitter. May subscribe to "log", "telemetry" and "error" events.
*/
static logger: LoggerType = logger;
/**
* Creates an instance of the framework
*/
static create(config: MockFx.Configuration): Promise<MockFx>;
/**
* Returns an object with endpoints for the mock server for registering, finalizing,
* and obtaining the plug-in code for a test.
*/
static getEndpoints(port: number, runId: string, testId: string): MockFx.Endpoints;
/**
* Starts the mock server
*/
start(): Promise<void>;
/**
* Stops the mock server
*/
stop(): Promise<void>;
/**
* Returns the mock plug-in client code to run in the browser
*
* Also accessible over HTTPS GET at `MockFx.getEndpoints(mockFxPort, runId, testId).plugin` end-point
*/
getPluginCode(runId: string, testId: string): string;
/**
* Registers a stage within a test. All requests occurring for a test will be tagged with the current stage.
* Used if parts of a test (a stage) are conditional or can be skipped.
*
* Also accessible over HTTPS POST at `MockFx.getEndpoints(mockFxPort, runId, testId).registerStage` end-point
* by sending the stage context object as the body
*/
registerStage(stageContext: MockFx.Stage): Promise<void>;
/**
* Registers a new test, creates instances for each target referenced, and loads their test data if in
* replay mode.
*
* Also accessible over HTTPS POST at `MockFx.getEndpoints(mockFxPort, runId, testId).registerTest` end-point
* by sending the context object as the body
*/
registerTest(context: MockFx.Context): Promise<void>;
/**
* Finalizes a stage within a test. During replay, removes any subsequent requests that were tagged with this stage
*
* Also accessible over HTTPS POST at `MockFx.getEndpoints(mockFxPort, runId, testId).finalizeStage` end-point
* by sending the stage context object as the body
*/
finalizeStage(stageContext: MockFx.Stage): Promise<void>
/**
* Calls `storeRequests` on each target the test has registered, then removes the references to the instance
* to free memory.
*
* Also accessible over HTTPS POST at `MockFx.getEndpoints(mockFxPort, runId, testId).finalizeTest` end-point
* by sending the finalization object as the body
*/
finalizeTest(finalization: MockFx.Finalization): Promise<void>
}MockFx Configuration
interface Configuration {
/**
* Send MockFx logs to the console in addition to the log event emitter.
*
* Setting this to true will affect the MockFx class and send logs to the console for all
* created MockFx instances.
*
* @default false
*/
logToConsole?: boolean;
/**
* ****The**** port for the server to listen on
*/
port: number;
/**
* A list of target services to be mocked. Any target which a test wants to mock must be defined here.
*/
targets: TargetDefinition[];
}MockFx End-points
Returned from MockFx.getEndpoints(...). Use these end-points to register, load client plug-in code for, and finalize your test runs.
interface Endpoints {
/**
* The stage finalization end-point
*/
finalizeStage: string;
/**
* The test finalization end-point
*/
finalizeTest: string;
/**
* The end-point for obtaining the client-side mock plug-in
*/
plugin: string;
/**
* The stage registration end-point
*/
registerStage: string;
/**
* The test registration end-point
*/
registerTest: string;
}Target Definition
A target definition describes how the framework should intercept requests for a service, how to handle its requests and responses, and how to load and store mock data for it.
A few target definitions are built into the framework for targetting ARM, storage data plane, and handling local static mock data.
If your team desires to target other services, it's possible to write your own definition by using the following interface:
interface TargetDefinition {
/**
* A unique name to identity the targeted service
*
* @example ARM
*/
name: string;
/**
* An array of additional hostnames used to correlate requests to a target. If an incoming request's
* host includes any of the strings in this array, it will be redirected to the framework.
*
* May include partial hostnames
*
* @example ["management.azure.com"] or [".blob.core.windows.net", ".file.core.windows.net"]
*/
hosts: string[];
/**
* *Record mode*
*
* Called after an intercepted request has completed to provide any metadata to save as the MockRequest's
* metadata property.
*
* @param {Express.Request} request - the request that was sent to the target
* @param {IncomingMessage} response - the response from the target
* @returns {object} the metadata object to set on the request
*/
getRequestMetadata?: (request: ExpressRequest, response: IncomingMessage) => Promise<Record<string, any>>;
/**
* Used to provide a custom Express router to handle special target routing (e.g. ARM batch calls)
*/
getRouter?: () => Router;
/**
* *Replay mode*
*
* Called during test initialization in replay mode to obtain the array of mocked requests to replay
*
* @param {Context} context - the registered mock context object for the currently running test
* @returns Promise<MockRequest[]>
*/
loadRequests: (context: Context) => Promise<MockFx.Request[]>;
/**
* *Replay mode*
*
* Called on incoming request to match it against the stored requests
*
* @param {Express.Request} request - the incoming request
* @param {MockFx.Request[]} unmatchedRequests - the unmatched stored mock requests
* @returns {MockFx.Request | undefined} the matched request or nothing if no match was found
*/
matchRequest: (
request: ExpressRequest,
unmatchedRequests: MockFx.Request[]
) => Promise<MockFx.Request | undefined>;
/**
* *Replay mode*
*
* Called before an incoming request has been matched to allow for modifying the stored requests during
* test execution.
*
* Useful when needing to normalize requests which contain dynamic data that changes between runs.
*
* @param {Express.Request} request - the incoming request
* @param {MockFx.Context} context - the registered mock context object for the currently running test
* @param {MockFx.Request[]} storedRequests - the stored mock requests
* @returns {MockFx.Request[]} a new array of mock requests which will replace the current stored requests
*/
onBeforeRequestMatch?: (
request: ExpressRequest,
context: MockFx.Context,
storedRequests: MockFx.Request[]
) => Promise<MockFx.Request[]>;
/**
* *Record mode or with `Context.proxyUnmatched`*
*
* Called when processing target response. The value returned will be set as the response property
* on the stored request.
*
* If not implemented, the response body will be converted to a UTF-8 encoded string.
*
* @param {IncomingMessage} response - the target response
* @param {Buffer} responseBody - the body of the response
* @returns {any}
*/
parseResponse?: (response: IncomingMessage, responseBody: Buffer) => Promise<any>;
/**
* *Record mode*
*
* Called once the running test is finalized
*
* @param {MockFx.Context} context - the registered mock context object for the currently running test
* @param {MockFx.Request[]} requests - the recorded requests
*/
storeRequests: (context: MockFx.Context, requests: MockFx.Request[]) => Promise<void>;
}Mock Request
interface Request {
/**
* The URL for the request
*/
url: string;
/**
* The request's HTTP method
*/
method: string;
/**
* Used by targets to store any additonal data about the request from the target definition's
* `getRequestMetadata` method
*/
metadata: Record<string, any>;
/**
* The request body (if present)
*/
request: any;
/**
* The response from the target. Value is returned from the target definition's `parseResponse`
* method
*/
response: any;
/**
* Any headers set on the response.
*/
responseHeaders: Record<string, string>;
/**
* The size of the response in bytes.
*/
responseSize: number;
/**
* The HTTP response code
*/
responseCode: number;
}Test Context
interface Context {
/**
* Whether to record or replay for this test
*/
mode: MockFx.Mode;
/**
* A unique ID to group a set of running tests together
*/
runId: string;
/**
* A unique ID for a single test instance within a run
*/
testId: string;
/**
* A list of target names to mock during this test run (e.g. ["ARM"]).
*
* Target must have been previously defined in the MockFx instance.
*/
targets: string[];
/**
* Provide any additional information about the test that you require.
*/
metadata?: any;
/**
* Set this to true to send unmatched requests to the target while in replay mode, otherwise
* an error response will be returned if a request cannot be matched
*
* Default: false
*/
proxyUnmatched?: boolean;
}Test Finalization
interface Finalization {
/**
* The runId for a registered test
*/
runId: string;
/**
* The testId for a registered test
*/
testId: string;
/**
* Whether or not the storeRequests target definition method should be called for this test run
* (i.e. finalizing a failed test without storing its mock data)
*
* Default: true
*/
shouldStore?: boolean;
}Stage Context
interface Stage {
/**
* The runId for a registered test
*/
runId: string;
/**
* The testId for a registered test
*/
testId: string;
/**
* The name of the stage
*/
stage: string;
}Telemetry
interface Telemetry {
/**
* The test context for this request
*/
context: MockFx.Context;
/**
* The source of the request (ARM client, Portal, etc.)
*/
source: string;
/**
* The name of the matched target
*/
target: string;
/**
* The request's original host
*/
originalHost: string;
/**
* When the request was processed
*/
time: string;
/**
* The URL of the request
*/
url: string;
/**
* HTTP method for the request
*/
method: string;
/**
* Whether the target matched the incoming request
*/
isMatch: boolean;
/**
* The size of the response received from the target
*/
responseSizeInBytes: number;
/**
* Any additional metadata a target wishes to add to the telemetry
*/
metadata?: Record<string, string>;
}License
Copyright (c) Microsoft Corporation
All rights reserved.
MIT License
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.