This is a TypeScript library that implements AdGuard's content blocking rules.
- TSUrlFilter
The idea is to have a single library that we can reuse for the following tasks:
- Doing content blocking in our Chrome and Firefox extensions (obviously)
- Using this library for parsing rules and converting to Safari-compatible content blocking lists (see AdGuard for Safari, AdGuard for iOS)
- Using this library for validating and linting filter lists (see FiltersRegistry, AdguardFilters)
- It could also be used as a basis for the VS code extension
Install the tsurlfilter:
npm install @adguard/tsurlfilter
type: string
Version of the library.
Engine is a main class of this library. It represents the filtering functionality for loaded rules
/**
* Creates an instance of Engine
* Parses filtering rules and creates a filtering engine of them
*
* @param ruleStorage storage
* @param configuration optional configuration
*
* @throws
*/
constructor(ruleStorage: RuleStorage, configuration?: IConfiguration | undefined)
/**
* Matches the specified request against the filtering engine and returns the matching result.
* In case frameRules parameter is not specified, frame rules will be selected matching request.sourceUrl.
*
* @param request - request to check
* @param frameRules - source rules or undefined
* @return matching result
*/
matchRequest(request: Request, frameRule: NetworkRule | null = null): MatchingResult
/**
* Matches current frame and returns document-level allowlist rule if found.
*
* @param frameUrl
*/
matchFrame(frameUrl: string): NetworkRule | null
const list = new StringRuleList(listId, rulesText, false, false);
const ruleStorage = new RuleStorage([list]);
const config = {
engine: 'extension',
version: '1.0.0',
verbose: true,
};
setConfiguration(config)
const engine = new Engine(ruleStorage);
const request = new Request(url, sourceUrl, RequestType.Document);
const result = engine.matchRequest(request);
const cosmeticResult = engine.getCosmeticResult(request, CosmeticOption.CosmeticOptionAll);
MatchingResult contains all the rules matching a web request, and provides methods that define how a web request should be processed
/**
* GetBasicResult returns a rule that should be applied to the web request.
* Possible outcomes are:
* returns null -- bypass the request.
* returns a allowlist rule -- bypass the request.
* returns a blocking rule -- block the request.
*
* @return basic result rule
*/
getBasicResult(): NetworkRule | null
This flag should be used for getCosmeticResult(request: Request, option: CosmeticOption)
/**
* Returns a bit-flag with the list of cosmetic options
*
* @return {CosmeticOption} mask
*/
getCosmeticOption(): CosmeticOption
/**
* Return an array of replace rules
*/
getReplaceRules(): NetworkRule[]
/**
* Returns an array of csp rules
*/
getCspRules(): NetworkRule[]
/**
* Returns an array of cookie rules
*/
getCookieRules(): NetworkRule[]
Cosmetic result is the representation of matching cosmetic rules. It contains the following properties:
/**
* Storage of element hiding rules
*/
public elementHiding: CosmeticStylesResult;
/**
* Storage of CSS rules
*/
public CSS: CosmeticStylesResult;
/**
* Storage of JS rules
*/
public JS: CosmeticScriptsResult;
/**
* Storage of Html filtering rules
*/
public Html: CosmeticHtmlResult;
/**
* Script rules
*/
public getScriptRules(): CosmeticRule[];
const css = [...cosmeticResult.elementHiding.generic, ...cosmeticResult.elementHiding.specific]
.map((rule) => `${rule.getContent()} { display: none!important; }`);
const styleText = css.join('\n');
const injectDetails = {
code: styleText,
runAt: 'document_start',
};
chrome.tabs.insertCSS(tabId, injectDetails);
const cosmeticRules = cosmeticResult.getScriptRules();
const scriptsCode = cosmeticRules.map((x) => x.getScript()).join('\r\n');
const toExecute = buildScriptText(scriptsCode);
chrome.tabs.executeScript(tabId, {
code: toExecute,
});
DNSEngine combines host rules and network rules and is supposed to quickly find matching rules for hostnames.
/**
* Builds an instance of dns engine
*
* @param storage
*/
constructor(storage: RuleStorage)
/**
* Match searches over all filtering and host rules loaded to the engine
*
* @param hostname to check
* @return dns result object
*/
public match(hostname: string): DnsResult
const dnsResult = dnsEngine.match(hostname);
if (dnsResult.basicRule && !dnsResult.basicRule.isAllowlist()) {
// blocking rule found
..
}
if (dnsResult.hostRules.length > 0) {
// hosts rules found
..
}
Before saving downloaded text with rules it could be useful to run converter on it. The text will be processed line by line, converting each line from known external format to Adguard syntax.
/**
* Converts rules text
*
* @param rulesText
*/
public static convertRules(rulesText: string): string {
This module is not used in the engine directly, but it can be used to validate filter rules in other libraries or tools
/**
* Validates raw rule string
* @param rawRule
*/
public static validate(rawRule: string): ValidationResult
/**
* Valid true - means that the rule is valid, otherwise rule is not valid
* If rule is not valid, reason is returned in the error field
*/
interface ValidationResult {
valid: boolean;
error: string | null;
}
This module is not used in the engine directly, but it can be used in other libraries
/**
* Checks if rule can be matched by domain
* @param ruleText
* @param domain
*/
public static isRuleForDomain(ruleText: string, domain: string): boolean {
/**
* Checks if rule can be matched by url
* @param ruleText
* @param url
*/
public static isRuleForUrl(ruleText: string, url: string): boolean {
Provides a functionality of conversion AG rules to manifest v3 declarative syntax. See examples/manifest-v3/ for an example usage.
/**
* Converts a set of rules to declarative rules array
*
* @param ruleList
*/
public convert(ruleList: IRuleList): DeclarativeRule[] {
- Regexp is not supported in remove params
- We cannot implement inversion in remove params
- We cannot filter by request methods
- Only one rule applies for a redirect. For this reason, different rules with the same url may not work. Example below:
Works ||testcases.adguard.com$removeparam=p1case6|p2case6
Failed ||testcases.adguard.com$removeparam=p1case6
Works ||testcases.adguard.com$removeparam=p2case6
This project is part of the @adguard/extensions monorepo.
It is highly recommended to use lerna for commands as it will execute scripts in the correct order and can cache dependencies.
npx lerna run --scope=@adguard/tsurlfilter:<script>-
start: Runbuildin watch mode -
test:watch: Run test suite in interactive watch mode -
test:prod: Run linting and generate coverage -
build: Generate bundles and typings, create docs -
lint: Lints code
On library development, one might want to set some peer dependencies, and thus remove those from the final bundle. You can see in Rollup docs how to do that.
Good news: the setup is here for you, you must only include the dependency name in external property within rollup.config.js. For example, if you want to exclude lodash, just write there external: ['lodash'].
There is already set a precommit hook for formatting your code with Eslint 💅