botmation

Botmation is a simple TypeScript framework to build web bots with Puppeteer in a declarative, functional and composable way.

“Everything should be made as simple as possible, but no simpler.” - Albert Einstein

Why choose Botmation?

It empowers Puppeteer code with a simple pattern to maximize code readability, reusability and testability.

Its compositional design comes pre-built with safe defaults for building bots with less code.

It encourages a learn at your own pace approach to exploring the possibilities of Functional programming.

It has 100% source code test coverage.

Introduction

Botmation is simple declarative framework for Puppeteer, to build web bots in a composable way. It provides a simple pattern focused on a single type of function called BotAction.

BotAction's handle everything from simple tasks in crawling and scraping the web to logging in & automating your social media. They are composable. They make assembling Bots easy, declarative, and simple.

The possibilities are endless!

Getting Started

Botmation is a NodeJS library written in TypeScript. You'll need node.js LTS installed and the TypeScript compiler (tsc) installed globally (or have a transpiling code step).

Install

Install Botmation with npm and save it as a dependency:

npm install --save botmation

If you're just getting started, install puppeteer & @types/puppeteer:

npm install --save puppeteer 
npm install --save-dev @types/puppeteer

Documentation

To get started with Botmation, learn about its design and pattern, view API Doc's, see examples, advanced techniques, and a tutorial on approaching these Bot problems, visit the official Botmation Documentation site.

Library Reference

After intalling through npm, import any BotAction from the main module:

import { chain, goTo, screenshot } from 'botmation'

As of v3.x, there are 14 groups of BotAction to compose from:

• abort
  • abort an assembly of BotAction's
• assembly-line
  • compose and run BotAction's in lines
• console
  • log messages to the nodeJS console
• cookies
  • read/write page cookies
• errors
  • try/catch errors in assembly-lines
• files
  • write files to local disk ie screenshots, pdf's
• indexed-db
  • read/write to page's IndexedDB
• inject
  • insert new injects into a line of BotAction's
• input
  • simulate User input ie typing and clicking with a mouse
• local-storage
  • read/write/delete from a page's Local Storage
• navigation
  • change the page's URL, wait for form submissions to change page URL, back, forward, refresh
• pipe
  • functions specific to Piping
• scrapers
  • scrape HTML documents with an HTML parser and evaluate JavaScript inside a Page
• utilities
  • handle more complex logic patterns ie if statements and for loops

Examples

In the ./src/examples directory of this repo (excluded from the npm module), exists a small collection of simple bots, to help you get going:

• Simple Object-Oriented
• Simple Functional
• Generate Screenshots
• Save a PDF
• Instagram Login
• LinkedIn Like Feed Posts

Dev Notes

Library Development

First, clone the repo locally, then install the npm dependencies. You can build the library locally with this command:

npm run build

The playground_bot is a dedicated spot for trying out new Bot Actions, etc. You can run it's code, after running the build command, with:

npm run playground

Library Testing

All our testing (e2e, unit, and integration) is done with Jest.

Learn more about the library's testing strategy and coverage with the Botmation: Tests documentation.

Issues & Feature Requests

Open Issues on Github. Please specify if it's a feature request or a bug.

When reporting bugs, please provide sample code to recreate the bug, relevant error messages/logs, and any other information that may help.

Contributors

Code

Michael Lage - Blog

Art

Patrick Capeto - Email