@stencila/sparkla

✨ Sparkla

Compute sessions for executable documents: fast-to-start, isolated, reproducible. Pick any three.

⚠️ Well, that's the aim ;) Sparkla is currently in early development. It started as an experiment in using Amazon's Firecracker as a more secure (but just as fast-to-start) alternative to Docker containers. It is intended to supersede stencila/cloud.

Install

npm install -g @stencila/sparkla

Sparkla creates sessions in either Firecracker microVMs or Docker containers. So you'll need at least one of these installed.

Firecracker

Firecracker is only available on Linux. Download the latest Firecracker binary from https://github.com/firecracker-microvm/firecracker/releases:

curl -L -o firecracker https://github.com/firecracker-microvm/firecracker/releases/download/v0.18.0/firecracker-v0.18.0
chmod +x firecracker

Firecracker is built on top of KVM and needs read/write access to /dev/kvm. Log in to the host in one terminal and set up that access:

sudo setfacl -m u:${USER}:rw /dev/kvm

Docker

If you want to use Docker-based sessions then you'll need to have docker installed.

Usage

Run Sparkla using the command line interface e.g.

sparkla serve --port 9001 --cpuTotal 1

Options

Name	Description	Type	Default
debug	Output debug level log entries?	boolean	false
host	The host address that the server should listen on.	string	"127.0.0.1"
port	The port that the server should listen on.	number	9000
jwtSecret	The JWT secret to use to sign and verify JWT tokens. If null then a random secret will be generated.	string, null	null
sessionType	The class of sessions created.	"firecracker", "docker"	"docker"
cpuTotal	The total number of CPUs that can be allocated to sessions. null = use the number of CPUs on the machine.	number, null	null
memoryTotal	The total number amount of memory (Gib) that can be allocated to sessions. null = use the total amount of memory on the machine.	number, null	null
expiryInterval	Interval in seconds between checks for expired sessions.	number	15
durationWarning	Number of seconds to provide clients with a warning prior to reaching maximum session duration.	number	600
timeoutWarning	Number of seconds to provide clients with a warning prior to a reaching session timeout.	number	60
staleInterval	Interval in seconds between checks for stale sessions.	number	60
stalePeriod	Number of seconds that a stopped session is considered stale and will be removed from the list of sessions.	number	3600
statsInterval	Interval in seconds for collecting system statistics	number	60
statsPrometheus	The port to serve Prometheus compatible metrics on. Set to 0 to turn off Prometheus exporting.	number	9464
logsSentry	The Sentry DSN (Data Source Name) to record warning and error log entries. Set to null to not send logs to Sentry.	string	"https://[e-mail]/1818023"
peerSwarm	The name of the peer swarm to join.	string, null	"sparkla"

All options can be set, in decending order of priority, by:

• a command line argument e.g. --port 80
• an environment variable prefixed with SPARKLA_ e.g. SPARKLA_PORT=80
• a .json or .ini configuration file, set using the --config option, or .sparklarc by default

Development

npm run serve:dev

Firecracker

To check for running FirecrackerSessions:

ps aux | firecracker

You can also check the logs and metrics of a Firecracker VM while it is running:

cat /tmp/vm-0d0b4b2fa73c5d4c10ed72ccac97b798530ba1b7ab8461fd313d320fb5d3562e/log.fifo
cat /tmp/vm-0d0b4b2fa73c5d4c10ed72ccac97b798530ba1b7ab8461fd313d320fb5d3562e/metrics.fifo

Docker

To check for running DockerSessions:

docker ps