Upload Transformation SDK (Example)

An example NPM package that can be referenced by transformation steps in Upload.

Getting started

See src/Example.ts for a working example of a custom transformation step.

The example performs the following steps:

1. Reads the original file.

2. Compresses the file.

3. Writes the file back.

Publishing to NPM

Upload requires 'fat' packages.

This means you must bundle all the node_modules your code depends on into the package you publish to NPM.

Why?

Because Upload does not npm install packages referenced by transformation steps.

Instead, Upload will download your package's tarball from the NPM registry and will directly include it into the transformations it's used in, invoking it using its default export. Simply put, Upload will download your tarball, extract it, and run it using: require("./your-package")(transformation, params).

The example project is configured to publish fat packages by default.

Try it using npm pack.

Reading and writing data

The Storage class

The Storage class is typically instantiated once, at the start of your transformation:

import { storageForRequest, Storage, Transformation } from "upload-transformation-sdk";

export default async function transform(transformation: Transformation, params: any): Promise<void> {
  const storage: Storage = storageForRequest(transformation.requestId);
  ...

The Storage class contains the following methods:

• read(path: string): Promise<Readable>
• write(path: string, data: Readable): Promise<void>

Reading and writing to "/"

You can access the original file using the path "/".

You can also overwrite this file, causing a new file to be returned from the root transformation URL:

https://cdn.upload.io/<fileId>/<transformationName>/

If you don't overwrite the "/" file, then the root transformation URL will simply host the original file. That is, the following two URLs will serve identical content (i.e. the original file):

https://cdn.upload.io/<fileId>/<transformationName>/

https://cdn.upload.io/<fileId>/

(Note that the last URL will always return the original file, and cannot be overwritten.)

Saving files to other paths

The paths you write to will be accessible from the following URL:

https://cdn.upload.io/<fileId>/<transformationName><path>

This allows you to output multiple files for a single original file.

For example, if your code performs a storage.write("/manifest.json", data), and you publish your code to NPM and subsequently use it in a transformation you name video, and you want to use your new transformation on the file xyz, you would request the following URL:

https://cdn.upload.io/xyz/video/manifest.json

Logging

The example in src/Example.ts demonstrates how to add logging to transformation code:

1. Create an array of logs: logs = []
2. Push log messages to this array: logs.push("Doing something...")
3. At the end of your code write the logs array to Upload using write("/logs", logs.join("\n")).

(Note: the write should be placed in a finally to ensure logs are written on error.)

Your log will be accessible from:

https://cdn.upload.io/<fileId>/<transformationName>/log