@runwayml/hosted-models

Hosted Models JavaScript SDK

A small library for interfacing with RunwayML Hosted Models using only a few lines of code. Works in both Node.js and the Browser.

Benefits

This library is a thin wrapper around the Hosted Models HTTP API. It provides a few benefits:

• Abstracts the HTTP requests to the Hosted Model, so you don't have to make them directly.
• Simplifies authorization for private models, just provide your token in the new HostedModels({ url, token }) constructor.
• Automatically retries failed requests if they timed out while the model wakes up or are blocked due to rate-limiting.
• Includes friendly error reporting with messages for humans, so you can better understand why something went wrong.

If your project is written in JavaScript, we encourage you to use this library!

Examples

See the examples/ directory for a full list of examples.

Node.js / Module Syntax

If you are using Node.js or packaging your front-end code via a bundler, you can install the module using npm.

npm install --save @runwayml/hosted-models

const { HostedModel } = require('@runwayml/hosted-models');

const model = new HostedModel({
  url: 'https://my-model.hosted-models.runwayml.cloud/v1',
  token: 'my-private-hosted-model-token',
});

const prompt = 'Hey text generation model, finish my sentence';
model.query({ prompt }).then(result => console.log(result));

Browser

If you prefer to access the library in the Browser using a <script> tag, you can include the code snippet below in your HTML files. Replace hosted-models.js with hosted-models.min.js if you prefer a minified build for production environments.

<script src="https://cdn.jsdelivr.net/npm/@runwayml/hosted-models@latest/dist/hosted-models.js"></script>

This injects the library into the window and exposes it via the rw namespace.

const model = new rw.HostedModel({
  url: 'https://my-model.hosted-models.runwayml.cloud/v1',
  token: 'my-private-hosted-model-token',
});

const prompt = 'Hey text generation model, finish my sentence';
model.query({ prompt }).then(result => console.log(result));

Usage

This library is super simple to use; It exposes a single HostedModels class with only four methods:

• HostedModel
  • .info()
  • .query()
  • .isAwake()
  • .waitUntilAwake()

Note: Be sure to use rw.HostedModel() if you are including the library via a <script> in the Browser.

HostedModels Constructor

The HostedModel constructor takes a configuration object with two properties, url and token (required only if the model is private).

const model = new HostedModel({
  url: 'https://example-text-generator.hosted-models.runwayml.cloud/v1',
  token: 'my-private-hosted-model-token', // not required for public models
});

.info() Method

This method returns the input/output spec expected by this model's query() method.

const info = await model.info();
console.log(info);
//// Note: These values will be different for each model
// {
//   "description": "Generate text conditioned on prompt",
//   "name": "generate_batch",
//   "inputs": [
//     {
//       "default": "",
//       "description": null,
//       "minLength": 0,
//       "name": "prompt",
//       "type": "text"
//     },
//     ...
//   ],
//   "outputs": [
//     {
//       "default": "",
//       "description": null,
//       "minLength": 0,
//       "name": "generated_text",
//       "type": "text"
//     },
//     ...
//   ]
// }

.query() Method

query() is used to trigger the model to process input, and return output.

const result = await model.query({
  prompt: 'Hey text generation model, finish my sentence',
});
console.log(result);
//// Note: These values will be different for each model
// {
//   generated_text: 'Hey text generation model, finish my sentence please.',
//   encountered_end: true
// }

.isAwake() Method

The isAwake() method returns true if the model is already awake and processing requests quickly, or false if it is still waking up. A model that is waking up can still process all requests (e.g. info() and query()), they just might take a bit longer to respond. Learn more about Hosted Model states here.

if (!model.isAwake()) {
  showLoadingScreen(); // this is a fake function for demonstration
}

// A model doesn't have to be awake for you to make requests to it
await model.query(input);

Note: A model that is waking up can still process all requests (e.g. info() and query()), they just might take a bit longer to respond. Learn more about Hosted Model states here.

.waitUntilAwake() Method

The .waitUntilAwake() method returns a promise that will resolve as soon as the model is awake. This is useful if you'd prefer to wait to perform some action until after the model can be expected to process them quickly.

setLoading(true); // this is a fake function for demonstration
await model.waitUntilAwake();
setLoading(false);

setProcessing(true); // this is also a fake function for demonstration
await model.query(input);
setProcessing(false);

Note: A model that is waking up can still process all requests (e.g. info() and query()), they just might take a bit longer to respond. Learn more about Hosted Model states here.

License

This library is released under the terms of the MIT license.

CHANGELOG

You can view a list of changes here.