Contractor is a purpose-built, open-source library created in TypeScript. Designed with a primary focus on streamlining the usage of OpenAI's function calling and enabling incremental, type-safe streaming of JSON outputs, Contractor seamlessly integrates AI functionality into a wide array of applications.
- incremental & type-safe streaming of JSON output from functions (array elements streaming)
- built-in auditing facility to log calls to OpenAI and keep track of token usage (streaming & sync)
- Type-Safe IDE hinting for OpenAI functions and robust handling of data received from OpenAI
- pluggable logging - wrap your logging library to pass on internal logs to your logging subsystem
- graceful error handling and recovery (JSON issues, networking issues)
Before getting started with Contractor, you should ensure that you meet the following requirements:
- Language Compatibility: Contractor is written in TypeScript, offering the advantage of compile-time type safety with well-annotated code. However, it is also compatible with any JavaScript Node applications.
- Runtime Environment: The library has been developed and tested to work in Node.js. Compatibility with other runtimes has not been established.
- OpenAI API Key: To utilize this library, you'll need to have your own OpenAI API key.
yarn add contractor
#or if you are using npm
npm install contractorCreating a Contractor instance
const apiKey = process.env.OPENAI_API_KEY;
const logger = prefixedLogger('typedStreamingObjectWithAuditor'); // optional..
if (!apiKey) {
throw new Error('OPENAI_API_KEY env var not provided');
}
const configuration = new Configuration({
apiKey: apiKey, // your openai key..
});
const openaiClient = new OpenAIApi(configuration);
const client = new OpenAIClient(openaiClient);
const auditor: IAuditor<{ userId: string }> = {
/**
* @param record type AuditRecord<MetaData> = {
* resultRaw: CreateChatCompletionResponse | undefined;
* result: { data: { content: any } } | { error: { message: string, details: any, receivedMessage?: string } };
* request: CreateChatCompletionRequest;
* requestType: string;
* requestSig: string;
* metaData?: MetaData;
* }
*/
auditRequest: record => {
// write records of audit entries to db, use MetaData attribute to pass customerIds
// or other identifiers to associate record with business use
logger.info(`auditor record [${JSON.stringify(record)}]`)
}
};
const contractor = new Contractor(
client, // instance of OpenAIClient
auditor, // OPTIONAL - count token usage and log responses
800, // OPTIONAL (defaults to 8000)- max tokens per single request (request + response)
'<|+|>', // OPTIONAL (defaults to |{-*-}|) - seperator to use when using streaming
logger, // OPTIONAL - intercept interaly generated log entries
);Using Contractor
// stream response
function streamingFunction(systemMessage: string,
messages: RequestMessageFormat[],
model: GPTModelsAlias, // gpt3/gpt4
functions: [ChatCompletionFunctionsWithTypes<T1, N1>], // array of functions to expose to GPT
transformObjectStream: (streamingObject: Result<T1, N1>) => Promise<OUT>, // handle resopnses from AI, make DB calls and pass on result downstream
responseSize?: number, // limit response size
logMetaData?: MetaData,
requestOverrides?: Partial<CreateChatCompletionRequest>, // override OpenAI arguments like top_p etc..
maxTokens?: number // limit total token usage for single call
): Promise<NodeJS.ReadableStream | undefined>;
// perform single function sync call (result returned via Promise)
function singleFunction<T>(systemMessage: string,
messages: RequestMessageFormat[],
model: GPTModelsAlias = 'gpt3', // gpt3/gpt4
gptFunction: {
name: string, // name of function
description: string; // helpful description
parameters: JSONSchemaType<T>; // JSPN schema to pass and validate against
},
logMetaData?: MetaData,
requestOverrides?: Partial<CreateChatCompletionRequest>,
responseSize: number = 2000,
maxTokens: number = this.maxTokensPerRequest,
): Promise<T | undefined>;Simple agent (see full example)
const MathMultiplyOperation: JSONSchemaType<{ firstNumber: number, secondNumber: number }> = {
type: "object",
properties: {
firstNumber: {type: "number", description: "first number"},
secondNumber: {type: "number", description: "second number"},
},
required: ["firstNumber", "secondNumber"],
};
const MathAddOperation: JSONSchemaType<{ firstNumber: number, secondNumber: number }> = {
type: "object",
properties: {
firstNumber: {type: "number", description: "first number"},
secondNumber: {type: "number", description: "second number"},
},
required: ["firstNumber", "secondNumber"],
};
const FinalResultOperation: JSONSchemaType<{ finalResult: number }> = {
type: "object",
properties: {
finalResult: {type: "number", description: "first number"},
},
required: ["finalResult"],
};
contractor.streamingFunction('you are a calculator agent',
[{
role: 'user', content: `your goal is: ${JSON.stringify(currentGoal)}
operations performed so far: ${operationsPerformed.map(_ => JSON.stringify(_)).join('\n')}`
}],
'gpt3',
[
{
name: 'math_add_operation',
description: 'add two numbers',
parameters: MathAddOperation
},
{
name: 'math_multiply_operation',
description: 'multiply two numbers',
parameters: MathMultiplyOperation
},
{
name: 'final_result_operation',
description: 'return final math result',
parameters: FinalResultOperation
},
],
async streamingObject => {
if (streamingObject.name === 'math_add_operation') {
const opStack = [...operationsPerformed,
{calculated: `${streamingObject.value.firstNumber}+${streamingObject.value.secondNumber}=${streamingObject.value.firstNumber + streamingObject.value.secondNumber}`}];
performSingleOperation(currentGoal, opStack)
.then(resolve, reject);
return {
"performed": streamingObject.value,
"stack": opStack
}
} else if (streamingObject.name === 'math_multiply_operation') {
const opStack = [...operationsPerformed,
{calculated: `${streamingObject.value.firstNumber}*${streamingObject.value.secondNumber}=${streamingObject.value.firstNumber * streamingObject.value.secondNumber}`}];
performSingleOperation(currentGoal, opStack)
.then(resolve, reject);
return {
"performed": streamingObject.value,
"stack": opStack
}
} else if (streamingObject.name === 'final_result_operation')
resolve(`final result: ${JSON.stringify(streamingObject.value.finalResult)}`)
return {
"performed": streamingObject.value,
"stack": operationsPerformed,
"finalResult": streamingObject.value.finalResult
}
})Incremental partial streaming (see full example)
const StoryOutput: JSONSchemaType<{ title: string, lines: string[] }> = {
type: "object",
properties: {
title: {type: "string"},
lines: {type: "array", description: "story line", items: {"type": 'string'}},
},
required: ['title', 'lines'],
};
contractor.streamingFunction('you are a story teller',
[{
role: 'user',
content: 'tell me a short story (50 lines) about an AI agent that went rogue, use available functions to answer'
}],
'gpt3',
[
{
name: 'print_output',
description: 'print content to user',
parameters: StoryOutput,
partialStreamPath: ['lines'],
},
],
async streamingObject => {
return `story so far: ${streamingObject.value.title}\n${streamingObject.value.lines.join('\n')}`
})Using with Express server
// your express app..
const expressApp = express();
const oaiConf = new Configuration({
apiKey: process.env.OPENAI_API_KEY,
});
const openaiClient = new OpenAIApi(oaiConf);
const client = new OpenAIClient(openaiClient);
const contractor = new Contractor(client);
const AnswerSchema: JSONSchemaType<{ richMarkdownAnswer: string }> = {
type: "object",
properties: {
richMarkdownAnswer: {type: "string", description: "richly markdown formatted answer"},
},
required: ["richMarkdownAnswer"],
};
expressApp.post('/ask-question-sync', async (req: Request, res: Response) => {
const {question} = req.body;
const value = await contractor.singleFunction("you are a calculator agent",
[{role: 'user', content: 'what is the sum of 2+2?'}],
'gpt3',
{
name: 'answer',
description: 'return answer',
parameters: AnswerSchema
}
);
res.json({result: value?.richMarkdownAnswer ?? 'no answer :('})
})
expressApp.post('/ask-question-async', async (req: Request, res: Response) => {
const {question} = req.body;
contractor.streamingFunction(
'you are a story teller', // system prompt
[{
role: 'user',
content: 'tell me a short story (50 lines) about an AI agent that went rogue, use available functions to answer'
}], // messages
'gpt3', // or gpt4
[
{
name: 'print_output',
description: 'print content to user',
parameters: StoryOutput,
// this is the important bit, the path inside `{ title: string, lines: string[] }`
// that should be streamed as the array parts come..
partialStreamPath: ['lines'],
},
], // array of functions to pass to ai, in the
async streamingObject => {
return `story so far: ${streamingObject.value.title}\n${streamingObject.value.lines.join('\n')}`
}
).then(stream => {
stream && stream.pipe(res);
});
})For more examples see the example directory.
Some additional plug-and-play stuff you will need to get stuff done, like tests or your own use cases
Place an instance of this class in a stream to intercept chunks
import {pipeline} from "stream";
const stream = ...;
const listener = new StreamListenerTransform((x) => console.log("look what I got!", x));
pipeline(stream,
listener,
downstream,...)Place an instance of this class to manipulate chunks
import {pipeline} from "stream";
const stream = ...;
// transform here will transform incoming string chunks by trimming them
const tranform = new SimpleStreamTransform((input: string) => input.trim());
pipeline(stream,
tranform,
downstream,...)Place an instance of this class to intercept string stream of stringified json, call a transform function (passed via constructor) and stringify back the result "down pipe".
import {pipeline} from "stream";
const stream = ...;
// transform here will transform incoming string chunks by trimming them
const tranform = new StreamMITMTransform((input: {text: string}) => input.text.trim());
pipeline(stream,
tranform,
downstream,...)import {gptUtils} from "contractor";
// truncate some string to certain size of tokens
const input = "Alfalfa sprouts bananas chili roasted brussel sprouts fig arugula cashew salad dill main course chili pepper cashew creamiest edamame chocolate.";
const model = "gpt3" // or "gpt4"
const maxTokenSize = 10;
const truncatedString = gptUtils.truncateInput(input, model, maxTokenSize);
console.log(`we truncated input [${input}] to size [${maxTokenSize}] and `)
const tokenCount = gptUtils.countTokens(input, model);
console.log(`for string [${input}] we counted [${tokenCount}] tokens!`);yarn testcd contractor
yarn link
cd <your project>
yarn link contractorWe appreciate any contribution to Contractor, and thank you for your interest in improving this open-source project! Here's how you can contribute:
-
Fork the Repository: Start by forking the Contractor repository to your own GitHub account.
-
Clone the Repository: Clone the forked repository to your local machine and create a new branch for your feature or fix.
git clone https://github.com/<your-username>/contractor
git checkout -b name-of-your-branch-
Make Changes: Implement your new feature or bug fix, making sure to add or update any relevant tests.
-
Run the Tests: Ensure that all tests pass with your changes.
yarn test- Push to GitHub: Push your changes and your new branch to your forked repository.
git push origin name-of-your-branch- Create a Pull Request: Navigate to your forked repository on GitHub and create a new pull request from your branch to the main branch of the Contractor repository.
In your pull request, please provide a clear description of the changes you've made. The more information you can provide, the easier it will be for us to review and accept your contribution.
Before submitting a pull request, please ensure that your code follows the existing style in the codebase.
Thank you for considering a contribution to Contractor. We're looking forward to your pull request!