genkitx-groq
TypeScript icon, indicating that this package has built-in type declarations

0.10.0 • Public • Published

Firebase Genkit + Groq

Firebase Genkit <> Groq Plugin

Groq Community Plugin for Google Firebase Genkit

Github lerna version NPM Downloads GitHub Org's stars GitHub License Static Badge
GitHub Issues or Pull Requests GitHub Issues or Pull Requests GitHub commit activity

genkitx-groq is a community plugin for using OpenAI APIs with Firebase Genkit. Built by The Fire Company. 🔥

Genit + Groq example

Installation

Install the plugin in your project with your favorite package manager:

  • npm install genkitx-groq
  • yarn add genkitx-groq
  • pnpm add genkitx-groq

Usage

Initialize

import 'dotenv/config';

import { configureGenkit } from '@genkit-ai/core';
import { defineFlow, startFlowsServer } from '@genkit-ai/flow';
import { groq } from 'genkitx-groq';

configureGenkit({
  plugins: [
    // Groq API key is required and defaults to the GROQ_API_KEY environment variable
    groq({ apiKey: process.env.GROQ_API_KEY }),
  ],
  logLevel: 'debug',
  enableTracingAndMetrics: true,
});

Basic examples

The simplest way to call the text generation model is by using the helper function generate:

// ...configure Genkit (as shown above)...

const response = await generate({
  model: llama3x70b, // model imported from genkitx-groq
  prompt: 'Tell me a joke.',
});

console.log(await response.text());

Within a flow

// ...configure Genkit (as shown above)...

export const myFlow = defineFlow(
  {
    name: 'menuSuggestionFlow',
    inputSchema: z.string(),
    outputSchema: z.string(),
  },
  async (subject) => {
    const llmResponse = await generate({
      prompt: `Suggest an item for the menu of a ${subject} themed restaurant`,
      model: openMixtral8x22B,
    });

    return llmResponse.text();
  }
);
startFlowsServer();

Tool use

// ...configure Genkit (as shown above)...

const createReminder = defineTool(
  {
    name: 'createReminder',
    description: 'Use this to create reminders for things in the future',
    inputSchema: z.object({
      time: z
        .string()
        .describe('ISO timestamp string, e.g. 2024-04-03T12:23:00Z'),
      reminder: z.string().describe('the content of the reminder'),
    }),
    outputSchema: z.number().describe('the ID of the created reminder'),
  },
  (reminder) => Promise.resolve(3)
);

const result = generate({
  model: llama3x70b,
  tools: [createReminder],
  prompt: `
  You are a reminder assistant.
  If you create a reminder, describe in text the reminder you created as a response.

  Query: I have a meeting with Anna at 3 for dinner - can you set a reminder for the time?
  `,
});

console.log(result.then((res) => res.text()));

Contributing

Want to contribute to the project? That's awesome! Head over to our Contribution Guidelines.

Need support?

[!NOTE]
This repository depends on Google's Firebase Genkit. For issues and questions related to Genkit, please refer to instructions available in Genkit's repository.

Reach out by opening a discussion on Github Discussions.

Credits

This plugin is proudly maintained by the team at The Fire Company. 🔥

License

This project is licensed under the Apache 2.0 License.

Package Sidebar

Install

npm i genkitx-groq

Weekly Downloads

11

Version

0.10.0

License

Apache-2.0

Unpacked Size

149 kB

Total Files

17

Last publish

Collaborators

  • epmatt
  • davidoort
  • thefirecompany