Convai Web SDK

A TypeScript/JavaScript SDK for integrating Convai's conversational AI into web applications, optimized for React developers.

Features

• React-first design - Optimized for React applications with hooks
• Vanilla JavaScript/TypeScript support - Core functionality without React dependencies
• Tree shaking support - Only import what you need
• Multiple build formats - ESM and UMD
• TypeScript support - Full type definitions included

Installation

npm install convai-web-sdk

Usage

React Applications (Recommended)

For React applications, import directly from the main package:

import {
  useConvaiClient,
  ConvaiClient,
  GetResponseResponse,
  ConvaiClientState,
  ConvaiClientActions,
  ConvaiClientRefs,
  UseConvaiClientReturn
} from 'convai-web-sdk';

// React Hook Example
function ChatComponent() {
  const { state, actions, refs, characterId } = useConvaiClient(
    'your-character-id',
    'your-api-key'
  );

  const handleSendMessage = async () => {
    await actions.sendMessage('Hello, how are you?');
  };

  return (
    <div>
      <div>Character: {state.npcName}</div>
      <div>User Text: {state.userText}</div>
      <div>NPC Text: {state.npcText}</div>
      <div>Is Talking: {state.isTalking ? 'Yes' : 'No'}</div>
      <button onClick={handleSendMessage}>Send Message</button>
      <button onClick={actions.startListening}>Start Listening</button>
      <button onClick={actions.stopListening}>Stop Listening</button>
    </div>
  );
}

// Direct ConvaiClient Example (React)
function DirectClientExample() {
  const [client, setClient] = useState<ConvaiClient | null>(null);
  const [response, setResponse] = useState<string>('');

  useEffect(() => {
    const convaiClient = new ConvaiClient({
      apiKey: 'your-api-key',
      characterId: 'your-character-id',
      enableAudio: true,
      sessionId: 'session-123',
      narrativeTemplateKeysMap: new Map(),
    });

    convaiClient.setResponseCallback((response: GetResponseResponse) => {
      if (response.hasAudioResponse()) {
        const audioResponse = response.getAudioResponse();
        if (audioResponse?.hasTextData()) {
          setResponse(audioResponse.getTextData() || '');
        }
      }
    });

    setClient(convaiClient);
  }, []);

  const sendText = () => {
    client?.sendTextChunk('Hello, how are you?');
  };

  return (
    <div>
      <div>Response: {response}</div>
      <button onClick={sendText}>Send Text</button>
    </div>
  );
}

Vanilla JavaScript/TypeScript

For applications that don't use React, import from the vanilla subpath:

import {
  ConvaiClient,
  GetResponseResponse,
  ConvaiGRPCClient,
  AudioRecorder,
  AudioPlayer,
  ActionConfigParamsType,
  ConvaiGRPCClientConfigType,
} from "@airsurfer09/web-sdk/vanilla";

// Vanilla TypeScript Example
class ChatManager {
  private client: ConvaiClient | null = null;

  constructor(apiKey: string, characterId: string) {
    this.client = new ConvaiClient({
      apiKey,
      characterId,
      enableAudio: true,
      sessionId: `session-${Date.now()}`,
      narrativeTemplateKeysMap: new Map(),
      languageCode: "en-US",
      enableFacialData: false,
      faceModel: 3,
    });

    this.setupCallbacks();
  }

  private setupCallbacks() {
    if (!this.client) return;

    this.client.setResponseCallback((response: GetResponseResponse) => {
      // Handle live transcript
      if (response.hasUserQuery()) {
        const transcript = response.getUserQuery();
        const isFinal = response.getIsFinal();
        console.log("Transcript:", transcript, "Final:", isFinal);
      }

      // Handle audio response
      if (response.hasAudioResponse()) {
        const audioResponse = response.getAudioResponse();
        if (audioResponse?.hasTextData()) {
          console.log("Response:", audioResponse.getTextData());
        }
        if (audioResponse?.hasAudioData()) {
          const audioBytes = audioResponse.getAudioData_asU8();
          this.playAudio(audioBytes);
        }
      }
    });

    this.client.setErrorCallback(
      (type: string, statusMessage: string, status: string) => {
        console.error("Convai Error:", type, statusMessage, status);
      },
    );
  }

  private playAudio(audioBytes: Uint8Array) {
    // Convert audio bytes to blob and play
    const blob = new Blob([audioBytes], { type: "audio/wav" });
    const url = URL.createObjectURL(blob);
    const audio = new Audio(url);
    audio.play();
  }

  public sendText(text: string) {
    this.client?.sendTextChunk(text);
  }

  public startAudioRecording() {
    this.client?.startAudioChunk();
  }

  public stopAudioRecording() {
    this.client?.endAudioChunk();
  }

  public resetSession() {
    this.client?.resetSession();
  }

  public destroy() {
    this.client?.resetSession();
    this.client = null;
  }
}

// Usage
const chatManager = new ChatManager("your-api-key", "your-character-id");

// Send text message
chatManager.sendText("Hello, how are you?");

// Start voice recording
chatManager.startAudioRecording();

// Stop voice recording
setTimeout(() => {
  chatManager.stopAudioRecording();
}, 5000);

// Reset session
chatManager.resetSession();

UMD Example (using unpkg)

For direct browser usage without a bundler:

<!DOCTYPE html>
<html>
  <head>
    <title>Convai Web SDK - UMD Example</title>
    <script src="https://unpkg.com/@airsurfer09/web-sdk@latest/dist/vanilla/umd/convai-web-client.umd.js"></script>
  </head>
  <body>
    <div id="chat-container">
      <div id="messages"></div>
      <input
        type="text"
        id="message-input"
        placeholder="Type your message..."
      />
      <button onclick="sendText()">Send Text</button>
      <button onclick="startRecording()">Start Recording</button>
      <button onclick="stopRecording()">Stop Recording</button>
      <button onclick="resetSession()">Reset Session</button>
    </div>

    <script>
      // Access the global variable
      const { ConvaiClient } = window["convai-web-core"];

      let client = null;

      function initializeClient() {
        client = new ConvaiClient({
          apiKey: "your-api-key",
          characterId: "your-character-id",
          enableAudio: true,
          sessionId: `session-${Date.now()}`,
          narrativeTemplateKeysMap: new Map(),
          languageCode: "en-US",
          enableFacialData: false,
          faceModel: 3,
        });

        // Set up response callback
        client.setResponseCallback((response) => {
          if (response.hasAudioResponse()) {
            const audioResponse = response.getAudioResponse();
            if (audioResponse && audioResponse.hasTextData()) {
              const text = audioResponse.getTextData();
              addMessage("NPC", text);

              // Play audio if available
              if (audioResponse.hasAudioData()) {
                const audioBytes = audioResponse.getAudioData_asU8();
                playAudio(audioBytes);
              }
            }
          }
        });

        // Set up error callback
        client.setErrorCallback((type, statusMessage, status) => {
          console.error("Convai Error:", type, statusMessage, status);
          addMessage("System", `Error: ${statusMessage}`);
        });
      }

      function addMessage(sender, text) {
        const messagesDiv = document.getElementById("messages");
        const messageDiv = document.createElement("div");
        messageDiv.innerHTML = `<strong>${sender}:</strong> ${text}`;
        messagesDiv.appendChild(messageDiv);
        messagesDiv.scrollTop = messagesDiv.scrollHeight;
      }

      function playAudio(audioBytes) {
        const blob = new Blob([audioBytes], { type: "audio/wav" });
        const url = URL.createObjectURL(blob);
        const audio = new Audio(url);
        audio.play();
      }

      function sendText() {
        const input = document.getElementById("message-input");
        const text = input.value.trim();
        if (text && client) {
          addMessage("User", text);
          client.sendTextChunk(text);
          input.value = "";
        }
      }

      function startRecording() {
        if (client) {
          client.startAudioChunk();
          addMessage("System", "Recording started...");
        }
      }

      function stopRecording() {
        if (client) {
          client.endAudioChunk();
          addMessage("System", "Recording stopped.");
        }
      }

      function resetSession() {
        if (client) {
          client.resetSession();
          addMessage("System", "Session reset.");
        }
      }

      // Initialize client when page loads
      window.addEventListener("load", initializeClient);

      // Handle Enter key in input
      document
        .getElementById("message-input")
        .addEventListener("keypress", (e) => {
          if (e.key === "Enter") {
            sendText();
          }
        });
    </script>
  </body>
</html>

Available Exports

Main Package Exports (React-first)

// React Hook and Types
import {
  useConvaiClient, // React hook for Convai integration
  ConvaiClientState, // State interface for the hook
  ConvaiClientActions, // Actions interface for the hook
  ConvaiClientRefs, // Refs interface for the hook
  UseConvaiClientReturn, // Return type of useConvaiClient
} from "convai-web-sdk";

// Core Client
import {
  ConvaiClient, // Main client class
  GetResponseResponse, // Proto response type
} from "convai-web-sdk";

// Proto Types
import {
  ActionConfig, // Action configuration
  AudioConfig, // Audio configuration
  TriggerConfig, // Trigger configuration
  GetResponseRequest, // Request type
  FeedbackRequest, // Feedback request
  FeedbackResponse, // Feedback response
  STTRequest, // Speech-to-text request
  STTResponse, // Speech-to-text response
  VisemesData, // Visemes data
  BlendShapesData, // Blend shapes data
  EmotionResponse, // Emotion response
} from "convai-web-sdk";

// Audio Utilities
import {
  AudioRecorder, // Audio recording utility
  AudioPlayer, // Audio playback utility
} from "convai-web-sdk";

// Device Management
import {
  getDeviceId, // Get current device ID
  generateDeviceId, // Generate new device ID
  generateSecureDeviceId, // Generate secure device ID
  regenerateDeviceId, // Regenerate device ID
  resetDeviceId, // Reset device ID
  hasDeviceId, // Check if device ID exists
  getStorageMethod, // Get storage method used
  getDeviceFingerprint, // Get device fingerprint
  isDeviceFingerprintingAvailable, // Check fingerprinting availability
  parseDeviceId, // Parse device ID components
} from "convai-web-sdk";

// Configuration Types
import {
  ConvaiGRPCClientConfigType, // GRPC client configuration
  ActionConfigParamsType, // Action configuration parameters
} from "convai-web-sdk";

// Narrative Design Functions
import {
  createSection,
  editSection,
  getSection,
  listSection,
  deleteSection,
  createTrigger,
  updateTrigger,
  deleteTrigger,
  getTrigger,
  listTriggers,
  getServiceUrls,
} from "convai-web-sdk";

// Client Management
import {
  manageClient,
  setGrpcConfig,
  generateNewCurrentClient,
  getCurrentClient,
} from "convai-web-sdk";

Vanilla Package Exports

// Import from vanilla subpath for non-React applications
import {
  ConvaiClient,
  ConvaiGRPCClient,
  AudioRecorder,
  AudioPlayer,
  GetResponseResponse,
  ActionConfigParamsType,
  ConvaiGRPCClientConfigType,
  // Device management functions
  getDeviceId,
  generateDeviceId,
  generateSecureDeviceId,
  regenerateDeviceId,
  resetDeviceId,
  hasDeviceId,
  getStorageMethod,
  getDeviceFingerprint,
  isDeviceFingerprintingAvailable,
  parseDeviceId,
  // Client management
  manageClient,
  setGrpcConfig,
  generateNewCurrentClient,
  getCurrentClient,
  // Narrative design functions
  createSection,
  editSection,
  getSection,
  listSection,
  deleteSection,
  createTrigger,
  updateTrigger,
  deleteTrigger,
  getTrigger,
  listTriggers,
  getServiceUrls,
  // Proto types
  GetResponseResponse,
  ActionConfig,
  AudioConfig,
  TriggerConfig,
  GetResponseRequest,
  FeedbackRequest,
  FeedbackResponse,
  STTRequest,
  STTResponse,
  VisemesData,
  BlendShapesData,
  EmotionResponse,
} from "convai-web-sdk/vanilla";

Narrative Design Functions

The SDK includes comprehensive narrative design functionality for managing character sections and triggers:

import {
  createSection,
  editSection,
  getSection,
  listSection,
  deleteSection,
  createTrigger,
  updateTrigger,
  deleteTrigger,
  getTrigger,
  listTriggers,
  getServiceUrls,
} from "convai-web-sdk/vanilla";

// Example: Create a new narrative section
const section = await createSection({
  objective: "Help users with customer service",
  sectionName: "Customer Support",
  apiKey: "your-api-key",
  characterId: "your-character-id",
});

// Example: Create a trigger
const trigger = await createTrigger({
  triggerName: "Support Request",
  triggerMessage: "I need help with my order",
  destinationSection: "section-id",
  apiKey: "your-api-key",
  characterId: "your-character-id",
});

Proto Types

// Direct proto imports (if needed)
import { GetResponseResponse } from "convai-web-sdk/dist/proto/service/service_pb";

Project Structure

src/
├── vanilla/           # Vanilla JavaScript/TypeScript files
│   ├── index.ts      # Vanilla entry point
│   ├── convai_client.ts
│   ├── convai_grpc_client.ts
│   ├── audio_recorder.ts
│   ├── audio_recorder_processor.ts
│   ├── streaming_audio_player.ts
│   ├── deviceId.ts
│   ├── manageClients.ts
│   ├── narrativeDesign.ts
│   └── CharacterUtil.ts
├── react/            # React-specific files
│   ├── index.ts      # React entry point
│   └── useConvaiClient.tsx
├── shared/           # Shared types and utilities
│   └── types.ts
└── index.ts          # Main entry point (exports everything)

dist/
├── vanilla/          # Vanilla builds
│   ├── esm/         # ES Modules format
│   └── umd/         # UMD format
├── react/           # React builds
│   └── esm/         # ES Modules format
├── types/           # TypeScript declarations
└── proto/           # Protocol buffer files

Build Outputs

The package provides multiple build formats optimized for React developers:

Entry Points

• Main (React): dist/react/esm/convai-web-client-react.js (ESM) - Primary entry point
• Vanilla: dist/vanilla/esm/convai-web-client.js (ESM) and dist/vanilla/umd/convai-web-client.umd.js (UMD)

Formats

• ESM (.js) - Modern ES modules for bundlers (React-first)
• UMD (.umd.js) - Universal module definition for browsers (vanilla)

Development

Building

# Build everything
npm run build

# Build specific targets
npm run build:vanilla
npm run build:react
npm run build:types

# Clean and rebuild
npm run lib:build

Build Scripts

• build:vanilla - Builds vanilla JavaScript/TypeScript files
• build:react - Builds React-specific files
• build:types - Generates TypeScript declarations
• lib:build - Full build process including Proto files

Tree Shaking

The package is optimized for tree shaking. When you import only what you need, unused code will be eliminated:

// Only imports React hook - vanilla exports are tree-shaken
import { useConvaiClient } from "convai-web-sdk";

// Only imports ConvaiClient - React exports are tree-shaken
import { ConvaiClient } from "convai-web-sdk/vanilla";

Dependencies

Peer Dependencies

• react >= 16.8.0 (for React functionality)

Dev Dependencies

• TypeScript and build tools
• Webpack for bundling
• Protocol buffer tools

License

Apache-2.0