agora-conversational-ai-denoiser

Agora AI Denoiser

This package provides AI denoiser feature for Agora Web SDK.

Requirements

• Agora Web SDK (v4.15.1 or later)

Highlights

• Smaller JavaScript file
• Better performance
• Friendly APIs

Usage

Install packages agora-rtc-sdk-ng and agora-conversational-ai-denoiser, then import these packages in your code.

import AgoraRTC from "agora-rtc-sdk-ng";
import { AIDenoiserExtension } from "agora-conversational-ai-denoiser";
import type { IAIDenoiserProcessor } from "agora-conversational-ai-denoiser";

Set WebAudio samplerate to 48000 with WebSDK parameter before AgoraRTC.createClient()

// @ts-ignore
AgoraRTC.setParameter('WEBAUDIO_INIT_OPTIONS', {
  latencyHint: 0.03,
  sampleRate: 48000,
});

Create an extension instance and register it.

const extension = new AIDenoiserExtension({
  assetsPath: "./external",
  fetchOptions: { cache: "no-cache" },
});
AgoraRTC.registerExtensions([extension]);

if (!extension.checkCompatibility()) {
  throw new Error("Browser unsupported");
}

Once join, let track be an ILocalAudioTrack, create a processor and pipe it to track.

processor.on("overload", () => {
  console.warning("processor may overload");
});
processor.on("pipeerror", (error: Error) => {
  console.error(`failed to pipe processor: ${error}`);
  processor.unpipe();
  track.unpipe();
  track.pipe(track.processorDestination);
});

const processor = extension.createProcessor();
track.pipe(processor).pipe(track.processorDestination);

Enable or disable the processor.

const enable = async () => {
  const enabled = processor.enabled;
  if (!enabled) {
    await processor.enable();
  }
};
const disable = async () => {
  const enabled = processor.enabled;
  if (enabled) {
    await processor.disable();
  }
};

If there is an audio issue, please dump the audio data.

processor.on("dump", (blob: Blob, name: string) => {
  const a = document.createElement("a");
  const url = URL.createObjectURL(blob);
  a.href = url;
  a.download = name;
  a.click();
  setTimeout(() => {
    URL.revokeObjectURL(url);
  }, 0);
});
processor.on("dumpend", () => {
  console.log("dumpend");
});

Unpipe and destroy the processor when it is not longer in use.

const destroy = async () => {
  processor.unpipe();
  track.unpipe();
  track.pipe(track.processorDestination);
  await processor.destroy();
}

Refer to Agora Docs for further details.

---

© Copyright Agora, Inc.