Have ideas to improve npm?Join in the discussion! »

    @picovoice/porcupine-react-native
    TypeScript icon, indicating that this package has built-in type declarations

    1.9.9 • Public • Published

    Porcupine Binding for React Native

    Porcupine

    Porcupine is is a highly accurate and lightweight wake word engine. It enables building always-listening voice-enabled applications using cutting edge voice AI.

    Porcupine is:

    • private and offline
    • accurate
    • resource efficient (runs even on microcontrollers)
    • data efficient (wake words can be easily generated by simply typing them, without needing thousands of hours of bespoke audio training data and manual effort)
    • scalable to many simultaneous wake-words / always-on voice commands
    • cross-platform

    To learn more about Porcupine, see the product, documentation, and GitHub pages.

    Custom wake words

    Porcupine includes several built-in keywords, which are stored as .ppn files. To train custom PPN files, see the Picovoice Console.

    Unlike the built-in keywords, custom PPN files generated with the Picovoice Console carry restrictions including (but not limited to): training allowance, time limits, available platforms, and commercial usage.

    Compatibility

    This binding is for running Porcupine on React Native 0.62.2+ on the following platforms:

    • Android 4.1+ (API 16+)
    • iOS 9.0+

    Installation

    To start install be sure you have installed yarn and cocoapods. Then add these two native modules to your react-native project.

    yarn add @picovoice/react-native-voice-processor
    yarn add @picovoice/porcupine-react-native

    or

    npm i @picovoice/react-native-voice-processor --save
    npm i @picovoice/porcupine-react-native --save

    Link the iOS package

    cd ios && pod install && cd ..

    NOTE: Due to a limitation in React Native CLI autolinking, these two native modules cannot be included as transitive depedencies. If you are creating a module that depends on porcupine-react-native and/or react-native-voice-processor, you will have to list these as peer dependencies and require developers to install them alongside.

    Permissions

    To enable recording with the hardware's microphone, you must first ensure that you have enabled the proper permission on both iOS and Android.

    On iOS, open your Info.plist and add the following line:

    <key>NSMicrophoneUsageDescription</key>
    <string>[Permission explanation]</string>

    On Android, open your AndroidManifest.xml and add the following line:

    <uses-permission android:name="android.permission.RECORD_AUDIO" />

    Finally, in your app JS code, be sure to check for user permission consent before proceeding with audio capture:

    let recordAudioRequest;
    if (Platform.OS == 'android') {
        // For Android, we need to explicitly ask
        recordAudioRequest = this._requestRecordAudioPermission();
    } else {
        // iOS automatically asks for permission
        recordAudioRequest = new Promise(function (resolve, _) {
        resolve(true);
        });
    }
    
    recordAudioRequest.then((hasPermission) => {
        if(hasPermission){
            // Code that uses Porcupine
        }
    });
    
    async _requestRecordAudioPermission() {
        const granted = await PermissionsAndroid.request(
        PermissionsAndroid.PERMISSIONS.RECORD_AUDIO,
        {
            title: 'Microphone Permission',
            message: '[Permission explanation]',
            buttonNeutral: 'Ask Me Later',
            buttonNegative: 'Cancel',
            buttonPositive: 'OK',
        }
        );
        return (granted === PermissionsAndroid.RESULTS.GRANTED)
      }

    Usage

    The module provides you with two levels of API to choose from depending on your needs.

    High-Level API

    PorcupineManager provides a high-level API that takes care of audio recording. This class is the quickest way to get started.

    Using the constructor PorcupineManager.fromKeywords will create an instance of the PorcupineManager using one or more of the built-in keywords.

    async createPorcupineManager(){
        try{
            this._porcupineManager = await PorcupineManager.fromKeywords(
                ["picovoice", "porcupine"],
                detectionCallback);
        } catch (err) {
            // handle error
        }
    }

    NOTE: the call is asynchronous and therefore should be called in an async block with a try/catch.

    The detectionCallback parameter is a function that you want to execute when Porcupine has detected one of the keywords. The function should accept a single integer, keywordIndex, which specifies which wake word has been detected.

    detectionCallback(keywordIndex){
        if(keywordIndex === 0){
            // picovoice detected
        }
        else if (keywordIndex === 1){
            // porcupine detected
        }
    }

    Available built-in keywords are stored in the constants PorcupineManager.KEYWORDS and Porcupine.KEYWORDS.

    To create an instance of PorcupineManager that detects custom keywords, you can use the PorcupineManager.fromKeywordPaths static constructor and provide the paths to the .ppn file(s).

    this._porcupineManager = await PorcupineManager.fromKeywordPaths(["/path/to/keyword.ppn"], detectionCallback);

    To add custom keywords to your app, copy the ppn files to your platform's asset folder and get the paths using react-native-fs. An example of this can be found in our Rhino demo.

    In addition to custom keywords, you can override the default Porcupine model file and/or keyword sensitivities. These optional parameters can be passed in like so:

    this._porcupineManager = await PorcupineManager.fromKeywordPaths(
        ["/path/to/keyword/file/one.ppn", "/path/to/keyword/file/two.ppn"],
        detectionCallback,
        'path/to/model/file.pv',
        [0.25, 0.6]);

    Once you have instantiated a PorcupineManager, you can start audio capture and wake word detection by calling:

    let didStart = await this._porcupineManager.start();

    And then stop it by calling:

    let didStop = await this._porcupineManager.stop();

    Once the app is done with using PorcupineManager, be sure you explicitly release the resources allocated to Porcupine:

    this._porcupineManager.delete();

    As you may have noticed, there is no need to deal with audio capture to enable wake word detection with PorcupineManager. This is because it uses our @picovoice/react-native-voice-processor module to capture frames of audio and automatically pass it to the wake word engine.

    Low-Level API

    Porcupine provides low-level access to the wake word engine for those who want to incorporate wake word detection into a already existing audio processing pipeline.

    Porcupine also has fromKeywords and fromKeywordPaths static constructors.

    async createPorcupine(){
        try{
            this._porcupine = await Porcupine.fromKeywords(["picovoice"]);
        } catch (err) {
            // handle error
        }
    }

    As you can see, in this case you don't pass in a detection callback as you will be passing in audio frames directly using the process function:

    let buffer = getAudioFrame();
    
    try {
        let keywordIndex = await this._porcupine.process(buffer);
        if (keywordIndex >= 0) {
            // detection made!
        }
    } catch (e) {
        // handle error
    }

    For process to work correctly, the audio data must be in the audio format required by Picovoice. The required audio format is found by calling .sampleRate to get the required sample rate and .frameLength to get the required frame size. Audio must be single-channel and 16-bit linearly-encoded.

    Finally, once you no longer need the wake word engine, be sure to explicitly release the resources allocated to Porcupine:

    this._porcupine.delete();

    Custom Wake Word Integration

    To add a custom wake word to your React Native application you'll need to add the ppn file to your platform projects. Android models must be added to ./android/app/src/main/res/raw/, while iOS models can be added anywhere under ./ios, but must be included as a bundled resource in your iOS project. Then in your app code, using the react-native-fs package, retrieve the files like so:

    const RNFS = require('react-native-fs');
    
    let wakeWordName = 'keyword';
    let wakeWordFilename = wakeWordName;
    let wakeWordPath = '';
    
    if (Platform.OS == 'android') {
        // for Android, extract resources from APK
        wakeWordFilename += '_android.ppn';
        wakeWordPath = `${RNFS.DocumentDirectoryPath}/${wakeWordFilename}`;
        await RNFS.copyFileRes(wakeWordFilename, wakeWordPath);
    } else if (Platform.OS == 'ios') {
        wakeWordFilename += '_ios.ppn';
        wakeWordPath = `${RNFS.MainBundlePath}/${wakeWordFilename}`;
    }

    Non-English Wake Words

    In order to detect non-English wake words you need to use the corresponding model file. The model files for all supported languages are available here.

    Demo App

    Check out the Porcupine React Native demo to see what it looks like to use Porcupine in a cross-platform app!

    Install

    npm i @picovoice/porcupine-react-native

    DownloadsWeekly Downloads

    94

    Version

    1.9.9

    License

    Apache-2.0

    Unpacked Size

    2.16 MB

    Total Files

    62

    Last publish

    Collaborators

    • avatar
    • avatar
    • avatar