You can use Augnito SDK to make use of the Speech Recognition AI. To get access credentials or talk to us about your project, get in touch with us at partnerships@augnito.ai.
Install the library in your Typescript project.
npm install augnitosdk
use https://cdn.jsdelivr.net/npm/augnitosdk@latest/dist/augnitosdk.js to import the library from npm using CDN url.
import { Augnito } from 'augnitosdk';
import { Augnito } from 'https://cdn.jsdelivr.net/npm/augnitosdk@latest/dist/augnitosdk.js';
Note - All other steps are common for Vanilla JS and Typescript implementations.
const augnitoConfig: AugnitoConfig = {
server: '<your domain server>',
accountCode: '<your accountcode>',
accessKey: '<your accesskey>',
userTag: '<your usertag>',
sourceApp: '<your sourceapp>',
lmId: '<your lmid>'
noiseCt: 1, //Optional parameter, to set noice cancellation value and default value is 1 which is off
enableLogs: false, // Optional parameter, to check debug logs set to true
listOfControls: [], // Optional parameter, set a list of type TargetEditor. Setting editor instance is necessary for EditorType.DEVEXPRESS and to handle navigation command between fields
editor: EditorType.HTML // Optional parameter, set EditorType enum for SDK to insert sppech-to-text display at cursor placed inside field and to handle commands dictated. Other enum values are EditorType.CKEDITOR4 | EditorType.CKEDITOR5 | EditorType.GENERIC | EditorType.DEVEXPRESS,
};
const augnito = new Augnito(augnitoConfig);
Now all you have to do is toggle the status when you want to connect/disconnect!
// Toggles the Speech API connection. Set optional parameter shouldPauseResume to true to toggle pause and resume dictation. This will ensure socket connection remains connected throughout. If no parameter is given, then the method will function as toggle of start and stop dictation. This will create a new socket connection on every start
augnito.toggleListening((shouldPauseResume = false));
augnito.requestMicOff(); //call this method to close the socket connection and clear all resources in case shouldPauseResume is set to true
Whether you want to use the Augnito Mic as your microphone, or you want to develop your own app, we got you covered.
All you have to do is initialize the mobile configuration. The SDK will generate a value that you need to use to generate a QR code which should be scanned in the App.
import { AugnitoMobileConfig } from 'augnitosdk';
const mobileConfig: AugnitoMobileConfig = {
enableLogs: false
};
const augnitoMobile = augnito.initMobileClient(mobileConfig);
//Generate a QR Code with the following value
const code = augnito.getQRCode();
If you have a Philips speech mic and want to use it as the input device, then include the below code.
import { PhilipsMicMode } from 'augnitosdk';
const philipMicConfig = {
philipsMicMode: PhilipsMicMode['HandsFree'] //other options include PhilipsMicMode.PushToTalk or PhilipsMicMode.BOTH
};
const philipMicHandler = augnito.initPhilipMicClient(philipMicConfig);
- HandsFree - where you push the record button speak and the push the button to stop
- PushToTalk - where you press and hold the record button to speak. Release of the button will stop
- BOTH - SDK will intelligently handle the other two modes based on user action
Name | type | Description |
---|---|---|
onFinalResult | (finalText: Recipe) => boolean | A final result is called when text has been processed by the Speech Server. Returns true to stop the processing in the SDK |
onCommandResult | (command: Recipe) => boolean | A command generated from the server. Returns true to stop the processing in the SDK |
onStateChanged | (isConnected: boolean) => void | Callback to indicate the status of the microphone listening state has changed. |
onSessionEvent | (data: AugnitoSocketResponse) => void | Callback to intercept Session Events |
onIdleMic | () => void | Callback when the Mic is idle for 5 minutes |
onMicrophoneOnError | (isWebClient: boolean) => void | Callback triggered when trying to start the connection, but it fails because it is already opened. (It can be opened in another tab, or it was not closed properly) |
onError | (error: AugnitoSDKErrorMessage) => void | Callback triggered when the SDK encouters an error |
Name | type | Description |
---|---|---|
onConnectionRequest | () => void | Callback when a mobile client is requesting to connect |
onMobileScan | () => void | Callback when a mobile client has scanned the QR Code |
Name | type | Description |
---|---|---|
onPhilipsMicPressed | () => void | Callback when record button is pressed on Philips Speech Mic |
prop | type | notes |
---|---|---|
toggleListening | void | Toggles the Speech API connection: if already connected, it will stop the web socket; otherwise, it will start a new connection. If optional parameter shouldPauseResume is set to true then listening to mic will be suspended and resumed, here web socket will remain alive throughout the session. Consumers of the sdk are expected to call requestMicOff to stop the web sockeet and clean the resources in the end |
isConnected | boolean | Returns the status of the web socket: true if connected; otherwise false. |
getQRCode | string | Returns the Generated Value for the QR Code (If and only if the DeviceId was provided). |
dispose | void | Dispose all resources. |
initMobileClient | AugnitoMobile | Creates a web socket for getting notifications from client mobile application: only to be used with Mobile App |
apisFactory | FactoryAPI | Returns an Abstract Factory to create different API Clients. |
requestMicOff | void | Sends a request to close all opened connections. |
initPhilipMicClient | PhilipsMicClient | To be used with Philips Speecch Mic. It will listen to press event of record button on the mic. |
name | type | notes |
---|---|---|
getMacros | Promise: MacroResult | Returns a list of macros for the specified user. |
upsertMacro | Promis: MacroResult | Creates or Updates a Macro for the specified user. |
deleteMacro | Promise: MacroResult | Deletes a Macro for the specified user. |
// If you already have an augnito client..
const macrosClient = augnito.apisFactory.getMacrosAPI();
// OR
import { MacrosAPI } from 'augnitosdk';
const macrosClient = new MacrosAPI(accountCode, accessKey, server);
macrosClient
.getMacros({
UserTag: userTag
})
.then((result) => {
console.log(result);
});
interface Edge
{
PK: string;
SK: string;
}
interface UserFormattingRequest
{
PageId?: Edge; // Optional page edge for next and previous pages of search results
Limit?: number; // Optional limit for number of search results
}
interface FormattingItem
{
SpokenForm: string; // The spoken form of the formatting preference
DisplayForm: string; // The display form of the formatting preference
ID: string; // The ID of the formatting preference
}
interface FormattingResult
{
Status: number; // Response status code
ErrorMessage: string; // Error message if any
Data: {
StatusMessage: string; // Response status message
FormattingPreferenceItems? :
{
PageSize: number; // Number of items in the current page
PageIdNext?: Edge; // Next page edge if any
PageIdPrevious?: Edge; // Previous page edge if any
Items: Array<FormattingItem>; // List of formatting preferences
};
};
}
interface UpsertFormattingRequest {
SpokenForm: string; // The spoken form of the formatting preference
DisplayForm: string; // The display form of the formatting preference
ID?: string; // The ID of the formatting preference if updating an existing preference
}
RemoveFormattingRequest {
ID: string; // The Id of the formatting preference to be deleted
}
name | type | notes |
---|---|---|
getFormatting(request: UserFormattingRequest) | Promise: FormattingResult | Returns a list of formatting preferfences for the specified user. |
upsertFormatting(request: UpsertFormattingRequest) | Promise: FormattingResult | Creates or Updates a formatting preference for the specified user. |
removeFormatting(request: RemoveFormattingRequest) | Promise: FormattingResult | Deletes a formatting preference for the specified user. |
// If you already have an augnito client..
const formattingAPI = augnito.apisFactory.getFormattingAPI();
// OR
import { FormattingAPI } from 'augnitosdk';
const formattingAPI = new FormattingAPI({
UserCode: <your usercode>,
SubscriptionCode: <your subscription code>,
AccessKey: <your access key>,
SmId: <your sm id>,
DeviceCode: <your device code>,
SourceApp: <your source app>,
AccountCode: <your account code>,
Server: <your server>,
EnableLogs: true | false,
});
formatApi
.getFormatting({
Limit: 10
})
.then((result) => {
console.log('formatting', result);
});
Vocabulary items can be added to the model for a specific user and model. When the word has been added and is ready to be recognised a notification will be sent to the client and the microphone will need to be restarted.
interface Edge
{
PK: string;
SK: string;
}
interface UserVocabularyRequest
{
PageId?: Edge; // Optional page edge for next and previous pages of search results
Limit?: number; // Optional limit for number of search results
}
interface VocabularyItem
{
Word: string; // The word
SpokenWord: string | null; // The , optional, spoken form of the word
IsAbbreviation: boolean; // Whether the word is an abbreviation or not
CreatedAt: Date; // The date the word was added to the vocabulary
Status: string; // The status of the word
}
interface VocabularyResult
{
Status: number; // Response status code
ErrorMessage: string; // Error message if any
Data: {
StatusMessage: string; // Response status message
WordItems? :
{
SubscriptionCode: string; // The subscription code
UserCode: string; // The user code the word has been added for
PageSize: number; // Number of items in the current page
PageIdNext?: Edge; // Next page edge if any
PageIdPrevious?: Edge; // Previous page edge if any
Items: Array<VocabularyItem>; // List of vocabulary items
};
};
}
interface UpsertVocabularyRequest
{
file: File; // Audio file of the word to be added
Word: string; // The word
SpokenWord: string; // The, optional, spoken form of the word
IsAbbreviation: boolean; // Whether the word is an abbreviation or not
}
interface RemoveVocabularyRequest
{
Word: string; // The word to be removed
}
name | type | notes |
---|---|---|
getVocabulary(request: UserVocabularyRequest) | Promise: VocabularyResult | Returns a list of vocabulary for the specified user. |
upsertFormatting(request: UpsertVocabularyRequest) | Promise: VocabularyResult | Creates or Updates vocabulary for the specified user. |
removeFormatting(request: RemoveVocabularyRequest) | Promise: VocabularyResult | Deletes a vocabulary for the specified user. |
Name | type | Description |
---|---|---|
onVocabularyAdded | (notification: AugnitoAppNotification) => void | Notification that a vocabulary word has been processed. |
Name | type | Description |
---|---|---|
startRecording() | (Promise) => void | Starts recording audio. Will turn off main Augnito mic if on |
stopRecording() | (blob:Promise) => void | Stops recording audio and returns the audio |
The audio for the vocabulary to be added must be between 1 and 8 seconds long. If you call stop recording less than 1 second after start the recording will wait for a second to have passed since it started before resolving the promise. If you call stop recording more than 8 seconds after start the returned recording will be the first 8 seconds recorded.
const vocabularyAPI = augnito.apisFactory.getVocabularyAPI();
// Add a callback to be notified when a vocabulary word has been processed
vocabularyAPI.onVocabularyAdded = (notification: AugnitoAppNotification) => {
console.log(notification);
};
// Start recording
vocabularyAPI.startRecording();
... // Record for 1-8 seconds
// Stop recording
vocabularyAPI.stopRecording().then((blob) => {
// Add the word to the vocabulary
vocabularyAPI.upsertVocabulary({
file: new File([blob], 'test.wav'),
Word: 'test',
SpokenWord: 'test',
IsAbbreviation: false,
});
});
const loadWords = (page?: Edge) => {
vocabularyAPI
.getVocabulary({ Limit: 10, PageId: page })
.then(function (result) {
if (typeof result != 'string') {
result.Data.WordItems?.Items.forEach((value) => {
console.log(value);
});
if (result.Data.WordItems?.PageIdNext) {
loadWords( result.Data.WordItems.PageIdNext);
}
}
});
};
loadWords();
Name | type | Description |
---|---|---|
getMedicineOrderData | Promise: MedicineOrderingResult Or String | Returns medicine ordering data for text specified by the user. |
// If you already have an augnito client..
const orderingClient = augnito.apisFactory.getMedicineOrderingAPI();
orderingClient
.getMedicineOrderData({
Text: ''
})
.then((result) => {
console.log(result);
});