Porcupine Wake Word

@capacitor-community/porcupine-wake-word

This plugin is a bridge to the native SDKs provided by Picovoice on iOS, Android and Web platforms for their product called Porcupine Wake Word.

Maintainers

Maintainer	GitHub
Julien Lecoq	JulienLecoq

Supported platforms

Name	Android	iOS	Web
Status	✅	✅	❌

Introduction

This plugin is a bridge to the native SDKs provided by Picovoice on iOS, Android and Web for their product called Porcupine Wake Word.

A wake word is a special word or phrase that is meant to activate a device when spoken. It is also referred to as 'hotword', 'trigger word', and 'wake up word'.

This plugin is a perfect fit to combine with the speech-recognition plugin to allow always listening feature in a power efficient manner.

⚠️ Never use speech-recognition alone to mimic always listening feature: doing that will result in a very high power consumption (see: Apple documentation on their Speech native API which says in section: Create a Great User Experience for Speech Recognition):

Speech recognition places a relatively high burden on battery life and network usage.

Porcupine

Porcupine is a highly accurate and lightweight wake word engine. It enables building always-listening voice-enabled applications using cutting edge voice AI.

Porcupine is:

• private and offline
• accurate
• resource efficient (runs even on microcontrollers)
• data efficient (wake words can be easily generated by simply typing them, without needing thousands of hours of bespoke audio training data and manual effort)
• scalable to many simultaneous wake-words / always-on voice commands
• cross-platform

To learn more about Porcupine, see the product, documentation, and GitHub pages.

Custom wake words

Porcupine includes several built-in keywords, which are stored as .ppn files. To train custom PPN files, see the Picovoice Console.

Unlike the built-in keywords, custom PPN files generated with the Picovoice Console carry restrictions including (but not limited to): training allowance, time limits, available platforms, and commercial usage.

Custom model file

In order to detect non-English wake words you need to use the corresponding model file. The model files for all supported languages are available here. By default, Porcupine will use a model file for the English language.

AccessKey

Porcupine requires a valid Picovoice AccessKey at initialization. AccessKey acts as your credentials when using Porcupine SDKs. You can get your AccessKey for free. Make sure to keep your AccessKey secret. Signup or Login to Picovoice Console to get your AccessKey.

Install

npm install @capacitor-community/porcupine-wake-word
npx cap sync

Android

Permissions

This API requires the following permissions be added to your AndroidManifest.xml:

<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />

Read about Setting Permissions in the Android Guide for more information on setting Android permissions.

The RECORD_AUDIO permission is a runtime permission that must be granted by the user before any usage of PorcupineWakeWord.start() (which will record audio from the user's device).

Plugin registration

Make sure to register the plugin in your main activity. This is a file placed at: /android/app/src/main/java/domainNameOfYourApp/MainActivity.java from the root of your project.

package io.ionic.starter;

import android.os.Bundle;

import com.getcapacitor.BridgeActivity;
import com.getcapacitor.community.porcupinewakeword.PorcupineWakeWordPlugin;

public class MainActivity extends BridgeActivity {
    @Override
    public void onCreate(Bundle savedInstanceState) {
        registerPlugin(PorcupineWakeWordPlugin.class);
        super.onCreate(savedInstanceState);
    }
}

Configuration

The root path for finding models is: /android/app/src/main/assets/. Hence, the following call will search for models/myKeywordModel.ppn in /android/app/src/main/assets/models/myKeywordModel.ppn. The same rule apply for models/myModel.pv.

PorcupineWakeWord.initFromCustomKeywords({
    accessKey: "myAccessKey",
    keywordPathOpts: [{
        keywordPath: "models/myKeywordModel.ppn",
        sensitivity: 0.8
    }],
    modelPath: "models/myModel.pv",
})

Usage

Example usage using a custom keyword.

import { PorcupineWakeWord, KeywordEventData, ErrorEventData } from 'capacitor-porcupine-wake-word';

async function listenForWakeWord(): Promise<void> {
    await PorcupineWakeWord.initFromCustomKeywords({
        accessKey: "myAccessKey",
        keywordPathOpts: [{
            keywordPath: "models/myKeywordModel.ppn",
            sensitivity: 0.8
        }],
        modelPath: "models/myModel.pv",
    })

    PorcupineWakeWord.addListener("keywordDetected", (keyword: KeywordEventData) => {
        console.log('Keyword detected:', keyword)
    })

    PorcupineWakeWord.addListener("error", (error: ErrorEventData) => {
        console.log('Error detected:', error.message)
    })

    return PorcupineWakeWord.start()
}

async function main() {
    const result = await PorcupineWakeWord.hasPermission()

    if (result.hasPermission) {
        this.listenForWakeWord()
    } else {
        const permissionStatus = await PorcupineWakeWord.requestPermission()
        if (permissionStatus.record_audio === "granted") {
            this.listenForWakeWord()
        }
    }
}

Example usage using a built in keyword.

import { PorcupineWakeWord, BuiltInKeyword, KeywordEventData, ErrorEventData } from 'capacitor-porcupine-wake-word';

async function listenForWakeWord(): Promise<void> {
    await PorcupineWakeWord.initFromBuiltInKeywords({
        accessKey: "myAccessKey",
        keywordOpts: [{
            keyword: BuiltInKeyword.OK_GOOGLE,
        }],
    })

    PorcupineWakeWord.addListener("keywordDetected", (keyword: KeywordEventData) => {
        console.log('Keyword detected:', keyword)
    })

    PorcupineWakeWord.addListener("error", (error: ErrorEventData) => {
        console.log('Error detected:', error.message)
    })

    return PorcupineWakeWord.start()
}

async function main() {
    const result = await PorcupineWakeWord.hasPermission()

    if (result.hasPermission) {
        this.listenForWakeWord()
    } else {
        const permissionStatus = await PorcupineWakeWord.requestPermission()
        if (permissionStatus.record_audio === "granted") {
            this.listenForWakeWord()
        }
    }
}

API

• initFromBuiltInKeywords(...)
• initFromCustomKeywords(...)
• start()
• stop()
• delete()
• addListener('error', ...)
• addListener('keywordDetected', ...)
• removeAllListeners()
• hasPermission()
• checkPermission()
• requestPermission()
• isListening()
• isInitialized()
• Interfaces
• Type Aliases
• Enums

initFromBuiltInKeywords(...)

initFromBuiltInKeywords(options: BuiltInKeywordInitOptions) => Promise<void>

Initialize Porcupine from built in keywords.

Rejects:

• JSONException: if there is an error while decoding the JSON from the method parameter.
• PorcupineException: if there is an error while initializing Porcupine.

Resolves when Porcupine finished its initialization.

Param	Type
options	BuiltInKeywordInitOptions

---

initFromCustomKeywords(...)

initFromCustomKeywords(options: KeywordPathInitOptions) => Promise<void>

Initialize Porcupine from custom keywords (path of trained models of keywords).

Rejects:

• JSONException: if there is an error while decoding the JSON from the method parameter.
• PorcupineException: if there is an error while initializing Porcupine.

Resolves when Porcupine finished its initialization.

Param	Type
options	KeywordPathInitOptions

---

start()

start() => Promise<void>

Starts recording audio from the microphone and monitors it for the utterances of the given set of keywords.

Rejects from native Porcupine iOS sdk:

• If porcupine is not initialized.
• If the user has not granted the record_audio permission.

Rejects:

• If porcupine is not initialized.
• If the user has not granted the record_audio permission.

Resolves when the recording from the microphone has started, hence Porcupine listening for the utterances of the given set of keywords.

---

stop()

stop() => Promise<void>

Stops recording audio from the microphone. Hence, stop listening for wake words.

Rejects from native Porcupine Android sdk:

• PorcupineException message: if the PorcupineManager.MicrophoneReader throws an exception while it's being stopped.

Resolves when the recording from the microphone has stopped.

---

delete()

delete() => Promise<void>

Releases resources acquired by Porcupine. It should be called when disposing the object. Resolves when resources acquired by Porcupine have been released.

---

addListener('error', ...)

addListener(eventName: "error", listenerFunc: (data: ErrorEventData) => void) => void

Register a callback function to run if errors occur while processing audio frames.

Param	Type
eventName	'error'
listenerFunc	(data: ErrorEventData) => void

---

addListener('keywordDetected', ...)

addListener(eventName: "keywordDetected", listenerFunc: (data: KeywordEventData) => void) => void

Register a callback function that is invoked upon detection of the keywords specified during the initialization of Porcupine.

Param	Type
eventName	'keywordDetected'
listenerFunc	(data: KeywordEventData) => void

---

removeAllListeners()

removeAllListeners() => Promise<void>

Remove all registered callback functions.

---

hasPermission()

hasPermission() => Promise<PermissionBool>

Check if the user has granted the record_audio permission.

Returns: Promise<PermissionBool>

---

checkPermission()

checkPermission() => Promise<PermissionStatus>

Check record_audio permission.

Returns: Promise<PermissionStatus>

---

requestPermission()

requestPermission() => Promise<PermissionStatus>

Request record_audio permission. Resolves with the new permission status after the user has denied/granted the request.

Returns: Promise<PermissionStatus>

---

isListening()

isListening() => Promise<ValueResult<boolean>>

Returns true if the plugin is listening for wake words, false otherwise.

Returns: Promise<ValueResult<boolean>>

---

isInitialized()

isInitialized() => Promise<ValueResult<boolean>>

Returns true if the plugin is initialized, false otherwise.

Returns: Promise<ValueResult<boolean>>

---

Interfaces

BuiltInKeywordInitOptions

Prop	Type
keywordOpts	BuiltInKeywordInitOption[]

BuiltInKeywordInitOption

Prop	Type	Description	Default
keyword	BuiltInKeyword	Built in keyword to listen for (keyword provided by Porcupine).
sensitivity	number	Sensitivity is the parameter that enables trading miss rate for the false alarm rate. This is a floating-point number within [0, 1]. A higher sensitivity reduces the miss rate at the cost of increased false alarm rate.	0.5

KeywordPathInitOptions

Prop	Type
keywordPathOpts	KeywordPathInitOption[]

KeywordPathInitOption

Prop	Type	Description	Default
keywordPath	string	Path to the trained model for the given keyword to listen for.
sensitivity	number	Sensitivity is the parameter that enables trading miss rate for the false alarm rate. This is a floating-point number within [0, 1]. A higher sensitivity reduces the miss rate at the cost of increased false alarm rate.	0.5

ErrorEventData

Prop	Type	Description
message	string	The message of the error.

KeywordEventData

Prop	Type	Description
index	number	The index of the keyword (index taken from the array passed during the initiliazation of Porcupine) that has been detected.

PermissionBool

Prop	Type	Description
hasPermission	boolean	Permission state for record_audio alias.

PermissionStatus

Prop	Type	Description
record_audio	PermissionState	Permission state for record_audio alias.

ValueResult

Prop	Type
value	T

Type Aliases

PermissionState

'prompt' | 'prompt-with-rationale' | 'granted' | 'denied'

Enums

BuiltInKeyword

Members	Value
ALEXA	"ALEXA"
AMERICANO	"AMERICANO"
BLUEBERRY	"BLUEBERRY"
BUMBLEBEE	"BUMBLEBEE"
COMPUTER	"COMPUTER"
GRAPEFRUIT	"GRAPEFRUIT"
GRASSHOPPER	"GRASSHOPPER"
HEY_GOOGLE	"HEY_GOOGLE"
HEY_SIRI	"HEY_SIRI"
JARVIS	"JARVIS"
OK_GOOGLE	"OK_GOOGLE"
PICOVOICE	"PICOVOICE"
PORCUPINE	"PORCUPINE"
TERMINATOR	"TERMINATOR"