Skip to content

Speech

Jump to bottom
Lance Gao edited this page Jan 13, 2021 · 53 revisions

Speech

temi's speech flow is comprised of four main components - wakeup, ASR (Automatic Speech Recognition), NLP (Natural language processing) and TTS (text to speech).

temi's SDK provides developers tools to utilize, customize and listen to any of the components.


API Overview

Return Method Description
void speak(TtsRequest ttsRequest) Ask temi to speak(play TTS)
void cancelAllTtsRequests() cancel TTS request
void wakeup() Wake up temi
String getWakeupWord() Get current wake-up word
void askQuestion(String question) temi speak actively and wait for user's reply
void finishConversation() Finish a conversation(Stop recording for ASR)
void startDefaultNlu(String text) Trigger default NLU service
Interface Description
TtsListener TTS status listener
WakeupWordListener Wake-up event listener
AsrListener ASR result listener
ConversationViewAttachesListener Conversation view attaches listener
OnTtsVisualizerWaveFormDataChangedListener Listener for wave form data changes of TTS audio visualizer
OnTtsVisualizerFftDataChangedListener Listener for ftt data changes of TTS audio visualizer
Model Description
TtsRequest TTS request instance

Methods

speak()

Use this method to let temi speak something that from the parameter ttsRequest of this method.

  • Parameters

    Parameter Type Description
    ttsRequest TtsRequest An object of type TtsRequest in this object you will add the text to be spoken.

  • Prototype

    void speak(TtsRequest ttsRequest);

  • Required permissions

    None.

  • Support from

    0.10.36

cancelAllTtsRequests()

Stops currently processed TTS request and empty the queue.

  • Prototype

    void cancelAllTtsRequests();

  • Required permissions

    None.

  • Support from

wakeup()

Use this method to trigger temi's wakeup programmatically.

  • Prototype

    void wakeup();

  • Required permissions

    None.

  • Support from

    0.10.49

getWakeupWord()

Use this method to get temi's wake word assistant.

  • Return

    Type Description
    String Wake-up word

  • Prototype

    String getWakeupWord();

  • Required permissions

    None.

  • Support from

    0.10.49

askQuestion()

Use this method to let temi actively speak to the user and wait for the user to answer.

  • Parameters

    Parameter Type Description
    question String The text to be spoken

  • Prototype

    void askQuestion(String question);

  • Required permissions

    None.

  • Support from

    0.10.63

  • Recommendation

    A custom dialog flow can be realized by cooperating with AsrListener. For details, please refer to Sample code.

finishConversation()

Use this method to finish the conversation (Stop recording for ASR).

  • Prototype

    void finishConversation();

  • Required permissions

    None.

  • Support from

    0.10.63

startDefaultNlu()

Use this method to trigger the system's default natural language understanding (NLU). If you want to directly trigger system skills such as weather or music in your skill, you can directly pass in "What's the wheather today" or "Play Music" as a parameter and invoke this method to achieve it.

  • Parameters

    Parameter Type Description
    text String Natural language text to be processed

  • Prototype

    void startDefaultNlu(String text);

  • Required permissions

    Selected Kiosk

  • Support from

    0.10.70

  • Note

    This interface can only be called once every 5 seconds.


Interfaces

TtsListener

Set your context to implement this listener and add the override method to get TTS status changes.

Prototype

package com.robotemi.sdk;

interface Robot.TtsListener {}

Abstract methods

  • Parameters

    Parameter Type Description
    ttsRequest TtsRequest TTS request object that holds its text content and status

  • Prototype

    void onTtsStatusChanged(TtsRequest ttsRequest);

Method for adding listener

  • Parameters

    Parameter Type Description
    listener TtsListener An instance of a class that implements this interface

  • Prototype

    void addTtsListener(TtsListener listener);

Method for removing listener

  • Parameters

    Parameter Type Description
    listener TtsListener An instance of a class that implements this interface

  • Prototype

    void removeTtsListener(TtsListener listener);

  • Support from

    0.10.36

WakeupWordListener

Set your context to implement this listener and add the override method to get wake word value when triggered by the user.

Prototype

package com.robotemi.sdk;

interface Robot.WakeupWordListener {}

Abstract methods

  • Parameters

    Parameter Type Description
    wakeupWord String The wakeup word used to trigger
    direction int
    • 0 - temi was triggered from the front
    • 90 - temi was triggered from the left
    • 180 - temi was triggered from the back
    • 270 - temi was triggered from the right

  • Prototype

    void onWakeupWord(String wakeupWord, int direction);

Method for adding listener

  • Parameters

    Parameter Type Description
    listener WakeupWordListener An instance of a class that implements this interface

  • Prototype

    void addWakeupWordListener(WakeupWordListener listener);

Method for removing listener

  • Parameters

    Parameter Type Description
    listener WakeupWordListener An instance of a class that implements this interface

  • Prototype

    void removeWakeupWordListener(WakeupWordListener listener);

  • Support from

    0.10.36

AsrListener

Set your context to implement this listener and add the override method to get the ASR result.

Prototype

package com.robotemi.sdk;

interface Robot.AsrListener {}

Abstract methods

  • Parameters

    Parameter Type Description
    asrResult String The ASR result

  • Prototype

    void onAsrResult(String asrResult);

Method for adding listener

  • Parameters

    Parameter Type Description
    listener AsrListener An instance of a class that implements this interface

  • Prototype

    void addAsrListener(AsrListener listener);

Method for removing listener

  • Parameters

    Parameter Type Description
    listener AsrListener An instance of a class that implements this interface

  • Prototype

    void removeAsrListener(AsrListener listener);

  • Support from

    0.10.53

ConversationViewAttachesListener

Set your context to implement this listener and add the override method to listen if the conversation view attaches.

Prototype

package com.robotemi.sdk;

interface Robot.ConversationViewAttachesListener {}

Abstract methods

  • Parameters

    Parameter Type Description
    isAttached boolean true means the conversation view attaches, false otherwise

  • Prototype

    void onConversationAttaches(boolean isAttached);

Method for adding listener

  • Parameters

    Parameter Type Description
    listener ConversationViewAttachesListener An instance of a class that implements this interface

  • Prototype

    void addConversationViewAttachesListenerListener(ConversationViewAttachesListener listener);

Method for removing listener

  • Parameters

    Parameter Type Description
    listener ConversationViewAttachesListener An instance of a class that implements this interface

  • Prototype

    void removeConversationViewAttachesListenerListener(ConversationViewAttachesListener listener);

  • Support from

    0.10.36

OnConversationStatusChangedListener

Set your context to implement this interface and override its' abstract method to listen to the status and text changes of the Conversation layer.

Note: Only the selected Kiosk App declared in the AndroidManifest.xml file to override the original conversation layer UI can receive the related callback data.

Prototype

package com.robotemi.sdk.listeners;

interface OnConversationStatusChangedListener {}

Static constants

All constants here are only for the status of Conversation layer.

Constant Type Value Description
IDLE int 0 Idle, no useriteraction
LISTENING int 1 Listening user's voice
THINKING int 2 Doing NLP
SPEAKING int 3 Playing TTS

Abstract methods

  • Parameters

    Parameter Type Description
    status int Status of Conversation layer
    text String Text of Conversation layer

  • Prototype

    void onConversationStatusChanged(int status, String text);

Method for adding listener

  • Parameters

    Parameter Type Description
    listener OnConversationStatusChangedListener An instance of a class that implements this interface

  • Prototype

    void addOnConversationStatusChangedListener(OnConversationStatusChangedListener listener);

Method for removing listener

  • Parameters

    Parameter Type Description
    listener OnConversationStatusChangedListener An instance of a class that implements this interface

  • Prototype

    void removeOnConversationStatusChangedListener(OnConversationStatusChangedListener listener);

  • Support from

    0.10.72

OnTtsVisualizerWaveFormDataChangedListener

Set your context to implements this interface and override its' abstract method to listen to the wave form data changes of the TTS audio visualizer.

Prototype

package com.robotemi.sdk.listeners;

interface OnTtsVisualizerWaveFormDataChangedListener {}

Abstract methods

  • Parameters

    Parameter Type Description
    waveForm byte[] Wave form data

  • Prototype

    void onTtsVisualizerWaveFormDataChanged(byte[] waveForm);

Method for adding listener

  • Parameters

    Parameter Type Description
    listener OnTtsVisualizerWaveFormDataChangedListener An instance of a class that implements this interface

  • Prototype

    void addOnTtsVisualizerWaveFormDataChangedListener(OnTtsVisualizerWaveFormDataChangedListener listener);

Method for removing listener

  • Parameters

    Parameter Type Description
    listener OnTtsVisualizerWaveFormDataChangedListener An instance of a class that implements this interface

  • Prototype

    void removeOnTtsVisualizerWaveFormDataChangedListener(OnTtsVisualizerWaveFormDataChangedListener listener);

  • Support from

    0.10.72

OnTtsVisualizerFftDataChangedListener

Set your context to implements this interface and override its' abstract method to listen to the fft data changes of the TTS audio visualizer.

Prototype

package com.robotemi.sdk.listeners;

interface OnTtsVisualizerFftDataChangedListener {}

Abstract methods

  • Parameters

    Parameter Type Description
    fft byte[] fft data

  • Prototype

    void OnTtsVisualizerFftDataChangedListener(byte[] fft);

Method for adding listener

  • Parameters

    Parameter Type Description
    listener OnTtsVisualizerFftDataChangedListener An instance of a class that implements this interface

  • Prototype

    void addOnTtsVisualizerFftDataChangedListener(OnTtsVisualizerFftDataChangedListener listener);

Method for removing listener

  • Parameters

    Parameter Type Description
    listener OnTtsVisualizerFftDataChangedListener An instance of a class that implements this interface

  • Prototype

    void removeOnTtsVisualizerFftDataChangedListener(OnTtsVisualizerFftDataChangedListener listener);

  • Support from

    0.10.72


Models

TtsRequest

Request object passed to temi, which contains all the information temi needs to in order to speak and for the skill to track its' request.

Prototype

package com.robotemi.sdk;

class TtsRequest {}

Subclass

  • Status

    • The status currently in use

      Status description
      STARTED Start playing
      COMPLETED Finish playing
      ERROR Errors occurred while playing
      NOT_ALLOWED Play is not allowed

    • Prototype

      enum Status {
    PENDING,
    PROCESSING,
    STARTED,
    COMPLETED,
    ERROR,
    NOT_ALLOWED
}

Attributes

Attribute Type Description
id UUID Unique number that identifies each tts request
speech String The text to be spoken
packageName String Skill package name so that temi knows who made the request
status Status Status of the request
isShowOnConversationLayer boolean Should the conversation line be shown when temi speaks the text. Note: Only relevant for 'Hey temi' assistant skills

Static methods

Create a TtsRequest object and pass it to speak(TtsRequest ttsRequest) method to play TTS.

  • Parameters

    Parameter Type Description
    speech String The text to be spoken
    isShowOnConversationLayer boolean true means the conversation view will appear when TTS is playing, false otherwise

  • Return

    Type Description
    TtsRequest TTS request object created by this method

  • Prototype

    static TtsRequest create(String speech, boolean isShowOnConversationLayer);

Override original voice flow

Override the NLP

Steps

  • Add the following <meta-data>s under the <application> element to AndroidManifest.xml file:

    <!-- Kiosk mode is required -->
<meta-data android:name="@string/metadata_kiosk" android:value="true" />

<meta-data android:name="@string/metadata_override_nlu" android:value="true" />

  • Listen to the ASR and access your own NLP service in its callback method.

  • Operate in Launcher: Settings > Kiosk > Select your skill. Or request to be the selected Kiosk App by method Robot.getInstance().requestToBeKioskApp().

Required permissions

Selected Kiosk

Support from

0.10.63

Override the ASR

Steps

  • Add the following <meta-data>s under the <application> element to AndroidManifest.xml file:

    <!-- Kiosk mode is required -->
<meta-data android:name="@string/metadata_kiosk" android:value="true" />

<meta-data android:name="@string/metadata_override_stt" android:value="true" />

  • Listen to the wake-up event and access your own ASR service in its callback method.

  • Operate in Launcher: Settings > Kiosk > Select your skill. Or request to be the selected Kiosk App by method Robot.getInstance().requestToBeKioskApp().

Required permissions

Selected Kiosk

Support from

0.10.70

Override the Conversation layer

Steps

  • Add the following <meta-data>s under the <application> element to AndroidManifest.xml file:

    <!-- Kiosk mode is required -->
<meta-data android:name="@string/metadata_kiosk" android:value="true" />

<meta-data android:name="@string/metadata_override_conversation_layer" android:value="true" />

  • Listen to onConversationStatusChangedListener and draw the UI according to the data(status, text) from its callback method.

  • Operate in Launcher Settings > App > Kiosk > Select your skill. Or request to be the selected Kiosk App by method Robot.getInstance().requestToBeKioskApp().

Required permissions

Selected Kiosk

Support from

0.10.72

General

Features

Advanced Features
temi GO

中文版

概要

功能

高级功能

temi GO 送餐送货机器人

Clone this wiki locally