Skip to content

Commit

Permalink
[PC-1828] Speech recognition tutorial check and revamp (#2151)
Browse files Browse the repository at this point in the history 
* Nicla Vision Added to requirements

* chris modification

* finished
  • Loading branch information
@mcmchris
mcmchris authored Sep 9, 2024
1 parent b3793b5 commit 115cdf7
Showing 1 changed file with 132 additions and 9 deletions.
141 changes: 132 additions & 9 deletions ...o/shields/portenta-vision-shield/tutorials/speech-recognition-engine/content.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ In case you are interested in unlocking the full potential of the tool, the tuto
To use the Arduino Speech Recognition Engine, you will need one of the following boards:

- [Arduino Portenta H7 (any variant)](https://store.arduino.cc/portenta-h7) + Portenta Vision Shield ([LoRa](https://store.arduino.cc/products/arduino-portenta-vision-shield-lora%C2%AE) or [Ethernet](https://store.arduino.cc/products/arduino-portenta-vision-shield-ethernet))
- [Arduino Nicla Vision](https://store.arduino.cc/products/nicla-vision)
- [Arduino Nano 33 BLE Sense Rev 1](https://store.arduino.cc/products/arduino-nano-33-ble-sense)
- [Arduino Nano 33 BLE Sense Rev 2](https://store.arduino.cc/products/nano-33-ble-sense-rev2)
- [Arduino Nano RP2040](https://store.arduino.cc/products/arduino-nano-rp2040-connect)
Expand Down Expand Up @@ -133,7 +134,7 @@ Once everything is ready, click on the **submit** button to get your license, it
};
```
**Remember to replace the g_lpdwLicense values with your license ones.**
***Remember to replace the `g_lpdwLicense` values with your license ones.***
Now switch back to the `VoiceRecognition` tab and upload the sketch.
Expand All @@ -149,7 +150,7 @@ Please note that the sketch and license that you are using are a "demo" version
### Customized Commands
To expand the features and human interaction of your projects, you can create your own voice commands with the Arduino Speech Recognition Engine. In this section, you will learn about how to use the _Cyberon Model Configuration_ webpage to create a new project with custom voice commands.
To expand the features and human interaction of your projects, you can create your own voice commands with the Arduino Speech Recognition Engine. In this section, you will learn about how to use the [Cyberon Model Configuration](https://tool.cyberon.com.tw/ArduinoDSpotterAuth/CTMain.php) webpage to create a new project with custom voice commands.
***These steps are compatible with the trial license, used in this tutorial, and the paid license***
Expand All @@ -169,18 +170,18 @@ Click next, you will see a new page to:
***Warning: each project is bound to a single Arduino board with the declared Serial Number. If you would like to use the Arduino Speech Recognition Engine on another Arduino board, you need to create a new project from scratch and assign it to the new Serial Number.***
To create a new project first you need to select the desired language for the speech recognition. Once is set, click **create**.
To create a new project first you need to select the desired language for the speech recognition. Once is set, click **Create**.
![Cyberon, New Project](assets/newProject.png)
Now you need to configure the following steps to create a new trigger:
* Create the **Input Trigger word**, for example "Hey Arduino".
The **Input Trigger word** will trigger the device, to let the board know that you are going to say a command after that.
![Cyberon Adding the Input trigger](assets/newProjectTrigger.png)
The **Input Trigger word** will trigger the device to let the board know that you are going to say a command after that.
![Cyberon Adding the Input trigger](assets/newProjectTrigger.png)
* Add the **Command** list.
These commands will be used to do tasks on your sketch. For example, if you have a command which is "Play", later you will be able to get that command and proceed with some job inside the sketch easily.
![Cyberon Adding the Command list](assets/newProjectCommands.png)
These commands will be used to do tasks on your sketch. For example, if you have a command which is `share`, later you will be able to get that command and proceed with some job inside the sketch easily.
![Cyberon Adding the Command list](assets/newProjectCommands.png)
* The next step is just to confirm that the data written is correct. Click on **Next** to finish.
On the next page you will see all the configurations already set. Check it out to confirm that everything is correct. In case something is wrong, click on **Back** to fix it.
Expand All @@ -201,9 +202,9 @@ On your sketch directory, paste the files you have got in your e-mail inbox befo
* `CybLicense_<id>.h`
* `Model_<id>.h`
Now to implement the **Input Trigger Command** and the **Command List** open the sketch and change the following `#include`
Now to implement the **Input Trigger Command** and the **Command List** open the sketch and change the following headers with the downloaded ones:
```cpp
```arduino
...
#include "CybLicense.h" -> #include "CybLicense_<id>.h"
Expand All @@ -220,6 +221,128 @@ Now to implement the **Input Trigger Command** and the **Command List** open the

At this point, the project is set to be used. Upload the sketch and open the Serial Monitor. Now you can test your new **Input Trigger Word** and the **Command** list that you have created. Pronounce the new trigger words out loud, you will see the recognized words on the **Serial Monitor**.

#### Modify the Example Codes

If you want to execute custom actions with the **trigger** and **commands** you defined, you can do it inside the `VRCallback` function.

First you need to know your trigger and commands IDs. You can find them listed in your Serial Monitor, in this case are the following:

| **Keyword** | **ID** |
| :---------: | :----: |
| Hey Arduino | 100 |
| read | 10000 |
| upload | 10001 |
| share | 10002 |

Here is the `VRCallback` function modified so you can easily give functionality to the commands detection:

```arduino
// Callback function for VR engine
void VRCallback(int nFlag, int nID, int nScore, int nSG, int nEnergy)
{
if (nFlag==DSpotterSDKHL::InitSuccess)
{
//ToDo
}
else if (nFlag==DSpotterSDKHL::GetResult)
{
/*
When getting an recognition result,
the following index and scores are also return to the VRCallback function:
nID The result command id
nScore nScore is used to evaluate how good or bad the result is.
The higher the score, the more similar the voice and the result command are.
nSG nSG is the gap between the voice and non-command (Silence/Garbage) models.
The higher the score, the less similar the voice and non-command (Silence/Garbage) models are.
nEnergy nEnergy is the voice energy level.
The higher the score, the louder the voice.
*/
//ToDo
switch(nID)
{
case 100:
Serial.println("The Trigger was heard");
// Add your own code here
break;
case 10000:
Serial.println("The Command -read- was heard");
// Add your own code here
break;
case 10001:
Serial.println("The Command -upload- was heard");
// Add your own code here
break;
case 10002:
Serial.println("The Command -share- was heard");
// Add your own code here
break;
default:
break;
}
}
else if (nFlag==DSpotterSDKHL::ChangeStage)
{
switch(nID)
{
case DSpotterSDKHL::TriggerStage:
LED_RGB_Off();
LED_BUILTIN_Off();
break;
case DSpotterSDKHL::CommandStage:
LED_BUILTIN_On();
break;
default:
break;
}
}
else if (nFlag==DSpotterSDKHL::GetError)
{
if (nID == DSpotterSDKHL::LicenseFailed)
{
//Serial.print("DSpotter license failed! The serial number of your device is ");
//Serial.println(DSpotterSDKHL::GetSerialNumber());
}
g_oDSpotterSDKHL.Release();
while(1);//hang loop
}
else if (nFlag == DSpotterSDKHL::LostRecordFrame)
{
//ToDo
}
}
```
We added a `switch...case` that evaluates the `nID` variable and compares it with the different IDs.
Inside each case add your custom code to be executed with the trigger or commands.

```arduino
...
switch(nID)
{
case 100:
Serial.println("The Trigger was heard");
// Add your own code here
break;
case 10000:
Serial.println("The Command -read- was heard");
// Add your own code here
break;
case 10001:
Serial.println("The Command -upload- was heard");
// Add your own code here
break;
case 10002:
Serial.println("The Command -share- was heard");
// Add your own code here
break;
default:
break;
}
...
```


#### Unlock Limitations (License)

In case you want to further customize your model or add additional commands or trigger words, we have the right licenses for you:
Expand Down

0 comments on commit 115cdf7

Please sign in to comment.