Tony

Build your own butler.
Uses speech recognition to control your device.
Press the button and talk with him.

• Flask
• xdotool
• python 3.x
• rasa NLU (TODO)
• Watson Speech-to-Text API

Setup

Currently only running under Linux, tested with Python 3.4.
You need xdotool to run the scripts under scripts/.

$ sudo apt-get install xdotool
$ virtualenv env
$ source env/bin/activate
$ pip install -r requirements.txt

Run

To start Tony:

$ python app.py
# visit <your-ip>:8080 and talk to Tony

Add your own actions

Use actions.py to define new commands.
There are some examples to control Spotify,
you have to modify them (update x and y values for the mouse clicks)
to make them work for you.

Look into the scripts/ directory to see how it works.
xdotool manual.
Don't use --sync, otherwise xdotool halts the whole process if an error occurs.

TODO: Use rasa NLU to parse complex commands.

There are two types of actions:

• pattern matching commands (words/short sentences with optional parameters):
  e.g. "next" to play the next song or "play music rick astley"
• complex commands: using natural language understanding (TODO)

Caution!

This system is not safe. It's passing user input to the shell.
So please just use it in safe private networks.