LAMatHome helps you expand the functionality of your rabbit r1.
By using your hole.rabbit.tech
account token, we can directly fetch journal entries from the API. By doing this, we can efficiently grab your latest journal entry.
By providing the user's utterance to llama3-70b-8192
via the Groq API we can determine:
- If the user is talking to r1, or if it is a command meant for LAMatHome
- Which integration to call, and what parameters to give it
By doing this, we avoid the user having to learn every command for every integration, and enables natural language commands. It also enables creativity for the LLM to craft texts for you, if you're into automating social interaction.
Examples:
User utterance in journal entry | LLM "rigid command" Output |
---|---|
"Can you text my roommate Justin on telegram, yelling in all caps, that his music is too loud? Add a few extra exclamation points." | Telegram Justin YOUR MUSIC IS TOO LOUD!!! |
"Can you turn the volume on my computer up to 100 to spite my roommate?" | Computer Volume 100 |
"What time does the moon come out today?" | x * |
*x
is the output when the LLM decides your intention was to talk to r1, or, that it didn't have enough information. e.g. Name of recipient, platform to send on, etc.
Then, llm_parse.py
takes your neatly formatted command, and executes it based on which integration is called, and the parameters.
Below is a list of our current integrations. This list is kept up-to-date.
Name | Category | Description | Example prompt | |
---|---|---|---|---|
Site | Browser | Opens/Searches in any website. | Open the _____ website |
|
Browser | Searches Google. | Search google on my computer for ______ |
||
YouTube | Browser | Searches YouTube. | Open a YouTube search for ______ on my computer |
|
Gmail | Browser | Searches Gmail. | Search my emails on my computer for ______ |
|
Amazon | Browser | Searches Amazon. | Search amazon on computer: ______ |
|
β | Run | Local Actions | Presses Windows key, searches for an app, and runs. | Open up the chat app for gamers on my computer |
β | Volume | Local Actions | Sets volume, turns up/down, and mutes/unmutes. | Change the volume on my pc to 50 |
Media | Local Actions | Skips media next/back, pause/unpause. | Pause on my pc , Skip twice backwards on my computer |
|
β | Power | Local Actions | Power options (Lock/Sleep/Restart/Shutdown) | Shutdown my PC , Please lock my computer |
Google Home | Local Actions | Activates Google Home automations. | Turn on my desk lamp , Use google home to turn on my lamp, but I forgot what it's called |
|
Open Interpreter | Local Actions | Send commands to Open Interpreter | Tell Open Interpreter to open the blender file on my desktop. |
|
LAMatHome | Local Actions | Only integration currently is "terminate", which closes LAH. | That's enough from you. Close LAM at home. |
|
Discord | Messaging | Sends a message on Discord to a specified person/channel. | Text poke on discord asking when he's going to be back online. Wait, no ask him on telegram. Actually no, discord is good. |
|
FB Messenger | Messaging | Sends a message on FB Messenger to a specified person. | Ask Justin what he thinks of my new sunglasses. Oh, send that on facebook. |
|
Telegram | Messaging | Sends a message on Telegram to a specified person. | Message Kevin on telegram asking him when he's gonna PR his new feature |
Note
Integrations marked with a β may need to be saved as a note to work. r1 could interpret these as commands to itself, and not save the query to the rabbithole. This is a result of r1 update 20240603.15/0.8.99(134D8DE) that enabled voice settings.
Integrations marked with a
1. Clone the repository and CD into it
git clone https://github.com/dot-justin/LAMatHome
cd LAMatHome
2. Install dependencies
pip install -r requirements.txt
playwright install
3. Obtaining your user token from the rabbithole:
-
Google Chrome
- Log into the rabbit hole from the link above
- Press F12 to bring up the developer console. If this doesn't work, right-click the page, and click inspect.
- Expand the developer console for better viewing
- Click the
Network
tab in the top navigation bar. - Press Ctrl + R to reload the page.
- Near the bottom of the middle pane, find and select
fetchUserJournal
. - In the new pane that opened, select
Payload
in the top navigation bar. - Select everything inside the quotes after
accessToken
. This is your user token.
-
Firefox
- Log into the rabbit hole from the link above
- Press F12 to bring up the developer console. If this doesn't work, right-click the page, and click inspect.
- Expand the developer console for better viewing
- Click the
Network
tab in the top navigation bar. - Press Ctrl + R to reload the page.
- In the
Network
tab, in theFile
column, find and selectfetchUserJournal
. - In the new sidebar that opened up, select
Request
in the top navigation bar. - Select everything inside the quotes after
accessToken
. This is your user token.
Note: Your token will expire 24 hours after you log in. This is out of our control but we are working on a better way.
4. Obtaining an API key from Groq:
- Go to console.groq.com/login
- Create an account.
- Once logged in, go to console.groq.com/keys
- Click
Create API Key
- Create a name for your API key. This can be anything you want. Click
Submit
. - Now, a window will pop up containing your API key. Click on
Copy
. After this point, you will not be able to access this key again from the Groq website. (Don't worry too much, you can create and delete keys at any time.) - Back in the ui that popped up earlier, enter your API key in the field for
Groq API Key:
Warning
NEVER share your user tokens or API keys. They will be stored locally in the program and will only be used to authenticate with official API's.
5. Running LAMatHome and entering your credentials:
py main.py
The first time you run LAMatHome, it will bring up a ui to enter your credentials. For any service that you want to use, enter your credentials. The only required values here are:
- Rabbit Hole Access Token
- GROQ API Key
After you enter these, and hit submit, you should be all set up and ready to go! Remember, you can edit your credentials any time by editing the .env
file inside the root directory of the project.
When you want to run LAMatHome, do the following:
Run main.py
from the root directory
py main.py
This will start the LAMatHome program, and (as long as your token is valid) you can start giving prompts to r1.
To stop LAMatHome, you have two options.
-
Via r1 voice prompt (If you can't get this to work, try method 2)
- Say
Please close/quit/terminate LAMatHome
.
- Say
-
Manually
- Find the command prompt where it's currently running from.
- Press
Ctrl + C
to exit the program. You may see some errors, this is expected and will not mess up your installation.
Open the config.json
file in the root directory of your project. (Use your favorite IDE. A free, lightweight one is notepad++)
mode
:
- The only options here are
rabbit
andcli
.rabbit
mode will listen to the rabbithole api for journal entries, whilecli
mode will turn LAMatHome into a dumb OpenInterpreter.
rabbithole_api_max_retry
:
- This determines how many times LAMatHome will try to connect after failure.
rabbithole_api_sleep_time
:
- This determines how many seconds LAMatHome will wait between refreshes.
rolling_transcript_size
:
- This determines how many of your past prompts will get passed to llm_parse. The higher the number, the more "memory" the LLM has.
If you don't want to use specific integration, no worries!
Find the integration you want to disable. Set each value to false
, and they will no longer be activated by llm_parse.
Examples:
"browsergoogle_isenabled": false,
TheGoogle
function of theBrowser
integration is disabled."computervolume_isenabled": true,
TheVolume
function of theComputer
integration is enabled."browser_isenabled": false,
= All of the browser functions are disabled, even if the child functions are set totrue
.
To get Telegram running and sending texts on your behalf, some setup is required. Follow this guide:
- Give LAMatHome any Telegram command. This can be a test, but be aware that the text will be sent after you log in.
- A Firefox Nightly window (Playwright instance) will open up and request your sign-in.
- Sign in, and watch the text go through. After this, your session is saved and you won't need to log back in for a long while.
To allow LAMatHome to use your Google Home, you need to follow these steps:
- Add your
G_HOME_EMAIL
andG_HOME_PASS
to your.env
file. - Create any Google Home automations that you want to run, using the Google Home mobile app.
- Example names that work well with LAMatHome:
Lamp on
,Lamp off
(one for on and off respectively, because there is no toggle option.),Goodnight
, etc. llm_parse needs to be able to link the user utterance to an automation.
- Example names that work well with LAMatHome:
- Add the verbatim Automation names to config.json, on the
googlehomeautomations
line. There are some examples there, just follow the pattern to add more."googlehomeautomations": ["Automation 1", "Automation 2", "Automation 3"],
- You should be set! This list that you just configured will be passed to
/utils/llm_parse.py
and if determined to be run, will run! Ask r1/LAMatHome to "Turn on the lamp in my room" or "Turn off my tv".
Error | Meaning/Fix |
---|---|
500 Server Error: Internal Server Error for url: https://hole.rabbit.tech/APIs/fetchUserJournal |
This means that for some reason, the request to the rabbit API failed. More often than not, this means that your token has expired. While this can happen if your token is valid, it is very rare. Instructions to obtain user token |
playwright._impl._errors.TimeoutError: Page.click: Timeout 30000ms exceeded. |
This means that, in waiting for a certain object on the website to load, it never loaded, and Playwright timed out. If this ever happens, please open a bug issue. |
Use case | Prompt |
---|---|
Get r1 to just say "ok" in response to commands: | From now on, whenever I give you a command and you think that you can't do it, just reply with okay. Don't reply with any other acknowledgment or any other warning or message, just reply with okay if you cannot do the command. Say "ok" if you understand. |
- Thanks to poke for the original idea rabbitWrighter
- Obligatory "There's no way you're that young" rabbitt