Add pure local mode

[bluecoconut]

Pure local mode lets you use sketch without sending any of your data over the network

Do this by setting 3 environment variables

1. os.environ['SKETCH_USE_REMOTE_LAMBDAPROMPT'] = 'False' (this turns off talking to us)
2. os.environ['LAMBDAPROMPT_BACKEND'] = 'StarCoder' (this sets StarCoder model as your backend (rather than relying on OpenAI) (if you want to be sure its set, check lambdaprompt.backends.backends and you can call lambdaprompt.backends.set_backend('StarCoder')

• os.environ['HF_ACCESS_TOKEN'] = 'your_hugging_face_token' (This is necessary for pulling star-coder, which has an OpenRAIL license --> IF you use MPT as your backend, you will not need the HF_ACCESS_TOKEN)

Notes!

The first install is pretty slow (downloading full weights). I timed it at ~9 minutes

Second runs (eg in a fresh kernel, where the weights are downloaded) still take ~2 minutes for the first request.

But after, that, requests are pretty quick (on an A100 80GB) (and it should work on an A100 40 GB)

I think overall, the more I've been testing, I think StarCoder is actually weaker than OpenAI davinci, but it definitely still works

Here are some screenshots

(note it took 1.8s to complete this one) (Note, this HTML list + description took 7.4 seconds)

(when outputting a lot of code, grows to ~13 seconds)

Overall, these outputs are reasonable but I think when i pay close attention, they're not as good as GPT3 answers. But for a purely open, entirely local model, seems pretty great!