ServiceNow flow action and subflow that can be used to test together.ai models.
- https://together.ai
- Sign in with Github
- Get your API key - https://api.together.xyz/settings/api-keys
Together.ai provides $5.00 worth of free credits for each account, and I've found with their optimizations discussed here, this is a decent amount of credits to get started with.
- Import from source control in Studio.
- Open Together.ai app in Studio and select the system property.
- Put in your Together AI API key and save it.
- Open the subflow and click test.
- Enter a model and prompt. Note: The list of models can be found at https://api.together.xyz/models and you will need to enter the full model string including author and model name. (The default in subflow is meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo)
Note: Embedding and Image generation model types are currently not supported. Chat models are supported by default. If you need to test a language model, please update the subflow's resource path input to '/v1/completions'
- Have fun 😁!
Since ServiceNow's Now Assist product is unavailable in PDIs, this allows for testing of GenAI responses using together.ai's library of serverless endpoint models. The subflow could even be modified and utilized in a production Flow if together.ai is allowed in your organization.
- Ability to support image generation / storage of the returned image in ServiceNow.
- TBD