Skip to content

A set of simple tools to speed up the prompt engineering iteration cycle

License

Notifications You must be signed in to change notification settings

bjsi/prompt-iteration-assistant

Repository files navigation

Prompt Iteration Assistant

A set of simple tools to accelerate the prompt engineering iteration cycle.

⚠️ Work in Progress ⚠️

Motivation

I work full time as an AI Engineer at RemNote where I am building the RemNote Flashcard Copilot. On the side I'm working with OpenPipe to build Open Recommender, an open source LLM-powered YouTube video recommendation system.

A huge portion of my time in both of these projects goes towards prompt engineering, iteration and evaluation. I realised recently that while each step in the prompt iteration cycle is very simple, there aren't any tools that support it in the way I want, so the process ends up feeling quite frustrating. Prompt Iteration Assistant is my attempt to build a set of simple tools to make prompt engineering 10x easier and faster.

Features

  • Simple Prompt class abstraction.
  • Methods to create promptfoo tests with minimal boilerplate.
  • Easily register prompts as CLI scripts.
  • CLI interface to quickly run tests or try out a prompt with new input.
  • zod integration for runtime type safety.
  • built in dialogs and prompts to help with the entire prompt engineering pipeline
    • creating prompt instructions
    • creating prompt input and output zod schemas
    • creating in-context examples
    • elaborating on examples
    • quickly run prompts to generate text inside the prompt you are editing, eg. to create example data
    • brainstorming possible inputs and edgecases, creating tests
  • easily build your own dialogs
  • easily extend for your purposes
  • no janky, unpolished UI, just good old fashioned CLI tools

How to Use

  • npm i prompt-iteration-assistant.
  • See the src/examples folder or get in touch with me if you get stuck!

About

A set of simple tools to speed up the prompt engineering iteration cycle

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages