Skip to content

Commit

Permalink
chore: ..
Browse files Browse the repository at this point in the history
  • Loading branch information
zeeshanlakhani committed May 1, 2024
1 parent 76d73ab commit 8693849
Showing 1 changed file with 7 additions and 0 deletions.
7 changes: 7 additions & 0 deletions example-llm-workflows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -438,6 +438,12 @@ This paradigm shift towards local-first approaches resonates deeply within ongoi

## Outputs and Discussion

How does a model with 8 billion parameters running on a laptop compare to a similar set of prompts rendering output on ChatGPT 3.5 for example?

I took our sequential chain output from above ([`chain1.json`](./chain1.json)) and evaluated it in in [Grammarly's readability score indicator][grammarly-score], which calcualtes the accuracy level of the document based on total word count, and the number and type of writing issues detected.

Our demo AI-chain workflow using Homestar and LLaMA rates out at a 94, which ChatGPT 3.5 rates out at a 91. This is by no means a fully qualified study, but showcases that local, smaller LLMs can work well enough for targeted use cases.

## What is Homestar in a Nutshell?

[Homestar][homestar] is a runtime engine for executing WebAssembly (Wasm) based workflows. It implements the [Interplanetary Virtual Machine (IPVM)][ipvm] in-progress protocol specification centered around workflows and invocations. These workflows contain content-addressed tasks—which means they're uniquely identified by their content rather than by their location—composed of [Wasm components][wasm-components] with runnable functions that can be scheduled and executed by any Homestar peer throughout the network. Pure Wasm Workflows can be executed deterministically while side-effecting operations like random and HTTP network requests are explicitly managed.
Expand All @@ -464,6 +470,7 @@ For this project, I forked and extended the Homestar codebase (of which I'm a le
[gemini]: https://arxiv.org/pdf/2312.11805
[gdpr]: https://gdpr-info.eu/
[gguf]: https://github.com/ggerganov/ggml/blob/master/docs/gguf.md
[grammarly-score]: https://support.grammarly.com/hc/en-us/articles/360007144751-What-is-Performance-and-how-is-it-calculated
[guest]: https://wasmbyexample.dev/examples/wasi-introduction/wasi-introduction.all.en-us.html
[homestar]: https://github.com/ipvm-wg/homestar
[host]:https://wasmbyexample.dev/examples/wasi-introduction/wasi-introduction.all.en-us.html
Expand Down

0 comments on commit 8693849

Please sign in to comment.