382nd Commit

README.md

@@ -180,6 +180,7 @@ This seed repository provides the following features:
 - [x] [Background Workers](./app/src/jobs) (Or use [Redis Store](./app/src/routes/queues/+handler.ts))
 - [x] [Cron Jobs](./app/src/jobs) (Or use [Redis Store](./app/src/routes/queues/cron/+handler.ts))
 - [x] [Delayed jobs](./app/src/jobs) (Or use [Redis Store](./app/src/routes/queues/delay/+handler.ts))
+- [x] [Streaming LLM Output](./app/src/routes/sse/model/+handler.ts)
 
 ## Configuration
 

app/src/routes/sse/model/+handler.ts

@@ -14,7 +14,13 @@ export default (async (app) => {
       },
     },
     async (request, reply) => {
-      const model = useModel({ model: 'gpt-4o-mini', temperature: 0 });
+      const model = useModel({
+        // Replace the model with a fine-tuned model or an embedding model
+        model: 'gpt-4o-mini',
+
+        // Allow some creative flexibility to handle variations in phrasing, while maintaining accuracy in responses
+        temperature: 0.3,
+      });
 
       const stream = await model.stream(request.body.message);