refactor: groundwork - add pkg/concurrency and the associated test file

[dave-gray101]

This small PR lays the groundwork for some future PRs of my refactor to cut free the backend management code from the http api. pkg/concurrency is intended to allow easy communication of asynchronous (Request, Response, Error) tuples throughout the codebase. The intended usecase here is that a developer creating an asynchronous backend service can easily:

1. Receive a request parameter for the function
2. Pass that to NewJobResult to recieve a matching pair of job result structures.
3. Immediately start a goroutine with the WritableJobResult as its parameter, performing the asynchronous work and eventually calling WritableJobResult.SetResult when the work is finished
4. Immediately return the JobResult from the function

The caller of this can proceed along until the result is actually required - JobResult.Wait() returns both the relevant Result and any applicable Error.

Rather than include this as a part of #2698 as I've previously done, this slim PR allows the basic structure can be evaluated on its own merits, and the unit tests can run before any code actually depends on it.

Golang channels support incremental / asynchronous work in many ways. This design was chosen in order to present something simple for the common usecase we have of a single request / single response (ie: not streaming mode token responses). For Streaming Text, I expect to either just handle bare channels or create a distinct structure pair if some abstraction is desirable.

[netlify]

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	66012c1
🔍 Latest deploy log	https://app.netlify.com/sites/localai/deploys/66998a01c064780008284d5f
😎 Deploy Preview	https://deploy-preview-2745--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[mudler]

This is just a style nitpick, so feel free to ignore.

I have to say the pattern is a bit confusing to me, I would expect something like (pseudo-code ahead):

asyncJob := func (wjr *WritableJobResult[string, string]) {
  // ... do something to write the job result
} 
futureResult := NewAsyncOperation(asyncJob)
result := futureResult.Wait()

[mudler]

we might also actually want even a ctx to pass down there to be sure we can short-circuits Waits call as well, but that is not a biggie as can be handled in the async func

type AsyncJobResult[T] *WritableJobResult[T, T]
asyncJob := func (ajr AsyncJobResult) {
  // ... do something to write the job result
} 
futureResult := NewAsyncOperation(asyncJob, opts ...)
result := futureResult.Wait(ctx)

[dave-gray101]

context on wait is a really good idea, I'll add that in

[dave-gray101]

I cleaned up the style in these tests a bit - there's no need to use an explicitly distinct "reader" goroutine.
I also added in a context parameter, to make things a lot more flexible. Now, with context.Background() things work as before, blocking for all time until a request arrives - but by adding a timeout here, the reader can indicate when to break that wait lock.

Notably, I didn't interpret canceling the read-only side Wait() call to indicate that the actual underlying task should be canceled - I'd assume that would be on the writable side. The test I've added that exercises contexts should validate that multiple readers play nice and allow early dropouts.

[dave-gray101]

@mudler turns out pkg/downloader was skating by without actually running its tests... when I noticed I forgot the suite test file here, I added it there as well.

[localai-bot]

This PR introduces a new package called concurrency with associated tests, as well as modifies the downloader package by adding a test suite.

In the concurrency package:

• A new structure called JobResult is introduced, which is designed to contain the result of an asynchronous action. It has three fields: request, result, and err. The request field holds the original request, while result and err represent the result and error, respectively, of the action. The structure also includes a once field to ensure that the result and err are only set once, and a done channel to signal when the result is ready.
• A WritableJobResult struct is also introduced, which extends the JobResult struct. It has a method called SetResult that allows updating the result and err fields of the associated JobResult struct.
• The NewJobResult function creates a pair of JobResult and WritableJobResult structs, initialized with a given request.
• The Wait method of the JobResult struct waits for the result to be ready (i.e., the done channel is closed) or until the context expires. It returns the result or nil and the associated error.

In the downloader package:

• A new test suite called Downloader test suite is introduced.

The changes in this PR appear to be the initial implementation of the concurrency package and its testing. The downloader package now has a test suite, but the test files themselves are empty. The purpose of these changes is likely to provide a framework for managing asynchronous actions and their results, which may be useful in other parts of the codebase. The downloader package may be using this framework to test its functionality.

In summary, this PR introduces a new package for managing asynchronous actions and their results, as well as modifies an existing package to include tests for its functionality.