Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Discussion] Validation fails when an output is not provided #13

Open
jfaldanam opened this issue Dec 13, 2023 · 0 comments
Open

[Discussion] Validation fails when an output is not provided #13

jfaldanam opened this issue Dec 13, 2023 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@jfaldanam
Copy link
Member

Example:
A function executes a query to a database, this query may return no response if there are no rows that matches the filters.

As such, by a naive implementation the result is an empty tuple, instead of the expected n outputs, which raises a validation error.

{"status": {"code": 500, "message": "Error: function return malformed results.\nNumber of output variables (0) does not match number of variables in the schema (7)"}, "data": null}

There are several ways to approach this issue, the first of which is:

  1. Is this an issue? The schema of the function defines n outputs, so if they are missing the output is invalid.

    1. This pushes the problem to the function implementation.
  2. Add a way to mark (and later validate) some output types as optional, as we do with inputs.

    1. This increases the complexity of the validation and the configuration json of the function.
  3. Deal with the with user facing warnings or errors. As the main goal of eidos is to be used to execute AI functions for an LLMs or future AI arquitectures, we can provide function implementers with a common set of Exceptions to define user facing warnings that the LLM can then actuate on.

    1. For example, in the previous example if we send the LLM a response explaining that there is no data on the database to answer the query proposed by the user, it allows the LLM to guide the conversation on how to make the query more relaxed or at least inform the user that there is no relevant data.
    2. The issue of this approach is that it forces functions to import a eidos-defined sdk to identify said exceptions, or use a predefine error, that is later parsed. But then we can enforce the output to always be correct unless a error has been raised.
@jfaldanam jfaldanam added the bug Something isn't working label Dec 13, 2023
@jfaldanam jfaldanam self-assigned this Dec 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant