Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return highest scoring response to the API #124

Closed
Tracked by #33
grantdfoster opened this issue Jun 27, 2024 · 3 comments · Fixed by #196
Closed
Tracked by #33

Return highest scoring response to the API #124

grantdfoster opened this issue Jun 27, 2024 · 3 comments · Fixed by #196
Assignees
Labels

Comments

@grantdfoster
Copy link
Contributor

grantdfoster commented Jun 27, 2024

From an API perspective, we want to return what the validator judges as the "best" response, or highest scoring, not a list of sanitized responses based on pre-defined types.

@juanmanso
Copy link
Contributor

This should be fixed by changing the return object from the list to the first element here right?

        print("FINAL RESPONSES ------------------------------------------------")
        print(response_with_uids)
        
-        return response_with_uids
+        return response_with_uids[0]

(Aside from the fact that the return type should be changed as well)

@juanmanso
Copy link
Contributor

This should be fixed by changing the return object from the list to the first element here right?

    print("FINAL RESPONSES ------------------------------------------------")
    print(response_with_uids)
  •    return response_with_uids
    
  •    return response_with_uids[0]
    

(Aside from the fact that the return type should be changed as well)

Isn't this blocked also by masa-bittensor/issues#197? Or could we fix it and then have the scoring be worked on in parallel 🤔 (I think we can but just checking here)

@mudler
Copy link
Contributor

mudler commented Aug 5, 2024

this is low hanging fruit, and what we could actually do is mutate the behavior such having a debug flag that returns all the scores, and when in product mode, returns only the highest score.

Outcome from planning: even better to have a separate endpoint and avoiding any user option, or maybe a parameter to the API endpoint

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants