-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Deployments should be cached for non-model-information #102
Conversation
cf0345e
to
d9f2959
Compare
[...deployments].reverse().forEach(deployment => { | ||
cache.set(getCacheKey({ ...opts, model: extractModel(deployment) }), { | ||
entry: transformDeploymentForCache(deployment) | ||
const cacheEntries = [...deployments].map(deployment => |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MatKuhr how do you like that?
my 2 cents: I think it is semantically better, however more error prone.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, after multiple tries and still failing tests, I am reverting this. Feel free to check the commit, if you are interested: 37eaadd
This reverts commit 37eaadd.
.flatMap(entry => { | ||
const entries = [entry]; | ||
// if the entry has a model, also cache it without model and version | ||
if (entry.model) { | ||
entries.push({ id: entry.id }); | ||
} | ||
// if the entry has a model version, also cache it for without model version | ||
if (entry.model?.version) { | ||
entries.push({ id: entry.id, model: { name: entry.model.name } }); | ||
} | ||
return entries; | ||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This could still be shortened if we remove the checks:
.flatMap(entry => { | |
const entries = [entry]; | |
// if the entry has a model, also cache it without model and version | |
if (entry.model) { | |
entries.push({ id: entry.id }); | |
} | |
// if the entry has a model version, also cache it for without model version | |
if (entry.model?.version) { | |
entries.push({ id: entry.id, model: { name: entry.model.name } }); | |
} | |
return entries; | |
}); | |
.flatMap(entry => [entry, { id: entry.id }, { id: entry.id, model: { name: entry.model.name } }]); |
Even though this is a little less performant, I think I even prefer that, but no strong opinion.
# Conflicts: # packages/gen-ai-hub/src/utils/deployment-cache.ts
Context
AI/gen-ai-hub-sdk-js-backlog#78.
Follow up from #96.
When retrieving a deployment for a model name without version and there are only deployments where each model has a version, the second time we use this deployment, the cache wasn't used.
Definition of Done