-
Notifications
You must be signed in to change notification settings - Fork 4.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve schema of CosmosDB chat history to handle long conversations #2312
Merged
+268
−154
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pamelafox
changed the title
WIP: Improve schema of CosmosDB chat history to handle long conversations
Improve schema of CosmosDB chat history to handle long conversations
Jan 28, 2025
mattgotteiner
approved these changes
Jan 28, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot reviewed 5 out of 14 changed files in this pull request and generated no comments.
Files not reviewed (9)
- app/backend/requirements.txt: Language not supported
- infra/main.bicep: Language not supported
- tests/snapshots/test_cosmosdb/test_chathistory_getitem/auth_public_documents_client0/result.json: Language not supported
- tests/snapshots/test_cosmosdb/test_chathistory_query/auth_public_documents_client0/result.json: Language not supported
- tests/snapshots/test_cosmosdb/test_chathistory_query_continuation/auth_public_documents_client0/result.json: Language not supported
- .pre-commit-config.yaml: Evaluated as low risk
- app/backend/chat_history/cosmosdb.py: Evaluated as low risk
- app/backend/config.py: Evaluated as low risk
- app/frontend/src/api/api.ts: Evaluated as low risk
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
The original approach to CosmosDB chat history was to store both the session information and all the chat messages inside the same document. However, we received feedback from the CosmosDB team that such an architecture is risky because a single session could have so many messages that it exceeds the 2MB document size limit. The safer approach is to store the sessions and messages separately, in one document each.
This PR migrates to that schema, and is based off the same schema used in this Cosmos DB sample:
https://github.com/AzureCosmosDB/cosmosdb-nosql-copilot
It should be just as performant, as we're using hierarchical partition keys (user_id/session_id) for all the operations.
However, this PR is a breaking change: if a developer already had this deployed with history and deployed the new version, users would lose their current history and start over (in a new container). If this is an issue for some folks, we could write a migration script to migrate over the data from the old container to the new one. I'm not sure how many folks are using history in production yet, since it's a fairly new feature.
Does this introduce a breaking change?
When developers merge from main and run the server, azd up, or azd deploy, will this produce an error?
If you're not sure, try it out on an old environment.
Does this require changes to learn.microsoft.com docs?
This repository is referenced by this tutorial
which includes deployment, settings and usage instructions. If text or screenshot need to change in the tutorial,
check the box below and notify the tutorial author. A Microsoft employee can do this for you if you're an external contributor.
Type of change
Code quality checklist
See CONTRIBUTING.md for more details.
python -m pytest
).python -m pytest --cov
to verify 100% coverage of added linespython -m mypy
to check for type errorsruff
andblack
manually on my code.