Skip to content
This repository has been archived by the owner on Aug 30, 2022. It is now read-only.

Limit Messages per Conversation #281

Closed
westei opened this issue Sep 17, 2018 · 3 comments
Closed

Limit Messages per Conversation #281

westei opened this issue Sep 17, 2018 · 3 comments
Milestone

Comments

@westei
Copy link
Member

westei commented Sep 17, 2018

The current storage model of conversations require to limit the number of messages per conversation to ensure respecting the document size limit of MongoDB of 16 megabytes.

Assuming ~200 chars average per message and ~200 chars required for additional information a limit of 5000 messages seams to be a reasonable limit.

@westei
Copy link
Member Author

westei commented Sep 17, 2018

Just as explanation:

The Smarti conversation was meant to represent a section within a communication channel. This lead to the decision to store the messages with the conversation.

However when integrations do establish a 1:1 mapping between communication channels and Smarti conversations their is a risk that the amount of messages in those channels do overrun the storage limit of MongoDB.
To avoid this situations this issue will limit the number of messages/conversation and only store the 5000 most recent messages in Smarti

Depending on the goals future versions might change the storage model to directly represent Channels and their messages. With messages stored as single documents in an own collections. Conversations would than be an additional class that just refer their the channel and the messages they contain

westei added a commit that referenced this issue Sep 20, 2018
* The limit is enforced on 2 places
    1. with the `ConversationRepoListener`: required when a Conversation with more as the maximum allowed number of conversations is directly added to the Repository
    2. with a `$slice` command in the update of `appendMessage(..)` of the `ConversationRepository`
* The limit is configured by `smarti.storage.mongodb.maxConvMsg` the default is `5000`
    * configuration changes are applied only on the next update to an conversation. This is sufficient as this limit is intended to prevent Mongo Documents getting over the limit of 16MByte
* Added a UnitTest for the MongoRepository that validates this feature. For testing the limit is set to `50` messages
@westei
Copy link
Member Author

westei commented Sep 20, 2018

The number of messages per conversation is configureable via smarti.storage.mongodb.maxConvMsg. The default is set to 5000

@westei westei closed this as completed Mar 18, 2019
@ghost ghost removed the in review label Mar 18, 2019
@westei
Copy link
Member Author

westei commented Mar 18, 2019

NOTE: This feature is also useful to avoid the memory issue while indexing public channels with many messages (see #299 and #302)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant