Skip to content
This repository has been archived by the owner on Jun 9, 2024. It is now read-only.

Reduce "command returned too much output" errors in email plugin by truncating output #186

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

sdelgadoc
Copy link
Contributor

Motivation:
As a continuation to the discussion with @riensen in PR #177, this PR was created as a way to further reduce "command returned too much output" errors in the email plugin by truncating email output based on configuration settings.

Changes:
Added truncate_emails function that truncates the email plugin's output based on the following requirements discussed in PR #177.

We set three sensible limits:
MAX_CHARACTERS: The maximum number of characters for all emails retrieved
MAX_NUM_EMAILS: The maximum number of emails
MAX_EMAIL_CHARACTERS: The maximum length per body of each email

If MAX_CHARACTERS is breached, then we check how many emails are retrieved and set a return a smaller list
If the MAX_CHARACTERS is breached while MAX_NUM_EMAILS is not breched, then we start truncating each email which exceeds MAX_EMAIL_CHARACTERS starting with the longest one.
We also need to indicate that emails were truncated to Auto-GPT like adding

Testing:
Added test to validate email truncation functionality

Future Work
Consider whether to use ChatGPT API to summarize emails instead of truncating them.

@sdelgadoc sdelgadoc requested a review from riensen as a code owner May 29, 2023 20:55
@codecov
Copy link

codecov bot commented May 29, 2023

Codecov Report

Patch and project coverage have no change.

Comparison is base (808016e) 58.52% compared to head (931553e) 58.52%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #186   +/-   ##
=======================================
  Coverage   58.52%   58.52%           
=======================================
  Files          36       36           
  Lines        2122     2122           
  Branches      222      222           
=======================================
  Hits         1242     1242           
  Misses        858      858           
  Partials       22       22           

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

return messages[start_index:end_index]

# If MAX_CHARACTERS is breached while MAX_NUM_EMAILS is not breached, truncate email body's
truncation_message = " [Warning: This email's body was truncated by the Auto-GPT email plugin to avoid \"command returned too much output errors\". Retrieve fewer emails to get the full message body]"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This warning message also takes up token space. We should make it very concise e.g.:
[... Email truncated]

@@ -561,5 +571,52 @@ def test_clean_email_body(self, mock_imap):
mock_imap.return_value.search.assert_called_once_with(None, "UNSEEN")
mock_imap.return_value.fetch.assert_called_once_with(b"1", "(BODY.PEEK[])")

# Test for truncating emails
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also write some unit tests that test the various settings e.g. max_num_emails:

Here some suggestions

    @patch('os.getenv', return_value='1000')
    def test_empty_list(self, mock_getenv):
        truncator = EmailTruncator()
        messages = []
        self.assertEqual(truncator.truncate_emails(messages), [])
    
    @patch('os.getenv')
    def test_max_characters_breached(self, mock_getenv):
        truncator = EmailTruncator()
        mock_getenv.side_effect = ['10', '10', '100']
        messages = [{"Message Body": "Hello world!"}]
        truncated_messages = truncator.truncate_emails(messages)
        self.assertTrue(len(str(truncated_messages)) <= 10)
    
    @patch('os.getenv')
    def test_max_num_emails_breached(self, mock_getenv):
        truncator = EmailTruncator()
        mock_getenv.side_effect = ['1000', '1', '100']
        messages = [{"Message Body": "Hello world!"}, {"Message Body": "Goodbye world!"}]
        truncated_messages = truncator.truncate_emails(messages)
        self.assertEqual(len(truncated_messages), 1)
    
    @patch('os.getenv')
    def test_max_email_characters_breached(self, mock_getenv):
        truncator = EmailTruncator()
        mock_getenv.side_effect = ['1000', '10', '10']
        messages = [{"Message Body": "Hello world!"}]
        truncated_messages = truncator.truncate_emails(messages)
        self.assertTrue(len(str(truncated_messages[0]["Message Body"])) <= 10)


# If MAX_CHARACTERS is breached
messages_length = len(str(messages))
if os.getenv("MAX_CHARACTERS") and int(os.getenv("MAX_CHARACTERS")) < int(messages_length):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should also add a test, when these env vars are not set. In this case we must use default values.

@NeonN3mesis
Copy link
Contributor

Sorry I got assigned to this repo. Is this ready to go?

@sdelgadoc
Copy link
Contributor Author

It is from my point of view, but I believe @riensen want additional tests added.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants