Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Tool Execution Order, Enhance Logging, and Update Documentation #520

Merged
merged 1 commit into from
Dec 12, 2023

Conversation

mrT23
Copy link
Collaborator

@mrT23 mrT23 commented Dec 12, 2023

Type

Enhancement, Documentation


Description

  • The order of automatic tool execution in bitbucket_app.py has been refactored. The PRDescription tool now runs after the PRCodeSuggestions tool.
  • Enhanced logging in pr_description.py by adding a log statement before publishing the answer as a comment.
  • Updated the documentation in Usage.md to clarify the configuration of automatic tools in the .pr_agent.toml file. The instructions now clearly state that the tools run automatically when a new PR is opened.

Changes walkthrough

Relevant files
Enhancement
bitbucket_app.py
Refactor Automatic Tool Execution Order                                                   

pr_agent/servers/bitbucket_app.py

  • Changed the order of automatic tool execution.
  • Moved the execution of PRDescription after PRCodeSuggestions.
+3/-3     
pr_description.py
Enhance Logging in PR Description Tool                                                     

pr_agent/tools/pr_description.py

  • Added a logging statement before publishing the answer as a comment.
+1/-0     
Documentation
Usage.md
Update Documentation for Automatic Tool Configuration                       

Usage.md

  • Updated the instructions for configuring automatic tools in
    .pr_agent.toml.
  • Clarified that the tools run automatically when a new PR is opened.
+3/-3     

✨ Usage guide:

Overview:
The describe tool scans the PR code changes, and generates a description for the PR - title, type, summary, walkthrough and labels. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on a PR.

When commenting, to edit configurations related to the describe tool (pr_description section), use the following template:

/describe --pr_description.some_config1=... --pr_description.some_config2=...

With a configuration file, use the following template:

[pr_description]
some_config1=...
some_config2=...
Enabling\disabling automation
  • When you first install the app, the default mode for the describe tool is:
pr_commands = ["/describe --pr_description.add_original_user_description=true" 
                         "--pr_description.keep_original_user_title=true", ...]

meaning the describe tool will run automatically on every PR, will keep the original title, and will add the original user description above the generated description.

  • Markers are an alternative way to control the generated description, to give maximal control to the user. If you set:
pr_commands = ["/describe --pr_description.use_description_markers=true", ...]

the tool will replace every marker of the form pr_agent:marker_name in the PR description with the relevant content, where marker_name is one of the following:

  • type: the PR type.
  • summary: the PR summary.
  • walkthrough: the PR walkthrough.

Note that when markers are enabled, if the original PR description does not contain any markers, the tool will not alter the description at all.

Custom labels

The default labels of the describe tool are quite generic: [Bug fix, Tests, Enhancement, Documentation, Other].

If you specify custom labels in the repo's labels page or via configuration file, you can get tailored labels for your use cases.
Examples for custom labels:

  • Main topic:performance - pr_agent:The main topic of this PR is performance
  • New endpoint - pr_agent:A new endpoint was added in this PR
  • SQL query - pr_agent:A new SQL query was added in this PR
  • Dockerfile changes - pr_agent:The PR contains changes in the Dockerfile
  • ...

The list above is eclectic, and aims to give an idea of different possibilities. Define custom labels that are relevant for your repo and use cases.
Note that Labels are not mutually exclusive, so you can add multiple label categories.
Make sure to provide proper title, and a detailed and well-phrased description for each label, so the tool will know when to suggest it.

Utilizing extra instructions

The describe tool can be configured with extra instructions, to guide the model to a feedback tailored to the needs of your project.

Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Notice that the general structure of the description is fixed, and cannot be changed. Extra instructions can change the content or style of each sub-section of the PR description.

Examples for extra instructions:

[pr_description] 
extra_instructions="""
- The PR title should be in the format: '<PR type>: <title>'
- The title should be short and concise (up to 10 words)
- ...
"""

Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

More PR-Agent commands

To invoke the PR-Agent, add a comment using one of the following commands:

  • /review: Request a review of your Pull Request.
  • /describe: Update the PR title and description based on the contents of the PR.
  • /improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback.
  • /ask <QUESTION>: Ask a question about the PR.
  • /update_changelog: Update the changelog based on the PR's contents.
  • /add_docs 💎: Generate docstring for new components introduced in the PR.
  • /generate_labels 💎: Generate labels for the PR based on the PR's contents.
  • /analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component.

See the tools guide for more details.
To list the possible configuration parameters, add a /config comment.

See the describe usage page for a comprehensive guide on using this tool.

…ging in pr_description.py, and update tool configuration instructions in Usage.md
@mrT23
Copy link
Collaborator Author

mrT23 commented Dec 12, 2023

/describe

Copy link
Contributor

PR Analysis

  • 🎯 Main theme: Refactoring tool execution order and adding logging
  • 📝 PR summary: This PR refactors the order of tool execution in the bitbucket_app.py file, adds logging in the pr_description.py file, and updates the tool configuration instructions in the Usage.md file.
  • 📌 Type of PR: Enhancement
  • 🧪 Relevant tests added: No
  • ⏱️ Estimated effort to review [1-5]: 2, because the changes are straightforward and mainly involve refactoring and logging.
  • 🔒 Security concerns: No security concerns found

PR Feedback

  • 💡 General suggestions: The PR is well-structured and the changes are clear. However, it would be beneficial to add tests to ensure the refactoring and logging changes work as expected.

  • 🤖 Code feedback:
    relevant filepr_agent/servers/bitbucket_app.py
    suggestion      Consider adding error handling for the async function calls to PRReviewer, PRCodeSuggestions, and PRDescription. This will ensure that if one function fails, it does not affect the execution of the others. [important]
    relevant line'+ await PRReviewer(pr_url).run()'

    relevant filepr_agent/tools/pr_description.py
    suggestion      It would be beneficial to add more detailed logging messages, including the PR id and the result of the publishing action. This will help in debugging and understanding the system's behavior. [medium]
    relevant line'+ get_logger().info(f"Publishing answer as comment")'

How to use

Instructions

To invoke the PR-Agent, add a comment using one of the following commands:
/review: Request a review of your Pull Request.
/describe: Update the PR title and description based on the contents of the PR.
/improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback.
/ask <QUESTION>: Ask a question about the PR.
/update_changelog: Update the changelog based on the PR's contents.
/add_docs: Generate docstring for new components introduced in the PR.
/generate_labels: Generate labels for the PR based on the PR's contents.
see the tools guide for more details.

To edit any configuration parameter from the configuration.toml, add --config_path=new_value.
For example: /review --pr_reviewer.extra_instructions="focus on the file: ..."
To list the possible configuration parameters, add a /config comment.

@github-actions github-actions bot changed the title feat: Refactor auto tool execution order in bitbucket_app.py, add log… Refactor auto tool execution order and enhance logging and documentation Dec 12, 2023
@github-actions github-actions bot added the enhancement New feature or request label Dec 12, 2023
Copy link
Contributor

PR Description updated to latest commit (2f10b4f)

Copy link
Contributor

PR Code Suggestions

💡 Suggestion:

The order of execution of the tools seems to have been changed. If the order is significant, consider adding a comment explaining the reasoning behind this order.

File: pr_agent/servers/bitbucket_app.py (100-108)

Example code:

Existing code:

auto_review = get_setting_or_env("BITBUCKET_APP.AUTO_REVIEW", None)
if auto_review is None or is_true(auto_review):  # by default, auto review is enabled
    await PRReviewer(pr_url).run()
auto_improve = get_setting_or_env("BITBUCKET_APP.AUTO_IMPROVE", None)
if is_true(auto_improve):  # by default, auto improve is disabled
    await PRCodeSuggestions(pr_url).run()
auto_describe = get_setting_or_env("BITBUCKET_APP.AUTO_DESCRIBE", None)
if is_true(auto_describe):  # by default, auto describe is disabled
    await PRDescription(pr_url).run()

Improved code:

auto_review = get_setting_or_env("BITBUCKET_APP.AUTO_REVIEW", None)
if auto_review is None or is_true(auto_review):  # by default, auto review is enabled
    await PRReviewer(pr_url).run()
# The order of execution is significant: auto_improve should run before auto_describe
auto_improve = get_setting_or_env("BITBUCKET_APP.AUTO_IMPROVE", None)
if is_true(auto_improve):  # by default, auto improve is disabled
    await PRCodeSuggestions(pr_url).run()
auto_describe = get_setting_or_env("BITBUCKET_APP.AUTO_DESCRIBE", None)
if is_true(auto_describe):  # by default, auto describe is disabled
    await PRDescription(pr_url).run()

💡 Suggestion:

Consider adding more context to the log message to make it more informative.

File: pr_agent/tools/pr_description.py (105-105)

Example code:

Existing code:

get_logger().info(f"Publishing answer as comment")

Improved code:

get_logger().info(f"Publishing answer {self.pr_id} as comment")

💡 Suggestion:

The instructions for setting the configuration variables could be clearer. Consider specifying where the .pr_agent.toml file is located.

File: Usage.md (243-243)

Example code:

Existing code:

You can configure in your local `.pr_agent.toml` file which tools will **run automatically** when a new PR is opened.

Improved code:

You can configure which tools will **run automatically** when a new PR is opened in your local `.pr_agent.toml` file, typically located at the root of your project.

@mrT23 mrT23 merged commit 1853b4e into main Dec 12, 2023
3 checks passed
@mrT23 mrT23 deleted the tr/bitbucket_app branch December 12, 2023 08:02
@mrT23
Copy link
Collaborator Author

mrT23 commented Dec 24, 2023

/describe

@qodo-merge-pro qodo-merge-pro bot changed the title Refactor auto tool execution order and enhance logging and documentation Refactor Tool Execution Order in bitbucket_app.py, Enhance Logging in pr_description.py, and Update Documentation Dec 24, 2023
@qodo-merge-pro qodo-merge-pro bot removed the enhancement New feature or request label Dec 24, 2023
Copy link
Contributor

PR Description updated to latest commit (2f10b4f)

@hussam789 hussam789 changed the title Refactor Tool Execution Order in bitbucket_app.py, Enhance Logging in pr_description.py, and Update Documentation Refactor Tool Execution Order, Enhance Logging, and Update Documentation Jan 21, 2024
@hussam789 hussam789 added the enhancement New feature or request label Jan 21, 2024
@hussam789
Copy link
Collaborator

PR Description updated to latest commit (2f10b4f)

1 similar comment
@hussam789
Copy link
Collaborator

PR Description updated to latest commit (2f10b4f)

@hussam789 hussam789 added the documentation Improvements or additions to documentation label Jan 21, 2024
yochail pushed a commit to yochail/pr-agent that referenced this pull request Feb 11, 2024
Refactor auto tool execution order and enhance logging and documentation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request Review effort [1-5]: 2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants