Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Improved server, security and commands #529

Merged
merged 2 commits into from
Dec 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 32 additions & 16 deletions pr_agent/servers/bitbucket_server_webhook.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import json
import os

import uvicorn
from fastapi import APIRouter, FastAPI
Expand All @@ -13,35 +14,55 @@
from pr_agent.agent.pr_agent import PRAgent
from pr_agent.config_loader import get_settings
from pr_agent.log import get_logger
from pr_agent.servers.utils import verify_signature

router = APIRouter()


def handle_request(background_tasks: BackgroundTasks, url: str, body: str, log_context: dict):
def handle_request(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you change the parenthesis styling?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Black... I linted and formatted with black

background_tasks: BackgroundTasks, url: str, body: str, log_context: dict
):
log_context["action"] = body
log_context["event"] = "pull_request" if body == "review" else "comment"
log_context["api_url"] = url
with get_logger().contextualize(**log_context):
background_tasks.add_task(PRAgent().handle_request, url, body)


@router.post("/webhook")
@router.post("/")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you change the webhook endpoint to /?

Copy link
Contributor Author

@lukefx lukefx Dec 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because on the configuration page in bitbucket, there is a Test connection button, if the webhook has the same get and post URL you can see if the call succeeds or not otherwise this test will always fail.

An example, it succeeds because the URL supports get and post
image

async def handle_webhook(background_tasks: BackgroundTasks, request: Request):
log_context = {"server_type": "bitbucket_server"}
data = await request.json()
get_logger().info(json.dumps(data))

pr_id = data['pullRequest']['id']
repository_name = data['pullRequest']['toRef']['repository']['slug']
project_name = data['pullRequest']['toRef']['repository']['project']['key']
webhook_secret = get_settings().get("BITBUCKET_SERVER.WEBHOOK_SECRET", None)
if webhook_secret:
body_bytes = await request.body()
signature_header = request.headers.get("x-hub-signature", None)
verify_signature(body_bytes, webhook_secret, signature_header)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you verify that it works as expected?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it works... the HMAC method is the same as GitHub just the header has a slightly different name (without -256)


pr_id = data["pullRequest"]["id"]
repository_name = data["pullRequest"]["toRef"]["repository"]["slug"]
project_name = data["pullRequest"]["toRef"]["repository"]["project"]["key"]
bitbucket_server = get_settings().get("BITBUCKET_SERVER.URL")
pr_url = f"{bitbucket_server}/projects/{project_name}/repos/{repository_name}/pull-requests/{pr_id}"

log_context["api_url"] = pr_url
log_context["event"] = "pull_request"

handle_request(background_tasks, pr_url, "review", log_context)
return JSONResponse(status_code=status.HTTP_200_OK, content=jsonable_encoder({"message": "success"}))
if data["eventKey"] == "pr:opened":
body = "review"
elif data["eventKey"] == "pr:comment:added":
body = data["comment"]["text"]
else:
return JSONResponse(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should return 200 in this case, this is not an error, unless you know differently?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well we can, but you have to configure which events you want to be pushed as webhook. If you push some events that are not supported it could be a bad request, or could be a 200 and who cares or, could also be a 501 not implemented which makes also sense.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm asking before in some cases, after a certain threshold of errors the webhook is disabled.

status_code=status.HTTP_400_BAD_REQUEST,
content=json.dumps({"message": "Unsupported event"}),
)

handle_request(background_tasks, pr_url, body, log_context)
return JSONResponse(
status_code=status.HTTP_200_OK, content=jsonable_encoder({"message": "success"})
)


@router.get("/")
Expand All @@ -50,15 +71,10 @@ async def root():


def start():
bitbucket_server_url = get_settings().get("BITBUCKET_SERVER.URL", None)
if not bitbucket_server_url:
raise ValueError("BITBUCKET_SERVER.URL is not set")
get_settings().config.git_provider = "bitbucket_server"
middleware = [Middleware(RawContextMiddleware)]
app = FastAPI(middleware=middleware)
app = FastAPI(middleware=[Middleware(RawContextMiddleware)])
app.include_router(router)
uvicorn.run(app, host="0.0.0.0", port=3000)
uvicorn.run(app, host="0.0.0.0", port=int(os.environ.get("PORT", "3000")))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change this to BITBUCKET_SERVER.PORT, use get_settings().get(), add to configuration file

Copy link
Contributor Author

@lukefx lukefx Dec 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's like this because some cloud providers uses an environment variable PORT to bind the ingress... also in bitbucket_app.py is done like this:

https://github.com/Codium-ai/pr-agent/blob/main/pr_agent/servers/bitbucket_app.py#L162

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't complain on my own code :(



if __name__ == '__main__':
if __name__ == "__main__":
start()
5 changes: 5 additions & 0 deletions pr_agent/settings/.secrets_template.toml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,11 @@ personal_access_token = ""
# For Bitbucket personal/repository bearer token
bearer_token = ""

[bitbucket_server]
# For Bitbucket Server bearer token
auth_token = ""
webhook_secret = ""

# For Bitbucket app
app_key = ""
base_url = ""
Expand Down
5 changes: 5 additions & 0 deletions pr_agent/settings/configuration.toml
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,11 @@ polling_interval_seconds = 30
# token to authenticate in the patch server
# patch_server_token = ""

[bitbucket_server]
# URL to the BitBucket Server instance
# url = "https://git.bitbucket.com"
url = ""

[litellm]
#use_client = false

Expand Down