Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Log HTTP request body on signature verification failure #3239

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

randombit
Copy link
Contributor

There have been reports both internally and in support channels of ingress messages failing signature verification. So far these have not been reproducible, and the cause is unknown. To assist debugging, if verification fails log the HTTP request body.

@randombit randombit requested a review from fspreiss December 18, 2024 18:50
@github-actions github-actions bot added the chore label Dec 18, 2024
There have been reports both internally and in support channels of
ingress messages failing signature verification. So far these have not
been reproducible, and the cause is unknown. To assist debugging,
if verification fails log the HTTP request body.
Comment on lines +285 to +288
pub fn signed(&self) -> &HttpRequest<SignedIngressContent> {
&self.signed
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just stumbled upon

impl AsRef<HttpRequest<SignedIngressContent>> for SignedIngress {
    fn as_ref(&self) -> &HttpRequest<SignedIngressContent> {
        &self.signed
    }
}

so theoretically, the new signed(&self) is not strictly necessary as the same can be achieved with the existing as_ref(). For people who don't know this code well (like me), this is not obvious, however.

Comment on lines +249 to +250
pub(crate) fn validation_error_to_http_error<C: std::fmt::Debug>(
request: &HttpRequest<C>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Optionally we could make this

pub(crate) fn validation_error_to_http_error<C: std::fmt::Debug + HttpRequestContent>(
    request: &HttpRequest<C>,
    err: RequestValidationError,
    log: &ReplicaLogger,
) -> HttpError {

i.e,. we don't need to take the message_id separately, because we can just get it from the request via request.id().

"msg_id: {}, err: {}, request: {:?}",
message_id,
err,
request.content()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have sufficient debug info when just logging request.content() rather than request?

If we log the entire request, we also get the HttpRequest::auth: Authentication part, which contains the signature and signer_pubkey (and sender_delegation) for authenticated requests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants