Add unit tests for file_util presigned URL logic HHS#11647 - #4
Open
davidhuanggg wants to merge 1 commit into
Open
Conversation
17 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #11647 HHS/simpler-grants-gov#11647
Changes proposed
Added:
pre_sign_uploadPOST policy construction (Content-Type, metadata pinning, content-length-range from FileConfig, IfNoneMatch)pre_sign_file_location(bucket/key parsing, localhost endpoint override, non-default duration)presign_or_s3_cdnify_urlCDN vs presign branching and non-s3://ValueErrorContext for reviewers
These three helpers (
pre_sign_upload,pre_sign_file_location,presign_or_s3_cdnify_url) were mostly only covered indirectly via API route tests. The security-relevant pieces especiallypre_sign_upload's POST policy conditions weren't pinned at the util layer, so a refactor of theconditionslist could slip through.This PR adds direct unit tests in
test_file_util.pythat assert on policy structure and parsed URL query params rather than full signed URL / signature strings, so they stay stable across botocore signing changes.Test only.
make testpasses inbackend/grants_shared.Validation steps
Local Reproduction
Output
Acceptance criteria
pre_sign_uploadpolicy construction:Content-Typeappears in bothFieldsandConditionsmetadataentry becomes anx-amz-meta-<key>field and a pinned conditioncontent-length-rangecondition is[1, max_file_upload_size_bytes]and tracksFileConfig, not a hardcoded numberinclude_if_none_match=Trueadds{"IfNoneMatch": "*"}; default (False) does noturlandfields, andfieldscarries the signed policy/signaturepre_sign_file_location:s3://path viasplit_s3_url(nested keys, keys with spaces/special chars)aws_s3_endpoint_urlis set, and the URL is left untouched when it is notpresigned_s3_durationis reflected inX-Amz-Expires(existing test covers the 900s default)presign_or_s3_cdnify_urlbranching:cdn_urlset → returns a CDN URL, no presigningcdn_urlunset → returns a presigned URLs3://path withcdn_urlset → raisesValueErrorbackend/grants_shared/tests/grants_shared/util/test_file_util.py, use the existingmock_s3_bucket/s3_configfixtures, and parametrize where cases are shape-identicalmake testpasses inbackend/grants_sharedOriginal PR linked prior to the repo move that will be closed:
HHS/simpler-grants-gov#11700