Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: Download Worksheet Results #1547

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

anilsoni7
Copy link

Feature or Bugfix

  • Feature

Detail

  • teamA in environmentA has a team IAM role roleA and a workgroup wgA created by data.all
  • in Worksheets data.all uses that team IAM role and workgroup to run the query
  • the results are stored in the workgroup S3 location = environment S3 bucket/ athenaQueries / teamAFolder/
  • Create the presigned url for the csv present under the s3 location with 2 hours of expiration
  • Storing presigned urls generated in database so that can be reused in future

Relates

Testcase

  • tests/modules/worksheets/test_worksheet.py/test_create_query_download_url

Security

Please answer the questions below briefly where applicable, or write N/A. Based on
OWASP 10.

  • Does this PR introduce or modify any input fields or queries - this includes
    fetching data from storage outside the application (e.g. a database, an S3 bucket)? - Yes
    • Is the input sanitized?
    • What precautions are you taking before deserializing the data you consume?
    • Is injection prevented by parametrizing queries? - Yes
    • Have you ensured no eval or similar functions are used? - Yes
  • Does this PR introduce any functionality or component that requires authorization? - Yes
    • How have you ensured it respects the existing AuthN/AuthZ mechanisms? - Yes
    • Are you logging failed auth attempts?
  • Are you using or adding any cryptographic features? - N/A
    • Do you use a standard proven implementations? - Yes
    • Are the used keys controlled by the customer? Where are they stored? - N/A
  • Are you introducing any new policies/roles/users?
    • Have you used the least-privilege principle? How?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@anilsoni7 anilsoni7 changed the title Query result download Feat: Download Worksheet Results Sep 13, 2024
@dlpzx dlpzx self-requested a review September 13, 2024 09:23
Copy link
Contributor

@dlpzx dlpzx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @anilsoni7, first of all, thank you for opening the PR! I have left some comments and requested changes, but it is less than it looks. The technical design of how results are downloaded and the user experience are great :) 🚀 Feel free to request more info in the comments if anything is unclear

Copy link
Contributor

@dlpzx dlpzx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @anilsoni7, the PR looks great, but there are still some changes regarding permissions that we need to solve

@dlpzx dlpzx self-requested a review October 21, 2024 10:11
Copy link
Contributor

@dlpzx dlpzx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @anilsoni7, I reviewed and tested the functionality locally and resolved all the comments that are solved or were misunderstandings or questions. Can you take a look at the pending open comments? Thank you again for the good work :)

@anilsoni7
Copy link
Author

Hi @dlpzx, I have pushed the latest changes resolving the comments which you mentioned. Can you review them

# Conflicts:
#	backend/requirements.txt
@anilsoni7 anilsoni7 requested a review from dlpzx October 22, 2024 13:23
Copy link
Contributor

@dlpzx dlpzx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Ready to be merged. Thank you @anilsoni7 🚀

@anilsoni7
Copy link
Author

Fixed

  • Integration test
  • alembic test
  • ruff checks & format

need to have a discussion with you regarding npm audit failing.
there is an package which is falling into critical severity

@dlpzx
Copy link
Contributor

dlpzx commented Oct 29, 2024

Hi @anilsoni7 sorry for bothering you again. We have fixed the issue with the npm-audit, could you merge the main branch into this branch? There is also an issue on the migration scripts that needs to be fixed before merging

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants