Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TNS API fix and a few others #282

Merged
merged 19 commits into from
Mar 17, 2025
Merged

TNS API fix and a few others #282

merged 19 commits into from
Mar 17, 2025

Conversation

djones1040
Copy link
Collaborator

@djones1040 djones1040 commented Feb 10, 2025

Fixes # .

Description of the Change

  1. TNS API change is now addressed for the custom uploads feature (Fixes [Bug] transient uploads broken #278 )
  2. Cutout reprocessing bug fixed (Fixes [Bug] Cutout downloads sometimes fail upon reprocessing #277)
  3. Aperture size adjustment in quadrature (Fixes aperture size adjustment should be done in quadrature #280 )
  4. Mag systems labeled explicitly in table (Fixes [Bug] mag system units unclear #269 )
  5. Attempted to fix a bug where new redshifts for transients are not retriggering processing correctly.
  6. Software version included in transient, host, cutouts, apertures, aperturephotometry, and SED params (Addresses Link Blast version with generated data products for provenance #264 )

Checklist

Please check all that apply to your proposed changes

  • HTML code change
  • Added package dependency
  • Added data
  • Changed django model
  • Documentation change
  • Added or changed TaskRunner

Additional context

Sorry, something went wrong.

@djones1040 djones1040 marked this pull request as ready for review February 10, 2025 17:13
@manning-ncsa
Copy link
Collaborator

@djones1040 I merged main into your branch and then added some commits. In particular, please review my changes to the TNS ingest function here. The logic of this function appears to have become convoluted over time so I refactored it to make it easier to understand and hopefully be less prone to error. Workflows are running in my local dev environment as of commit 309815d, and the output of a test transient I uploaded yields different calculated values for equivalent workflow task statuses.

image image

@manning-ncsa
Copy link
Collaborator

There is another concern I have about the consistency of TNS ingest and processing logic. Search the code for the four instances of transient_workflow.delay where workflows are actually launched (recall that we switched to using the Celery Canvas workflow system to run transient tasks as quickly as possible). There are other functions that query the TNS and launch workflows but with inconsistent processing of the TNS data compared to this "primary" TNSDataIngestion.run_process function.

@manning-ncsa
Copy link
Collaborator

There is another concern I have about the consistency of TNS ingest and processing logic. Search the code for the four instances of transient_workflow.delay where workflows are actually launched (recall that we switched to using the Celery Canvas workflow system to run transient tasks as quickly as possible). There are other functions that query the TNS and launch workflows but with inconsistent processing of the TNS data compared to this "primary" TNSDataIngestion.run_process function.

@djones1040 and I examined the code together and determined that there is no inconsistency.

@manning-ncsa
Copy link
Collaborator

Workflows are running in my local dev environment as of commit 309815d, and the output of a test transient I uploaded yields different calculated values for equivalent workflow task statuses.

@djones1040 determined that there is a bug in the aperture photometry code introduced by commit b065674 that he will fix prior to merging this branch.

@djones1040 djones1040 merged commit 3747fd1 into main Mar 17, 2025
1 check passed
@djones1040 djones1040 deleted the djones-tns-api-2 branch March 17, 2025 22:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants