Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

The CCD reduction does not work if there is no Science Data. #58

Closed
simontorres opened this issue May 8, 2017 · 6 comments
Closed

The CCD reduction does not work if there is no Science Data. #58

simontorres opened this issue May 8, 2017 · 6 comments
Assignees
@simontorres @Allonck
Labels
March 2024 Sprint Solve long standing issues and implement new requirements. Need Testing

Comments

@simontorres
Copy link
Member

Trying to process some engineering data taken during daytime the ccd pipeline did not process any data.
I have made some changes but so far can't get it to work and it appears to be a problem due to the time the data was taken, early in the morning, so it thinks it is in the previous night, most likely has to do with UTC midnight.

I have some ideas and I will post here what solution I find.
@simontorres simontorres self-assigned this May 8, 2017
@simontorres simontorres added Need Testing March 2024 Sprint Solve long standing issues and implement new requirements. labels Mar 6, 2024
@simontorres
Copy link
Member Author

@Allonck will test this and @simontorres will implement solutions.

@Allonck
Copy link
Collaborator

Allonck commented Mar 11, 2024 •
edited
Loading

I made a test directory without science images (only BIAS and FLATS), and here is the output:

(goodman_pipeline) cmontalban@lswlan59 2021-11-18_test % redccd --debug --red-path RED_test_58
/Users/cmontalban/anaconda3/envs/goodman_pipeline/bin/redccd:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  __import__('pkg_resources').require('goodman-pipeline==1.3.7')
[14:56:41][    INFO]: Starting Goodman HTS Pipeline Log [core.setup_logging:2938]
[14:56:41][    INFO]: Local Time    : 2024-03-11 14:56:41.204070 [core.setup_logging:2939]
[14:56:41][    INFO]: Universal Time: 2024-03-11 17:56:41.204119 [core.setup_logging:2941]
[14:56:41][   DEBUG]: Starting new HTTPS connection (1): api.github.com:443 [connectionpool._new_conn:1055]
[14:56:41][   DEBUG]: https://api.github.com:443 "GET /repos/soar-telescope/goodman/releases/latest HTTP/1.1" 301 213 [connectionpool._make_request:549]
[14:56:42][   DEBUG]: https://api.github.com:443 "GET /repositories/62915938/releases/latest HTTP/1.1" 200 847 [connectionpool._make_request:549]
[14:56:42][   DEBUG]: Starting new HTTPS connection (1): api.github.com:443 [connectionpool._new_conn:1055]
[14:56:42][   DEBUG]: https://api.github.com:443 "GET /repos/soar-telescope/goodman/releases/latest HTTP/1.1" 301 213 [connectionpool._make_request:549]
[14:56:42][   DEBUG]: https://api.github.com:443 "GET /repositories/62915938/releases/latest HTTP/1.1" 200 847 [connectionpool._make_request:549]
[14:56:42][    INFO]: Pipeline Version: 1.3.7 (latest) [core.setup_logging:2952]
[14:56:42][   DEBUG]: Initializing DataClassifier instance [goodman_ccd.__init__:168]
[14:56:42][   DEBUG]: Starting argument checks. [goodman_ccd._check_args:274]
[14:56:42][   DEBUG]: Raw data folder "/Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test" exists. [goodman_ccd._check_args:278]
[14:56:42][    INFO]: Found 51 fits files in /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test. [goodman_ccd._check_args:289]
[14:56:42][ WARNING]: Reduction folder doesn't exist. [goodman_ccd._check_args:359]
[14:56:42][    INFO]: Created directory for reduced data. [goodman_ccd._check_args:361]
[14:56:42][    INFO]: New directory: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/RED_test_58 [goodman_ccd._check_args:362]
[14:56:42][   DEBUG]: Calling data_classifier Instance of DataClassifier [goodman_ccd.__call__:206]
[14:56:42][   DEBUG]: keywords in setter before pruning: *. [image_collection.keywords:231]
[14:56:42][   DEBUG]: keywords after pruning ['*']. [image_collection.keywords:239]
[14:56:42][   DEBUG]: should be building new table... [image_collection.keywords:253]
[14:56:42][   DEBUG]: Raw path: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test [data_classifier.__call__:77]
[14:56:42][   DEBUG]: Detected Red camera. [data_classifier._get_instrument:124]
[14:56:42][    INFO]: Instrument: Red Camera [data_classifier.__call__:81]
[14:56:42][    INFO]: Detected Spectroscopy Data from Red Camera [data_classifier._get_obs_technique:159]
[14:56:42][    INFO]: Observing Technique: Spectroscopy [data_classifier.__call__:90]
[14:56:42][   DEBUG]: Initializing night organizer procedure [goodman_ccd.__call__:237]
[14:56:42][   DEBUG]: Calling night organizer procedure [goodman_ccd.__call__:245]
[14:56:42][   DEBUG]: keywords in setter before pruning: ['naxis', 'date', 'slit', 'date-obs', 'obstype', 'object', 'exptime', 'obsra', 'obsdec', 'grating', 'cam_targ', 'grt_targ', 'filter', 'filter2', 'gain', 'rdnoise', 'roi', 'wavmode']. [image_collection.keywords:231]
[14:56:42][   DEBUG]: keywords after pruning ['rdnoise', 'object', 'gain', 'date-obs', 'obsra', 'obstype', 'cam_targ', 'filter2', 'naxis', 'exptime', 'grating', 'filter', 'obsdec', 'wavmode', 'grt_targ', 'roi', 'slit', 'date']. [image_collection.keywords:239]
[14:56:42][   DEBUG]: should be building new table... [image_collection.keywords:253]
[14:56:43][    INFO]: Organizing data for this configuration: Gain: 1.48, Noise: 3.89, ROI: Spectroscopic 2x2 [night_organizer.__call__:140]
[14:56:53][   DEBUG]: Sun Set 2021-11-18T23:15:44.494 [core.get_twilight_time:1908]
[14:56:53][   DEBUG]: Sun Rise 2021-11-18T09:40:57.560 [core.get_twilight_time:1909]
[14:56:53][    INFO]: Found valid data, appending to data container list [night_organizer.__call__:178]
[14:56:53][   DEBUG]: Initializing image processing procedure [goodman_ccd.__call__:259]
[14:56:53][   DEBUG]: Calling image processing procedure. [goodman_ccd.__call__:264]
[14:56:53][ WARNING]: Determining trim section. Assuming you have only one kind of data in this folder [core.define_trim_section:1039]
[14:56:53][    INFO]: Trim Section: [26:2055,2:948] [core.define_trim_section:1077]
[14:56:53][   DEBUG]: Overscan Sample File /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0071_BIAS_18-11-2021.fits [core.get_overscan_region:1611]
[14:56:53][    INFO]: Overscan regions has been tested for ROI Spectroscopic 1x1, 2x2 and 3x3 [core.get_overscan_region:1625]
[14:56:53][    INFO]: Overscan Region: [3:24,1:948] [core.get_overscan_region:1677]
[14:56:53][   DEBUG]: Creating Master Bias [image_processor.__call__:105]
[14:56:53][    INFO]: Creating master bias [core.create_master_bias:400]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0055_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0056_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0057_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0058_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0059_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0060_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0061_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0062_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0063_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0064_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0065_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0066_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0067_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0068_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0069_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0070_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0071_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0072_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:53][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0073_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:54][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0074_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:54][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0075_BIAS_18-11-2021.fits [core.create_master_bias:404]
[14:56:54][    INFO]: Combining 21 images to create master bias [core.create_master_bias:408]
[14:56:59][    INFO]: Saving FITS file to master_bias_RED_SP_2x2_R03.89_G01.48.fits [core.write_fits:3323]
[14:56:59][    INFO]: Created master bias: master_bias_RED_SP_2x2_R03.89_G01.48.fits [core.create_master_bias:436]
[14:56:59][ WARNING]: Determining trim section. Assuming you have only one kind of data in this folder [core.define_trim_section:1039]
[14:56:59][    INFO]: Trim Section: [26:2055,2:948] [core.define_trim_section:1077]
[14:56:59][   DEBUG]: Overscan Sample File /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0099_GHTS_R_400m2_2x2_18-11-2021.fits [core.get_overscan_region:1611]
[14:56:59][    INFO]: Overscan regions has been tested for ROI Spectroscopic 1x1, 2x2 and 3x3 [core.get_overscan_region:1625]
[14:56:59][    INFO]: Overscan Region: [3:24,1:948] [core.get_overscan_region:1677]
[14:56:59][   DEBUG]: Process Data Group [image_processor.__call__:140]
[14:56:59][ WARNING]: Only flats found in this group [image_processor.process_spectroscopy_science:473]
INFO: using the unit adu passed to the FITS reader instead of the unit adu in the FITS file. [astropy.nddata.ccddata]
[14:56:59][    INFO]: using the unit adu passed to the FITS reader instead of the unit adu in the FITS file. [ccddata.fits_ccddata_reader:721]
[14:56:59][    INFO]: Creating Master Flat [core.create_master_flats:507]
[14:56:59][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0091_GHTS_R_400m2_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:56:59][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:56:59][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0092_GHTS_R_400m2_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:56:59][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:56:59][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0093_GHTS_R_400m2_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:56:59][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:56:59][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0094_GHTS_R_400m2_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:56:59][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:56:59][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0095_GHTS_R_400m2_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:56:59][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:56:59][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0096_GHTS_R_400m2_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:56:59][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:56:59][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0097_GHTS_R_400m2_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:56:59][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:56:59][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0098_GHTS_R_400m2_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:56:59][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:56:59][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0099_GHTS_R_400m2_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:56:59][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:56:59][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0100_GHTS_R_400m2_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:56:59][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:56:59][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0101_GHTS_R_400m2_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:56:59][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:56:59][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0102_GHTS_R_400m2_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:56:59][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:56:59][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0103_GHTS_R_400m2_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:56:59][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:56:59][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0104_GHTS_R_400m2_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:56:59][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:56:59][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0105_GHTS_R_400m2_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:56:59][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:56:59][    INFO]: Combining 15 images to create master flat [core.create_master_flats:544]
[14:57:03][    INFO]: Saving FITS file to master_flat_400m2_GG455_1.0_dome.fits [core.write_fits:3323]
[14:57:03][    INFO]: Created Master Flat: RED_test_58/master_flat_400m2_GG455_1.0_dome.fits [core.create_master_flats:563]
[14:57:03][   DEBUG]: Process Data Group [image_processor.__call__:140]
[14:57:03][ WARNING]: Only flats found in this group [image_processor.process_spectroscopy_science:473]
[14:57:03][   DEBUG]: Found 5052.815 Angstrom as central wavelength [core.get_central_wavelength:1484]
INFO: using the unit adu passed to the FITS reader instead of the unit adu in the FITS file. [astropy.nddata.ccddata]
[14:57:03][    INFO]: using the unit adu passed to the FITS reader instead of the unit adu in the FITS file. [ccddata.fits_ccddata_reader:721]
[14:57:03][    INFO]: Creating Master Flat [core.create_master_flats:507]
[14:57:03][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0076_GHTS_R_400m1_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:57:03][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:57:03][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0077_GHTS_R_400m1_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:57:03][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:57:03][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0078_GHTS_R_400m1_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:57:03][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:57:03][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0079_GHTS_R_400m1_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:57:03][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:57:03][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0080_GHTS_R_400m1_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:57:03][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:57:03][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0081_GHTS_R_400m1_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:57:03][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:57:03][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0082_GHTS_R_400m1_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:57:03][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:57:03][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0083_GHTS_R_400m1_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:57:03][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:57:03][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0084_GHTS_R_400m1_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:57:03][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:57:03][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0085_GHTS_R_400m1_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:57:03][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:57:03][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0086_GHTS_R_400m1_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:57:03][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:57:03][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0087_GHTS_R_400m1_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:57:03][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:57:03][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0088_GHTS_R_400m1_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:57:03][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:57:03][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0089_GHTS_R_400m1_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:57:03][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:57:03][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2021-11-18/2021-11-18_test/0090_GHTS_R_400m1_2x2_18-11-2021.fits [core.create_master_flats:513]
[14:57:03][   DEBUG]: Set saturation_threshold level as 69257 [core.get_saturation_value:4072]
[14:57:03][    INFO]: Combining 15 images to create master flat [core.create_master_flats:544]
[14:57:07][    INFO]: Saving FITS file to master_flat_400Custom_5053nm_NO_FILTER_1.0_dome.fits [core.write_fits:3323]
[14:57:07][    INFO]: Created Master Flat: RED_test_58/master_flat_400Custom_5053nm_NO_FILTER_1.0_dome.fits [core.create_master_flats:563]
(goodman_pipeline) cmontalban@lswlan59 2021-11-18_test %

The red-path directory only has:

  • master_bias_RED_SP_2x2_R03.89_G01.48.fits
  • master_flat_400Custom_5053nm_NO_FILTER_1.0_dome.fits
  • master_flat_400m2_GG455_1.0_dome.fits

What I see is just the creation of masters. In fact, the flats were not normalized, and when comparing the counts with the RAW images, I would say that they were not bias subtracted either. In summary, the images weren't processed.

@Allonck
Copy link
Collaborator

Allonck commented Mar 11, 2024

I made another test with the Blue camera and I have the same results.

(goodman_pipeline) cmontalban@lswlan59 2023-03-09_test_58 % redccd --debug --red-path RED_test_58
/Users/cmontalban/anaconda3/envs/goodman_pipeline/bin/redccd:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  __import__('pkg_resources').require('goodman-pipeline==1.3.7')
[15:12:22][    INFO]: Starting Goodman HTS Pipeline Log [core.setup_logging:2938]
[15:12:22][    INFO]: Local Time    : 2024-03-11 15:12:22.163553 [core.setup_logging:2939]
[15:12:22][    INFO]: Universal Time: 2024-03-11 18:12:22.163601 [core.setup_logging:2941]
[15:12:22][   DEBUG]: Starting new HTTPS connection (1): api.github.com:443 [connectionpool._new_conn:1055]
[15:12:23][   DEBUG]: https://api.github.com:443 "GET /repos/soar-telescope/goodman/releases/latest HTTP/1.1" 301 213 [connectionpool._make_request:549]
[15:12:23][   DEBUG]: https://api.github.com:443 "GET /repositories/62915938/releases/latest HTTP/1.1" 200 847 [connectionpool._make_request:549]
[15:12:23][   DEBUG]: Starting new HTTPS connection (1): api.github.com:443 [connectionpool._new_conn:1055]
[15:12:24][   DEBUG]: https://api.github.com:443 "GET /repos/soar-telescope/goodman/releases/latest HTTP/1.1" 301 213 [connectionpool._make_request:549]
[15:12:24][   DEBUG]: https://api.github.com:443 "GET /repositories/62915938/releases/latest HTTP/1.1" 200 847 [connectionpool._make_request:549]
[15:12:24][    INFO]: Pipeline Version: 1.3.7 (latest) [core.setup_logging:2952]
[15:12:24][   DEBUG]: Initializing DataClassifier instance [goodman_ccd.__init__:168]
[15:12:24][   DEBUG]: Starting argument checks. [goodman_ccd._check_args:274]
[15:12:24][   DEBUG]: Raw data folder "/Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58" exists. [goodman_ccd._check_args:278]
[15:12:24][    INFO]: Found 37 fits files in /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58. [goodman_ccd._check_args:289]
[15:12:24][ WARNING]: Reduction folder doesn't exist. [goodman_ccd._check_args:359]
[15:12:24][    INFO]: Created directory for reduced data. [goodman_ccd._check_args:361]
[15:12:24][    INFO]: New directory: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/RED_test_58 [goodman_ccd._check_args:362]
[15:12:24][   DEBUG]: Calling data_classifier Instance of DataClassifier [goodman_ccd.__call__:206]
[15:12:24][   DEBUG]: keywords in setter before pruning: *. [image_collection.keywords:231]
[15:12:24][   DEBUG]: keywords after pruning ['*']. [image_collection.keywords:239]
[15:12:24][   DEBUG]: should be building new table... [image_collection.keywords:253]
[15:12:24][ WARNING]: unable to get FITS header for file /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/repeated_fits: [Errno 21] Is a directory: '/Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/repeated_fits'. [image_collection._fits_summary:623]
[15:12:24][   DEBUG]: Raw path: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58 [data_classifier.__call__:77]
[15:12:24][   DEBUG]: Detected Blue camera. [data_classifier._get_instrument:124]
[15:12:24][    INFO]: Instrument: Blue Camera [data_classifier.__call__:81]
[15:12:24][    INFO]: Detected Spectroscopy Data from Blue Camera [data_classifier._get_obs_technique:159]
[15:12:24][    INFO]: Observing Technique: Spectroscopy [data_classifier.__call__:90]
[15:12:24][   DEBUG]: Initializing night organizer procedure [goodman_ccd.__call__:237]
[15:12:24][   DEBUG]: Calling night organizer procedure [goodman_ccd.__call__:245]
[15:12:24][   DEBUG]: keywords in setter before pruning: ['naxis', 'date', 'slit', 'date-obs', 'obstype', 'object', 'exptime', 'obsra', 'obsdec', 'grating', 'cam_targ', 'grt_targ', 'filter', 'filter2', 'gain', 'rdnoise', 'roi', 'wavmode']. [image_collection.keywords:231]
[15:12:24][   DEBUG]: keywords after pruning ['filter2', 'obsdec', 'object', 'cam_targ', 'grt_targ', 'naxis', 'exptime', 'roi', 'slit', 'obsra', 'wavmode', 'rdnoise', 'filter', 'date-obs', 'obstype', 'date', 'gain', 'grating']. [image_collection.keywords:239]
[15:12:24][   DEBUG]: should be building new table... [image_collection.keywords:253]
[15:12:24][ WARNING]: unable to get FITS header for file /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/repeated_fits: [Errno 21] Is a directory: '/Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/repeated_fits'. [image_collection._fits_summary:623]
[15:12:24][    INFO]: Organizing data for this configuration: Gain: 1.40, Noise: 4.74, ROI: Spectroscopic 2x2 [night_organizer.__call__:140]
[15:12:35][   DEBUG]: Sun Set 2023-03-09T23:03:26.364 [core.get_twilight_time:1908]
[15:12:35][   DEBUG]: Sun Rise 2023-03-09T10:42:57.770 [core.get_twilight_time:1909]
[15:12:35][    INFO]: Found valid data, appending to data container list [night_organizer.__call__:178]
[15:12:35][   DEBUG]: Initializing image processing procedure [goodman_ccd.__call__:259]
[15:12:35][   DEBUG]: Calling image processing procedure. [goodman_ccd.__call__:264]
[15:12:35][ WARNING]: Determining trim section. Assuming you have only one kind of data in this folder [core.define_trim_section:1039]
[15:12:35][    INFO]: Trim Section: [26:2055,2:948] [core.define_trim_section:1077]
[15:12:35][   DEBUG]: Overscan Sample File /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0037_BIAS_09-03-2023.fits [core.get_overscan_region:1611]
[15:12:35][    INFO]: Overscan regions has been tested for ROI Spectroscopic 1x1, 2x2 and 3x3 [core.get_overscan_region:1625]
[15:12:35][    INFO]: Overscan Region: [1:8,1:948] [core.get_overscan_region:1677]
[15:12:35][   DEBUG]: Creating Master Bias [image_processor.__call__:105]
[15:12:35][    INFO]: Creating master bias [core.create_master_bias:400]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0030_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0031_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0032_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0033_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0034_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0035_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0036_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0037_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0038_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0039_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0040_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0041_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0042_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0043_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0044_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0045_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0046_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0047_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0048_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0049_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][   DEBUG]: Loading bias image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0050_BIAS_09-03-2023.fits [core.create_master_bias:404]
[15:12:35][    INFO]: Combining 21 images to create master bias [core.create_master_bias:408]
[15:12:41][    INFO]: Saving FITS file to master_bias_BLUE_SP_2x2_R04.74_G01.40.fits [core.write_fits:3323]
[15:12:41][    INFO]: Created master bias: master_bias_BLUE_SP_2x2_R04.74_G01.40.fits [core.create_master_bias:436]
[15:12:41][ WARNING]: Determining trim section. Assuming you have only one kind of data in this folder [core.define_trim_section:1039]
[15:12:41][    INFO]: Trim Section: [26:2055,2:948] [core.define_trim_section:1077]
[15:12:41][   DEBUG]: Overscan Sample File /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0062_GHTS_B_400m1_2x2_09-03-2023.fits [core.get_overscan_region:1611]
[15:12:41][    INFO]: Overscan regions has been tested for ROI Spectroscopic 1x1, 2x2 and 3x3 [core.get_overscan_region:1625]
[15:12:41][    INFO]: Overscan Region: [1:8,1:948] [core.get_overscan_region:1677]
[15:12:41][   DEBUG]: Process Data Group [image_processor.__call__:140]
[15:12:41][ WARNING]: Only flats found in this group [image_processor.process_spectroscopy_science:473]
[15:12:41][   DEBUG]: Found 5052.815 Angstrom as central wavelength [core.get_central_wavelength:1484]
INFO: using the unit adu passed to the FITS reader instead of the unit adu in the FITS file. [astropy.nddata.ccddata]
[15:12:41][    INFO]: using the unit adu passed to the FITS reader instead of the unit adu in the FITS file. [ccddata.fits_ccddata_reader:721]
[15:12:41][    INFO]: Creating Master Flat [core.create_master_flats:507]
[15:12:41][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0051_GHTS_B_400m1_2x2_09-03-2023.fits [core.create_master_flats:513]
[15:12:41][   DEBUG]: Set saturation_threshold level as 49928 [core.get_saturation_value:4072]
[15:12:41][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0052_GHTS_B_400m1_2x2_09-03-2023.fits [core.create_master_flats:513]
[15:12:41][   DEBUG]: Set saturation_threshold level as 49928 [core.get_saturation_value:4072]
[15:12:41][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0053_GHTS_B_400m1_2x2_09-03-2023.fits [core.create_master_flats:513]
[15:12:41][   DEBUG]: Set saturation_threshold level as 49928 [core.get_saturation_value:4072]
[15:12:41][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0054_GHTS_B_400m1_2x2_09-03-2023.fits [core.create_master_flats:513]
[15:12:41][   DEBUG]: Set saturation_threshold level as 49928 [core.get_saturation_value:4072]
[15:12:41][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0055_GHTS_B_400m1_2x2_09-03-2023.fits [core.create_master_flats:513]
[15:12:41][   DEBUG]: Set saturation_threshold level as 49928 [core.get_saturation_value:4072]
[15:12:41][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0056_GHTS_B_400m1_2x2_09-03-2023.fits [core.create_master_flats:513]
[15:12:41][   DEBUG]: Set saturation_threshold level as 49928 [core.get_saturation_value:4072]
[15:12:41][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0057_GHTS_B_400m1_2x2_09-03-2023.fits [core.create_master_flats:513]
[15:12:41][   DEBUG]: Set saturation_threshold level as 49928 [core.get_saturation_value:4072]
[15:12:41][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0058_GHTS_B_400m1_2x2_09-03-2023.fits [core.create_master_flats:513]
[15:12:41][   DEBUG]: Set saturation_threshold level as 49928 [core.get_saturation_value:4072]
[15:12:41][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0059_GHTS_B_400m1_2x2_09-03-2023.fits [core.create_master_flats:513]
[15:12:41][   DEBUG]: Set saturation_threshold level as 49928 [core.get_saturation_value:4072]
[15:12:41][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0060_GHTS_B_400m1_2x2_09-03-2023.fits [core.create_master_flats:513]
[15:12:41][   DEBUG]: Set saturation_threshold level as 49928 [core.get_saturation_value:4072]
[15:12:41][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0061_GHTS_B_400m1_2x2_09-03-2023.fits [core.create_master_flats:513]
[15:12:41][   DEBUG]: Set saturation_threshold level as 49928 [core.get_saturation_value:4072]
[15:12:41][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0062_GHTS_B_400m1_2x2_09-03-2023.fits [core.create_master_flats:513]
[15:12:41][   DEBUG]: Set saturation_threshold level as 49928 [core.get_saturation_value:4072]
[15:12:41][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0063_GHTS_B_400m1_2x2_09-03-2023.fits [core.create_master_flats:513]
[15:12:41][   DEBUG]: Set saturation_threshold level as 49928 [core.get_saturation_value:4072]
[15:12:41][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0064_GHTS_B_400m1_2x2_09-03-2023.fits [core.create_master_flats:513]
[15:12:41][   DEBUG]: Set saturation_threshold level as 49928 [core.get_saturation_value:4072]
[15:12:41][   DEBUG]: Loading flat image: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_test_58/0065_GHTS_B_400m1_2x2_09-03-2023.fits [core.create_master_flats:513]
[15:12:41][   DEBUG]: Set saturation_threshold level as 49928 [core.get_saturation_value:4072]
[15:12:41][    INFO]: Combining 15 images to create master flat [core.create_master_flats:544]
[15:12:45][    INFO]: Saving FITS file to master_flat_400Custom_5053nm_NO_FILTER_1.0_dome.fits [core.write_fits:3323]
[15:12:45][    INFO]: Created Master Flat: RED_test_58/master_flat_400Custom_5053nm_NO_FILTER_1.0_dome.fits [core.create_master_flats:563]
(goodman_pipeline) cmontalban@lswlan59 2023-03-09_test_58 %

The masters created:

  • master_bias_BLUE_SP_2x2_R04.74_G01.40.fits
  • master_flat_400Custom_5053nm_NO_FILTER_1.0_dome.fits

@Allonck
Copy link
Collaborator

Allonck commented Mar 11, 2024

I made a test with only BIAS imaging data and the redccd process not even started.


(goodman_pipeline) cmontalban@lswlan59 imaging % redccd 
/Users/cmontalban/anaconda3/envs/goodman_pipeline/bin/redccd:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  __import__('pkg_resources').require('goodman-pipeline==1.3.7')
[15:27:20][I]: Starting Goodman HTS Pipeline Log
[15:27:20][I]: Local Time    : 2024-03-11 15:27:20.103691
[15:27:20][I]: Universal Time: 2024-03-11 18:27:20.103739
[15:27:22][I]: Pipeline Version: 1.3.7 (latest)
[15:27:22][I]: Found 21 fits files in /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_GP/imaging.
[15:27:22][I]: No special reduced data path defined. Proceeding with defaults.
[15:27:22][W]: Reduction folder doesn't exist.
[15:27:22][I]: Created directory for reduced data.
[15:27:22][I]: New directory: /Users/cmontalban/Desktop/SOARdata/Haro_5-2/2023-03-09/2023-03-09_GP/imaging/RED
[15:27:22][C]: Unable to determine which camera was used.
[15:27:22][I]: Make sure you only have 'Blue' or 'Red' camera data only, not both.

And the output in the RED directory is empty.

Here is the header of one BIAS image:

TELESCOP= 'SOAR 4.1m'
INSTRUME= 'ghts_blue_imager'
OBJECT = 'BIAS_200ATTN0_IM_2x2'
NOTES = ''
INSTCONF= 'Blue'
CAMSWV = 'Beta 29'
CONSWV = '8359,10'
INSTSWV = 'GSCS 2.0.17.6.6-Alpha'
CAMERA = 'Si 600-658 '
RA = '08:24:36.242' / right ascension [hh:mm:ss.sss]
DEC = '-17:33:38.374' / declination [dd:mm:ss.sss]
AIRMASS = 1.660000 / airmass at approx. start of exposure
UT = '22:13:10.480' / time at approx. start of exposure [UTC]
FOCUS = -894.210000 / SOAR telescope focus
MOUNT_AZ= 90.007236 / SOAR mount azimuth
MOUNT_EL= 37.012836 / SOAR mount elevation
ROTATOR = 90.000000 / Nasymth cage rotator angle [deg]
POSANGLE= 0.000000 / position angle [deg. E of N]
SEEING = -1.000000 / DIMM seeing [arcsec]
LST = '04:39:36.550' / Local Sidereal Time [hh:mm:ss.sss]
OBSRA = '18:09:43.711' / target right ascension [hh:mm:ss.sss]
OBSDEC = '-2:19:47.500' / target declination [hh:mm:ss.sss]
DOME_AZ = 180.006404 / SOAR Dome Shutter azimuth
HA = '-03:45:00.695' / Hour Angle [hh:mm:ss.sss]
ADCSTAT = 'IN' / ADC Status
ADCPOS = 'DONE_50.000' / ADC Position
LAMP_HGA= 'FALSE' / Hg(Ar)
LAMP_NE = 'FALSE' / Neon
LAMP_AR = 'FALSE' / Argon
LAMP_FE = 'FALSE' / Iron
LAMP_CU = 'FALSE' / Copper
LAMP_QUA= 'FALSE' / Quartz
LAMP_QPE= 0.000000 / Quartz Percent
LAMP_BUL= 'FALSE' / Bulb
LAMP_DOM= 'FALSE' / Dome
LAMP_DPE= 0.000000 / Dome Percent
CAM_ANG = 0.000111 / camera angle [deg]
GRT_ANG = 0.661300 / grating angle [deg]
CAM_TARG= 0.000000 / camera target [deg]
GRT_TARG= 0.661000 / grating target [deg]
CAM_FOC = 402 / camera focus
COLL_FOC= 1001 / collimator focus
FILTER = 'NO_FILTER' / primary filter wheel
FILTER2 = 'NO_FILTER' / secondary filter wheel
GRATING = 'NO_GRATING' / VPH grating [1/mm]
SLIT = 'NO_MASK' / slit [arcsec]
COL_TEMP= 14.296875 / coll ext temp (deg C)
CAM_TEMP= 16.390625 / cam ext temp (deg C)
WAVMODE = 'IMAGING' / Wavelength
EXPTIME = 0.000000 / integration time
RDNOISE = 4.740000 / CCD readnoise [e-]
GAIN = 1.400000 / CCD gain [e-/ADU]
OBSTYPE = 'BIAS' / observation type
OBSERVER= 'Diego Gomez'
PROPOSAL= ''
EQUINOX = 2000.000000 / equinox of coordinates
ROI = 'Imaging 2x2'
ENVWIN = 5.104080 / Wind Speed [km/hr] at start of exposure
ENVPRE = 737.750000 / Atmospheric Pressure [hPS] at start of exposur
ENVDIR = 136.100000 / Wind Direction at start of exposure
ENVTEM = 15.000000 / Outside Temperature [C] at start of exposure
ENVHUM = 63.500000 / Relative Humidity at start of exposure
DISPAXIS= 1
DETSIZE = '[1:4096,1:4096]'
TRIMSEC = '[1:3096,1:3096]'
CCDSIZE = '[1:4096,1:4096]'
CCDSUM = '2 2'
OBSID =
CONFID =
REQNUM =
PROPID = 'calibrate'
SITEID = 'sor'
TELID = '4m0a'
RLEVEL = 0
BLKUID = 0
L1PUBDAT= '2023-03-09T22:13:12.153296Z'
IMAGEID = '77'
HEADERVE= '1.0.1'
BLOCK_ID= 1678399931761036
COMMENT Triggered, Exp Time= 00 - Image Transform: Flip Y - Image Transform: Fl
COMMENT ip Y, Saved as: 0077_BIAS_09-03-2023.fits
END

@Allonck
Copy link
Collaborator

Allonck commented Mar 11, 2024

It occurs to me that the problem happens because the whole process revolves around science images. If the process happens the same as in the Jupyter notebook (Create and bias-correct science images, create masterflat [masterflats are bias-corrected when created] and then normalize it for science image correction), then it makes sense that calibration images are not processed.

Changing this I understand may be complex and I don't know to what extent advantageous, perhaps to better structure the software so that it can at least process calibration images?

If instead, all images are processed by BIAS, then the masterFLAT is created with these images processed by BIAS and everything is corrected by masterFLAT (except the BIAS) [After being normalized] could be a solution to this problem.

@Allonck
Copy link
Collaborator

Allonck commented Mar 22, 2024

Fixed in #395

@Allonck Allonck closed this as completed Mar 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@simontorres simontorres

@Allonck Allonck

Labels
March 2024 Sprint Solve long standing issues and implement new requirements. Need Testing
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@simontorres @Allonck