Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Threshold for flat center first guess #145

Open
detoma opened this issue Jan 25, 2024 · 1 comment
Open

Threshold for flat center first guess #145

detoma opened this issue Jan 25, 2024 · 1 comment
Assignees
Labels
bug Fix for existing functionality of the pipeline needs testing Draft done, testing needed to complete

Comments

@detoma
Copy link
Contributor

detoma commented Jan 25, 2024

The current code comp_find_flat_initial_guess.pro uses a fixed threshold of 250 for all times, wavelengths, and exposures. This thresholding should be applied to the dark corrected flats.

The threshold was determined empirically and seems to work pretty well for 1074 and 1079 even if the flat intensity changes over time (2013 to 2018) by over a factor of 2 (as the diffuser degrades, transmission decreases, and measured counts in the flats go down).
It seems all 1074 and 1079 data were taken at 250ms, but a better equation would have been:

threshold = exposure_time

An even better equation would have made this threshold a function of counts in the dark-corrected flats to take into account degradation over time, wavelength, and exposure.

For HeI 10803, the threshold is more complicated to determine than for Fe XIII 1074 and 1079 and the measured count must be considered. We have 250ms, 50ms, and 25ms exposure and different ND filters. The fixed 250 threshold does not always work. This may be part of why the centering and flat-fielding for HeI fails in some cases. We do not have the ND filter recorded early on, so it is not easy to set a fixed number. However the following threshold seems to work:

threshold = 20.0 * median(flat - dark)

This should be tested on HeI before reprocessing HeI data.

I also would like to test it on 2011- 2012 data. A better threshold may improve the centering code for 2011 and 2012.

Since the data in 2013-2018 were processed with the fixed value of 250, we need to add a comment in the code that explains a threshold of 250 seemed to work pretty well for data with exposure time of 250ms and was applied to the Fe 2013-2018 Fe XIII data, but the real threshold should be a function of exposure, wavelength, and time, i.e. a function of the measured counts, and a better equation was used later on.

@detoma detoma added this to the Decommissioning reprocessing milestone Jan 29, 2024
@detoma detoma added the bug Fix for existing functionality of the pipeline label Jan 29, 2024
@mgalloy mgalloy self-assigned this Jan 31, 2024
@mgalloy mgalloy added the needs testing Draft done, testing needed to complete label Jan 31, 2024
@detoma
Copy link
Contributor Author

detoma commented Feb 6, 2024

The fixed 250 threshold is too low but mine was too high. Let's try the following equation for 2011 and 2012

threshold = 10.0 * median(flat - dark)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fix for existing functionality of the pipeline needs testing Draft done, testing needed to complete
Projects
None yet
Development

No branches or pull requests

2 participants