Define expanded exposures table

[dkirkby]

The exposures table currently contains the minimal set of columns necessary to track survey progress, with no redundancy, for efficient and straightforward simulations.

However, a standardized set of additional columns are useful for downstream users. This issue to specify these columns and how they will be calculated / filled.

More generally, we need to determine what constitutes the authoritative history of exposures. In simulations, this is currently a record of the decisions taken by the next tile selector and online ETC. However, in operations, the raw data FITS headers (or a concurrently written db) should perhaps take precedence?

[dkirkby]

Expanded columns derivable from TILEID:

• TILERA
• TILEDEC
• EBVMED
• DESIGNHA

Expanded columns derivable from MJD:

• MOONFRAC
• MOONALT
• SUNALT
• LST

Expanded columns derivable from (MJD, TILEID):

• ALTITUDE
• AZIMUTH
• AIRMASS
• MOONSEP
• HA

For columns that depend on MJD, there are multiple options:

• shutter open / closed timestamp.
• exposure midpoint in time or SNR2.
• time when the ADC is set to null atmospheric dispersion.

Some quantities (e.g., SUNALT during twilight) might need more than one time value for interpolation.

The minimal exposures table includes SNR2FRAC, but this is based on the best info available when the next tile selector ran, updated by the online ETC estimate. How do we incorporate better information from QL or the offline pipeline?

[sbailey]

Thanks for itemizing this. Let's get this in for the 18.12 release (Dec 10 or before) to make the exposure list more useful for downstream usage. Two use cases on my radar:

1. including the necessary information to pass to specsim to simulate spectra for this exposure (minimalist columns + MOONFRAC, MOONALT, MOONSEP, and eventually SUNALT).
2. basic set of columns one would use for survey QA / survey progress tracking, independent of surveysim (i.e. all of your above columns).

Although this would come from surveysim for now, I'd like this to be a placeholder for a file that we could generate from real survey data by querying the OpsDB and/or scraping raw data headers. QL and/or offline pipeline could then provide row-matched companion files and/or augment this table with additional columns, but let's keep a clear distinction between the ICS+ETC information vs. downstream, and focus these updates on what ICS+ETC can provide.

[sbailey]

For the record, I've moved this off of the 18.12 critical path, and will use a workaround in the integration test in the meantime.

[michaelJwilson]

@changhoonhahn will have some requirements for the new bright time background model?