Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Moon code #108

Merged
merged 1 commit into from
Feb 3, 2025
Merged

Remove Moon code #108

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
235 changes: 26 additions & 209 deletions database/spectroscopic-production-database.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,109 +27,60 @@
"## DR1 TO DO\n",
"\n",
"* Update desimodules release/Jupyter Kernel\n",
"* Currently the `desi` user is needed for testing. Switch back to `desi_public`."
"* Currently the `desi` user is needed for testing. Switch back to `desi_public`.\n",
"* Working on removing the \"Moon\" exercise. This needs testing after NERSC is restored.\n",
"* Some imports may be unnecessary."
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 1,
"metadata": {
"tags": []
},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"sqlalchemy==2.0.30\n",
"specprodDB==1.4.0.dev260\n"
]
}
],
"source": [
"#\n",
"# Imports\n",
"#\n",
"import os\n",
"import sys\n",
"import itertools\n",
"from types import MethodType\n",
"# import sys\n",
"# import itertools\n",
"import numpy as np\n",
"import matplotlib\n",
"# import matplotlib\n",
"import matplotlib.pyplot as plt\n",
"from matplotlib.font_manager import fontManager, FontProperties\n",
"# from matplotlib.font_manager import fontManager, FontProperties\n",
"from sqlalchemy import __version__ as sqlalchemy_version\n",
"from sqlalchemy import inspect, and_\n",
"from sqlalchemy import inspect #, and_\n",
"from sqlalchemy.sql import func\n",
"from sqlalchemy.orm import aliased\n",
"# from sqlalchemy.orm import aliased\n",
"import astropy.units as u\n",
"#\n",
"# DESI software\n",
"#\n",
"from desiutil.log import get_logger, DEBUG\n",
"from desitarget.targetmask import (desi_mask, mws_mask, bgs_mask)\n",
"# from desisim.spec_qa import redshifts as dsq_z\n",
"from desisurvey import __version__ as desisurvey_version\n",
"from desisurvey.ephem import get_ephem, get_object_interpolator\n",
"from desisurvey.utils import get_observer\n",
"from desitarget.targetmask import desi_mask # , mws_mask, bgs_mask\n",
"from specprodDB import __version__ as specprodDB_version\n",
"import specprodDB.load as db\n",
"#\n",
"# Set the spectroscopic production run.\n",
"#\n",
"specprod = os.environ['SPECPROD'] = 'iron' # Change this to 'guadalupe' if needed.\n",
"#\n",
"# Initialize ephemerides, to find Moon, etc.\n",
"#\n",
"os.environ['DESISURVEY_OUTPUT'] = '/global/cfs/cdirs/desi/public/epo/example_files'\n",
"ephem = get_ephem()\n",
"#\n",
"# get_ephem() will run freeze_iers(), so we import these after that.\n",
"#\n",
"from astropy.time import Time\n",
"from astropy.coordinates import ICRS\n",
"#\n",
"# Working directory.\n",
"#\n",
"workingdir = os.getcwd()\n",
"print(f'sqlalchemy=={sqlalchemy_version}')\n",
"print(f'specprodDB=={specprodDB_version}')\n",
"print(f'desisurvey=={desisurvey_version}')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This function will compute various Moon paramters needed below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def moon(self, mjd, ra, dec):\n",
" \"\"\"Compute relative location of the Moon.\n",
" \n",
" Parameters\n",
" ----------\n",
" mjd : float\n",
" Time of observation\n",
" ra : float\n",
" Right Ascension\n",
" dec : float\n",
" Declination\n",
" \n",
" Returns\n",
" -------\n",
" tuple\n",
" Moon separation, Moon altitude, Moon illumination fraction\n",
" \"\"\"\n",
" observation_time = Time(mjd, format='mjd')\n",
" position = ICRS(ra=ra*u.deg, dec=dec*u.deg)\n",
" zenith = get_observer(observation_time, alt=90 * u.deg, az=0 * u.deg).transform_to(ICRS())\n",
" alt = 90 * u.deg - position.separation(zenith)\n",
" moon_dec, moon_ra = get_object_interpolator(self.get_night(observation_time), 'moon', altaz=False)(observation_time.mjd)\n",
" moon_position = ICRS(ra=moon_ra*u.deg, dec=moon_dec*u.deg)\n",
" moon_sep = position.separation(moon_position).to(u.deg).value\n",
" moon_alt = (90 * u.deg - moon_position.separation(zenith)).to(u.deg).value\n",
" moon_frac = ephem.get_moon_illuminated_fraction(observation_time.mjd).tolist()\n",
" return (moon_sep, moon_alt, moon_frac)\n",
"\n",
"ephem.moon = MethodType(moon, ephem)"
"print(f'specprodDB=={specprodDB_version}')\n"
]
},
{
Expand Down Expand Up @@ -708,142 +659,6 @@
"* Determine which tiles and exposures were involved in the determination of a HEALPixel-based redshift. *Hint*: the `fiberassign` table can connect `targetid` to `tileid`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Fly me to the Moon\n",
"\n",
"How does the Moon affect redshifts? First, let's find exposures that exposures that had the Moon above the horizon."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"moon_up = [e.expid for e in db.dbSession.query(db.Exposure).all() if ephem.moon(e.mjd, e.tilera, e.tiledec)[1] > 0]\n",
"len(moon_up)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"moon_up"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"So there are a few. But there is a subtle issue: redshifts are based on *all* exposures, and the exposures are deliberately designed to enforce the bright/dark dichotomy in targeting. There are execptions though: certain LRGs also get targeted in the BGS & MWS, so that's not hard to capture. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"various_lrgs = (desi_mask.LRG | desi_mask.BGS_ANY | desi_mask.MWS_ANY)\n",
"various_lrgs"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```SQL\n",
"SELECT z.targetid, z.z, z.zerr, z.zwarn\n",
" FROM iron.ztile AS z\n",
" JOIN iron.target AS t ON z.targetphotid = t.id\n",
" JOIN iron.fiberassign AS f ON t.targetid = f.targetid\n",
" JOIN iron.exposure AS e ON f.tileid == e.tileid\n",
" WHERE z.spgrp = 'cumulative'\n",
" AND t.desi_target & 3458764513820540929 != 0\n",
" AND e.expid IN (90250, 87505, 87382[...]);\n",
"```\n",
"\n",
"*WARNING*: This query will take several minutes, please be patient."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"q_up = db.dbSession.query(db.Ztile.targetid, db.Ztile.z, db.Ztile.zerr, db.Ztile.zwarn)\\\n",
" .join(db.Target, db.Target.id==db.Ztile.targetphotid)\\\n",
" .join(db.Fiberassign, db.Target.targetid==db.Fiberassign.targetid)\\\n",
" .join(db.Exposure, db.Fiberassign.tileid==db.Exposure.tileid)\\\n",
" .filter(db.Ztile.spgrp=='cumulative')\\\n",
" .filter(db.Target.desi_target.op('&')(various_lrgs) != 0)\\\n",
" .filter(db.Exposure.expid.in_(moon_up)).all()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```SQL\n",
"SELECT z.targetid, z.z, z.zerr, z.zwarn\n",
" FROM iron.ztile AS z\n",
" JOIN iron.target AS t ON z.targetphotid = t.id\n",
" JOIN iron.fiberassign AS f ON t.targetid = f.targetid\n",
" JOIN iron.exposure AS e ON f.tileid == e.tileid\n",
" WHERE z.spgrp = 'cumulative'\n",
" AND t.desi_target & 3458764513820540929 != 0\n",
" AND e.expid NOT IN (90250, 87505, 87382[...]);\n",
"```\n",
"\n",
"*WARNING*: This query will take several minutes, please be patient."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"q_dn = db.dbSession.query(db.Ztile.targetid, db.Ztile.z, db.Ztile.zerr, db.Ztile.zwarn)\\\n",
" .join(db.Target, db.Target.id==db.Ztile.targetphotid)\\\n",
" .join(db.Fiberassign, db.Target.targetid==db.Fiberassign.targetid)\\\n",
" .join(db.Exposure, db.Fiberassign.tileid==db.Exposure.tileid)\\\n",
" .filter(db.Ztile.spgrp=='cumulative')\\\n",
" .filter(db.Target.desi_target.op('&')(various_lrgs) != 0)\\\n",
" .filter(~db.Exposure.expid.in_(moon_up)).all()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Unfortunately however, the database currently only contains cumulative tile redshifts, not per-exposure redshifts, so it's not really meaningful to say whether the Moon was up or not. We'll just call this a work in progress."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"q_up"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"q_dn"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down Expand Up @@ -1101,7 +916,9 @@
"SELECT p.*, z.*, q3c_dist(p.ra, p.dec, 180.0, 0.0) AS radial_distance\n",
" FROM iron.photometry AS p JOIN iron.zpix AS z ON p.targetid = z.targetid\n",
" WHERE q3c_radial_query(p.ra, p.dec, 180.0, 0.0, 1.0/60.0); -- 1 arcmin\n",
"```"
"```\n",
"\n",
"Note that q3c functions are associated with the 'public' schema and therefore do not need to be schema-qualified in a pure SQL query. That is, always use `q3c_function_name()` not `iron.q3c_function_name()`. When translating to SQLAlchemy, public functions are associated with the module `sqlalchemy.sql.func` (imported in this notebook as `func`), so q3c functions become, *e.g.*, `func.q3c_dist()`."
]
},
{
Expand Down Expand Up @@ -1147,7 +964,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.14"
"version": "3.10.16"
}
},
"nbformat": 4,
Expand Down