Update Matching Notebooks

[jbkalmbach]

Update matching notebooks to use gcr-catalogs. Also make sure that they are doing what we want for DC2.

[jbkalmbach]

@drphilmarshall @jiwoncpark @dannygoldstein

This PR isn't quite ready but I just wanted it here for you all to follow along.

These notebooks will have do 2 things as mentioned in #477

[jbkalmbach]

Ok, these are ready for comments.

Basically, MatchingLensGalaxies.ipynb and MatchingLensGalaxies-SN.ipynb run first and use an empirical fit between velocity dispersion and redshift to stellar mass in order to assign each lens galaxy in OM10/Danny's Lensed SNe catalog a stellar mass. It then matches this lens galaxy to a galaxy in the extragalactic catalog based upon this stellar mass and uses the radius, SED and SED normalization magnitude of this galaxy for input to PhoSim.

If @drphilmarshall and @jiwoncpark could look over MatchingLensGalaxies.ipynb and make sure everything looks good for strongly lensed AGN and @dannygoldstein could look over MatchingLensGalaxies-SN.ipynb to make sure everything looks good for the strongly lensed SNe that'd be great.

The other notebook dc2_sprinkler_cache_production.ipynb matches the lensed systems to the AGN that will be replaced by OM10 systems and the non-AGN galaxies that will be replaced by strongly lensed SNe systems in the uDDF field. This involves matching based upon redshift and magnitude for the AGN and only redshift for the SNe. If all of you could also take a look at this notebook and make sure that the bounds and matching criteria are reasonable that would be good.

I think the final thing I discussed with @jiwoncpark was making sure that the final matched AGN systems are good for analysis in DC2 and this will have to wait until we run all these notebooks with the final DC2 but I have included data/cosmoDC2_v0.1_test_matched_AGN.fits in order to provide an example of the catalog we will want to test.

[drphilmarshall]

Nice work, @jbkalmbach ! Just looking at the matching lens galaxies notebook, I have some comments:

1. The BIC analysis suggests that only a small number of Gaussians are justified by the data, I think you are right to try to get away with just one. Still, I worry that with the SL2S sample so small, you will overfit the training sample. I didn't see any cross validation happening, to find the GMM parameters that minimize the generalized error, so I wasn't too surprised to see the model misfitting some of the test set. On the whole, I think it's unlikely that you are in danger of overfitting to the whole SL2S sample with 1, 2 or 3 Gaussians, so am inclined to recommend that, just this once, you don't bother with the training/test split, but instead just model the whole SL2S sample and go with that. What do you think? At least, it'd be good to see the training points as well as the test points, overlaid on the predictions in the corner plot.

2. I agree that the radius is still unlikely to be well fitted, so its maybe worth ignoring this from the start (having explained that you can't model it).

3. All plots need axis labels.

4. I would expect the lens galaxy SEDs to come out as being early type spectra with low metallicity and old (~few to 10 Gyr) ages. Is that what you see? The final example seems to have an "Exp" SED, which got me worried.

[jbkalmbach]

Hi @drphilmarshall. I have updated the notebooks based upon your comments and used the cosmoDC2 v1.0 galaxies.

[drphilmarshall]

This is looking pretty sweet, @jbkalmbach - I think the 1-Gaussian GMM is working well. You could clean up the notebook by either reducing the 2 and 3 Gaussian tests to a bare minimum, or just make sure that the narrative make sense in terms of your choices of the number of Gaussian components.Overall though I agree with your final conclusion - looks good!

[jbkalmbach]

I've updated the caching files for cosmoDC2_v1.0 and with approval on the notebooks will be merging this after 4pm Pacific time if nobody objects.

[drphilmarshall]

Great! Thanks Bryce.