Check for required keys

[raphaelshirley]

Bug report
I am trying to run an old Fortran para file. It hung at the prepare stage. I checked keys that were missing but were in the default cosmos config. After adding these:

ADDITIONAL_MAG
ADD_DUSTEM
EM_DISPERSION
EXTERNALZ_FILE
LIMITS_MAPP_CUT
LIMITS_MAPP_REF
LIMITS_MAPP_SEL
LIMITS_ZBIN
PDZ_OUT
PDZ_TYPE
RF_COLORS
RM_DISCREPANT_BD
VERBOSE
ZGRID_TYPE
Z_METHOD

With the default values it ran.

Should we have a list of required and allowed keys and a config checker?

Before submitting
Please check the following:

• I have described the situation in which the bug arose, including what code was executed, information about my environment, and any applicable data others will need to reproduce the problem.
• I have included available evidence of the unexpected behavior (including error messages, screenshots, and/or plots) as well as a descriprion of what I expected instead.
• If I have a solution in mind, I have provided an explanation and/or pseudocode and/or task list.

[johannct]

The code should not hang. Let's not wipe this under the rug by creating additional layers of code. How can one reproduce this behavior? And which is the parameter at the root of the problem?

[raphaelshirley]

import lephare as lp
config={'ADAPT_BAND': '6',
 'ADAPT_CONTEXT': '-1',
 'ADAPT_LIM': '17.5,22.5',
 'ADAPT_MODBIN': '0,2000',
 'ADAPT_ZBIN': '0.2,5',
 'ADD_EMLINES': 'YES',
 'AGE_RANGE': '0.,14.e9',
 'APPLY_SYSSHIFT': '0.05991,-0.22920,-0.08224,0.04400,0.01752,0.02375,-0.04355,0.27143,-0.00222,-0.00165,-0.06937,-0.00832,0.00000,0.00000,0.00000,0.00000,0.00000,0.00000,0.04685,0.04490,0.01923,0.01636,0.04057,0.08311,0.00945,-0.17312,-0.02210,0.01726,0.03319,-0.01975,0.00000,0.00000,0.00888,0.05750,-0.00681,0.00567,-0.04302,-0.16036,0.00000,0.00000,0.00000,0.00000,0.00000,0.00000,0.00000,0.00000,0.00000,0.19607,-0.03855',
 'APPLY_ZMAX': 'NO',
 'AUTO_ADAPT': 'NO',
 'BD_SCALE': '0',
 'CAT_FMT': 'MEME',
 'CAT_IN': 'bidon',
 'CAT_LINES': '-99,-99',
 'CAT_MAG': 'AB',
 'CAT_OUT': 'zphot.out',
 'CAT_TYPE': 'LONG',
 'CHI2_OUT': 'NO',
 'COL_NUM': '3',
 'COL_SEL': 'AND',
 'COL_SIGMA': '3',
 'COSMOLOGY': '70,0.3,0.7',
 'DZ_WIN': '0.25',
 'EBV_RANGE': '0,0.5',
 'EB_V': '0.,0.05,0.1,0.15,0.2,0.25,0.3,0.35,0.4',
 'EM_LINES': 'YES',
 'ERROR_ADAPT': 'NO',
 'ERR_FACTOR': '1.5',
 'ERR_SCALE': '0.01',
 'EXTINC_LAW': 'SMC_prevot.dat',
 'FAST_MODE': 'NO',
 'FILTER_CALIB': '0',
 'FILTER_FILE': 'filter_cosmos_OI.dat',
 'FILTER_LIST': 'cosmos/u_cfht.lowres,cosmos/B_subaru.lowres,cosmos/V_subaru.lowres',
 'FIR_CONT': '-1',
 'FIR_FREESCALE': 'YES',
 'FIR_LIB': 'NONE',
 'FIR_LMIN': '7.0',
 'FIR_SCALE': '-1',
 'FIR_SUBSTELLAR': 'NO',
 'FIT_PDFz': 'NO',
 'FORB_CONTEXT': '-1',
 'GAL_FSCALE': '1.',
 'GAL_LIB': 'LIBOI_GAL',
 'GAL_LIB_IN': 'LIBOI_GAL',
 'GAL_LIB_OUT': 'GALOI_SCOSMOS',
 'GAL_SED': 'examples/COSMOS_MOD.list',
 'GLB_CONTEXT': '-1',
 'INP_TYPE': 'M',
 'LIB_ASCII': 'YES',
 'MABS_CONTEXT': '-1',
 'MABS_FILT': '1,2,3,4',
 'MABS_METHOD': '1',
 'MABS_REF': '2',
 'MABS_ZBIN': '0,0.5,1,1.5,2,3,3.5,4',
 'MAGTYPE': 'AB',
 'MAG_ABS': '-24,-8',
 'MAG_REF': '2',
 'MIN_THRES': '0.05',
 'MOD_EXTINC': '1,30',
 'PARA_OUT': 'output.para',
 'PHYS_CONT': '-1',
 'PHYS_LIB': 'NONE',
 'PHYS_NMAX': '100000',
 'PHYS_SCALE': '-1',
 'QSO_FSCALE': '1',
 'QSO_LIB': 'LIB_QSO',
 'QSO_LIB_IN': 'LIB_QSO',
 'QSO_LIB_OUT': 'QSO_SCOSMOS',
 'QSO_SED': f'{lp.LEPHAREDIR}/sed/QSO/QSO_MOD.list',
 'SPEC_OUT': 'NO',
 'STAR_FSCALE': '3.432E-09',
 'STAR_LIB': 'LIB_SSTAR',
 'STAR_LIB_IN': 'LIB_SSTAR',
 'STAR_LIB_OUT': 'SSTAR_SCOSMOS',
 'STAR_SED': 'sed/STAR/STAR_MOD_ALL.list',
 'TRANS_TYPE': '1',
 'ZFIX': 'NO',
 'ZPHOTLIB': 'GALOI_SCOSMOS,STAR_SCOSMOS',
 'Z_INTERP': 'YES',
 'Z_RANGE': '0.025,7',
 'Z_STEP': '0.5,0.,7.',
 'FILTER_REP': f'{lp.LEPHAREDIR}/filt'}
lp.prepare(config)

Hangs on my system

[raphaelshirley]

I think it is hanging at the sedtolib GAL run.

[johannct]

in sedtolib.py you have the list of parameters used by sedtolib :

sedtolib_config_keys = [
    "GAL_SED",
    "GAL_FSCALE",
    "GAL_LIB",
    "SEL_AGE",
    "AGE_RANGE",
    "QSO_SED",
    "QSO_FSCALE",
    "QSO_LIB",
    "STAR_SED",
    "STAR_LIB",
    "STAR_FSCALE",
    "LIB_ASCII",
]

So the one missing would be SEL_AGE, which is used if there are BC03 templates.

[raphaelshirley]

It appears to be the VERBOSE keyword for lephare.MagGal.run. If that is not set then lephare.MagGal.run hangs

[johannct]

Was this solved? It does not seem to hang for me....

[raphaelshirley]

It is still hanging on my system for the QSO stage. It produces all the required outputs for stars and galaxies but takes a large amount of cpu for QSO and can not run the mag_gal stages. I am trying to narrow it down to create a minimal example to reproduce it on my machine...