Adapt ERA5 atmospheric forcing generation script #22
Conversation
- remove old download scripts - new download request - add year, month and output directory as input for download script
- adapt script to new CDSAPI - add argument parser for script - rename of SHUM to QBOT
- add argument parser - add input variables and defaults - add running_jobs functionality
replace unzip with python as unzip is not available at JSC compute nodes.
- Use python3 in hashbang for cross-platform and rely on it - Exit on error - No tab characters
ATM, the link https://cds.climate.copernicus.eu/how-to-api is deadish. Referring now also to ECMWF GitHub README.
|
|
||
| # Go into working directory and create temporary directory | ||
| if [ -z ${wrkdir} ];then | ||
| wrkdir=${iyear}-${imonth} |
There was a problem hiding this comment.
This cannot be right (and the behaviour will be hard to predict when there is more than one element in iyear or imonth).
Moreover, I don't like the whole naming here. When I see a word prefixed with i, I think of index or a scalar or maybe an integer, but NOT an array. It is also all confusing in mkforcing/extract_ERA5_meteocloud.sh and I think picking a natural naming scheme would help.
I'd use singular for elements and plural for arrays, so for month in ${months} &c.
(Interestingly, I'd read for imonth in ${months} correctly as well.)
There was a problem hiding this comment.
The date command parameters are not standard and fail on BSD (and probably macOSX).
Is a wrapper (in Bash) needed? Why not pass the user arguments to download_ERA5_input.py?
There was a problem hiding this comment.
Ah, the linked standard is maybe overly restricted. At least on OpenBSD there is no -I or -d.
Also breaks on OSX: no -I and -d does something else.
I don't know how to make it platform indenpendent in (Bourne) shell. Python would be nice (and better to use python3 and rely on shebang).
| @@ -0,0 +1,80 @@ | |||
| #!/usr/bin/env python | |||
| import calendar | |||
| import cdsapi | |||
There was a problem hiding this comment.
Should we include a pip install cdsapi in the README?
This would be the first time that this is needed (the rest we relied on modules).
edit: ah, except for the NCL script they need to evben do a conda install ncl!
To some extend, mess must maybe be accepted…
There was a problem hiding this comment.
Not needed; it is written in the linked documentation.
| mkdir -pv $tmpdir | ||
|
|
||
| if $lrmp; then | ||
| python -m zipfile -e ${pathdata}/download_era5_${year}_${month}.zip ${tmpdir} |
There was a problem hiding this comment.
FYI, there is an Lmod module UnZip, which may be nicer for a Bash script.
But I also don't want to fix what's not broken®.
There was a problem hiding this comment.
I get some warnings:
Process 2017-07-31-23 prun: 744
cdi warning (gribapiScanTimestep1): Record 1509 (name=z id=4.3.0 lev1=1 lev2=0) timestep 1: Inconsistent verification time!
...
There was a problem hiding this comment.
I use here years instead of the faulty iyear (to be fixed by @s-poll), so it's consistent with near-future changeset.
./extract_ERA5_meteocloud.sh years=2017 months=07works- ./extract_ERA5_meteocloud.sh years=2017 months=7` fails
./extract_ERA5_meteocloud.sh iyear=2017 imonth=7 ihour=(00 01)fails./extract_ERA5_meteocloud.sh iyear=2017 imonth=7 ihour=(00 01)fails- There is no
days, but maybe this was intentional - Check for
meteocloud_${year}_${month}.{grb,nc}at start of script is nice to have (to catch unnecessary recomputation) but optional
closes #13