Examples of analysis of CESM LENS data publicly available on Amazon S3 (us-west-2 region) using xarray and dask.
Try these notebooks on Pangeo Binder. Note that the session is ephemeral. Your home directory will not persist, so remember to download your notebooks if you made changes that you need to use at a later time!
The main catalog URL is:
https://raw.githubusercontent.com/NCAR/cesm-lens-aws/main/intake-catalogs/aws-cesm1-le.json
This catalog is an ESM collection catalog. The data is stored in Zarr format and meant to be opened with Xarray.
Using this catalog requires the following package versions:
- Intake-esm >=
v2020.11.4
To open the catalog and load a data set from Python, you can run the following code:
In [1]: import intake
In [2]: col = intake.open_esm_datastore("https://raw.githubusercontent.com/NCAR/cesm-lens-aws/main/intake-catalogs/aws-cesm1-le.json")
In [3]: col
Out[3]: <aws-cesm1-le catalog with 55 dataset(s) from 391 asset(s)>
In [4]: col.df.head()
Out[4]:
component frequency experiment ... dim_per_tstep start end
0 atm daily CTRL ... 2.0 0402-01-01 12:00:00 2200-12-31 12:00:00
1 atm daily CTRL ... 2.0 0402-01-01 12:00:00 2200-12-31 12:00:00
2 atm daily CTRL ... 2.0 0402-01-01 12:00:00 2200-12-31 12:00:00
3 atm daily CTRL ... 2.0 0402-01-01 12:00:00 2200-12-31 12:00:00
4 atm daily CTRL ... 2.0 0402-01-01 12:00:00 2200-12-31 12:00:00
[5 rows x 9 columns]
In [5]: col_subset = col.search(experiment="RCP85", frequency="monthly", variable=["hi", "aice"])
In [6]: dsets = col_subset.to_dataset_dict(zarr_kwargs={"consolidated": True}, storage_options={"anon": True})
--> The keys in the returned dictionary of datasets are constructed as follows:
'component.experiment.frequency'
|████████████████████████████████████████████████████████████████████████████████████████████████████| 100.00% [2/2 00:00<00:00]
In [7]: dsets.keys()
Out[7]: dict_keys(['ice_sh.RCP85.monthly', 'ice_nh.RCP85.monthly'])
In [8]: ds = dsets['ice_sh.RCP85.monthly']
In [9]: ds
Out[9]:
<xarray.Dataset>
Dimensions: (d2: 2, member_id: 40, ni: 320, nj: 76, time: 1140)
Coordinates:
* member_id (member_id) int64 1 2 3 4 5 6 7 8 ... 34 35 101 102 103 104 105
* time (time) object 2006-01-16 12:00:00 ... 2100-12-16 12:00:00
time_bounds (time, d2) object dask.array<chunksize=(1140, 2), meta=np.ndarray>
Dimensions without coordinates: d2, ni, nj
Data variables:
aice (member_id, time, nj, ni) float32 dask.array<chunksize=(1, 1140, 76, 320), meta=np.ndarray>
hi (member_id, time, nj, ni) float32 dask.array<chunksize=(1, 1140, 76, 320), meta=np.ndarray>
Attributes:
comment3: seconds elapsed into model date: 0
conventions: CF-1.0
nco_openmp_thread_number: 1
source: sea ice model: Community Ice Code (CICE)
NCO: 4.3.4
contents: Diagnostic and Prognostic Variables
comment2: File written on model date 20060201
comment: All years have exactly 365 days
intake_esm_dataset_key: ice_sh.RCP85.monthly- For details about intake-esm API, see the reference documentation
- CESM LENS on AWS Site
The source code for https://doi.org/10.26024/wt24-5j82 resides in the site directory of this repository.
The site is built with sphinx.
To build the site locally, please use conda to set up a build environment with all dependencies.
First, make a local clone of this source repository on your machine. For example:
git clone https://github.com/NCAR/cesm-lens-awsSet up your a conda environment:
conda env create -f site/environment.yml
conda activate cesm-lens-aws-site
bash site/install-extension.shYou can then build the site with:
make live