You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[release/public-v2.2.0] Update MacOS configuration and compatibility (#953)
Updated modulefiles, scripts and configurations for running SRW v2.2.0 on MacOS platforms, summarized below. The changes include adaptation of the code for two MacOS architecthures, Intel/x86_64 and M1/arm64 (could be aarch64 if using Rosetta)
* modulefiles in ./modulefiles, ./modulefiles/tasks/macos/, srw_common.lua (successfully tested on all platforms but cloud)
* machine file in ./ush/machine/macos.yaml
* ./etc/lmod-setup.*
* scripts in ./ush/wrappers/* ./jobs/ , ./scripts/*.sh - the header changed to #!/usr/bin/env bash; a small bug in one of the scripts
* a patch file is applied for the ./sorc/CMakeLists.txt to build SRW on M1/arm64 machine; patch is located in ./patches/patch_macos_arm64_sorc_cmakelists.txt, and it is applied in ./devbuild.sh when needed
* small fixes in ./ush/job_preamble.sh, ./ush/preamble.sh (use mac-specific commands for runtime estimates)
* fix for ./ush/python_utils/filesys_cmds_vrfy.py: function vrfy_ln creates links with gnu-ln on MacOS (from coreutils)
* updates to the documentation for the MacOS users and adding options to install workflow conda environments.
---------
Co-authored-by: Natalie Perlin <[email protected]>
Co-authored-by: Natalie Perlin <[email protected]>
Co-authored-by: Gillian Petro <[email protected]>
Copy file name to clipboardexpand all lines: docs/UsersGuide/source/BuildingRunningTesting/BuildSRW.rst
+3-4
Original file line number
Diff line number
Diff line change
@@ -15,7 +15,7 @@ The Unified Forecast System (:term:`UFS`) Short-Range Weather (SRW) Application
15
15
16
16
To build the SRW App, users will complete the following steps:
17
17
18
-
#. :ref:`Install prerequisites <HPCstackInfo>`
18
+
#. :ref:`Install prerequisites <StackInfo>`
19
19
#. :ref:`Clone the SRW App from GitHub <DownloadSRWApp>`
20
20
#. :ref:`Check out the external repositories <CheckoutExternals>`
21
21
#. :ref:`Set up the build environment and build the executables <BuildExecutables>`
@@ -358,9 +358,8 @@ Additional Details for Building on MacOS or Generic Linux
358
358
359
359
The SRW App can be built on MacOS and generic Linux machines after the prerequisite software has been installed on these systems (via :term:`HPC-Stack` or :term:`spack-stack`). The installation for MacOS is architecture-independent and has been tested using both x86_64 and M1 chips (running natively). The following configurations for MacOS have been tested:
360
360
361
-
#. MacBookPro 2019, 2.4 GHz 8-core Intel Core i9 (x86_64), Monterey Sur 12.1, GNU compiler suite v.11.3.0 (gcc, gfortran, g++); mpich 3.3.2 or openmpi/4.1.2
362
-
#. MacBookAir 2020, M1 chip (arm64, running natively), 4+4 cores, Big Sur 11.6.4, GNU compiler suite v.11.3.0 (gcc, gfortran, g++); mpich 3.3.2 or openmpi/4.1.2
363
-
#. MacBook Pro 2015, 2.8 GHz Quad-Core Intel Core i7 (x86_64), Catalina OS X 10.15.7, GNU compiler suite v.11.2.0_3 (gcc, gfortran, g++); mpich 3.3.2 or openmpi/4.1.2
361
+
#. MacBookPro 2019, 2.4 GHz 8-core Intel Core i9 (x86_64), OS Monterey 12.6.1, 32 GB RAM; GNU compiler suite v.12.3.0 (gcc, gfortran, g++); openmpi/4.1.5
362
+
#. MacBookAir 2020, M1 chip (arm64, running natively), 4+4 cores, OS Ventura 13.0.1, 16 GB RAM; GNU compiler suite v.12.3.0 (gcc, gfortran, g++); openmpi/4.1.5
364
363
365
364
Several Linux builds have been tested on systems with x86_64 architectures.
On generic Mac and Linux systems, users need to create a conda |wflow_env| environment. The environment can be stored in a local path, which could be a default location or a user-specified location (e.g., ``$HOME/condaenv/venvs/`` directory). (To determine the default location, use the ``conda info`` command, and look for the ``envs directories`` list.) The following is a brief recipe for creating a virtual conda environment on non-Level 1 platforms. It uses the aarch64 (64-bit ARM) Miniforge for Linux and installs into $HOME/conda. Adjust as necessary for your target system.
195
+
On generic Mac and Linux systems, users need to create a conda |wflow_env| environment that contains python packages required for running the workflow. Other conda environments may need to be activated for running graphics generation tasks (|graphics_env|) or when testing the AQM/CMAQ (|cmaq_env|). Python packages in these other environments may conflict with those in |wflow_env|. The environments can be stored in a local path, which can be a default location or a user-specified location (e.g., ``$HOME/condaenv/venvs/`` directory). (To determine the default location, use the ``conda info`` command, and look for the ``envs directories`` list.)
196
+
These conda environments can be added to the existing python or conda modules.
196
197
197
-
.. code-block:: console
198
+
There are several options available for building virtual conda environments on non-Level 1 platforms. The examples in this section use the aarch64 (64-bit ARM) Miniforge for Linux and install into ``$HOME/conda``. Users should adjust as needed for their target system.
conda create -y -n workflow_tools -c local workflow_tools
208
-
conda activate workflow_tools
200
+
**Options:**
209
201
210
-
In future shells, you can activate and use this environment with:
211
202
212
-
.. code-block:: console
203
+
1) Users can add the following environment .yaml files:
204
+
205
+
a) workflow_tools.yaml for |wflow_env|
206
+
207
+
.. code-block:: console
208
+
209
+
name: workflow_tools
210
+
channels:
211
+
- conda-forge
212
+
- defaults
213
+
dependencies:
214
+
- python=3.9*
215
+
- boto3=1.22*
216
+
- black
217
+
- f90nml=1.4*
218
+
- jinja2=3.0*
219
+
- numpy=1.21*
220
+
- pylint
221
+
- pytest
222
+
- pyyaml=6.0*
223
+
- tox
224
+
225
+
b) regional_workflow.yaml for |graphics_env|
226
+
227
+
.. code-block:: console
228
+
229
+
name: regional_workflow
230
+
channels:
231
+
- conda-forge
232
+
- defaults
233
+
dependencies:
234
+
- python=3.9.*
235
+
- f90nml
236
+
- jinja2
237
+
- pyyaml
238
+
- scipy
239
+
- matplotlib=3.5.2*
240
+
- pygrib
241
+
- cartopy
242
+
243
+
c) regional_workflow_cmaq.yaml for |cmaq_env|
244
+
245
+
.. code-block:: console
246
+
247
+
name: regional_workflow_cmaq
248
+
channels:
249
+
- conda-forge
250
+
- defaults
251
+
dependencies:
252
+
- python=3.9.12
253
+
- f90nml=1.4*
254
+
- jinja2=3.0*
255
+
- pyyaml=6.0*
256
+
- scipy
257
+
- matplotlib
258
+
- pygrib
259
+
- cartopy
260
+
- netcdf4
261
+
- xarray
262
+
263
+
2) Users can instead install Miniforge. This example uses the aarch64 (64-bit ARM) Miniforge for Linux that installs into ``$HOME/conda``. Users should adjust as needed for their target system.
conda create -y -n workflow_tools -c local workflow_tools
276
+
conda activate workflow_tools
277
+
278
+
In future shells, you can activate and use this environment with:
279
+
280
+
.. code-block:: console
281
+
282
+
source ~/conda/etc/profile.d/conda.sh
283
+
conda activate workflow_tools
284
+
285
+
See the `workflow-tools repository <https://github.com/ufs-community/workflow-tools>`__ for additional documentation.
213
286
214
-
source ~/conda/etc/profile.d/conda.sh
215
-
conda activate workflow_tools
287
+
3) A third option is to build miniconda3 and create an Lmod modulefile that can be loaded with other modules during the workflow. The module can be added to the user's ``wflow_<platform>.lua`` modulefile, and the environments can be activated or deactivated as needed for a particular workflow task. A repository with full installation instructions, a modulefile template, and environment configuration files can be accessed in `NOAA-EPIC/miniconda3 repository <https://github.com/NOAA-EPIC/miniconda3>`__. Full instructions can be viewed in the `README.md file <https://github.com/NOAA-EPIC/miniconda3/edit/master/README.md>`__.
216
288
217
-
See the `workflow-tools repository <https://github.com/ufs-community/workflow-tools>`__ for additional documentation.
218
289
219
290
Modify a ``wflow_<platform>`` File
220
291
``````````````````````````````````````
@@ -632,7 +703,7 @@ This can be helpful when conducting multiple experiments with different types of
632
703
Plotting Configuration (optional)
633
704
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
634
705
635
-
An optional Python plotting task (PLOT_ALLVARS) can be activated in the workflow to generate plots for the :term:`FV3`-:term:`LAM` post-processed :term:`GRIB2`
706
+
An optional Python plotting task (plot_allvars) can be activated in the workflow to generate plots for the :term:`FV3`-:term:`LAM` post-processed :term:`GRIB2`
636
707
output over the :term:`CONUS`. It generates graphics plots for a number of variables, including:
Use the instructions in the `UPP User's Guide <https://upp.readthedocs.io/en/upp-srw-v2.2.0-docs/InputsOutputs.html#control-file>`__ to make modifications to the ``fv3lam.xml`` file and to remake the flat text file, called ``postxconfig-NT-fv3lam.txt`` (default), that the UPP reads.
0 commit comments