Merge pull request easybuilders#352 from branfosj/eb5

EasyBuild 5

Author: branfosj

docs/easybuild-v5/deprecated-functionality.md

@@ -110,13 +110,18 @@ The following checksum types are deprecated and should no longer be used: `md5`,
 
 ## `EnvironmentModulesC` or `EnvironmentModulesTcl` modules tools {: #modules-tools }
 
-*(more info soon)*
+The use of the older [Environment Modules](https://modules.readthedocs.io/) versions is deprecated.
+This corresponds to setting the module tool in EasyBuild to either `EnvironmentModulesC` or
+`EnvironmentModulesTcl`.
+
+Those using Environment Modules should use at least version 4.3 or newer.
 
 ---
 
 ## GC3Pie as job backend {: #gc3pie-job-backend }
 
-*(more info soon)*
+The use of the GC3Pie job backend is deprecated. Those [generating jobs][submitting_jobs] from
+EasyBuild should use one of the other available backends instead, such as Slurm or PBS Python.
 
 ---
 
@@ -128,7 +133,8 @@ The following checksum types are deprecated and should no longer be used: `md5`,
 
 ## `COMPILER*_FLAGS` attributes in `Compiler` class {: #compiler-constants }
 
-*(replaced by `Compiler.COMPILER*_OPTIONS`, more info soon)*
+In the `Compiler` class, the `COMPILER*_FLAGS attributes` were renamed to change `FLAGS` to `OPTIONS`.
+This change was made to make it clearer as to the function of these attributes.
 
 ---
 

docs/easybuild-v5/enhancements.md

@@ -63,14 +63,17 @@ If you would like to see other types of easyconfigs added as templates, please
 
 ## Mark support for installing extensions in parallel as stable (no longer experimental) {: #parallel-extensions-install-stable }
 
-*(more info soon)*
+The support for [installing extensions in parallel][installing_extensions_in_parallel]
+no longer requires the use of the `--experimental` option. Currently this supports
+installing R extensions in parallel.
 
 ---
 
 
 ## Mark easystack support as stable (no longer experimental) {: #easystack-stable }
 
-*(more info soon)*
+[Easystack files][easystack] are now considered stable and no longer require the use
+of the `--experimental` option.
 
 ---
 
@@ -168,7 +171,12 @@ status codes are reported in the corresponding logs.
 
 ## Copy build directory and/or log file(s) if installation failed to path specified via `--failed-install-build-dirs-path` or `--failed-install-logs-path` {: #copy-build-log-failed-installs }
 
-*(more info soon)*
+Often local storage (such as `/tmp`) or `/dev/shm` is used for the build path. When these are used with `--job`
+then the build files and log files will be removed when the job finishes. If a build fails this can make
+determining the reason for failure difficult. The options `--failed-install-build-dirs-path` and
+`--failed-install-logs-path` have been added to copy the build directory of log files to a specified path in
+the event of a failed build.
+
 
 ---
 
@@ -186,9 +194,13 @@ See [separate dedicated section](../implementing-easyblocks.md#module_load_envir
 ---
 
 
-## `keep-debug-symbols` configuration option to set default value of '`debug`' toolchain option {: #keep-debug-symbols }
+## `--keep-debug-symbols` configuration option to set default value of '`debug`' toolchain option {: #keep-debug-symbols }
 
-*(more info soon)*
+A configuration option `--keep-debug-symbols` was added to set the default value for the `debug` toolchain option.
+Enabling this will add `-g` to flags such as `$CXXFLAGS`.
+
+!!! Warning
+    For some software enabling debug symbols will result in significant increase in disk space usage.
 
 ---
 
@@ -316,7 +328,7 @@ This option is also available as easyconfig parameter
 
 ## Support not using `$PYTHONPATH` to specify the location of installed Python packages (via `--prefer-python-search-path`) { : #PYTHONPATH-vs-EBPYTHONPREFIXES }
 
-*(more info soon)*
+See [dedicated page](../python-search-path.md).
 
 ---
 

docs/easybuild-v5/index.md

@@ -4,12 +4,6 @@ A high-level overview of changes in EasyBuild version 5.0.0 is listed below.
 
 Click on a particular item for more information.
 
-!!! warning
-
-    (Tue 18 March 2025)
-    The sections linked to from this overview below are known to be incomplete.
-    Extra information will be gradually added over the next couple of days.
-
 ---
 
 ## Breaking changes {: #breaking-changes }
@@ -76,7 +70,7 @@ Various significant enhancements are included in EasyBuild v5.0, including:
 - [Copy build directory and/or log file(s) if installation failed to path specified via `--failed-install-build-dirs-path` or `--failed-install-logs-path`](enhancements.md#copy-build-log-failed-installs)
 - [Specify changes that should be made by generated module files via `module_load_environment`](../implementing-easyblocks.md#module_load_environment)
 - [Add support for alternate easyconfig parameters/templates/constants](enhancements.md#alternative-easyconfig-parameters-templates-constants)
-- [`keep-debug-symbols` configuration option to set default value of '`debug`' toolchain option](enhancements.md#keep-debug-symbols)
+- [`--keep-debug-symbols` configuration option to set default value of '`debug`' toolchain option](enhancements.md#keep-debug-symbols)
 - [Provide control over how generated modules update search path for header files (via `--module-search-path-headers`)](enhancements.md#module-search-path-headers)
 - [Provide control over how EasyBuild specifies path to header files during installation (via `--search-path-cpp-headers`)](enhancements.md#search-path-cpp-headers)
 - [Provide control over how EasyBuild specifies path to libraries during installation (via `--search-path-linker`)](enhancements.md#search-path-linker)
@@ -94,12 +88,12 @@ Functionality that was deprecated a while ago in EasyBuild v4.x has been ***remo
 
 - [EasyBuild bootstrap script](removed-functionality.md#bootstrap_script) 
 - [Experimental support for `.yeb` easyconfig](removed-functionality.md#yeb)
-- [`accept-eula` configuration setting](removed-functionality.md#accept-eula)
+- [`--accept-eula` configuration setting](removed-functionality.md#accept-eula)
 - [`--wait-on-lock` configuration setting](removed-functionality.md#wait-on-lock)
 - [`is_generic_easyblock` function from `easybuild.framework.easyconfig.easyconfig`](removed-functionality.md#easyconfig-is_generic_easyblock)
 - [`copytree`, `rmtree2` functions from `easybuild.filetools`](removed-functionality.md#filetools-copytree-rmtree2)
 - [`fetch_extension_sources` method in `EasyBlock` class](removed-functionality.md#EasyBlock-fetch_extension_sources)
-- [`mod_exists_regex_template` options in `ModulesTool.exist` method](removed-functionality.md#ModulesTool-exist-mod_exists_regex_template)
+- [`mod_exists_regex_template` option in `ModulesTool.exist` method](removed-functionality.md#ModulesTool-exist-mod_exists_regex_template)
 - [`Toolchain.add_dependencies` method](removed-functionality.md#Toolchain-add_dependencies)
 - [`disable_templating` + `default_fallback` options in `get_easyblock_class` function](removed-functionality.md#get_easyblock_class)
 - [`skip_lower` option from `template_constant_dict` function](removed-functionality.md#template_constant_dict)
@@ -143,13 +137,6 @@ Some functionality is being deprecated in EasyBuild v5.0, and is scheduled to be
 
 - [Toolchain support policy](../policies/toolchains.md)
 
----
-
-## FAQ
-
-*(coming soon)*
-
-
 ---
 
 ## Other links

docs/easybuild-v5/removed-functionality.md

@@ -42,93 +42,108 @@ format and little suggestion that anyone was using it at all.
 
 ---
 
-## `accept-eula` configuration setting] {: #accept-eula }
+## `--accept-eula` configuration setting] {: #accept-eula }
 
-*(more info soon)*
+The `--accept-eula` option was changed to `--accept_eula_for` in EasyBuild 4.3.4 where it was also enhanced to
+support reular expressions. Use of `--accept-eula` has now been removed.
 
 ---
 
 ## `--wait-on-lock` configuration setting {: #wait-on-lock }
 
-*(more info soon)*
+The `--wait-on-lock` option was replaced and split into `--wait-on-lock-limit` and `--wait-on-lock-interval` in
+EasyBuild 4.2.1. Use of `--wait-on-lock` has now been removed.
 
 ---
 
 ## `is_generic_easyblock` function from `easybuild.framework.easyconfig.easyconfig` {: #easyconfig-is_generic_easyblock }
 
-*(more info soon)*
+`is_generic_easyblock` was moved to `easybuild.tools.filetools` in EasyBuild 4.2.0. Use of `is_generic_easyblock`
+from `easybuild.framework.easyconfig.easyconfig` has been removed.
 
 ---
 
 ## `copytree`, `rmtree2` functions from `easybuild.filetools` {: #filetools-copytree-rmtree2 }
 
-*(more info soon)*
+The use of `rmtree2` was replaced by `remove_dir` and the use of `copytree` by `copy_dir` in EasyBuild 4.2.0.
+Use of `rmtree2` and `copytree`, from `easybuild.filetools`, has now been removed.
 
 ---
 
 ## `fetch_extension_sources` method in `EasyBlock` class {: #EasyBlock-fetch_extension_sources }
 
-*(more info soon)*
+`fetch_extension_sources` was replaced by `collect_exts_file_info`. in EasyBuild 4.5.0. Use of
+`fetch_extension_sources` has now been removed.
 
 ---
 
-## `mod_exists_regex_template` options in `ModulesTool.exist` method {: #ModulesTool-exist-mod_exists_regex_template }
+## `mod_exists_regex_template` option in `ModulesTool.exist` method {: #ModulesTool-exist-mod_exists_regex_template }
 
-*(more info soon)*
+The `mod_exists_regex_template` option in `ModulesTool.exist` was unused and has now been removed.
 
 ---
 
 ## `Toolchain.add_dependencies` method {: #Toolchain-add_dependencies }
 
-*(more info soon)*
+`Toolchain.add_dependencies` has been removed. The dependencies should be passed to the `prepare`
+method instead, using the `deps` argument.
 
 ---
 
 ## `disable_templating` + `default_fallback` options in `get_easyblock_class` function {: #get_easyblock_class }
 
-*(more info soon)*
+`disable_templating` was changed to be a function on the EasyConfig class in EasyBuild 4.4.0.
+`disable_templating` and the `default_fallback` option for `get_easyblock_class` have now been removed.
 
 ---
 
 ## `skip_lower` option from `template_constant_dict` function {: #template_constant_dict }
 
-*(more info soon)*
+The `skip_lower` option in `template_constant_dict` had no effect in EasyBuild 4 and has now been
+removed.
 
 ---
 
 ## `use_git_am` option for `apply_patch` function {: #apply_patch-use_git_am }
 
-*(more info soon)*
+The `use_git_am` option for `apply_patch` was renamed to `use_git` in EasyBuild 4.3.0 . This option
+has now been removed.
 
 ---
 
 ## `skip_symlinks` option for `adjust_permissions` function {: #adjust_permissions-skip_symlinks }
 
-*(more info soon)*
+The `skip_symlinks` option in the `adjust_permissions` function has been removed. Symlinks are not
+followed in this function.
 
 ---
 
 ## `log_error` option in `which` function {: #which-log_error }
 
-*(more info soon)*
+The `log_error` option in the `which` function was changed to `on_error` in EasyBuild 4.4.0 to allow
+enhanced options on what to do when an error is encountered in this function. This option has now
+been removed.
 
 ---
 
 ## `descr` option for `simple_option` function {: #simple_option-descr }
 
-*(more info soon)*
+The `descr` option in the `simple_option` function was removed. This has been unused in EasyBuild for
+a long time.
 
 ---
 
 ## `dummy` toolchain {: #dummy-toolchain }
 
-*(more info soon)*
+The `dummy` toolchain was replaced by the `SYSTEM` toolchain in EasyBuild 4. The `dummy` toolchain has
+now been removed. As part of this the `--add-system-to-minimal-toolchains` option replaces
+`--add-dummy-to-minimal-toolchains`.
 
 ---
 
 ## Support for 32-bit targets {: #32-bit-targets }
 
-*(more info soon)*
+EasyBuild has support for 32-bit targets for `imkl` versions before 11. These have now been removed.
 
 ---
 

docs/easystack-files.md

@@ -4,7 +4,7 @@ This documentation covers aspects of specifying a software stack to
 install with Easybuild with *easystack files*.
 
 !!! note
-    Some breaking changes were made to the experimental easystack support in EasyBuild v4.7.0.
+    Some breaking changes were made to the easystack support in EasyBuild v4.7.0.
 
     Easystack files must now use the `easyconfigs` top-level key to list *easyconfig filenames*,
     as opposed to the `software` top-level key and separate subkeys like `version`, `versionsuffix` and
@@ -21,7 +21,7 @@ specify to EasyBuild what to install.
 To build software with *Easystack*, type:
 
 ``` shell
-eb --easystack example.yaml --experimental
+eb --easystack example.yaml
 ```
 
 where `example.yaml` is the file with specifications that you just
@@ -136,7 +136,7 @@ easyconfigs:
 and you run with
 
 ``` shell
-eb --experimental --easystack my_easystack.yaml --dry-run
+eb --easystack my_easystack.yaml --dry-run
 ```
 
 this will have the same effect as running
@@ -163,7 +163,7 @@ they take priority over the the ones on the command line, if the same configurat
 For example, running:
 
 ``` shell
-eb --experimental --easystack my_easystack.yaml --disable-debug
+eb --easystack my_easystack.yaml --disable-debug
 ```
 
 will effectively cause `PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb` to be installed with debug logging enabled,

docs/installing-extensions-in-parallel.md

@@ -11,10 +11,6 @@ in the background, to exploit the availability of multiple cores.
 To let EasyBuild install extensions in parallel, the
 `parallel-extensions-install` configuration option must be enabled.
 
-In addition, since the support for installing extensions in parallel is
-an experimental feature, the `experimental` configuration option must be
-also be enabled (see also [Experimental features][experimental_features]).
-
 How many extension installations should be started (at maximum) at the
 same time is controlled by the `parallel` configuration option.
 
@@ -45,9 +41,6 @@ There are a couple of caveats and known issues with installing
 extensions in parallel, which we hope to resolve in future EasyBuild
 releases.
 
-This is also why the support for installing extensions in parallel is
-currently marked as an experimental feature.
-
 ### Only works for R extensions {: #installing_extensions_in_parallel_caveats_known_issues_only_r }
 
 The support for installing extensions in parallel currently only works

docs/submitting-jobs.md

@@ -24,17 +24,17 @@ by the software dependency graph(s).
 
 The job backend to be used can be specified using the `--job-backend` EasyBuild configuration option.
 
-Since EasyBuild 3.8.0, three backends are supported:
+Three backends are supported:
 
-* `GC3Pie` *(default)* (supported since EasyBuild 2.2.0)
-    * `GC3Pie` version 2.5.0 (or more recent) required (<https://gc3pie.readthedocs.org>)
-    * works with different resource managers and job schedulers, including TORQUE/PBS, Slurm, etc.
-    * **note**: requires that a GC3Pie configuration file is provided, see [Configuring the job backend][submitting_jobs_cfg_job_backend_config]
+* `Slurm` *(default)*
+    * requires Slurm version 17.0 (or more recent), (<https://slurm.schedmd.com/>)
 * `PbsPython`
     * `pbs_python` version 4.1.0 (or more recent) required (<https://oss.trac.surfsara.nl/pbs_python>)
     * **note**: requires TORQUE resource manager (<https://adaptivecomputing.com/cherry-services/torque-resource-manager>)
-* `Slurm` (supported since EasyBuild 3.8.0)
-    * requires Slurm version 17.0 (or more recent), (<https://slurm.schedmd.com/>)
+* `GC3Pie` *(deprecated)*
+    * `GC3Pie` version 2.5.0 (or more recent) required (<https://gc3pie.readthedocs.org>)
+    * works with different resource managers and job schedulers, including TORQUE/PBS, Slurm, etc.
+    * **note**: requires that a GC3Pie configuration file is provided, see [Configuring the job backend][submitting_jobs_cfg_job_backend_config]
 
 
 ### Configuring the job backend (`--job-backend-config`) {: #submitting_jobs_cfg_job_backend_config }
@@ -59,8 +59,8 @@ which job backend is being used:
 * if the `PbsPython` job backend is used, the (most common) number of available cores per workernode in the
   target resource is determined; this usually results in jobs requesting full workernodes (at least in terms of cores)
   by default
-* if the `GC3Pie` or `Slurm` job backend is used, the requested number of cores is left unspecified,
-  which results in falling back to the default mechanism used by GC3Pie/Slurm to pick a number of cores;
+* if the `Slurm` or `GC3Pie` job backend is used, the requested number of cores is left unspecified,
+  which results in falling back to the default mechanism used by Slurm/GC3Pie to pick a number of cores;
   most likely, this results in single-core jobs to be submitted by default
 
 
@@ -79,7 +79,7 @@ can be specified via the `--job-deps-type` configuration setting:
 The default value for `-job-deps-type` depends on the job backend being used
 (see [Configuring the job backend][submitting_jobs_cfg_job_backend_config):
 
-* for the `GC3Pie` and `Slurm` backends, `--job-deps-type=abort_on_error` is the default;
+* for the `Slurm` and `GC3Pie` backends, `--job-deps-type=abort_on_error` is the default;
 * for the `PbsPython` backend, `--job-deps-type=always_run` is the default (because of historical reasons,
   and for the sake of backward compatibility)