|
9 | 9 | "source": [
|
10 | 10 | "# Scaling CMA-MAE on the Sphere Benchmark\n",
|
11 | 11 | "\n",
|
12 |
| - "_This tutorial is part of the series of pyribs tutorials! See [here](https://docs.pyribs.org/en/latest/tutorials.html) for the list of all tutorials and the order in which they should be read._\n", |
| 12 | + "_This tutorial is part of the series of pyribs tutorials! See [here](https://docs.pyribs.org/en/stable/tutorials.html) for the list of all tutorials and the order in which they should be read._\n", |
13 | 13 | "\n",
|
14 | 14 | "One challenge in applying CMA-MAE is its quadratic time complexity. Internally, CMA-MAE relies on CMA-ES, which has $\\Theta(n^2)$ time and space complexity due to operations on an $n \\times n$ covariance matrix. Thus, CMA-ES and hence CMA-MAE can be computationally intractable when a problem involves millions or even just thousands of parameters. To address this issue, [Tjanaka 2022](https://scalingcmamae.github.io) proposes to replace CMA-ES with more efficient evolution strategies (ESs), resulting in several variants of CMA-MAE. The CMA-MAE variants proposed in the paper and supported in pyribs are as follows:\n",
|
15 | 15 | "\n",
|
16 | 16 | "- LM-MA-MAE: Replaces CMA-ES with [Limited Memory Matrix Adaptation ES (LM-MA-ES)](https://ieeexplore.ieee.org/document/8410043), which has $\\Theta(kn)$ runtime.\n",
|
17 | 17 | "- sep-CMA-MAE: Replaces CMA-ES with [separable CMA-ES (sep-CMA-ES)](https://inria.hal.science/inria-00270901v1/document), which constrains the covariance matrix to be diagonal and has $\\Theta(n)$ runtime.\n",
|
18 | 18 | "- OpenAI-MAE: Replaces CMA-ES with [OpenAI-ES](https://arxiv.org/abs/1703.03864), which performs search by sampling from an isotropic Gaussian and has $\\Theta(n)$ runtime.\n",
|
19 | 19 | "\n",
|
20 |
| - "This tutorial shows how these different variants of CMA-MAE can be accessed by changing the `es` parameter in the [`EvolutionStrategyEmitter`](https://docs.pyribs.org/en/latest/api/ribs.emitters.EvolutionStrategyEmitter.html).\n", |
| 20 | + "This tutorial shows how these different variants of CMA-MAE can be accessed by changing the `es` parameter in the [`EvolutionStrategyEmitter`](https://docs.pyribs.org/en/stable/api/ribs.emitters.EvolutionStrategyEmitter.html).\n", |
21 | 21 | "\n",
|
22 |
| - "_Note: This tutorial is based on the [CMA-MAE tutorial](https://docs.pyribs.org/en/latest/tutorials/cma_mae.html). As such, we skim over details like how to set up the archives. For more details on these steps, please refer to that tutorial._" |
| 22 | + "_Note: This tutorial is based on the [CMA-MAE tutorial](https://docs.pyribs.org/en/stable/tutorials/cma_mae.html). As such, we skim over details like how to set up the archives. For more details on these steps, please refer to that tutorial._" |
23 | 23 | ]
|
24 | 24 | },
|
25 | 25 | {
|
|
154 | 154 | "source": [
|
155 | 155 | "## Emitter Setup with the `es` Parameter\n",
|
156 | 156 | "\n",
|
157 |
| - "Exactly like in the regular CMA-MAE, scalable variants of CMA-MAE are built with the [`EvolutionStrategyEmitter`](https://docs.pyribs.org/en/latest/api/ribs.emitters.EvolutionStrategyEmitter.html). The key difference is the choice of ES, which is indicated with the `es` parameter. `es` has the following options:\n", |
| 157 | + "Exactly like in the regular CMA-MAE, scalable variants of CMA-MAE are built with the [`EvolutionStrategyEmitter`](https://docs.pyribs.org/en/stable/api/ribs.emitters.EvolutionStrategyEmitter.html). The key difference is the choice of ES, which is indicated with the `es` parameter. `es` has the following options:\n", |
158 | 158 | "\n",
|
159 | 159 | "- `lm_ma_es`: Indicates LM-MA-ES\n",
|
160 | 160 | "- `sep_cma_es`: Indicates sep-CMA-ES\n",
|
|
193 | 193 | "id": "9df5e6fc-1840-4365-8f71-9b22f54731b0",
|
194 | 194 | "metadata": {},
|
195 | 195 | "source": [
|
196 |
| - "It is also possible to pass in `es` as a class, as the string options to `es` are simply translated to a corresponding ES class (the documentation for all ES classes is [here](https://docs.pyribs.org/en/latest/api/ribs.emitters.opt.html)). All ES classes are subclasses of [`EvolutionStrategyBase`](https://docs.pyribs.org/en/latest/api/ribs.emitters.opt.EvolutionStrategyBase.html)." |
| 196 | + "It is also possible to pass in `es` as a class, as the string options to `es` are simply translated to a corresponding ES class (the documentation for all ES classes is [here](https://docs.pyribs.org/en/stable/api/ribs.emitters.opt.html)). All ES classes are subclasses of [`EvolutionStrategyBase`](https://docs.pyribs.org/en/stable/api/ribs.emitters.opt.EvolutionStrategyBase.html)." |
197 | 197 | ]
|
198 | 198 | },
|
199 | 199 | {
|
|
224 | 224 | "id": "80c2f899-2d38-468b-a42d-c3a162855a67",
|
225 | 225 | "metadata": {},
|
226 | 226 | "source": [
|
227 |
| - "Some ESs also accept kwargs. For instance, in [LMMAEvolutionStrategy](https://docs.pyribs.org/en/latest/api/ribs.emitters.opt.LMMAEvolutionStrategy.html), the size of the approximation can be adjusted with the `n_vectors` parameter. kwargs can be passed to the ESs with the `es_kwargs` parameter in `EvolutionStrategyEmitter`, as shown below." |
| 227 | + "Some ESs also accept kwargs. For instance, in [LMMAEvolutionStrategy](https://docs.pyribs.org/en/stable/api/ribs.emitters.opt.LMMAEvolutionStrategy.html), the size of the approximation can be adjusted with the `n_vectors` parameter. kwargs can be passed to the ESs with the `es_kwargs` parameter in `EvolutionStrategyEmitter`, as shown below." |
228 | 228 | ]
|
229 | 229 | },
|
230 | 230 | {
|
|
269 | 269 | "id": "cb8ef8be-945e-475e-a7c0-7ea2c0039e51",
|
270 | 270 | "metadata": {},
|
271 | 271 | "source": [
|
272 |
| - "Finally, [`GradientArborescenceEmitter`](https://docs.pyribs.org/en/latest/api/ribs.emitters.GradientArborescenceEmitter.html) also has as `es` parameter. However, it is less common to use a scalable ES in `GradientArborescenceEmitter` since the ES only operates in objective-measure coefficient space, and that space usually has only a few dimensions." |
| 272 | + "Finally, [`GradientArborescenceEmitter`](https://docs.pyribs.org/en/stable/api/ribs.emitters.GradientArborescenceEmitter.html) also has as `es` parameter. However, it is less common to use a scalable ES in `GradientArborescenceEmitter` since the ES only operates in objective-measure coefficient space, and that space usually has only a few dimensions." |
273 | 273 | ]
|
274 | 274 | },
|
275 | 275 | {
|
|
0 commit comments