Update Sphinx documentation, commit e38afb1 [skip ci].

_sources/changelog.rst

@@ -40,6 +40,9 @@ New
 Changes
 ~~~~~~~
 
+- The custom NumPy memory manager that prevents memory leaks
+  with ``real`` arrays is now disabled by default
+  (`#129 <https://github.com/bluescarni/heyoka.py/pull/129>`__).
 - The step callbacks are now deep-copied in multithreaded
   :ref:`ensemble propagations <ensemble_prop>`
   rather then being shared among threads. The aim of this change

changelog.html

@@ -540,6 +540,9 @@ <h3>New<a class="headerlink" href="#new" title="Permalink to this heading">#</a>
 <section id="changes">
 <h3>Changes<a class="headerlink" href="#changes" title="Permalink to this heading">#</a></h3>
 <ul class="simple">
+<li><p>The custom NumPy memory manager that prevents memory leaks
+with <code class="docutils literal notranslate"><span class="pre">real</span></code> arrays is now disabled by default
+(<a class="reference external" href="https://github.com/bluescarni/heyoka.py/pull/129">#129</a>).</p></li>
 <li><p>The step callbacks are now deep-copied in multithreaded
 <a class="reference internal" href="notebooks/ensemble_mode.html#ensemble-prop"><span class="std std-ref">ensemble propagations</span></a>
 rather then being shared among threads. The aim of this change

notebooks/Batch mode overview.html

@@ -1306,7 +1306,7 @@ <h2>Ensemble propagations<a class="headerlink" href="#ensemble-propagations" tit
 </div>
 </div>
 <div class="cell_output docutils container">
-<img alt="../_images/74927d01e6034c387ef4b6a2900bcf815df1e30022ddebf766c6e7e0c1080b9e.png" src="../_images/74927d01e6034c387ef4b6a2900bcf815df1e30022ddebf766c6e7e0c1080b9e.png" />
+<img alt="../_images/f29fc07daa622ba2fca337814af20bb3620e22f95ddc98b2908cf63f261112c2.png" src="../_images/f29fc07daa622ba2fca337814af20bb3620e22f95ddc98b2908cf63f261112c2.png" />
 </div>
 </div>
 </section>

notebooks/Customising the adaptive integrator.html

@@ -560,8 +560,8 @@ <h2>Compact mode<a class="headerlink" href="#compact-mode" title="Permalink to t
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>CPU times: user 4.66 s, sys: 61 ms, total: 4.72 s
-Wall time: 4.72 s
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>CPU times: user 7.15 s, sys: 75.4 ms, total: 7.23 s
+Wall time: 7.29 s
 </pre></div>
 </div>
 </div>
@@ -576,8 +576,8 @@ <h2>Compact mode<a class="headerlink" href="#compact-mode" title="Permalink to t
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>CPU times: user 181 ms, sys: 65 µs, total: 181 ms
-Wall time: 181 ms
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>CPU times: user 237 ms, sys: 0 ns, total: 237 ms
+Wall time: 236 ms
 </pre></div>
 </div>
 </div>

notebooks/Optimal Control of the Lotka-Volterra equations.html

@@ -1022,9 +1022,9 @@ <h2>Implementing a single shooting method<a class="headerlink" href="#implementi
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/tmp/ipykernel_10435/2332358995.py:11: DeprecationWarning: Conversion of an array with ndim &gt; 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/tmp/ipykernel_12891/2332358995.py:11: DeprecationWarning: Conversion of an array with ndim &gt; 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
   ta.pars[6] = np.heaviside(switching_function(x_0,y_0,lx_0,ly_0, ps), 1.)
-/tmp/ipykernel_10435/2332358995.py:14: DeprecationWarning: Conversion of an array with ndim &gt; 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
+/tmp/ipykernel_12891/2332358995.py:14: DeprecationWarning: Conversion of an array with ndim &gt; 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
   ta.state[0] = x_0; ta.state[1] = y_0; ta.state[2] = lx_0; ta.state[3] = ly_0
 </pre></div>
 </div>
@@ -1066,7 +1066,7 @@ <h2>Implementing a single shooting method<a class="headerlink" href="#implementi
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/tmp/ipykernel_10435/685660463.py:12: DeprecationWarning: Conversion of an array with ndim &gt; 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/tmp/ipykernel_12891/685660463.py:12: DeprecationWarning: Conversion of an array with ndim &gt; 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
   ta.pars[6] = np.heaviside(switching_function(x_0,y_0,lx_0,ly_0, ps), 1.)
 </pre></div>
 </div>

notebooks/Periodic orbits in the CR3BP.html

@@ -661,7 +661,7 @@ <h2>Putting all together and integrating some initial conditions<a class="header
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>--- 10.439852237701416 seconds --- to build the Taylor integrator
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>--- 15.712157011032104 seconds --- to build the Taylor integrator
 </pre></div>
 </div>
 </div>
@@ -692,7 +692,7 @@ <h2>Putting all together and integrating some initial conditions<a class="header
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>--- 0.061354875564575195 seconds --- to propagate
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>--- 0.06314849853515625 seconds --- to propagate
 </pre></div>
 </div>
 </div>
@@ -750,7 +750,7 @@ <h2>Putting all together and integrating some initial conditions<a class="header
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x7efdb2552b30&gt;
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x7f78688fec50&gt;
 </pre></div>
 </div>
 <img alt="../_images/38d7685311d4c0702c894ea7ed3a7c8adce228eb1b7d34e0f04fe4a77277e9da.png" src="../_images/38d7685311d4c0702c894ea7ed3a7c8adce228eb1b7d34e0f04fe4a77277e9da.png" />

notebooks/arbitrary_precision.html

@@ -569,7 +569,7 @@ <h2>Getting to know your real self<a class="headerlink" href="#getting-to-know-y
 </div>
 </div>
 </div>
-<p>Please be aware that NumPy support is to be considered <strong>experimental</strong> at this time (see the “Limitations and caveats” section below).</p>
+<p>Please be aware that NumPy support is to be considered <strong>experimental</strong> at this time (see the <a class="reference internal" href="#mp-caveats"><span class="std std-ref">“Limitations and caveats”</span></a> section below).</p>
 </section>
 <section id="numerical-integration">
 <h2>Numerical integration<a class="headerlink" href="#numerical-integration" title="Permalink to this heading">#</a></h2>
@@ -797,11 +797,26 @@ <h2>Compiled functions<a class="headerlink" href="#compiled-functions" title="Pe
 </div>
 </section>
 <section id="limitations-and-caveats">
-<h2>Limitations and caveats<a class="headerlink" href="#limitations-and-caveats" title="Permalink to this heading">#</a></h2>
+<span id="mp-caveats"></span><h2>Limitations and caveats<a class="headerlink" href="#limitations-and-caveats" title="Permalink to this heading">#</a></h2>
 <p>Similarly to <a class="reference internal" href="ext_precision.html"><span class="doc std std-doc">extended-precision datatypes</span></a>, NumPy support for the <code class="docutils literal notranslate"><span class="pre">real</span></code> dtype is quite limited: while basic features such as array slicing, indexing, arithmetic, special functions and explicit conversions to/from other types are supported and working as intended, more advanced features such as random number generation, nontrivial linear algebra, etc. will not work with <code class="docutils literal notranslate"><span class="pre">real</span></code>.</p>
-<p>Additionally, because <code class="docutils literal notranslate"><span class="pre">real</span></code> wraps a C++ class with nontrivial lifetime management, heyoka.py must (ab)use a <a class="reference external" href="https://numpy.org/neps/nep-0049.html">recently-introduced NumPy memory management feature</a> in order to ensure that <code class="docutils literal notranslate"><span class="pre">real</span></code> objects stored in a NumPy array are properly initialised before access and properly destroyed when the array is garbage collected. There are a couple of known issues in the custom memory management scheme employed by heyoka.py.</p>
-<p>To begin with, under certain circumstances (i.e., typically during the conversion of <code class="docutils literal notranslate"><span class="pre">real</span></code> arrays to/from another dtypes), heyoka.py will be forced to allocate on-the-fly <code class="docutils literal notranslate"><span class="pre">real</span></code> values that are never deallocated until the termination of the program (thus leading to potential memory leaks). See <a class="reference external" href="https://github.com/numpy/numpy/issues/22686">here</a> for more details. Although in practice this behaviour does not seem to create issues in most cases, long-running computations that frequently call into the NumPy API might eventually lead to memory exhaustion.</p>
-<p>Secondly, the fact that heyoka.py overrides the default NumPy memory management functions with custom implementations renders heyoka.py incompatible with other packages which might also be overriding the NumPy memory allocation functions. We are not aware at this time of any widely-used package that overrides the default NumPy memory allocation functions.</p>
+<p>Additionally, because <code class="docutils literal notranslate"><span class="pre">real</span></code> wraps a C++ class with nontrivial lifetime management, creating and manipulating arrays with <code class="docutils literal notranslate"><span class="pre">real</span></code> dtype will result by default in <strong>memory leaks</strong>. Specifically, when a <code class="docutils literal notranslate"><span class="pre">real</span></code> array is garbage-collected, the memory allocated by the individual <code class="docutils literal notranslate"><span class="pre">real</span></code> objects in the array will <strong>not</strong> be freed.</p>
+<p>This leaky behaviour can be avoided by (ab)using a <a class="reference external" href="https://numpy.org/neps/nep-0049.html">recently-introduced NumPy memory management feature</a>. Specifically, it is possible to invoke the <code class="docutils literal notranslate"><span class="pre">install_custom_numpy_mem_handler()</span></code> function to install a custom memory manager for NumPy arrays that will ensure that all <code class="docutils literal notranslate"><span class="pre">real</span></code> objects are properly destroyed when an array is garbage-collected:</p>
+<div class="cell docutils container">
+<div class="cell_input docutils container">
+<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="n">hy</span><span class="o">.</span><span class="n">install_custom_numpy_mem_handler</span><span class="p">()</span>
+</pre></div>
+</div>
+</div>
+</div>
+<p>Note, however, that this custom memory manager results in a measurable performance penalty when (de)allocating NumPy arrays for <strong>all</strong> dtypes (and not only <code class="docutils literal notranslate"><span class="pre">real</span></code>). Hence, the custom memory manager is disabled by default.</p>
+<p>Note that the original NumPy memory manager can be restored via the <code class="docutils literal notranslate"><span class="pre">remove_custom_numpy_mem_handler()</span></code> function:</p>
+<div class="cell docutils container">
+<div class="cell_input docutils container">
+<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="n">hy</span><span class="o">.</span><span class="n">remove_custom_numpy_mem_handler</span><span class="p">()</span>
+</pre></div>
+</div>
+</div>
+</div>
 <p>If you encounter issues with NumPy support for the <code class="docutils literal notranslate"><span class="pre">real</span></code> class, please do not hesitate to <a class="reference external" href="https://github.com/bluescarni/heyoka.py/issues/new/choose">open an issue</a>.</p>
 </section>
 </section>

notebooks/ensemble_mode.html

@@ -561,10 +561,10 @@ <h2>A simple example<a class="headerlink" href="#a-simple-example" title="Permal
  Taylor order            : 20
  Dimension               : 2
  Time                    : 20
- State                   : [0.04760306011272994, 0.05487222312847258],
+ State                   : [0.05023369765600805, 0.056819409529623166],
  &lt;taylor_outcome.time_limit: -4294967299&gt;,
- 0.20197701082694358,
- 0.21826551051657198,
+ 0.19984740911611365,
+ 0.2189204289521889,
  97,
  None)
 </pre></div>
@@ -616,7 +616,7 @@ <h2>A simple example<a class="headerlink" href="#a-simple-example" title="Permal
 </div>
 </div>
 <div class="cell_output docutils container">
-<img alt="../_images/aaf5c630363e5d1a6024f690ed37880df8f2caeb485deb2281d9dd4e810730d3.png" src="../_images/aaf5c630363e5d1a6024f690ed37880df8f2caeb485deb2281d9dd4e810730d3.png" />
+<img alt="../_images/00f4323467b843cbb4a7cf94e498a5a80e5b48f7e9247b434b911e09fd7f56a7.png" src="../_images/00f4323467b843cbb4a7cf94e498a5a80e5b48f7e9247b434b911e09fd7f56a7.png" />
 </div>
 </div>
 </section>

notebooks/second_integral.html

@@ -595,7 +595,7 @@ <h2>Running the experiments<a class="headerlink" href="#running-the-experiments"
 </div>
 </div>
 <div class="cell_output docutils container">
-<img alt="../_images/ea3f04302eaa81b133d1292c576620e2dd9a591e2ab109d892639b5e28d943a7.png" src="../_images/ea3f04302eaa81b133d1292c576620e2dd9a591e2ab109d892639b5e28d943a7.png" />
+<img alt="../_images/0cead41a95fb17e9124e45a3219193dea38008bb6f8a953781f86331900102ff.png" src="../_images/0cead41a95fb17e9124e45a3219193dea38008bb6f8a953781f86331900102ff.png" />
 </div>
 </div>
 <p>This Poincare’ section is suggesting that the integral of motion <span class="math notranslate nohighlight">\(\mathcal{C}\)</span> does indeed exist, because the intersections of all random orbits are clearly tracing curves on the <span class="math notranslate nohighlight">\(x=0\)</span> plane.</p>
@@ -610,7 +610,7 @@ <h2>Running the experiments<a class="headerlink" href="#running-the-experiments"
 </div>
 </div>
 <div class="cell_output docutils container">
-<img alt="../_images/96a838fdf0fbb793681d398d94817f014cee71941b61c5fc2ebe1eb71b2d0374.png" src="../_images/96a838fdf0fbb793681d398d94817f014cee71941b61c5fc2ebe1eb71b2d0374.png" />
+<img alt="../_images/cb61f488d87637e6f987fe09d40ffbf9a21c811a0c3a1272d1eef75c36a98f7a.png" src="../_images/cb61f488d87637e6f987fe09d40ffbf9a21c811a0c3a1272d1eef75c36a98f7a.png" />
 </div>
 </div>
 <p>The qualitative picture is now radically different. Although islands of stable motion persist, now most trajectories seem to be densely filling the plane, rather than tracing curves on it. Let us further increase the energy level:</p>
@@ -621,7 +621,7 @@ <h2>Running the experiments<a class="headerlink" href="#running-the-experiments"
 </div>
 </div>
 <div class="cell_output docutils container">
-<img alt="../_images/e8603a65639c8f5752cbf4aad119d7f2257467eca3552300051a99f14f2201df.png" src="../_images/e8603a65639c8f5752cbf4aad119d7f2257467eca3552300051a99f14f2201df.png" />
+<img alt="../_images/49971a71351f07f2166610fdd9b90162510597543302862789851bad2421b79f.png" src="../_images/49971a71351f07f2166610fdd9b90162510597543302862789851bad2421b79f.png" />
 </div>
 </div>
 <p>At higher energies, the densely-filling behaviour (which Henon and Heiles call <em>ergodic</em>) is even more prevalent than before.</p>

notebooks/sympy_interop.html

@@ -587,8 +587,8 @@ <h1>Interoperability with SymPy<a class="headerlink" href="#interoperability-wit
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>CPU times: user 2.19 s, sys: 1.22 ms, total: 2.19 s
-Wall time: 2.19 s
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>CPU times: user 2.37 s, sys: 710 µs, total: 2.38 s
+Wall time: 2.37 s
 </pre></div>
 </div>
 <div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>(&lt;taylor_outcome.time_limit: -4294967299&gt;,
@@ -607,8 +607,8 @@ <h1>Interoperability with SymPy<a class="headerlink" href="#interoperability-wit
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>CPU times: user 2.21 s, sys: 544 µs, total: 2.21 s
-Wall time: 2.21 s
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>CPU times: user 2.45 s, sys: 448 µs, total: 2.45 s
+Wall time: 2.45 s
 </pre></div>
 </div>
 <div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>(&lt;taylor_outcome.time_limit: -4294967299&gt;,