diff --git a/15-ConjGrad/Seminar15en.ipynb b/15-ConjGrad/Seminar15en.ipynb new file mode 100644 index 0000000..e469718 --- /dev/null +++ b/15-ConjGrad/Seminar15en.ipynb @@ -0,0 +1,1024 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "# Seminar 15\n", + "# Conjugate gradient method" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## Reminder\n", + "\n", + "1. Newton method\n", + "2. Convergence theorem\n", + "4. Comparison with gradient descent\n", + "5. Quasi-Newton methods" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## Linear system vs. unconstrained minimization problem\n", + "Consider the problem\n", + "\n", + "$$\n", + "\\min_{x \\in \\mathbb{R}^n} \\frac{1}{2}x^{\\top}Ax - b^{\\top}x,\n", + "$$\n", + "\n", + "where $A \\in \\mathbb{S}^n_{++}$.\n", + "From the necessary optimality condition follows\n", + "\n", + "$$\n", + "Ax^* = b\n", + "$$\n", + "\n", + "Also denote gradient $f'(x_k) = Ax_k - b$ by $r_k$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## How to solve linear system $Ax = b$?\n", + "\n", + "- Direct methods are based on the matrix decompositions:\n", + " - Dense matrix $A$: dimension is less then some thousands\n", + " - Sparse matrix $A$: dimension of the order $10^4 - 10^5$\n", + "- Iterative methods: the method of choice in many cases, the single approach which is appropriate for system with dimension $ > 10^6$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## Some history...\n", + "\n", + "M. Hestenes and E. Stiefel proposed *conjugate gradient method* (CG) \n", + "\n", + "to solve linear system in 1952 as **direct** method. \n", + "\n", + "Many years CG was considered only as theoretical interest, because\n", + "\n", + "- CG does not work with slide rule\n", + "- CG has not a lot advantages over Gaussian elimination while working with calculator\n", + "\n", + "CG method has to be considered as **iterative method**, i.e. stop after \n", + "\n", + "achieve required tolerance!\n", + "\n", + "More details see [here](https://www.siam.org/meetings/la09/talks/oleary.pdf)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## Conjugate directions method\n", + "\n", + "- Descent direction in gradient descent method is anti-gradient\n", + "- Convergence is veryyy **slow** for convex functions with pooorly conditioned hessian\n", + "\n", + "**Idea:** move along directions that guarantee converegence in $n$ steps.\n", + "\n", + "**Definition.** Nonzero vectors $\\{p_0, \\ldots, p_l\\}$ are called *conjugate* with tespect to matrix $A \\in \\mathbb{S}^n_{++}$, where \n", + "\n", + "$$\n", + "p^{\\top}_iAp_j = 0, \\qquad i \\neq j\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "**Claim.** For every initial guess vector $x_0 \\in \\mathbb{R}^n$ the sequence $\\{x_k\\}$, which is generated by conjugate gradient method, converges to solution of linear system $Ax = b$ not more than after $n$ steps.\n", + "\n", + "```python\n", + "def ConjugateDirections(x0, A, b, p):\n", + " \n", + " x = x0\n", + " \n", + " r = A.dot(x) - b\n", + " \n", + " for i in range(len(p)):\n", + " \n", + " alpha = - (r.dot(p[i])) / (p[i].dot(A.dot(p[i])))\n", + " \n", + " x = x + alpha * p[i]\n", + " \n", + " r = A.dot(x) - b\n", + " \n", + " return x\n", + "\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Example of conjugate directions\n", + "\n", + "- Eigenvectors of matrix $A$\n", + "- For every set of $n$ vectors one can perform analogue of Gram-Scmidt orthogonalization and get conjugate dorections\n", + "\n", + "**Q:** What is Gram-Schmidt orthogonalization process? :)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Geometrical interpretation (Mathematics Stack Exchange)\n", + "\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## Conjugate gradient method\n", + "\n", + "**Idea:** new direction $p_k$ is searched in the form $p_k = -r_k + \\beta_k p_{k-1}$, where $\\beta_k$ is based on the requirement of conjugacy of directions $p_k$ and $p_{k-1}$:\n", + "\n", + "$$\n", + "\\beta_k = \\dfrac{p^{\\top}_{k-1}Ar_k}{p^{\\top}_{k-1}Ap^{\\top}_{k-1}}\n", + "$$\n", + "\n", + "Thus, to get the next conjugate direction $p_k$ it is necessary to store conjugate direction $p_{k-1}$ and residual $r_k$ from the previous iteration. \n", + "\n", + "**Q:** how to select step size $\\alpha_k$?" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Convergence theorems\n", + "\n", + "**Theorem 1.** If matrix $A \\in \\mathbb{S}^n_{++}$ has only $r$ distinct eigenvalues, then conjugate gradient method converges in $r$ iterations.\n", + "\n", + "**Theorem 2.** The following convergence estimate holds\n", + "\n", + "$$\n", + "\\| x_{k+1} - x^* \\|_A \\leq \\left( \\dfrac{\\sqrt{\\kappa(A)} - 1}{\\sqrt{\\kappa(A)} + 1} \\right)^k \\|x_0 - x^*\\|_A,\n", + "$$\n", + "\n", + "where $\\|x\\|_A = x^{\\top}Ax$ and $\\kappa(A) = \\frac{\\lambda_n(A)}{\\lambda_1(A)}$ - condition number of matrix $A$\n", + "\n", + "**Remark:** compare coefficient of the linear convergence with \n", + "\n", + "corresponding coefficiet in gradient descent method." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Interpretations of conjugate gradient method\n", + "\n", + "- Gradient descent in the space $y = Sx$, where $S = [p_0, \\ldots, p_n]$, in which the matrix $A$ is digonal (or identity if the conjugate directions are orthonormal)\n", + "- Search optimal solution in the [Krylov subspace](https://stanford.edu/class/ee364b/lectures/conj_grad_slides.pdf) $\\mathcal{K}(A) = \\{b, Ab, A^2b, \\ldots \\}$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Improved version of CG method\n", + "\n", + "In practice the following equations for step size $\\alpha_k$ and coefficient $\\beta_{k}$ are used.\n", + "\n", + "$$\n", + "\\alpha_k = \\dfrac{r^{\\top}_k r_k}{p^{\\top}_{k}Ap_{k}} \\qquad \\beta_k = \\dfrac{r^{\\top}_k r_k}{r^{\\top}_{k-1} r_{k-1}}\n", + "$$\n", + "\n", + "**Q:** why do they better than base version?" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Pseudocode of CG method\n", + "```python\n", + "def ConjugateGradientQuadratic(x0, A, b):\n", + " \n", + " r = A.dot(x0) - b\n", + " \n", + " p = -r\n", + " \n", + " while np.linalg.norm(r) != 0:\n", + " \n", + " alpha = r.dot(r) / p.dot(A.dot(p))\n", + " \n", + " x = x + alpha * p\n", + " \n", + " r_next = r + alpha * A.dot(p)\n", + " \n", + " beta = r_next.dot(r_next) / r.dot(r)\n", + " \n", + " p = -r_next + beta * p\n", + " \n", + " r = r_next\n", + " \n", + " return x\n", + "\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## Using CG method in Newton method\n", + "\n", + "- To find descent direction in Newton method one has to solve the following linear system $H(x_k) h_k = -f'(x_k)$ \n", + "- If the objective function is strongly convex, then $H(x_k) \\in \\mathbb{S}^n_{++}$ and to solve this linear system one can use CG. In this case the merhod is called *inexact Newton method*.\n", + "- What's new?\n", + " - Explicit storage of hessian is not needed, it's enough to have function that perform multiplication hessian by vector\n", + " - One can control accuracy of solving linear system and do not solve it very accurate far away from minimizer. **Important**: inexact solution may be not descent direction! \n", + " - Convergence is only suprlinear if backtracking starts with $\\alpha_0 = 1$ similarly to Newton method" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## CG method for non-quadratic function\n", + "\n", + "**Idea:** use gradients instead of residuals $r_k$ and backtracking for search $\\alpha_k$ instead of analytical expression. We get Fletcher-Reeves method.\n", + "\n", + "```python\n", + "def ConjugateGradientFR(f, gradf, x0):\n", + " \n", + " x = x0\n", + " \n", + " grad = gradf(x)\n", + " \n", + " p = -grad\n", + " \n", + " while np.linalg.norm(gradf(x)) != 0:\n", + " \n", + " alpha = StepSearch(x, f, gradf, **kwargs)\n", + " \n", + " x = x + alpha * p\n", + " \n", + " grad_next = gradf(x)\n", + " \n", + " beta = grad_next.dot(grad_next) / grad.dot(grad)\n", + " \n", + " p = -grad_next + beta * p\n", + " \n", + " grad = grad_next\n", + " \n", + " if restart_condition:\n", + " \n", + " p = -gradf(x)\n", + " \n", + " return x\n", + "\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Convergence theorem\n", + "\n", + "**Theorem.** Assume \n", + "- level set $\\mathcal{L}$ is bounded\n", + "- there exists $\\gamma > 0$: $\\| f'(x) \\|_2 \\leq \\gamma$ for $x \\in \\mathcal{L}$\n", + "Then\n", + "\n", + "$$\n", + "\\lim_{j \\to \\infty} \\| f'(x_{k_j}) \\|_2 = 0\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Restarts\n", + "\n", + "1. To speed up convergence of CG one can use *restart* technique: remove stored history, consider current point as $x_0$ and run method from this point\n", + "2. There exist different conditions which indicate the necessity of restart, i.e.\n", + " - $k = n$\n", + " - $\\dfrac{|\\langle f'(x_k), f'(x_{k-1}) \\rangle |}{\\| f'(x_k) \\|_2^2} \\geq \\nu \\approx 0.1$\n", + "3. It can be shown (see Nocedal, Wright Numerical Optimization, Ch. 5, p. 125), that Fletcher-Reeves method without restarts can converge veryyy slow! \n", + "4. Polak-Ribiere method and its modifications have not this drawback" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Remarks\n", + "- The great notes \"An Introduction to the Conjugate Gradient Method Without the Agonizing Pain\" is available [here](https://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf)\n", + "- Besides Fletcher-Reeves method there exist other ways to compute $\\beta_k$: Polak-Ribiere method, Hestens-Stiefel method...\n", + "- The CG method requires to store 4 vectors, what vectors?\n", + "- The bottleneck is matrix by vector multiplication" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## Experiments" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Quadratic objective function " + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "A is normal matrix: ||AA* - A*A|| = 0.0\n" + ] + } + ], + "source": [ + "import numpy as np\n", + "n = 100\n", + "# Random\n", + "# A = np.random.randn(n, n)\n", + "# A = A.T.dot(A)\n", + "# Clustered eigenvalues\n", + "A = np.diagflat([np.ones(n//4), 10 * np.ones(n//4), 100*np.ones(n//4), 1000* np.ones(n//4)])\n", + "U = np.random.rand(n, n)\n", + "Q, _ = np.linalg.qr(U)\n", + "A = Q.dot(A).dot(Q.T)\n", + "A = (A + A.T) * 0.5\n", + "print(\"A is normal matrix: ||AA* - A*A|| =\", np.linalg.norm(A.dot(A.T) - A.T.dot(A)))\n", + "b = np.random.randn(n)\n", + "# Hilbert matrix\n", + "# A = np.array([[1.0 / (i+j - 1) for i in xrange(1, n+1)] for j in xrange(1, n+1)])\n", + "# b = np.ones(n)\n", + "\n", + "f = lambda x: 0.5 * x.dot(A.dot(x)) - b.dot(x)\n", + "grad_f = lambda x: A.dot(x) - b\n", + "x0 = np.zeros(n)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "#### Eigenvalues distribution " + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZkAAAEHCAYAAAB8yTv9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAGrtJREFUeJzt3VtsXded3/HfXzc78iWHlEfBtElHOjJmCnQCORQ9g85jTGMeAhSIJdtwHgoUdcS85C0xo/YhLYrCI7tvAWZGst86EOqILOZhMEBjatq3tjBFZ6boFPZETNyHuSilSHlIJqIl/vuw1yY3jzZJnX0ua5+9vh+AEM/em/ssLZxz/met/7qYuwsAgEE4FLsAAIDmIsgAAAaGIAMAGBiCDABgYAgyAICBIcgAAAaGIAMAGBiCDABgYAgyAICBORK7ALE988wzfurUqdjFAICRcvPmzf/n7r9y0HUjGWTM7IKkVUkTklbd/WrVe506dUoLCwt9KxsApMDMPnmU60YuyJhZS9Jldz8jad7MbpnZvLsvxS4bAGC3kcvJuPuqpHMdh+/EKAsAYH8jF2Sk7UAjM2tLWswfAwDqJWp3mZlNSLrk7i+XnLuonRZK293f6jg/JWla0szACwoAqCRKkAnB5dXwsF1y/qIkuftseNw2syvuPp1f4+7z2snJvEhOBgDqJ0p3mbsvuvuMpPf2uGS6OGIsBJApKQtQYXRZblFZiwYAUDO1G10WRo9NlJxaDV1kLUnjheMtSR8Mo2wARtvavfv607/4G63dux+7KLUw9sRRff0rXxzoc9QuyCjrPitL5N+RNOHub5nZhdCldkZZ4n+2mycwsxOSTkjS2bNney0vgBHx+//1J/r9/3YrdjFq4ze+8FSSQWZc5UOSVxUCQ7dBpcS3JX1fkm7fvt3jrQCMik/ubEiSnnr8iE4+9Vjk0sT3pfHPDfw56hhkhuEHkq5J0smTJz+KXBYAQ7IeusnOT3xR/+af/ZPIpUlDXefJjJcca0la7sfN3X3Z3T9294+PHEk1zgLpyYPM8WOHI5ckHXUMMgvKAkqncWUjyfrCzM6b2bWVlZV+3RJAza3feyBJeuIxvlwOS+2CTJi9vxRGmRW1wtyYfj3PnLt/Y2xsrF+3BFBz65tZS+ZJgszQxA4yZd1iknRZ0qX8QZi82bcAAyBNeXcZLZnhiTXjv61sAuWUpAkzuyLpZj4B092vmtnFwryYdnG2f5/KcF7S+Xb7oQUHADRUPj/mycfIyQxLlCATZvDvu+ZYL3vEPGIZ5iTNTU5OvjbI5wFQD/cfbOmXn21JoiUzTLG7ywBgKNY3H2z/TpAZHoIMgCSsF5aSeeIYQWZYkg0yDGEG0rKxWQgy5GSGJtkgwxBmIC1r93a6yxjCPDzJBhkAadnVXUaQGRqCDIAk5MOXjx05pKOH+egblmRrmpwMkJb1e8z2jyHZIENOBkjLzmx/kv7DlGyQAZCWPPHP8OXhIsgASEI+hJmk/3ARZAAkYY3FMaMgyABIwjqLY0aRbJBhdBmQlnVyMlEkG2QYXQakhe6yOJINMgDSwhDmOAgyAJJASyYOggyAJGyE/WSY8T9cBBkASdjuLiPxP1TJBhlGlwFpobssjmSDDKPLgHTcf7Cle/e3JNFdNmzJBhkA6VgvbFjG6LLhIsgAaLy1TTYsi4UgA6DxNtgVMxqCDIDGWysEmScZXTZUBBkAjUdOJh6CDIDGy1syjx05pCOH+dgbpmRrm3kyQDp2lvmnq2zYkg0yzJMB0rHOrpjRJBtkAKQj7y47fox8zLARZAA03sY9FseMhSADoPFYtyweggyAxiPxHw9BBkDj7ST+yckMG0EGQOOthZwM3WXDR5AB0Hh0l8VDkAHQeOvbQ5gJMsNGkAHQeHlO5klyMkNHkAHQeOvkZKJJNsiwdhmQDubJxJNskGHtMiANnz3Y0ub9LUkk/mNINsgASMM6u2JGRZAB0Gi7dsUk8T90BBkAjVbcFZMhzMNHkAHQaPnwZYnushgIMgAabVdOhv1kho4gA6DR8iDz+NFDOnKYj7xho8YBNNoaG5ZF1XWtm9nTksYl3XH3T8Ox1yWdkfSBu//n/hYRAKpbZyJmVFVaMm9JmpE0JUlm9p6klyX9UNJdM3upf8UDgN6ssThmVFVq/X13n5MkMzst6YKksUKr5nwfy1fKzC4oa02dkXTL3a8O+jkBjKadZf5J+sdQJcgUF/uakvRhHmAC761I+zOztqRVd58Nj2+a2YK7Lw7yeQGMpo1NFseMqdfE/wVJ/6njmPV4z4NMSJouPF5Q6LoDgE4sjhlXlVofM7OvSjon6Xll+Zi86+wNSVf6V7yHufusmc0XDk1Kuj7I5wQwura7y8jJRNF1SybkY0xZt9lpd//UzL6irHVxV9mH/iMxswkzKw0QZnbRzC6Enzc6yrAarmkrG+U2X3YPAKAlE1fVWr8l6UVlrZh33f1DMxtXNoT50/3/NAsukl4ND9sl5y9KWaslPG6b2RV3n+64dMbdX6z4fwCQABL/cXXdkgmtlrzl0MqPu/sNSc+b2amD7uHui+4+I+m9PS6ZLo4Yc/cldeRdQutmJvzeEgCUyBfIPE5LJooqtf6Kuz8rPTxc2d1vhHkyP6taoBAwJkpOrZrZlLvPm9mUpKt5t5myADRb9TmBJti8v6X/+D8+0d99+svYRamVv777C0l0l8VSpdYXCr+XDVcer1iWXFvSasnxO5ImzOyOpPclyWx7INu5bp7AzE5IOiFJZ8+erVxQoE5+9Jd/q3/3J38Zuxi1NXb8aOwiJKlKkCkGlrLhys9WLEtuXFlA6bQq6USYD9PrMOlvS/q+JN2+fbvHWwH18Ld3sxbM8WOHNfGP2Fa86B+2Pqev/uOTsYuRpCpBxszsdXd/V4WAY2bPSbocfuruB5KuSdLJkyc/ilwWoC/yUVS//oWn9Eev/3bk0gCZqkOYx0K31Ttm9ldmtizphqTr7v5nfShXWZdbS9JyH+4td19294/d/eMjR+inRTPsjKLiNY36qPRqdPe3zeyqsjkxbUlLkhbc/W4fyrSgwqi1gnFJfVs6JgxaON9uPzSCGhhJ+ZL2TzBUFzVSeVkZd7/r7jfc/Z3w711pe9n/ysKIsaWSYcmtfk66dPc5d//G2Bh912gGlrRHHXX1agx7yez31X9c2dyVdx/xlnuNRLss6ZJ25sFMaGduDoAS20GG5VNQI92+Gs9IuqlsSZmVjnNtZSsBlA0/3iUsBzOtbH7LhJldkXQzn4Dp7lfDsjJTyrrO2iWz/XtCdxmaZn2Tlgzqp9tX45KySZDfKjtpZi+ofPjxLmEG/8wB1wx0j5gwgGFucnLytUE+DzAs69vbDJOTQX10lZMJeZc9g0NYWuZ0r4UC0D1yMqijKkOY+zGCDECfsdow6qjXTcu2mdnTYULm8/265yCZ2Xkzu7ay0plaAkYT82RQR10FGTP7ipk9KPtRNhDgHe29snKtMIQZTbK15VoP2wwfP0ZOBvVRJfE/5+6vDKIwAKrZ+OzB9u+0ZFAnfU38A4hjI3SVSeRkUC9VEv8/3e98yMvUHjkZNMlaIcjQkkGd9C3xX3BpAPfsO3IyaJJ8joxESwb1suerMWyzvLDX+b3+TOUbmQEYoLVd3WUk/lEf+7VkliTdcPfDXfwckjQ3pLIDCPLhy0cPmx47QpBBfewZZEKS/5HXCwuLZ0oMDACGLl+37DiLY6Jm9n1FHpTkz5nZ5yVNhuX5XdIj/V1MLJCJJlljIiZqqnLiP8zwP2VmpySNKVuBeUnSq/0p2mCR+EeTbLBhGWqq6689odVyXdky/S4pX8uspWwbALrLgCFj3TLUVZWWzPckzYQk/yvuPh5+DoVzBy71D6C/WLcMdVUlyCy4+4fh9119TWGp/712uwQwINsblpH4R81UCTLFeTA3zeyljvOtHsoDoIK17ZwMQQb1UiXImJmdNrP3lG21/K/M7F/mAwE0Iol/oEl2Niwj8Y96qbJ22ZykCWVL+y9LekXZUjKrykaYXelnAQeFtcvQJOyKibrqOsiY2XNh+O+33P1Td19y92cljYVZ/382gHL2HUOY0SR5TobEP+qmSnfZ9bKDbMsMxJMvkPkEG5ahZqoEmRNm9qaZvV5YSgZARMyTQV1VeUWey5ebMbMXzOy0pKVR6SYDmoh5Mqirrl+RxfXMwrwYmdnnw1DmcUnz7v6zvpUQwL62tlwbmwxhRj31a9OyFyR9S9Jb6mLlZgC9y5P+EkOYUT9VRpe9FP59zsz+0MzuKAssV8LyMiOxMybQFHkrRqIlg/qp8oqcNbNbyrrG3pR0ehRHlrHUP5pi166YLCuDmqnSXbYqadrdT7j7fxjFACMxTwbNsV4IMiT+UTdVgsyb+40kM7Ov9lAeAF3a1ZIhyKBmqowue1vKcjIlp8clXZb0fI/lAvCI8omYxw4f0rEj/RrLA/RHlU3Lziub9X9LkpVccrrXQgF4dCyOiTqr0rZ+PmxQVsrM/rCH8gDoUt5ddpykP2qoStv6gwPOs/0yMEQbLI6JGhtEBy7dZcAQ7WxYRncZ6qdKkHnfzL4aJmPuWiAzPGYyJjBE7CWDOqvyqlwtPjCz4nbMpt3bMwMYMBbHRJ1VacksKtug7FD4OVz4OSRprs9lBLAPlvlHnVUJMt88YJY/iX9giLa7y9iwDDXUdZBx9w/N7FS+cVl+POwt83RxK4A6M7PzZnZtZWUldlGAnmzviklLBjVUZRXmr0iaV5Z/aeXHw94yz5vZqX4VbpBYuwxNkS/1T5BBHVV5Vb7i7s9K27P/t7n7jbAVwM/6UDYAj4DEP+qsSk5mofB72Uiy8YplAVDBGt1lqLEqQaZzyHKnZyuWBUAFOy0ZEv+onypBxgoJfy8cfM7M/oukH/WlZAAO9GDL9YvPaMmgvqqMLpuTNBa2XX7HzP7KzJYl3ZB0fb+9ZgD0V570l1ggE/VU6VXp7m+b2VVJk5LakpYkLYzqLpnAqGJXTNRdlSHM75nZKXe/6+433P2d8O/dwjUvhete6m9xARTlc2QkFshEPVXJyfzQ3X+210kze0HZIpm/J+m32I4ZGBxaMqi7Skv9m9l3zOwnIR/z9Y7TFyXNuPuH7v49SRM9lxJAqWKQIfGPOqoSZNqSnpE0razF8pqZPVc4P6EsR5MbyDIzZtY2s8uDuDcwKvLFMY8dOaSjhwexPRTQm0pL/bv724XHs2FI84/D4/GO7rS+L/1vZhckvdjv+wKjZp1dMVFzvU7GzJVNyhwYd5+VdH2YzwnUUT7b/zgrMKOmqgSZsbAC80tm9nqYgHlL2k76m5n9WuH6M/0oKICHbbBuGWquymTMtyXdkfSWpJclXVUWWL4raUrSaUmXwhbNfyDp/b3uZWYTZlbaIjGzi2Z2Ify80W05gRSw9TLqrvJkTElvdxy+kf8SEvLTkmbd/ccd18nMJiS9Gh62S85fDM8zGx63zeyKu09XKS/QVCyOibrr+yszTNT8qaTv7XWNuy9KWgzBZqrkkml3P1e4fsnMyq4DksbimKi7QXz9uaydVkrXzKyl8rk1q2Y25e7zlUu28xwnJJ2QpLNnz/Z6O/To2v/8v/rvS8uxizGSFj/JdnZ9gnXLUFP7vjLN7CfKFr28FB6/oP1XWTb1PmS5LWm15PgdZcFnPrRqpiVNmNlFd7/a5XN8W9L3Jen27du9lBU9Wl67p3/9x/9L3veB7ml55qnHYhcBKHXQ158Z7Z5YeUfSnLu/stcfmNkPeyzTeHieTqsKrY/QmumlRfMDSdck6eTJkx/1cB/06M765naA+dqXf1WfYyhu155+/Kj+xe+cil0MoNS+QSYs6198/KGZffOAe870XKoBc/dlScuSNDk5Gbk0aVsrLIvy77/+m2odPxaxNAD6rcoQ5n2X8w9J/16VbeHcUggM/WBm583s2srKSr9uiQo2NourCJNXAJrmoJzM08pyJG1lH/zzxSVjzOy0dkaHnZO0IumWu7/bQ5kWlAWUTuOSFnu47y6hlTY3OTn5Wr/uie6x9hbQbAe9q1eUjRZbcvd3O5f4d/efhv1k3nH3bykLBFd6KZC7r0paCqPMilr9GFmGellnxjrQaAcFmbvu/rtlEyrLhMmS3eyOWdYtJmWB7VL+IMynIcA00M6MdRL+QBMd9PVx1we7mZ3qvKBkA7MDg4GZtZUNQZ5SNgz5iqSb+VBkd78alpWZUtZ11u73bH8zOy/pfLv90IIDGKLtGevM8wAa6aB39lLH4zFlgeH3lK1d9t4j/M1D3H1JB4xCqzD3pSvkZOqBtbeAZjvonb1ripy7fyjpQzN7MZ+gedDfAPtZI8gAjVZ1OE/fRnkhbRubrL0FNNlBQaZsKLG0f2tlr7+pFebJ1MM6ORmg0Q4KMtNmttz5I2mm7Hg4d3EI5e6Zu8+5+zfGxsZiFyVpdJcBzfYoif/LXdzPJH23enGQGubJAM124BBmd3+nmxuGOS3AI6ElAzTbvt1lYRZ/V6r8TQzkZOphfZPJmECTJbtYFDmZeiDxDzRbskEG9cBkTKDZCDKI5v6DLd27vyWJxD/QVAQZRJN3lUnkZICmSjbIkPiPb21zZ1dMWjJAMyUbZEj8x7de2HqZnAzQTMkGGcS3VgwyjC4DGokgg2g2yMkAjUeQQTR5S+axI4d05DAvRaCJeGcjGtYtA5qPIINodpaUIcgATZVskGEIc3wsjgk0X7JBhiHM8e10l5H0B5oq2SCD+PIZ/8cZvgw0FkEG0ZD4B5qPIINo2EsGaD6CDKJZy/eSoSUDNBZBBtHQXQY0H0EG0bBhGdB8yQYZ5snExzwZoPmSDTLMk4lvuyVzjMQ/0FTJBhnEt75J4h9oOoIMovjswZY2729JIvEPNBlBBlGwKyaQBoIMoijuisnaZUBzEWQQxfquXTFpyQBNRZBBFMWWDAtkAs1FkEEUu3IyDGEGGosggyg2wuKYjx89pCOHeRkCTcW7G1Hki2MyfBloNoIMomDdMiANyQYZ1i6La3vdMpL+QKMlG2RYuywulvkH0pBskEFceZA5zkRMoNEIMoiCXTGBNBBkEEU+hPlJcjJAoxFkEAUblgFpIMggip3EPzkZoMkIMohinZwMkASCDKKguwxIA0EGUaxv5kGG7jKgyUbya6SZTUlq5Y/dfTZicVDBRt5dxugyoNFG9R0+7e4vS5KZXTezeXdfjV0oPJrN+1vafLAliRn/QNONXHdZaMXcKRz6QNIrkYqDCnbtJUOQARpt5IKMpLakYqtlVdKZSGVBBWsEGSAZUd/hZjYh6VLe9dVx7qJ2Wixtd38r/N6StNxxeUsYGXnSX6K7DGi6KO/wEFxeDQ/bJecvSjsJfTNrm9kVd59WecuFfMwI2d1dxugyoMmiBBl3X5S0GILNVMkl0+5+rnD9UsjFSNKSpHOFa1vK8jIYEfnimJJ0nNFlQKPVLidjZi1JEyWnVs1syt3nJY0Xjp+RND+UwqEvNkJL5nNHD+vwIYtcGgCDVMevkZ2J/dwdZcFnXtKbZnYhXPd+t8OXzeyEpBOSdPbs2UqF/M71P9cny+uV/jZ1y2ubkkj6Aymo47t8XLuHKOdWFQJD3t3Ww3N8W9L3Jen27duVbvC///pT/Z+/+bSHIuAftB6PXQQAA1bHIDMMP5B0TZJOnjz5UZUbvPZbX9LP//5eXwuVksOHTF/78q/GLgaAAatrkBkvOVY2dLkSd1/O7zU5OVnpHv/8n57qR1EAoNFql/iXtKDyeS/j6q2LbBczO29m11ZWVvp1SwBAh9oFmZDEXwqjzIpaYWRZv55nzt2/MTY21q9bAgA6xA4yZd1iknRZ0qX8QZhPwzBlABgxsWb8tyVNK5uIOWFmVyTddPerkuTuV83sYmFJ/3aY7Q8AGCGxZvwvSZo54JqrgyyDmZ2XdL7dfmhVGwBAn8TuLouGnAwADF6yQQYAMHjm7rHLEJWZ/VzSJxX+9LCkL0j6O0kPDri2yaiHHdRFhnrINL0efs3df+Wgi5INMnlORtKcu89V+Ptfl/SRpN9w94/7Xb5RQT3soC4y1EOGesjUdcb/wIXA0nVwAQA8OnIyAICBIchUtyzp36pP66mNMOphB3WRoR4y1IMSzskAAAaPlgwAYGAIMgCAgSHIAAAGhiADABgYggwAYGAIMgCAgSHIAAAGhiADABiYZNcu64WZXZR0Jzxsu/tbMcszLOH/LUnnwr8z7r7acT6pejGz6+7+csexpOrBzN6QtKrwf3b32cK5JOqi8N5oSToh6c3U3xs5gkyX8hdT/kYys7aZXWn69tBmdrG4W2moh5uSzhQeJ1UvZjYh6ULHsaTqwcyuK/uysRQeu5mNuftqKnURguzVjqByXdLL4fck6mEvdJd1b7r4YRveXFMRyzNwZtbqPBbqYNzM8v97cvUiabzkWDL1ED48P8gDTHCm8GGbSl08XwwwwVLhfZNKPZQiyHQhvGgmSk6tFj5sm6gt6UpJsFmS1E6xXszsgrvPdxxLrR4uS5otHii0aFKqi3Zo1Ra1QmsupXooRZDpTltZ33OnOyp/ITWCuy9KOlfyba2tEGiUUL2ED5TFklPJ1EP48GyF3y+Y2ZSZvVH4IpJMXUj6pqSbodtMIXhcCedSqodSBJnujGsneVe0qizZ11gh0GwzswuSlsK3+dTqpd3RRZRLqR7yD8+Wu8+G18FVSTfC+WTqIrw3zki6ZGYrhWNSQvWwF4IMuha+rV6S9ELssgxb6CabPfjKxhtX1pLZDrZ5SzeVbqCcmbWVDQA5rSzQvl8YbZY8Rpd1ryzZ21JaGxNdlvRyR/dZ4+slfJiUtWCKGl8PwZK0E1gK8m6gRaVTFzOFkWIzZvaepBtmlr9WUqmHUgSZ7iwo9EN3GFd5H33jhH7nyx3dRanUy5SkVuc39cI8kR8qjXqQuy+Z2V6nV5XIayK8Ft4vHnP3RTN7WdKLkt5UAvWwH4JMF8JokSUza3V8g2t1jjRqotAFMFsMMGY25e7zKdRLcRhqzswuFyfWpVAPBYtm1pmfaktaSP29oqylt0w9kJOp4rKyfISk7ZFGjX+xhG9sC8Uhqh3f6JOslxIp1cNM+JG0/X9dKiS9G18XIVC8WnLqgrL8jJRAPezH3D12GUZO+Ea/pKwZ3PglIkIu4tYep8cKCd9k6iUE2GllHyazkq7k30wTq4cLylovknTC3Wc6zje+LgoDYZYVRtzp4RZ/4+thLwQZAMDA0F0GABgYggwAYGAIMgCAgSHIAAAGhiADABgYggwAYGAIMgCAgSHIAAAGhiADABiY/w+6f1od0+5OQQAAAABJRU5ErkJggg==\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%matplotlib inline\n", + "import matplotlib.pyplot as plt\n", + "plt.rc(\"text\", usetex=True)\n", + "plt.rc(\"font\", family='serif')\n", + "import seaborn as sns\n", + "sns.set_context(\"talk\")\n", + "\n", + "eigs = np.linalg.eigvalsh(A)\n", + "plt.semilogy(np.unique(eigs))\n", + "plt.ylabel(\"Eigenvalues\", fontsize=20)\n", + "plt.xticks(fontsize=18)\n", + "_ = plt.yticks(fontsize=18)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "#### Exact solution" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "outputs": [], + "source": [ + "import scipy.optimize as scopt\n", + "\n", + "def callback(x, array):\n", + " array.append(x)" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "||f'(x*)|| = 1.0369185281750917e-05\n", + "f* = -11.862243075233534\n" + ] + } + ], + "source": [ + "scopt_cg_array = []\n", + "scopt_cg_callback = lambda x: callback(x, scopt_cg_array)\n", + "x = scopt.minimize(f, x0, method=\"CG\", jac=grad_f, callback=scopt_cg_callback)\n", + "x = x.x\n", + "print(\"||f'(x*)|| =\", np.linalg.norm(A.dot(x) - b))\n", + "print(\"f* =\", f(x))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "#### Implementation of conjugate gradient method" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "outputs": [], + "source": [ + "def ConjugateGradientQuadratic(x0, A, b, tol=1e-8, callback=None):\n", + " x = x0\n", + " r = A.dot(x0) - b\n", + " p = -r\n", + " while np.linalg.norm(r) > tol:\n", + " alpha = r.dot(r) / p.dot(A.dot(p))\n", + " x = x + alpha * p\n", + " if callback is not None:\n", + " callback(x)\n", + " r_next = r + alpha * A.dot(p)\n", + " beta = r_next.dot(r_next) / r.dot(r)\n", + " p = -r_next + beta * p\n", + " r = r_next\n", + " return x" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\t CG quadratic\n", + "Convergence in 4 iterations\n", + "Function value = -11.862243075235172\n", + "Norm of gradient = 3.8187218127914515e-09\n", + "\t Gradient Descent\n", + "Convergence in 100 iterations\n", + "Function value = -5.587954614458977\n", + "Norm of gradient = 4.080288108928145\n", + "Condition number of A = 1000.0000000003688\n" + ] + } + ], + "source": [ + "import liboptpy.unconstr_solvers as methods\n", + "import liboptpy.step_size as ss\n", + "\n", + "print(\"\\t CG quadratic\")\n", + "cg_quad = methods.fo.ConjugateGradientQuad(A, b)\n", + "x_cg = cg_quad.solve(x0, tol=1e-7, disp=True)\n", + "\n", + "print(\"\\t Gradient Descent\")\n", + "gd = methods.fo.GradientDescent(f, grad_f, ss.ExactLineSearch4Quad(A, b))\n", + "x_gd = gd.solve(x0, tol=1e-7, disp=True)\n", + "\n", + "print(\"Condition number of A =\", abs(max(eigs)) / abs(min(eigs)))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "#### Convergence plot" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhIAAAGKCAYAAABZ4KgMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xd8VFX6+PHPmZkAAdIhIAiE0CF0sGFF3LWuimJZXbtgWdeCom752n6uS5FiB3VlLagUCxZQitgQJIXeE5JAKOmFJKTMnN8fd+ZmhplAMmlM8rxfL143c8+UM0nIfeY5zzlHaa0RQgghhPCHpak7IIQQQojAJYGEEEIIIfwmgYQQQggh/CaBhBBCCCH8JoGEEEIIIfwmgYQQQggh/CaBhBBCCCH8JoGEEEIIIfwmgYQQQggh/GZr6g4Eig4dOuiYmJim7oYQQgjRKBISErK11h1Pdj8JJGooJiaG+Pj4pu6GEEII0SiUUmk1uZ8MbQghhBDCbxJICCGEEMJvEkgIIYQQwm8SSAghhBDCbxJICCGEEMJvEkgIIYQQwm8SSAghhBDCbxJICCGEEMJvEkgIIYQQwm8SSAghhBDCbxJICCGEEMJvEkgIIYQQwm8SSDRHOcnwyS2QvbepeyKEEKKZk0CiOVr/Fuz8Gn6f29Q9EUII0cxJINEcZSQax8wdTdsPIYQQzZ4EEs1NZTkc3mJ8nbWzafsihBCi2ZNAornJ3A72MuPr4iwozmna/gghhGjWJJBobg4met7OkuENIYQQDUcCiebGVR9hCTKOUichhBCiAUkgcSo6VgCHNvn32INJxrHfpcZR6iSEEEI0IAkkTkVfPghzz4f9v9fuceUlRgZCWWHwDca5TAkkhBBCNBwJJE41ZUWw+zvja9expg5vBm2H6AHQdYRxTmokhBBCNCAJJE41yT+Avdz4OvWX2j3WVR/RZTiEdoXWoVCSA0ez6rePQgghhJMEEqeaXcuqvs5IgPLimj/WNWOj6whQCjr2M25LVkIIIUQDafaBhFIqVik1tan7USMOO+xxDme06wiOitrVSZgZCeewRsf+xlFmbgghhGggzTqQUEpdDzwJhDd1X2rkQLwxFBERA0NuNM7VdHijNB9yk8HaGjoNMs5FDzSOEkgIIYRoIM06kNBaLwYWNXU/amy3c1ij72UQc57xdU0DiUMbjWPnwWB1riER7cxIyBRQIYQQDaRZBxIBZ9dy49jvUuhxNihLzeskMtzqI1w6DjCOmTtA6/rtqxBCCEGABBJKqRFKKZ+ZBaXURKXU9c5/Uxq7b37ZshgW32XURACLtvzEo589gz1rB7QOgx5joE0YnDbUWSex/uTPefC4+giAkM7G8xzLh6NHGuCNCCGEaOlsTd2BE1FKjQCcxQLE+mifCOYQhquwcq7WelLj9bKWSvPgm8nGxb19Z5KG3Mvz8ZPBcowv2rfjmu5jsbqGJmLOhYNJ5CavItHmYGz3sVhUNbFfhnNFS/eMhFJGVmL/OiMrEdK5Yd+bEEKIFueUzkhorRO11k8Cn1Zzl0la63lu908BxjVK5/wVHAE3fgCWIBzrXucf394JlmMAvBkRxttZfSivdBj3jTkPO/DXg8t4dM2jfLzzY59PGb9vJXe1LWd32zCI6uPZGN2fCuCb5KUcLT9abbeOVR5Dy/CHEEKIWjqlA4kTUUqFAyN8NOUrpU7tYKLn+fCnV/gwNIT9QTm0srcmtryCIzYbbxzN4a8LEqmwO6D7WSwJDWGLqgBg7qa5XsHAscpj/HP982wIbsML0dFopTxfq+MAZkeG89Shlfzr13/57E7ikUTGfDyGaRum+Wwvrihm8prJfLLzk2rf0vLU5fx+qPqpqtml2RwpluEVIYRobgI2kMAY6sj3cT4XZ4DhDCgmAeNcwyCnipXtBzEzIhKAl7MO8Giu8VZ0x7V8vyOdhxYkcaSynNmRxn06tgolryyP97a95/E87217j4yyPAA2qjJWpa/yaE9pH86C0BDjNdNXkpSZ5NFud9h5cf2LlDvKWbBzAbvzdnv1de7muXyf9j1TN0wlvTDdq339ofU88eMTPLDqAQ4XH/ZqLygr4Pql13Ptl9eSVeK9yqbWmik/TuHKz68k91iuz+/X8n3LeeLHJygoK/DZnnE0g/9u/S8lFSU+2ysdlaw/tJ5KR6XPdoCi8iLJygghRC0FciARiRE0HC8fiALQWq/UWk/QWvdyHwKpKaVUlFKqr1Kqb2Vl9Reg2iouK2PKj09ht2guKAvjwtJSLigtZUibaLAWExL9G8u3HeZv3z9PkdKcW1LKzLCRAHyw/QPzYnzw6EHe3fw2AFceNS6gsxNnU+GoMF9r+oEVVCpFpN0YLnk5/mWPi+WSPUvM4MGhHcxMmOnR1/2F+/lw+4eAcTGenTjbo73SUcnUDcZ6X2X2Ml5NetXr/b616S1yjuVQVFHE6xtf92pfs38Ny1KXkVaYxpsb3/RqzyrJ4v/W/h/LU5fz1qa3vNod2sFjax5jVsIsXtv4mle7633f8/09zE6Y7bN9zf41nPvJudU+PuNoBuOXjufdLe/6bK+wV/DCby+wcNdCn+0AX6d8zcq0ldW2783by8bMjdW2Hy0/yoGiA9W2a60ps5dV2y6EEA0hkAOJxvAQsAvYlZmZWW9Pes/SF6mwHcBSGcW/bvgMYi9CtWrPwyMeBaBNx5+xhWxhe9EaWisbf8/JZdjBHVzc/WJKK0t5c5NxsZ2x7kXKHBVcdrSY5+MmEhMaQ1phGot3LwbgpwM/8cuR32nv0Hx48DCRrcPZlLWJlenGxaywvJDXkowL5z/O/Aftg9rza8av/HbwN7OvM+JnUOGo4Lyu59HG2oYVaSs8Lnaf7fmMPXl7iG4bTZAliK+Sv2JHTtUCWCkFKXyy8xMUCquy8vnez9mbt9dsr7BXMCN+hnl70e5F7CvY5/H9mpM4h9LKUgA+2fkJaYVpHu1fJX/F9pztAHy882P2F+73aE8pSDHrSz7a+RH7izzby+xl/Of3/+DQDv679b9ejwcjENmTt4fXkl7zen2ABTsXsHD3Qv69/t9e/QdIOJLA0z8/zeQfJ3u8f5ec0hxuW3Ybty+/nZ253ut+2B127v3+Xq76/Cq2Zm/1ageYtmEaYz4eU20wsmzfMv64+I9eWSmX3Xm7uee7e0g8kuizvaCsgP+37v9V+/x2h50FOxawJWuLz3Ywsle+sl4uB4oOeP183JVUlFSbtQIjqKywV1TbLoSof4EeSET6OBcO5NTT878K9AP6RUdH18sTHizMZWvR92iteGrUc3QKj4ZbP4Mn9nJGnys567SzOGYvJrjrAgCGtbuGbnbgYCJ/i7sHq7Ly2Z7P+GTHx6zI+Ilgh4PH2vUl6LzHeWTEI4CRAcg/ls/0DdMBuI9wulVW8kBXo3RkdsJsKuwVvLXpLfLK8hgRPYIb+93I3YPvBmBmwkwc2sH6Q+tZvX81wbZgnj3nWf4y8C9AVVbDPRCZMnoKN/e/GY32yHrM2DCDSl3J+D7jmdB3glfWY8HOBaQXpRMTGsPVva7Gru0eWYNtOdv4MvlLbBYbY7qMoVJXMidxjtleUlFi3u4R2oNKRyVzkqraXf21azvBtmCjPdGz/YPtH5BxNAPA5+M3HN7AirQVRrv2fnzusVwzU2LXdmYlzPJod2iHWX/iK+sD8MbGNyiqKMKhHcyIn+E1xLI0eSlbc7ZSqSuZvmG6V/uOnB18tOMjyuxlTP19Kg7t8GgvKCvg3+v/zcHig7yw7gXszqnH7n18du2zrD+8nv9b+38+L8azE2fz6a5PmfLTFI5VHvNqX7JnCS/9/hJ/Xf1Xiiu81z5Jykzinu/v4Y7ld5B/zHtUMrs0mxu+uoEJX03wWU9T4ajgtmW3cdmSy3wOsQH869d/cd6n57Enb4/P9o92fMTYhWPZlr3NZ/u6Q+u4fun11QZL+wv38+CqB9lweIPP9qPlR3lx3YusO7TOZ7tDO3h/2/vVtoORHavu+QF25e46YeYquzSbXbm7qm0vs5dV+/0DI7OVdyyv2nbA6/dHtGyBHEjE43vp60jA90eqWtJa52itd2utd9ts9TNTtktoJB9fvpBrT3+Mm4deYJy0WCAoGIC/Df+bcU5p7GUdWbd5FJWdh4Kjktilj3Ft5BDs2s6Lv/8bgHuKK+h87btgsTK2+1iGdRxG7rFcblt+G6mFqcSExvDnDsawyPi8HGJCupNelM6M+Bl8vONjFIqnBtyB2vQxt54+jk5tO7Ezdydf7v3SHLK4d/C9RLeN5q64u4hsE8nGrI2sTF/pEYj8occfmDhkIiGtQlh/eD0/Z/zMLxm/8HPGz7QPas9Dwx/i/mH30y6oHT9n/My6Q+s8LsBPjH6Ch0c8TLAtmNX7VxN/OB6tNVN/N/pw64Bbee6c57yyIu9seYes0iwGdxjMO394hzbWNnyX+p3Z/mvGr/x04CfaBbXjvUvfo7W1tUd7dmk2bzuHh14Y84LX4+0OuxmQ3dD3BvP13T/Vv7HxDY5WHGVE9AiCbcH8sP8HjwuBK2MSHRxtvn/3rM/uvN0s3rMYq7LSPqg96w+t55eMqhVNiyuKzeDFZrGRmJnI6vTVZrvWmmkbpqExgoutOVtZvm+5x+/dvM3zyC8zLt578vawNHmpR/u3+75lS7aRSUgrTGPhbs8hmt15u/lsz2cAHCo+xIc7PvRoLyqvGrbKPZbLf7f+16PdoR1M+32aed+5m+dyvNeSXqOooojiimKfQ2CLdi1iV94uSipLvIbYwAj4liYvpbii2CPL5XK4+DCzE2aTVZrFv3//t1cwVmYv49m1z7IrbxfPrn3Wq55Ga80L617gpwM/8c9f/ulzGOnNTW/yya5PePKnJ33Okvpy75dMj5/OIz88Qk6p9+edTVmbeGj1Q0xaMckMbt1llWSZmStfmZ1yezl3Lr+TCV9NqDbz9PiPj3Pl51d6/I65e23ja5z/6fksT13us335vuWM/HAkn+/53Gf7tuxtjF04lo92fOSzPbMkk1u+uaXaYcIyexlP/PiEz2FMMH4OryS+whsb3/DZDsb/uTc3vekVULskHEng3S3vVlszdaDoAAt2LKg2u3W0/ChLk5dWO5Rod9j56cBPPgNulx05O6qt6QI4UnzkhO0lFSWnzFBmwAYSWut8IMU5e8NduNa6+oHoU0Bc5+68MO4On22DOw7msp6XYbPY6G+7g6NlsKjtzRDUDvav54Gkrwh2GH8AT6+o4PaL/gNhXQFQSjF51GQAM70+ZfQUgrqOAiAo8X88us9Iiy/YucDIFNhbMWD+NfDF/bT56mH+NsIIZJ5f9zx78vbQtX1Xbht0GwDtW7XngaEPAEYa3QxEzngKpRRhrcOYNMRYwmNWwizzAjxpyCSigqOIbBPJPYPvAWBm/ExeSXyFoxVHGdN1DOeffj4d23bkzkF3AkYWYXnqcpIyk4hsE8nEIRPp1K4Ttw+6HYDp8dPJOJrB/7b9z3yfndt19siaVDgqzD5MHDKRQVGDuG3gbWa71ppXk16lpLKEC0+/kGt6X2O+V1f70uSl7MjdQae2nXh89OPm68/YYGQN9uTtYdHuRViVlWfOfoY746r679AOj4zJIyMfMd//y/EvY3fY0VozfcN0HNrBDf1u4L6h95ntrj9y7255l5xjOQzpOIQnRj1hfP8SZpp/5Fanryb+SDzhrcN5fNTjgDEc5Pojk1aYxoKdC1Ao8/v7atKr5h+p0spSMwv0hx5/AIysVmF5IYBHH4d1HAYYAZz7EMPbm98m91gu3UO6A/D+tvc9Cm+/3fctW3O2EtY6DIXyGqLalbuLz/Z8hk3ZsCkbX+z9wuNTdUFZAW9sMi4cFmXxGmJzD/gA1h5cy68Zv+LulcRXOGY3/rBvztrMd2nfebR/tOMj8+KdXJDM53s9L5Q/Z/zMb4eMAPBg8UEW7Fjg0e76PoPvYKqkooRXkl4BjODQNUTp4goIwci+HJ/5AuMiX1JZUm1m6+OdH5NamIpGm7+j7n7J+IU1+9egMX6mx19I0wvTzX7P2DDDHFJ0Ka4o5j+//we7tvNywstexc8O7eDF9S+SVZrF7ITZPjNLcxLnsDl7M68lveZzGPDD7R+yPHU5r2983ecw2Zr9a3h7y9u8uelNr58xQHJ+Mv/69V+8sfENlu1b5tVeUFbAIz88YmbYjmd32Hn4h4d56feXeGfLO17tAM+sfYZ//PIPZsZ7/wwA3tr8Fg+uepBnf3vWZ/vy1OXc8PUNPP7j4z7bt+ds57LPLmPSikk+g6Hs0mwu/+xybvnmllNiKC9QAglfQxgAU4GnXTecC1id0kFETbx47ousmrCKFy69GoBndnYj456NcNUcOnYaxv35+di05p8Ro2k9aLzHY4dFD+OSHpcAcF7X8zjv9PNg2J/hmjeh+zlcVJTPiGPGH9P2Dgd/PZBsBCmWIEhZw5Xhg+gf2d/8A/PYyMdobW1tPv/4vuOJCY3hcPFhKnUl1/a5lgFRA8z2m/vfTNf2Xdmbv5eUghS6h3TnlgG3mO23DriVTm07sSN3B0v2LMGqrObFEeD2QbfTMbgjW3O28szaZwD46/C/EtLKmHlyZ9ydRLaJZHPWZiZ+P5FyRzmX97ycYdHGBe7uwXebWZNHf3iU5IJkTm9/OrcOuBXAI6vy2sbX+HzP59iUzQzA3Nu/TP7S/MP/6MhHCbYFc2fcnUS1iWJz9ma+S/2OaRummUFAbHgstw80+r8tZxvL9i3jv1v/S1ZpFnFRcVwRe4X5/nfl7eLrlK/56cBPrDu0jpBWITww9AHz++e6kB08etAjWJrQbwIxoTGkF6WzcPdCyu3lvJzwMgAPDHuAWwfcSp+IPh4XOldQcnXvq3lk5CPERcWRVZplPu//tv2PIyVH6B/Zn2nnT2Nkp5Hkl+Xzzmbjj6h7H18d+yrndj2X4opi8xPh/qL9ZoZi2vnT+EOPP3DMfswsvHUPVCaPnMzVva+mUlea51yBikZzU/+buLH/jcaF0G2IZ+7muRSUFTC682jujjOG4KbHVw3xuAd8rmB3RvwMMwW/NXsrX6V8RZAlyAzmZifMptxeDhg1KvM2G/XY1/e9HjAyJK4hmgpHVS3PRd0uAozgyX0IYFbCLCodlQyPHg7A+9s9g6l3t75Ldmk2MaExWJSFxbsXk5KfYrYvT13O5qzNRLaJpJWlFcv2LfO4kO7M3Wn+vrYLasevGb+yNmOt2Z53LI+5m4xMT7At2Pwddal0VDJjg/EebMpGSkEKS3Yvwd3MhJnm//0jJUf4YPsHHu2uoBaMC7Irm+eybN8yM7N1zH7Mq3h5W/Y2MxtWqSu9gqGc0hze3lL1nMcP81XYK8zfd1f78cMsrqFM8AyoXdyzc+4Bs8sXe78wsz3vbXuPzBLP+rjEI4l8n/Y9AAt3LSS1INWj/XDxYeZvnQ/ANynfeAVDxyqP8XK88R5+zvjZKxhyBZQVjgo2Zm3k233fcrzXN75OzrEcduXt8soeNoVTOpBw2wJ8KjBCKTXXfRqncyZGslJqnHOnz3Gn9KqWNRRkCSKyTSSDuoRx9bAulNsdvPzjQRh5B0z8gTtvXkb80KcZc7Xv1OA/z/onDwx7gOfHPG+csFiNYOKuZaiHEvl796uIcVh4KrgPHSZ8AFOSIc4ISCwbF/D4qMexKAtnnnamGZS49+3RkUZRaLugdjw0/CGP9lbWVmatBsDkUZMJcq3UCbSxtTGzHgA39ruRXuG9zNttg9ry4LAHAeMC1DeiL+N7VwVL7YLame3pRem0sbYx++Nqd11IfjzwIwCPj3qcVtZWgGdWZd7meWg0Nw+4mZiwGK/nf2btM2SXZjOk4xAu73l5Vftwo/353573CAJc/Xd9T2YmzGT+tvkATDljChZloY2tDQ+PeBiAV5JeMS9O9w+9n/A24cb3b6Tx/Xs96XVe+v0lM1ga2nEoQZYgHhv5GGD8EZy3eR77i/YTGxbLhL4TsFqsPD7S+JTz9ua3+S71O37Y/wPBtmD+NvxvWJSFx0cb7e9te49tOdvMT6BTRk/BaqkK7D7c8SFphWlmH+8bch/hbcKZPHJy1YWwIIVZCbOocFTwp15/YlCHQTwy8hFsFptZeOsKVAZEDuDq3lfz12F/JdgWzMr0lSQeSeTHAz+y/vB6QluFct/Q+7hvyH2EBIWw7tA6fsn4hbTCND7eaWS/poyewt2D7zaCOWdWwf2T/sMjHuauwXeZwewXe7/w+KR/68BbeXDYg/QO703G0Qwz2Hpj4xsUVxRzbtdz+b+z/o8hHYd4ZBUW717MvoJ9dAvpxowLZjCmyxiKKorM9PuGwxtYlb6KYFswMy6YwaUxl1JmL+OVRKNfh4sPm4HbC2Ne4Lo+12HXdvNCeqzymFlb87fhfzMza65gSWtnhsEZbE0cYvwZnJFQdSF9c9ObFFUUcfZpZzNltLFbwOzE2eaF9LM9n5mB9QvnvmC8701vmEMw7u/hpfNeAozAIbs0G4BDRw/x/vb3AaM4W6FYsHOBWRxbWllqDjndO/hebBYbX+790iwe1lozPd7IGv2p159oa2vrVQ/y+sbXKa4o5szOZxLROoLEzESPKe2f7vqUtMI0YkJjzJ+xe+ZobcZaczg1NizWGIbbXjUMl16YbmbnYsNiPQJmMDIurgC4a/uulFaWmnVgYGRcXJmvyDaRPoMhV+arfVB742d0XDD0wfYPOFx82PyAdnwwtDp9NQlHErAqK2AEQ+5DJO7DjGD83KubFt9YTulAQmudorV+Ums9UmuttNYeK1k67zPPOc1zsdba94pKAWzyJf0Isio+T8pgb6ZzzLXzYKzDbwFbK5+PiWwTyf1D76dDcAfvxqhe9LtsJl/duYmrb/oCBlxp1GcMN/5wsXEBZ3YaxdfXfs2rY19FHb/AFcYnshfPfZE3Ln7D52v8MeaP3NTvJm4feLv56c3dlbFXMqrTKLq278r9Q+/3ar+m9zX0i+hnXjisFqtH+/g+44kNM1ZMvyPuDjq381z6e3zf8fQM6wnA6M6jGdt9bLXt4a3DzeEY9+fvGdbTTClOGT3F4/twbe9r6RXWi6KKIqAqCHD5U68/0SeiD5klmZTZy7g05lLzUyrAFbFXMCByAJklmWYdy039bqr6/vX4I0M6DCHnWA5r9q+htbW1R3B2YbcLGdVpFPll+WatweOjHsdmMep4zul6jnmhm/KTcUG5O+5uOrbtCMDITiPNGUB3Lr+T0spSLu5+MaM7jwZgUIdBXBF7BRWOCu5cfiephan0CO3Bzf1vBqB3RG/G9xmPXduZvGYyK9JWmIEKQLeQbvy5/5/RGDUFrovxE6OfwKIsnkNUG6abn87uH3o/Ya3DCG8Tbl4oX45/2SjYdVRyTe9r6B/Z3yOYm50wm7c2v0V2abaZ9XH/fr228TW+2PtF1RDZ4InYLFUZqHmb57Hh8AazRuWJUU+glDKDqfe3vc+evD1m9uWxkY/RytqKx0Y9hkVZWLhrISkFKebF5a64u4huG83DIx42ZjGlfMW2nG3mBf3SmEsZFj2MB4Y9QFtbWyOIOrSeD3d8yKHiQ/SN6Ms1va/hnsH3ENkmkqTMJFamr2TN/jUewdYtA26hS7su7Mnbw5fJX5KSn8LCXQvNQPHa3td6BEvuNSyPjXqMK3pewfDo4eQey+Xdre96XCDviruLK2Ov5ILTL6CkssR873OSjE/3l8Vcxk39b+KqXldR4agwM0uuC2T/yP48OOxBbup3k0dmaVX6KhKOJBDROoInz3iSu+LuMn8HHNrBnrw9Zpby72f+nfuHGX8bZiXMosJeQUFZgTkcNHnUZDMgd2WOKh2VZqBy75B7efKMJwHPYThX1uhPvf7Ev8816sw+3PGhOa3afRjxrXFveQ2zuYboOgZ35MPLPvSqiXLPfL136XtewVB2abY5XDLrwllewZB7hvGJ0U/QN6Ivh4sPmxk/V0Dp0A5u6ncTozuP9pkZamzWZ599tkk7ECjmzZv37MSJjb+mVVjbILZkFJCcVcyomEj6dQ5pmBcK7w5bFkJBOnQdRViXEQRZgnzeVSlFv8h+nNb+tKqTZUdB28EahFKK808/n3O6nuMzEFFKcXWvq7llwC0EO4tM3VmUhUt6XMIVsVcwNHqoz/azTjuLru27cvug270CDauyMihqELnHcvn7mX/3uMi72nuF9WLtwbU8OfpJBncc7PX83UO68+2+b7mm9zXmBdS9vWv7rnyz7xtiQmN4YcwLHn2wKAvd2nfj65SvaWVpxStjXzGHZlzvv0doDzPF+8KYFzyyMkopeoX3Mv+43D34bi7ucbFHe5/wPizeY0zzHdN1jJlFcekb2ZfFuxfj0A46t+vM1POnmoEGQP/I/izatYhyRzk2i41XLnqFsNZhZvvAqIEs3LXQDJaeP+d5ekf0NtvjOsSxaNcijpQYY+ATB0/kwu4XerQv3r2Y/UX7qXRUMrbbWHNWEEBcVBxf7P2C1MJU8svyje/juS+Yn8IGRg3km5RvSCtKI7UwlWBbMHMumkO7oHYA9Ivox6r0VaQVppm1EjMunEGX9l0A6BXei18zfmVf4T7W7F8DGMHW8E5GQNctpBsbMzeSXJDMsn3LqNSV3NjvRq7ubQwndm7Xmb35e9mdt5tv931LYXkhozqN4tGRj6KUIio4iiPFR9iWs43V6atJK0yjU9tOTD1/KkGWIEJbh3K04igbszYSfzieDYc30MrSijlj5xDaKpS2QW0BWH94PdtztvPj/h+pcFTw0nkv0T20O62srQi2BfNzxs9sz9nO+kPryS/L59GRjzK682hsFhtRbaJYmb6Srdlb2Z6znf1F+7mu73Vc1+c643cwxPgd3Jq9lSMlR0jMTGRE9AgeG/mY+Tv22Z7PzOnE3+77lui20Uw7fxpBliD6Rxm/I9tzt9O5bWfmbZ7n8R4GRQ0yC2D7RvRlerxRczH1vKl0C+3G4A6DWbR7ESkFKfSL6MfMhJkUlhcyedRkRnYayaAOg/gy+UvSCtPoHtKd+dvmk16Uzk39b+JPvf9E/8j+fJf6HWlFaYS1DjMu2Ec2cGbnM3lo+EP0Du/Nrxm/klqYilVZ2X90P5/QVWN+AAAgAElEQVTv/Zyu7bvy7/P+Tc+wnmzJ3sLe/L0cqzxGW1tbZiXOItgWzOyLZtMzvCfpRenszN1JTmkOgzsM5qmfn8Ku7cy4YAb9o/pTUFbA5uzNpBelc0mPS3h49cMUVxTz9JlPc+ZpZ1KpK9lweAO783ZzXd/rmPLTFA4VH+L2QbdzVa+raGNrw88ZP7MjZwc39L2Bl+NfZnP2Zi7sdiH3Db2PDsEdWJG2gq3ZW5nQbwILdy1keepyYsNieX7M88SExvBVyldszd7Ktb2vJf5IPHM3zyUkKITZF81mcMfBLN69mO0527k89nKP/8P14bnnnjv07LPPnnQNJgkkaqipAgmAVTuOsOtwEX8c1Jn+p4U2zIsoZWxXvu9HqDxmDnXUSHkxvDIcNn9qDKFYfWdKPF9O+QwyXIJtwb4zKk4RbSIYFj3MK4hw6dSuE5f3vJzw1r4m9kDXkK7cMegO+kT08dnePbQ7E/pO4NKYS31ulNYjtAdnnXYWfxn4F0Jbe/9MuoV2o1tIN67rex0Dowb6fP02tjaM6DSC6/tc7/W96NyuM2X2Mtra2vL0GU97DA8BRLeNJr8snwNFB5hx/gwi2kR4tEcFR5FbmsvWnK08d85z9Ivs59Ee3ibc/CN5+8DbubTnpR7tIa1CKK0sJSkziTM6n8HDIx726GPboLY4tIPfD/9OdNto4wJ63BBWK2srfj34qxGojPUMVIKsQYQEhbDmwBrAO5iyWqzmH1kwhlXO73a+2e4ezIFRJOrKcoB3MNY7vDfPnP2M+bNUStE3oi+Ldi/Cru3mH+ZgW1VgOzByIJ/u/pRjlcdQKGZdNIvotlXTwAdFDWLh7oVmWvkfZ/6DQR0Gme1xHeJYsmeJGWzdPuh2/hDzh6rnjxrI0uSl7C/aT4WjggtOv4BJQ6uyY/0j+/N92vekFaaZwdbzY543g63e4b35JeMXUgtTOXD0AO2C2jH7otlmkNI9tDubsjaRXJDMjlxjbZeZF86kU7tOgPF/JLUglV15u8xP1O7vIaJNBNml2WzN3soP+38A4I5Bd5jvoX2r9pTZy0g4ksCKtBVUOCq4qNtF3DPkHvN3wGaxsfbgWlanryavLI/YsFieO+c5LMpCkCWI8NbhrN6/mnWH1pFSkEJIUAizLjIu9lZlpUv7Lizbt4zNWZvZmLkRjWbWRbPo2Lajx894W/Y2kjKTKK0s5ZmznzF/3/tH9GfR7kVsz9lOUmYS+WX5HkGv62e4K28XG7M2crD4IJfFXMZfBhkZ2rioOBbvMYbwtudsZ0fuDgZEDuAfZ/0DpRSDogaxdO9S0orS2Fewj18yfiGidQQvX/gyra2tPYKh7NJss6B49kWziWgTYQS8B41gqKi8iE92fkK5o5z/d+7/o2dYT7qFdGNr9lb25u+lqLyIJXuWkF+Wz99G/I2zupxFdNtoDhQdYEfuDrJLsz1+v+qDBBL1rCkDiRXbj7DjUBHjBnZiYJcGCiQAInrCujcgJxlG3gWt2tXscenrYcPbUJIN5SXQ59Te6qSm2ga1rX63VeC09qeZn5B96RfZj24h3aptHx49nNGdR1cbUJ3d5Wyu7HWlVxDhct7p53FX3F1eQYTLuV3P5bo+1/nM6gCcddpZjOo8ivF9xvt8nyM7jaRL+y5MHDLRvDi5G9JxCK0srZg4ZCJdQ7p6tQ+MHMjRiqNM6DuBM08706u9X0Q/Mo5mMKrTKG4dcKvX96FXeC+S85MJax3GP876h1eGrHtId/bm7yW/LJ/pF0z3Cug6t+vM/qL9JOcn85/z/0P30O4e7R2CO5BVmsX2nO08MvIRrz6GtQ6jqLyITVmbuKb3NdzY70aP9nZB7cxPpIOiBvH0mU97vIfW1ta0trbml4xfiGwTycsXvGzW6oBRbxTRJoJV6auwKRtzxs7x+FlalIUu7bqYxXa+Mlc9w3ryxd4vAKPY9tyu53p+jyP7sXj3YjSaq2Kv4uYBntm1AVEDWLhrIQ7t8PkeBkUNYtHuRVQ4Kny+h7gOcXy+53NKKkuM93DRHI8M4MCogXyb8i15zmX8XRdIl74Rfflx/48cLD4IYF4gXXqE9iDhSAL7CvfhwGGuR+PinjkqrSxlWMdhPD7qcfM9RAZHklmSybacbeSX5XsFva6AOTEzkcySTI+MC3gGQ65akKnnTeX0kNONn6E1yMyWJOcnA0bma0QnYxso92DIFczdPOBmrup1lfkz7BXmDIZytlHuKOecLufw4LAHzffQP7I/C3cvZHvOdvLL8ukW0o0Xz33R/BA1qMMgFu4ygqFzupzjNdRbFzUNJMxCHvl34n8jR47UTeWJRRt1jye/1p/8ntbwL/bRDVo/E6r1L3Nq/pifZxmPeSZU62fCtE5d23D9E8KNw+HQlfbKatsr7BU6qySr2vZye7nekrVFOxyOattXpq3UxyqPVdu+eNdifejooWpff/7W+TrpSJLPdrvDrt9IekMv3bvUZ7vD4dBzEuboOQlzqu3jnIQ5+uHVD1fbx9eSXtPXfXmdPnz0sM/2tze/rc9ecLbelLnJZ/tH2z/Sg+cPrraPX+z5QsfNj9Mz42f6bF+ZulLHzY/T96+43+d7WH9wvY6bH6cvX3K5Lqss82rfnr1dD54/WI/+cLTOLM70ak8vSNfD3h+m4+bH6c2Zm73as0qy9BkfnqHj5sfpL/d+6dVeVFakz//kfB03P07Pip/l1V5WWaYvXXypjpsfpx9a9ZBXu91h19cvvV7HzY/TV39+ta6wV3i0OxwOfdfyu3Tc/Dh9zoJzdP6xfK/neOyHx3Tc/Dg95H9D9K7cXV7tz619TsfNj9Nx8+P0itQVXu1zEubouPlx+s/f/Lna3xN/APG6BtfHJr9AB8q/pgwknlqyWfd48mv94brUhn+x7V8ZAcGro7Su6S/kJ7caj3ntTOM4Z7jWZcUN208hRKMpLj/x/+eDRQdPeAHbnbtbl1SUVNu+KXNTtYGO1lr/fuh3vT17e7Xtvx38Ta9OW11t+9qMtfqDbR9ou8Pus339wfV66u9Tq32fSUeS9OQ1k6sNGLdlb9N3Lr+z2mBsb95eff3S6/U3yd/4bD9QdEBf+dmV+tXEV322Z5Vk6T8u/qN+aNVDPr/PR8uP6vM/OV+f9/F5Or0w3edz+KOmgYQy7itOZtSoUTo+Pr5JXvufX2zhw3XpPH/1IG47O6ZhX8xeATMHQnEm3PU9dPdOSXuZOQgKD8Ckn+CzSZC1A87+K/zxRXA4YO9KY+gjpDNc9YpRjyGEEKLebMraRGxYrEdhd10ppRK01qNOdr/6WfdZNCibxRi/tjsaIeizBsGwm+HXOZD0/skDiaIjRhDRKgQ6xcE1r8M74+C316F1CGz7HLLcNqEa8whE9ar++YQQQtTa0I6+a6Eawym9joQwWC3GJ/hGCSQAhhmrQLL9K6g8yVruB53bmnQZZix81XUkjHkY0LDmJSOICOkCkc7g4UD1mxEJIYQIPBJIBACbM5CobKxAomNfI7tQVgDJP5z4vhkJxrHryKpzFzwFsRcZ566dBw9vghHOBa8kkBBCiGZFhjYCQKNnJAAGXQNHthpDE/0urf5+vgKJoDZw2xee9zvdWDWR/b/Xbz+FEEI0KclIBAAzI2FvzEDCuSDVzm+gopqtcLX2HUj40mU4KCsc2WYsYCWEEKJZkEAiAFjNYkvv7WQbTFQv6DwEyosgeZXv++SmwLECaN8ZQruc+PlatYNOg4xltA8m1X9/hRBCNAkJJAKAzdrINRIurmWyt37mu909G1GTKZ3dzjCOUichhBDNhgQSAcCimqBGAmDgNcZx1zKoKPVuNwOJ4d5tvph1EhJICCFEcyGBRACwNUWxJUBkT+gyAiqKYc8K7/YM59TPk9VHuLgCiQMbjPoKIYQQAU8CiQBgbezpn+4GXWsctx03vGGvgEObjK+71DAjERkLbaOMVTPz0+qvj0IIIZqMBBIBwFUj0egZCTCmgQLs/s5ztsWRbWAvg6jeEOx790kvSrllJZpmuXEhhBD1SwKJANCkGYnw7sbFv6IE9nxfdb6m0z6Pd7pz2XZZT0IIIZoFCSQCQFWNRCNO/3TnWlPipxlVhZK1rY9wOV1mbgghRHMigUQAcK0j0SQZCYDB10P7TsZKl++Og48mwL6fjLbaBhJdR4CywOHNvmeCCCGECCgSSAQAq/On1CQ1EgDto+GBdXDe49CqvTHEUZAOliBjT47aaB0C0QPBUVlVrAmQ9htsWVy//RZCCNHgJJAIAE2ekQBoGwkX/wse3mzs7mkLhv5XGPtq1JarTuLABmP2x4pn4L1LYcndxmqZQgghAoZs2hUAXDUSjqYMJFzaRcElz8O45/x/jtPPgIT5xkJX25fCAbfCy+w9xjRRIYQQAUECiQDQpLM2qlOTJbGr45oCmvarcQztaswOSf8N8lLr3DUhhBCNR4Y2AkCTrWzZUKJ6Q9sOxtd9L4X7fjGOIIGEEEIEGMlIBIBTMiNRFxYL3PQRFByAuOuM7EZkT6Mtd1/T9k0IIUStSCARAGxNsY14Q+t+luftiBjjKBkJIYQIKDK0EQDMjIS9mWQkfHEPJGRDLyGECBgSSAQAa3OrkfClTRgER0JlKRw90tS9EUIIUUMSSAQAM5Bo7p/UZXhDCCECjgQSAaDZzdqojhRcCiFEwJFAIgC0iBoJkIyEEEIEIAkkAoDN2kIyEhJICCFEwJFAIgDYzHUkmtH0T18inEMbeTK0IYQQgUICiQBgNdeRkIyEEEKIU4sEEgHAqprZypbVCe0C1lbG9M/ykqbujRBCiBqQQCIAWFtKjYTFamzeBZKVEEKIACGBRABoMdM/QYY3hBAiwEggEQBaxMqWLmbBZWqTdkMIIUTNSCARAGzNbffPEzEzEjJzQwghAoEEEgGgZWUkYoyjZCSEECIgSCARAFzbiDf7dSSgaplsCSSEECIgNPtAQikVq5Sa2tT9qIsWlZEI72Ec89KgJQROQggR4GxN3YGGpJS6HrikqftRV9aWVCPRuj20i4biTCg6BGFdm7pHQgghTqBZZyS01ouBRU3dj7pyxhFoDY6WEEzUpE7CXgnF2Y3RGyGEECfQrAOJ5kIpVbWWhG5JgcQJZm6sfh6m94IFN0HmzkbplhBCCG9NGkgopUYopXxmDJRSE5VS1zv/TWnsvp1qWlSdRE0KLrd9bhx3L4M3z4alD0HhoQbvmhBCCE9NUiOhlBoB3Oi8GeujfSKYQxOugsm5WutJzttTgKhqnn6F1npl/fe6adksijJaSJ3EyYY2CjIgPx1ah8KQGyD+PUh8H7YshruWw2lDG6unQgjR4jVJIKG1TgQSnQHFOB93maS1Hul2/xSl1Di329MaoZunFDMjYW8JgYQzI5FbzdBG+m/GsduZcMXLcOb98PkkyIiH1F8kkBBCiEZ0ytVIKKXCgRE+mvLdg4mWxmZtQWtJnCwj4Qokup9lHDv0hgFXGl8XHmzIngkhhDjOKRdIYAx15Ps4n4vvAKNazsBjEjDONVxSy8dHKaX6KqX6VlZW1vbh9cqiWlCNRPtOYGsDJdlQVuTdnuYMJHqcU3UupItxlEBCCCEa1akYSERiBA3Hy6f6ugiftNYrtdYTtNa9tNbz/OjLQ8AuYFdmZqYfD68/LWq/DYulKiuRtcuzrTQPMreDtRV0cYsrQ52BRJEUXAohRGPyK5BQSsUopV5SSt3jdu5ipVRo/XXtlPAq0A/oFx0d3aQdaVGzNgBizjOOrtkZLunrAQ1dR0JQm6rzrkCiMKNRuieEEMJQ60BCKTUcWAkoINx1Xmu9ChitlIqph35F+jgXDuTUw3PXmNY6R2u9W2u922Zr2kVAbdYWFkgMvck4bllkLD7lYtZHnO15/5DTjGPRYVlaWwghGpE/GYkbtNa9tdZPAR5l9c5golZ1DD7E4xaguIkEEuv43AGrRS2TDUbGIbIXHD0C+9ZUna8ukGjVFtqEg70cSho13hRCiBbNn0Ai3u1rX1c1X9mEGtNa5wMpztkb7sKb4/oQNWVraUMbSlVlJTZ9ahwrSiEjEVDQ7Qzvx5h1ElJwKYQQjcWfQML9SqZ8tPeuxXNVF3RMBZ42X8RYb6LFBhEA1pa0lbjLkBuM486voewoZCSAowI6xUGwj6RVqMzcEEKIxuZPIKHciiy128lhSqnvgO9r8ASurb2nAiOUUnPdp2c6Z1gkK6XGOXfwHOda1bKlanEZCTBmbnQ/GypKYMdXVcMaPc72fX9XnYQEEkII0WhqXUGotV6ilHpCKZULaGdA4MosPKm1Xl2D50gBnjzJffyZrtlsWVpajYTLkBuNAGLzJ6Csxrnj6yNcQp1bjksgIYQQjcavqQha6+lKqXnAKIwFpFKAeK11QX12TlRxZSRaxDbi7gZdA8umQMqPxiJVcIJAwjVzQ9aSEEKIxuL3glRa6wKt9Sqt9dvABnwXXop60uJmbbgER0DfSwENlaXGcIcrYDieZCSEEKLR+bOOxD0+TvcCLlFK3auUGlv3bonjtcgaCRfX7A2A7udUfz+pkRBCiEbnz9CG16wMrXUSkASglBpf104Jby02IwHQ+xIIjoTS3OoLLUGWyRZCiCZQ1+mfHpyrWl7ib2dE9aoyEi1o+qeLrRVc/C/oMQb6X1n9/YIjjDqKskLfm30JIYSodyfNSDiXxB4H3AgMd56b4uuuQDIwoT47KAzmOhL2FpiRABh1l/HvRJQyshK5KVB4CDqGNE7fhBCiBTtpIOE2bDHdOdUzDB9TN2XGRsNq0TUStRHiCiQyoGPfpu6NEEI0e7Wtkfg3xl4bXkGDUipUa11YP90Sx2vRNRK1IVNAhRCiUdUqkHAGEG8ff14pFQaMcu6PobXWn9VT/4STK5BwaAkkTki2ExdCiEbl9zoSSqlQpVSMs8AyAqM+IgWjlkLUM9fQRoutkaipEFcgIRkJIYRoDLWe/unMPizCKMDUgGuYIxxI4CRLXwv/WKVGomZk4y4hhGhU/mQknsLYU8OCUS8R6fxncbbl1msPBQA2q9RI1IhsJS6EEI3Kn0Ai3jmTA4whDZPWehXVbw0u6sDakteRqI1QGdoQQojGVNcFqRJ8rGQZXof+iGrYXOtISEbixNpFg7JAcSZUljd1b4QQotnzJ5BQSqmeSqlPgXzg70qpu13Fl0ixZYOwKKmRqBGrDdp3Nr4+erhp+yKEEC1ArQMJrfUSYASQB+QANwBPYwQVycDc+uygMEiNRC2EyuZdQgjRWPzZtMsVTCxx3iwEeiulwmR1y4YjszZqIbQLZCRIICGEEI3An23Ex/ra4VOCiIYlS2TXQohMARVCiMbiT43EfUBsfXdEnJgskV0Lsky2EEI0Gn8CiRVa6xnVNfrKVoi6a9HbiNdWaFfjKMtkCyFEg/MnkPhUKTVeKTWsmnaZtdEArDL9s+ZCXMWWkpEQQoiG5k+x5T5AAeFKqeOvagrPdSZEPTEzErLXxsnJMtlCCNFo/MlI7ANGaq0tWmvrcf8sVM3mEPXIIjUSNWcuk30IZChICCEalD+BxL1a630naH/J386I6tlkG/GaCwqG4AhwVEBJTlP3RgghmjV/FqRKqku78I/M2qglcwpohjHEseY/8P41kLmjafslhBDNjF8LUonGJzUStRTaBTK3wdK/wpHtoO3G+fVz4arZTds3IYRoRvwZ2hBNQDISteSqkzi8BZSC7ucYtw9saLo+CSFEMyQZiQDh2mtD1pGooZG3Q34a9DgXRvwF2oTBS6dD5nYoK4LWIU3dQyGEaBYkkAgQso5ELXUdCbd96Xmu8xA4mGjswxF7YVP0Sgghmh0Z2ggQVtlGvO5OH20cZXhDCCHqjQQSAUJqJOpBtzOM434JJIQQor74FUgopWKUUi8ppe5xO3exUiq0/rom3JnrSEgg4T/3jISsxyGEEPXCn23EhwMrcS6T7TqvtV4FjFZKxdRX50QVq1UyEnUW3h3ad4LSXMhJbureCCFEs+BPRuIGrXVvrfVTGMtlm5zBxIh66ZnwULX7pwQSflPKLSvxe9P2RQghmgl/Aol4t699XdUi/eyLOIGqGgmZ/lknUnAphBD1yp9Awj14UD7ae/vZF3ECNuf0T8lI1JEUXAohRL3yJ5BQbkWW2u3kMKXUd8D39dIz4UFmbdSTLsPBYjOWzy4raureCCFEwPNn064lQIRSKhd4Wym1RymVA6wCFmutV9d3J0VVICEZiToKCobOg0E7ICOxqXsjhBABz6/pn1rr6UBP4AZgmvMYq7V+ux77JtxIsWU9Ot05vCEFl0IIUWd+ryMBXKy1XqW1fts5WyNCpn42HMlI1COz4NKtbrg4B359BfLTm6ZPQggRoPxZR2IYMBJ4x/k1AFrrVIxgYlh1jxX+s0mNRP3pdtzCVGlr4a1zYcW/4MdpTds3IYQIMP5s2jVKa/0OsOT4Bq11klJqPLCxzj2rB0qp64F8jLUt8rXW85q4S36TjEQ9Cu8B7aKhOBO+fRzi/2vUTADk7G3avgkhRIBpiL02Tol1JJRS4cBUrfVKrfU04EmlVGxT98tfNnP3T1lHos6UqpoGuuEdI4gYfqtxO3df9Y8TQgjhxZ9A4mTrRJwS60horfMxhmDc5TZFX+qDa4lsu10yEvWixznGsW0HuHUJXPUKWILg6GEoL2navgkhRADxJ5D4VCn1nVJqqPtJpdRYpdQGTqF1JJzBBM5MRKLrdiBybSMuNRL1ZOSdcPXrcP9a6D0OLFZjLw6QgkshhKiFWtdIOOsgpgNLlFI93ZrygYm1WUdCKTUCeFprPcFH20SqMgixzuGJWlNKjQMmAU/68/hThdRI1LNWbauGM1wiYiA3GfL2QXT/JumWEEIEGn+KLdFarwR6Oz/p9wRStNY1Hlx2BhA3Om961S04gwi01oudt2OVUnO11pOct6cAUdU8/Qpn/9z7ulIplayUukRrnVLTfp5KzHUkZPvrhhPZE5KBvNSm7okQQgQMvwIJF+dF2ePCrJQae7KshNY6EUh0BhTjfNxlktZ6pNv9U5yZBdftk2YnnM8d6wpGgEQCODMhNRKNICLGOErBpRBC1JjfgUQ160VEAlOB0XV43nB8b0Wer5Qa555tOIlYPGeQhAMBu1OTrCPRCCKcI3WSkRBCiBqrdSChlLoOWISRBPa1+2dPH+dqIxaj3uJ4uRgBRo0CCa31YqXU9c5hkl4YxZaLT/a4U5XUSDQCV0YiTzISQghRU/5kJEZrraud7aGUeqsO/QEji+BrmmY+1ddF+FTXwEEpFeV6zaFDh57k3g1L1pFoBBE9jGNeGjgcYGmIZVaEEKJ58ecv5cmGBwKyBqEaDwG7gF2ZmZlN2hFnQgKHBodkJRpG6xBo1xHsZcZ6EkIIIU6qIT5y1XVoA3yvjhkO5NTDc9fGq0A/oF90dHQjv7QnpVTV8IbM3Gg4UnAphBC14k8gscK5+NQwpVSoe4Pz9tN17FM8RtBwvEiMmReNRmudo7XerbXebbPVaYJLvZA6iUYgBZdCCFEr/lwdPQohlVLuVzUF1Okqp7XOV0qlKKXCj1uJMrwWMzaaJZtFUY4EEg1KCi6FEKJW/MlIJAIRWmuL85/V7Z8FH7uCnkB1G3xNxS2z4VwTokUHEVCVkZApoA3IDCRSm7IXQggRMPzJSNyrtS44QftJiy2dK2JOwliMaoRSai6Q4NrmW2s9Tyk10bkIVTjGwlKT/Ohrs2KToY2GFylDG0IIURv+7rURgxEIJGut3wFQSl0MbKjJUtnOFTFPGHC4ggpRxSpTQBueFFsKIUSt1HpoQyk1HGOYQeFWFKm1XgWMdgYZogFIRqIRtO8M1tZQkg1lRU3dGyGEOOX5UyNxg9a6t9b6KcDjY5szmPC1vLWoB2aNhOy30XAsFqmTEEKIWvAnkIh3+9rXFa26AkpRRzL9s5FIICGEEDXmTyBx/HTP4/X2sy/iJGQr8UYiBZdCCFFj/gQSSil1j/Nr7XZymFLqO+D7eumZ8CIZiUYiBZdCCFFjtQ4ktNZLgAilVC7wtlJqj1IqB1gFLNJar67vTgqD1Eg0EhnaEEKIGvNr3Wet9XSl1DxgFMa23ylA/EnWlxB1ZLNKRqJRmMtk1yIjUVYE71wC3c+Cq2Y3TL+EEOIU5M/0z0+VUjFa6wKt9Sqt9dvOY4HbfcY77ze+frvbssk6Eo3EtZ14fjo47FXn7RXVP2bfT5C1A5I+hPKShu2fEEKcQvypkViotU6trtG5MNXTwH+AM5RSY/3smziOrCPRSIKCIeQ0cFRCwQHj3I6vYGpP+O4fvh+TttY4OirgwO+N008hhDgF+LWNuFLqcaXUXmd9xLXHNU8EntRaJznXmpB1JeqJVcleG43GvU4i8X1YeBuUF8Gmj8FXRsgVSADs+7kxeiiEEKcEf2okYoEojCWyI4CblVL7tNYbne0jMGomXKT0vZ7IrI1GFBED6b/BDy/C/vXGOWtrKMmBI1vgtKFV9y07Coc2Vd1OlUBCCNFy+JORyNdaP+Wsi1istb4Bo+jSJfK4oQ+56tUTKbZsRK6Cy/3rAQWXz4AhE4xzyT943vfABtB2iOoNygIZCVBe3KjdFUKIplLXBalcfC1MJeqZZCQaUWSscbTY4Lp34Ix7IfYi41zKcYFE+m/Gsc8fjEyFoxLS1zVeX4UQogn5E0hEKKVecs7MuMe5CFUymIWWSinVw+3+veqjo6Kq2FJqJBpB/8vhrAfgL5/D4OuNc7EXGse036CitOq+rvqI7mdDzHnG16m/NFZPhRCiSfmzINV0IA+YBkwA5mEED08A44CewNNKqbFKqTeBFfXY3xatKiMh0z8bXKt2cOlL0PP8qnPtOkDnIWAvq8pCVJbDAef2M93Prrq/1EkIIVoIfxekmoYRSLhb5fpCKTUVoxhzsVsRpqgjm7mOhGQkmkyvi+DwZqNOotdYo8iyshSi+kD7jq8Oe1cAACAASURBVMaCVMoKGYnGIlWtQ5q6x0II0aD8WZBq7MkWmtJa73MVZPrfNXE8qZE4BRxfJ5HuHNbocbZxbB0CXYYbxZfp6xu/f0II0cj8qZG4D2MKqGhkstfGKaD7WcY00MNb4GiWUS8B0P2cqvvEnGscU39q/P4JIUQj8yeQWKG1nlFdoyyL3XCsso140wsKrso+pKypqpXo4RZI9JSCSyFEy+FPIPGpc8bGsGrab6xLh0T1ZInsU4RreGP9W3AsH0K7Qnj3qvZuZxnTRg9uhGOFVee19r0qphBCBDB/Aol9wDtAolLKftw/B3B9/XZRuFhl+uepoZczkMhwm62h3JZSad0euoxw1kn8BpVlsH4ezBwA/7uy8fsrhBANyJ9ZG/uACVprn0tfK6UW1q1LojpmRsIun2qbVKfB0LYDlGQbt11DHe56nmds3vXrHPhmMhTsN84XHZLZHEKIZsWfjMS91QURTi/52xlxYlaZ/nlqsFgg9oKq2+6Fli6ugsu0X40gouMAaNfROJeX2uBdFEKIxuLPglRJSqkY5+qW97jOK6UuVkqFaq2T6reLwkX22jiFuOok2oRDx/7e7d3PhuhBxvoS49+G+381hjtAAgkhRLNS66ENpdRwYBGwGAh3nddar3IGE8nHbdol6olFthE/dfS/Ata9aSylbfERjwcFwwNrPc+5tibPlQ1xhRDNhz81EjdorXsDKKWuc29wBhPjgdR66Js4jszaOIW0jfQOFE4m0rmjqGQkhBDNiD81EvFuX/u6okX62RdxErKyZYBzZSTyJCMhhGg+6rqNuK/tw3v72RdxEpKRCHARkpEQQjQ//gQSyq3IUrudHObcUvz7eumZ8GK1So1EQIvoYRzz08Fe2bR9EUKIeuLPrI0lQIRSKhd4Wym1RymVg7H75yKt9er67qQw2GQb8cAWFAwhp4GjEgozmro3QghRL/yZtTFMaz1dKTUPGIWxgVcKEK+1LqjvDooqso5EMxARYyxKlbevKkMhhBABzJ9ZG4uAPs6gYZXzn2gEUiPRDET0NJbNljoJIUQz4U+NRJRrMSqlVGi990hUyyJ7bQQ+c+ZGalP2Qggh6o0/GYmRriWynQtQxQLJUhvR8FwZCYcEEoHLtZaELEolhGgmah1IuO+zobVeBaxSSoU5F6KKBFbKypYNQ3b/bAYkIyGEaGb8Gdrw5WLgPmAaMKmenlMcR2okmgFZlEoI0czUOpBwZh5c60a85ZwGOgmYq7WO1Fo/Xd+dFAbJSDQD7TpCUDs4VgCleU3dGyGEqDN/MhKLlVJ7MGZr7AV6aq3/6FxfQjQgm3P6p6wjEcCUks27hBDNij+BRD4wSWsdpbWeIWtHNB4zI2GXjERAk827hBDNiD+BxEsyQ6NpyKZdzYTUSQghmhF/lsiefqJ2pdRY/7sjTsQmNRLNg8zcEEI0I/6sIwEYxZY+TkcCU4HRfvdIVMuVkXBoCSQCWoSsJSGEaD782WvjOoxlspPxvY14z7p2qr4opVYA4zDqOl7SWk9r4i7ViU1qJJoHs0YirWn7IYQQ9cCfjMRorXW1QyJKqbfq0J/6NheYoLXOb+qO1AepkWgmwroBCgoPQGU52Fo1dY+EEMJv/hRbbjhJ+5P+dKShNJcgAsBmddVIyPTPgGZrBWGng3ZAwf6m7o0QQtRJfa1s6e6UGdoAYpVS45RS1yulJjZ1Z+rKaq4jIRmJgCdrSQghmgl/AokVSqmxzpUtPXb/dN6u8cqWSqkRSqlF1bRNdAYA1yulpvjRT4B5WuuVWuvFwATnBmMBS2ZtNCMyBVQI0Uz4UyPhMVSglHK/qingpFc5pdQI4EbnTa+Luyt74AwAUErFKqXmaq0nOW9PAaKqefoVWuuVzse79zUFo/By3sn6d6qyKKmRaDZkUSohRDPhTyCRCFxc3YqWSqmFJ3sCrXUikOgMKMb5uMskrfVIt/unKKXGud0+6ewL5/0nuIKP5sBVIyGBRDMga0kIIZoJf4Y27j3Jsth1KrZUSoUDI3w05bsHEzWQgjFN1WUUcNIg51QmszaakQjJSAghmodaZyS01klg1kOMw1h8KhlYqbVO1VrXddA3luOGT5xyMQKMlTXsZ4qzBmMiEA48GegzOKRGohlxL7bU2tjMSwghApBfK1sqpe7FWMEyBeMCfwkwVyk1RWv9ch37FOl8zuPlU31dhE+uGgt/KaWiXK85dOjQujxVvZCMRDPSNhLahBnbiZfkQLsOTd0j8f/bu38Yx7H8TuDfV9WobSxudtgqeNaAA++wgFnAsBO1OvNdUhQuODgxpKrMBm5vJFywmSFakX1RreTQBu7E2sBODFSJuOguMMQ2HDgrFWFgE/sAPdvAJTeHUXMuahS65l3AP0VRlERR/8iq7wcodIuiKOrVH/74e7/3HhHlkndmS00pVUl57hdCiN9XSv33rZzd4f0cwJ8AwDfffHPgU3laRpzzSDwTrb8D/s2vAyc/PPSZ0JqUUnh4eIDidPVUAkIInJycQOwo85mnRqKyaOEupdQfAzjb7JT890jZpgH4dgvHXsefA/gpgJ9+8cUXe37recxIPDMVnUFEySil8OHDB3z33Xd4fHw89OkQZfL4+IjvvvsOHz582Enwm6drY9XF/EOeE4kZww8akirwR4zsjVLqWwSft1ar7fOtUx2zRoLooDzPw2effYZXr3Kvd0h0ED/84Q/x6dMneJ6HN2/ebPXYuTISK57f6AyDgkgZjN6I08L5IV6qKCPBRbuI9k4pBSEEgwgqrVevXkEIsfWsRJ5A4p+FEFdCiM/iG4UQPxJC/FcA92sca1FQ0kNshsxgvokXHUQAT6M2HtkvS7R3Dw8PODnhAmtUbicnJ3h4eNjqMfMM/3wvhHgD4F+FEPFuDh3+RFJ/u+oYwVTVbfjDR6tCiAGAe6WUFbyHFUyRbcDv5tCf08RSebFrg+hwlFI4OtrF8kRE+7OLjESuHF0wrNIWQpzDDyAkgPGKiarir5dYMXFVGFTQk1cstiSiF0ZKCV3f/jJJ2z6uZVnwPH+qIl3XoWlPvfOGYaTuG+5TqVSi/yf33bZdjNxYGkgEgcLnic2OUur/AVF2YhpOUkW7FR+1EfbXEhHti+d5uLq6Qq/Xi7a1220MBoONjmuaJur1+txF1HXdnQUSruvC8zxUq2kTKWf/rFJKmKaJbrc7cywpJc7Ozmbu/tfZt0xWZSSm8DMHBoABgGEYRMR4wQRVSin1yx2cIwWEEDg+Enj8XuHxexWtvUFEtA/T6RRSyplt4/E402td18XNzQ1OT/15BTsdf1Fnz/PQ7/dxeno6E0h4nofBYLBxkLJIo9FAu91Gr9ebyR6EsnxWz/NQr9cxGo3mgh1d19FqtXLtWzZLO/yCTMMQwJlS6j+n1T8opf5ZKXWtlPqlEOKPdnWi5GOdBBGV0fn5ObrdLu7u7jAajaLtjuPX0ScvpFdXVzDNjZZuWsk0TVxdXeV+fbPZRLvdXpgxiZ//OvuWzaqujd+Hvyz3v2Q83rUQ4j8xM7E7x1xKnIhKxrKsqA7g+vp6JgMwGo3QarXmsgKu6+6kSyNO13W4br7piTzPg+M4GA6HC/cJzz/cd1l2ZdefdZdWlSC/WyOIQFBsmTaZFG0JF+4iorK5v7+PagKSAcPt7S3a7dlBebZto9ls7uXcms0mLGv92v7xeAxN01K7RQBEhZfhvsB8sGDbNtrtNprNJmx7o6WhDmoXY5nWWliL1nMc1EV8z0CCiErCcRzU6/W57WEhZbLg8ebmBhcXF3s5t4uLi6VZhUVWZRDCLptl+zYajSgQaTQaa59DUawKJPLkWsqbnykBZiSIqCz6/T6azSaklBiNRmi32zNdCY7joNvtzr3Odd2Fd/pSSliWhX6/H2Uy+v0+LMvKlcXQNC1zwWicruswDCM1k2Db9kxgoOs6Go3G3L6e58G27dQgq0xWjdoQQoifZO3eEEJ8CYBDCXaIC3cRUVl0Oh24rgvbtlPv+g3DWKs2IKw1CAsz6/U66vU6hsMhHMeBbdsz8zPs2nA4hGmakFJC07SoDiQtu5C2LwD0er2FQ1DLYlUg8Qv4wz7/fcbj/TesmGiKNsOlxImoTMbj8cJgIW37skDg9vZ2ZnTHdDqFYRjRxXsymeQKInRdzx2AxOeZ2Oa+ZbJq+KcL4L0Q4m+EEL+5aD8hxE+EEH8Df7Kqf9j2SdITZiSIqEzihZZZSCmju/WkZN2E67oz3QLJwMQ0TZydna18z0qlMjdnBGW3copspVRfCHEKf7Gue/iLZ4VrbJwiWC8DQF8p9Wc7O1MCwBoJIiqX8XiMy8vLzPsvywrEnwuLGZdNKf3u3buZ0RN535eWy7TWhlLKFELcwF+Rs42nIZ4e/MCixWmy9+OIGQkiKhHXdddK6WfNDoxGo5WZjtFolKkAc1kWZBnXdTEej+F5Hu7u7tDr9Uo9H0RemRftCro59jOwlxbiwl1ExfSTP/6fhz6FTP7lF/9hb+8VBgS1Wi3zazRNw3Q6XXi88ELtOM7McT3Pw3g8nslQhJNASSkhpcRgMEgt+pxOp2tnJML3C2s2wiGuk8lkreM8B1wTt2RYI0FEZREO41z3Ip12V+84Ds7OziCljIaQxo9rWdZMEBEGHeG+hmEs7GLJk0WQUs5kWmq1GqSUmbtSnpNcy4jT4bBGgqiY9nmnXxZ3d3e5lsWu1WpwXXem6yJc2CqsjXj//j1M04xmpUyu1eE4TpSJCIdjpg3LdBxnrRqOULVanVkzZNVMl88ZA4mSecpIcPgnERXbogmnVgmX646vTaHr+txaFcvWrhiNRhgOh5BSRqtupgnnd8gjnskYDAa4vr7OdZyyY9dGyUTzSDwyI0FExeO6btSd4LpuroxEtVrdeDhmmNEwDCOquYhPWx2aTqcbF0haloXLy8tST3O9CQYSJcMaCSIqsvPzc9zc3MBxHHQ6ndyp/ni3xbqklFEAo2kaDMOYK84E/AAguWDYuhzHiabAfqkYSJTMMWskiKjAut0uzs7OMBwON5rJ0TAMTCaTXMWLyW6QXq8XzYAZ8jwPnuflypiEXNdFpVKJjlHmFTw3wRqJkmFGgogOpVKpzHUDJB93Op2tvV+v14NlWXOFlNsQZkwWWfVZpZQ4Pz+fCXReamZCKMULUha1Wk3lWSFu2372l3d4/4/f4Jd/UIPxWz8+9OkQvRgfP34EALx+/frAZ0KU3zo/x0KIe6XUyklA2LVRMuzaICKiImEgUTKvjtm1QURExcFAomSOuYw4EREVCAOJkuFaG0REVCQMJEqGNRJERFQkDCRK5lgwI0FERMXBQKJkjllsSUREBcJAomRYI0FEREXCQKJkWCNBRERFwkCiZF5xGXEiIioQrrVRMk/zSDAjQURUFJZlRetu6Lo+s0BYcmGwcN9wn0qlEv1/k0XEDoWBRMlEGYlHBhJE9PxJKecWzyrScaWUME0T3W4X1Wp1ZvvZ2Rni61mts2+ZsGujZI5YI0FEB+J5HkzTnNnWbrc3Pq5pmnAcZ26767pwXXfj46dZdewsn9XzPNTrdfR6vZnAAPCzEvFVS9fZt2wYSJQMR20Q0aFMp1NIKWe2ZV0V2XVdmKaJfr+Pfr8fbfc8D/1+f+6i7nkeBoPBzpblbjQaGAwGM8uAx2X5rM1mE+12e2FmIx6IrLMv4Hd/nJ2dwbZtWJY1F8RYloU3b95Ezydfv08MJEomHLXxWNIUGBG9TOfn5+h2u7i7u8NoNIq2h5mI5B351dXVzi+Opmni6uoq12s9z4PjOEszCWHQEO67LChKBhi1Wg2GYaDRaKDVakHTNFiWNfN8q9WKnpdSpmZ19oGBRMkwI0FEZWNZVlRQeH19PRNIjEaj6EIZ57ruTmoj4nRdz911Mh6PoWna3HmH4pmOMJOR/Dy2baPdbqPZbMK27ZnnHMfB27dvZ45XqVRmnq/X69FzjuPsvL0WYSBRMtE8Eiy2JKKSuL+/j+oCkhfe29vbubS9bdtoNpt7Obdmszlzp5/Vqot2PDuwaN9GoxG1RzJbcXd3h1qtBsAPFKSUM/vc3d0B8IO0r7/+Gu/fv2cgQdlwHgkiKpv43XNcOHIiWXx4c3ODi4uLvZzbxcUFhsPh2q/TdR2GYcxlEgA/EIpf9HVdR6PRmNvX8zzYtp3aNq7rRt0VjuPMZHHC1xqGgVarhUqlMteG+8ThnyVzfMx5JIioHPr9Pu7u7iClxGg0wv39PdrtdnTRcxwH3W537nWu6y7sMggvrp7nYTKZYDAYoN/vQ9M0jEajtYMCTdMyF4wmDYdDmKYJKSU0TYu6b9JqIdL2BZA6iiPsFllUUxGfgwJA1B6L2mzXGEiUDGskiKgsOp0OXNeFbdupF3jDMNZKxycLHOv1Our1OobDIRzHgW3be7+g9nq9re87Ho+XTkw1Ho9n2i0MTg4VTLBro2TCZcSZkSCiMkhe9OLSti+7GN7e3s6MkphOp6hWq1EWYDKZ5LqQ6rq+cBjovkkp0ev1MJ1OU88pfD7+XKPRgOM4mE6n+zzVyLPOSAghkldbRyk13xlVIsfMSBBRicQLLbOQUs6MTohL1k24rjtzl58MTEzThG3bmEwmS9+zUqlASnnQOoOQrutz9RCrns9T47FNzzaQEEJUAZwppWTscTFCzg28OmYgQUTlMR6PcXl5mXn/ZRmFZF0AsHxtinfv3mXONByqvuA5eM5dGzIMIgJ64nEpMSNBRGXiuu5ad/phdmCV0Wi08rij0SjTMNJlWRBa7dkGEkqpKAwVQrSUUvNjdEroVbTWBod/ElGxhQFBOB9CFpqmLezrjwcYjuPMHDcsxIxzHAeGYUQjPRYFFdPpNFdGwnVdWJaFfr+PZrOZKQB6jg7atRF0N3SVUnPfXSFEC0D406QrpfrJfdZwtsFrCyVcRpwZCaKC+dPPD30G2fzpd3t7q3AY57oX6bQizHAuislkEnVXxI9rWRY6nU70OJyjIjwHwzAWdnPkmcjJ8zyMx+Oo+DN+fi/NQQKJIIAIO83mvoNBEIEwiyCE0IUQA6VUO3jcAXC64PAjpVQUlgohGgCezXf2FVf/JKKSuLu7W1rDsEitVpvrEglXyAyzDu/fv4dpmtGslMk1LxzHgZRyZkbItHkZHMdZq4YjFI6eCN+3VqtBSnnQ+RwO5SCBhFLKBeAGAUXaT1lbKfU2tr8UQhixx+tkJ+oAFpfAlswRaySIimmPd/plsWjCqVXa7TYGgwEGg0G0Tdf1mccA5h7HhZNTSSlRr9cXjoQIJ4paV7VanTnmqrU3nrPC1UgIITQAaRU0XjyYWIMO4Nl0XL3iWhtEVGCu60YLYbmumysjUa1WN643CDMahmFENRdpq2NOp9Pca1TEXzcYDHB9fZ3vZEuucIEE/At/WkfWFOkBRha5hn0KIU6FEF8JIb769OlTzrfeLi4jTkRFdn5+jpubGziOg06nk/sOPd5tsS4pZRTAhPURyeJMwK+rSC4YlodlWbi8vFy6TPhzVsR5JCp4KrKM87C4LmKhDSeg+jmAPwGAb775ZoPDbA+nyCaiIut2u9A0DcPhcGnXwyqGYcA0zVw1B8lukLSpqT3Pixa+2kS4fPemxymzIgYSRfLnAP4aAL744ot/OvC5AIgtI85Agoj2rFKpzHUDJB/HR05sqtfrwbKsuULKbQgzJotk+ayu686svJlc9fOlEOqAKfKg2PI6XlgZ1EEMlVJvEvuO4I/I2GQYaG61Wk3lXSFum371v7/D7/3F3+O3f+NH+B8//7eHPh2iF+Pjx48AgNevXx/4TKgIpJR4+/btzJBSXdcLP/xznZ9jIcS9UmrlJCBFzEiMAaTlsSoA3D2fS+Ecs9iSiOjgdF3Hhw8fDn0ahVC4YstgRkoZjN6I0+LzQ7xUXGuDiIiK5NCBxKLJzXsAosHHQRfIiw8iAOBIMJAgIqLiONTMljqANvzJqKpCiAGAe6WUBQBKKUsI0QrqJTT4U2RvPkbnGeDMlkREVCSHmtlSAlg6lVgYVNAsrv5JRERFcuiuDVoTaySIiKhIGEiUDOeRICKiImEgUTKvomXEvz/wmRARETGQKB1mJIiIqEgYSJQMiy2JiKhIGEiUDId/EhFRkRRximxaIsxIfM9AgoheACnl3GJZRTyuZVkzK5VWKpXo/899ZVBmJErmWDxlJA654BoRvTye58E0Z6cAarc3nyvQNE04zvzkxa7rwnV3s8TSqmNn/axSSjSbTdRqNXQ6HbRaLbRaLVSrVdTr9WcfRADMSJTO0ZHAkQC+V/5XMK0EEdHOTadTSClntmVdFdl1Xdzc3OD09BTA03Ljnueh3+/j9PR05qLreR4GgwEGg8GWzn5Wo9FAu91Gr9eLMgdxWT6r53mo1+sYjUapS47vYvnzImJGooTCIaCfOASUiEri/Pwc3W4Xd3d3GI1G0fYwE5G86F5dXc1lBLbNNE1cXV3lfn2z2US73V7YRbLr8y8KZiRK6PhIAI8cuUFE5WBZVlQzcH19PZMBGI1GaLVac1kB13V3UhsRp+t67q4Tz/PgOM7SjMmuz78omJEoIY7cIKIyub+/R7VaBYC5gOH29nau9sC2bTSbzb2cW7PZhGWtv7RT2M2RDBZs20a73Uaz2YRt21s5x6JjIFFCR+FcEo8MJIio+BzHQb1en9sejpwIg4zQzc0NLi4u9nJuFxcXGA6Ha79uUbah0WhEwVKj0djo3MqCgUQJMSNBRGXQ7/fRbDYhpcRoNEK73Z7pSnAcB91ud+51ruumFkACfvBhWRb6/X6Uyej3+7AsK1cWQ9O0zAWjcbquo9FozGUdPM+DbdupgdNzxRqJEormkuDwTyIqsE6nA9d1Ydt26l2/YRhr1RGEdQlhYWa9Xke9XsdwOITjOLBte2Yuh10bDocwTRNSSmiahkqlAgDo9XpzWZbnjIFECTEjQURlMR6PFwYLaduXBQK3t7czozum0ykMw4CmaWg0GphMJrmCCF3XcwcgvV5v7dc8N+zaKKHjY9ZIEFE5xAsts5BSRnf2Scm6Cdd1Z7oQkoGJaZo4Oztb+Z6VSmVuzgjKjhmJEuI8EkRUFuPxGJeXl5n3X5YViD8Xzj+xbObId+/ewfO8jd+XlmNGooS4AigRlYXrumtlJLJmB0aj0crjjkajTAWYy7Igy7iuGxV+hkWlLxEzEiXEGgmi4vmdv/qdQ59CJr/6w1/t7b3CC2utVsv8Gk3TMJ1OFx4v7L5wHGfmuJ7nYTwez2QowgmjpJSQUmIwGKQWfU6n07UzEuH7hTUb4RDXyWSy1nGeA2YkSuhIMCNBRMUXDuNc9yKdVoTpOA7Ozs4gpYyGkMaPa1nWTBARBh3hvoZhLOxiyTMDpZRyptCyVqtBSpm5K+U5YUaihF4dM5AgKpp93umXxd3dXa7VL2u12lyXSLgIVlgb8f79e5imGc1KmVyrw3GcKBMRTgyVNkGU4zhr1XCEqtXqzJoh4/E4V9D0HDCQKKFjdm0QUQksmnBqlXa7Pbfyp67rc+taLFvnYjQaYTgcQkoZrdCZJpwLIo94JmMwGOD6+jrXccqOXRsl9IrFlkRUUK7rRt0JruvmykhUq9WNCxfDjIZhGFHNRZjNiJtOpxsvrmVZFi4vL1/MlNhJDCRK6CkjweGfRFQs5+fnuLm5geM46HQ6uVP98W6LdUkpowBG0zQYhjFXnAn4AUBywbB1OY4TTZf9UjGQKKFwHglmJIioaLrdLs7OzjAcDjea9dEwDEwmk1zFi8lukF6vF82AGfI8D57n5cqYhFzXRaVSiY7xUlb7TGKNRAmxRoKIDqFSqcx1AyQfdzqdrb1fr9eDZVlzhZTbEGZMFln1WaWUOD8/nwl0XmpmQigu/JRJrVZTeVaI24X/+Jd3+Nt//Aa//IMajN/68aFPh+hF+PjxIwDg9evXBz4TovzW+TkWQtwrpVZOAsKujRJiRoKIiIqCgUQJveIy4kREVBAMJEqIGQkiIioKBhIl9DSPBId/EhHRYTGQKKHjcBnxR2YkiIjosBhIlBBntiQioqJgIFFCX/36Z/h3X/0afvwjDkMjIqLsdjHlAyekKqGf/e6X+Nnvfnno0yB6UYQQeHx8PPRpEG1EKYWjo+3mEJiRICLK4OTkBA8PD4c+DaKNPDw84OTkZKvHZCBBRJSBEAJKKXz69OnQp0KUy6dPn6CUghBiq8dl1wYRUUaapsHzPAghcHJyAiHE1v8oE22TUgpKKTw8PEAplXs11mUYSBARZSSEwJs3b2b+MBMVmRACR0dH+Pzzz3cW9DKQICJakxACP/jBDw59GkSFwBoJIiIiyo2BBBEREeXGQIKIiIhyYyBBREREuTGQICIiotwYSBAREVFuDCSIiIgoNwYSRERElJvgzGzZCCH+L4B/3dLhjgH8GMD/AcDlBPNhG26ObbgdbMfNsQ03t4s2/E2l1K+t2omBxAEIIb4C8E8AfqqU+l+HPp8yYhtujm24HWzHzbENN3fINmTXBhEREeXGQIKIiIhyYyBxGN8C+C/Bv5QP23BzbMPtYDtujm24uYO1IWskiIiIKDdmJIiIiCg3BhJERESUGwMJIiIiyo2BBBEREeXGQIKIiIhyYyBBREREuTGQICIiotxeHfoEXhohRAvANHioK6X6hzyfMgjaDADeBv+aSikv8TzbNCMhxFAp1UxsYxtmIIToAPAQtJVSyo49xzbMIPb7rAE4BXDF3+flhBBVAN3k723w3NL22kd7MpDYo/AXKPzjI4TQhRADpVT7sGdWXEKIllLKij8GcA/gLPaYbZpR8AepkdjGNsxACDGEH8TK4LESQrxRSnlsw2yCQMxKBA5DAM3g/2zHmOD39TJ4qKc8v7S99tWenNlyj4QQ90qpt4ltE6XU2aHOqciEEBqAi3ggEWz/AKCplHLYpusRQhgARkopEdvGNlwh+IOsxe/mhBB6LKhgBRnF4gAAB3RJREFUG2awIBvWQ5CVYDumCwKK65S2Wdpe+2pP1kjsSXBRrKY85QV/3GmeDmAQtF2cBKCzTdcjhGgopZzENrZhNj0AdnxDLIhgG2anBxfFOC0IItiOa1jVXvtsTwYS+6PD71tNmiL9m/3iKaVcAG/jadCAjiCYANs0k+CPt5vyFNtwheAPshb8vxH8ke7EAly2YXZfA7gPujjCDNkgeI7tuJ5V7bW39mQgsT8VPBW8xHnwC44oRRBMRIQQDQAyuLNmm2YXpeET2IarhX+QNaWUHfzsWQDeB8+zDTMKfp/PAHSDLsr47zjbcT2r2mtv7clAgkojuAPsAjg/9LmUSdClYa/ekxaowM9IRIFYmCVjyn09QggdfrHvl/CDsVFsFAeVFEdt7FclZZuGA6wfX1I9+EWW8XQd23SJ4A93WiYijm24nASegoeYMEXsgm2YlRkbMWAKIW4AvBdChD+jbMf1rGqvvbQnA4n9GSPoZ02oIL3vmmKCPtVeIj3PNl3NAKAl75xj8yHcgm24lFJKCiEWPe2BP4eZhCOG4tuUUq4QogmgDuAKbMd1rPq529vPJQOJPQmqkqUQQkvc2WjJSnqaFaQ+7XgQIYQwguGfbNMlkkNnAX+4XWIYI9twNTc+3DOgAxjzd3tjEsC3bMf1ZGmvfbUnayT2qwe/jx9AVEnPX5AlgruYcXyoXeLumm26ObbhambwBSBqIxkrFGQbrhBcvC5TnmrAr5cA2I6LpHVRAKvbay/tyQmp9iy4u5bwU06c/nWJoH9/suDpN7GCN7ZpBkEA1ob/h9sGMIjdubANVwhGDIWzC54qpczE82zDFWIF098iGAmD+Wwj2zEQ/A1sw++irMIPuO5TZvtd2F77aE8GEkRERJQbuzaIiIgoNwYSRERElBsDCSIiIsqNgQQRERHlxkCCiIiIcmMgQUS0QjB/CVegJErBQIJoj4QQAyHERAihhBDDYG6CQguWzl40n8ezFwQQQwD3hz6XJCGELoQYBT9Pg9WvINo+ziNBtGfBBDE9pdSblOeqyaXT92XRewcX0svkBEwvSTCR0gel1MJFNw5JCKHgL2jHVV5p75iRICqWQy5LnfreSin3JQcRQOrKn4UR63LhVNJ0EAwkiIolbS2Cl/DelJ8Bf92PwgY79LwxkCAqgKCYrwd/Pv1CvbcQoppchrxI0gohw21Bl8RzVwezEXRAXEacqBgu4C+qg+CiDgCTxOI88cLMdwBulFJucBG9hr+g1JfBseqYXZQr+dqr2B3swveOLbLUADBTHxAEFzqAKfzVCadhH33snKoA3uCp2+QdgLtVfflZXx8satSLn19wzj0ALQBv4S8Bvuh4dQAjpZQdayMdANIWNwreT4/tN037LJt8r3IwADRjn70VbJ9bWIxoJ5RS/OIXv/b4Bf8P/YeU7Zr/K5n6mg6AVmLbBP5qfoB/YfoAoBHbP/x/I9wveFyFf/HM9N7B8yrxuAG/YHThOYbHBGAsO9aS98z8+gXbPgCoZjleyrZJ/LWx/RqJbYOU70vu71WOn6VqcF5a8NWKndfkED/f/Hp5X8xIEBVcmBVQ86M8bPhLDJtKKRnsJ4G5u+k6/PqHZvCcK4QwhBCayt+vfg3/jjqilOoLIT4IIW6VUp5SyhNCAH7GIk5mGZ2y6euTr1t2PDxlGUIugFrwb/wYyeyDCSD6zFv4Xq3LCI8D4EI9ZbCYiaC9YSBBVHw1YC5dDgDfApgJBBZcXE0AlSCtXsPThbSSfH0WQZfGdEEQIuFf3OzEtk1s+vosx8v1HrHgpAa/TmHT79W66sG/8SACGwSIRGtjIEFUfAv74jOqwJ9Q6Sq82AghNrnQJO/e46bwawKic930orbti+KC4yWzFOuIZzQ2/V6tywDQB/BWCHEP4OstBShEmXHUBlFBxUYdSPjBQF4j+MV8aUWBqaMaVox4WHb3XoFfD/CS6Hhqk02/V5nFRqpcKaXaAG7gB4xEe8VAgqggUu6Uw7vcMTBz4YisGpYZBAM6gNvEU2GQUFtQK7Es6zAGoC8INKo40FDElPNZ9hm29Z46/C6LcbAp9/cqh+T8EVEQEwx/Lfz06/Q8MJAgKhYZXJwAoBIWLcKvc+jFdwwuTOPkAeJiF5noLjl4nYug0j9+IUq+95JjtlPOp4egmDCxPS3gyHzXnvH1LmKBQ/AZU7MDG5yPl3Jx7sHvTvCAqG1yfa/CcwvWY8ky/0Xa/BHxwktOl037cehhI/zi10v6QjAsD/6QvSHmhxNWg30amB+SaOBpfoRoSGfwmuGSYxrh9uD/1fi2Ze8N/+IcP7aeOG4nOJ9O4ljJ11XhBy6DYNt98vMlznmt1wfPh/NJNILH90Fbt9Y5XnCcD+FrY+/RCY5jxD576mfY4Hs1MzR0xc/SPeaHqA6T3wt+8WvXX1y0i4ioYIQQDcWMApUEuzaIiIqHwzepNBhIEBEVSJCN4NoZVBoMJIiIioXZCCoV1kgQERFRbsxIEBERUW4MJIiIiCg3BhJERESUGwMJIiIiyo2BBBEREeXGQIKIiIhy+/8vz30uvW7efAAAAABJRU5ErkJggg==\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plt.figure(figsize=(8,6))\n", + "plt.semilogy([np.linalg.norm(grad_f(x)) for x in cg_quad.get_convergence()], label=r\"$\\|f'(x_k)\\|^{CG}_2$\", linewidth=2)\n", + "plt.semilogy([np.linalg.norm(grad_f(x)) for x in scopt_cg_array[:50]], label=r\"$\\|f'(x_k)\\|^{CG_{PR}}_2$\", linewidth=2)\n", + "plt.semilogy([np.linalg.norm(grad_f(x)) for x in gd.get_convergence()], label=r\"$\\|f'(x_k)\\|^{G}_2$\", linewidth=2)\n", + "plt.legend(loc=\"best\", fontsize=20)\n", + "plt.xlabel(r\"Iteration number, $k$\", fontsize=20)\n", + "plt.ylabel(\"Convergence rate\", fontsize=20)\n", + "plt.xticks(fontsize=18)\n", + "_ = plt.yticks(fontsize=18)" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[8.845930974954362, 14.015059201881973, 10.16744487372933, 5.610074694671215, 3.8187218127914515e-09]\n" + ] + } + ], + "source": [ + "print([np.linalg.norm(grad_f(x)) for x in cg_quad.get_convergence()])" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhEAAAGKCAYAAACy1xMPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xlc1Oe1+PHPM6yiwgiIqGh00DRuURE10WgTwSa1SVMNmKZJm9s2SnvbdLltpSbtbe/tvbWaLr/b3psW0ts17a0Bm10TGRMTY6ICI+51YUTFDQQHtygCz++PWRxWZ4ZZ4bxfL17AfIfvHFpfmcN5znMepbVGCCGEEMJbhlAHIIQQQojIJEmEEEIIIXwiSYQQQgghfCJJhBBCCCF8IkmEEEIIIXwiSYQQQgghfCJJhBBCCCF8IkmEEEIIIXwiSYQQQgghfBId6gDCXWpqqh4zZkyowxBCCCGCorKy8pzWeqgnz5Uk4ibGjBlDRUVFqMMQQgghgkIpdczT58pyhhBCCCF8IkmEEEIIIXwiSYQQQgghfCJJhBBCCCF8IkmEEEIIIXwiSYQQQgghfCJJhBBCCCF8InMihBAiAmitaW5uRmsd6lBEhFBKERsbi1IqYK/RJ5MIpdRyoNHxrUlrvSaU8QghhK+01thsNtcbgsEgBWThmdbWVpqamtBaYzQaA5JM9LkkwpFAoLUudXxvUkoVaa0LQhuZEEJ4z2azMXjwYKKj+9x/rkUQJCQk0NLSgs1mY8iQIX6/f1/8V1mgtZ7h/EZrbVVK5YYyICGE8IXWGqWUJBCiV6Kjo1FKuf49+VOfqosppYxAVheXbOGQSLS1tdFw5WKowxBCRIjm5mZiY2NDHYboA2JjY2lubvb7fftUEgGYAFsXjzfSdXLRJaVUilLqVqXUrS0tLX4L7nMv/ojc/8vnYP0pv91TCNF3aa2lB0L4hbMS4W997V9nMjcaKt3ZgBQv7vMkcBA4WFdX54+4qL90gb22d2mJPsnSVx6j8mS1X+4rhBBC3EygdmjIQlvXfgX8FSAtLe2gP244dFAiJQ8+z8Mvf4Hr0bV8/s1/4pd3/5q7TZP9cXshhBBdsFqtmEymXt2juLgYm81e5DaZTBiNRte13NzcLp/rfE5ycrLr69zcXL/EE076YhKR3MVjRqDB0xtorRucz8/OzvZTWDA+dTiv3v/fLH7923wYdYQnNy/jx9d+yQMTZvrtNYQQQthZLJZevWlbrVYKCwtZuXIlWVlZ7R7PzMxstzzg6XMtFgs2m63dcyJZX0siKrAnDB0lA5Ygx9JZ0UcZebqKjV/awaI3fsjFqL08te2f0VE/4ZO35oQ6OiGEiBgWi4W1a9eSkmJfqV6xYkW76zabjaKiIoqKiny6v81mY+HChZSVlXVKQkwmE8uXL/fpuXl5eRQUFLB69ep2FY1I1ad6IrTWNsDq2KXhzqi1NocipnYS7EUSY+NByh75A4NbssFwlac/+Ca/rvo1bbotxAEKIURkyMnJYeXKlZSXl1NWVtbp+qpVqygsLPT5/vn5+RQUFHRbxXC/tzfPdX6/atUqn2MLJ30qiXBYDax0fqOUygJCn0AAjJhu/3y6ioFxcTxqWsm1unsBeHbXs3ztra9xoflCCAMUQojwV1xc7Oo1eO6557pMIiwWi8/LGDabDbPZ3K6C0JHz3s7n5uXl3fS57t9bLKEvjvtDn0sitNbFQLVSKlcplQfkhs20yuHT7J9P7QRgakYyzQ33MPzqV0mKS+Kd2nf49GufZl/DvhAGKYQQ4a2ystLVU9DVkkBpaSn5+fk+37+iogKj0djtcoOzydL5XOicKJSWllJQUEB+fj6lpaWd7pGfn09xcbHPMYaLPpdEgD2R0FqbtdalYXVuhrMScaoKtGbyyEQAjp0YxV8+/lduS76NExdP8Nj6x/jd3t/J8oYQQnTBbDazcOHCbq+vXbuWpUuX+nz/m1UwzOYbxe3unpuXl+dKQrqqUixdupSSkhKfYwwXfTKJCFtJGZCQAh82gu04xoRYRicncK2ljStXjDy/6Hk+c9tnaGlr4ReVv2B52XLOXj4b6qiFECIsrFmzhvz8fKxWK2VlZRQUFHS5LGCxWLqtIlitVoqLi1mzZg0FBQWu+xYXF7uqFyaTidzc3C4rCKWlpe2SApPJRF5eXqfn2mw2SktLu012jEajq4oRyZQcK9uz7Oxs7df/o59/CI6YYemfYOKDfOUvFl7fc5o1ebezNHsUAO/Wvsv3t36fxquNGOOMPD37ae4dc29Aj3MVQoSfq1evAhAfHx/iSMKHxWJhxowZPU5fzMzMpLq680A/m83GCy+84Op1cL7Bl5SUYDabyc/P5/z5864EpLCwkJSUFIxGo6sHo+NcCKeOz3XKysrqtloxZMgQzp8/79kv3kve/FtSSlVqrT2ab9DXtniGv+HT7EnEqZ0w8UGmZCTx+p7T7KltciUR8zPms+6T6/je1u+x9eRWvvPud3ij5g2+d8f3SB2QGuJfQAghQqeioqLH5Qb3QU8duScQAI2NjeTm5mI0GsnLy6O6urrdz65evdrjuLx5rpPJZOox3kggyxnB5uqLsDdXThmZBMCek03tnpY6IJVf5/yaf73zXxkYM5BNxzfxqZc/xavVrwZk/rkQQkQC96bKrlit1naVAHcd+yQsFku75YZgT5JMTk7GarUG9TX9TZKIYOvYXDnCnkTsP32B663tGymVUuTfms+Ln3yROSPm0HStiafee4ovb/oytRdrgx25EEKEXEVFBTNndj/lt6e/6t2vOZsju1ueCJZIrkKAJBHBlzgCBg6FqzY4X0NSQgy3pCTQ3NLG4bOXuvyR4YOG85vc3/Dvc/6dwbGD2XpyK4tfXswf9v6Bljb/nTIqhBDhzmKx9FiJ8PSv+7KyspuOni4sLCQzM9PrGD3VU9UkUkhPRLApZa9GHN4Ip6sgeSxTRiZxrOEKe07amDgisZsfUywev5h5GfNYvWM1b9S8wc8qf8brR1/n+3d8n9uH3h7kX0QIEWpjvvt6qEPwSM1PPuGX+ziTg57ONDIajTQ2dnWYc/vDuMxmc7v72Gw2Kioq2lUmZs6c2W4mhL81NjZKJUL4oMPQqe76IrqSOiCVZz76DM/mPMuIgSP4R+M/eGz9Y/zw/R9iuxq4f+xCCBFqzq2bN3vj7aq3wWw2k5mZidVqdW0Ldb9PcXFxp6WNsrKyXg2tupm+cJqnVCJCwb0vArckovbmSYTTvIx5vPjgixTtLuJP+/7EusPrMB83842sb7Bk/BIMSvJDIfo6f/2FHynKy8s96mHIzs7utOzhPAjL2QuxadMmCgsLXVMjuxpxbTabKSoqwmq1YrVaKSoq8tuAKLPZzMMPP+yXe4WSJBGhMMJRiThtb66c5EgiDpy5yPXWNmKiPEsAEmIS+OaMb/Jg5oP8ePuP2X5mO//2wb+x7tA6vjv7u0wdOjVQv4EQQgSd2Wxm5cqVN31eQUFBpxM8TSZTpxM9ezrh07n04ax+5Obm+nVpo6SkpFcHhIUL+XM1FAYPh0HD4GoTnD9K0oAYxjiaKw+dvej17UxGE8997Dme+egzpCWksbdhL4+tf4yn33uacx+eC8AvIIQQwWGxWFzLDxaLxaNKRFZWVq+3TprNZlcFwrns0NMhW95qbGzsE8sZkkSEgrO5Em70RWTY1+a8WdJof0vFfWPu49VPvcqyKcuIMcTwSvUr3P/i/fxu7+9obm32S+hCCBFMOTk5rF27FrPZzIoVKzxuRHRfqvBFWVmZa+mip3M6fFFcXOwauR3pJIkIlU7NlfZdGZ40V/YkISaBr2V9jZcffJl7Rt3D5euX+UXlL/jkS59kY81GGVQlhIgoK1euJDMzk5KSEq+mQubm5lJdXe3zEoSzpyI3N9e128P94C1f2Ww2bDZbyOdT+Iv0RIRKp+ZKRyWil0mE06jEUfxywS95/9T7PFP+DEdsR/jWO98iKy2LFTNXMCl1kl9eRwghAmnFihU+/+zq1aspLi7usmmyJ1ar1fUm7+yH6Lgl1FfOikpfIQdw3YTfD+ByungGfvYRiEuEwmNcaG7l9h9uJDbKwN5/u5fYaP8ViVraWvj74b/z3zv/m/PX7Ie9LBq7iK9nfZ0Rg0b47XWEEP4lB3AJfwnUAVyynBEqg9PtDZbXLkCjlcT4GEypA2lu9a25sifRhmiWfmQpry15jc9P+jwxhhjWH13PAy8+wM8rf86F5gt+fT0hhBD9gyQRoTT0I/bP52sAGJc2CIBjDVcC8nKJsYn8S/a/8OriV1k0dhHNbc38fu/vWfT3Rfxx3x+51notIK8rhBCib5IkIpTi7fMhuGbvgxgcHwPA5ebAnocxctBIVs9fzd8+8Teyh2XTdK2Jn1b8lAdefIBXql+hta01oK8vhBCib5AkIpScScRV+3LCwLgoAK5cC86hWpNSJ/G7e3/HsznPMn7IeE5fPs3T7z1N3qt5bD6xWXZyCCGE6JEkEaEU5zhs65o9iUiItW+WudwcvEqAUop5GfMoub+EH9/1Y4YPHM4R2xGefOtJPrvhs5SfKQ9aLEIIISKLJBGh5KpE2JczBjkqEZeDVIlwF2WI4oHMB3ht8WsUziwkOT6ZXfW7+MKbX2D5xuXsqd8T9JiEEEKEN0kiQslZibjavhJxJYiViI5io2J5bOJjbFiyga9O+yqDYgbxwekP+Mz6z/DkW09ysPFgyGITQggRXiSJCKX49ssZzp6ISyGoRHSUEJNAwdQCNizZwBcnf5EB0QPYfGIzea/m8a3N3+LI+SOhDlEIIUSISRIRSt1WIkKfRDgZ4418Y8Y3WL9kPY9NeIxYQywbj21kyStL+M4736HaVh3qEIUQQoSIJBGh5NriaU8iBsU5Giuvhd8Wy9QBqRTOKmT9kvV8+iOfJtoQzRs1b7D45cWseGeFJBNCCNEPSRIRSvEdKxGOLZ5hVInoaNjAYTx9x9OsX7Kehz/yMFGGKDbUbGDxy4v59jvf5tD5Q6EOUQghRJDIAVyh5FrOsO/OGOioRFwKw0pER+kD0/neHd/jiSlP8Ns9v+Xvh//OmzVv8mbNm+SOzmXZ7cuYmDIx1GEKIUSvFRcXu04DNZlM7Y4j73gap/O5zuckJye7vu4rJ3e6kyQilDpMrHQmEeFciejIPZn4/d7fU3qoFPNxM+bjZuaNnMfy25czLW1aqMMUQvRTVqsVk8nk888WFhaycuVKsrKy2j2emZnZbiCfN8/tjd78PoEgSUQoxQ22f752EbRmYKxzTkT4VyI6Sh+YzsrZK3liyhP8cd8feeHQC2w5uYUtJ7cwK30WX5zyRe4cfidKqVCHKoToJywWi89vujabjYULF1JWVtbp500mU7vjxb15bm9ZLBZsNlu7RCWUpCcilKJiICYBdBs0XyIhAisRHQ1NGMq3Z36bNx96k2VTljEoZhA7zuygoKyAT7/+aczHzLTptlCHKYSIcBaLhcLCQtasWcOaNWs6XbfZbBQVFZGXl+fT/fPz8ykoKOg2ASksLPTpuWBf8sjMzKS0tJTi4mIKCgraXRsyZIjrWsefzcvLo6ioyLW8EmqSRISa2zbPhBhnY2UrbW2RfW7FkPghfC3ra7yZ9yZfz/o6yfHJ7G/Yzzc3f5MHX3qQvx/+O82tzaEOUwgRoXJycli5ciXl5eWUlZV1ur5q1apOb8CestlsmM3mHisIzoTB+dyekpWOyUV2dja5ubnk5eWxfPlyjEYjxcXFrmvLly93XbNarZjN5nY/X1hYyKpVq3z63fxNkohQc9vmaTCoGzs0rkfekkZXEmMTeWLKE7z50Js8NfspRgwcQc2FGn7w/g/4+LqP8/u9v+dS86VQhymEiCDFxcWuhsXnnnuuyyTCYrH43DtQUVGB0Whs10Dpzr0KUFFRAXROFEpLSykoKCA/P5/S0tJ218xmMzNmzGh3v+TkZNe1hQsXuh43m81dLpFYLBaffjd/kyQi1Dpt83QsaYTB1Ep/io+O55HbHuG1Ja+xat4qxg8ZT92Hdfy88ucsLF3Izyt/ztnLZ0MdphAiAlRWVrp6Arp6oy8tLSU/P9/n+98s+XCvDHT33Ly8PFdsHasU5eXlZGdnA/ZEwWq1up5TXm4/9LC4uJhly5axadOmLl8jPz/fVb0IJUkiQq3TNs/wGX0dCDGGGO433c+6B9bxbM6zZA/L5tL1S/x+7++57+/38fR7T8usCSFEj9z/Wu/K2rVrWbp0qc/3N5lM5ObmdqoggD1BcU8KTCYTeXl5nZ5rs9koLS3tMk5nw6fZbMZsNrerpNhsNnJzc1m+fDnJycndNlAuXbqUkpISX39Fv5HdGaEW3/Vx4KE8hCsYnEeQz8uYx576Pfxh3x8wHzfzSvUrvFL9CnNGzOFzEz/HnBFzZEeHEAKANWvWUF5ejtVqpaysjMrKSgoKCjq90Voslm6XIpxv3jabjerqaoqKilizZg1Go5GysjLXG3NJSQmFhYVYrVaMRqNr+aSr3oeunguwevXqTrE5l0K6uo/7fAnAFWdXv4vRaHQtpYSSJBGh1qESEcrjwENlytAp/Ozun3Hi4gn+vP/PvHTkJd4/9T7vn3qfccZxfHbiZ/mE6RPERcWFOlQhRAitWLECi8VCaWmpT3+Fd2yYXLhwIQsXLqSkpASz2UxpaWm7N+3Vq1d7fG9Pn1tRUdHt0KmKiop2SxfOpKS7RCIcyHJGqPXTSkRXRg0exVOzn6Isr4yvZ32dtAFpHLEd4Qfv/4CPlX6M/6n6H859eC7UYQohQqjjG21HPb3hvvDCC+12XDQ2NpKVleWqMFRXVwf0zdpqtbJ69WoaGxs7bdF0XnN/PC8vD7PZTGNjY5f3M5lMId/qKZWIUItz7M64Gn7HgYdKUlwST0x5gscnPs4bNW/w5/1/5kDjAX6z6zf8ds9vWTR2EY9OeFTGagvRD7k3VXbFarW6lhM66tgnYbFY2lUQAj0J0mQydbmTpLtrN6u2JCcnY7VaQzp4qs8lEUopZ5rp3D9TqLUOj6kcXelwkufAMDwOPFRiomJ4IPMB7jfdT+XZSp4/8DxvHX/L1TeRlZbFZyZ8hpzROUQb+tw/ZSFEFyoqKnj44Ye7vd5TJaFjvwFE/nkWoV7m6FP/5VVKLddaF7t/D1QCmaGL6iY6bPEcGMbHgYeKUors9Gyy07M5cfEE//eP/+PFwy9iqbNgqbMwLGEYD3/kYZaMX0LKgJRQhyuECKCO1YOOnH+d30xZWVnYjI72VU9Vl2DpM0mEUqpTOqa1LlZKrVZK5WqtzV39XMh1aKyMhOPAQ2nU4FGsmLmCr077Ki9Xv8xfD/yVmgs1/HLnL/n1rl9z35j7+MyEzzA5dXKoQxUi8H6YFOoIPPPDJr/cxpkcOGcsdMVoNHbbQ+B+jobZbG53H5vN1qnpsbCwkNLSUqqrq/0Rvt81NjaGvBLRlxorTUBRF8mE1XHNY0qpFKXUrUqpW1taAvxm3qGxMpKOAw+lhJgEHrntEV7+1Mv8Jvc33J1xNy1tLbxqfZVHXn+ER157hJeOvMTVlquhDlUI4SfOrZs3e+PsqrfBbDaTmZmJ1Wp1TXt0v09xcXGnpY2ZM2eG9XJHOJzm2WcqEVpri1JqRhf9DybsiYQ3ngR+AFBXV+eP8LoX12E5QyoRXjEoA3NHzmXuyLnUXqzlhYMvsO7wOvY27GXv1r38tOKnLB63mKW3LmVU4qhQhyuEf/npL/xIUV5e7tGbenZ2NhaLpd1yhfM0TWcvxKZNmygsLHRNfezqnIyysrJeTb4MJLPZ3GNvSLAof51xHo6UUnnASq31jJs+uf3PpQApAFOnTj1YVVUViPDsztfAf02FpFHwzb28UHGCFaW7eSgrg58tnRq41+3DrrZcZcPRDfzt4N/Y37Df9ficEXNY+pGlfDTjo9KIKSLC1av2Slp8fHyIIwkPM2bMYOXKlTc9mdNisVBUVERRUVGvXi8zM5Pq6mqsVitWq5WioqKwmBIJUFBQQGFhocfVCG/+LSmlKrXW3a8Zuemz/yV1LGusBHK8/VmtdQPQAD2vvflFfIctnrHOxkqpRPgqPjqexeMXs3j8Yvae28vf/vE33qh5wzXAaljCMJaMX8KS8UtIH5ge6nCFED1wLj1kZWVhsVg8qkRkZWV51FzZE2f/hHMJJTc3N+QzGdw1NjaGxXJGWPZEKKWWK6XKPPzobnFsNZAf1ts74cZyxrUL0NbmmhNxWZYz/GJy6mT+467/YFP+Jr6T/R3GJI7h7JWz/HrXr7l33b08uelJ3q19l9Y26UERIhzl5OSwdu1azGYzK1as8LiR0H2pwhdms9lVgXC+Wd+sAhIsxcXFFBQUhDoMIEwrEY5tmj7/v6+UWgGs1lr3LhUNBkMUxA6C5kvQfMnVWNkfJ1YGUlJcEp+b9Dk+O/Gz7Dizg5JDJWw6vonNtZvZXLuZ9IHpLBm3hMXjF0t1QogwsnLlSoxGIyUlJV4tT+Tm5lJYWOjzyGjnORpWq5WFCxd2OyQq2Gw2m+uQrnDQ53oiHLMhzO4JRG+2eGZnZ+uAH3Lyswlw8RR8Yy/7riTyiV++x23pg3njG/MD+7r9XMOHDbx05CVKD5VSe6kWsDdq3jXyLh4a/xDzMuYRY4gJcZSiP5OeiN4rLi7usmnyZpz9EDabjZycHCorKzGbzSF/8+54iqinpCfCA0qpXKDCmUA4ljoC3NTgB/GJ9iTi2gUGxtoHh8hyRuClDEjhi1O+yOcnf54dZ3ZQeqiUTcc38W7tu7xb+y6pA1L5ZOYnWTJ+Cbck3hLqcIUQPvAlgbBara5kwdkP0XGuRKiEy5KKU5+pRCilTEB3E0GG+NobEZRKxG8XQu0O+Pwb1CdnMfM/zaQMjKXy+53PoReB1Xi1kVerX2Xd4XUcbTrqenzGsBksGb+EhbcsZED0gBBGKPoTqUQIf5FKxE04qg8q1HH4xG3glDRWhlZyfDKPT3qcz038HFX1Vaw7tI6NxzZSebaSyrOV/Hj7j/n42I+zeNxipqROQanI/CcnhBD+0GeSiIjmts0zPjoKpeDq9TZa2zRRBnmTCgWlFNPTpjM9bTrfnfVd3qh5gxcPv8juc7spPVRK6aFSTEkmPjXuUzyQ+QCpA1JDHbIQQgRdWG7x7Hdc2zybMBgUCTFSjQgng2IHkXdrHn/5xF948ZMv8rmJnyM5Phlrk5WfV/6c3JJcvrLpK2ys2Uhza3OowxVCiKCRSkQ4iG9/CNfAuGguN7dy5VorifGyOyCcjBsyju/M/A7fmPENttRu4aUjL7GldourGTMxNpFFYxfx4LgHmZQySZY7hBB9miQR4aDj+Rlx0XDxmlQiwliMIYYFoxewYPQCGj5sYP3R9bx85GUOnj/I3w7+jb8d/BumJBMPZD7A/ab7ZfaEEKJPkuWMcODsiXCc5Ok8DlxGX0eGlAEpfHbiZyn9ZCklD5Tw2ITHXMsd/2X5Lz5W+jGeePMJXjryEpevXw51uEII4TdSiQgHXVUigMtyHHjEuS35Nm6bdRv/kv0vfHDqA16pfoW3j7/N9jPb2X5mO/+57T+5Z/Q9PGB6gDtH3CkHgQkhIpr8FywcuG3xBDkOvC+IMcQwP2M+8zPmc6H5AhtrNvJq9atY6ixsOLqBDUc3kByfzMfHfpz7TfdL/4QQIiJJEhEOOpzkmeCsRMj5GX1CYmwiebfmkXdrHrUXa3nd+jqvWV+j5kINfznwF/5y4C+MSRzDorGLWGRaJNMxhRARQ5KIcBDXdSVCeiL6nozBGRRMLWD57cvZ37Cf16yvsf7oemou1PDsrmd5dtezTE6ZzCdMn+C+sffJ/AkhRFiTJCIcdLHFEySJ6MuUUkxKncSk1El8K/tbbD+9nfVH12M+ZmZvw172NuzlmYpnmJU+i0VjF5FzSw6JsYmhDluIiON+lLeviouL250Gmpyc7Po6NzfXL68RqSSJCAcdGytj5Tjw/iTaEM3ckXOZO3Iu37vje7xz4h3WH13PlpNb2HZ6G9tOb+NH237EXSPvYtHYRczPmE9CTEKowxYi7Fksll69wVutVgoLC1m5ciVZWVntHs/MzMR59pTFYsFms7V7Tn8hSUQ4iBts/9x8EdpaSYiT5Yz+akD0AO4bex/3jb2PpmtNbDq+ifXW9ew4s4O3T7zN2yfeZkD0AO7OuJt7x97LXSPvIi4qLtRhCxF0FouFtWvXkpKSAsCKFSvaXbfZbBQVFVFUVOTT/W02GwsXLqSsrKxTEmIymdqdDpqXl0dBQQGrV692VSj6C0kiwoEhCmIH25OIaxcZ5GqslCSiP0uKS2LJ+CUsGb+E+iv1bDy2kQ1HN7CrfhcbajawoWYDg2IGsWD0Au4dcy93Dr+TmCiZcCr6h5ycHI4ePcqyZcuw2WydkohVq1ZRWFjo8/3z8/MpKCjotorR8d6FhYWsWrWK1atX+/yakUiSiHARn+hIIi6Q4FzOkDkRwmFowlAenfAoj054lJOXTvJmzZu8cfQNDjQe4JXqV3il+hUSYxPJGZ3DvWPuZdbwWcQYJKEQfVNxcbGrL+G5557r8q9/i8Xi8zKGzWbDbDb3WMXoqjphsVh8er1IJklEuIhPggsn4eoFBsbay3NSiRBdGTloJF+Y/AW+MPkL1DTV2BOKmjc4YjvCi0de5MUjL2KMM5IzOoeP3fIxZg6fKQmF6FMqKytd/QddJRClpaXk5+f7fP+Kigqgc6JQWlpKWVkZjY2NPPzww+Tl5bW7np+fT3Fxcbuljr5Oxl6HC7dtngkysVJ4aEzSGAqmFvDigy/y0oMv8c9T/xlTkgnbNRvrDq+jwFzAghcW8MP3f8j7J9/netv1UIcsRK+ZzWYWLlzY7fW1a9eydOlSn+/fXQUjLy/PlbR0TCAAli5dSklJic+vG4mkEhEu3LZ5DoqXo8CF9zKNmXx52pf50tQvccR2hI3HNvJmzZscbTrKusPrWHd4HYmxiSwYvYCP3fIx7hh+h/RQiIiyZs0aysvLsVqtlJWVUVlZSUFBQaddERaLpdsGR6vVitlsxmazUV1dTVFREWvWrMFoNFJWVkZJSQkmk4m8vDxKS0vbJQs2m43S0tJuey2MRqOtzKtVAAAgAElEQVSritFfKOcWFdG17OxsHZR/FKVfhL2lsLiYA2kf5+P/tYWPDBvMm9+cH/jXFn2W1tqVUGys2Yi1yeq6NjhmMHePupvcW3KZM2IO8dHxIYxUdOXq1asAxMfL/zdOFouFGTNm0NN7V2ZmJtXV1Z0et9lsvPDCC67lBmc1o6SkBLPZTH5+PufPn3clIIWFhaSkpGA0GklOTnbdJysrq9tqxZAhQzh//rzPv1+gePNvSSlVqbXO9uS+UokIF27nZzjnREglQvSWUorxQ8Yzfsh4vjLtK1Tbql0JxRHbEV61vsqr1ldJiE5gXsY8cm/JZf5ImUMhwldFRUWPDZPuQ6E6ck8gABobG8nNzcVoNJKXl0d1dXW7n/Vlp4XJZOoxhr5GkohwEXdjOUPmRIhAyTRm8mXjl/ny1C9ztOko5mNmyo6VcaDxAG/WvMmbNW8Sa4hlzsg5LLxlIR/N+ChJcUmhDlsIF/emyq5YrdZ2VQN3HfskLBZLu0TBH1Mnk5OTsVqt/WbwlCQR4cJ5CNe1C25zIqSxUgTO2KSxLLt9GctuX0btxVo2Hd+E+ZiZqvoqNp/YzOYTm4lW0WSnZ5M7OpcFoxcwNGFoqMMW/VxFRQUPP/xwt9d7qgC4XzObzYB9bLW/9ZcqBMjujPARf2P0dVy0AYOC5pY2rre2hTYu0S9kDM7g8UmP8+dFf2ZT/iaemv0Us4fPRqPZdnob/7H9P1hQsoBH1z/K7/b+jmMXjoU6ZNFPWSyWHv/Kd1YCbqasrOym1YLCwkIyMzO9iq+nSkhfJJWIcOEsGV9tQinFwNhoLl5r4cq1VpISJNcTwZOWkMYjtz3CI7c9gu2qjXdq38F83MwHpz5gd/1udtfv5heVv2CccRz3jLqHnNE5TEyZiFIq1KH3O1P+OCXUIXhkz+N7/HIfZ3KQnd19z5/RaKSxsbHbn3cuWZjN5nb3sdlsVFRUtKtMzJw5E5vN5lWMjY2N/aoSIUlEuIjvcBx4nD2JuNzcQlKCbMMToWGMN/LguAd5cNyDXLl+ha2ntvLW8bd4p/YdjtiOcMR2hOf2PEdaQhr3jLqHBaMWMDN9pmwdFQHh3Lp5szfprnobnLMlqqurXYmB+32Ki4s7jc4uKyvzemhVfzvNU5KIcNHhJE9nc+UV2aEhwkRCTAILb1nIwlsWcr3tOhVnKnjr+Fu8deIt6q7UsfbgWtYeXMugmEHMGzmPe0bfw10j72Jw7OBQh95n+esv/EhRXl7uUQ9DdnZ2p2UP56FZzl6ITZs2UVhYSHFxMUCXUyado6+tVitWq5WioqIeh0mZzeYe+zX6IkkiwkXHSkSsTK0U4SvGEMOdI+7kzhF38tTsp9jXsI+3jr/F2yfe5ojtiOuAMGdj5t2j7uaeUfcwYtCIUIcuIpjZbGblypU3fV5BQUGnEzxNJlOnszB6OhvDufThrH7k5ubedGmjpKSkV4d+RSJJIsJFx0pErGzzFJFBKcXk1MlMTp3M17K+xokLJ3jrxFtsPrEZS52Fbae3se30Nn6y4yd8ZMhHXAnFhJQJGJT0+4ieOQ+1ysrKwmKxeFSJyMrK8qi5sidms9lVgXBOrexq1LW7xsZGWc4QIeK2xROQbZ4iYo1KHMXjkx7n8UmPY7tqY8vJLbx94m22ntzKwfMHOXj+IEW7ixg6YCjzM+Zz96i7mT18NgOiB4Q6dBGGcnJyWL58OY2NjaxYscLjpkXnUoWvh2E5R2BbrVYWLlxIWVlZj88vLi6moKDAp9eKZJJEhIvYQYCC5kvQ2uI6hEt6IkQkM8YbeSDzAR7IfIDm1mbKz5Tz9om32XxiM2evnHWd6REXFccdw+9gfsZ8PprxUYYNHBbq0EWYWLlyJUajkZKSkh6XHzrKzc2lsLDQ5+mRzp4Kk8nEqlWrAHt1oqtKiM1mw2azBWTmRLiTJCJcGAz2JY1rTY7R187lDKlEiL4hNiqWuSPnMnfkXJ6e/TQHzx9k84nNvHPiHfY27OWd2nd4p/YdfsSPmJA8wZVQTEqdJMse/VjHHRPeWL16tU/VCKvV6koInP0QHbeEujObzb2KM5LJAVw3EbQDuAB+MRmaTsDXd/GjrVf43/eO8vSiCSyb37/W2ET/U3+lni0nt7D5xGa2nd7Ghy0fuq4lxyczP2M+8zPmc+fwOxkUOyiEkQaXHMAl/EUO4OoP3JorB8bGAXIIl+gfhiYMZcn4JSwZv4RrrdfYcXoH79S+w7u173L68mleOvISLx15iWhDNDPSZjAvYx7zM+YzJnGMDLkSIoS8TiKUUmOAAqBaa/1bx2M5QLnW+oJfo+tv3LZ5JsQNB+CKNFaKfiYuKo55GfOYlzHPdZT5O7XvsKV2C1X1VWw/s53tZ7bz04qfkjEog/kZ85mXMY/sYdlynLkQQeZVEqGUmg6UAKWAq1NFa71JKZWjlKrWWtf4N8R+JN7xP+nFMwyMzQDgkmzxFP2Y+1HmT0x5gqZrTWw9uZV3T77L1pNbqb1Uy1//8Vf++o+/Eh8Vz6zhs5g30p6AjBw0MtThC9HneVuJWKq1HgeglHrI/YIjkVgC1Pgptv5n9Gw4tAEObmCg6Q4ArkgSIYRLUlwSi0yLWGRaRGtbK3vO7WHLyS1sqd3CgcYDvFv7Lu/Wvgvb7aeUzhs5j7tG3sWMYTOIjYoNdfhC9DneJhHuHYZddWT2n6PLAmHip8D8Qzi4gUHjvw/InAghuhNliGJa2jSmpU3jyelPUn+lnvdOvseWk1v44NQHHG06ytGmo/xp/58YED2A2emzmTtyLneNvIuMwRmhDl+IPsHbJMI9ceiqm2lcL2IRyWNhxHQ4tZOMc1uAZJkTIYSHhiYMZfH4xSwev5jrbdepqqvivZPv8d7J9zh0/hCbazezuXYzAGMSx3DXyLuYO3Ku9FII0QveJhFKKfWEo6FSuz04DVjt+AgbSqkSrbV3R7CF2qQlcGon6Sc2AI9ySeZECOG1GEMMM9NnMjN9Jt+c8U3OXj7L1lNbee/ke2w7tY2aCzXUXKjh+QPPExcVR/awbPsMixFzGZs0VnZ8iD5Hax2Qf9dez4lQSn0HWIk9iWjkxhJGoXO3RjhQSmUBlVrrXv2vFtQ5EQC24/D/ptAWPYBJl/6HjLRUyv7lo8F7fSH6uOtt19lTv4f3Tr7H1lNb2d+wv9314QOHM2fEHO4aeRezh88O6Smk165do7W1lYSEhJDFIPqGK1euEBUVRVxc3E2f682cCJ+GTSmlkoBswARYgQqtdZPXNwogpVQuUBZxSQTAb3OhtpyvNH+NqsR72PrdBcF9fSH6kYYPG3j/1PtsPbWVD059QOPVRte1KBXFlNQpzBk5h7kj5jIpZRJRhqigxaa1pqmpyaexzUK4s9lsJCUleVSNCHgS0cMLPxEO1QilVJ7WulQppX1JIpRSKUAKwNSpUw9WVVX5PcYeffA/8OZTrG+dxVMx36HqXz8W3NcXop9q020caDzA+yff572T77G7fjct+kZfUmJsIncMv4M5I+YwZ8Qchg8aHvCYzp8/z+DBg4mOltmAwjctLS1cvHiRIUOGePT8gCQRSqlE7JWH7iQDRVrr8R7dMEAcyxg2rbW1F0nED4EfAAwfPpxTp075OcqbaDoJv5jIVR3DHS1FVP3nQzf/GSGE311qvsSOMzvslQrHXAp3Y5PGcufwO5kzYg4z02eSEOP/ZQetNTabDaUUsbGxKKWkZ0PclNYarTXNzc1orTEajR7/uwlUEjEdqATOOz7cmYBq7G/eMz26YYA4qxCOryOzEgHo/70XdWIbX2v+Kj/99x8RGy0HEAkRaicunHAtfew4s4PL1y+7rkUbopk6dCp3Dr+TO0fc6felD/c3BCE84Z54evlzAUkikoDVWusvdXM9B2jUWu/0OFI/c08gHN/7lES4C0lPBMD2Itiwgo2tM5i18g2MCTIoR4hw4mzQ/OD0B7x/6n32nttLm25zXR8cO5jZ6bO5c8Sd3DH8DkYNHiUVBBERAtYToZRK6qmBUim1RGv9d49v2P19lgOebs3M11rblFImwKi1trjdJ3KTiAunafv5BK7rKM59eT8j04cFPwYhhMearjVRfqacD059wAenP+DExRPtro8cNJI7ht/BHSPu4I70OzDGS7OkCE+hbKz0SxLh42svx+08D4fVQCH2ZZZiX+4bsiQC2PWjuUxt3cvpnF8xfN7nQhKDEMI3Jy6eYNvpbXxw6gO2n97OheYb5xMqFLcl3+ZKKKYPm86A6AEhjFaIG4KeRLg1XT6stV7Z6xv6SURXIoA/rPkm/3Tld5y79dOkfqYoJDEIIXqvta2VA40H2HZ6G9tOb2Pn2Z00tzW7rscYYpiWNo07ht/B7OGzmZQyiWiD7MYQoeFNEuHxv1JHY2VP76YWYJmn9xM3dzhhGlyBgac/CHUoQoheiDJEMTl1MpNTJ/PElCe42nIVS52F7ae3s+30Ng40HKD8TDnlZ8r51c5fMShmENnp2fakIn02mcZM6acQYcmbVNcKrNNaLw1UMP7iGDRV4Pi6BPvWU3Noo/Je/aDbuFg/gMEXj9m3fSbJ0cZC9AXx0fGuWRMAtqs2dpzZwfbT29l+ZjvHLhxj84nNbD6xGYCU+BRmDZ/F7PTZzB4+Ww4QE2HD28bKsVrrowGMJ+yEcjnj63/byYP7vsGCqCpY8hzcHvb5mxDCD05fOs2209vYfmY7O07voP7D+nbXRw4ayaz0WcwePptZ6bMYmjA0RJGKviggyxkAN0sglFLTtNbBH6rQR8VFG9jWNsGeRNRskSRCiH5i+KDhrhNJtdYcbTrKttPb2HFmBzvO7ODkpZO8eORFXjzyImAfeuVMKrKHZTMk3rPJhEL0lr87d1YCD/v5nv1WdJSBbW0T7d/UvBfaYIQQIaGUwmQ0YTKa+MyEz9Da1so/zv+D7ae3s+PMDixnLRxtOsrRpqOsPbgWgFuH3Mqs9FnMSp/FjPQZJMYmhvi3EH1Vl0mEB02UXf4YbseDi96LjTKwT4+hOWogsY1W6YsQQhBliGJSyiQmpUziC5O/wPW26+w7t49tp7dRfqacqroqDp0/xKHzh3j+wPMYlIHbkm9jVvosZqbPZMawGQyMGRjqX0P0Ed1VIqzAJq21Vyc/KaVe6H1IwinaoGglijPG6YxueA+ObZUlDSFEO87todPSpvGlqV/iWus1dtXtci197Dm3h/0N+9nfsJ8/7PsDUSqKiSkTmZk+k5npM5meNl2SCuGzLpMIrXWTUqrA05sopRK11hewD3YSfhLjOC+jNmmGPYmQvgghxE3ERcUxa/gsZg2fBcCV61eoqq+i/Ew5O87sYN+5few5t4c95/bwu72/I0rZKxvZ6dmSVAivddsT4ekuDMeZGtlKKSP25Yx+tXsjkGIM9n3hxwZPZw5IX4QQwmsJMQnttpNeuX4FS53FNZdif8N+dp/bze5zu9slFTPSZzBzmD2pGBQ7KMS/hQhXPjVWOiZUJrs9VA0MAb4LhGTsdV8UE2WvRJwcMB5iB4P0RQgheikhJoG7Rt7FXSPvAuDy9cvsrNvZZVLx+72/x6AMTEiewMz0mWQPy2b6sOnSqClcvEoiHFWHEiAXe9XBeRiXEfsx4bKc4UfRjiTielsU3HInHN4ofRFCCL8aGDOwU1JRVWdf/ig/W87+c/vZ17CPfQ37+MO+P7jO/ZgxbAbZ6dnMSJshh4n1Y95WIr4LFGqtdyqlHtJar3NecB4F7tfo+rmYKPtyRnNrG4y5y55ESF+EECKABsYMZO7IucwdORe40VNRcaaCirMV7Dm3hwONBzjQeIDnDzwPwDjjuHZJhQy/6j+8TSIqtNY7HV+3m2aitd6klFrgn7AE3FjOaGnV9iQCpC9CCBFUHXsqPmz5kN31u6k8W0nF2Qp21+/miO0IR2xHXHMqbkm8hay0LGYMm8GMYTMYOWiknP3RR3mbRLjPgajs4uhvqWn5kTOJuN7aBulTpS9CCBFyA6IHMHu4/QwPgObWZvae2+tKKqrqqjh24RjHLhxzTdRMS0izVyqGZZOVloXJaMKgDKH8NYSfeJtEKKXUWOAn2Jc2nlJKDcHeJ5GMfVqlNFb6iXM543qrhqjoG30Rb3wXPr4aEkeEOEIhRH8XGxVL1rAssoZlsYxltLS1cLDxIBVn7csfO+t2Uneljg1HN7Dh6AYAjHFGpqdNZ8awGWSlZXFbym3EGGJC/JsIX3h7dsY6pdRDwHmgAVgKbASew16lWOj3CPuxdpUIgFkFUP02HHgFjphh7tdhzpMQK3u6hRDhIdoQzaTUSUxKncTjkx6nTbdRbavGctZC5dlKKs9WUvdhHW+feJu3T7wN2Ksbtw+9naw0ezJye+rtJMQkhPg3EZ7w9hTPLg/YUkolaa2buvqZSBfKUzxf332ar/zVwqIp6Tz76Az7g41WKPtXOPCq/fvBI+Dz6yF5bEhiFEIIb2itqb1U60oqdtbtpOZCTbvnRKkoJiRPsFc40rKYljaNlAEpoQm4HwrYKZ7Yly3Gd3ywryYQoRbt3J3R4pboJZvg4eehZius/w7U7QPLHyH3hyGJUQghvKGUYtTgUYwaPIoHxz0IwLkPz7GzbieWsxYsdRb+0fgP9jbsZW/DXv60/08AjEkcw/S06UxPm07WsCxGDx4tzZphwNskIkUptQr7cKkXHKOuRYDEdlzOcDdmLsz5Krz0ZWiqDXJkQgjhP6kDUll4y0IW3mJfEb98/TK76ndhOWuhqq6K3ed2U3OhhpoLNa5mzeT4ZFeVYnradCYkTyAmSvoqgs3bJGKGcxy2UirH0WRp1Vq/5f/QhLMS0dLWRRIBkOjYodF0MkgRCSFE4A2MGdhuW+n1tuscbDxI5dlKquqqsNRZaLzaiPm4GfNxMwDxUfFMTp3M9LTpTEubxtShU0mKSwrlr9EveNtYedTt601g74dQSi3BvjvDrLWu8WuE/ZirsbKlm76VpAz7Z6lECCH6sBhDDJNTJzM5dTKPT3ocrTXHLx63Vyrqq9hZt5OjTUddO0KcxhnHuSoV04dOJ2NwhiyB+JlPZ2d0kAN8CcgGioCVfrinwG2L580qERdPQVsrGKKCFJkQQoSOUopbEm/hlsRbWDx+MQDnr56nqq6KnfU7qaqrYu+5va4hWKWHSgFIiU+xH5s+1H50+sSUicRGxYbyV4l43p6dsURr/Xel1DTsicNSoBwo0lp/LBAB9medtnh2ekI8JKTClXNw6azMjRBC9FtD4odwz+h7uGf0PQBca73G/ob99sSizp5YNFxtYNPxTWw6vgmwVzgmpUxyJRZT06aSOiA1lL9GxPG2ElGqlKrGvnSxChgrOzMCJ9rgNva6O0kZ9iSi6aQkEUII4RAXFefazfF5Pu9aAnEmFLvqd3HEdoSq+iqq6m9MLsgYlMHUtKmuasU44ziiDf4o2vdN3v4vYwMKpJEyOGKj3Q7g6k5SBpyugqYTMGpmkCITQojI4r4E8qlxnwKg6VoTu+t3s6t+F1X1Veyu303tpVpqL9XyuvV1wD4Ia0rqFKYOncq0tGncnnq7nFrqxtskYpUkEMHjcSUC4ILs0BBCCG8kxSUxL2Me8zLmAdDS1sLh84fZVb/LnljUVVF7qZYdZ3aw48wO18+NSRzD7UNvZ+rQqUwdOpVxxnFE9dOeNG93ZzwTqEBEZzHRN+mJANnmKYQQfhJtiGZCygQmpEzg07d9GrAPwtpdv5uq+ip21e1iX8M+18yKV6pfASAhOoEpqVNcicWUoVNIjk8O5a8SNLLQE8ZiDG4HcHXHtc3zRBAiEkKI/iV1QCoLRi9gwegFAFxvvc7B8wdd1Yrd9bs5eekk289sZ/uZ7a6fGz14NLcPvd31ceuQW/vkIWOSRISxm+7OAFnOEEKIIIqJujGz4tEJjwL2aoUzodhVv4t95/Zx/OJxjl88zmvW1wB7o+eklEmupGJK6hTSB6aH8lfxC0kiwphrYqVHyxkycEoIIUIhdUAqOaNzyBmdA9gnbB4+f9iVVOw5t4djF45hqbOfDeKUlpBmX/5IncKU1ClMTJkYcaeXShIRxm5UInpYzhicDioKLtfD9av22RFCCCFCJsYQw8SUiUxMmejqrTh/9Tx7zu1hd/1u9pzbw576PdRdqaPsWBllx8oA++ml44zjXJWK24feztiksRiUIZS/To8kiQhjriSirQ2tddfjWg1R9vkQTSfsSxopmUGOUgghxM0MiR/C/Iz5zM+YD0CbbqPmQo09qajfw+5zuzl8/jAHzx/k4PmDlBwqAezniExOmcyUoVNciUU4DcSSJCKMRRkUBgVtGlrbtGt5o5OkDEkihBAighiUAVOSCVOSyTW34sr1K+xv2G+vVDiqFmevnO3UtJk+MN21BDI5dTKTUiaFbBnEr0mEUmqBzJHwr5goA9da2mhp00R3tw1ZtnkKIUTES4hJIDs9m+z0bNdjdVfq2HNuD3vP7WVP/R72NuzlzOUznLl8xrUMYlAGyvLKSEtIC3rMPiURjrMzOkoGVgMyNtGPnElEc2sb8THdZBFJ0lwphBB9UVpCWrumzTbdxtGmo67EYnf9bhquNjB0wNCQxOftAVwPASVANdBVbX2sP4ISN8S4dmj0NCtilP3zBUkihBCiLzMoA5nGTDKNma5lkJa2lpAdce5tJWKm1rrbNlGl1G96GY/oINqTWRGyzVMIIfqtUB4Q5u2+kfKbXC/0NRDRtVhHEtHc4sHAKemJEEIIEUT+3nwqyxl+5ho41ebJ6GupRAghhAgeb5OIMqXUAqXUNKVUovsFx/cr/ReaAA9HXw8YAjEJ0HwRrjYFKTIhhBD9nbcLKTb3b5RS7n8eK6CHP5eDRym1AnusjQBa69LQRuS7aNchXD0kEUrZ+yIaDtuXNOKTghSdEEKI/szbJMIC5Gitu/xzVyn1Qu9D6h2lVAlQqLW2Or7XSqkhWmvbTX40LMVGezD6GuzbPBsO25c0hk0MQmRCCCH6O2+XM5Z1l0A4hLSxUim1HCh3JhAOmZGaQMCNSkSPh3CB22me0hchhBAiOLyqRGitd4Kr/yEX+2CpasCsta7RWh/1f4heWQ3McH+gQ0LhEaVUCpACMHXqVP9E5iNnT0TzzZKIRGmuFEIIEVxe785QSi0DaoCnsL9hfwmoVkp9y7+heR2XETA6vs5TSuUqpVY4HvfWk8BB4GBdXZ0/w/SaM4nocdgUyDZPIYQQQefLxEqj1jq5i2s/UUot0Vr/3W/ReceEvZnS6GykVEpVAJvoUJ3wwK+AvwKkpaUd9GeQ3nJOrOyxsRJk9LUQQoig87YSkay1fqarC1rr7wKhPEIyGXslwrV84eyFUErlenMjrXWD1vqQ1vpQdHRoDzq9MbHyJpWIROmJEEIIEVzevkM23OT6eV8DcedokMz38On5jmTBCjcSBzeNQBZg9kdswRbryZwIuFGJuHAK2trA4O85YkIIIUR73iYRnZYxOhjiayDutNbFQLGXP2Pt4QCSyN2d4ZpYeZMkInagfejUh+fhcj0MHhaE6IQQQvRn3v65elQptUopNdj9QaVUolLq10Cl/0LziUUpZerwmAmoCEUw/uCaWNniwRwv2eYphBAiiLxKIrTWm7AnCseUUoedH9iXMSq11m8FIkgvFOI2q0IplQVYtdaW0IXUO64k4maVCJBtnkIIIYLK665Bx86HUqVUDva/8q1AxU2GUAWF1tqslDI6xl4DpGitF4Y0qF5y7c7o6RRPJ9nmKYQQIoh83nrgqEpscn9MKTVGa13T26B6I5LPyeiKa05ET6d4OiWOsH++IEmEEEKIwPN3C/9qP9+v33M2Vt50YiXAAMdcrWsXAxiREEIIYddtEqGUOqKUWuX2fY5SqrWHjzYgLyhR9yOxnk6sBIh19LtKEiGEECIIeqpEFALup3I2Auu01lHdfBiAdQGNth+KNng4JwIgzpFENF8KYERCCCGEXbc9EVrrdR2+3+k4N6MnIT3Fsy+KifZiOSNOKhFCCCGCx9stnp12YDhmRCQ6rof6FM8+J8bgxXJG3CD7Z0kihBBCBIFXSYRS6okuHs4EFiqlnlBKLfBPWMLJ4wO4QCoRQgghgsrbLZ7jOj6gtd4J7ARQSi3xR1DiBo8P4AKIS7R/liRCCCFEEHi7xbPbdzKl1Bggogc7hSOPD+ACiJXlDCGEEMHTYyVCKTUdyAUeBqY7HlvR1VOBajw/eVN4yHUAlydJRHQcGGKg7Tq0XLN/L4QQQgRIj0mE21LFM0qp1UASXezACIeR131VjDfLGUrZ+yI+bLRXIySJEEIIEUDeLGf8GPshW00dPwIVnPCysRLcdmhcCFBEQgghhJ3HSYQjYXhOKTW2YwOl47Ex/g5OuFciPE0inM2VMnBKCCFEYHm7xXMakAX81vE14JoPMcT9MeEf0d4cwAWyzVMIIUTQeLs7I1trvU5rnay1rnK/4OifMPkvNAE3ljOaPTkKHGSHhhBCiKDx9ymeyX6+X78X62slQs7PEEIIEWDeJhGdhk15eV14KdrrngjncoY0VgohhAgsb5OItUqpN5VSU90fVEotUEqVAxv9F5oA990Z0hMhhBAivHg19tpxkuczwDql1Fi3SzZgudb6Lb9GJ3zYneFMImQ5QwghRGB5e3YGWmszME4pZQLGAlY5vTNwnEmERxMrQSoRQgghgsbrJMJJa20FrO6PKaUWSDXCv6INXi5nyO4MIYQQQeJTEtHNPIhkYDUws1cRiXZio31czmiWJEIIIURgeZVEKKUeAkqwH7alunjK2C4eE71woxLh7cRKSSKEEEIElreViJla6253dCilftPLeEQHMdFeHMAFbmdnSBIhhBAisLzd4ll+k+udTvgUvRNjkN0ZQgghwpO/J1bKcoafeX+Kp+zOEEIIERzeJhFljsFS05RSie4XHN+v9F9oAiDK0RPRpqHVk9HXsmcdWVYAABiCSURBVDtDCCFEkHjbE2Fz/0Yp5f6upgAPF+6Fp5RSxEYZaG5t43prG1GGqJ5/wH13htaguup/FUIIIXrP20qEBRiitTY4PqLcPgzAugDE2O9FO5Y0PDqEKyoGogeAboPrVwIcmRBCiP7M2yRimda6qYfr0lgZAK7R154eBy59EUIIIYLAqyRCa73zJk/J6UUsohuu5so2T5MI6YsQQggReB73RDgaJ009PCUZeyXit70NSrR34xAuOclTCCFE+PCmsTITqATOOz7cmbBPsbR1/CHRe66eCJlaKYQQIox4k0RYgWKt9Ze6uqiUygEa/RKVaMfr48Blm6cQQogg8LgnwtFQ2W3jpNZ6EzJsKiBifV3OaJaplUIIIQLH28bKnnZmiACJlqmVQgghwpBfxl4rpRIdx4PLMeAB4H1jpXM540KAIhJCCCG8250xHajo4SkWYFmvIxKdyCFcQgghwpFHSYRSagyQCKzTWi8NZEC9pZRa7vjSCKQAq7TWEb1rJCbauTvD00qE7M4QQggReN0mEUqpw44vS4Ey4ChhPpFSKbUC+w4Sm9tjJUB+6KLqvWhvKxGyO0MIIUQQ9FSJyARMWuuaIMXiDzO11ms6PGZVShkjuRrh9RZP2Z0hhBAiCHpqrDR3TCAcDZTtPgIbntdMSqmsDo95nUAopVKUUrcqpW5taWnxY3i+cY299npipTRWCiGECJyekgir+zdKqSRgIfax1ueB1UB24ELzyTKg0rGsgVIqFyjy4T5PAgeBg3V1dX4Mzzc+VyJkOUMIIUQA9ZREtPvrXWvdpLV2Nlbu1Fp/WWv9VmDD847W2oJ9GWalUuq822Pe+hXwEeAjaWlpfozQN77PiZDlDCGEEIHTUxLRU+3c3NWDSqlv9y6c3lFKmYA87JMzi4Eyt90aHtNaN2itD2mtD0VHezMZPDB8nlgplQghhBAB1NM7pLGHa929m/ll2JTjjd/THRX5bj0PhVrrAufXSqm1wCallFVr3WXiEwlcB3B5ehS47M4QQggRBD0lEQVKqe5mQgzp5i/8nhIPj2mti7FXEjzm6H8o63Afi1IqH3svR8QmEc6eiOYWL5OI65ehrRUMUQGKTAghRH/WUxJhxd486SkFfKd34QSEFWgIdRC94UwiWto8XM4wGCB2MDRftG/zjE8KYHRCCCH6q56SCLPW+jlvbuboSQgJrbXZMViqtMOlPLysaoQb1xZPTysRYD8/o/mifUlDkgghhBAB0G0SobX+krc301p/t3fh9NoypdRq7JUHG/blldJIHjQFbhMrPa1EgL258uJp2aEhhBAiYEK/9cCPHMlCWI/m9kVstJdzIkB2aAghhAg4vxwFLgIr2uA8gMuLJCJWjgMXQggRWJJERIAYb+dEgJyfIYQQIuAkiYgAMT4tZ8hx4EIIIQJLkogIEGPwcuw12HdngCQRQgghAkaSiAjgmhPhy3KG7M4QQggRIJJERADn2Otmn3ZnSGOlEEKIwJAkIgLE+lKJkPMzhBBCBJgkEREgOqoXjZWyO0MIIUSASBIRAVxjr72dWAlSiRBCCBEwkkREANecCK/OzpAkQgghRGBJEhEBYnxazpCeCCGEEIElSUQEiPZpOUOGTQkhhAgsSSIiQKwsZwghhAhDkkREAGcloqXNhwO4ZHeGEEKIAJEkIgL4dABXzABQUdByFVqaAxSZEEKI/kySiAgQY/ChsVIpOclTCCFEQEkSEQFion04gAukL0IIIURASRIRAaINPoy9BkkihBBCBJQkERHAuTvDqwO4QJIIIYQQASVJRARwLmd4XYmQHRpCCCECSJKICBDtS2MlyHHgQgghAkqSiAgQ45oTodFaDuESQggRHiSJiABKKaINzh0aviQRspwhhBDC/ySJiBDOgVNeTa2USoQQQogAkiQiQrgO4WqR5QwhhBDhQZKICOE6hMubSkS80f7ZdiwAEQkhhOjvJImIEK5KhDc7NDIXAAoOb4QrjYEJTAghRL8lSUSEcB3C5c1yRtJIMN0Nrc2w7+8BiUsIIUT/JUlEhIjxZTkDYNpn7J+r/s/PEQkhhOjvJImIEDG+LGcA3HY/xA6GkxVQfygAkQkhhOivJImIED4fwhWbAJMetH+9669+jkoIIUR/JklEhIiJ9vEQLoCpjiWNXWuhrdWPUQkhhOjPJImIEDEGHw/hAhh9JxhvgYun4Og7fo5MCCFEfyVJRIRwNVb6UokwGGDqI/avpcFSCCGEn0gSESF8mhPhbuqn7Z8PvApX5VRPIYQQvSdJRIRwTaz0ZTkDIHksjJ4DLR/C1v8HV5v8GJ0QQoj+SJKICOGsRLT4WokAyPqc/fOWn8GaTPjzEij/XzlbQwghhE+iQx2AL5RSWcBKrXV+F9eWA84Zzyat9ZqgBhcgzp4In3ZnOE39NLReg90lcPx9qN5k/zj+ATz0Wz9FKoQQor+IqCTCkTw87PjW1MX15QBa61LH9yalVJHWuiB4UQaGcznDp90ZTkrBjH+yf1w+B5Y/waZ/g7oDfolRCCFE/xJRyxlaa4vWuhBY281TCrTWxW7PtwK5QQkuwHrdWNnRwFSY/pj96wsn/XNPIYQQ/UpEJRH/v737jZHjvus4/vnad07jJM76HDsJDU2zJjbir3p2+SOhgNqzqgqhqujOEZCCAOX8qOojfLIEQjxA1uVpy4OzhSgSSE18rUBIPMBO4RFUcHcPSqCywYtMKMVOfL42tEByzpcH85u98dzs7uzszuzO3vslje52ZufPffd257u/v92YWUPSbMamLTPrK5EwsyNmdsLMTmxvbw/nAge0M3fGACURaQefkPZNS/9zT3r3e8M7LgBgT5iYJEJR9cZWxvpNZScX3XxW0nVJ1+/cuTPodQ3FziyeQyqJkKLxIw49Hf3+zreGd1wAwJ4wSUnEjHYaVCZtSTrS57E+L+mkpJPHjh0b9LqGIp6Aa7vfWTx7OfTB6CdVGgCAPk1SEjE07n7X3W+4+42pqfFoezo16DgRnRz6vujnd/5zuMcFAEy8kdwhQy+KXd0zO1hw96xqiiwzGesaku7m3H9sDTTsdTeURAAAChpJEhF6UFzq+cT+rClKGNJmJG0M+VyViyfgKi+JoCQCANCfianOCKUVrdBLI6nh7tdGcU3DFE8FTnUGAGBc1DWJyKq2kKRlSRfiB2FwqtonEJI0VXpJBNUZAID+jEerwZzMrCnpnKIBpGbNbEXSejzAlLtfMrPFMC5EQ9Gw17UfrVKSDkyV1SaCkggAQDG1SiLCCJRLPZ4z7LYWY2Fq3xCGvc7y6DHJ9kvffUva/j9p6qHhHh8AMLHqWp2x58TjRAw0AVeWffulxxhwCgDQP5KImpgexgRcnVClAQAogCSiJkobJ0LaSSK+TeNKAEB+JBE1sTOLZxklEfTQAAD0jySiJg5UURJBdQYAoA8kETXRbhMx7Am4pEQSQUkEACA/koiaaFdnbJdZnUFJBAAgP5KImmg3rCyjJOJxkggAQP9IImpien9Jw15L0qNPSrZP+u/b0v33hn98AMBEIomoiVLHidg/HSUScumd/xr+8QEAE4kkoiZKG7EyRg8NAECfSCJqotSSCIkeGgCAvpFE1MRUmeNESPTQAAD0jSSiJqbLHLFSojoDANA3koiamN5XVUkE1RkAgHxIImpieqrsJIKSCABAf0giamJqX1SdUX7DSpIIAEA+JBE1EffOePf++3IvIZF47Ono5zvfkt6/P/zjAwAmDklETezfZwqFEbr/fglJxNRD0iNHJb8fjVwJAEAPJBE10p4/gyoNAMAYIImokVIn4ZKkQ89EP+mhAQDIgSSiRtpjRWzTQwMAMHokETUSj1q5XUabCImhrwEAfZka9QUgvwNxD43SSiLCgFO3/0n65oY0/XC0NJ6VzMo5JwCgtkgiaiSuziitJOLxkETc/Gq0xJ75CemlVekDj5dzXgBALVGdUSOlT8L1/T8pfeQl6UM/LT3949ITJ6WHDkn/8ffSn56V3v1uOecFANQSJRE1Ml12ErF/WvrUHzy47t4t6Y8+Kb35NelLvyL90pek6Q+Uc34AQK1QElEjpc/kmeXws9Kv/nk0EFXrr6XVX5fuv1fd+QEAY4uSiBqJSyK2yyqJ6OSJ56XP/Jn0xZ+Xrv+ltPKC9NSPSoefk2aa0pEfkI6elB56tNrrAgCMFElEjcSTcL1bdRIhSU/9iPTSV6Q/+bR055+jJa3xrHTsh6TDH5YeOSIdDMtjT0eJxsGZyi8bAFAekogaOTAVl0RUWJ2R9Mwp6XNfl26/IW3+m7TZipa3/0V6+4a0dStaOjl4RDryvPShn5Je+C1KLgCg5kgiaiQuiSitYWUeDzekD/9MtCTdf0+6ezMqofjON6Xv3Q3LpvTtN6W3/3Vn3Ztfi6pFFr4oPfnDI/kzAACDI4mokdIn4BrE/mnp2A9GSxb3aDjtO9+Q/uq3pbe+IV3+mPTJZWn21xjMCgBqiN4ZNVJ6F88ymUWDWT0/J738Vekjn5G2/1f6i89JX/5NxqAAgBoiiaiRnS6eNUwikg4clD71BekXL0vTj0hvfFn6w09IW/8+6isDAPSBJKJG2hNwjWN1RhE/dlZa/Btp5rh0+x+lSz8n3frbEV8UACAv2kTUSFydMZIunmU5ekJ6+XVp9Tei+Tr++Bekj/1ONDaFLLSVsJxtJsJz4n3GXfNno7YkAFBTtUwizGxW0gV3X8jYthh+PRV+Lrn7VmUXV6L2BFyTlERI0sOHpV++Il37XenvvhD93AuWbkW9XQCgpmqVRITk4cXwsJmxfdHdLyUfS1qXdLyaKyzX808+phdOHNVTjz886ksZvv1T0id+X/rgrPTGV6T370vyqFeHclTfePwcT/w+5vbV6u0HALuY1+UDNyEkE5fd/VRiXUPS2WQSEdbfk7Tg7teKnOv06dO+trY20PUCAFAXZrbu7qfzPHeSGlY2Ja2EZCKppYxSi27M7IiZnTCzE9vb20O7QAAAJsnEJBHuviHpVEb7h6aiRKIfn5V0XdL1O3fuDOPyAACYOBOTREjtRKLNzOYltQpUZXxe0klJJ48dOzasywMAYKJMVBKRFKo1Lkj6eL/7uvtdd7/h7jempmj8BgBAlpHcIUOviV3dMztYKNhFc3mAfQEAQA8jSSJCD4pLPZ9YkJmdl7Ts7v22hQAAADlNXHVGKOVYTSYQZjY3wksCAGAi1bXCfyZrZUgW1uIEIrSLyNXXFQAA9KdWSYSZNSWdkzQnadbMViStu/ulsO1qeF5618OVXigAAHtArZKIUMKw1GVbDWZdAgBgMkxcmwgAAFANkggAAFAISQQAACiklrN4VsnM3pJ0a0iH2y/pSUm3Jd0f0jH3GmI4HMRxcMRwcMRwcGXE8Fl3P5rniSQRFTKzE4om9jrp7jdGfT11RAyHgzgOjhgOjhgObtQxpDoDAAAUQhIBAAAKIYmo1l1Jvxd+ohhiOBzEcXDEcHDEcHAjjSFtIgAAQCGURAAAgEJIIgAAQCEkEQAAoBCSCAAAUAhJBAAAKIQkAgAAFEISAQAACpka9QXsFWa2KGkzPGy6+yujvJ46CDGTpFPh55K7b6W2E9OczOyKuy+k1hHDHMzsvKQthVi5+2piGzHMIfF+bkg6Iuki7+fuzGxW0oX0+zZs6xqvquJJElGB+M0Tf/CYWdPMVtz93GivbHyZ2aK7X0o+lrQu6XjiMTHNKXwYzafWEcMczOyKogS2FR67mR129y1imE9Iwi6lkoYrkhbC78QxIbxfXwwPmxnbu8aryngyYmUFzGzd3U+l1t109+OjuqZxZmYNSWeTSURYf0/SgrtfI6b9MbM5SVfd3RLriGEP4cO4kfwWZ2bNREJBDHPoUAq2rFAaQRyzhWTickZsusarynjSJqJk4YY4m7FpK3ywY7empJUQu6SWpCYx7Y+Zzbv7tdQ6YpjPsqTV5IpEAkEM82uGG2JSIyQQxLEPveJVdTxJIsrXVFSXmrap7Bd6z3P3DUmnkkWfQVMhkRAxzSV8cG9kbCKGPYQP40b4fT58QJ9PJLfEML+XJa2Hao24ZGwlbCOO/ekVr0rjSRJRvhntNG5J2lLUuAgZQiLRZmbzklrhGzUxza9d9J5CDHuLP4wb7r4a/vcuSXo9bCeGOYX383FJF0K1ZPI9Thz70ytelcaTJAJjL3zzuyDp46O+ljoJ1RirvZ+JDmYUlUS0k7C4dIxi9v6YWVNRw97nFCViVxO9NVBj9M6oxkzGuoZGNP97DS0ralCZLKIjpl2ED+2sEogkYthdS9pJHBLiYuENEcO8lhI9A5bM7FVJr5tZ/D9KHPvTK16VxZMkonxrCvWqKTPKrqtGQqhDXU4VyRPT3uYkNdLfmBPjHbwmYtiVu7fMrNPmLfF/mEvcMyi5zt03zGxB0hlJF0Uc+9Hr/67S/0uSiJKF1sctM2ukvtE00i3m8aBQ3LmaTCDMbC508SSmXaS7x0pRl7pUV0Vi2NtGsktn0JS0xnt7YC1Jd4ljf/LEq8p40iaiGsuK6vQltVvM8+boInx7WUt2p0t9qyamgyOGvS2FRVI7Rq1Eo0Bi2EO4cb2YsWleUfsIiTh2klUtIfWOV2XxZLCpioRv1S1FxUwM6dpFqM+/2WHz4UTjNmKaQ0i+zin60F6VtJL4xkIMewg9g+JRA4+4+1JqOzHsIdE4+q5CjxftLmUkjkH4DDynqFpyVlGytZ4xim/HeFUVT5IIAABQCNUZAACgEJIIAABQCEkEAAAohCQCAAAUQhIBAAAKIYkAgA7C+CTMJAl0QBIBVMDMVszsppm5mV0JYw+MtTD9dafxOiZeSB6uSFof9bWkmVnTzK6G/6eV3nsA5WCcCKAiYfCXZXc/nLFtNj39eVU6nTvcRF9MD660l4RBku65e8dJNEbJzFzR5HTM1oqRoCQCGA+jnFo689zuvrGXEwgpcwbPsZGoZmF4aIwMSQQwHrLmFtgL50Zxc4rm8RjbRAeTjyQCGKHQcG9Z0fj4Y3VuM5tNTyU+TrIaPcbrQjXEpDsjSiEwYkwFDozWWUUT5Cjc0CXpZmqinWQjzI9KetXdN8IN9LKiyaGeC8c6owcn2ErvezHxzbXjuRMTJs1LeqA9QEgsmpI2Fc0yuBnXySeuaVbSYe1UlXxU0j/0qrvPu3+YoGg5eX3hmpclLUo6pWga707HOyPpqruvJmLUlKSsiYrC+ZqJ521m/S2DvFYFzElaSPzti2H9rknCgNK4OwsLSwWLog/5exnrG9FbMXOf85IWU+tuKpqVT4puSvckzSeeH/8+Hz8vPJ5VdOPMde6w3VOP5xU1Du14jfExJc11O1aXc+bev8O6e5Jm8xwvY93N5L6J582n1q1kvC6FX6sC/0uz4boaYVlMXNfNUfx/s+zNhZIIYEzFpQG+uzfHqqJpgpfcvRWe15J2fYs+o6i9w0LYtmFmc2bW8OL16JcVfZNuc/dXzOyemb3m7lvuvmVmUlRSkdTK0wtl0P3T+3U7nnZKF2Ibkk6Hn8ljpEsdliS1/+YhvFb9mouPI+ms75RcUQKBSpFEAOPrtLSriFyS7kp6IAnocGNdkjQTitJPa+cmOpPeP49QjbHZIQFpKbqxrabWDWLQ/fMcr9A5EonJaUXtEgZ9rfp1JvxMJhAaIDkECiGJAMZXx7r3nGYUDZZ0Mb7RmNkgN5n0t/akTUVtANrXOugNbdg3xA7HS5dO9CNZkjHoa9WvOUmvSDplZuuSXh5ScgL0hd4ZwJhJ9C5oKUoEirqqqOFeVgPAzN4LPXo2dPvWPqOo/n8vaWonJoO+VrkleqRcdPdzkl5VlCwClSOJAEYs4xty/O12TXrgptHWq+tlSASakl5LbYoThNMd2kZ0K21Yk9TskGTMakTdDTOup9vfMKxzNhVVU6yFVYVfqwLS40O0E5jQxXXsh1TH5CCJAMZDK9yYJGkmbqCoqF3DcvKJ4aa0lj5AUuIG0/52HPbbUGjRn7wJpc/d5ZjnMq5nWaHhYGp9VrKR+9t6zv03lEgawt+YWSowwPVsZdyYlxVVIWxJ7dgUeq3iawvzq+QZ3yJrfIhkI0uGwEZ1Rt09hIVlLywKXe8Udcu7ot1dBmfDc+a1u9vhnHbGP2h32wz7XOlyzLl4ffh9Nrmu27kV3ZiTx26mjns+XM/51LHS+80qSlpWwrr19N+Xuua+9g/b4/Ei5sPj9RDrxX6OF45zL943cY7z4Thzib89828Y4LV6oPtnj/+lde3uhnol/VqwsFSxMAEXAIwJM5t3ShJQI1RnAMD4oIsmaoUkAgDGQCiFYC4M1ApJBACMB0ohUDu0iQAAAIVQEgEAAAohiQAAAIWQRAAAgEJIIgAAQCEkEQAAoBCSCAAAUMj/AzlTnW6e7wGyAAAAAElFTkSuQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plt.figure(figsize=(8,6))\n", + "plt.plot([f(x) for x in cg_quad.get_convergence()], label=r\"$f(x^{CG}_k)$\", linewidth=2)\n", + "plt.plot([f(x) for x in scopt_cg_array], label=r\"$f(x^{CG_{PR}}_k)$\", linewidth=2)\n", + "plt.plot([f(x) for x in gd.get_convergence()], label=r\"$f(x^{G}_k)$\", linewidth=2)\n", + "plt.legend(loc=\"best\", fontsize=20)\n", + "plt.xlabel(r\"Iteration number, $k$\", fontsize=20)\n", + "plt.ylabel(\"Function value\", fontsize=20)\n", + "plt.xticks(fontsize=18)\n", + "_ = plt.yticks(fontsize=18)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Non-quadratic function" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Initial function value = 0.6931471805599454\n", + "Initial gradient norm = 1.9334391059533118\n" + ] + } + ], + "source": [ + "import numpy as np\n", + "import sklearn.datasets as skldata\n", + "import scipy.special as scspec\n", + "\n", + "n = 300\n", + "m = 1000\n", + "\n", + "X, y = skldata.make_classification(n_classes=2, n_features=n, n_samples=m, n_informative=n//3)\n", + "C = 1\n", + "def f(w):\n", + " return np.linalg.norm(w)**2 / 2 + C * np.mean(np.logaddexp(np.zeros(X.shape[0]), -y * X.dot(w)))\n", + "\n", + "def grad_f(w):\n", + " denom = scspec.expit(-y * X.dot(w))\n", + " return w - C * X.T.dot(y * denom) / X.shape[0]\n", + "# f = lambda x: -np.sum(np.log(1 - A.T.dot(x))) - np.sum(np.log(1 - x*x))\n", + "# grad_f = lambda x: np.sum(A.dot(np.diagflat(1 / (1 - A.T.dot(x)))), axis=1) + 2 * x / (1 - np.power(x, 2))\n", + "x0 = np.zeros(n)\n", + "print(\"Initial function value = {}\".format(f(x0)))\n", + "print(\"Initial gradient norm = {}\".format(np.linalg.norm(grad_f(x0))))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "#### Implementation of Fletcher-Reeves method" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "outputs": [], + "source": [ + "def ConjugateGradientFR(f, gradf, x0, num_iter=100, tol=1e-8, callback=None, restart=False):\n", + " x = x0\n", + " grad = gradf(x)\n", + " p = -grad\n", + " it = 0\n", + " while np.linalg.norm(gradf(x)) > tol and it < num_iter:\n", + " alpha = utils.backtracking(x, p, method=\"Wolfe\", beta1=0.1, beta2=0.4, rho=0.5, f=f, grad_f=gradf)\n", + " if alpha < 1e-18:\n", + " break\n", + " x = x + alpha * p\n", + " if callback is not None:\n", + " callback(x)\n", + " grad_next = gradf(x)\n", + " beta = grad_next.dot(grad_next) / grad.dot(grad)\n", + " p = -grad_next + beta * p\n", + " grad = grad_next.copy()\n", + " it += 1\n", + " if restart and it % restart == 0:\n", + " grad = gradf(x)\n", + " p = -grad\n", + " return x" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "#### Convergence plot" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "scrolled": true, + "slideshow": { + "slide_type": "fragment" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\t CG by Polak-Rebiere\n", + "Norm of garient = 1.4300709199609023e-05\n", + "Function value = 0.49027579401250215\n", + "\t CG by Fletcher-Reeves\n", + "Convergence in 114 iterations\n", + "Function value = 0.4902757939862087\n", + "Norm of gradient = 8.247750869146107e-06\n", + "\t CG by Fletcher-Reeves with restart n\n", + "Convergence in 82 iterations\n", + "Function value = 0.4902757939880006\n", + "Norm of gradient = 7.016046988177081e-06\n", + "\t Gradient Descent\n", + "Convergence in 361 iterations\n", + "Function value = 0.49027579399156757\n", + "Norm of gradient = 9.535301105947104e-06\n" + ] + } + ], + "source": [ + "import scipy.optimize as scopt\n", + "import liboptpy.restarts as restarts\n", + "\n", + "n_restart = 60\n", + "tol = 1e-5\n", + "max_iter = 600\n", + "\n", + "scopt_cg_array = []\n", + "scopt_cg_callback = lambda x: callback(x, scopt_cg_array)\n", + "x = scopt.minimize(f, x0, tol=tol, method=\"CG\", jac=grad_f, callback=scopt_cg_callback, options={\"maxiter\": max_iter})\n", + "x = x.x\n", + "print(\"\\t CG by Polak-Rebiere\")\n", + "print(\"Norm of garient = {}\".format(np.linalg.norm(grad_f(x))))\n", + "print(\"Function value = {}\".format(f(x)))\n", + "\n", + "print(\"\\t CG by Fletcher-Reeves\")\n", + "cg_fr = methods.fo.ConjugateGradientFR(f, grad_f, ss.Backtracking(\"Wolfe\", rho=0.9, beta1=0.1, beta2=0.4, init_alpha=1.))\n", + "x = cg_fr.solve(x0, tol=tol, max_iter=max_iter, disp=True)\n", + "\n", + "print(\"\\t CG by Fletcher-Reeves with restart n\")\n", + "cg_fr_rest = methods.fo.ConjugateGradientFR(f, grad_f, ss.Backtracking(\"Wolfe\", rho=0.9, beta1=0.1, beta2=0.4, \n", + " init_alpha=1.), restarts.Restart(n // n_restart))\n", + "x = cg_fr_rest.solve(x0, tol=tol, max_iter=max_iter, disp=True)\n", + "\n", + "print(\"\\t Gradient Descent\")\n", + "gd = methods.fo.GradientDescent(f, grad_f, ss.Backtracking(\"Wolfe\", rho=0.9, beta1=0.1, beta2=0.4, init_alpha=1.))\n", + "x = gd.solve(x0, max_iter=max_iter, tol=tol, disp=True)" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhIAAAGKCAYAAABZ4KgMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzsnXlYW9eZ8H9H7KuF8J54iXBsZ3MSgZu1SdqIpNtM2xjsJp1u01i0/b5pO23G1LO0nS6TQpdpp98sQLpPk2LLmel0TZDTyb4AymrHSYy8Jd4BYWMDZjnfH1dXaAMkIRDg9/c8enTvPeee+0os973vqrTWCIIgCIIgJIMl3QIIgiAIgjB7EUVCEARBEISkEUVCEARBEISkEUVCEARBEISkEUVCEARBEISkEUVCEARBEISkEUVCEARBEISkEUVCEARBEISkEUVCEARBEISkyUy3ALOF+fPn65UrV6ZbDEEQBEGYFtrb209qrRdMNE8UiThZuXIlbW1t6RZDEARBEKYFpdSBeOaJa2MClFIblFL3d3d3p1sUQRAEQZhxiCIxAVrrHVrru0pKStItiiAIgiDMOESREARBEAQhaUSRmABxbQiCIAjC2IgiMQHi2hAEQRCEsZGsDUEQZixaawYHBxkZGUm3KIIwJ7BYLGRlZaGUSt2aKVtpjiKuDUFID0NDQ3R2djI4OJhuUQRhzjA4OEhnZydDQ0MpW1NprVO22FymoqJCSx0JQZg+urq6KCkpSemTkyAIhqWvu7sbm8027jylVLvWumKi9cQiIQjCjGNkZASLxSJKhCBMAUopLBZLylyGokgIgjDjGBoaIisrK91iCMKcJSsrK2XuDQm2nACl1AZgg91uT7cognDeMDIyQkZGRrrFEBKgsbERAJvNRldXFzabDavVitPpTGiOMD2k0iIhisQEaK13ADsqKiruTLcsgiAIM5Hq6mq2bt2Kw+EIHisrK6O9vX3cOeXl5WFzhOlDsjbmAC99/2u8/N63cerwm+kWRRCEWYDf70+3CDGpqalh06ZNYQqCedxqtQa3Kysro+Zs3bp12uQUpg5RJNJE1n88QGbHaXyf/2S6RREEYYqpra0NbtfX1yd8vtvt5tZbb4055vP5kpYr3nXGkt/v97Nt2zaqqqqiztmyZUvYHJfLFTUn9Dy/3095eTlutxu3201NTQ1erzc4Xl5eTmNjIx6PJ0weIf2IIpFmsve8lW4RBEGYYkJv0q2trRPOr62tpb6+PhhT0NzczKZNm2Ku6/F4UiKjx+MZU5kYS36Px0NFxfjZgR6PJyoGwu12U11dHfx8AFarFbvdTlVVFU6nk7q6OkJj0+x2Oy6XC6fTidfrnbEWmvMRUSQmYKoKUmVkG0Eu6twww/IHIQhCgPr6esrKyrBarcE8f4/HE/OJvq6uLubxZHC5XDQ0NCR8XqxA9FBLAhBVr6CqqirqM5mKgdvtZtu2bVit1qBrxMTtdlNZWRnmNpnNhCpSM3G9eBFFYgKmrtfGaCGw3scfT/HagiDMVlpaWnA6nbhcLqqqqvB6vTidzqgbZ2NjI9XV1Sm99qZNmxJyvTidTiIL9UVaCmLN8Xg8UQpIW1sbmzZtClokIuebY5WVlXMmyyPVgabpClwVRSINjAwPMzw4+tX3vfhSGqURBGEm0dbWFnaT9Xg8MYMSt2/fnvIbqsPhoKWlJe75VquVpqYmamtrg7ENbW1tYUGVsebYbDZqamrC1mppaQmeF6lkeL3e4FhraytdXV3JfsQZgd/vp6amJmWfI9XrJYqkf6aBDz9YxZf1aOpN/yuvpFEaQRBmAh6Ph4aGBmw2G/X19TidThwOR/A9lPHiA0zzdkdHB5WVlfh8PlpaWmhqaorLHWC1WvH5fDFdFrFwOBxR8sUzJ3Tf5/MF3RaR1/X5fDQ3NweVpvXr1wddJ7FkNIMx7XZ7UFlpaWmhrKwsypXS2NgYXMPn8wUDRGNhrltRUUFlZSXNzc1UVlbicrnw+/3ce++9rF+/ntbW1jCrSX19PQ6HA7/fT2trK5s2bcLn89HV1YXX66W+vh6r1RqUzev1BmNSWlpaqKmpCX5XY8lg1uWItd60oLWWVxyv8vJynSo+/KPb9e41a/XuS9bo3WvW6levvEqPDA6mbH1BmO309fXpvr6+dIuRMqqqqmJuR1JXV6e3bNky4XotLS0x523fvj247XA4dENDQ9Q1Ozo6tN1u193d3WPKELpOIvLPFLZv367tdrvu6OjQWmvd3d2tjdvdKA6HI+w7aGlp0S6Xa9x1Gxoagt9de3u7bm9v11rrsGuZ+93d3bqhoSHsu2xpaQmes3379pjfpcPhCJ7T0dGhrVZrXDKMtd5YxPM3BrTpOO6PYpFIA8V9xrsuGmZEFZPhP8tAh4/cNavTK5ggCGnFfGKdCL/fT2lpadTx0Kd8r9fLxo0bAcMNYmK327Hb7WNaJ6xW66x3HZifzbQ2mPt+vx+r1RrMdAn9DpxOJ5WVldTV1Y353dhstuB3Z37Xbrc77Fpg/BzMOBDT5eB0OuNyRe3cuTNK/lALUSwZ0o3ESKSBgj4j0HI4WzO8sAgQ94YgCOGxAOMx1o3evNl4vd4xlYV43BaJpFY2NjbS2NhITU1NympapILx3Dherzdm50ur1RoVGDrRuj6fL6icmK/169cHS383NDQEXSvl5eUTfrfmWo2NjUElJZHPlg7EIjEBU9Fro7DPSP0czoaRBfnwOvTvegU23JGyawjCXGTll36XbhEA2P+t907JuvHGJthsNjo6OsYcj6zdELqux+OhvLw8rAZFZOxAvP/vvF4vFRUVwSfw6urqWVHy2m63x1TGEvnsoWsBYd+3uW3+HMz92tpaGhsbo2IxfD4ffr8fh8NBZWUl1dXVwZ/J5s2bE5Incr3pQCwSE6CnIP0z/+wwAIM5mpEFuQD07dqVsvUFQZh9xGuNAOPmFflk6/V6gxUfm5ubKS8vB0ZvKiahWRkbN26MMrd3dnbG/cTr8/mCtScqKipmlEViPKqqqvD7/WHyut1uqqqqElYkzOqcoWv5/X68Xi9erzesYFio28putwfP8fl8OByO4Pyx6muMR6z1pguxSKSBgr6AIpGtybQZP4L+V/cw1N9PZm5uOkUThBnNVFkCZgKxKkCOhcPhiLppm/tut5vt27fT0NCA2+2O6q5pzjNvWJFKg9frpa6uLi45Qms+JCL/VGLKb2YwuFwu7r33XsCwCJgZHe3t7cFMi66uLvx+f1gsSax1GxoaaGtrC65rfnc7d+4MrmVSVVVFW1tbMCMFwjNDHA4HFRUVwSwL85jL5aK+vj7ommpoaKCuro7q6upxZYi13rQRT0SmvFKbtfGTzdfr3WvW6kc2XaRbv3uH3n3T2/XuNWv1M9/cnLJrCMJs5nzM2qiqqgpG4MeDy+UaM/NiLLq7u4PXdzgcWmsdlm3Q3d0dM3MhXvkTlUdIH6nM2hDXRhrI6xsCoD8HLCPnyM7vBGDNq79Jp1iCIKQBMwAvUXO06W9PBLNCJBhmdrfbHWbKN4MmE6W+vj7uOhXC3ENcG2kgXJEYpHheDycpoq87C0v/IMW5WWmWUBCE6cLpdOLxeGhqakroPNP0nUjxqNBum7EC/pJJKXS73UET+0xxbwjTi1gk0kBevxEjcSYHbP0HyS8ZAKC/K4sX959Ip2iCIEwBkV0sQ6mrq6Oqqiqp4DiXy5XS7p9jVUMcS36v18vmzZspLy+npKQk7tgKYW6hDDeIMBEVFRV6ovzieHnceQXz3xzi8T8bwFXQyfCg4vUdS1AWzdP/9C984gO3peQ6gjBb6e/vByBXgo8FYUqI529MKdWutR6/TzxikUgL2QOGRaI3x+i3kZGlyS4aRI8ojj75GO0HuujpG0yniIIgCIIQF6JITIBSaoNS6v7u7u6UrZk9YFiBTocogrk2Q3FY+ObrbPj3p6n83qMMDo+k7JqCIAiCMBWIIjEBegoKUh38wI003W7hZNFoB9CMBdkArBswMjiOnx7Ad+JMyq4pCIIgCFOBKBJpYOlfumhxWDiVO6pIDC+/EID5J49RvSaLDIbZc/RUukQUBEEQhLgQRSIN5GXmAdCvRhWJc2vWAZr+k0PU+ap5KLuWPYdT504RBEEQhKlAFIk0YCoSfWr06z9b9k6yFxfDiGKgJ4tVlsP0Hno5XSIKgiAIQlyIIpEGghYJy6hFIrughLxr3gFAT6ZRr73g+PPTL5wgCIIgJMCcrmyplHICwZqtWuvx26dNE7mZgY6fIa6NnIJici+7jJ5f/w8jvQVQDGUDr9LZO0BpYU66RBUEQYgLs1y3zWajq6sLm80W1TAsnjnC7GNOKxJAjda6GkAptV0p5dFa+yc6aaqJFSORV2gl9/LLjeNvnYalcLVlL9vb3+RTN5elRU5BEIR4qK6uZuvWrWHVOcvKymhvbx93Tnl5edgcYXYyZ10bAWtEV8ihVmBjmsQJIyfDsDD0WyyYlSJyC4rJXbsWLBYGDhxmcCSbVZbD/Psf2viXnW/Qd244fQILgpB2/P60PwPFpKamhk2bNkWV+K6pqQk28aqpqaGysjJqztatW6dNTmHqmLOKBGAHQv/y/MCMeLS3KEuUVaKgcB6W/HxyyuwwNMRg7lUAOC1evtfyOr989kDa5BUEYXLU1tYGt+vr6xM+3+12c+utt8Yc8/l8ScsV7zpjye/3+9m2bVtYMzATsymYOSdWH4/Q8/x+P+Xl5bjdbtxuNzU1NXi93uB4eXk5jY2NeDyeMHmE9DMrFAmllEMptX2MMZdSqirwCm1nZwU6I6bPmB63uRnhcRKWjAzj+GUB94ZaC8CHC1oBaNl9bLpFFAQhRYTepFtbWyecX1tbS319fTCmoLm5Odj+O3LdVDbtGkuZGEt+j8dDRcX4rRhidQR1u91UV1eHtUG3Wq3Y7XaqqqpwOp3U1dVFNQtzuVw4nU68Xu+MtdCcj8xoRSKgQNQBmzAsDJHjLjCCKAOBlG6lVENg2A+URpwyY37zgimgIZkbwGicRHcOWDK5eugFFlpO03agm56z0n9DEOY69fX1lJWVYbVasdlswNidOevq6sbs2JkoLpeLhoaGiSdGEKuFeaglAQh+DpOqqqqoz2QqBm63m23btmG1WoOuERO3201lZWWY22Q6CFV4ZuJ66WZGKxJaa6/WuhZoHmNKjda6MWS+DzBVXx/hFggrRpzEjMDM3OhX4T+C3MsuNY7veQNWXI/Sw2xcdJjhEc3/vn582uUUBGF6aWlpwel04nK5qKqqwuv14nQ6o26cjY2NVFdXp/TamzZtSsj14nQ6ieyKHGkpiDXH4/FEKSBtbW1s2rQpaJGInG+OVVZWTnuWR6oDQudagOmMViTGQyllBRwxhvxKKafW2gOEqsFlQGpsgClgtCiVYkiP/hhy166FzEwG9u5lYHgRANcsGADgGV+kp0YQhLlGW1tb2E3W4/HEDErcvn17ym+oDoeDlpaWuOdbrVaampqora0Nxja0tbWFBVXGmmOz2aipqQlbq6WlJXhepJLh9XqDY62trXR1dTEd+P1+ampqUna9VK83U5jN6Z+RwZQmXRgKhge4VylVFZjXkmjqp1KqlIB75Morr5yctBGEujYGyQz+ICx5ecx7/5/Ts+NBjv1mH8vWQVmO0XPjxUM9KZVBEISZg8fjoaGhAZvNRn19PU6nE4fDEXwPZbz4ANNs3tHRQWVlJT6fj5aWFpqamuJyB1itVnw+X0yXRSwcDkeUfPHMCd33+XxBt0XkdX0+H83NzUGlaf369UHXSSwZzWDMiooKKisraW5uprKyEpfLhd/v595772X9+vW0traGWTfq6+txOBz4/X5aW1vZtGkTPp+Prq4uvF4v9fX1WK3WoDvG6/UGY0daWlqoqakJfqaxZDDrZ8RabzYzmxUJG+HpnSbB2AittRfwxpgTL38FfAXg+PHUuhXys/IB6LVYGIoYW3jPPfR6dnLm1cOcWZjDwitOYlHw2rHT9J0bJi87I6WyCIKQfswgQrvdHsx4AGLepCOf+k3cbnfwxlReXk5ZWRkul4uWlpagEuHz+aisrKS9vT2mYmHeqONVJFKB3W6no6NjzLFQV0DodxMLp9NJTU0NdXV1UQGb5eXltLS0BIM6zVoX27Ztw263B5UK83sJzSqJvO7mzZvZunUrVVVVOBwOysvL6e7uHleG0J/ZRJ9jNjGbFYnp4IfA/QALFy58LZULl+YacaCdGRaGIuIkMktKKN18N8e/811O7ipixbWHWb2oiD1HT7P7SA/lK2yxlhQEYZZjPglPhN/vp7Q0MpY8/Ebl9XrZuNEonbN9+2jSm91ux263j2mdsFqts970brPZgp/R/E7cbqOwcahi4XA4gvEapsvB6XTG5TLauXNn8Ds01wy15MSSYa4ya2MkAsS6o8ZK+0wKrXUncAXw1dOnT6diySDz8+YDcDIjg6EY+pz1Q3eSUVxI38ls+l5/iysvNH5hXxD3hiDMWUJjAcZjrBu9eRMzLQqxlIV43BaJpFY2NjbS2NhITU1NympapILIz+7z+bBarXg8nuBr/fr1wRLdDQ0NtLS0UFZWRnl5+YTfgblWY2NjUEmZSIa5ymy2SLQRuy6Ejcm5M8LQWu8AdlRUVNyZqjUBSvNMi0QGwzF+DBmFBVirq+j80U/xv3iaqysz2XWyg10dNrjxolSKIgizh6/OS7cEBl+dGoU+3tgEm802pisAoms3hK7r8XgoLy8Pq0ERmYYZr1vD6/VSUVERfLKvrq6esRkJ5mcK/V7MbfP7Mvdra2tpbGyMcj/4fD78fj8Oh4PKykqqq6uD393mzZsTlil0vdnMrLVIBAInfYHsjVCsgYyNGY1pkejMyGBIxdbnrBs/BECPL5srP7OBbz/x71y848fTJqMgCNNHvNYIMG6KkU/MXq83WPGxubmZ8vJyYPRmZRKalbFx48YoM35nZ2fcT9I+ny9Ye6KiomJGWSQiMeMdQmX0+/14vV68Xm9YYa9Q95Ldbg+e4/P5cDgcwflj1cEYj1jrzXZmi0VirKCAOmArUAtGAStSnOKplNoAbEh14FGYa2MMRSJ7xQryl1o4e3gkeOzygy/xZvdZLizJT6k8gjArmCJLwEwgVgXIsXA4HFE3bXPf7Xazfft2GhoacLvdUd01zXnmjTBSafB6vdTV1cUlR2jNh0Tkn0q8Xi8NDQ20tbVRX18f9hl37twZzNowqaqqoq2tLZg5AsZ3ZFojHA4HFRUVwSwL85jL5aK+vj7oQmpoaKCuro7q6upxZYi13qxHaz1jXxgpnnVAO6CBBsAVMceFUYSqCtgyVbKUl5frVLLPv09f/tPL9eU/vVzf+rPr9NOHn445r+/bt+u33r1Mn/1Nk267ar3evWat/t3vntFaa91+oEu72w6lVC5BmAn09fXpvr6+dIuRMqqqqmJuR85pb2+Pe02Xy6W7u7sTkqO7uzt4fYfDobXWuqOjI2zc5XLFlC3WduScROUR0kc8f2NAm47j/jijXRtaa5/WulZrXa61VlrrsEqWgTmNWmuPNspkJ94NJ02YFgmAY/o0X37yyzHn5a4qY+nbesiz9dN7yToATjz+FAB3/NtTfHH7i+w5emrqBRYEYUowA/sSNXObfvxEMCtEgmG+d7vdYfEQZtBkotTX18ddp0KYe8xoRWImoJTaoJS638wPThUFWQXBxl0wWqAqiosDpsLnf0HRNdcAsPjJhzm9/2BwyhF/f0plEwRh+nA6nXg8HpqamhI6zzSpJxKXUFVVFYwV2LJlS1idBDOrIVGfvVm7wsxiEM4/RJGYAK31Dq31XSUlJSldVylFUXZRcL/MOkaH87Xvg8LFcGIPl5z7OX1Z2Sw/vp9D1dVc2rkPgBOnB1IqmyAIqSWyi2UodXV1waJGieJyuVLa/XOsKotjye/1etm8eTPl5eWUlJTEHVshzC2U4QYRJqKiokJHNp6ZLFf+/EpGtBFI+a6V7+LbN3879sRH6+FP3wTgf8+Ws/9ZK9cce5WzmTl8/ubPsuGDN/J55+qUyiYI6aS/37Cy5ebmTjBTEIRkiOdvTCnVrrUev088YpGYkKlybQBBJQKgf2gc98QNn4ebjAjii4uP87VrPs7LqyrIHxrg7577OcdPSIyEIAiCkB5EkZiAqXJtALzP/r7gdv/wOIpEZjbc+NcALB46jMWi+fLaD3KwcCErTh/D/tBYXdYFQRAEYWoRRSKN/P21f49rneGTHBieIM4hOx/mLSeDYZapE/Rn5vD9q406+tc++3t6X9/LE2+cpH9weKrFFgRBEIQgokikkYKsAt65/J3ABK4Nk/kXA1CmDgPwaulK/rjibWToEZ7/1vf5ix89y8+e2j9V4gqCIAhCFKJITMBUxkgA5GUYaZ/jujZMAorE2swjAHzmljJ+fdltDCsLtqf/xOIznbxyWOIlBEEQhOlDFIkJmMoYCYCczBwABobiSOEMKBKuS4f50z23sOVda8m44AIeudCBRY/wyVd+y+X//RP6X311SmQVBEEQhEhEkUgzORmGIhGfRcJI8Szu3cdF8wsAuKAkn+bVhnvkxiMvc9OLLRz9x69xZmBI4iUEQRCEKWe2NO2as5gVLeOKkVhwifF+Yg9oDUqxemEhjxUt5NELruTmt14EoO+FF/j0FxtQ9lX87Au3T5XogiAIQcxy3Tabja6uLmw2W1TDsHjmCLMPUSTSjGmRGBgeQGuNUmrsyYULIH8+nD0JPW+CdRlrlxQD0HT5n9OdU8za7v2s7T7EVs//Y1/xYl6/6zpWLy6ejo8iCMJ5SnV1NVu3bg2rzllWVkZ7e/u4c8rLy8PmCLMTcW1MwFQHW2ZaMsm0ZDKshxkaGZr4hEWXGu/HdwOwdrFRZrszbx4N697Pzy59T3DqRaeO8sRDz6RcZkEQph+/359uEWJSU1PDpk2bokp819TUBJt41dTUUFlZGTVn69at0yanMHWIIjEBUx1sCQSbd8UVJ7EwXJFYtbAwbPiFBRfzt9e7eKnUqIff49mZOkEFQUiK2tra4HZ9feJNit1uN7feemvMsUSado3HeOuMJb/f72fbtm1hzb9MtmzZEjYnVh+PyPPKy8tpbGzE7XaHXTN0zOPxRI0J6UUUiRlAMOAynjiJhYE4ieNGZkZuVkbUlOcXrsZ98S0AvKv1f3i98jZ8H7yDw3/7d/Q+/gR6ZCTqHEEQpo7Qm3Rra+uE82tra6mvrw/GFDQ3Nwfbf0eum8qmXWMpE2PJ7/F4qKgYvxWDx+OJioFwu91UV1dHtUG32Wy4XK6gguH1eoNjdrsdl8uF0+nE6/XOWAvN+YgoEjOA3MwkLBLHdgUPWQJhFe+6bDEZgZ0XF1zMqax8AIYPHWLg1VfpefBBDm3ezP4P3Unfrl0IgjDzqK+vp6ysDKvVis1mA8buzFlXVzdmx85EcblcNDQ0JHxeZDdTCFcAgODnMKmqqhq32ygYykukK8TtdlNZWRnmNhHSjygSMwDTtRFXLYlFl0FmrqFI9LwJwB8+dxN3vm059dXruPLCeQCcy8jilx/7Mn93/Wbe/P5PWfHA/Sz43GfJXLCA/pdeYv+H7qS7eduUfSZBEJKjpaUFp9MZfDL3er04nc6oG2djYyPV1dUpvfamTZsScr04nU4iuyJHWgpizfF4PFEKiMfjwWq14vF4cLvdNDU1hY1t2rSJqqoqKisrJctjhiGKxAzALEr1yYc/yY7Xd4w/ObsAVt8OaHjlQQDWLC7i3juuoDg3i6uXj8ZyzF9zMd6Fa+jIm0/+1Vcz/9OfpuyPf6DkrrtgcJCjX/kKXT/72VR9LEEQkqCtrS3sJuvxeGIGJW7fvj3lN1SHw0FLS0vc861WK01NTdTW1uJ2u3G73bS1tYVZEmLNsdls1NTUhK3l9XqpqanB6XRSVVUVpjh5vd7gmq2trXR1dU3ykwqpRNI/ZwCmRaKrv4uvPv1VNqzeMP4JV1TD7l/Dy9vghs+GDX3mljKe29fFe9ctITfT0BP/y/sWe4/3UpSbyV9Xrmbxl/+BnItXcfQfv8axe7+FHh6h9C8/MSWfTRCE+PB4PDQ0NGCz2aivr8fpdOJwOILvoYwXH2DGHXR0dFBZWYnP56OlpYWmpqa43AFWqxWfzxfTZRELh8MRJV88c0L3fT4fzc3NMRWjyLH169cHXSfxyihMLaJITIBSagOwYSp/Yc0Yibi5+DbImQdHX4aTbwRLZwOUFubwm7+6EYA/7TkOwGvHTvPasdMAlC0o5C+uXUHJnXdCRgZHv/JVjtfXk71iOUVjRIULgjD1mEGEdrs9mPEAxLxJRz71m7jd7mDcQXl5OWVlZbhcLlpaWoJKhM/no7Kykvb29piKhXmjns6btN1uH7OeRORY6HcjzAzEtTEB05H+aWZtxE1mDqx9r7G967/HnLa8ND/qWMeJ3uB2ycaNLLzniwAcrv0SA/v2JSaHIAgppbW1lfXr1084z+/3U1paGnU8VLnwer1s3LgRMNwgJna7HbvdPqZ1wmq1iutASAhRJGYACVskAC77oPG+67/GnHJhSV5we35hNgD7Tp4Jm2P7y7+k6PbbGent5a3PfpaRs2cTl0UQhJQQGgswHmPd6E0rgmlRiKUsxOO2SCS1srGxkcbGRmpqalJW00KYXYhrYwZgxkgkhP0WyJ0Hx3fBsd2jFS9DyMkcrTHxmVtW8bXf7mZ/hCKhlGLJN7/JwBtvMPDGXk784F9YtPVLicsjCNPAFT+7It0iAPDyx16eknXjjU2w2Wx0dHSMOR5ZuyF0XY/HQ3l5eVgNitA0TL/fH7dbw+v1UlFRgcPhwOPxUF1dLSWvz0PEIjEDSNi1AZCZDZfdYWy/8Msxp+349HX886Yrueua5SgFh7r7GBwOL0iVUVjA0m/XQ0YGXT//OX0vvJC4PIIgTIp4rRFgWB4irQZerzdY8bG5uZny8nLAUCJC54ZmZWzcuDEqwLGzszPuGg0+ny9Ye6KiokIsEucpYpGYAZgdQE2GRobItMTxo7n6I9D+E3jxAbj1K4ZyEUH5ChvlK4xiMEvn5fGWv49DXWexLwgvrZ132WWU/uUn6Gy6j8N///dc9OCDWLKj1xN9FbtZAAAgAElEQVSEdDJVloCZQKwKkGPhcDiibtrmvtvtZvv27TQ0NOB2u6O6a5rzzIJQkUqD1+ulrq4uLjmqqqqCaycivzC3EIvEDMCsI2EyMBxHYSqACxxGa/GznXDwqQmnXzS/AID9nWdijs//P/+H7BUrOLe3g8777otPBkEQUkJra2vMMthjEWmVqKqqoq6ujqqqKux2e3A79OZuui1Cq1iGKiSJuDVMTEWkubk5rIiUcP4gisQMwBLxY4ir5waAUnBhoM59594Jp6+cb2Rx+E7EViQsubks/trXjOWa7mPw2LH45BAEIWnKy8vx+/0xS0KPR21tbVSviokwK0SCUcXS7XaHKQ5m0GSi1NfXx12nQph7iCIxA+gd7A3bj6vnhokt8E+g40/wqw/DidfHnLoq4M54PVBTIhYF17yNospKdF8fJ773z/HLIQhCUjidTjweT8JP82ZWRiJxCVVVVcGGWFu2bAnrvunz+bBarQkpMzBau8Isby2cf4giMQFKqQ1Kqfu7u7un7BqRikRcPTdMTEViz2+N19M/HHPqZRcYfTh2HT417pIL/+YeVFYWPb/+NX0vz12ftCBMF6FP/ZGuA9MFkegNHIxsi1R2/xyridZY8nu9XjZv3kx5eTklJSVxx1YIcwultU63DLOCiooKHdl4JlV845lv0Pxac3D/V+/7FZeVXhbfyUdegoa3j+4vvAw+EzteondgiCu++hCZFsWuf3wX2Zlj65HHv/MdOu/7EXkV5az4xS9QSsUnjyCkgP5+wyqXm5tEarQgCBMSz9+YUqpdaz1+n3jEIjEjcK1zUbFo9GfV099D77necc4IwXZR+P7x3TAQ23VRmJPJytICBof1uO4NgNKaGjLmzaOvrZ0zTzwZnyyCIAjCeYcoEjOAhfkL+cm7fsI1S64BoMZTw3UPXMfQyNDEJ+cURRzQ8JZ3zOmXLi0GYPcE7o2MoiJKXZsBOPH973PqD3/gZGMTemRk3PMEQRCE8wtRJGYQkRUu+4b6klvozefGHLp8qRkn0TPhMiV33UXGgvn079rFW3/9BU5873sc+fKXOfDRj3HuzbeSk00QBEGYU4giMYOI7LkRdz2JJVcZ70VLjffnmozYieFoi8bqRUbmhu9k7BTQUCx5ecz/1KfCjvW4d3D2uec4+cN/iU82QRAEYU4z5xUJpZRdKTUrQokjS2XHbZG481fwrm/BZ72w4kboPWYEYP7nB6OmLrcZtSQOdsXXnKukupqcSy8ha9mysOOnHnqY4Z6JrRqCIAjC3GZOKxJKqSqgFpgVVVIiS2XHnQZavASu/TRk5UHVj2BlIIvjwNMwMhw2dVlAkXiru4+hQM+Nlt3HuO/x2LnoKjsb+4MPUvbwQ1zwgx8w7wMfIPeKK9D9/fh3PJjApxMEQRDmInNakdBau4Ht6ZYjXiItEgkVpjIpWgwf/y0ULoaRQTgVHsuQm5XBouIchkY0R3r6Odh5ls0/b+Mbv3uVN8bJ5FBKUXz7bSz91r3M//SnASMI8+Ddm0WhEARBOI+Z04rEbCNp10YszLTQrn1RQytsRs+Ng11n+affvxo8/toEKaEmhTffROaSJehz5zjzxBMc+bu/Y7inh5GBBAppCYIgCHOCtHb/VEo5gK1a6+oYYy6gK7Br11rXT6twaSDKtRFvsGUsSlbCwaehex9wc9jQMls+z+3vYu/xXh7afTR4/I1j8dWuUBkZ2D7yEY7Xj/5IXr/mWrJWLKfst79FZWUlL7cgCLMSs++HzWajq6sLm80W1XlUmJskZZFQSq1USt2rlLo75NitSqniOM93BAIgNwFRreYCSgRaa3fAPeFWSjWEjG9RStWN8Zq1v7VRro14m3fFoiRgkejeHzVkBly2H+gmtLDp3uNxFsECbB/5C5Z84+uUfmq0wc/ggYOc3rmTwcOHkxJZEOYqtbW1we36+uSeiUI7fYaSSK+N8RhvnYnkr66upqKiApfLRVVVFS6Xi3vvvVeUiPOEhC0SSqmrMeIO3IQEMWqtdwaUiQ6t9f7x1tBaewFvwCIR6zetRmtdHjLfF6ogzFXrRGT6Z1IxEiYlK433WK6NUkORaN1vGHwsCkb0+M28IlFZWVirqhg+fZpTv/s9g4cOAfDW5/8aMjK4aIeb3LVrk5dfEOYQoTfp1tbWCefX1tZSWlqK1WrF5XLhdru59957aW9vj1p3vB4ZieDxeHA6nTHbiI8nf01NDZWVlVG9QrZu3TppmYTZQTIWiY1a61Va6y8BYXcprfVOIPHOMyEopaxjrOGfzdaGeIhSJCZjkTBjJLqjFYnlAUXiSI+x/g2r5qMU7Dt5hsHhxCpXZhQVUfbwQ6x69NHRg8PD+B98ED08PPaJgiDEpL6+nrKyMqxWKzabDYDm5uZg++9Q6urqUqJEgNEArKGhYeKJIfj9frZt2xZThtDOosLcJhlFIrRzVayOX7YkZTGxA7FseF0kqKQEFI8awGm6SxI8v1QptVoptXpoKI5y1ZNkulwbKwKujeB+aT4XluQxNKI50DlxoapIlFJkLVpI6ac/RUZpqXHZn/+CPeuu5NTDDye8niCcz7S0tOB0OoNuAojdmbOxsZHq6qjwskmxadOmhFwvphUjFLfbTXV1dTBmQpj7JKNIhCoPsVpCrkpSFhMbo0GWofiB0kQW0lp7tNbVWusyrXUyv9V/BbwGvHb8+PEkTk+MyGDLSbk2CuZDVgH090BfeAt0W0E2RTmjXq1FRbmsWmBUvNx7PHFFwmTh5z7HxU88TubixcaB4WG6fvZzRs6do/O+++h9/PGk1xaE84W2traoVt1OpxOrNbwczvbt21Meg+BwOGhpaUnoHNNqYlJVVZUyd4swO0hGkVAhQZY65OBVSqmHgLn0CPpDYA2wZuHChVN+sZRaJJQC63Jj238wYkixYv6oVWJRcS4r5xspofuTsEhErr3wi18kc8kSAPq8Xg7ceRfHv/Nd3rrnb9Dnzk1qfUGYq3g8Hqqrq7HZbNTX1+P1eoPHI+MNxgq8BMNS0djYSG1tLR6PJ2i5GO+cUKxWa9wBnE6nk7a2trBjHo8nZpyFMHdJWJHQWu8ASpRSXUCTUuoNpVQnsBPYrrV+JAVyxXKPWIHOFKwdN1rrTq3161rr1zMzpz5TNlKRmFT6J0DJCuM9QpEAWFFaENxeWJzDysB+Mq6NSOb92fu4+E+PUFTpBK3p37ULgJGeHs4888yk1xeEuYjT6WT9+vVUVVWxZcuWYPCi0+mMCmRsa2uLOgaGW8HlcuFyufB4PPh8vqBlINKiMRbr168PKjETYbVaaWpqora2FrfbjdvtxmazUVNTM/HJwpwhqbuj1vrbSqlGoAIjpsEHtGmtU9F8oY3YJa1tQHy/3SlEKbUB2DAdGnaka2NSBalg1CLRfSBqKDROYvG8XJQyvFT74mjmFS8ld93F6Uf+RME1byO7bBXdv/gFh1w15Kxdy8pf/ieWgoKJFxGE84jW1taooMpYCoPf76e0NNrTGzrX6/WyceNGwHCDmPh8PiorK2lvb4+pXFitVrq6YnmXY+NwOKJkjCWzMHdJurKl1rpHa71Ta90EtBI78DKZdf2AL5C9EYpVa+1JxTUSlGeH1vqukpKSKb9WpiVcr5u0RWIM1wYQtECAESNxUWB//8n4mnnFQ8F117H62WdY/uMfU/Kh0X+OA3v20Bua5SEIAmDc/OO5CY91ozcfeLxeL3a7PaaiYLfbxxwzidcNAqOulJqampTVtBBmF8nUkbhba31fxOEywK6UsgEdCbg3xsrwqAO2YjTcMitgTrsSEbj2tFkkIplUjASA1XRtRFskFhaPulGs+VkU5WaSaVEcPdVP37lh8rIzJnftABmFRhBnTlkZCz7/OU7867/B4CD+//5vzjz3HKcfbuGC736HguuuS8n1hLnNq2svSbcIAFyy59WJJyWBz+eLK77AZrPR0dEx5nhkNkXouhNdw+/3xx3j4PV6qaiowOFwBGM8ImtdCHOfZCwSUVkZWuvnA0/uTcTRaTOktXcd4FBKNYSmZwYyLDqUUs5AB0+n1jotTrfptEjY59l590Xv5hKb8c9y8orE2BaJS5cYRUgzLQqlFJkZlmDFy1ePnprcdcdg/qc+xaqHHwLgzGOP4/9VM8NdXXT9/BdTcj1BmE3Ea40Aw6oQaTXwer3BCpTNzc2Ulxs1/Xw+X9hcj8dDeXk5Pp8vaE0IpbOzM+54Cp/PF6w9UVFRIRaJ85RkYiTGdGEopVYClcC47SC11j4C1oZx5px3SchKKepvquepw09R01IzufRPCA+21NrI5AiwsDiXP37+7ZTkZwePrZxfgO/kGe74t6f4tw87eM8VSyZ3/RhkLVlC/vr1nG1tJXfdOvpfeokzTz7JwL599Pz3r8m76kqK3vGOlF9XmBtMlSVgJhCrJsNYOByOqJu2ue92u9m+fTsNDQ243e6ofhctLS1UVlYCsHHjxig3idfrpa6uLi45qqqqgmsnIr8wt5hQkQiUxHZi9MW4OnBsS6ypQAeQ2gopaSYdro3cDKPC5aQtErlWyCmGgVPQc2jUQhFg7eLw1ijvuWIJj+wx6mV4dh+bEkUC4ILv/zMDezvIX1/BgTvvou/FF/G9930wMoKlqIjVTz0pjb+E847W1taEykqbVgnTelBVVRVWTXIsZcBUOMxaD6HWh0TcGibm+c3NzTQ1NSV0rjA3mNC1EXBbfFtrXQF8B2jCiG0Ie2mtLVrri7XWL0ypxNPMdLo2TMxS2ZMOtlQKVtxgbO+4G4YHx51eVX4h//WZ6wHYfWRq3BsAmaWlFFzzNpTFYqSIAoyMoHJyGDl9mrMReemCMJcpLy/H7/fj8/kSynaora1NuHqkqSiElsMOtWyYQZOJUl9fT1NTU9wuEWFukWiMxD8B7YGMjbBXvJ0/hYkxFYlJp38C/NkPoHAxHHoW9k9cWXLt4mIsyugE2j849b0ySj7yERZ88Qus+OV/YvvYxwB46wtf5NTvfz/l1xaEmYDT6cTj8ST8NG9mXiQSl+DxeILppZs2bcLtdocFYVqt1oRTN83aFVarFY8nLTHxQppJKEYiUCci6rddKTUPqAikbGqt9bgxEsL4BF0bk42RAChaBBc74fn/hC4flL1z3Ol52RnYFxSy93gve4/3cvkF8yYvwzhYcnKYv3kzACojg87GRoa7u3nrC18k5+KLybn44im9viBMB6HugkjXQbzxCLFwuVw0NjbGXY461PWxZUu4h3q8stZjye/1etm8eTM2m42uri4qKiokTuI8JOk6EkqpYqXUykCAZQlGfIQPI5ZizqCU2qCUur+7u3viySnCtEgcPXOUpw8/jdaTLNFhthSPUZgqFmZGx67DqagvFj+569ZRcMMNwf3TO40s4pG+Ps48+xwj/SlQrAQhDYQqC5NRHGKRyu6fYzGW/A6Hg+7ubjo6Ouju7k64T4cwN0hYkVBKzVNKPYzRRKsDo9qkF0OJaAQS60M7w0lLjETGaDtxV4uL1qOtk1vQutJ4D60n4XsUTh2OOf3SpYYi8fJb06tIKIuF5T+6jwv/9f8BcOqPf+T4d7/H3lvewcGPfYzj3/vetMojCIIgTEwyFokvAbVaawuwUWttC7wsgbH4a6sKMYnsufGG/43JLWimgZoWiUOt8PM/h//+dMzp61caStOjr5+YvDUkCQquuw6Vk8PAnj10NjUx3GMoNKf/+FBa5BEEQRDGJhlFok1r/XxgO+wxXWu9k7GrVQpxkmEJrypZkDXJnhSRFS59fzLeDz4Lw0NR069aVsL8whwOdfWx5+jpyV07CSz5+cx7//shM5Pi976XFQ/cT+bChQwdP07/rt2cee45Bsap6icIgiBMH8koEqGPhO1KqTsixudU/k86YiQiOTs4yf4XhQshMw/6uqH/FBx4yjg+1Acnogv8ZFgUzkuMtukP7zoWPP7InmP83X+9jP/s1LcCX/yPX2XtC89zwXe/Q/7VV1N4880AHLjrLg5+9GMcuOvDjAxMMj1WEAThPCWV1t1kFAmllLpIKdWMESfxt0qpT5rBl8yxYMt0xEhEcnZokoqEUqPFqLo64M2QmIu3YjdUvf2yxQD85Kl9HOg8w+DwCF/c9iK/fPYgVf/xNGcGoi0ZqUQphQpp3V7kvBUAfc5QYoZ7ejjbKvUm5ioWi4Xh4alPPxaE85WRkREslqTzLcJIeBWt9Q7AAXQDncBGjAZbZvDlnAq2nAmcGUxBa28zTmLP7+Bc7+jxt2I32Ll59QLesWYB/rODbHG/RMvuY3SfNQpa7T3eyx9fOTp5mRKg4KabuOAHP2DFL35OaSBdtPexRxnu7aXnN7/h+He+w1AarUZCasnMzGRwcPwCaoIgJM/g4CCZmcl0yYgmKXUk8JT+Ka31Ka21T2u9CijRWmck0PlTGIev3/D14PaZwTMMjUzSAlB8gfG+N1AwZl7AQnE4tkXCYlHUVa0DYPfhUzzwnNH4Ky/LiN840pOCYlkJoJSi+PbbyF+/nsJ33AKA372DN66/gcN/s4XO+35EZ1NkU1phtmKxWBgZGZHgWkGYArTWKbVIJNNG/J2ANbLoVKBYlZAiPrDqA2it+fJTX+ah/Q+x/fXt1N9UT+WKyuQWzA/EwHYFquCteid4fw7HdsNgH2TlRZ2yoDCHrAzF6YEh2g8YT/sfvX4FDY/6OHE6ffEJeevWkVFSwnB3NyhFzpo1DLz2Gmcefwy2/E3a5BJSS3FxMZ2dneTl5WGxWFAhTecEQUgcU4Ho6+tLaTnzZNSRTwHT18EqzaQz2DI/y2jr3dXfxdDIEF/43y8kv1hu4JemP6DvFV8ICy8FPQxHXop5ilKK+YVGKurZc8NkWlSwWNXJ3qkPuBwLlZnJssYGlnzzG6x69H+5aPs2LPn5DLyxl3OHDtH30kvi5pgDZGZmUlpaSlZWligRgpAClFJkZWVRWlqaMrcGJNdGvEVrPWZReKXUHXOpRHYgJmRHRUXFndN97fzM/NQtlhcRLJpnhQsccOwVI05i+TUxT5tfmMORHqOi5MKiHBYWGcWy0mmRAMi74gryrrgiuJ9/3XX07txJx7vfA0ND5F65jouam9MooZAKlFJkZ2dPPFEQhLSRjEWiWSl1h1LqqjHG51TWRjqJrB+RZZlEa+0oRaIElgaa84wRcAkwv3D0n/jC4lwWFBkWihO9Myv1MthFdMiIJel/8SUGjxxJo0SCIAjnB8lYJPYBCrAqpSIjoRThdSaESRCpSERWvEyIWIrE/NXG9itusNnhHX9rpIqGYCoOAIuKc0YViTRbJCKZ9/73k7V4MVkXXsixb32LXs9Oeh97nJJNGxnu7WWkp4esCy5It5iCIAhzjmQsEvuAcq21JZClEfqyADtSLON5ixkjYZKdMQkTb15EYE1eiREjkV9q7D9WD8eji1OZMRIAC4tyKc7NJDvTQu/AEGfPTW0tiURQSlFw7bVkX3ghhW+/CYCun/6UfZs28fo117L3VidnWyfZs0QQBEGIIhlFYrPWet844/cmK4wQTmSMRMotEhmZcLcHVODX4MSeqNNCFYlFxTkopVgQOHbydPoCLsej8Ka3A3Bu3z76X3wJAoWNTv3hD+kUSxAEYU6STEGq5yczPttIZ9bGlLs2wHBp3PA5Y/vEa1Gnhbo2Fhbnhh070Tsz23pnLVnC4q9/jdLNm1nW1Mjyn/wYgN7HHpe6BIIgCCkmdfkfc5R0Zm1MSnGIJCsPMnNhqN+wQOQUj44tWGu8T2iRiFAkZlicRCgl1dXBbT08TMa8eQy++SYnvvtdBt7YS9/uXcx7z3tYtHVrGqUUBEGY/aSmrJUwJUTmzk+654ZZSyJ3HoRWNFuwxniPaZEYjctYVJwTODbzFYlQVEYGBTfcAEDnfT+i99FHGT5xku4HfsVI3/RW6BQEQZhriEViFtE3NMmbXl4J9B6NdnOY2Rude2F4EDJG00wXFOYGtxcFakiYMRKzRZEAKN18N8Nnesletpy8q66is6mJgdde42xrK4U33ZRu8QRBEGYtokjMIlKiSIS+m2QXGL03eg5C1z5YsDo4VJyXyYUleWgN1nxDwSgN1JboPDMzgy1jkXvJJSxvGO0nN9Cxl4HXXuPUQw+BstC/6xX6d+1i6PgJFn/9a+SuXj3OaoIgCIKJKBKziKGRIQZHBpMvTDWWIgEwf1VAkfCFKRJKKf7wubejGXW1lOQbikTXLFIkIil8+9vp/Pf/oGfHg/TsCC/E2v3AAyz5ylfSJJkgCMLsQmIkZhn9Q5PIlDBrScRSJMzuoKfejBoqys2iOHdUeSktGFUknu7onFUuDpO8devIufQSVH4+eeXl2D72URZ87rMAnHnyqTRLJwiCMHtIyiKhlFoJ1AAdWuv7AsduBVq11qdSJp0QRd9QH0XZRcmdPJ5FYt6FxnvPWxMuYwu4Np7d18WdTc9gUfCHz93EmsVJypUGVGYm9gcfRI+MoAKBp3p4mM6f/ozBgwc5d/AgKieXgTfeIHvFcrKXLUuzxIIgCDOThC0SSqmrAQ+BMtnmca31TmB9QMkQpohJxUnYLjLeSy6KHgtaJA5PvExBeIXNEQ2f+9XsLB+iQrJXVEYGBdddB4Dvz9/P3ptv5tDdd7P/rrvQQzOniqcgCMJMIhnXxkat9Sqt9ZcwymUHCSgTjpRINkNIZ0EqgE9c9omw/UkpEo6Pwcd/B+vvjh6bZyoSE1skzBiJUPYcPY3/7OyNmTApvq0SAN3fj6W4GJWXx/CJk/S9/HKaJRMEQZiZJKNItIVsxyoTaEtSlhmJ1nqH1vqukpIY7oBp4K/L/5pHqh9h3YJ1wCRjJDKyYOWNkBmjZ4dpkeiJjpGIJCvDQnFutFfspTd7kpdthlD07ndz0X89yKo/PcLqZ5/B+sEPABI3IQiCMBbJxEiEKg8qxviqJGURYqCUYkH+AvIy84AUFKUai1DXhtZRXUAjKS3M4VS/Ye5fZsvjUFcfH/3xc1xzkY0vvXstVy9Pj+I1WZRS5F5ySXC/4IYb6L7/AU4/9EdUZgYDHT4GOvYaaaL/8A8U335bGqUVBEFIP8lYJJRSyrSN65CDVymlHgIeTolkQhimImG6Nnad3MULx19I3QVyCo2Kl8MDcLZzwumhcRLvvnxJcPvZfV1sanyGN7unSOGZZvKvuQYyMhh4Yy8nvv8DTv3mNwzsfpXhkyfpfuCBdIsnCIKQdpJp2rUDKFFKdQFNSqk3lFKdwE7ArbV+JNVCCuGKhNaaD/3uQ3zkDx9hcHgwdRcpNjM3JnZvhMZJvPvyxWFj54ZG5oSbAyCjsJCF99xDwc03Yfv4x1nyja+z7Ef3gVL0tbdLiW1BEM57kkr/1Fp/WynVCFQAdsAHtGmt58bdYwZithTvG+qjf3g0TqJvuI+sjCQLVEUy7wI4vgt6DsHSq8admpUx6vq48kIrjuVWhkc0ly6dxwPPHeSwf+7cYEs/8XFKP/HxsGO5l11G/yuvcLa1lZw1azi3bz+ZCxeQY7enR0hBEIQ0MZk6Eg6t9YMYlgiUUiuVUiVa6/0pk26SKKWqMII/yzBqXjSmWaSkyc00+lz0DfZxZvBM8Hj/UD/F2cVjnZYYiy6DNx6GJ38Aq98NGWP/egwMjQS3LRbFg5+5Aa01TY/7AHhrDikSsSi4/nr6X3mFQzWfMmJKAJWXx6pHdpKZpsBcQRCEdJBMHYmrgHLgvsA2AAEFoiT0WDpRStkBv9a6UWtdC9QopWZtaqrp2ugf7uf0udPB45PK4ojkhs9D0VJ4sxVe+OXo8fafwfP/GTa1f3A46nSlFEuthpxzySIRi6LbbzM6qGpNRkkJluJidF8fZ595Jt2iCYIgTCvJBFtWBFIibVrrsGg/rfXzGK6OmYADo/qmSRvgTJMskyY0RqL3XG/w+KQbeYVdxArX/19j+3CgwNRb7fCbz8Kv/w+c7QpO3VhhVHp87xVLwpYYVSRSqODMQPIuu4yLH3+M1c88zeqnn6L0biP++MxTkiYqCML5xVQ07ZoRdSS01m6llCfkUAWwPV3yTJZQReL04KhFIqWKBIxWvew5ZLz/6Z9Gx95sg9VGuuP7r1rKqoWFrFpYGHb6BeeJRQIgs7Q0uF1ww/Wc+N736H3scbq3bWPw0JucO3SIoePHsX38YxTfJmmigiDMTZJRJCaqExF3HYmAq2Gr1ro6xpgLMB+B7Vrr+vhFNNBa+wNr2YEurbVnglNmLKGKRGiMRMoVCety491/CE4fhb0hX9mhZ4OKhFKKyy+YF3X6gsIcsjIUnWfO0T84TG5WRmrlm6HkXnIJGSUlDB07xtEvh3cOPXnmjCgSgiDMWZJRJJoD9SK2aK1fNA8qpd4J1AG1Ey0QUCA2BXajXCEBJQKttTuwb1dKNWitawL7W4DSyPMCtMRQGGq11pUTyTWTKcw2nvw7+zrDXBspjZEAsAaaU/kPwpmT4WOHnp3wdItFsXheLoe6+jjs78O+oHDCc+YCymJh4T1fpOfX/0PWkiVkLV9G9rJlHPnyVxh47TWGTp4kc/78dIspCIKQchJWJLTWzyulvg3sUEqFdn/yA6546khorb2AN6BQxIpbqNFal4fM9ymlnCH7cVsnAkpHbWDbalopZhvr5hslstuPtVO+KPjVhKWCpoTcecarvwe69xvHSi6C7n1GvMTw0LjZHGC4Nw519XGw6+x5o0gAWDdswLphQ9ixnv/5DWeeeIIzTz/NvD/7M/S5cwweOQJak71yZXoEFQRBSCHJBFuitfZorVcBFwO3Aau01qWBYlWTQillJXbjL3+oMhHnWk6gMUR5mLXBlksLl7KyeCW9g708c2Q0MyDlrg2AeQH3xrFdxrvtIrDZYfAsHJu4edXqRUY78c/80ssl//BH7v5ZG/tOnpngrLlJwfXXA3Csvp43bnkHe668io7b30XHu97NWa83zdIJgiBMnsmctlIAACAASURBVKQUCROttU9rvVNrHewCGnBxTAY7hnUjki4S6CwasHa0AN1KKa2U0hiFs+JGKVWqlFqtlFo9NAPaSF+75FoAnjo8mhkwJYqEGSdxfLfxnlMEy4xrc3Bi98bnbr2YG1fN5+y5YfoGh/G8eowv//qV1Ms5Cyi85WawWBg+cZKho0dBKSyFhpXm9MMtaZZOEARh8iSdtTFGvQgbRpzE+qQlMtboinHcz9hxEVEE3Cfjd56amL8CvgJw/PjxSS41ea5beh2/eu1XYcdSHiMBo3ESoYrEhevhxfvh0DNw7afGPb20MIdffPJtHOrqo+vsOT7wr0/yxrHecc+Zq+TY7axsbmak9zRZF15I1uLFnG33cvDjH5dUUUEQ5gQJKxJKqQ0YaZQdxL5RXxTj2Gzlh8D9AAsXLnwtzbKwxrYm6tiUWiROvm685xTDsmuM7YPPxtUdVCnF8tJ8LijJIytDcfRUP2fPDZGfPRUZxzObvCsuD993XI3Ky2Pg9dfpfeJJRk6fYvDwYQaPHCV//XrpKCoIwqwimf/q67XWY7pElFL/MQl5TGLVorACE7elTCFa607zmhUVFdN56Zgszl9MpspkSI+6WabEIlEUXmSKnCKYv8YIwjx9GP7tWvjIf0Hx0gmXyrAoltvy6Thxhv0nz3Lp0hSV857FWLKzya+o4Mzjj3Po7rvDxvzbtlF409ux5OWlSTpBEITESCZGonWC8QnTPyegDUNpiMQGTHt0mlJqg1Lq/u7u7um+dBQZlgyWFobfvFOetQGQH+FByi40ykGvfrexf2IPvPaHuJe7aL4RE7C/8/wMuIyF7S8+TNYFF5Bz6SUUOm+l5KMfIWvZMvS5c5xtlyBMQRBmD1NhZ74IeGHCWWOgtfYrpXwxUjWt6SgoFchE2VFRUXHndF87FsuKlnHw9MHg/pS4NiIViRwjC4P3/yuc64U9v4Xe+GNG7AsK4FXO28yNWBTefDOrdt4cdux4Tg6dTfdx5qmnKLzxhuBxPTLCyOnTZMyLLgAmCIKQbpKxSLQopd6plLpKKRVmpw7sb01grbHKadeFrhPIwEhLVcqZZJEAuLDowrD9aVUkMjLBfouxfSZ+RWJlaQEAvhOiSIyHmSp66ve/53Dtlzjw0Y+xt/I29lx5Fa9fcy3H//n7aZZQEAQhmmQsEmGpmYG0yuAuoJmAQMnqGoy6Dg6lVAPQbrb51lo3KqVcgToQVowS2TVjrzh1zESLRChTEiORH6Hf5YToiwULjPcELBIXzTcUCXFtjE+ew4HKy2Po6FF6fv3rqPGe3/wPCz7/OdQEga6CIAjTSTKKhBe4VWvdE2tQKbVtogW01j4miKUwlQohnEUFi8L2p8QikZkD2UVgtis3LRIAhYHrJ6BIrJyfD8CBzrOpknBOYsnJ4cIf/pC+558na8lispYuJXPJErIWLWLvO29l6PARBg8ckIqYgiDMKJJRJDaPpUQEmGyw5YwikO66wW6fGd3RS3PD3Q5TYpEAwyoRVCRCylwXLjTee4/FvdTColwyLIqTvQMMDA2Tk3l+NPJKhsIbbwiLjzDJv+5aTv/hj/Q+/gRFOTkMHjnC4JEjqIxMim6/TawUgiCkjYRjJAK9NlYqpe5VSgVz15RStyqlikOrXM4FtNY7tNZ3lZSUpFsUAK5acBWXll7K5aVGbYIpydqA8DiJMItEQJE4c8KoJxEHGRbF4uJcAI71DKRKwvOKYKntb36Tve94Jwfu+jCHv3gPb33+8/T+7/+mVzhBEM5rElYklFJXYwQ+KkLSNLXWO4H1SqmVqRJOiCYrI4vm9zVTf7PRt2xKXBsQoUiExEhkF0JmntF341z81SqXzDMUibf8UyTvHKfolluwBLI2MhbMJ3fdOnLWGAXKzjz+eDpFEwThPCcZ18bGQMMu0+wfRGu9Uyl1B7A/BbIJ45CXaRQsmjpFIiTgMjvEtaGUYZXwHzDiJEKtFeOw1JoHB7o50iOKRDJkLljA6scfQ2MUtAI4+/zzHLjzLs48KaW2BUFIH8koEm0h27Fs22OldM5KZlqMhImpSExZjERW/uh2Znb4WKgiUVoW13JLrIZF4rBYJJJGZWeH1aTPu+IKLIWFnDtwgK7772fkdC9Dx44yeOw4hTfeQMmdMyLRSBCEOU4yikRkumckq5KUZUYy09I/TXIycgDDIqG1Tn2wXUb22GPBzI34Ay4vsBqKz1v+KVJ8zkNUZib511xD786dHPva18PGeh97jHl//udYCgrSJJ0gCOcLySgSSil1t9b6PkKUikA30LrAS5hiMi2ZZFmyGBwZ5NzIuaBikTIyssYeM2tJnDkR93JL5hmKhLg2Usv8Ghf63DkyiorIXLyYrMWL6d7WzLm9HZxtb6fwppvSLaIgCHOchBUJrfUOpdTfKKW6AK2UqmPUnVGrtX4kpRIKY5KXmcfguUH6h/qnQJEYxyJhNvXqeTPu5cxgS3FtpJa8detY3hRecmWoq5POvR2cefKpMEVipK+P4Z4eMhctknRRQRBSRlK9NrTW31ZKNQIVgB3wAW0T1JeYlczUGAmA3MxcTp07Rd9QH/NyUtyHYcX18MT3IDM3emzBauP9xJ64lzNdG4f9/YyMaCwWuZFNFQXXXU/nfzTQ87vfMbB/H0NHjzF09CjDPcafZ+ndn2ThPfekWUpBEOYKyaR/NiulVmqte7TWO7XWTYH3npA5dwTm3ZFacaefmVZHIpSiLCNjomdgCvS3VU740P3wf9uixxZeZrwf2x33ctb8LJbMy6V3YIgX3/RPfIKQNHlXX4WlsJDhkyc58+hjDLz2mqFEZBrPDT2//z06zhoggiAIE5GMRWKb1nr/WINKqVsxGm65gE1KKb+4O6aGZUXL6Ojp4ODpg6yxrUnt4krB2vfGHrPZISMHeg5C/ynILY49L2w5xbsuX8xPntzPb148wtrFxeRlS4XLqcCSnc3yH/+IvldeIWvxYjIXLSJr8WIy5s3jjRtuNEptHzxI9ooV6RZVEIQ5QDLdP1FK3aOU2quUekMp9cGIYRdGrMTzWusvAY5JSynEZHnxcgAOnDowvRfOyEzKvfGeK4zYih8/uY/yb7RwUHpvTBl569Zhu+suit75TvIuu4zM0lIjy+O66wDoffRRzh06xNnW/9/ence3VV4JH/89WrxIXuU1e+KswADBcQgJWwpJodAFmoSlM4WWaUOnLe9Mp6xtZ9q3My0Nhem80yklUDpdoZAApXQokAAFEptAVgghC3b2xPsur5Kf9497Zcu2vMmKriSf7+fjj6Qr6d5zcx3r6FnO8y5NL/yZ5pdfkVYKIURYwkkkioBcjNU77wNuNmdsBBRjjJkISKiS2bFkRobxjfJI05HoHzz/bOO2evTdG4umZzM12xgr0dblZ9OHo58+KiLDvcxIJKp+eD/lKz/O0c/fwqm77uLkP/6jVMgUQoQlnESiUWt9rzkuYqPW+gaMQZcBngFdH3H9NUcptUop9URDQ4PVoQwyM2MmAMdajkX/4IFEYgzjJGw2xZNfvog1i6YCUPpR7ZmITAwj/WMfw+7xgMOBY/IkUhcu7C213fqmJBJCiLELJ5EIlRgk7BD8WB5sGWiRiHrXBkChsWgYle+N6W3TPC6+sdLoFnl1fzVX/+eb7DoWe0laonLk5TH3rTdZsGc3c197jZl/eJLCf/0XALylUmpbCDF24Qy2zFZK3Q+8i1E/Yg1mESpzoKVSSs3QWgc+3UZXQ1mMWb4rn1RHKvUd9TR1NkV+CuhwJpm9Waffgx4/2EY/cHJyVirZLicNbd3sr2zh6e0nuGB67CVqiUrZ+1+r1PPOw+Zy0VVRQcMfnsLf0oyvqrq33HbK2WdR+N3vSu0JIURI4Swj/mOgAXgAI4l4FCN5uAtYAcwC7lNKXaGU+jmwKYLxiiBKKaanGwMujzVHuXvDnQsZU6HbC3UfjfntN184vff+R9Utg57v6PZLFcwoUU4nrgsvBKDye9+j5qH/oOF3v6Nl02Y63nuPxj88RddhGeokhAgt3IJUD2AkEsFeDdwxq13eDmzUWu8OPzwxkklpkzjQcIDq9uroH3zyQmg+Aad2Q97Ypp/e+fH5fHrhZK7+z7f48HTLoCJVX/v9Tt46VMuL/3gJc/JHt8KoCF/O7WvRPh/2jHQc+QXmlNECGp95Fu+WLXhLy0iOwaJsQgjrjTmRUEpdAWRprZ8d6jVa68PAveMJTIxOdrLRJdDYYUGRp0nnw/4/w87fwDnXgWP0ZbptNsWCwgzy05OpbunkREM703OMFUf3Vzbz6n4jMfrjrlPceVWEa2SIQVwXXMD0Xzw2aHtPW5uRSJSV4fm7vwVA9/TQ09aOPU0WBBNChDfY8isYU0AnhFietQGQlZIFQEOnBfFNNkuEHN0CT4a3OOrZk41iVi99cJpOnx+AX5ce6X1+i8zssJTbrDvh3bqVIzd/jo+uuJL95y/kYEkJtY88YnF0QohYEE4isUlr/eBQTyZCWexgsTxrAyxukShaDheuNe6f3GHcHn8H3njAGIA5CmdNMhKJH764n6/8dgdaazbt6+um2XOikbrWzggGLcbCOXkyyfPmoTs6aN+1i+5Tp6C7G4CmPz5vcXRCiFgQTiLxlLmWxsIhnr9xPAGJsclKtrBFwu6Aq9eBskFHI/i64PGV8PoPYM+To9rFlQvye++/fqCG53adpLa1E7tNcfGcHLSWVgmrTX34Yab8x0PM+O1vmP3Ky8zbvh2b203XkSNGYiGEmNDCSSQOA78Adiql/AN+eoDVkQ1RDCc7xWiRqOuo44PaD/D1+KIbgM0Grlzjfltd3/ZAC8UISmZ62Pf9q/jB9UZdiu//2ShwlZeWzJJZOQDsOZ5wi8rGlaSpU8i45hpcixeTNH069jQ3riVLAKP2hK+ujo59+2h5/XUa/vAU9U88QU9Xl8VRCyGiJZxZG4eBNeaAykGUUk+PLyQxFoEWia0nt7L15FbWnreWOy64I7pBuPPAWw3emr5twUnFCFxJDq6/YArffm4vjW1Gs3lBZgrnTTXqYrx/UlYLjTXuZctofe01Tn/nX4Z8jedzn4tiREIIq4TTIvHloZII0/3hBiPGLpBIBPzy/V9GPwi32SLRGFRhs+nkmHbhSnKQl94366MwI5nzphrntvdkMz5/z7jDFJGTfuUV2NzGrA17ZibJc+fivuQSXBddBIB3y1YrwxNCRNGYWyS01ruUUjMx6kSUa61/Ab1VLd/VWu+KaIRiWIGujaEeR4U7z7gNXnej5gBobSxHPkrTPS5qWoyBlYUZKXjcSUzNTuVEQzvlNV7mF0o9iVjhnDSJuaVbQWtsKSm927tPn+ajj11B27Zt6O5ulNNpYZRCiGgIp47EBcAGYCPQ+3VYa/2qUupKpVT5gEW7xBmUnpSOTdno0cY39sB00KgKJBLVH/Rt62qBphOQNW3Uu5nucbHjqDFotCDT+HA6b2omJxra2XOiURKJGGNLHlw3xDlpEkmzZtF1+DC1P/852Oz4qqrorq7CV1WNs6CAKT/9L2xJSRZELIQ4E8IZI3GD1noOGDUWgp8wk4nPAkciEJsYBZuy4bQ56fQb3+TdDguKBAW6NgauBFqzf0yJxDSPq/d+YYaRSMwvyODF9ys5Wucdd5giOtxLlxqJxMM/H/Rc5/79tL3zLmmXXGxBZEKIMyGcRGJ70P1QK4F6wowlJpnJ0qqiGC4PHEgiAFq7W6MfQKBFou5Q/+01B2DuylHvZnqIRCI/w/jWW90stSTiheeLX8BXX48tJQVHQQGOgnychYW0vPoqTc88i7esVBIJIRJIOIlEcPIQqgN8TpixxCSt9TPAMyUlJeGVboyylq7BC2CdcWn5/R8npRtdGwMTixEEJxKBro18cwBmdUsnPn8Pdzy5i0UzsvnSpbGb2E10SdOmMfU/fzJouy011Uwkynq3ab8ff1MT9uxsWV1UiDgVzqwNpZT6knlfB21cqJR6GXglIpGJsFiSSARaJAJmLDNua8e2KuiMnBAtEunGbXVLJx+ebuEveyt5fIusRBmPUouLUcnJdO77kMNrbuDQZZez/9zzOLTsYk5/5ztWhyeECFM4y4g/A2QrpeqBx5RSh5RSdRirf27QWr8W6SDF8JZPW957v83XFv2iVIExEgGBRGKMLRJ5acmcOyWTkhnZuJONxrJA10ZNSwcnG41lxWtbO9HayGF7ekL1rolYZEtOxr3M+N3oeP99fNXV0GMMEm75y0tos/S2ECK+hDNrY6HW+sdKqUeBEowFvCqA7VprKUFogfsvuZ+d1Tu5+8278XZ7ae1qje7sjYEtElOKwZECrVXQ0QwpGaPajc2meP5r/fvOc9xJKAV13i6O1RsDLrv9mqb2bn72+kc8t+skr3zjcjxumQUQDyb98Ae079yJPduDsyAfR14eFdddT1dFBe3vvYdr0SKrQxRCjFE4YyQ2AHPNpOFV80dYKC0pjcumXkZWchbebi8tXS3RTSSS3DDvajj4khlQIXhmG9NB6w7BlNF/ONhs/fvJHXYbOe4kalu7eO9EX55a09LJY28ZXRx/2Xuav10yY/znIc44R3Y26Vde2W+be9kyuioqaH3rLZxTp+KrrsZXXU13VRW6vYOs1auwZ2ZaFLEQYiThJBI5Sqn7gXLgaa11c4Rjihil1GqgESgGGrXWj1oc0hmVkZTBSU7S3G3BJbnhN/DKd6DlNOTMhtw5RiLx9K3whT9D9sywd52XnkJtaxe7j/eVyq5u6ZvFkZYczq+xiBXuZUtp+N3vqHtkPXWPrB/0vL+xgfxvftOCyIQQoxHOYMtFWuv7zIqWi5VSX1ZKXRHpwMZLKZUFrNNab9ZaPwDco5RK6KH+6UlGwabmTgsSCUcyXPNjuPF3YLND4bnG9qbj8Nq/j2vXgZkbJxrae7eV1/RNc3XYjF/jwLgJEV/cS5bgnDwZAHtODslnn0Xa5ZeTtsJouWh9a4uV4QkhRhBOiezDQfdfBV5VSmWahag8wOZYqGyptW5USg1sU6+3JJgoyUgyxiJYMnNjoCVfgdZqeOdROPb2uHaVnz64guL7Qd0c7d1+3jlcz62/fIf/++lzuGHx6ItgCevZ3G5mb3oF/H5UUMXLns5ODl64hM79+/HV1uLIzR1mL0IIq0SqTfhK4CsYgy/XA/dFaL/jorVuBDBbInYGHieqQItETCQSyelw9Y9g95NGq0TzaciYFNau8kIkEsHjJdq7fNz0aBk9Gu599j1JJOKQstvBbu+3zZacjGvRIrylpdT89L+xe7LxVVXjq6rCV12FcrmY9sgjOLItWF9GCNErnFkbn9VaP6uUWoiRPNwAvAus11p/fIz7Kgbu01qvCfHcWvpaEIrM7okxU0qtwFhg7J5w3h9PAonEkeYjdPu7cdotXjDJZoepi6Dir3DiHTj7M2HtpjAzZdC2A1V9ydKx+jYCs0CzXDJ7I5G4L16Gt7SUxqeeCvl8y+bNZK8Z9OdDCBFF4bRIbFRKlWN0Y9wPzBrrtE8zgbjRfDho3IKZRKC13mg+LlJKrdda324+vhvIGWL3m7TWmwMPzPublVLlSqmVWuuKscQaTwKJxK8++BW17bXcf2kMrOg+9UIjkXjtBzB9GaTljfiWga49dxI/fPFDOrp7SLLb6BqwpPiGHSd677d2+ujp0YNmf4j4lLV6NZ0VFSibDUdefm/J7fbdu6l7ZD1tZWWSSAhhMTXWAWpmIarVkSg8ZSYUj2mtFw3YviPEtnKt9ewx7rsoKBnZAFRorcNqmSgpKdHbt28f+YUW+uXeX/KTHX2lid+/9X0LozGVvwa/vd64X3gefOWtsHbT1N7N41sOU5CRzLef2zvsa7feewVTslLDOo6ID11HjlB+9SewZ2czd+sWlM1GT1dX79RR55QpOAsKrA5TiLhmfhaXjPS6cFok7j+T1SvN2RbFIZ5qVEqtCG5tGEER/RcQy8LogklY09L7xgY4bA601tavXzBrOVx1P7x8H1S+B946cA/VmDS0zFQn/7xyHlXNHSMmEkdqvZJIJDjnjBk4Jk/Cd+o0FZ/+NP66evwNDb3P23Nzmfvaq/0GbwohzoxwSmT/eLjnIzAVtAij9sNA9YROMEIyWyLqlVJrlVLrMAZbbhxnbDFt+bTlfGeJsWaBr8dnzUqgA9lssPSrMOMS4/HxbePaXY47KeQsDuirJ3G4VpYcT3RKKdI/Zvyp6fqo3Egi7HYckyZhc7nw19bS/t57FkcpxMQQTh0JoHeRroE/VwDrxhmTh9DTNBsZelxESFrrjVrrR7XW94TTpaGUylFKzVNKzfP5orx+RRicNic3LriRGRlGlceathqLIwoy/SLj9ljZ8K8bgcNu43++uJiZQQt8BfzNFGP669E6SSQmgvw7v8m0x3/BzA0bmPPmGyx4bw9zX3+NzNWrAPCWju93TQgxOuHM2liFUSa7nNDLiM8ab1Ax5A7guwDV1dUWhzJ6+a58jjYf5Yn9T1CcX8w1RddYHVJfIjHOFgmAcyZn8uo3l/PyB5V89fc7e7efOyWTtyvqOVzbNu5jiNhnS00l7eKLB213L11Kw29+S+ubb5J22aV0VxrTRbsrq/DX15N5/fW4l1xoQcRCJKZwxkgs1loP2ZKhlHpkHPEEeEJsywLqIrDvsfgp8ARAfn7+gSgfO2z5rnwAnjrwFE8deIqVM1fitFk8FXTqYuP21C7wd8M4p6babYosV/99LCg0WiRON7WHeouYIFyLLwS7nY69ezly082Dnu/Yt4+iPz1vQWRCJKZwEomRBiyOt17DdoykYSAPsDPE9jNGa12nlFoOrGppiYEiT6MUSCQCOnwdOJMsTiRSs4z1NhqOQF055C8Y9y5dSf1/fecVGNNfK5s6xr1vEb/saW6y//ZztLz0Mo68PByFhcZKo/kF1D7yCJ0HD+KrqcGRN/apyEKIwc7EakezgN3hvtksbV2hlMoaUIkyawwzNiJGa/0M8ExJScngrzYxKj+1fyLR6e8knXSLogmSf7aRSFTvi0gikersXwlxZq4Lu01R5+2io9tPyoDnxcRR+K1vUfitbw3a3rZ9O94tW/C+vY3MT33SgsiESDzhDLbcpJS6whxcmRH8hPl4LOWxQ3VhgDFgs3c/Zk2IqCcR5rFXKaWeaAiaWhbrBrZItPtipKk//yzjtvrDiOzOldSXKNhtirRkBwXmjI7q5s6h3iYmMPfSpQDU/+pXnLr3Po7ddhvl136S8ms/SdfRoxZHJ0R8CieRaMT4UN+JUdvBH/gxn1s90g7MSpXrMBKGYqXU+kA1SwBzue9ypdQKcynwFYGqltGmtX5Ga/257Diq5x+qayMm5J9t3Fbvi8juUoMSibRkB0qp3nLaMk5ChOK+xJiG3PHBBzT98Y94S8voKi+nq7ycpudl3IQQ4Qina2MncOVQZbGVUk+PtAOzTPWwYynMZEKEodBd2O9x7CQSZ65FIj3F+FWelJkKNFLZHCPnLGJKyvx5TPrBv9N9uhJHQT7OggK6Tpyg6vv/hndrKXn/5/9YHaIQcSecFokvj7C2RkItjhWPXRuF7kIevvJhUh1GdccOf4x8qObMBZsD6svh5xdD+/j+TVMcfYmE2xx42dciESPnLGJO1qpV5H39a2SvWUPaZZeR+enPgMNB+/vv429uBqCnrY2uI0fwvvMOnYcPWxyxELFtzC0SWutd0DseYgWwGKOmxGat9RGtdUL9r4vHwZYAl069lOKCYrae3Bo7YyQcSTD7Sjj0MlTthUOb4bzwF1wKXpjL6TDuTzITCZm5IUbLnuYmdeH5tG/fQcV119HT0kpP0CwtlZLCnM2bcOTmWhilELErrMqWSqkvA0eAbwGLMJYTL1dKfTNyoYnxSrWbLRKx0rUBcPMf4EJzOMyJdyK2W7sKJBLGOcsYCTEW6StWAOA7dZqelhaU04lz6lTsWVnojg68ZVIlU4ihjDmRMCtbZmmtPVrrEq31x81bO5CnlPps5MMU4UhxGN/OO/0xNIPBZoMF5rS7E5FbQ81utk4EujZe/qCKf3467FnIYoLx/N3fMePJJ5j1/B+ZW1bK/Pf2MGfzJnLWGkmvd2upxREKEbvCGWzpGWrhLq31vUqpu8YZU0wxE6dVRUVFVocyZsl2YypkzHRtBEwpBmWDyvehux2c41+pM5BIzC9MpyAjmarmTp7deZIHVp2Hwx72kjJiglAOB64LLhi03X3xMgC8paV433kHX3UNviqj5Lavto70lSvIuPrqaIcrREwJJ5EYqUx1/IxKHIV4HSMB9A22jKWuDYDkdGMqaNVeo2T2jGXj3mUgkUhLdrD1nis4+19fpsvfQ31bF/npKePev5iYkufOxZ6Tg6+6mmO33Dro+dYtW0hfuRJll+JnYuIK56vaUEWkAuKn4EKCC3RtxMysjWBTzBXhT0dmqWeHre9X2WG3MTPXWB20rrUrIvsXE5Oy2ci9fS3OGdNJveAC0q++muxbPk/+XXfiKCykp6mJjn2RqYsiRLwKp0XisFLqfuCHWuveoc3mLI51GCuDihiQYjcTiVhrkYC+4lQ1kakpYbf1X4g2Ny2Zg1Wt1LZ20uDtYvfxRi6fl9dvpocQo+G55RY8t9wyaHvX0WM0Pv003tIyUs89t3e71hrd2YktRVrCxMQQzvTPV5VS2cBRpVRwN0cRcLvW+rWIRRcD4nmMRKBFIubGSADkmWttVO+PyO6mZPcfZ5GTZowPqWvt4sZHyzhY1cpDa85n1aKpETmeEO5lS2l8+mkaN26k88B+uquq8VUbP7qzk4J//Rc8n/uc1WEKccaFNQpNa71Ra+3BmPb5gHnr0Vr/IpLBxYJ4LJEdEBgjEVOzNgICVS5rPgStw97N07cv5TMLJ3P3VfP7bc9xJwFQ29rJwapWAMoqor0KvUhkriVLwOGg+/hxml/8C+07dtB9/Di60/j/jBo/rAAAIABJREFU1vRHKbktJoZhWySUUlcCmQM2b9ZaN0Nv60R9oEiViC29YyRisWsjrQBSMqGjCVoqIWNSWLu5cJaHC2cNHraTm2YkEnXevjES2S6Ll1IXCcWRnc20h39Gx4EDOAsKcOQX4CjIx56RwaHlH6Nj7178TU3YMwf+CRUisYzUtVGPUfJ6BbAe2BBIIoI0mgWqdCK2SMSzwPTPmBxsqRTknQXH3zZaJcJMJIYS6No41djXrZOeIomEiKy0yy4j7bLLBm13LVxI27vv4n17G+4lF5rdHlX4qqpAKTKvu05meoiEMWwiobXepZTagDH2IWTpa3P7YwBKqTu11g9GPkwRjkDXRkyOkQDIX2AkEid2wOwrIrrrQNfG+yf7loXp9PkjegwhhuJetpS2d9/l5D/9U8iuO5WUTOanPmlBZEJE3rBjJMwqlZvGsH7GY0qpL40/rNgRj4t2BcT0rA2AeZ8wbkt/Cq3VEd11oEWiosbbu62tSxIJER3pV12FcjpBa2xuN0lFRbiWXkTq+ecD4N3ylsURChE5I3VtLNZaPzvanWmtm5RSWeOMKabEc0GqmB4jATDvKpizEj7aBGX/DSu/H7FdB8ZIBGvrlERCREdyURFzzfU57Gnu3u0dBw9y+NOfwVtahtYapWQ6soh/4dSRGEnOGdinCENMF6QCY5xE8S1GIlFzMKK7DrRIBPN2+SJ6DCGGE5xABCTPnYs9LxdfTQ31jz+O7u6mu6oKX1U1vqoqkmbOZPKPH5DxEyKujJRIhFM8If4KLiSomFz9c6CMycZty6m+bV1e8HdBavhTbt1JdlKcNjq6e3q3tUvXhrCYUgr30qU0/+kFqh98aNDzHfv24bnl86QuXGhBdEKEZ6REQimlZmqtj4xmZ0qpWYC01cWImG+RAEg3Z2u0VBq3WsOvPw0Nh+GOnZAaXk+ZUoqlRTm8fqCmd1ugReJkYzs57iRSnPKtT0Rfzpe+RE9bGzaXq9+00eYX/0LLSy/hLSuTRELElZESiR9hTPu8apT7ewRjuqiIAckOc/pnLLdIpBUYK4G2VoO/G2oOwMntxnNHt8KCa8Pe9c/+tpgHXjrAhu3H8Xb5ae/ys+tYA9c/XMpNi6fxo1XnRegkhBi9lHnzmPbf/x3yuZaXXsJbWkbuP/wDANrnw1dbi6+mlqRZs0J2lwhhtWFnbWitdwKvKqVeVkrNGOp1SqmZSqmXMYpV7Y50kFaK51kbcdG1YXeAOx/Q0FoF+4KqAR4rG9euXUkOvvfpc3juaxcD4O3ys/6NCgD+8O7xce1biEhzL1kCNhttu3ZxePUaDl16GfvPO5+Pln+MI2vWcOKrX7U6RCFCGrFEttb6AWA3xmJd7yql7ldK3Wn+3K+UehcoB3ZqrX98pgOOtnguke2wObArOz7to7un2+pwhpZeaNw2n+6fSBwdXyIR4EoyujDaOn1UNvclVd5OGXwpYoc9MxNXcTH4fHTs3Yuvpga0xp6XC3Y7be++iy8Ov9CIxDeqWRta63uUUk8B9wG3A4GO60ZgM7BWymTHHqUUKY4UvN1eOnwdOJNitLJjxmQ4vRvqK6D2QN/207uNgZdJ42vOdSUZv+Z13i5qWvvWHTlQ1ULx9PhLEEXimvKT/6B9zx7sOTk4Cwtx5OainE6O3voF2rZto23bNjKuvtrqMIXoZ9TTP81ujjVnMBZxBqTYjUSi099JOulWhxNaYMBljbkSqDu/L7k4WgpzV45r94EWiU5fT7/t+09LIiFiiyMvj/QVKwZtdy9bRtu2bbS8+hrOyZP7TRn1t7aQc+utJM2cGf2AheDM1JEQMSRQJrulq4Xc1FyLoxlCxoBEIiXDKFZ1ejfs//O4E4lkhw27TeHv6V+q+EDlwGVjhIhN7mXLqPnJT2h+4QWaX3hh0PM9zS1MeUhWJxDWCGsZcRE/pqVPA+Bo81GLIxlGullLIpBIJGfAAnMdgv0vQk9P6PeNklIKV9BUz0vnGgnVh6dbxrVfIaIl5eyzcC9bht3jIfnss0hbvpysm27E8/e3AeAtK0OP8/+JEOGSFokENyd7DmWnyzjUcIiSghLSktKsDmmwwGDLemNGBSkZUHguZE2HxmPw/Wy47WWYflHYh3Al22kxB1dePi+Ptw7V8lFN63gjFyIqlN3O9F8+Pmi71prmP/8vvqoqOg8dImX+/EHPSxlucaZJi0SCm5s1F4D/2vVfLH1yKXtq9lgcUQjuAV0uyRlG+ezFQeu/7f/zuA4RGHAJsHBaFq4kO/XeLhrbusa1XyGsFKiUCVD94EOcuudejn7xi5Rfcy0HFpVwcMlFdFZUWBylSHSSSCS4OVlz+j1+oXxw/6rlUj39H6dkGLcX/yN8+qfG/YYj4zpEYMAlwKSsVGblGjNBKmq9Q71FiLjgvtisk/LWWzQ9/zxtZW/TVVFBj9dLT3MzzS/+xeIIRaKTro0RKKVWAauKiuJzCZHZWbP7PZ6SNsWiSIbhGrDOW0pQWeyCc4zbcSYSSY6+nDk/PZlZuW4+ONXM4RqvzNwQcS3jE1fTffIk2ufDWViAo8D46TxwgFN33Y23tJS8r3/N6jBFApNEYgTxvIw4gMvp6vc4JtfdSHKBIwUCFTiTM/qey55l3DYcNdbhCLO/t7Wjr/iU026jKM8YK1JRK+MkRHxTDge5X7l90HbnpElgt9O+Zw/+1laUUnRXVeOrqqS7qoqk6TNwFV9gQcQi0UgiMQHcNP8m/nDgDwC0+9otjmYIrhxoPmncTwlKJFKzjcSisxnaG8DlCf3+ETR39K/sWRTo2qiRrg2RmOzp6aSeey7tu3dz6NLL0O0D/u87ncz96+s4cnJC70CIUZIxEhPAvRfeyx0X3AHE8LobweMkglsklIJsc5mXhsNh776pfUAikWckEodljIRIYBmfMKpg6vZ2VFISzmnTcJWU4CgshO5uvGVvWxyhSATSIjEB2G323mJUMZtIBLc0BLdIAGTPhMr3jXESUxaFtfuObmOOvc3sGZlptkgcqfPKFDmRsLJvuYW0K6/E5nZjz8rq/T2ve/xxqn/8IN6yUjI/Gf4Ku0KAJBITRoo9BYiXRCKz/3PZM43bcQy4vHXpDH5ddpR/WG4MPs1IcZLitNHR3YO3y09asoN9p5r5j00H+O6nzmGaxzXCHoWIfUopkqZOHbTdvWwZAN4tW2n605/orqzCV1VFd1Ul/to6Mq+7juybbox2uCJOTYhEQilVBKw2VzKdkFIcRiLR7o/hMRIBySFaJADqw+/a+Pa1Z/PJ8ydzwbS+GSE57mRONrZT39pFWrKDT/33lt4y2r+4dXHYxxIi1iXPn489OxtfVRWn7r5n0PNdR46QdcMalE16v8XIJkQiAdyDsVLphNWbSMTqYMvU4bo2AjM3joS9+ySHjcUz+w/UzE1L4mRjO7XeTgozU3qTiIa2GF5yXYgIUDYb+Xd+k6bn/4QjNwdHQSGOgnychYVUrXsA3+nTdHz4IannnGN1qCIOJHwioZRaAewAZo/02kTmchhN9bHbtRHcIjGga8NjJhLjaJEIJSctGYC61i7ePVLfu73bL2sWiMSXtWoVWatWDdreumULTRufwVta2ptIaL8fX20dPV4vSTNnSEuF6CfhEwlTPRM8kQi0SMRsIpEaVBRqYItE5nSwOYzpod0d4EyJyCE97iQA6lo7ebu6r57E4VoZgCkmLvfSpTRtfIaG3/6O1s2vGkuW19SA3w9A/l13kWMuFiYEWDz9UylVrJTaMMRza5VSq82fu8Pc/wqt9ebxRZkYegdbxmJBKuifHNid/Z+zOyBzGqChMXKrmOakmYmEt4tDQYlES4dPujfEhOVeuhSVlISvupr2PXvwVVaC348tPR2A5ldetjhCEWssaZFQShUDgSHBg2pPK6XWAmitN5qPi5RS67XWt5uP7waGqqKySWu92RxgKavVmGJ+jITNOfzzniKjjkR9BeTNH/61o5Tr7uvaqG7un2AdqfP2tlgIMZE4PB5m/P53dB07hrOw0Bg/kZ8H3d0cWHIRHe/vxd/cjD0jY+SdiQnBkkRCa70T2GkmFCtCvOR2rfWioNdXmGMdAo9HM/uiGPCYzdOLgKKJ3EKR6kgFYjiRmHahcZt/dujnPbOgnIiOk+hrkeikpqUTgAtneXjncD1HamUNDjFxpZ57Lqnnntt/Y1ISqQvPp337Dlo2bSZ5/vzectu+yqreUt0qSRLwiSbmxkgopbIwkoCBGseSCARaM8x9rgUaJ2oSAX2JRMyOkXDnwt2HISkt9PMes+GqPnKNTIEWh8qmDuq8XdgULJqR3ZtICCH6cy9dSvv2HZz+9rdDPp80YzqZn/lMlKMSVovFobdFhJ6qWU/oBGNYZqvHGmBFcKvGROO0ObEpG9093fh6fCO/wQouDziG+DYTSCRqD0bscLnmrI2DVS3GIdzJvcuLH6tvi9hxhEgUmddeiz0nB3t2NskLFpB2+eVk3XADaR/7GADe0jKLIxRWiLkWCcCDkTQM1MjQ4yKGZHajrAwnEKVUTuCY559/fji7iBlKKVLsKbT52uj0d+KwxeKlH8akhcbtyR3g9xkDMMcp0LURGFiZn57MtGxjmuyJhhjtAhLCQkkzZzJv65ZB2zsOHqT19dfxlpbKjKcJKM4+TaLuDuC7ANXV1RaHMn4pDiORaPe143a6rQ5nbDImGa0S9RVQ+R5MGXPj1CADB1PmZyQzzWN0AR1vkBYJIUYree5c7Lm5+GpqqPrBD+nxes3xE9X4KitJnj+fGb/9DcputzpUcQbEYtcGGK0SA2UBdVGO46fAfGB+fn5+lA8decEDLlu7WunRcVZ4abqxPgBHSyOyu2SHnfSUvlw6Pz2ZwowU7DZFVXMnHd3+iBxHiESnlMK9bCkADb/7HU3PPYe3tIyu8nJ6vF7ad+6kY+9ei6MUZ0osJhLbMZKGgTzAzmgGorWuA84FvtfS0hLNQ58RgUTiYMNBlj65lLveuMviiMZohplIvPJteC9k+ZExO3dKXxXNvPRkHHYbk7OMqbInG6V7Q4jRyrvjDjy33kru179O4b99n2mPPcqs558n8/rrAfCWyfiJRBVziYTWuhGoMGdvBMuyYtaF1voZrfXnsrPjfypgoCjV0weeBuCVo69YGc7YFV1uVLgEeP6rxliJcbpiQV9LU3668e8zNUvGSQgxVknTplFw373kff1rZK9ZQ9qll5Iyfx5py5cD4N1qtCT2eL10VlTgLSuj6YU/0336tIVRi0iwOpEI1YUBsA64L/DAnHkxYaduRkqgKFVDR4PFkYQpcyqsfcO47++CllPj3uXy+X2JRGAWR2CcxK2/fIdH3ywf9zGEmMjcFy0BpWjbsYMDJYs5sKiEimuu5dgXb+PUXXdx8hv/bHWIYpwsSSTMSpXrMBKGYqXU+kA1SwCt9aNAuVJqhVJqNbAiUNXSglhXKaWeaGiI0w/fIIFEoq492kNNIqjwb2DaEuN+47Fx7252Xt+g08Dgy8DMDYAfvrif5g4ply1EuOyZmbiXLoWeHnpaW1FJSTinT8dVUgJOp1GGOwH+vk5kVlW2rMBY2nu41zwapXCGpbV+BnimpKTkZqtjGa/AGIn6jlCza+NI1nQ4vi0iiYRSimf+YSm7jjVyUZHRQFaY2X9RsL+8f5obF08f97GEmKimPvwzuo4dw5GXhz0rq3d66NFbv0Dbtm20bdtGxtVXWxylCJdM/5xAAomET8doQarRyjI/1Bsis4DXohkeFs3o62W7YkE+503NRGt4/2QTz+48KYmEEONgS0khZd68Qdvdy5bRtm0bTS/8Gd3Z2TtdtLu6ih6vl/xv3knq35xjQcRiLCSRGIFSahWwqqho0NpicScw2DLuZc0wbge2SLRWgysHbOObq56Tlsyfvn4JLR3dlPz7ZrYdrqeyqWNQS4UQYnzcy5ZS85Of0Prqq7S++uqg5+v/53+Y8tCDFkQmxsLqwZYxL6FmbTgS5IMw0CIRnEjs/194cC6U/lfEDpOe4uRj5mDMF9+XkeVCRFrKOeeQtWY1qSWLyLj2Wjy33UbBt+6j8Lv/ChhTRnVPnNW7mYCkRWICSehE4oV/NG43fw8u+UbEDnXteZN46YNK/vzeKW67ZBYAnT4/z+86xYqzC2SpcSHGQdlsTPq3fxu0XWtN7fpH8VVW0nnwICkLFhjbfT58tbUoux1HXl60wxVDkERiBInUteFJGTzbtkf3YFNx1jCVORVQ0Hyyb90Nb80ZOdSVZ+WT6rSz81gjx+vbmOZxcffG93h+9yluPj6N+z973hk5rhATmVIK99KlND33HKfu+xbK4cBXVYWvthZ6esBmY9Zzz5Iyf77VoQqka2NEidS1MStj1qBt3T1xOLXRkQxZ00D74eR26AkqZZ0a2evkSnKw8uwCAJ7ffZITDW08v9uoX/HkO8cjeiwhRJ+0yy8DoPPDD+l4/3181dWgNSo5GXp6aHllk8URigBpkZhAZmUOTiQ6/Z0k25MtiGaczvksbP1P2PlbSOkrcx2JapcDXV88hT/tOcWzu07S6evrr7UpaO30kZYs/42EiLT0j3+cKf/5EwAc+QU4Cwtw5OXR+tZbnPjq1/CWlZF3x9ctjlKAJBITSoG7YNC2Ln+XBZFEQPEtRiLxwbMweWHf9q4W6PJCUuRWN710Ti65aUlU1Hh581Bt7/YeDe+daGTZ7NyIHUsIYVA2W8jaEq4LLwS7nfY9e+isOExPSzPdVVX4qqrxVVeRvGABmddea0HEE5ckEiNIpDESNmUjyZZEV09f8tDtj8OuDYCc2TD5Aji1C3b/vv9zrdXgGdz6Ei6H3caCwgy2fFTLh6eaAZiSlcrJxnZ2HZNEQohosqelkXreebTv2kXFNdcMfoFSuJcswZEr/y+jRcZIjCCRxkgAZKf0P4/gpCLuZJvJQuWA5YnPwMDL3DRjdkaX3+jaWHGWMS109/HGiB9LCDG8rDVrUMnJ2HNzSTn7bNKuuIKsm28iafZs0Brv29usDnFCkRaJCcaT4qGqrar3cdx2bQCkTzJuAwNGnW7o9kJr1dDvCVNOWv9xJJfMzePXZUfZX9kc8WMJIYaX9dnrybz+ut5S2wF1jz9O9Y8fxFtaSuYnpXsjWiSRmGC+WfJNvvTKl3ofx3ciMWDMx6Tz4VjpGUkkcgckEiUzskmy2zhe3y4DLoWwwMAkAoyS2wCtb75J9UMP0V1ZZUwbrarCV1+P5/OfJ+//3BHtUBOedG1MMEsmLWHLTVs4L8+ofxDXXRuBFomASecbt63VET9UoGsDwGlXZLmczMlPA+BAZUvEjyeEGLvk+fOxezz4a2upe+wXNL/wAm3vvEPX0aP0tLTQ8Pvfo/3+kXckxkS+Ro0gkQZbBmQmZ/ZO+YzvFonCvvtOtzEAE85QItHXIuFxJ6GUYkFhOvtON3OgsoVFMxJjDI0Q8UzZbExet47W11/HkZ+HI78AR0E+zsJCjq+9ne6TJ+nY9yGp5/6N1aEmFEkkRpBIy4gHS7IFBg/GcyIR1CKRXghpZlfHGe7a8LiN+wsmpcMu2HO8kZsvnBayqVUIEV1pl15C2qWXDNruXraMxg0b8JaVkfI359DT3DdttKe9jbTLLsOWkiDLCESZJBITlNPuBOK9ayOoRSJ9Ul9i0Xwq4ofKTe/r2sgx19eYX5gBwFPbj1Pd0sH/fPHCiB9XCBEZ7mVLadywgdqf/5zan/8c3d7e7/ncO75O3te+ZlF08U3GSExQCdEikZxudGmAkVRkTjHuN5+M+KGCF+cK3F80I5t5BcY4idcP1NDSEac1OYSYANxLl2JLS0O3t6Pb27G5XCTNmkXKOecA0PrXNyyOMH5Ji8QElWRPgEQCjASivryva8NmLuDl6zTW5IiQZIedjBQHzR2+3kQiLdnBK9+4nE/8v7f48HQzh6pbKZ4uYyWEiEX2rCxmv/QX/E1NOAoKsKcZXwJ62to4sOQiOvbuxd/UhD0zc4Q9iYEkkZigegdbxnPXBhjdGfXlkDEZbHbjcdNxo1XCE9kBsrnpyTR3+Hq7NgLm5qfx4elmPqpq5Widl19tPcIvbl1MXnocrmEiRAJz5OYOqnhpc7lwLVxI27vvUv+b3+KcVNhXcruqCpWayuQf/Ds2d+TK7icaSSRGkIizNgCcNnOMRLy3SOTOgaNbIGeu8ThjipFINJ2BRMKdTEWNF09a/0Qi0L1xsKqFX2w5DMDvtx3ln1bMi+jxhRBnhvviZbS9+y61P/tZyOebl19O1nXXRTmq+CFjJEaQaCWyAwJdG3G71kbAiu/B3z4Dc1YYjwPjJJ5dC3/9UUQPNSvX+EYyw9P/m8ncgnQAntvVNzYjeJVQIURsy7z+s7iWLMG19CIyP/MZctaupeBfvkPWDTcA4C0ttTjC2CYtEhNU7xiJeO/aSM2GuSv6HmdMNm5bTsFf74eij8H0JRE51H3XLOCT509i2eycftvnmYlEnbfv3/JkQ/8R4UKI2OUsyGfGr381aHvnoUM0Pv003rIytNa9U7x7urrwVdfg8GRjc7miHG3skURiggrM2uj0d1ocSYRlTO3/+I0fweefi8ius1xJXDo3b9D26R4XKU4bHd19rRDHG9oickwhhHWS5szBkZeHr6aGY7fdhr+hEV9VFf6GBgCcU6Yw++WXUI6J/VEqXRsTVMJ0bQwU6NoIKH8NvLVn9JB2m+LfrzuXz14whX9YblTXPF4viYQQ8U4phfvyywBoK3ubzv37jSTCbgeHg+6TJ2l//32Lo7TexE6jJrCE6doYKC1oIa/M6dB0zJgO6s4d+j0RsHrRVFYvmkpPj+bxtw5T29qFt9OHWxbzEiKuFdx5J65FJdgz0ntLbjtycqj6wQ9oeOJJvGVluC64wOowLSV/5SaohChIFcqURXDeTTD5AvjgOSORaKuP2uFtNsVUTyoVNV6ON7SxwKx+KYSIT/asLLKuHzxjw7V0qZFIvLWFzE9+El91de+0UX9DPelXX02qWewq0UkiMUElTEGqgWx2+Ox64/5hs1Jde/QSCTDGTFTUeDlWJ4mEEInKvWQJ2Gy079pF+cevGvR869atFD37rAWRRZ+MkZigetfaSLREIliqOWW3vSGqh53uMUZxV9R62byvirrWBBvQKoTAnpFB5qc+hXI6cUyeROrChaRfdRXZn/88KjmZzn0f4quP7pcYq0iLxAgStSBVwlS2HE4gkYhi1wbA+VOzgKM8+mYF9d4uZua42PzPl+OwS94uRCKZvO5HTPrR/YNW/u0q/whvaRnesjIyr73WouiiRxKJEcgy4nGst0UiuonEhbM8ANSbdSWO1LXx67Kj/P0ls6IahxDizBuYRICxZLm3tIyG3/2e9j17esttd1dXods7mPLQg7iXLbMg2jNDviJNUAmxjPhIXMYHerS7NqZmp1KYkdJv2++3HY1qDEII6wSShPZdu2j4zW9pefll2nfvxnfqNP6GBho3brQ4wsiSFokJKtAikXB1JIKlmolElLs2lFJcOMvDn/acYkpWKpXNHRyp9dLR7SfFaY9qLEKI6Es+6ywK7ruX7lOnzCmjBTgL8tF+P8e+8EW8ZW+je3pQtsT4Li+JxASVsLM2glk02BJgxdkF/GnPKdaUTOV/3zvNoepWDlW1cu5UWaJYiESnlMJz662DtmutcUyehO/UaTo+/JCUBQvw1dYa3R5VVfgqq0Brsm++CeV0WhB5eCSRmKACiURnTwLPKLCoawPgU+dNYm5+GvMK0vmoupVD1a3sr2yWREKICUwphXvZMpo2PsOxW79AT3s7+P2DX5eaQvaaNRZEGJ7EaFcZglJqk1JKK6UalFJ3Wx1PLEnYEtnBLOraAOMPxlmTMrDbFAsKjUW9DlS2RD0OIURsybjKqDnR09oKfj92j4fks88ibflyXBddBIB3y1YrQxyzRG+RWA+s0Vo3Wh1IrJlwsza0hhCjq6NhvlmUar8kEkJMeGmXXsrsl18Cux1Hfj62pKTe57qOHaP841fR9vbbaL8fZY+PMVWJnkggSURoKY4UPCkeslKyrA7lzElygSMFfB3Q3QZJbkvCCLRIvHeikaa2bjJd8dP3KYSIvKQZM0Jud06bhnPKFLpPnqT2Zz9D+3vwVVaa4ycqcU6dyrSHf4YKSj5iQaInEkVKqRVAFuDRWj9qdUCxIjc1lzdufMPqMM68VA+0nDK6NyxKJKZmp7J4ZjbvHmngX57fy3/dPLEX+BFChBYYQ9G4YQO1D/980PNdR47QtnMX7ouWWBDd0CxNJJRSxcB9WutBo0qUUmuBQOd2kdb6gTAO8WigRcIcL7FZa10RfsQi7qRmG4lEez1kTbMkBKUUD645n0/8v7f4055TfOHimRRPz7YkFiFEbMv5+9voaWvD5nLhKCzAWViIo6CQ5hdfpOnZZ/GWlUkiAb0JxI3mw0G1p80kAq31RvNxkVJqvdb6dvPx3UDOELvfpLXebL4/uFujAlgBSKvEROI2f028tZaGMSPHzReWzeThv5bzs9c+4vEvLLY0HiFEbEqaOZMpDz04+AndYyQSpaXwjX9Ca01Payu+ykp8NTUkz5+PI2eoj8Uzy5JEQmu9E9hpJhQrQrzkdq31oqDXV5hdFIHHI7ZOmK9fE0g+xATlzjNuLU4kAP7+kln8cuthXt1fzbG6NqbnuKwOSQgRJ1yLFoHTScfevZRfcy2+ykp62tp6n0855xxmPWNNxcyYm/6plMoCikM81RicTIxCBbAh6HEJ8PR4YhNxyJ1v3HqrrY0DyElL5qIi4xvDvtPNFkcjhIgnNpcL97KloDVdFRX0tLWhUlNJmjkT5XTS8cEHdFdWWhJbLA62LAJCzbSox0gwNo9mJ2YrRrHZTZIF3CMzOCYgd65x662xNg5TUW4afz1QQ0Vtq9WhCCHizJQHHqBj3z7sOTk4CwuxpaejlOL47V+h9Y038JZf/EmxAAAMWklEQVS9Tdb110U9rlhMJDz0DbIM1sjQ4yJCCoyxCJdSKidwzPPPP388uxJWSTNbJFpjJJHIM2aOVNR4LY5ECBFv7JmZuJcuHbTdffEyM5EotSSRiLmujRhzB3AAOFBdbX3TuAhD7xiJWEskpEVCCBEZgeTCW1aG1jrqx4/VRMITYlsWUBflOH4KzAfm5+fnR/nQIiJiaIwEwOy8NAAqaqVFQggRGUlz5hgrjOYX4G+I/tpCsdi1sR0jaRjIA+yMZiBa6zql1HJgVUuLlDeOS71jJKyftQGQn56MO8lOY1s39d4uPO7YqlAnhIg/Silmv/wStpQUS44fcy0S5oDICnP2RrCsQH2IKMfzjNb6c9nZUkAoLgV3bVjQ5DeQUoqiQKuEdG8IISLEqiQCrE8kQnVhAKwD7gs8MOtNRD2JEAkgyQVJaeDvgo4mq6MBgsZJSPeGECIBWFXZsgi4HaMYVbFSaj2wI7AWhtb6UaXU2qB1MoqsKiyllFoFrCoqGlSAU8QLdx50tRrdG6nWL1J258fnc+8nFlCYYd03CCGEiBSrKltWAPeM8JqYKGWttX4GeKakpORmq2MRYXLnQcNhY8Bl7hyro2GaRypaCiESh9VdG0KceYFaEjEyBVQIIRJJLM7aiCnStZEAphSDrxNSrO/WEEKIRKOsKF4Rj0pKSvT27dutDkMIIYSICqXUDq11yUivk64NIYQQQoRNEokRKKVWKaWeaLCgWpgQQggR6ySRGIEUpBJCCCGGJomEEEIIIcImiYQQQgghwiaJxAhkjIQQQggxNEkkRiBjJIQQQoihSSIhhBBCiLBJIiGEEEKIsEkiIYQQQoiwSSIxAhlsKYQQQgxNEokRyGBLIYQQYmiSSAghhBAibLL65ygppWqAoxHanR0oAKoAf4T2GUsS/fwg8c9Rzi++yfnFv1g4xxla67yRXiSJhAWUUvOAA8B8rfVBq+OJtEQ/P0j8c5Tzi29yfvEvns5RujaEEEIIETZJJIQQQggRNkkkrFEH/F/zNhEl+vlB4p+jnF98k/OLf3FzjjJGQgghhBBhkxYJIYQQQoRNEgkhhBBChE0SCSGEEEKETRIJIYQQQoRNEgkhhBBChE0SCSGEEEKETRIJETalVJFSaoXVcYjxk2uZGOQ6xrd4vX4OqwOYaJRSa4F682GR1voBK+MZp2LgMaVUFtAIbAfu0VrvDLwg3s5XKVUM3Ke1XhPiuWHPJR7OdZjzS5hracYJsMi8vUdr3Tjg+bi9jiOcX9xfRzO+LPPhbGCd1rpiwPPxfv2GOr+4vH6SSERR4A+A1nqj+bhIKbVea327tZGFT2udrZTKCv5DHRBP52t+wN5oPiwK8fyw5xLr5zrS+UFiXEul1Fqt9aPBj4EdGH+wE+E6Dnt+EN/XUSl1d/AHo1JqNbCJxLl+w54fxOn101rLT5R+gB0htpVbHdc4zmd1op0vxjeCUHEPey7xcq7DnF/cX0uMb3lrQ2xvAFbE+3Uc5fnF9XUEyoPPASPp1UBWvF+/UZ5fXF4/GSMRJWZTVXGIpxrjsU9sJIl0viOdSyKdayhxdH5FwHoz3mAVQFECXMdhz2+kN8fB+QGs1Oa3bVMR0Ki1bkyA6wfDnN9Ib4zl85NEInqKMPq8Bqon9C9HXDD/Awd+7g76I5dI5zvSuSTEucb7tdRGP/KiEH+Ui+j7sI3b6ziK8wPi+zrqoLEQpnuAwHieuL5+MOL5AfF5/WSMRPR46BsgE6wRyIlyLJGyE/r+cyilKoANwEoS63xHOpdEONeEuJY6aFAa9PZBV2itN5vf2uL6Og53fuamhLiO5nmtxBiIGDi3hPl/OMT5QZxeP2mREGHTWlcEZ9jm/SJzYJ+II4l4Lc1vcvcBV1ody5kQ6vwS5TpqrTdqYwBhsVJqvdXxRNpQ5xev108SiejyhNiWRRysNz8GjUCJeT+Rznekc0mkcw2I92u5DlgzoCsgka5jqPMLJW6vozZmONwQNAYgka5fqPMLJeavnyQS0bOdvrnDwTyYzVnxxJx2pEM8VW/+JNL5jnQucX2uiXgtlVJ3M6D+AAl0HUOdX7xfR6VUsVKqIcRTFRhN+3F9/UY6v3i+fpJIRIn5raEixIjrrAF9ZPGiHgg1d7kE2JlI5zvSuSTAuSbUtTTn2m8c8CG7IlGu41DnR/xfRw/waIjtRRhTHOP9+g17fsTx9ZNEIrrWYfRpAr1FgmLhF3zMQjWnmn/gng76AxeP5xuq6RBGPpd4OddB55dI19L8QN0eNFgta0CzcVxfx+HOL96vY6gPw6CxAU+bt3F7/UY6v3i+fsosaCGixPzFqMBoooqJ8qbjYTaxNmI2uQ08n3g5X6VUEca3gRUYU6kexSj+MrCK4JDnEsvnOsrzi+traZ5j+RBPZwf+UMfrdRzD+cXtdTS/ba8N2jRUiey4u34w6vOLu+sniYQQQgghwiZdG0IIIYQImyQSQgghhAibJBJCCCGECJskEkIIIYQImyQSQgghhAibJBJCCDECs15DTK93IIRVJJEQIoqUUuuVUuVKKa2U2mCuAhjTlFKrlVJD1S9IeGYCsQHYYXUsA5lllTeZv08Jt7iViA9SR0KIKDMLyqzTWmeHeK544FLR0TLUsc0P0hu11vdYEFZMMAsJNWitldWxhGKu0bBGa73R6ljExCMtEkLEluFWAbTk2FrrnRM5iYDQ5adjRVCXi+WlksXEJImEELHlxgl6bBG+FUBFLCc7IrFJIiFEDDAH863DWAcjpo5tLn9sZUvJsEINhAxsC7FSYiJaibRGCAs5rA5ACAHADZiL9Jgf6mAsnRy8qFbwwMzFwFNa653mh+hjGMsRzzL3tRJYH1hxMMR77w/6Bjvksc0P4vuA1UC/8QFmclGEsfyxB6gP9NEHxVQMZNPXbbIYeHekvvzRvt9cyGpdcHxmzOswFkdaBOwcZn8rgU1a641B/0ZFMHixpKDjFQW9rj7UuYznWoVhBbAm6NwDi0LlTPQuKRElWmv5kR/5ieIPxh/6hhDbs4z/kiHfczewdsC2cozV/8D4YGoAVge9PnB/deB15uNijA/PUR3bfF4PeLwaY8DokDEG9gmsGG5fwxxz1O8fYlsDUDya/YXYVh783qDXrR6wbX2I6xL2tQrjd6nYjCvL/FkbFFe5Fb/f8jPxfqRFQogYF2gV0INneWzEWBr8Hq11hfm6Chj0bXolxviHNeZzO5VSK5RSWTr8fvXHML5R99JaP6CUalBKPa21btRaNyqlwGixCFYxmtkp433/wPcNtz/6WhkCdgIl5m3wPga2PtwD9J5zBK7VWK0I7Ae4Qfe1YElLhIgaSSSEiH0lMKi5HKAO6JcIDPHheg/gMZvVS+j7IPUMfP9omF0a9UMkIRUYH24bB2wbj/G+fzT7C+sYQclJCcY4hfFeq7Faad4GJxGMI0EUYswkkRAi9g3ZFz9KHoyCSvcHPmyUUuP5oBn47T1YPcaYgN5Yx/uhFukPxSH2N7CVYiyCWzTGe63GagXwALBIKbUD+HKEEhQhRk1mbQgRo4JmHVRgJAPh2oQxmC/UoMCQsxpGmPEw3Ld3D8Z4gImkiL5/k/Feq1ELmqlyv9b6duApjIRRiKiSREKIGBHim3LgW+526PfB0WukaZlmMlAEPD3gqUCSUDLEWInhWh22A0VDJBrFWDQVMUQ8w51DpI5ZhNFlsd3cFPa1CsPA+hG9SYw5/TXmy6+LxCCJhBCxpcL8cALwBAYtYoxzWBf8QvODafvAHQQL+pDp/ZZsvm8n5kj/4A+igcceZp+3h4hnHeZgwgHbQyUco/7WPsr37yQocTDPMWTrwDjiaQzx4bwOozuhEXr/bcK6VoHYzPVYRlP/IlT9iOCBl1IuW0SH1dNG5Ed+JtIP5rQ8jCl7Gxg8nbDYfM1qBk9JXEFffYTeKZ3mezYMs88Vge3m/eLgbcMdG+PDOXjfRQP2e7cZz90D9jXwfcUYict6c9uOgec3IOYxvd98PlBPYrX5eIf5b712LPsz99MQeG/QMe4297Mi6NxDnsM4rlW/qaEj/C7tYPAU1Q0Dr4X8yM+Z/pFFu4QQIsYopVZraVEQcUK6NoQQIvbI9E0RNySREEKIGGK2RsjaGSJuSCIhhBCxRVojRFyRMRJCCCGECJu0SAghhBAibJJICCGEECJskkgIIYQQImySSAghhBAibJJICCGEECJskkgIIYQQImz/H/xGwQAdO4ZxAAAAAElFTkSuQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plt.figure(figsize=(8, 6))\n", + "plt.semilogy([np.linalg.norm(grad_f(x)) for x in cg_fr.get_convergence()], label=r\"$\\|f'(x_k)\\|^{CG_{FR}}_2$ no restart\", linewidth=2)\n", + "plt.semilogy([np.linalg.norm(grad_f(x)) for x in cg_fr_rest.get_convergence()], label=r\"$\\|f'(x_k)\\|^{CG_{FR}}_2$ restart\", linewidth=2)\n", + "plt.semilogy([np.linalg.norm(grad_f(x)) for x in scopt_cg_array], label=r\"$\\|f'(x_k)\\|^{CG_{PR}}_2$\", linewidth=2)\n", + "\n", + "plt.semilogy([np.linalg.norm(grad_f(x)) for x in gd.get_convergence()], label=r\"$\\|f'(x_k)\\|^{G}_2$\", linewidth=2)\n", + "plt.legend(loc=\"best\", fontsize=16)\n", + "plt.xlabel(r\"Iteration number, $k$\", fontsize=20)\n", + "plt.ylabel(\"Convergence rate\", fontsize=20)\n", + "plt.xticks(fontsize=18)\n", + "_ = plt.yticks(fontsize=18)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "#### Running time" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "11 ms ± 1.79 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)\n", + "620 ms ± 5.31 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n", + "533 ms ± 3.54 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n", + "1.73 s ± 8.11 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n" + ] + } + ], + "source": [ + "%timeit scopt.minimize(f, x0, method=\"CG\", tol=tol, jac=grad_f, options={\"maxiter\": max_iter})\n", + "%timeit cg_fr.solve(x0, tol=tol, max_iter=max_iter)\n", + "%timeit cg_fr_rest.solve(x0, tol=tol, max_iter=max_iter)\n", + "%timeit gd.solve(x0, tol=tol, max_iter=max_iter)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## Recap\n", + "\n", + "1. Conjugate directions\n", + "2. Conjugate gradient method\n", + "3. Convergence\n", + "4. Experiments" + ] + } + ], + "metadata": { + "anaconda-cloud": {}, + "celltoolbar": "Slideshow", + "kernelspec": { + "display_name": "Python 3 (cvxpy)", + "language": "python", + "name": "cvxpy" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.4" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +}