Updated description of week5 assignment

week2/em_assignment.ipynb

@@ -3,7 +3,7 @@
   "nbformat_minor": 0,
   "metadata": {
     "colab": {
-      "name": "Copy_of_em_assignment (1).ipynb",
+      "name": "em_assignment.ipynb",
       "version": "0.3.2",
       "provenance": [],
       "collapsed_sections": []

week5/vae_assignment.ipynb

@@ -3,9 +3,10 @@
   "nbformat_minor": 0,
   "metadata": {
     "colab": {
-      "name": "vae_assignment.ipynb",
+      "name": "Vae_assignment.ipynb",
       "version": "0.3.2",
-      "provenance": []
+      "provenance": [],
+      "collapsed_sections": []
     },
     "kernelspec": {
       "name": "python3",
@@ -165,7 +166,7 @@
       },
       "cell_type": "markdown",
       "source": [
-        "### Variational Lower Bound"
+        "### Negative Variational Lower Bound"
       ]
     },
     {
@@ -177,11 +178,13 @@
       "source": [
         "**Task 1** Derive and implement Variational Lower Bound for the continuous mixture of Binomial distributions.\n",
         "\n",
-        "**Note** that to pass the test, your code should work with any mini-batch size.\n",
+        "**Note** that in lectures we discussed maximizing the VLB (which is typically a negative number), but in this assignment, for convenience, we will **minimize** the **negated** version of VLB (which will be a positive number) instead of maximizing the usual VLB. In what follows we always talk about _negated_ VLB, even when we use the term VLB for short.\n",
         "\n",
-        "**Also note** that although we need a stochastic estimate of VLB: \n",
+        "**Also note** that to pass the test, your code should work with any mini-batch size.\n",
+        "\n",
+        "To do that, we need a stochastic estimate of VLB: \n",
         "$$\\text{VLB} = \\sum_{i=1}^N \\text{VLB}_i \\approx \\frac{N}{M}\\sum_{i_s}^M \\text{VLB}_{i_s}$$\n",
-        "where $N$ is the dataset size, $\\text{VLB}_i$ is the term of VLB corresponding to the $i$-th object, and $M$ is the mini-batch size; in the function below you need to return just average across the mini-batch $\\frac{1}{M}\\sum_{i_s}^M \\text{VLB}_{i_s}$. People usually optimize this unscaled version of VLB since it doesn't depend on the dataset set size - you can write VLB function once and use it for different datasets - and it doesn't affect optimization (it does affect the learning rate though). The correct value for this unscaled VLB should be around $100 - 170$."
+        "where $N$ is the dataset size, $\\text{VLB}_i$ is the term of VLB corresponding to the $i$-th object, and $M$ is the mini-batch size. But instead of this stochastic estimate of the full VLB we will use an estimate of the negated VLB normalized by the dataset size, i.e. in the function below you need to return average across the mini-batch $-\\frac{1}{M}\\sum_{i_s}^M \\text{VLB}_{i_s}$. People usually optimize this normalized version of VLB since it doesn't depend on the dataset set - you can write VLB function once and use it for different datasets - the dataset size won't affect the learning rate too much. The correct value for this normalized negated VLB should be around $100 - 170$ in the example below."
       ]
     },
     {
@@ -193,7 +196,7 @@
       "cell_type": "code",
       "source": [
         "def vlb_binomial(x, x_decoded_mean, t_mean, t_log_var):\n",
-        "    \"\"\"Returns the value of Variational Lower Bound\n",
+        "    \"\"\"Returns the value of negative Variational Lower Bound\n",
         "    \n",
         "    The inputs are tf.Tensor\n",
         "        x: (batch_size x number_of_pixels) matrix with one image per row with zeros and ones\n",