updated cv-reg demo

DS-100 · Mar 8, 2023 · 31a4706 · 31a4706
1 parent 336e483
commit 31a4706
Show file tree

Hide file tree

Showing 71 changed files with 13,532 additions and 780 deletions.
diff --git a/..._loss_transformations/loss_transformations_files/figure-pdf/cell-6-output-1.png b/..._loss_transformations/loss_transformations_files/figure-pdf/cell-6-output-1.png
diff --git a/..._loss_transformations/loss_transformations_files/figure-pdf/cell-7-output-1.png b/..._loss_transformations/loss_transformations_files/figure-pdf/cell-7-output-1.png
diff --git a/..._loss_transformations/loss_transformations_files/figure-pdf/cell-8-output-1.png b/..._loss_transformations/loss_transformations_files/figure-pdf/cell-8-output-1.png
diff --git a/cv_regularization/.ipynb_checkpoints/cv_reg-checkpoint.ipynb b/cv_regularization/.ipynb_checkpoints/cv_reg-checkpoint.ipynb
diff --git a/cv_regularization/cv_reg.ipynb b/cv_regularization/cv_reg.ipynb
diff --git a/cv_regularization/cv_reg.qmd b/cv_regularization/cv_reg.qmd
@@ -13,7 +13,7 @@ format:
 jupyter: python3
 ---
 
-We ended last lecture with a question: how do we control the complexity of our model? To answer this, we must know precisely when our model begins to overfit. The key to this lies in evaluating the model on unseen data using a process called Cross-Validation. A second point this note will address is how to combat overfitting -- namely, through a technique known as regularization.
+Consider the question: how do we control the complexity of our model? To answer this, we must know precisely when our model begins to overfit. The key to this lies in evaluating the model on unseen data using a process called Cross-Validation. A second point this note will address is how to combat overfitting -- namely, through a technique known as regularization.
 
 ## Cross Validation
 
@@ -101,7 +101,21 @@ For `k` = 5, we have the following partitions. At each iteration, we train a mod
 
 ![](images/kfold.png)
 
-Note that the value of the hyperparameter is fixed during this process. By doing so, we can be confident in the hyperparameter's performance on the *entire* dataset. To compare multiple choices of a hyperparameter, say `m`, we run k-fold cross validation `m` times. The smallest of the `m` resulting errors corresponds to the best hyperparameter value.
+Note that the value of the hyperparameter is fixed during this process. By doing so, we can be confident in the hyperparameter's performance on the *entire* dataset. To compare multiple choices of a hyperparameter --say `m` choices of hyperparameter-- we run k-fold cross validation `m` times. The smallest of the `m` resulting errors corresponds to the best hyperparameter value.
+
+#### Hyperparameter Selection Example
+
+K-fold cross validation can aid in choosing the best hyperparameter values in respect to our model and loss function.
+
+Consider an example where we run k-fold cross validation with `k` = 3. We are implementing a model that depends on a hyperparameter $\alpha$, and we are searching for an $\alpha$ that minimizes our loss function. We have narrowed down our hyperparameters such that $\alpha = [0.01, 0.1, 1, 10]$.
+
+![](images/cv_hyperparam_selection.png)
+
+The losses of the model are shown per `k` fold of training data (arrows are pointing to the loss value) for each value of $\alpha$. The average loss over the k-fold cross validation is displayed to the right of all the k-fold losses.
+
+To determine the best $\alpha$ value, we must compare the average loss over the k-folds of training data. $\alpha = 0.01$ yields us an average loss of $5$, $\alpha = 0.1$ yields us an average loss of $4.67$, $\alpha = 1$ yields us an average loss of $7$, and $\alpha = 10$ yields us an average loss of $10.67$.
+
+Thus, we would select $\alpha = 0.1$ as our hyperparameter value as it results in the lowest average loss over the k-folds of training data out of our possible $\alpha$ values.
 
 #### Picking K
 
@@ -321,6 +335,10 @@ lasso_model.coef_
 Notice how we standardized our data first. Lasso Regression then set the coefficients of our squared features to $0$ -- presumably, these are the least important predictors of `mpg`.
 
 ### Summary of Regularization Methods
+A summary of our regression models is shwon below:
+
+![](images/reg_models_sum.png)
+
 
 Understanding the distinction between Ridge Regression and Lasso Regression is important. We've provided a helpful visual that summarizes the key differences.
 

diff --git a/cv_regularization/images/cv_hyperparam_selection.png b/cv_regularization/images/cv_hyperparam_selection.png
diff --git a/cv_regularization/images/reg_models_sum.png b/cv_regularization/images/reg_models_sum.png
diff --git a/docs/constant_model_loss_transformations/loss_transformations.html b/docs/constant_model_loss_transformations/loss_transformations.html
@@ -2,7 +2,7 @@
 <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>
 
 <meta charset="utf-8">
-<meta name="generator" content="quarto-1.2.280">
+<meta name="generator" content="quarto-1.2.313">
 
 <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
 

diff --git a/...loss_transformations/loss_transformations_files/figure-html/cell-6-output-1.png b/...loss_transformations/loss_transformations_files/figure-html/cell-6-output-1.png
diff --git a/...loss_transformations/loss_transformations_files/figure-html/cell-7-output-1.png b/...loss_transformations/loss_transformations_files/figure-html/cell-7-output-1.png
diff --git a/...loss_transformations/loss_transformations_files/figure-html/cell-8-output-1.png b/...loss_transformations/loss_transformations_files/figure-html/cell-8-output-1.png
diff --git a/docs/eda/eda.html b/docs/eda/eda.html
@@ -2,7 +2,7 @@
 <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>
 
 <meta charset="utf-8">
-<meta name="generator" content="quarto-1.2.280">
+<meta name="generator" content="quarto-1.2.313">
 
 <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
 
@@ -1799,7 +1799,7 @@ <h2 data-number="6.5" class="anchored" data-anchor-id="reproducing-data-compute-
       <td>3028122</td>
       <td>2.120747</td>
       <td>1.957405</td>
-      <td>2.278640</td>
+      <td>2.27864</td>
     </tr>
     <tr>
       <th>4</th>
@@ -1859,12 +1859,12 @@ <h2 data-number="6.5" class="anchored" data-anchor-id="reproducing-data-compute-
       <td>51.000000</td>
       <td>51.000000</td>
       <td>51.000000</td>
-      <td>5.100000e+01</td>
-      <td>5.100000e+01</td>
-      <td>5.100000e+01</td>
-      <td>51.000000</td>
-      <td>51.000000</td>
-      <td>51.000000</td>
+      <td>51.0</td>
+      <td>51.0</td>
+      <td>51.0</td>
+      <td>51.0</td>
+      <td>51.0</td>
+      <td>51.0</td>
     </tr>
     <tr>
       <th>mean</th>
@@ -1874,9 +1874,9 @@ <h2 data-number="6.5" class="anchored" data-anchor-id="reproducing-data-compute-
       <td>2.102549</td>
       <td>1.782941</td>
       <td>1.971961</td>
-      <td>6.436069e+06</td>
-      <td>6.500226e+06</td>
-      <td>6.510423e+06</td>
+      <td>6436069.078431</td>
+      <td>6500225.72549</td>
+      <td>6510422.627451</td>
       <td>2.104969</td>
       <td>1.784655</td>
       <td>1.969928</td>
@@ -1889,9 +1889,9 @@ <h2 data-number="6.5" class="anchored" data-anchor-id="reproducing-data-compute-
       <td>1.498745</td>
       <td>1.337414</td>
       <td>1.478468</td>
-      <td>7.360660e+06</td>
-      <td>7.408168e+06</td>
-      <td>7.394300e+06</td>
+      <td>7360660.467814</td>
+      <td>7408168.462614</td>
+      <td>7394300.076705</td>
       <td>1.500236</td>
       <td>1.338263</td>
       <td>1.474929</td>
@@ -1904,11 +1904,11 @@ <h2 data-number="6.5" class="anchored" data-anchor-id="reproducing-data-compute-
       <td>0.170000</td>
       <td>0.000000</td>
       <td>0.210000</td>
-      <td>5.787590e+05</td>
-      <td>5.776050e+05</td>
-      <td>5.794830e+05</td>
+      <td>578759.0</td>
+      <td>577605.0</td>
+      <td>579483.0</td>
       <td>0.172783</td>
-      <td>0.000000</td>
+      <td>0.0</td>
       <td>0.210049</td>
     </tr>
     <tr>
@@ -1919,9 +1919,9 @@ <h2 data-number="6.5" class="anchored" data-anchor-id="reproducing-data-compute-
       <td>1.295000</td>
       <td>1.210000</td>
       <td>1.235000</td>
-      <td>1.789606e+06</td>
-      <td>1.820311e+06</td>
-      <td>1.844920e+06</td>
+      <td>1789606.0</td>
+      <td>1820311.0</td>
+      <td>1844920.0</td>
       <td>1.297485</td>
       <td>1.211433</td>
       <td>1.233905</td>
@@ -1934,9 +1934,9 @@ <h2 data-number="6.5" class="anchored" data-anchor-id="reproducing-data-compute-
       <td>1.800000</td>
       <td>1.520000</td>
       <td>1.700000</td>
-      <td>4.467673e+06</td>
-      <td>4.507445e+06</td>
-      <td>4.506589e+06</td>
+      <td>4467673.0</td>
+      <td>4507445.0</td>
+      <td>4506589.0</td>
       <td>1.808606</td>
       <td>1.521612</td>
       <td>1.694502</td>
@@ -1949,9 +1949,9 @@ <h2 data-number="6.5" class="anchored" data-anchor-id="reproducing-data-compute-
       <td>2.575000</td>
       <td>1.990000</td>
       <td>2.220000</td>
-      <td>7.446805e+06</td>
-      <td>7.451987e+06</td>
-      <td>7.502811e+06</td>
+      <td>7446805.0</td>
+      <td>7451987.0</td>
+      <td>7502811.0</td>
       <td>2.577577</td>
       <td>1.993607</td>
       <td>2.219482</td>
@@ -1964,9 +1964,9 @@ <h2 data-number="6.5" class="anchored" data-anchor-id="reproducing-data-compute-
       <td>7.910000</td>
       <td>7.920000</td>
       <td>7.920000</td>
-      <td>3.951222e+07</td>
-      <td>3.950165e+07</td>
-      <td>3.914299e+07</td>
+      <td>39512223.0</td>
+      <td>39501653.0</td>
+      <td>39142991.0</td>
       <td>7.928425</td>
       <td>7.913519</td>
       <td>7.899949</td>
@@ -2812,7 +2812,7 @@ <h2 data-number="6.6" class="anchored" data-anchor-id="bonus-eda-reproducing-the
       <td>3028122</td>
       <td>2.120747</td>
       <td>1.957405</td>
-      <td>2.278640</td>
+      <td>2.27864</td>
     </tr>
   </tbody>
 </table>

diff --git a/docs/index.html b/docs/index.html
@@ -2,7 +2,7 @@
 <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>
 
 <meta charset="utf-8">
-<meta name="generator" content="quarto-1.2.280">
+<meta name="generator" content="quarto-1.2.313">
 
 <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
 

diff --git a/docs/intro_lec/introduction.html b/docs/intro_lec/introduction.html
@@ -2,7 +2,7 @@
 <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>
 
 <meta charset="utf-8">
-<meta name="generator" content="quarto-1.2.280">
+<meta name="generator" content="quarto-1.2.313">
 
 <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
 
@@ -155,6 +155,16 @@ <h1 class="quarto-secondary-nav-title"><span class="chapter-number">1</span>&nbs
   <div class="sidebar-item-container"> 
   <a href="../intro_to_modeling/intro_to_modeling.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">10</span>&nbsp; <span class="chapter-title">Introduction to Modeling</span></a>
   </div>
+</li>
+        <li class="sidebar-item">
+  <div class="sidebar-item-container"> 
+  <a href="../constant_model_loss_transformations/loss_transformations.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">11</span>&nbsp; <span class="chapter-title">Constant Model, Loss, and Transformations</span></a>
+  </div>
+</li>
+        <li class="sidebar-item">
+  <div class="sidebar-item-container"> 
+  <a href="../ols/ols.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">12</span>&nbsp; <span class="chapter-title">Ordinary Least Squares</span></a>
+  </div>
 </li>
     </ul>
     </div>

diff --git a/docs/intro_to_modeling/intro_to_modeling.html b/docs/intro_to_modeling/intro_to_modeling.html
@@ -2,7 +2,7 @@
 <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>
 
 <meta charset="utf-8">
-<meta name="generator" content="quarto-1.2.280">
+<meta name="generator" content="quarto-1.2.313">
 
 <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
 

diff --git a/docs/intro_to_modeling/intro_to_modeling_files/figure-html/cell-2-output-1.png b/docs/intro_to_modeling/intro_to_modeling_files/figure-html/cell-2-output-1.png
diff --git a/docs/intro_to_modeling/intro_to_modeling_files/figure-html/cell-3-output-1.png b/docs/intro_to_modeling/intro_to_modeling_files/figure-html/cell-3-output-1.png
diff --git a/docs/ols/ols.html b/docs/ols/ols.html
@@ -2,7 +2,7 @@
 <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>
 
 <meta charset="utf-8">
-<meta name="generator" content="quarto-1.2.280">
+<meta name="generator" content="quarto-1.2.313">
 
 <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
 

diff --git a/docs/pandas_1/pandas_1.html b/docs/pandas_1/pandas_1.html
@@ -2,7 +2,7 @@
 <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>
 
 <meta charset="utf-8">
-<meta name="generator" content="quarto-1.2.280">
+<meta name="generator" content="quarto-1.2.313">
 
 <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">