Skip to content

Commit

Permalink
post-meeting changes in risk minimization and new latexmath definitio…
Browse files Browse the repository at this point in the history
…n of \Pxy
  • Loading branch information
ludwigbothmann committed Aug 6, 2024
1 parent f449ede commit 6db9559
Show file tree
Hide file tree
Showing 80 changed files with 6 additions and 6 deletions.
2 changes: 1 addition & 1 deletion latex-math/basic-ml.tex
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
\newcommand{\nset}{\{1, \ldots, n\}} % set from 1 to n
\newcommand{\pset}{\{1, \ldots, p\}} % set from 1 to p
\newcommand{\gset}{\{1, \ldots, g\}} % set from 1 to g
\newcommand{\Pxy}{\mathbb{P}_{xy}} % P_xy
\newcommand{\Pxy}{P_{xy}} % P_xy
\newcommand{\Exy}{\mathbb{E}_{xy}} % E_xy: Expectation over random variables xy
\newcommand{\xv}{\mathbf{x}} % vector x (bold)
\newcommand{\xtil}{\tilde{\mathbf{x}}} % vector x-tilde (bold)
Expand Down
10 changes: 5 additions & 5 deletions slides/advriskmin/slides-advriskmin-risk-minimizer.tex
Original file line number Diff line number Diff line change
Expand Up @@ -89,26 +89,26 @@
\begin{vbframe}{Optimal Constant Model}

\begin{itemize}
{\footnotesize
\item We would like a loss optimal, constant baseline predictor
\item A "featureless" ML model, which always predicts the same value
\item Can use it as baseline in experiments, if we don't beat this
with more complex model, that model is useless
\item Will also be useful as component in algorithms and derivations
}
\end{itemize}

%By restricting $\Hspace$ to constant functions $f_c$ independent of $\xv$, the constant model minimizing the theoretical risk


$$f_{c}^{\ast} = \argmin_{c \in \mathbb{R}} \E_{xy} \left[L(y,c)\right] = \argmin_{c \in \mathbb{R}} \E_{y} \left[L(y,c)\right]$$

and

$\fx = \theta = c$ that optimizes the empirical risk $\risket$ is denoted as as $\hat{f}_c = \argmin_{c \in \mathbb{R}} \risket$.
{\footnotesize and $\fx = \theta = c$ that optimizes the empirical risk $\risket$ is denoted as as $\hat{f}_c = \argmin_{c \in \mathbb{R}} \risket$.}

\vspace*{-0.5cm}
\vspace*{-0.2cm}

\begin{center}
\includegraphics[width = 0.2\textwidth]{figure/l1_vs_l2.png}
\includegraphics[width = 0.38\textwidth]{figure/l1_vs_l2.png}
\end{center}
\end{vbframe}

Expand Down
Empty file modified slides/regularization/figure/bias_var_decomp.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/classifi_nn_err_decay.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/classifi_nn_err_size.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/classifi_nn_size_1.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/classifi_nn_size_2.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/classifi_nn_size_3.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/classifi_nn_size_4.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/classifi_nn_size_5.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/classifi_nn_size_6.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/classifi_nn_w_size_1.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/classifi_nn_w_size_2.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/classifi_nn_w_size_3.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/classifi_nn_w_size_4.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/classifi_nn_w_size_5.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/classifi_nn_w_size_6.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/l2_reg_hess_01.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/l2_reg_hess_02.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/l2_reg_hess_03.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/l2_reg_hess_04.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/lasso_contour_cases.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/lin_model_regu_01.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/lin_model_regu_02.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/model_eval_01.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/model_eval_02.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/model_eval_03.png
100755 → 100644
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified slides/regularization/figure/multicollinearity_example.png
100755 → 100644
Empty file modified slides/regularization/figure/poly_ridge_01.png
100755 → 100644
Empty file modified slides/regularization/figure/poly_ridge_02.png
100755 → 100644
Empty file modified slides/regularization/figure/reg_contours_01.png
100755 → 100644
Empty file modified slides/regularization/figure/reg_contours_02.png
100755 → 100644
Empty file modified slides/regularization/figure/reg_surfaces_l1_lam0.png
100755 → 100644
Empty file modified slides/regularization/figure/reg_surfaces_l1_lam1.png
100755 → 100644
Empty file modified slides/regularization/figure/reg_surfaces_l1_lam10.png
100755 → 100644
Empty file modified slides/regularization/figure/reg_surfaces_l2_lam0.png
100755 → 100644
Empty file modified slides/regularization/figure/reg_surfaces_l2_lam1.png
100755 → 100644
Empty file modified slides/regularization/figure/reg_surfaces_l2_lam10.png
100755 → 100644
Empty file modified slides/regularization/figure/ridge_perspectives_01.png
100755 → 100644
Empty file modified slides/regularization/figure/ridge_perspectives_02.png
100755 → 100644
Empty file modified slides/regularization/figure/ridge_perspectives_03.png
100755 → 100644
Empty file modified slides/regularization/figure/ridge_perspectives_04.png
100755 → 100644
Empty file modified slides/regularization/figure/ridge_vs_sgd_path.png
100755 → 100644
Empty file modified slides/regularization/figure/shrinkage_01.png
100755 → 100644
Empty file modified slides/regularization/figure/shrinkage_02.png
100755 → 100644
Empty file modified slides/regularization/figure/soft_thresholding.png
100755 → 100644
Empty file modified slides/regularization/figure/solution_paths_01.png
100755 → 100644
Empty file modified slides/regularization/figure/solution_paths_02.png
100755 → 100644
Empty file modified slides/regularization/figure/weightdecay_lambda_01.png
100755 → 100644
Empty file modified slides/regularization/figure/weightdecay_lambda_02.png
100755 → 100644
Empty file modified slides/regularization/rsrc/avoid_overfitting.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/bias_var_decomp.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/classifi_nn.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/data_func_utils.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/data_regu_example_1.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/data_regu_example_1.RData
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/data_regu_example_2.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/data_regu_example_2.RData
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/early_stopping.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/graddes_vs_weightdecay.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/l1_reg_hess.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/l2_reg_hess.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/lasso_contour_cases.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/lin_model_regu.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/model_eval.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/multicollinearity_example.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/nn_size.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/ozone_mse_boxplot.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/poly_ridge.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/reg_contours.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/reg_surfaces.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/ridge_perspectives.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/ridge_vs_sgd_path.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/shrinkage.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/soft_thresholding.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/solution_path.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/table_equivariance.R
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/table_equivariance_ols.tex
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/table_equivariance_ridge.tex
100755 → 100644
Empty file.
Empty file modified slides/regularization/rsrc/weightdecay_lambda.R
100755 → 100644
Empty file.

0 comments on commit 6db9559

Please sign in to comment.