1.md

author	title	documentclass
Wu Zhenyu (SA21006096)	Homework 1 for Statistical Learning	article

2*

Learn to install Anaconda (version $\geqslant 3.7$) and OpenCV-Python (version $\geqslant 3.4$) on your PC. Write down the installation steps, which can be very helpful for the following exercises and projects.

---

$ sudo pacman -S anaconda python-opencv
warning: python-opencv-4.5.3-4 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...

Packages (2) anaconda-2021.05-1  python-opencv-4.5.3-4

Total Download Size:   1108.32 MiB
Total Installed Size:  3269.08 MiB
Net Upgrade Size:      3260.19 MiB

:: Proceed with installation? [Y/n] n

4

Prove that the maximum likelihood estimator for $\sigma^2$ of a Gaussian distribution, i.e.

$$\hat{\sigma}^2 = \frac1N\sum_{i = 1}^N {(x_i - \bar{x})}^2$$

where $\bar{x} = \frac1N\sum_{i = 1}^N x_i$, is biased.

---

$$\begin{aligned} S^2 & = \frac1N\sum_{i = 1}^N {(X_i - \bar{X})}^2 \ & = \frac1N\sum_{i = 1}^N {\Big((X_i - \mu) - (\bar{X} - \mu)\Big)}^2 \ & = \frac1N\sum_{i = 1}^N \Big({(X_i - \mu)}^2 + {(\bar{X} - \mu)}^2

• 2(X_i - \mu)(\bar{X} - \mu)\Big) \ & = \frac1N\sum_{i = 1}^N \Big({(X_i - \mu)}^2 + {(\bar{X} - \mu)}^2\Big)
• \frac1N\sum_{i = 1}^N \Big(2(X_i - \mu)(\bar{X} - \mu)\Big) \ & = \frac1N\sum_{i = 1}^N \Big({(X_i - \mu)}^2 + {(\bar{X} - \mu)}^2\Big)
• \frac1N\sum_{i = 1}^N \Big(2(X_i - \mu)\Big)(\bar{X} - \mu) \ & = \frac1N\sum_{i = 1}^N \Big({(X_i - \mu)}^2 + {(\bar{X} - \mu)}^2\Big)
• \frac1N\sum_{i = 1}^N \Big(2(\bar{X} - \mu)\Big)(\bar{X} - \mu) \ & = \frac1N\sum_{i = 1}^N \Big({(X_i - \mu)}^2 + {(\bar{X} - \mu)}^2\Big)
• \frac1N\sum_{i = 1}^N \Big(2(\bar{X} - \mu)(\bar{X} - \mu)\Big) \ & = \frac1N\sum_{i = 1}^N \Big({(X_i - \mu)}^2 + {(\bar{X} - \mu)}^2
• 2(\bar{X} - \mu)(\bar{X} - \mu)\Big) \ & = \frac1N\sum_{i = 1}^N \Big({(X_i - \mu)}^2 - {(\bar{X} - \mu)}^2\Big) \ & = \frac1N\sum_{i = 1}^N {(X_i - \mu)}^2
• \frac1N\sum_{i = 1}^N {(\bar{X} - \mu)}^2 \end{aligned}$$

$$\begin{aligned} ES^2 & = E\frac1N\sum_{i = 1}^N {(X_i - \mu)}^2

• E\frac1N\sum_{i = 1}^N {(\bar{X} - \mu)}^2 \ & = DX - D\bar{X} \ & = DX - \frac1NDX \ & = \frac{N - 1}NDX \end{aligned}$$

5*

Choose a pair of parameters $(\mu, \sigma^2)$ to generate $N$ random numbers that follow the Gaussian distribution, then calculate the mean and variance of the random numbers and compare with the ground-truth $(\mu, \sigma^2)$. Set $N$ to 100, 1000, 10000, 100000, $\ldots$, to observe the change of results.