Deep Learning: Theory and Experiments

Notes

1. Background Materials

• Linear Algebra
• Probability
• [Numerical Computation] (https://docs.google.com/presentation/d/1Nt9vr-8_Tcgez8jmz7sPsbXc0PDpU0d04-f2MFR5XZg/edit?usp=sharing)

2. Machine Learning Basics

• [Concepts, Capacity, Estimators, Linear Regression] (https://docs.google.com/presentation/d/1Xn6FaPiGTLnCRKQjIXySxpdteLVT1F8WDFketIUiTlI/edit?usp=sharing)
• [MLE, Bayesian, Other ML Algorithms] (https://docs.google.com/presentation/d/1Dp2IBWnxQmKMszX0uL5HSfK302q6kpMfiexYAoT9z-k/edit?usp=sharing)
• [Stochastic Gradient Descent, etc] (https://docs.google.com/presentation/d/1Ss2BhwyarFGFiEIgqbQ-CW9zq0LJxmaOcHZKMzRWJ5k/edit?usp=sharing)

3. Deep Neural Networks

• Deep Feedforward Neural Networks, Cost, Output, Hidden Units
• Back Propagation

4. [Regularization] (https://docs.google.com/presentation/d/1lg4oxRDvfUIEtzMJ7E-Lqv1cDNiwoNeT1r5T-XnFIQI/edit?usp=sharing)
5. Optimization
6. Convolutional Neural Networks
7. Recurrent Neural Networks

• RNN
• GRU
• LSTM

Tensorflow (tf) Experiments

1. Hello World!
2. Linear Algebra
3. Matrix Decomposition
4. Probability Distributions using TensorBoard
5. Linear Regression by PseudoInverse
6. Linear Regression by Gradient Descent
7. Under Fitting in Linear Regression
8. Optimal Fitting in Linear Regression
9. Over Fitting in Linear Regression
10. Nearest Neighbor
11. Principal Component Analysis
12. Logical Ops by a 2-layer NN (MSE)
13. Logical Ops by a 2-layer NN (Cross Entropy)
14. NotMNIST Deep Feedforward Network: Code for NN and Code for Pickle