Masons, when they start upon a building,
Are careful to test out the scaffolding;

Make sure that planks won’t slip at busy points,
Secure all ladders, tighten bolted joints.

And yet all this comes down when the job’s done
Showing off walls of sure and solid stone.

So if, my dear, there sometimes seem to be
Old bridges breaking between you and me

Never fear. We may let the scaffolds fall
Confident that we have built our wall.

Seamus Heaney

Machine Learning

• A classifier is a system that inputs (typically) a vector of discrete and/or continuous feature values and outputs a single discrete value, the class.
• Learning = Representation + Evaluation + Optimization
  • Representation: A classifier must be represented in some formal language that the computer can handle = choosing a set of classifiers that it can possibly learn. This set is called the hypothesis space.
  • Evaluation (Objective Function): Distinguish good classifiers from bad ones.
  • Optimization: A method to search among the classifiers in the language for the highest-scoring one.

Representation	Evaluation	Optimization
Instances	Accuracy/Error Rate	Combinatorial Optimization
K-nearest Neighbor	Precision & Recall	Greedy Search
Support Vector Machines	Squared Error	Beam Search
Hyperplanes	Likelihood	Branch-and-bound
Naive Bayes	Posterior Probability	Continous Optimization
Logistic Regression	Information Gain	Unconstrained
Decision Trees	K-L Divergence	Gradient Descent
Sets of rules	Cost/Utility	Conjugate gradient
Propositional rules	Margin	Quasi-Newton methods
Logic programs		Constrained
Neural Networks		Linear programming
Graphical models		Quadratic programming
Bayesian networks
Conditional random fields

Linear Regression

• Univariate Linear Regression
• Multivariate Linear Regression
• Assumption for Linear Models

Logistic Regression (Naive Bayes)

• Linear model for binary classification
• Maximum Likelihood Estimation
• Assumptions:

Trees

Neural Networks

• Feed-forward
• Backpropagation

Papers

1. A Few Useful Things to Know About Machine Learning
2. ImageNet Classification with Deep Convolutional Neural Networks
3. Sequence to Sequence Learning with Neural Networks
4. Attention Is All You Need
5. Generative Adversarial Nets
6. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
7. Deep Residual Learning for Image Recognition
8. Mastering the game of Go with deep neural networks and tree search
9. A Survey on Deep Learning in Medical Image Analysis