An introduction to methods for automated learning of relationships on the basis of empirical data. Classification and regression using nearest neighbour methods, decision trees, linear and non-linear models, class-conditional models, neural networks, and Bayesian methods. Clustering algorithms and dimensionality reduction. Model selection. Problems of over-fitting and assessing accuracy. Problems with handling large databases.
In this assignment, I implemented both the Gradient Descent and Newton's Method algorithms in order to observe and compare the two in terms of their behaviour when minimizing an objective function.
I also implemented Least Squares Polynomial Regression with the goal of learning the problems of overfitting and underfitting aswell as the effects of the training set size on the performance of the model. The concept of Bias-Variance tradeoff is also another major topic of learning in this problem.
As for the second assignment, I derived the negative log-posterior objective value for Multinomial Logistic Regression. I then implemented the algorithm in order to observe the performance of Logistic Regression on different datasets with distinct characteristics.
Secondly, I implemented and observed the similarities and differences between clustering algorithms such as K-Means and GMM. The effects of sparse data and the dimensionality of the data was also observed.
For the final, I took a close examination of the CBOE volatility index (VIX) which is a well known measure for the stock market's implied volatility. The first goal was to learn of MoG model on the monthly log returns of the VIX from January 1990 to December 2019 using a Genetic Algorithm Solver. Then using the OLS, LASSO, Ridge, and Elastic Net Regression algorithms, I attempted to build a model to predict the future values of the VIX index using the equity market volatility (EMV) tracker. The overall goal of this report is to investigate and determine underlying phenomenons that dictate the behaviour of the stock market during certain periods of time and to also determine possible factors driving the behaviours of the market.