Supervised Machine Learning from First Principles
Greetings students! I’m thrilled to present our course on the exhilarating world of Supervised Machine Learning. This course offers a unique opportunity to gain a comprehensive understanding of modern statistical learning techniques for modeling, prediction and inference.
We will start from first principles – understanding basic concepts like overfitting, bias vs variance tradeoffs, model selection methods. We will build intuition on why and when to apply supervision to our models. We will then systematically progress towards regression, classification, validation techniques, regularization, dimension reduction and treebased methods.
The course strikes the perfect balance between theory and practical application. You will thoroughly learn linear regression, logistic regression, PCA, regularized models, tree pruning through extensive coding assignments. We want a seamless blend of core ideas and their implementation.
This is an applied course focusing on model accuracy, precisionrecall tradeoffs, ethics. We want to make you experts in supervised techniques with both depth and breadth. I will highlight modern advances and limitations to provide perspective. By the end, you will have an integrated grasp of the what, why and how behind supervision.
I can’t wait for our pioneering journey into the frontier of Supervised Machine Learning. Buckle up for an intellectual joyride! Do reach out if you have any questions. Now let’s get started…
Introduction to Statistical Learning

1Modelling for prediction versus modelling for interference

2Parametric versus Non Parametric Methods

3Model Prediction Accuracy and Model Interpretability Trade Off

4Supervised versus Unsupervised Learning

5Regression versus Classification

6Assessing Model Accuracy  Measury of Quality of Fit

7Bias Variance Trade Off

8Assessing Model Accuracy Classification Setting

9Classification Example K Nearest Neighbours KNN

10Confidence Intervals for Coefficient Estimates: Simple Linear Regression

11Estimating Model Coefficients: Simple Linear Regression

12Hypothesis Test of Coefficient Estimates: Simple Linear Regression

13Accuracy of Coefficient Estimates: Simple Linear Regression
Linear Regression

14Introduction to Linear Regression

15Assessing Simple Linear Model Accuracy

16Residual Standard Error (RSE)

17R Squared Statistic

18Multiple Linear Regression

19Estimating Multiple Linear Regression Coefficients

20Question 1  Is There a Relationship Between Response and Predictors

21Question 2  Variable Selection

22Question 3  Model Accuracy

23Dealing with Qualitative Variables

24Including Interaction Terms in the Model (Non additive Linear Models)

25Including Non Linear Terms in Linear Models

26Problem #1  Non linearity of the Data

27Problem #2  Correlation of the Error Terms

28Problem #3  Nonconstant Variance of Error Terms

29Problem #4  Outliers

30Problem #5  High Leverage Points

31Problem #6  Collinearity
Classification

32Introduction to Classification

33Why Linear Regression Will Not Work Perfectly In Classification

34Introduction to Logistic Regression

35The Logistic Model

36Estimating Logistic Regression Coefficients Maximum Likelihood Method

37Making Predictions with Logistic Regression

38Multiple Logistic Regression

39Introduction to Linear Discriminant Analysis (LDA)

40Bayes' Theorem of Classification

41Linear Discriminant Analysis One Predictor Clean

42Linear Discriminant Analysis with More Predictors

43The Confusion Matrix, Sensitity and Specificity

44The ROC Curve

45Quadratic Discriminant Analysis
Validation and The Bootstrap Methods
Linear Model Selection and Regularization

52Introduction to Model Selection and Regularization

53Best Subset Selection Method

54Forward Stepwise Selection Method

55Backward Stepwise Selection Method

56Model Selection Choosing the Optimal Model

57Cp Estimate for Test Error

58Akaike Information Criterion (AIC) Estimate for Test Error

59Bayesian Information Criterion (BIC) Estimate for Test Error

60Adjusted RSquared Estimate for Test Error

61Validation Set and Cross Validation Estimates for Test Error

62Introduction to Shrinkage Methods

63Introduction to Ridge Rigression

64Ridge Regression Example

65The Lasso

66The Lasso Example

67Lasso & Ridge Rigression using Mathematical Optimisation