GateXAIMLMachine Learning: Supervised
  • Chapter 1: Introduction to Machine Learning
    • What is Machine Learning
    • Overview of Machine Learning Paradigm
    • Types: supervised, unsupervised, and reinforcement learning
    • Data
      • Basic Types of Data
      • Exploring Structure of Data
      • Data Quality
      • Data Pre-Processing
    • Parametric and Non-Parametric Functions
      • Parametric Functions
      • Non-Parametric Functions
      • Comparison: Parametric vs Non-Parametric
    • Issues in ML
      • Accuracy and Precision
      • Issues
      • Bias and Variance
    • Evaluation Metrics for Classification
    • Applications
  • Chapter 2: Regression Models
    • Simple Linear Regression
      • Model
      • Estimating Coefficients: Ordinary Least Squares (OLS)
      • Residuals and Residual Standard Error
      • R-Squared ($R^2$)
      • Population vs Sample Regression
      • Standard Errors and Confidence Intervals
    • Multiple Linear Regression
    • Regularized Regression
      • Need for Regularization
      • Ridge Regression (L2 Regularization)
      • Bias–Variance Trade-off in Ridge Regression
      • Comparison with OLS Regression
  • Chapter 3: Classification Models - I
    • Logistic Regression
      • Sigmoid Function and Log-Odds Interpretation
      • Decision Boundary Analysis
    • k-Nearest Neighbour (k-NN)
      • Distance Metrics in k-NN
      • Choice of k and Model Complexity
      • Effect of Dimensionality on Performance
    • Naive Bayes Classifier
      • Bayes’ Theorem and Conditional Independence
      • Gaussian, Multinomial, and Bernoulli Naive Bayes
      • Parameter Estimation and MAP Decision Rule
  • Chapter 4: Classification Models - II
    • Linear Discrimination Analysis (LDA)
    • Fisher’s Linear Discriminant Analysis (LDA)
      • Two-Class Fisher LDA: Step-by-Step Derivation
      • Numerical Example
      • Multiclass Fisher LDA Derivation
      • Assumptions of Fisher LDA
    • Bayesian View of Linear Discriminant Analysis
      • Assumptions in Bayesian LDA
      • Example: Bayesian LDA (Two Classes)
  • Chapter 5: Model Evaluation and Generalization
    • Cross-Validation Methods
      • Hold-Out Validation
      • Leave-One-Out (LOO) Cross-Validation
      • k-Fold Cross-Validation
      • Model Selection and Hyperparameter Tuning
    • Examples: Cross-Validation Methods
    • ROC & AUC
      • Confusion matrix and rates
      • What is a decision threshold?
      • ROC curve and AUC (intuition)
      • Worked numeric example (spam detector)
      • Plotting the ROC curve (from these points)
      • Computing AUC (brief)
      • Perfect and random classifiers (illustration)
      • Practical notes
      • Conclusion
    • Bias--Variance Trade-off
      • Understanding the Trade-off
      • Mathematical Derivation
  • Chapter 6: Regression & Classification Models
    • Decision Trees
      • Entropy, Gini Index, and Information Gain
      • Tree Construction and Stopping Criteria
      • Tree Pruning and Overfitting
      • Bias–Variance Characteristics
    • Support Vector Machine (SVM)
      • Introduction
      • Hyperplane
        • Definition
        • 1D (\,\(\mathbb{R}\)\,) -- a point
        • 2D (\,\(\mathbb{R}^2\)\,) -- a line
        • General $d$ (\,\(\mathbb{R}^d\)\,) -- derivation (orthogonal projection)
        • Decision function (linear classifier)
      • Hard-Margin SVM: geometry, formulation and solution
        • Geometric figure and support vectors (2D)
        • Concept (informal)
        • Derivation of the optimization problem
        • Lagrangian, KKT and consequence
        • Dual statement (brief) and solution strategy
        • Bias $b$ (practical formula)
        • Summary (final formulas)
      • Soft Margin - Support Vector Classifier
        • Motivation: why soft margin
        • Primal formulation (soft margin)
        • Lagrangian and stationarity
        • Dual problem
        • KKT conditions and practical consequences
        • Summary table: \(\alpha_i\), \(\xi_i\) and geometric meaning
        • Hinge Loss
      • Kernel Trick
      • Feature Mapping $\phi(\cdot)$
        • Example: 1D to 2D Mapping
        • Kernel Function
        • The ``Trick'' with Dot Products
        • Common Kernel Functions
    • Neural Network Models
      • Feed-Forward Neural Networks (FFNN)
      • Network Architecture and Notation
      • Activation Functions (Sigmoid, ReLU, Tanh)
      • Forward Propagation and Loss Computation
      • Backpropagation Algorithm and Gradient Flow
      • Vanishing/Exploding Gradient Problem
    • Multi-Layer Perceptron (MLP)
      • Universal Approximation Theorem (Conceptual Insight)
      • Training and Regularization Techniques
      • Comparison with Traditional ML Models
      • Advantages and Limitations of MLPs
GateXAIML · ML Concept Notes

Machine Learning: Supervised Learning

Chapter 1
Introduction to Machine Learning
18 sections
What is Machine Learning, Overview of Machine Learning Paradigm, Types: supervised, unsupervised, and reinforcement learning...
Chapter 2
Regression Models
13 sections
Simple Linear Regression, Multiple Linear Regression, Regularized Regression
Chapter 3
Classification Models - I
11 sections
Logistic Regression, k-Nearest Neighbour (k-NN), Naive Bayes Classifier
Chapter 4
Classification Models - II
9 sections
Linear Discrimination Analysis (LDA), Fisher’s Linear Discriminant Analysis (LDA), Bayesian View of Linear Discriminant Analysis
Chapter 5
Model Evaluation and Generalization
19 sections
Cross-Validation Methods, Examples: Cross-Validation Methods, ROC & AUC...
Chapter 6
Regression & Classification Models
47 sections
Decision Trees, Support Vector Machine (SVM), Neural Network Models...
GateXAIML · Machine Learning: Supervised — Concept Notes
Home | About Us | Contact Us | Privacy Policy | Disclaimer
© 2026 GateXAIML. All original content. Not affiliated with IIT / GATE authorities.