Summary of content during Deepest PRML study.\

Source: Nasrabadi, Nasser M. “Pattern recognition and machine learning.” Journal of electronic imaging 16.4 (2007): 049901.\

1. Introduction

  • training set / target vector
  • training / learning
  • test set / generalization
  • preprocess / feature extraction
  • supervised learning
    • classification / regression
  • unsupervised learning
    • clustering / density estimation
  • reinforcement learning
    • exploration / exploitation

      1.1 Example: Ploynomial Curve Fitting

  • over-fit
  • regularization
  • validation set

    1.2 Probability Theory

  • sum rule / prodect rule

    1.2.1 Probability densities

  • probability density over x / cumulative distribution function
  • probability mass function

    1.2.2 Expectations and covariances

  • expectation / conditional expectation
  • variance / covariance

    1.2.3 Bayesian probabilities

  • prior / likelihood / posterior
  • estimate the probability distribution over w

    1.2.4 The Gaussian distribution

  • variance / standard deviation / precision
  • sample mean / sample variance

    1.2.5 Curve fitting re-visited

  • Maximum likelihood Estimation
  • Predictive distribution
  • Maximum posterior

    1.2.6 Bayesian curve fitting

    1.3 Model Selection

    1.4 The Curse of Dimensionality

    1.5 Decision Theory

    1.5.1 Minimizng the misclassification rate

    1.5.2 Minimizing the expected loss

    1.5.3 The reject option

    1.5.4 Inference and decision

    1.5.5 Lossfunctions for regression

    1.6 Information Theory

    1.6.1 Relative entropy and mutual information

2.Probability Distributions

2.1 Binary Variables

2.1.1 The beta distribution

2.2 Multinomial Variables

2.2.1 The Dirichlet distribution

2.3 The Gaussian Distribution

2.3.1 Conditional Gaussian distributions

2.3.2 Margianl Gaussian distributions

2.3.3 Bayes’ theorem for Gaussian variables

2.3.4 Maximum likelihood for the Gaussian

2.3.5 Sequential estimation

2.3.6 Baysian inference for the Gaussian

2.3.7 Student’s t-distribution

2.3.8 Periodic variables

2.3.9 Mixtures of Gaussians