source: kdnuggets: 10 probability concepts for machine learning explained simply
level: technical
a model is almost never sure of anything. it spreads its confidence across options and picks the most likely one. random variables represent unknown outcomes, like whether an email is spam. a probability distribution maps which values a variable can take and how often. expectation gives the average outcome, while variance and standard deviation measure spread. conditional probability updates beliefs with new evidence, and bayes' theorem formalizes that update. joint, marginal, and conditional distributions describe relationships between variables.
likelihood measures how well model parameters explain observed data. maximum likelihood estimation picks parameters that make the data most probable, often using log-likelihood for stability. sampling lets us work with subsets of data. the law of large numbers says sample averages approach true averages with more data. the central limit theorem states that sample means become normally distributed, with uncertainty shrinking as sample size grows. these ideas justify using mini-batches and validation sets.
entropy quantifies uncertainty in a distribution. cross-entropy measures how well one distribution approximates another, and minimizing it is equivalent to maximizing log-likelihood. kullback-leibler divergence measures the difference between two distributions. these concepts underpin loss functions and model training. understanding them helps debug models, choose metrics, and interpret outputs. they turn probability from a prerequisite into a practical tool for building and evaluating machine learning systems.
why it matters: these concepts form the foundation for interpreting model outputs, designing loss functions, and making reliable predictions in ai and data science.
source: kdnuggets: 10 probability concepts for machine learning explained simply