Multi-Armed Bandit
An optimization algorithm that balances exploration of unknown options with exploitation of known good options, dynamically allocating more traffic to better-performing variants during an experiment.
The multi-armed bandit problem gets its name from a gambler facing multiple slot machines (one-armed bandits) with unknown payout rates. The gambler must balance trying different machines to learn their payouts (exploration) with playing the machine that seems best so far (exploitation). In optimization, each "arm" is a variant being tested.
Unlike traditional A/B tests that split traffic equally for the entire experiment, bandit algorithms dynamically shift traffic toward better-performing variants. Thompson Sampling, Upper Confidence Bound (UCB), and epsilon-greedy are common algorithms. As data accumulates, more traffic flows to the winning variant, reducing the opportunity cost of showing inferior variants to users.
For growth teams, bandits are ideal for optimizing continuous choices: which headline to show, which recommendation algorithm to use, which pricing tier to display. They converge faster than A/B tests for clear winners and continuously adapt to changing conditions. The trade-off is reduced statistical rigor: bandits optimize for cumulative reward rather than producing clean causal estimates of treatment effects.
Related Terms
Cosine Similarity
A measure of similarity between two vectors based on the cosine of the angle between them, ranging from -1 (opposite) to 1 (identical), commonly used to compare embeddings.
Dimensionality Reduction
Techniques that reduce the number of dimensions in high-dimensional data while preserving meaningful structure, used for visualization, compression, and noise removal.
Batch Inference
Processing multiple ML predictions as a group at scheduled intervals rather than one-at-a-time on demand, optimizing for throughput and cost over latency.
Real-Time Inference
Generating ML predictions on-demand as requests arrive, typically with latency requirements under 200ms for user-facing features.
Data Pipeline
An automated sequence of data processing steps that moves data from source systems through transformations to destination systems, enabling reliable and repeatable data flows across an organization.
ETL (Extract, Transform, Load)
A data integration pattern that extracts data from source systems, transforms it into a structured format suitable for analysis, and loads it into a target data warehouse or database.