Epsilon-Greedy
Epsilon-greedy is a simple exploration strategy where an algorithm usually chooses the currently best-looking action, but with a small probability chooses randomly to keep learning.
Key points
- In Yildirim’s contextual-bandit overview, epsilon-greedy first estimates which treatment is performing best for a given context, serves that treatment with probability
1 - epsilon, and explores other treatments with probabilityepsilon[src-021]. - The value of epsilon can be scheduled: larger early in an experiment when uncertainty is high, smaller later as more evidence accumulates [src-021].
- For high-dimensional contexts, the article recommends using a model to estimate outcome values instead of trying to compute averages for every exact context value [src-021].
- Epsilon-greedy is easy to explain and implement, but it explores randomly rather than directing exploration toward actions with high uncertainty or high posterior promise [src-021].
Related entities
_(none yet)_
Related concepts
Source references
- [src-021] Ugur Yildirim — “An Overview of Contextual Bandits” (2024-02-02)
Recommended next
Keep reading from this thread
From 491 indexed pages and articles.
- Wiki concept Contextual Bandits Bandit problems where each decision arrives with observable context, and the algorithm learns which actions work best for which contexts from Related by algorithm
- Wiki concept Multi-Armed Bandits A framework for algorithms that make repeated decisions under uncertainty, learning from partial feedback while balancing exploration of uncertain actions against Related by exploration
- Insight Recommendation Systems in Production How recommendation systems become production decisioning systems through signals, ranking, constraints, feedback loops, and experimentation Readers have engaged with this next