Offline Policy Evaluation
Offline policy evaluation is the practice of estimating how a new decision policy would have performed using historical logged data, before deploying that policy live.
Key points
- Yildirim treats offline policy evaluation as a necessary companion to contextual bandits because teams often need to evaluate a candidate policy using logged data rather than only live traffic [src-021].
- Causal-inference approaches such as inverse propensity scoring and doubly robust estimation estimate the counterfactual outcome of a different policy, but require knowing the logged policy's action probabilities [src-021].
- Sampling/replay approaches evaluate a new policy by replaying logged examples and keeping only the cases where the new policy's chosen action matches the logged action; non-uniform logging policies require adjustments such as rejection sampling or propensity weighting [src-021].
- The operational warning is metadata-heavy: logged bandit or A/B-test data should preserve action propensities, context, chosen action, and observed outcome, otherwise offline evaluation becomes biased or impossible [src-021].
Related entities
_(none yet)_
Related concepts
Source references
- [src-021] Ugur Yildirim — "An Overview of Contextual Bandits" (2024-02-02)
Recommended next
Keep reading from this thread
From 500 indexed pages and articles.
- Wiki concept Upper Confidence Bound A bandit exploration strategy that scores actions by combining estimated reward with an uncertainty bonus, favouring actions that look Related by 021
- Wiki concept Epsilon-Greedy A simple exploration strategy where an algorithm usually chooses the currently best-looking action, but with a small probability chooses randomly to keep learning Related by 021
- Wiki concept Exploration-Exploitation Trade-off The exploration-exploitation trade-off is the core tension in bandit problems: an algorithm must spend some decisions learning about uncertain actions while also taking actions Related by 021