Overall Evaluation Criterion
An overall evaluation criterion is a stable bundle of metrics used to judge whether an experiment is good for customers and the business in the long run, not only whether one fast KPI moved.
Key points
- Statsig positions OEC as a maturity step beyond single-point KPIs such as click-through rate or conversion rate [src-036].
- The article’s example warning is that a near-term metric can improve while retention or customer value worsens, making the single metric misleading [src-036].
- A comprehensive OEC may blend revenue, lifetime value, engagement, support tickets, risk scores, and customer sentiment [src-036].
- Enterprise teams struggle with OEC design because each domain team owns a slice of data, and merging them requires shared definitions, agreements, and latency-tolerant pipelines [src-036].
- Statsig recommends a centralized metric catalog, standardized primary and guardrail metrics for similar experiments, and an analytics team that owns and maintains the core OEC list [src-036].
- OECs create a useful counterpoint to Input and Output Metrics: bundled evaluation can help align experiment judgment, but teams still need transparent component metrics so the bundle does not hide trade-offs [src-036].
Related entities
Related concepts
- Enterprise-Scale Experimentation
- Input and Output Metrics
- Proxy Metrics in Experiments
- Outcome-Obsessed Product Management
- Statistical Significance Testing
Source references
- [src-036] Yuzheng Sun — “Addressing complexity in enterprise-scale experimentation” (2025-04-23)