Overall Evaluation Criterion

An overall evaluation criterion is a stable bundle of metrics used to judge whether an experiment is good for customers and the business in the long run, not only whether one fast KPI moved.

Key points

• Statsig positions OEC as a maturity step beyond single-point KPIs such as click-through rate or conversion rate ^[src-036].
• The article's example warning is that a near-term metric can improve while retention or customer value worsens, making the single metric misleading ^[src-036].
• A comprehensive OEC may blend revenue, lifetime value, engagement, support tickets, risk scores, and customer sentiment ^[src-036].
• Enterprise teams struggle with OEC design because each domain team owns a slice of data, and merging them requires shared definitions, agreements, and latency-tolerant pipelines ^[src-036].
• Statsig recommends a centralized metric catalog, standardized primary and guardrail metrics for similar experiments, and an analytics team that owns and maintains the core OEC list ^[src-036].
• OECs create a useful counterpoint to Input Output Metrics: bundled evaluation can help align experiment judgment, but teams still need transparent component metrics so the bundle does not hide trade-offs ^[src-036].

Related entities

• Statsig

Related concepts

• Enterprise Scale Experimentation
• Input Output Metrics
• Proxy Metrics In Experiments
• Outcome Obsessed Product Management
• Statistical Significance Testing

Source references

• ^[src-036] Yuzheng Sun — "Addressing complexity in enterprise-scale experimentation" (2025-04-23)