/ portfolio /
Measurement and experimentation for AI products.
AI product leadership needs a measurement spine. Without incrementality, experiments, and guardrail metrics, teams can mistake model enthusiasm for business value.
Focus areas
Incrementality
Separating value created by the system from value that would have happened anyway.
A/B testing
Designing experiments for AI features where ranking, timing, targeting, and learning loops can all move.
Statistical significance
Making uncertainty understandable for product and executive decisions without flattening the nuance.
Contextual bandits
Using continuous optimisation where static campaign testing is too slow or too wasteful.
Planned assets
- Incrementality measurement primer for AI leaders.
- A/B testing checklist for AI features.
- Guardrail metrics for personalisation and recommendation systems.
- Contextual multi-armed bandits for campaign optimisation.
Recommended next
Keep reading from this thread
From 491 indexed pages and articles.
- Insight AI Measurement and Experimentation How to measure AI product impact with evals, adoption metrics, online experiments, guardrails, and cost tracking Related by measurement
- Insight Insights Essays and executive notes on enterprise AI in production, recommendation systems, ecommerce AI, measurement, and AI operating models Related by measurement
- Wiki concept Braze A customer engagement platform represented in this wiki by its articles and whitepaper on multi-armed bandits, contextual bandits, and AI decisioning for lifecycle Related by contextual