At the 10th Berlin Experimentation Meetup, our Principal Research Scientist, Olivier Jeunen, took the stage to share why experimentation can’t just be about measuring outcomes, it has to be about making better decisions over time.
Drawing from our recent research (also presented at RecSys25 in Prague), Olivier unpacked the limits of traditional A/B testing:
Segmentation and static assumptions flatten user differences, making experiments less powerful.
One-off winners ignore the fact that user preferences and contexts shift constantly.
Single metrics oversimplify outcomes and fail to capture the richness of user behavior.
Instead, Olivier showed how agentic infrastructure changes the game:
Using difference-in-differences to isolate true causal impact.
Treating each user’s full event stream (not just one conversion) as signal.
Leveraging modular decision-making loops to adapt in real time.
The result? A system that doesn’t just report outcomes, but continuously learns and improves.
That shift unlocks personalisation at scale.
🎥 Watch the full talk and see why we believe experimentation is evolving into decision infrastructure.
