Inside Eleanor Hanna’s RecSys 2025 Talk on Cross-Channel Personalization at Scale
Why A/B Tests and Segmentation Don’t Cut It
CRM teams today rely heavily on A/B tests and basic segmentation to optimize messaging. But this approach assumes a dangerous level of user homogeneity. As Eleanor put it, "an aggregate value can be misleading depending on the distribution"; and in many cases, average performance actively masks harm.
Picking a single "best" message based on an A/B test often means finding the least bad option across a wildly heterogeneous user base. It might work OK for no one. And when you consider the multi-dimensional space of CRM variables (value proposition, tone, CTA, timing, channel), the limitations of static testing become obvious.
What Agentic CRM Actually Does
Rather than treating the user base as a monolith, agentic CRM assigns a lightweight agent to each user. That agent makes modular decisions at send time: which value proposition to use, which tone, which channel, when to send. Every decision is informed by the user’s past behavior. and optimized independently through live experimentation.
Each message sent is a controlled test. The agent measures its impact using interrupted time series analysis, weighting events before and after the message with exponential decay. For each component (like tone or CTA), the agent maintains its own posterior belief, updating over time through Thompson sampling.
These modular routines come together at send time to create a new experiment for each user. And they keep learning.
Handling Sparsity and Cold Start
Agents don’t operate in isolation. They learn from the broader user base through cross-user imputations. If a specific user hasn’t seen enough examples of a certain value prop or timing window, the agent can use population-level data to infer likely outcomes. As more user-specific data is collected, those priors are refined.
This isn’t just a fancier recommender. The core difference is that agents don’t optimize for clicks or views. They optimize causal effect: what actually drives the desired outcome, given the full messaging context.
Real-World Results
Aampe deployed this system with a large multi-vertical app operating across multiple regions and use cases. The agentic CRM system was tested against their legacy rule-based system. Across all KPIs, features, and segments, the agentic system showed substantial performance gains.
CRM managers didn’t need to pick one "winner" message. They needed to give the system options to explore. The agents did the rest.
Bottom Line
Agentic CRM turns personalization into a learning process. Every message is a test. Every outcome is a lesson. And the result is a system that adapts to users as they evolve, without overfitting to averages or overloading marketers with decision fatigue.
