Jun 2, 2025
Schaun Wheeler

Reimagining Customer Engagement: Beyond Hierarchical Reinforcement Learning

Jun 2, 2025
Schaun Wheeler

Reimagining Customer Engagement: Beyond Hierarchical Reinforcement Learning

Jun 2, 2025
Schaun Wheeler

Reimagining Customer Engagement: Beyond Hierarchical Reinforcement Learning

Jun 2, 2025
Schaun Wheeler

Reimagining Customer Engagement: Beyond Hierarchical Reinforcement Learning

Following up on one of my recent posts about using bandits as the anchor point for how we think about agentic learning:

Yes, we’ve stretched the bandit framing well beyond its usual territory — multi-dimensional action spaces, non-ergodic structure, per-user learning - but it’s still a better conceptual fit than alternatives like hierarchical RL.

Customer engagement isn’t about planning journeys. It's about reacting to the present — selecting the best next action based on what we currently know, not on where we imagined the user might be at this point in a pre-designed arc.

Hierarchical RL assumes temporal structure: first A, then B, then C. You learn sub-policies that unfold over time. That works for robotics, navigation, task decomposition. It doesn’t match the shape of customer behavior, which is noisy, nonlinear, and context-sensitive.

What we’re doing instead is selecting, in parallel, the best options from a bunch of overlapping action sets — tone, CTA, incentive, channel, timing — to form a single composite action to take *right now*. Then we see what happens and update. No storyline, no path planning. Just a series of bets, each one grounded in local context.

So yes, we’ve moved pretty far from textbook bandits. But I'd say we’re even farther from hierarchical RL. And more importantly, this framing reflects how we think engagement really works: not as a narrative, but as a stream of decisions under uncertainty.

0

Related

Shaping the future of marketing with Aampe through innovation, data.

Jun 3, 2025

Schaun Wheeler

Explore why focusing solely on short-term metrics like lift can be misleading when assessing adaptive systems, and discover alternative approaches for meaningful evaluation.

Jun 3, 2025

Schaun Wheeler

Explore why focusing solely on short-term metrics like lift can be misleading when assessing adaptive systems, and discover alternative approaches for meaningful evaluation.

Jun 3, 2025

Schaun Wheeler

Explore why focusing solely on short-term metrics like lift can be misleading when assessing adaptive systems, and discover alternative approaches for meaningful evaluation.

Jun 3, 2025

Schaun Wheeler

Explore why focusing solely on short-term metrics like lift can be misleading when assessing adaptive systems, and discover alternative approaches for meaningful evaluation.

May 29, 2025

Schaun Wheeler

Discover how Aampe's semantic-associative agents adapt to varying user contexts, from fully known to unknown users, ensuring personalized experiences through continuous learning and contextual imputation.

May 29, 2025

Schaun Wheeler

Discover how Aampe's semantic-associative agents adapt to varying user contexts, from fully known to unknown users, ensuring personalized experiences through continuous learning and contextual imputation.

May 29, 2025

Schaun Wheeler

Discover how Aampe's semantic-associative agents adapt to varying user contexts, from fully known to unknown users, ensuring personalized experiences through continuous learning and contextual imputation.

May 29, 2025

Schaun Wheeler

Discover how Aampe's semantic-associative agents adapt to varying user contexts, from fully known to unknown users, ensuring personalized experiences through continuous learning and contextual imputation.

May 28, 2025

Schaun Wheeler

Explore how Aampe's semantic-associative agents differ from traditional multi-armed bandit models. Learn about their multi-dimensional action space and non-ergodic learning approach that tailors user experiences without generalization.

May 28, 2025

Schaun Wheeler

Explore how Aampe's semantic-associative agents differ from traditional multi-armed bandit models. Learn about their multi-dimensional action space and non-ergodic learning approach that tailors user experiences without generalization.

May 28, 2025

Schaun Wheeler

Explore how Aampe's semantic-associative agents differ from traditional multi-armed bandit models. Learn about their multi-dimensional action space and non-ergodic learning approach that tailors user experiences without generalization.

May 28, 2025

Schaun Wheeler

Explore how Aampe's semantic-associative agents differ from traditional multi-armed bandit models. Learn about their multi-dimensional action space and non-ergodic learning approach that tailors user experiences without generalization.

May 26, 2025

Schaun Wheeler

Traditional customer engagement tools often constrain strategies by bundling orchestration and analysis within fixed campaign structures. Adopting an agentic approach—separating orchestration from analysis—enables more dynamic, user-centered communication, allowing for nuanced decision-making and broader impact.

May 26, 2025

Schaun Wheeler

Traditional customer engagement tools often constrain strategies by bundling orchestration and analysis within fixed campaign structures. Adopting an agentic approach—separating orchestration from analysis—enables more dynamic, user-centered communication, allowing for nuanced decision-making and broader impact.

May 26, 2025

Schaun Wheeler

Traditional customer engagement tools often constrain strategies by bundling orchestration and analysis within fixed campaign structures. Adopting an agentic approach—separating orchestration from analysis—enables more dynamic, user-centered communication, allowing for nuanced decision-making and broader impact.

May 26, 2025

Schaun Wheeler

Traditional customer engagement tools often constrain strategies by bundling orchestration and analysis within fixed campaign structures. Adopting an agentic approach—separating orchestration from analysis—enables more dynamic, user-centered communication, allowing for nuanced decision-making and broader impact.

Load More

Load More

Load More

Load More