Before you judge an agent, check what it can see
Last week one team asked whether an engagement event from one feature of their product could be a good target event, the thing their agents learn toward. Using that feature tends to lead, eventually, to the action the whole product is built around. The team had already run a repeat-rate check on the event. It came back in the mid-teens, a few points under the bar of around 20 percent they were testing against. The team then had two questions for us. Did the result rule the event out, and if so, what should the agents aim at instead?
A few days later, a team in its first week live asked a related question. They wanted to know where they could see their signal density. Their hypothesis was that sending more messages would produce more of the behavior the agents need to watch.
Both questions are about the raw material of learning. An agent watches what a person does and tries an intervention. Then it checks whether the person’s next actions differ from what they would likely have done anyway. That loop needs two things from the event stream. It needs a target the person can move toward on purpose. It also needs enough of the surrounding journey to show movement before the target is reached. When either one runs thin, learning slows down, and the cause usually sits in what the agent has to look at.
Test a target event for repeatability first
Before asking how common a candidate target event is, ask whether the same person can do it again. Then ask whether they chose to do it the first time. Neither property has much to do with how common an event is. A subscription renewal happens to nearly everyone, on a schedule nobody chooses, which leaves an agent nothing to cause. Booking a training session might touch fewer users, but each of them can book again next week because they want to, and the training session is the far better target.
A two-by-two chart of what makes a good target event. The vertical axis asks whether the same person can choose to do an action again, and the horizontal axis shows how common it is. The top band, labeled Strong target, holds booking a training session (marked with the orange star) and making a purchase. Bottom left, Supporting signal, holds completing onboarding. Bottom right, Trigger, holds a subscription auto-renewing.
Whether a person can choose to repeat an action decides what job it does. How common it is matters much less.
The candidate target event they had in mind passes the test, because a person chooses to do it and can choose to do it again tomorrow. The repeat rate then tells you how often that happens, which is a measurement question about whether agents will see enough repeats to tell if a message moved anything. Our guidance treats it as a floor, and a low one. A rare action that people choose freely usually clears it once you look at the right group. So a result in the mid-teens is a reason to ask who repeats and who doesn’t. It is also a reason to check whether the group you measured is the group the feature serves. On its own, it is a weak reason to drop a use case.
When the value is one step removed
The harder part of the first question was the indirect path. Using the feature helps someone reach the core action, but only later and only some of the time. A reward function written for the whole product won’t credit that engagement at all. That leaves the feature without a target event of its own.
There are two sensible routes from there, and they can run together. The first is to target the most repeatable, deliberate action the feature produces, so the downstream goal rises behind it. Teams make the same move when a goal like upgrading or booking a trip is too rare to target directly. The second route is to keep the feature’s engagement events flowing as supporting signal. Agents give that kind of event partial credit, so they can learn from everyday behavior while conversions are still some way off. Once events are flowing into Aampe by MoEngage, the team can trace which earlier actions actually feed the goal. With that, the customer’s own data can settle the choice.
Signal density is a weekly count, and it takes weeks to read
Signal density sounds technical, but the idea is simple. It is a relative measure of how often each person produces events in a week. For the account that asked, it came in low. Across the other accounts we have looked at so far, it hasn’t been a problem. A customer can calculate it from their own data models, since it is just their own events per person per week.
The idea that more messages will produce more events is reasonable. It is also something nobody can confirm in the first few days. Agents typically need two to three weeks to burn in. The team’s plain answer at the time was that they would not know before then whether low density was hurting learning. They planned an early, preliminary read on the sends-to-events question the following week.
Raise target event density by sending more of the journey
It also helps to be precise about what raising density means. More events only help if a person chose to produce them. Some events fire hundreds of times a day from a background sync or a screen redraw. They bury the handful of real decisions under machine noise, so they can carry less usable signal than a rare event.
Coverage is the better lever, which means sending the full event stream, including small actions that feel too minor to matter. Only true duplicates and pure system noise should be dropped. The aim, to borrow the title of a certain Nathan Fielder documentary, is an agent that can see everything. Here that means every action a person chose to take, which is a less dramatic premise but a far more useful one. An agent that only sees the big outcomes is like a basketball player blindfolded except at the moment the ball goes through the hoop. The player knows they scored, but learns nothing about how they got there.
Where to start with target events
Both teams were asking the right questions early enough for the answers to matter. The decision that does the most work is which few events to point your reward functions at. A good test for each candidate target event is whether a person can choose to do it again. After that, make sure everything else on the path reaches the agents, so they can see someone moving well before they arrive.
