How to Tell Whether an AI Agent Is Still Exploring

Grid of user windows with evenly split bars, one window with a single tall orange bar and the Aampe star

Two people signed up for the same app in the same week. Each one has an agent, a small piece of software that learns what works for that person and picks their messages. Both agents choose the tone of each message from three options: warm, direct and playful. After a few weeks, the first person’s agent would pick direct about nine times out of ten. The second person’s agent still splits close to evenly, about a third each. Same start date, same content to choose from, and two very different pictures. That gap between the two people is where AI agent learning shows up, and the calendar will not tell you about it.

“Is it learning yet?” is the first question almost every team asks a few weeks after agents go live. We hear it from subscription apps, finance apps and marketplaces, and it usually arrives as a request for one number. The useful answer is one small picture per person, and once you know what to look for, AI agent learning is easy to read.

Start with how a message is built

To read that picture, it helps to see what an agent is actually choosing between. Take a simple push notification, the kind a meal-planning app might send: “Hi Sam, dinner sorted in 15 minutes. Tap to see tonight’s pick.” It reads as one message, but it is built from parts. There is a greeting, a reason to care (saving time) and a tone (friendly and direct).

Each part has a few versions a marketer has written and approved. The reason to care could be saving time, saving money or trying something new. In Aampe by MoEngage, each part is called a component. Each version gets a label that says what it means, like “saves time” or “warm”. Agents learn at the level of those labels. Over many messages, an agent works out which labels move each person to act. It measures that against what the person would have done anyway.

Exploring and exploiting, in plain terms

Think about a new barista at a café you visit often. In the first week, she suggests different things and watches what you go for, which is exploring. Once she notices you always come back for the same order, she starts making it before you ask. She still suggests something new now and then, in case your taste has changed. That is exploiting, acting on what she has learned while still checking now and then.

Agents do the same thing, and AI agent learning leaves a trace you can see for each person. For every person, we can estimate how often their agent would pick each label within a part. The technical name is label-selection propensity, and all it means is how often the agent would pick this. If tone has three labels and the agent knows nothing about someone, each label sits at about a third. Once the agent has learned which tone moves that person, one label climbs well above that and the others drop.

 Two bar charts showing AI agent learning for two people: one agent picks the Direct tone about 90 percent of the time, the other splits about evenly across three tones near chance

So a person at 90, 5 and 5 has an agent that is mostly exploiting. A person at 35, 35 and 30 has an agent that is still exploring, because it has not seen enough to prefer anything yet. Schaun Wheeler explains the machinery behind this in his post on how agents balance exploration and exploitation. Each agent starts out unsure about every option and becomes more certain as evidence comes in.

Count the chances each agent has had

The barista cannot learn your order if you have only been in twice, and agents are no different. They learn from each message they send and what happens afterwards. So the number that matters for AI agent learning is how many messages each person has received, far more than weeks since launch.

Most teams limit how often each person can be messaged, which slows this down more than people expect. In one recent account, the typical person had received around eight messages by the time the learning question came up. There were also around eight different reasons to care to try. So each agent had roughly one look at each option. What looked like a slow start was the agents working through every option once, which they have to do before they can compare them.

This also explains something that confuses people early on. Every time an agent sends a message, it has to pick something, so it picks its current best guess. Early on, that guess is often an option the person has never received. An untried option is the one the agent knows least about. One pick is one guess for one message. It says little about what the agent has concluded.

Judge only the options a person has actually seen

Before an agent has tried an option with someone, it still needs a starting point. It borrows one from what has worked for similar people. The new barista does the same when she assumes you want what most customers order. The post on how learning systems actually behave explains why this matters. The borrowed starting point keeps early choices sensible, and the person’s own behavior takes over as it builds up.

That starting point comes from other people, so it says nothing about this person. Depending on the business, it can come out generous or cautious. A tempting check, the share of people whose favorite option sits above a third, can mislead for exactly this reason. Early on, nearly everyone clears it, often on options they have never received. The more useful read of AI agent learning is how clearly each agent prefers one option among the ones that person has actually seen.

Track AI agent learning by who moves between groups

Most teams still want something they can watch week to week, and there is a good candidate. Every person sits in one of three rough groups. Some are new to their agent, so it leans almost entirely on what works for similar people. Some are partly known, so it has real evidence on a few options and borrowed starting points for the rest. Some are well known, so it has tried enough options to rely on that person’s own history.

Three stacked bars for week 1, week 4 and week 8, with the share of people the agent knows well growing from a small slice to the largest group, an illustration of AI agent learning over time

The healthy pattern is movement, with the well-known group growing over the weeks. The agents for those people should also get more decisive about the options they respond to. That is AI agent learning you can show a leadership team. An average lift figure tells you what happened across the crowd, while this tells you how much the system knows about each person.

One more detail helps when people look at individual users. The agent does not remember that someone opened one particular email. It keeps how that person reacted to the labels on that email. That carries over to any future message with the same labels, in any channel. This is why well-labeled content matters so much. An agent can only become sure about an option it has had the chance to send.

Neither agent is broken

Go back to the two people from the start. The first agent has found something about that person and is acting on it. The second is still asking, and will keep asking until the answers separate. The useful question for a team is how many people look like the first one, judged on options they have actually received. Then watch whether that group grows every month, because that is what AI agent learning looks like from the outside.

More conversations like this are on our YouTube channel.

Table of Contents

Related Posts