Understanding and working with AI agents is going to be one of the most transformative technological shifts in human history. As humans, we have a long tradition of anthropomorphizing the world around us, projecting human-like qualities onto deities, animals, weather, and even social constructs. Historically, this has often been a flawed exercise, as it imposes human cognitive frameworks onto systems that operate in fundamentally different ways.
But with AI agents, particularly agentic learners, anthropomorphization might actually serve a practical purpose. Not because these systems truly learn the way we do, but because the language we use to describe them helps us bridge the gap between human cognition and machine capability. In a way, we’re hacking our own understanding: using familiar human-psych terms to make sense of something that is distinctly non-human, yet capable of synthesizing concepts in ways our own brains cannot. By leaning into this framing, we don’t just clarify how agents function, we gain new insights into our own learning processes and how we can use these systems to better understand each other.
We’re talking about changing agents’ beliefs. Specifically, we’re talking about when you might want to manipulate an agent to believe that two apparently distinct concepts are actually the same thing.
Part 1: what it looks like when an agent believes something
Before we start manipulating our agents, let’s get a sense of how their brains work.
Aampe agents make decisions along many different dimensions, but I’m interested today in how they make decisions about their user’s copy preferences. Speaking anthropomorphically, the agents feel out things like what kinds of tones, value propositions, or incentives resonate with their user by trying a few different types of copy and zeroing in when they find something a user seems to like. (While they’re at it, they’re always paying attention to when a user might like a different kind of copy more, or when a user’s preferences change).
Let’s open up this artificial brain and see how it does this. We simulate the psychological process I just described by using a variation on Thompson sampling, which is a decision-making algorithm used across a variety of machine learning problems to intelligently search for the contexts that provide the optimal reward across several context options.
An agent’s belief about a user’s preference for any given copy label is operationalized as a beta distribution, where alpha and beta parameters are derived from weighted event counts after and before (respectively) all of the messages that bore that copy label. When an agent is trying to decide between several copy labels, it makes a random draw from the distribution of each copy label, and the high draw wins. The shape of the distribution for each copy label influences how likely that copy label is to win over the other ones - or put anthropomorphically, what the agent believes about each copy label helps it decide which label it should send the user next time around.
A concrete example:
A few things are notable right off the bat.
- The central tendencies of Convenience and Ease of Use are a bit higher than chance, and the central tendency for Luxury is quite a bit lower. We might think of the mean of the distribution as the expectation - “what” Agent X believes about how User X sees a copy label. (The mean of the beta distribution is + ; essentially, the percent of the time out of all of the trials that you got a success.)
- Agent X thinks User X likes both Convenience and Ease of Use okay. (And it thinks User X likes them both about the same amount.)
- Agent X thinks User X is none too jazzed about Luxury.
- We can think of how “peaky” a distribution is the confidence Agent X has in its belief. In a beta distribution, the kurtosis of the distribution varies with the amount of signal that defines the alpha and beta parameters. Recall that alpha and beta parameters are just counts of events (after we’ve chopped them up and reconstituted them a bit). The more events we’ve got, the more evidence we have to look at when coming up with our beliefs. A more platykurtic (flatter) distribution represents less certainty, because there are fewer events; a more leptokurtic (peakier) distribution represents more certainty. Confidence matters because it reduces the exploration that an agent tends to engage in. It’s pretty sure of its belief - no need to explore.
- Agent X is relatively confident in its belief User X liking Ease of Use. (This makes sense - Agent X sent its user Ease of Use messages about 2.5x as many times as either of the other two, and it got similar results each time.)
- Agent X is not terribly sure about its beliefs around User X’s preferences for Luxury and Convenience. Gun to its head, Agent X would say its user preferred Convenience to Luxury, but look at how broad those distributions are - can’t be super certain.
In other words: We’ve come up with a few principles about how agents think about their beliefs. We know how to derive an agent’s expectation (higher beta distribution mean = higher user preference for this label!), as well an agent’s confidence in its belief (more leptokurtic = more confident agent!).
Side note: This is not actually that distinct from how we, as humans, use concepts to form beliefs about the world in general. Take the well known visualization of perceptions of various terms that convey probability. Some people are a lot more confident when they say “almost certainly” than others, but almost all people that use the term “almost certainly” use it to convey a lot more confidence than the term “probably”.
Good job, you just finished your agentic neurosurgery rotation. Now scrub in, we’re going to operate.
Part 2: when and how to change an agent’s beliefs - and how to decide if you should
Why?
Where do agents get the labels to form beliefs around? They get them from the human user of Aampe - the business’s marketing professional.
Humans use their expertise about their userbase to come up with copy labels that encapsulate what they think matters to all their different users. The agents are the ones who help the humans get a deeper view into their userbase, but it’s the humans who have to start out by telling agents what kinds of options are on the table.
Here’s the thing - sometimes the human operator wants to change their label taxonomy. Maybe you’ve been using Aampe for a couple of weeks, look back at your labels, and realize that you have two labels that really get at the same underlying user preference - like “Convenience” and “Ease of Use.” Should you merge them together? And if you do, what will happen to the things that the agents have already learned based on those old labels?
In general, it’s beneficial to be precise with labels. You want to do your best to encapsulate the breadth of user preferences in your userbase in as few labels as possible. Intuitively, if your agent has fewer places to look, it’s going to find the best place more quickly. (More formally, a smaller exploration space in a multi-armed bandit problem tends to minimize overall regret.)
Basically: if you’ve got two labels that you have reason to think are really getting at the same thing, you’re going to do yourself and your agents a favor by combining them.
How?
As with actual biological brain surgery, the procedure itself is trivially simple, which is why everyone is doing brain surgery all the time…or would be if humans didn’t involve so much blood.
Remember that the alpha and beta parameters the agents use are pseudocounts of events after all the messages with a certain label (analogous to “successes” - reasons to think the message moved the needle on the user’s engagement) and events before all those same message (analogous to “failures” - reasons to think the user would have been engaging that much around the same time anyway).
Instead of grouping by the two parent labels and summing over all the messages with those labels, we can group by the new label instead. That’s all there is to it.
It’s as if you’ve erased the memory in the agent’s brain of Label A messages as being different from Label B messages, and replaced all those memories with the idea that it was simply always testing how messages with Label (A+B) tended to perform.
Let’s take a look at this example with User X, and see what happens when we perform this operation on Convenience and Ease of use. When we merge these two labels by summing the original parameters, we arrive at basically the same belief about how much User X likes the new merged label - which makes intuitive sense, because User X liked both of the original labels about the same.
Merging labels like this will have some consequences, though. I said it was easy, not that it was necessarily always a good idea. So how do we know whether or not to do it?
Whether or not?
First, since we’ve already got the agent on the table, let’s see what kind of beta distribution (“belief”) is demonstrated by the label we just made.
Let’s go back to the principles we learned earlier. First off, the central tendency of the new label is the same as the central tendencies of the parent labels; in other words, the agent’s expectation hasn’t changed. (We saw this in the table above, even before plotting the distributions.)
This feels intuitively right for this user. User X liked Ease of Use and Convenience about the same. If we’re brainwashing the agent to assume all messages under Ease of Use and Convenience were always the same thing, we wouldn’t want the agent to suddenly believe that the merged label hit User X differently than the parent labels.
Secondly, the distribution has become more leptokurtic; in other words, the agent is more confident in its belief about this merged label than it was in its belief about either of the parents. Think about it in anthropomorphic terms. The agent’s new understanding is that instead of 10 messages about Ease of Use and 3 messages about Convenience, it’s actually looking at 13 trials of one Convenience-Ease-Of-Use label. More trials means more evidence, which means more confidence.
A similar thing would happen if we simulated the agent seeing the exact same stuff about one of the original parent labels two times. If you’d gotten double the pseudocounts for Ease of Use (for both parameters), you’d see a version of the Ease of Use distribution with the same mean, but more leptokurtic. The agent holds the same belief but with more confidence, because it ran the same experiment more times and got the same results. In principle, that’s what you’re doing when you’re merging the data from two labels that are, at their core, the same thing.
But that’s only when labels are in fact, at their core, the same thing. Let’s open this agent’s brain back up again and make them think Ease of Use and Luxury are the same thing.
The parent distributions here had very different means - the agent believed two very different things about the user’s preference for each label. It had different levels of confidence about them, so its new belief is more swayed by Ease of Use than by Luxury, but at the end of the day, the new belief centrally tends towards chance.
It’s also more leptokurtic, because we are still giving the agent more evidence to work with. But greater kurtosis just means higher confidence. If the belief itself is chance, then increased confidence just means the agent is quite sure that something is unpredictable.
After all, the more times you flip a fair coin, the more you can be sure it’s fair.
In sum: whereas before Agent X felt pretty darn sure that User X liked Ease of Use (and sort of sure that User X disliked Luxury), now Agent X just feels quite sure that User X’s feeling about the imaginary Luxury-Ease-Of-Use label is totally unpredictable.
Why (redux)?
What we’ve really uncovered here is that while it’s not too hard to manually change an agent’s beliefs, you run the risk of the agent losing a lot of useful information about a user.
In cases where an agent is forced to believe that labels on either side of the chance meridian are really the same thing, you’re going to bring your agent to a state of confident uncertainty. If that’s true across enough users, the new label itself will become pretty useless - and rightly so, since the psychological underpinnings of the two parent labels are probably pretty different, if they caused such different responses in so many users.
On the other hand, in cases where the labels are similar to each other, you’ve given your agents a boost in confidence and a more straightforward belief system - which will make them more able to efficiently perform their jobs of getting to know their user.
So when do you do it? At the end of the day, it boils down to what matters to you as a business (the Aampe customer). The human operator has to be comfortable with the number of users who would shift their beliefs dramatically following a merge - not the agents’ confidence in the beliefs (since that will always go up in a label merge), but the actual beliefs themselves.
You might want to assign a cutoff percentage for the number of users who would end up with a merged label score closer to chance than to either of their scores for the parent label. You might want to identify the percent of users who would shift their action score by more than 0.1.
You might also decide that you know best. If the merge comes as part of a larger label taxonomy restructuring, or if you add more variants to the labels (changing the content can sometimes change the “meaning” of the label, too), you might take numbers like these as just a warning to proceed with caution.
After all, you didn’t go to four years of agent medical school to be called Mister or Miss. You read a blog post, and that told you all you needed to know about what can happen when you perform brain surgery on your agents. Our main job as the human side of Aampe is to give you as much usable information as possible to inform how you conduct your agent orchestra.
The Brain Surgery Afterparty
So there you have it—you’ve just performed brain surgery on an AI agent. You poked around in its neural pathways (or, more accurately, its probability distributions), reshaped its beliefs, and hopefully didn’t lobotomize its ability to help you make smarter marketing decisions.
If this were a real operating room, you'd be scrubbing out, tossing your gloves in a hazardous waste bin, and dramatically removing your mask to deliver some profound final words to your team. Maybe it’s not going to be as profound-sounding as the aftermath of a human surgery, but you’ve definitely got some wise words: if you merge labels that are fundamentally different, you’re not refining knowledge. You’re making your AI deeply confused.
So, as you wield your scalpel (or, let’s be honest, your trackpad) remember: Precision is key. Sometimes it’s worth it to fine-tune your beliefs. Other times… maybe just leave your agent's brain alone.
.png)
