“Check out the new SWISSGEAR Scan Smart TSA Lapto…”
Most consumer retail apps send a lot of generic notifications like this:
…and one reason they do is because the product names and descriptions stored in their CMS are super long and clunky.
For example, here are two relatively random items in the Walmart app (left) and the Target app (right):
…and just so it’s clear we’re not picking on any particular app, here is a sample of Product (Item) Names from major retail apps:
One of Aampe’s customers is a large retail app. We’ve been helping them move from the industry-standard style of short and generic notifications to targeted notifications that reference specific, recommended SKU/items.
A concern they had as we began the process was that their product catalog and Content Management System was not designed for notifications or short-form content. Like the examples we just looked at, the product names render fine in the Desktop browser window UI they were designed for…just not in notifications.
The CRM team was afraid these long item names wouldn’t fit in a notification and the result would be ugly and ill-formed notifications that would lower their open rate.
Like all CRM teams, they are not responsible for their app's product catalog and they don't have much control or influence over it. Moreover, they didn’t have the time or energy to make all of the product names shorter and more compelling to fit a push notification format—They could do that for 10 or 30 sure, but that wouldn't be very specific or targeted given they have millions of users.
All of a sudden the effort involved seemed overwhelming and they considered pushing it off to the next quarter.
So, here’s the question:
Is an overly long notification worse than a generic one?
If it is then we can just stop the exercise and keep sending our short, generic messages.
…but if specific + long is better than generic + short, then we’re losing value every day we send generic notifications when we could be sending specific ones. And we’re not just missing conversions, we’re also wasting our customers’ time and screen-space as well.
Thankfully, the CRM team was willing to implement the experiment, as they recognized that without trying there can be no learning. By learning fast we can try a lot more things, knowing that we can very strictly limit downsides (by stopping the thing that’s causing a problem) while massively exploiting the upsides (by scaling up the things that are bringing in big wins). This is why Aampe massively parallelizes notifications and continuously optimises based on that parallelized experimentation.
Our customer’s product catalog had nearly 2 million inventory items—The shortest item name was just a few characters, while the longest was just over 250 characters.
This plot shows a cumulative distribution of item name lengths (You can see that by the time we hit items with a character length of 50, we’ve already covered 80% of the product catalog):
By the second week of running the recommendations we could see significant performance improvements (we’ll write about those in a later post!), but our focus for this discussion was whether we could detect any damage caused by long item names.
Is it true that long or truncated item names hurt push notification performance?
We launched the recommendations and monitored closely. After over 6 million push recommendations, we were pretty confident in what we were seeing. The plot below shows the distribution of the length of the item’s name split by if the customer added the item to their cart or not (The box plot on the far right re-shows the distribution of item name length for the entire population of items).
…and here’s the result: They’re the same.
The outliers are a function of the larger population of recommendations that did not lead to an Add to Cart event (and the very small number of items - <1% - that had a name longer than 100 characters) . So let’s remove to make the picture a little clearer:
I have to admit I was surprised—I would have expected to see a small performance effect for shorter item names (assuming that shorter names are cleaner), though I expected the performance hit of longer names wouldn’t be near as large as the lift created from sending an item recommendation. But there was no performance hit for longer items.
That doesn't mean the item names can’t be cleaned and improved (There are various text transformations that can be run, including more naive truncation methods. And we’ve already put one into production). Maybe we’ll write about that in the future if we see some interesting performance effects.
But the lesson for today is: Try things
Don’t wait for perfection and don’t wait until you’ve eliminated all risk. If you do, you’re missing out on real value, and your customers are too.
What questions about push notifications do you have? Drop me a line and I’ll try to find the answers.
