A walk through our conditioned experimentation process
We explained our approach to designing “conditioned” experiments in a previous post. Briefly: humans are complicated so if you want experiment results that don’t lie, you have to assign your treatments that take that complexity into account. The traditional approach of just randomly assigning treatments only works if you have a huge sample size – […]
The retention paradox
Businesses need customers. That’s so obvious it hardly needs saying. Businesses grow through acquisition (new customers give you a try) and retention (past customers come back). That’s also obvious. Businesses grow most – both in terms of speed and size – when they focus most on the customers they already have. Maybe that’s a little […]
Why you need conditioned experiments
An A/B test is perhaps the most basic form of an experiment: take a group of people, randomly split them into two subgroups, send a different message to each group, and look at the difference in response. A/B tests are easy and widely available. They are also a really easy way to get misleading, and […]
Machine learning isn’t enough
I’ve worked as a data scientist for over a decade, much of that time spent on training, testing, and calibrating machine learning models to address specific business problems at various scales. I’ve done a lot of data engineering, and data product management, and a whole lot of plain-old, business-intelligence-style analytics, but more than anything else, […]
Change is your business
There are many names for the people who keep a company in business. User, customer, buyer, seller, member, player, contributor, among others. We’re going to use “consumer” because a single word makes things simpler to talk about, and consumer is a very broad word: even if you’re using a company to sell your product or […]