Guy Reams (00:01.102)
This is day 345, the concept of monotonic lift.
One of my many cross-country travels, I ended up in a non-Mariott facility. This is a rare occurrence, but in this instance, I had no choice and my loyalty was temporarily shifted to another large brand hotel operator. The next morning I woke up and went to take a shower. I was not used to the tub and shower faucet mechanism. I had learned the various Marriott brands, so I knew without thinking how to operate their showers.
However, this one had me stumped. And the reason why will explain this concept of monotonic lift. As I lifted the shower handle upward, my expectation was the temperature would change accordingly. As I slowly lifted the handle, the water grew warmer. This remained consistent. However, when I got to about three quarters of the way up, the water started to turn colder. When I went back down, the water got warmer.
But when I moved it up again, there was a sudden drop in temperature. This was not monotonic at all. It was unpredictable. And I felt confused and did not trust this new shower faucet contraption. If there had been a monotonic lift, then I, as I lifted the handle, the temperature would have risen steadily along with the handle lift. In essence, monotonic lift is predictable.
You know the effort it takes to change or improve results. When you slide the dimmer switch up, you expect the light to get brighter at a steady rate. In most businesses, a lift that is predictable in nature across segments and markets is more valuable than any one growth metric. Predictable growth almost always beats the unpredictable spike. The point is that once you figure out a formula that consistently improves results,
Guy Reams (02:00.814)
and that can be replicated in both a controlled scenario and multiple other scenarios, you have something a business person would call valuable. Often this concept of lift applies to buckets or different measurements. The lift will be different depending on the bucket that you are considering, but the lift will follow a consistent pattern nonetheless. The lift may be greater or smaller depending on the bucket, but the ratio will remain largely the same.
This is the holy grail of metrics, the one that performs the same way even when working with different buckets or populations. Often as you work your way towards more efficiency, you look for metrics such as these because once identified, you can start to rank buckets and set thresholds for them. A useful case would be the largely elusive TTFV metric.
This is called the time to first value. This is an important metric because marketing research has shown that the time it takes for a potential customer to reach the first realization of value has a direct correlation with the conversion rate of users becoming paid subscribers. The goal in this instance would be to get a consistent conversion rate across several time duration buckets. Customers who reach TTFE in less than 15 minutes
should have or will have a higher conversion rate, say, let's say 25%. Users who take longer to get to value, like let's say a 24-hour TTV, TTFV, will have a conversion rate of, let's say, 5%. As you see this fall off, it should fall off consistently across all of your buckets. When you achieve that, you know you have a predictable TTFV expectation.
Just as I expect the water to get hotter as I pull the lever upward, you would expect that the shorter time people take to experience value with your product, the higher the conversion rate will be. As you monitor this, you can quickly understand when something is out of alignment. When the lift is not predictable or shows a significant variance from expected results, you can look at noise or potential confounding issues
Guy Reams (04:22.306)
that might be distracting your user population from realizing value faster. If the results you are getting are not monotonic, it is not always a new feature that you need. Often it is better to improve the sample size, look at different timeframes, or address things that might be confounding the people in a particular bucket. Unintended crossover often impacts TTFE, and you may not even realize it's happening.
This is why a monotonic metric, once established, is highly valuable, like you might find a canary in a coal mine to be valuable.