The Danger with Numbers
Once when I was working as an engineer at a large software company, my team received a report that a serious bug had been discovered in a feature we had launched several days prior. Unfortunately this wasn’t a bug that would show up in our automated alerts and it was only reached by following a specific and unusual sequence of steps in the app. As a first order of business, I began to spot check a few user accounts to see if they had been affected. I had a hunch about what the problem was and knew that if I could confirm that hunch, I’d be able to then run a query to determine how widespread the bug was across our systems. From there I could then triage the issue with my team and work on a solution to fix or at least mitigate it.
As I was doing so, a Product Manager (not from our team) came to me and asked in an hurried voice if I knew what percentage of users had been affected. I responded that I did not, and that it would take some time to properly ascertain that. Apparently unsatisfied with that answer, this PM pressed me again to ‘at least give an estimate.’ I repeated that I didn’t have an accurate estimate yet and that we needed more time to diagnose the problem. The PM would not yield and insisted that we give them some number to go off. At one point I think they actually said “just give us some number to work with.” Exasperated at this point, and just wanting them to go away, I said well I’ve tested six devices and I’ve only encountered the bug one time. Before I could qualify that any further, the PM dashed off. I didn’t reflect too much on the conversation at the time because I wanted to get back to investigating the problem. Later that afternoon, with the bug still not resolved, we had a cross team meeting to discuss what we had found. To my horror, the second slide on the powerpoint deck led with a bullet point asserting that “our engineers have determined that the bug is affecting 17% of users.” My highly anecdotal and entirely unscientific finding of 1 in 6 had been massaged into a hard, crisp, and confident ‘17%.’ No context was included for how that number was arrived at, nor any qualifiers to convey how confident (or not) we were in its accuracy. As it turns out, the real incidence rate ended up being much lower than that, and quite a lot of panic was generated for no reason.
I don’t fault that PM — I’m sure they were in turn under pressure from one of their superiors to come up with a concrete figure. In fact I slightly fault myself for not putting my foot down harder on refusing to give a number that was inevitably going to be meaningless. But I thought about it more. This wasn’t the first time I’d seen numerical values (principally percentages) used misleadingly in this way in the corporate tech world. Whenever I’ve tried to protest this, I’m told ‘well we know the number isn’t exactly accurate but it’s better than nothing.” But is it better than nothing? I’m not so sure. We humans don’t have a very impressive track record when it comes to rationally interpreting numbers. Very large numbers in particular we have trouble making sense of. Cognitive scientists at Indiana University once did a study where they asked participants to draw a line on a sheet of paper that represented the numbers between 1 and one billion. They were then asked to place a tick mark on the line where one million should appear. About half of the participants placed it somewhere in the middle.
But my bigger problem with numbers is that they belie the fuzzy, imprecise and all around messy world from which they come. We tend to think of numbers as these precise, platonic forms, almost divine in their origins; and that to question them is to question truth or perhaps even reality itself. We even have a saying “numbers don’t lie” — which I guess is technically true but sort of meaningless. It’s like saying “verbs don’t lie.” Numbers are just referential symbols. They can’t lie not because they are infused with any angelic agency, but because they don’t have agency at all. People on the other hand do have agency and people do lie and often do so very effectively by using numbers — some fiddled, some presented without context, some just entirely made up. The mere presence of numbers themselves doesn’t tell you anything about the truth value of a particular statement.
Numerical estimates presented without an error rate are particularly deceiving, to the point of actually being worse than nothing. In fact, often the error rate is the more important metric than the estimate itself. Nicholas Taleb makes this point in his seminal work, The Black Swan, when he says:
“Don’t cross a river if it is four feet deep on average. You would take a different set of clothes on your trip to some remote destination if I told you that the temperature was expected to be seventy degrees Fahrenheit, with an expected error rate of forty degrees than if I told you that my margin of error was only five degrees. The policies we need to make decisions on should depend far more on the range of possible outcomes than on the expected final number.”
In the case of how prevalent the bug I mentioned earlier was, the error rate, or confidence interval, would have been very wide indeed. Most real scientists and statisticians, of course, already understand this, which is why you’ll see their academic papers festooned with p-values, confidence intervals, and hefty doses of qualifying and cautious language. They know that numbers are powerful but often radioactive tools, and so go to great lengths to encase and tame them with statistical techniques that temper their power. Lay people though, and, in my experience, especially the tech industry, seem to have discovered the power of numbers but not the tools by which they can safely be used. We are like kids playing in a chemicals factory without protective equipment. In most cases I think we’d be better off just sticking to good old fashioned prose and using numbers only sparingly.
Tech culture often looks down on prose, regarding it as too ‘fluffy’ and inexact, but in fact I think this precise inexactness is its great strength, since it better captures what is often our own foggy understanding of a problem. Describing a problem with human language, rather than with mere numerical symbols, often allows you to better convey both your own doubts about the situation, and the relative importance of what being wrong would mean. Human language is an extremely sophisticated technology — in fact it’s the most mature piece of technology we have — and it has been refined over millennia to be incredibly robust. Everyone is an expert in it, and listeners can therefore detect traces of nuance or subtlety that would simply be lost in a barrage of numbers and percent signs. Like any mature piece of technology, prose also tends to have lots of built in redundancy, such that changing a single word or several words in a paragraph often does not drastically change the overall message, in a way that changing a single number can. To return to our original example, an alternative way of presenting what we knew at the time might have been:
“We’re still unsure as to how widespread this bug is. So far we have only found one case of it in the six devices we have tested.”
See how hearing the raw counts leaves you with a much different feeling from hearing the percentage? You’re suddenly up close and personal with the issue, not surveying it from 30,000 feet with a telescope. Yes this phrasing is more verbose, but often verbosity is the price for accuracy. Note that we still use numbers but, firstly, we don’t lead with them, and secondly, we don’t map them to percentages, a move which is really not justified given the very low sample size we are working with. Once you start using percentages, people somehow get it into their heads that you must be drawing from a giant, randomly selected pool of data, and begin to attach far more confidence to the figure than they would do if they merely heard the raw counts. Einstein’s quote bears remembering here: “Make things as simple as possible, but not simpler.” The “not simpler” part is often forgotten.
I’ve always loved mathematics and how it allows us to express concepts that exist beyond our sensory world. I agree with Paul Erdős that “if numbers aren’t beautiful I don’t know what is.” They are a way we can express abstract truths. But they will only serve that purpose as long as people use them conservatively and honestly, rather than just as decoration put on to provide a patina of credibility. Otherwise rather than clarifying and furthering our understanding of the world, they will confuse and distort it.
