When I was in grad school, many years ago, my research supervisor and I debated the differences between precision, accuracy, and reality. Old Dr. Clark and I did not see quite eye to eye. We hashed out the effect of imperfect data on the design of our new measurement tool. This was an esoteric discussion. We debated about variance, standard error and confidence.
Dr. Clark taught me more through that discussion than he could ever have imagined. The lesson was indelibly burned into my outlook on my profession and my life, but I thought it would end there. I never dreamed these concepts had any meaning beyond the arcane field of aerosol measurement. But then, along came climate change! Now I find myself, some forty years later, confronting the same issues in my climate resiliency work.
Precision, Accuracy and Reality Are Different
Why am I obsessed with the difference between precision and accuracy? Is it because I am a statistics nerd, or perhaps I long for the good old days, back in grad school. Actually, I find more and more often that the distinction causes confusion and angst in our day to day work on climate change. Folks confuse the two and often exploit that confusion as a lever to undermine productive discussion. In fact, they often use it to dispute that climate change is even happening. I worry about this and seek ways to cut through the confusion to get to a meaningful dialogue.
The distinction centres on three concepts, reality, our ability to generate data that reflects reality (accuracy), and our ability to reproduce that data over many tests (precision). Of these three reality is seemingly the easiest to grasp. We all know the real world we perceive through our five senses. It is concrete. We can touch it and we can smell it. The trouble arises when we try to measure reality with our rulers or, these days, with our computer models and algorithms. When we do this, do we generate a true picture of reality?
Understanding Precision and Accuracy – An Analogy
Over the years, I realized that these concepts are best understood through analogies. My favourite is tossing five darts at a dartboard, trying to hit the bullseye. When the average of all five darts is near the bullseye, they are accurate. When the darts all cluster around the same point they are precise. But, often precision and accuracy do not coincide. In an ideal world, with a great dart player, the darts will be both accurate and precise, but this outcome demands practice. It is the same in the scientific world. Achieving both precision and accuracy requires work.
Precision, Accuracy, Reality, and Climate
Climate is what we expect, weather is what we get.
– Mark Twain –
So where does the discussion of precision, accuracy and reality take us in the climate debate? While we experience weather, we measure climate. Climate is the data that we use to describe weather. It is full of averages, maximums, minimums and other, arcane statistics.
Good data about climate is both precise and accurate, giving us a clear picture of what we expect to see. But, just as with the dart player, getting consistent and reliable results takes a great deal of work. Further, the weather we feel on any given day, is only one dart of multiple darts included in the climate statistics.
This is where the confusion between accuracy and precision creeps in. When we experience a bout of cold weather, we often hear folks say things like:
So where is all of that global warming now?
This statement confounds three concepts. It confuses one or two days of cold weather with many years of temperature measurements. It imposes a demand for accuracy in the climate data we expect of few other things in life; the ability to predict weather with complete certainty. Also, it presumes that data that does not accurately predict one day’s outcomes is, by definition, bad. Last, but not least, the argument concludes that since the data is bad, climate change is bogus. There are a lot of logical leaps in this chain of thought, most of which cannot stand up to deep scrutiny.
The “Precise is Right” Fallacy
Frequently, I have seen climate specialists cite the low variance of their work as a sign of the quality of their data. However, being able to reproduce the same result only tells us that the measurement is consistent. It can be consistently wrong. So, pointing to the precision of our data provides no overall assurance of the truth of our work.
Several undesirable outcomes arise from this fallacy. First, we know that we cannot predict weather with five decimals of precision. We never hear that the high tomorrow will be 14.14325 ℃. Correctly, folks would say:
That is ridiculous!
However, we have frequently seen climate projections that cite these levels of precision. We know that it does not reflect reality and the sceptics can point to that fact to undermine the value of the projection. Thus, while the data is precise, they argue that it isn’t accurate. Citing precision to defend the accuracy of the data is a fool’s errand.
Leaning on the precision of our data sends a message to the average person that we are out of touch with reality. This is literally the opposite of our expected “rational” response to the argument. They conclude that if the date, and the folks who generate it, are out of touch with the real world…
The modelling must be wrong!
In our work we have had the honour of working with many climate specialists. They are dedicated to their work and they are legitimate experts in their field. Their work is sound, and indeed, it carries the level of precision they cite. It just answers a different question than the one that the average layperson is asking. We need to recognize that hurdle and answer the questions that they ask.
Precision, Accuracy and the Real World
If the precision of our work doesn’t win the day, what does? We find that folks are much more easily convinced if the work reflects their knowledge of reality. That does not mean that we compromise the integrity of our work to make our case. Rather, we need to show the work actually can predict observable events. This is the basis of sound science. In science, we observe the real world, postulate explanations for what we observe, devise ways to measure and predict those observations and then test our work against further observations. We constantly revise, based on new observations or identified weaknesses in our earlier postulates. That is science. Science is not about certainty, it is about our ongoing mission to describe the real world.
The same applies to climate change work. We observe the weather record, postulate why we observe the trends, work out ways to test those postulates, refine, and move forward. Over the last thirty years I have observed continual growth in climate science’s ability to predict accurately things we see in the real world. In this case, accuracy is king. Over the range of climate models, we can project with relative accuracy the impacts of climate change. Is the science perfect? No. Science is never perfect. But, the method works. If we demand perfection of climate science, we require levels of precision and accuracy we rarely impose on any other activity in life.
Understanding Precision and Accuracy Helps Manage Uncertainty
We work with uncertainty in pretty much everything we do. Our plans are not perfect. We accept that we can make adjustments along the way, and with constant vigilance move forward. Once again, this isn’t perfect, but it works.
The key to the precision, accuracy and reality question is constantly testing our modelling efforts with real world observations. This not only improves the science, it is a much more rational argument for our day-to-day debates on these critical issues. Will we convince everyone? Of course not! Once again that demands a level of perfection we expect of few other things in life. But, we can move the discussion forward, based on sound science, and a very strong dose of pragmatism.
Call to Action
You are not alone. There are folks here to help you out. Seek the advice of climate risk and resiliency experts. Do not be afraid to engage in the debate. We all have something valuable to offer.
We provide ongoing commentary on these issues. Feel free to contact us, we are always happy to discuss your climate, risk and resiliency.