Trusting the Numbers: The McKinsey Survey

McKinsey & Company made waves earlier this month by releasing a report stating that “Overall, 30 percent of employers will definitely or probably stop offering employer-sponsored insurance (ESI) in the years after 2014.”

This number differed from the 7 percent estimate previously released by the Congressional Budget Office. Acknowledging the gap, McKinsey, a global management consulting firm, explained that its estimate came from an “early-2011 survey of more than 1,300 employers…as well as other proprietary research.” In weeks following, the report came under scrutiny and McKinsey did not discuss its methodology until several weeks later. Then representatives explained that the survey was “not an economic forecast, but rather a measure of attitudes.”

The number was not quite what it seemed to be.

My intent is not to dissect McKinsey’s methodology, but rather to talk about the trust placed in numbers. At CHI, we strive to be the most trusted and leading source of health information for Colorado leaders. When we publish, we stand firmly behind our numbers, make public our methodology, are open to dialogue, and will acknowledge and work to amend errors when they are made. In other words, we believe you can trust us for our rigor, integrity and transparency.

But what happens when you see a number batted around in the mainstream media and you are unaware of the rigor of the research methodology and you don’t have a relationship with the organization?  Here are three clues on how to spot a number you can (or can’t) trust:

1. If a number in the headlines causes you to do a double-take, then the second look is definitely worth your time. A deeper dive into the data may explain the real story.
2. If the organization providing you with the number can’t clearly and definitively tell you how they arrived at the number, it is probably questionable.
3. Comparison is key. If other reputable organizations have come to similar conclusions or at least cast an eye of critical approval toward a number, you can place confidence in it.

Information is everywhere. While Google and many media outlets make finding it a lot easier, the one thing they don’t do is help you sift the wheat from the chaff. Fortunately, that’s where CHI comes in.  We strive to be your trusted resource for health information in Colorado.

Part of our community benefit function is to field information requests. In fact, we answered over 200 last year. So if you have a question, please call or email. We will do our best to provide you with the information you need, and if we don’t have it, we will point you to someone who does.