"60% of the time, it works every time." - Anchorman
Economists are experts at using quantitative analysis to make a strong case. Decision-makers put a lot of faith in numbers. People like numbers. They are quick and easy to interpret. $10,000 of sales is better than $9,000, therefore we should choose the option that gives us $10,000. Easy as. They are a lot quicker and easier to digest than qualitative arguments. Numbers are helpful as it reduces the need for judgment. Decision-makers know where they stand with numbers, or so they think.
It's relatively straightforward to make a model say whatever you want it to say. Below are some of the ways to lie with numbers, I've tried to pick some of the less common examples:
Hide the lies with stuff that is factual - refer to research articles that are credible while not referencing other assumptions or sources creating the illusion of credibility.
Bury the lies - create so much work that people don't know where to look, for example by having a huge number of spreadsheets or data assumptions.
Shape the outcome in advance - for example, by running surveys that don't allow people to adequately present their opinion.
Unnecessarily lumping groups together (insufficient disaggregation) - analysis may be presented for a large group when in reality the result only applies to a subset of that group. For example, group A's average income is $100,000, but then there might be group B and group C that are a subset of group A that don't really fit together neatly when talking about income. This is very hard to pick up on and happens all the time.
Lie by omission - omit important details or structural model elements that are critical and hope people don't pick it up.
Creative use of percentages - Odds of winning increased by 50%! Wow! But meaningless without context. If the odds increased from 1/200,000 to 1/100,000 is that meaningful?
Imply without proof on something that seems sensible - talk about the result as if it will achieve something when you don't really know. For example, the campaign increased viewership by 30% and will increase sales. There may be no connection between viewership and sales at all.
There are many other examples, the ones above are just some of the common ones that are difficult to pick up on without some form of specialist knowledge.
99 times out of 100, people using or making a piece of quantitative analysis will have a purpose in mind. That purpose may be completely honest. Sometimes people make mistakes, not everyone is out to maliciously fudge the numbers. The point is it's easy to manipulate data to prove a point. Basic knowledge of analytical techniques can help with understanding if the analysis is sensible and representative.
▼▼ Thank you for reading. Please share using the links below. ▼▼