Consider the Hollywood actor giving the classic “follow your dreams and never give up” line is bad advice and is pure survivorship bias at work.  Well what is surviorship bias?  Let’s take a look friends and learn. :)

 

“Survivorship bias, or survival bias, is the logical error of concentrating on the people or things that “survived” some process and inadvertently overlooking those that did not because of their lack of visibility. This can lead to false conclusions in several different ways. The survivors may be actual people, as in a medical study, or could be companies or research subjects or applicants for a job, or anything that must make it past some selection process to be considered further.

Survivorship bias can lead to overly optimistic beliefs because failures are ignored, such as when companies that no longer exist are excluded from analyses of financial performance. It can also lead to the false belief that the successes in a group have some special property, rather than just coincidence (Correlation proves Causation). For example, if three of the five students with the best college grades went to the same high school, that can lead one to believe that the high school must offer an excellent education. This could be true, but the question cannot be answered without looking at the grades of all the other students from that high school, not just the ones who “survived” the top-five selection process.

Survivorship bias is a type of selection bias.”

 

“During World War II, the statistician Abraham Wald took survivorship bias into his calculations when considering how to minimize bomber losses to enemy fire. Researchers from the Center for Naval Analyses had conducted a study of the damage done to aircraft that had returned from missions, and had recommended that armor be added to the areas that showed the most damage. Wald noted that the study only considered the aircraft that had survived their missions—the bombers that had been shot down were not present for the damage assessment. The holes in the returning aircraft, then, represented areas where a bomber could take damage and still return home safely. Wald proposed that the Navy instead reinforce the areas where the returning aircraft were unscathed, since those were the areas that, if hit, would cause the plane to be lost.[8][9]”

 

 

So, they said: the red dots are where bombers are most likely to be hit, so put some more armor on those parts to make the bombers more resilient. That looked like a logical conclusion, until Abraham Wald – a mathematician – started asking questions:

– how did you obtain that data?
– well, we looked at every bomber returning from a raid, marked the damages on the airframe on a sheet and collected the sheets from all allied air bases over months. What you see is the result of hundreds of those sheets.
– and your conclusion?
– well, the red dots are where the bombers were hit. So let’s enforce those parts because they are most exposed to enemy fire.
– no. the red dots are where a bomber can take a hit and return. The bombers that took a hit to the ailerons, the engines or the cockpit never made it home. That’s why they are absent in your data. The blank spots are exactly where you have to enforce the airframe, so those bombers can return.

This is survivorship bias. You only see a subset of the outcomes. The ones that made it far enough to be visible. Look out for absence of data. Sometimes they tell a story of their own.

BTW: You can see the result of this research today. This is the exact reason the A-10 has the pilot sitting in a titanium armor bathtub and has it’s engines placed high and shielded.

 

If you want to think scientifically, ALWAYS ask what data was included in a conclusion. And ALWAYS ask what data was EXCLUDED when making a conclusion.

[Source:dieselpunksnotdead]