Respected as a quiet, gentle, and brilliant man, Abraham Wald (1902-1950), one of the most renowned mathematicians of his day, published more than 90 books and papers, including seminal works on sequential analysis, topology, set theory, lattice theory, econometrics and mathematical economics, and was also renowned as an outstanding teacher. Although best known for his work in sequential analysis and for founding statistical decision theory, perhaps his greatest contribution was solving the problem of how best to protect American fighter planes being shot down by enemy artillery in unsustainable numbers during World War II.
Prior to the Nazi rise to power, Germany was perhaps the world’s leader in mathematics, both theoretical and applied, and a large number of its leading mathematicians were Jews. This is the amazing, but largely unknown, story of how Wald played a major role in the victory of the Allies in WWII which, like so many others, begins with the Nazis hounding Jews like Albert Einstein out of Europe – and coming to regret it.
Born in Cluj in the Austrian Empire the grandson of a rabbi and the son of a kosher baker, Wald was raised in a strictly Orthodox home, which precluded his attendance at the local gymnasium on Shabbat, as strictly mandated by the Hungarian school system; as a result, he was homeschooled by his parents and brother. Although he was an early mathematical prodigy, he was initially denied admission to the University of Vienna because he was Jewish but, with persistence, he was finally admitted to the University, from which he graduated with a mathematics degree (1928) while simultaneously serving in the Romanian army. He went on to earn a Ph.D. in mathematics with a specialization in pure mathematics (1931).
Despite his brilliance, antisemitic discrimination blocked his obtaining a desired university position in Vienna, and the fact that he was an Eastern Jew with an identifiable accent from a poor immigrant community that flooded Vienna after World War I did not help him. Wald was forced to accept a research offer that barely paid a living wage from Oskar Morgenstern, the director of the Austrian Institute for Economic Research who later emigrated to the United States and, along with Jewish polymath John Von Neumann, invented game theory.
Throughout the early 1930s, the Rockefeller Institute was awarding some grants to researchers to study abroad. Wald applied for such a fellowship, but the widespread Nazi attack on German scientists forced the Institute to reevaluate its continued support for German scientific organizations. One of those decisions was to deny Wald’s application because he was a Jew; as John Van Sickle, a Foundation officer, wrote:
. . . I have no doubt that he is one of the very ablest of the men working on the problem of statistical techniques as applied to business cycle analysis. It is a pity that his nationality and race combined to make his future so precarious . . . Wald should be kept under observation, but I am not inclined to recommend an early award.
Meanwhile, Wald continued to search for a meaningful position that would offer him some means of financial support. One possibility he investigated was a position in Eretz Yisrael through Abraham Fraenkel, an early Zionist and the first Dean of Mathematics at the Hebrew University in Jerusalem, but a position never materialized. Wald nonetheless planned to go to Eretz Yisrael if he could obtain financing and an entry permit, but his search for a position proved fruitless and he remained in Vienna.
Van Sickle again recommended denying Wald a fellowship because, although the Jewish mathematician was “obviously a man of exceptional ability,” he was also . . .
[a] man without a country [such that] it is impossible to see what the future holds for him . . . In spite of his guarantee of employment in the [Vienna] Institute upon his return to Vienna, I doubt there is any real future for [him] there. Growing antisemitism has closed the door to such men throughout most of Central Europe. It is a tragic situation, but I don’t see how we can use our fellowships to combat the trend.
Van Sickle also feared having Wald enter the American labor force because, as he wrote, “if an award were made to Wald to study in this country, I am convinced that he would use the sojourn here to seek permanent employment.” Moreover, Wald’s poor financial status was obviously also an issue: “Wald is responsible for his parents in Romania and he has not been able to save anything.”
Wald’s experience in economics and econometric research, however, earned him a fellowship offer in 1938 at the Cowles Commission, a Colorado Springs economic institute. He preferred to pursue theoretical mathematics in Vienna, but with the Nazi annexation of Austria after the Anschluss and nationalist student fraternities rallying against the disproportionate number of Jewish instructors and students at the University of Vienna, discrimination against Wald and his family became untenable, and he accepted the position in Colorado, a decision that saved his life. On May 3, 1944, Nazi troops entered Cluj, collected the city’s approximately 17,000 Jews, brought them to the Iris Brickyard on the north side of town, and transported them to Auschwitz, where most were exterminated. Among those murdered in Hitler’s ovens were eight members of his immediate family, including his parents, sisters and brother, their spouses and children as well as other relatives, with only his brother Hermann surviving. (In the late 1940s, Wald managed to arrange for his brother’s passage to America.)
Only a few months after commencing work with Cowles in Colorado, Wald accepted a statistics professorship at Columbia University, which turned Columbia into a leading center of mathematical statistics. He continued to teach and conduct research at Columbia while simultaneously performing critical work for the Statistical Research Group (SRG) that helped the United States win the war.
SRG, a classified WWII program that harnessed 18 of the greatest American mathematical minds, was located at Columbia. Comparable to a Manhattan Project, except that its focus was on developing equations rather than bombs, it enlisted Wald as one of its first members, followed by the likes of Leonard Savage, the pioneer of decision theory; Norbert Wiener, the MIT mathematician and the creator of cybernetics; and Milton Friedman and George Stigler, future Nobelists in economics. Although Wald was viewed by his fellow researchers as mathematical royalty, he was nonetheless still technically an “enemy alien” who, ironically, was denied access to the very classified reports that he was producing. Although his strong inclination remained the abstraction of pure mathematics, he dedicated himself passionately to the SRG’s anti-Nazi applied mathematics effort.
Wald’s work for SRG, for which he became renowned, concentrated on the invention of sequential analysis, which describes an important technique for improving quality control in production, such as military ordnance, and he introduced many fundamental concepts of decision theory. He was the first to describe how studying data as it is produced sequentially yields far more meaningful results than aggregating all available data before commencing analysis. Two of the many advantages of this approach are that, first, the researcher is not bound by his original choice of sample size and may add data to his study as it becomes available and, second, if the results in sequentially unfolding data hold consistent, the researcher may terminate the testing and save considerable effort and expense.
But Wald’s greatest contribution began when he was assigned responsibility for solving perhaps the Air Force’s greatest problem in the war: aircraft casualties and the devastating frequency with which U.S. and Allied aircraft were shot down by enemy artillery. (Data compiled after the war show that over 40,000 planes were shot down by the German and Japanese.) Aircraft were being lost faster than American manufacturers could build them, and the loss rates were such that it was becoming a virtual statistical impossibility for an airman to survive lengthy tours of duty in Europe.
The core problem was that the Air Force needed armor to reinforce its aircraft, but it had to prioritize where to install the steel plating because the added weight had three significant adverse repercussions: First, each additional pound of armor reduced the plane’s attack capability by reducing payload capacity by a corresponding pound; second, the weight of the added armor rendered the plane less maneuverable and therefore less able to avoid being struck by anti-aircraft weapons; and, third, the plane would burn more fuel, limiting its range and, in some instances, the ability to carry out bombing missions at all.
Thus, the problem was one of optimization; i.e., how to minimize the use of heavy armor while maximizing the fortification provided by such armor. The SRG chose Wald, its best mathematician, to design and develop the protocol that optimally balanced protection and military capability.
The Air Force provided Wald with the data it had compiled on bombers that had returned from their missions covered in artillery holes. It showed that the damages were not homogeneously distributed, that a high percentage of the damage was to the fuselage (main body) with far less to the engines, and that the wings also took disproportionate fire:
Bullet holes
Section of plane per square foot
Engine 1.11
Fuselage 1.73
Fuel system 1.55
Other 1.80
Accordingly, the military authorities determined that the wings, tail and, in particular, the fuselage that housed the pilot and crew, were most vulnerable to enemy artillery. It concluded that the obvious optimum protocol was to concentrate reinforced steel in the sections where the planes were getting hit with the highest frequency; i.e., that reinforcing the fuselage would do the most good and afford the most protection. However, Wald’s brilliant and counterintuitive conclusion was that the reinforcement armor should not go to areas most frequently struck by bullets but rather to those least struck by bullets: the engines.
Obviously, Wald’s only certain information about aircraft that failed to return was that they didn’t return and, although he had to make certain assumptions in his study, he was very careful to ensure that they were reasonable. First, he assumed that the bullet holes should have been evenly distributed across the entire plane because it would be manifestly illogical to assume that anti-aircraft gunners and German fighters were successfully targeting specific sections of the American bombers rather than the planes themselves. (That assumption would be invalid today, when weapons of war include precision armaments such as heat-seeking Stinger missiles.)
Second, he assumed that planes failing to return were, indeed, shot down by enemy fire, even though it is possible that at least some sustained mechanical failure; or, less probably, that the plane ran out of fuel; or, less likely still, that a flier had defected to the enemy. In any event, these highly unlikely occurrences were statistical outliers that would not materially bias Wald’s analysis and conclusions.
Wald turned to identifying the aircraft areas with the “missing holes”; that is, the aircraft sections that evidenced less holes than expected, assuming a normal distribution. Concluding that these missing holes would be found on the planes that never returned to base, he argued that the high distribution of shell patterns on certain sections of the returning planes actually evidenced the sections where the planes were most able to survive taking flack absent additional reinforcement and that the planes could sustain heavy fire in sections such as the wings and fuselage and still remain airborne.
Wald asserted that Air Force data had ignored the thousands of bombers that had been unable to return to base because they’d been struck in other plane sections, including the engines, the cockpit, and the tail which, he maintained, are the very areas that must be more heavily armored. He described how the Air Force’s error was attributable to “survivorship bias,” the statistical term for the error of looking only at subjects who have survived to a certain point in time without considering those who have not.
| Probability of surviving a single hit
Engine 61% Fuselage 95% Fuel system 85% Other parts 98% |
The results of Wald’s model demonstrate that the engine area is the most vulnerable in the sense that a hit there is most likely to down the plane.
The frequently cited classic example of survivorship bias is the incorrect conclusion by medical researchers that more victims sustain gunshots to the arm than the chest because there are far more such patients in hospital recovery rooms. However, this conclusion fails to consider that more victims may be shot in the chest but are not part of the researchers’ sample because they did not survive to be treated in recovery rooms. Similarly, when wearing motorcycle helmets became mandatory, hospitals noted seemingly counter-intuitive data evidencing that the number of bikers treated for head injuries had risen steeply. Of course, while true, the increase in head injury patients was not because the helmets were ineffective but, rather, because the helmets protected the bikers so that more of them survived to arrive the emergency room alive.
Wald’s proposal was immediately adopted and, although it is impossible to quantify the number of planes – and the number of pilots’ lives – that he saved, they surely number in the thousands. The benefits of his model were also used by the U.S. military in Korea and Vietnam and are still used in armoring American military aircraft to this day, saving countless additional planes and lives.
But it is not only in the military theatre that Wald’s work saved resources and lives. For example, in a 2010 trial at Harvard Medical School and Beth Israel Deaconess Medical Center designed to improve patient survival following trauma, researchers focused upon abnormal blood clotting following trauma, which increases the probability of patients bleeding to death. Setting up an experiment to investigate whether administering blood-clotting factors would improve survival, they concentrated their study on patients who had received 4-8 blood transfusions within 12 hours of their injury before realizing their “survivorship bias”; e.g., that the trial would only include patients who had survived their initial accident, received care in the Emergency Department, and were sent to Intensive Care with sufficient time passing to have been administered at least four bags of blood. Because patients who died could obviously not be included in the study, the researchers concluded that it was pointless to continue with the trials.
In fact, Wald’s survivorship bias manifests itself in virtually every area of health research including, for example, cancer research, where “survival rates” are expressed in terms of patient survival after five years – but omits patients who died of something other than cancer. A more contemporary example is the federal government and various state governments, most notably New York, purposely manipulating data and blaming Covid for the death of anybody testing positive for Covid.
In fact, survivorship bias permeates virtually every aspect of contemporary life. For example, in Bullet Holes & Bias: The Story of Abraham Wald, James Thomas provides a lovely example of how survivorship bias can affect finance: as per customary practice, a fund manager seeking investments will tout growth results for various funds that are still active, but not many investors think to make inquiry regarding funds he managed that have failed and closed because of negative growth. A lone self-made billionaire is held up as compelling evidence that a college education is not necessary to attain great success, without considering the thousands of people who followed the same track and failed. Another classic manifestation of survivorship bias is the idea that old things reflect superior craftsmanship; the fact that a small number of cars are still drivable after twenty years and that some buildings still stand strong more than a century after construction does not mean that they were built better than the cars and buildings of today . . . and the list goes on.
Wald and his wife, Lucille Lang, died in 1950 when, on an extensive lecture tour at the invitation of the Indian government, their Air India plane crashed in the fog in the Nilgiri Mountains in southern India, leaving their two young children behind. He had visited the Indian Statistical Institute at Calcutta and was scheduled to attend the Indian Science Congress at Bangalore. His seminal paper for the military on protecting American planes and survivorship bias, A Method of Estimating Plane Vulnerability Based on Damage of Survivors, was not declassified until July 1980.
