What Is This Explanation Taking for Granted?

The returning aircraft

As the story is usually told, the question was where to add armour to bombers. Should it go where returning aircraft were hit, or where they were not?

The usual account of Wald’s work points at the aircraft that did not return. The planes available for inspection were assumed to represent the planes that flew.

I need to be careful with this story. A Feature Column on the American Mathematical Society’s site warns that the famous bullet-hole picture and the tidy story are embellished. It finds little evidence for the best bits, but a solid germ of truth. I am using the standard reading of Wald’s work, with that caution attached.

The method I want to test is this: find one assumption an explanation quietly relies on. Remove it, then ask what the evidence says.

What catches me here is how little needs to change before the question changes. The damage can remain exactly as recorded. I have to reconsider what those records can tell me.

What the explanation assumes

An explanation can fit the evidence available while depending on something outside that evidence. In the bomber account, that something is the route into the sample.

Looking at returning aircraft puts return inside the problem. If I want to understand vulnerability, I also need to consider what return leaves out.

That is my reading of the example. I cannot use the visible damage alone to establish what happened to aircraft I cannot inspect.

The buried assumption connects an observation to a wider claim. Here, it lets damage among survivors stand for vulnerability among all the aircraft that flew.

Writing that connection down gives me something specific to question. I can ask whether the evidence supports carrying the conclusion that far.

What changes without it

The Berkeley admissions case puts the assumption somewhere else.

The usual summary of Bickel, Hammel and O’Connell’s paper describes pooled figures that looked biased against women. Department by department, they did not. I have not read the paper itself, so I am keeping to that standard summary.

The assumption here is that the whole behaves like its parts. Once I question it, I need to examine the level at which the comparison is made.

The pooled result no longer settles what the departmental comparisons show. Equally, that summary cannot establish that every admissions decision was fair. Such a claim would reach beyond the evidence described here.

I find this useful because it makes the next enquiry concrete. For the bomber account, I would examine what is missing from the sample. For admissions, I would compare the pooled and departmental results.

Each question names evidence that could constrain the explanation. Removing the assumption has earned that much. Whether it improves the answer still needs testing.

How the inference changes

I think what the two cases share is narrow. A conclusion depends on treating the way evidence is selected or combined as harmless. Questioning that treatment changes which inference the evidence can support.

I can put those relations beside each other. The historical claims below retain the limits of the standard accounts.

Relation	Returning aircraft	Berkeley admissions	Match and limit
Evidence reaches the analysis in a particular form	The usual Wald account concerns aircraft available after returning.	The usual paper summary compares pooled and departmental figures.	Both require checking how the evidence reaches the comparison.
An assumption carries the conclusion further	Returning aircraft stand for those that flew, in the standard account.	The whole is assumed to behave like its parts, in my reading.	A match in the inference being questioned.
Changing the evidence boundary changes the enquiry	Consider aircraft absent through non-return.	Examine comparisons within departments.	A proposed check for each case; neither guarantees a better explanation.
Non-return removes observations	Central to the usual bomber account.	No corresponding loss is established by the admissions summary.	Cut from the transfer claim.
Pooling hides differences between parts	No corresponding pooling effect is established here.	Pooled and departmental readings differ in the standard summary.	Cut from the transfer claim.

The table stops me claiming a shared cause for both outcomes. The match concerns the relation between evidence and conclusion. Survival and pooling remain different mechanisms.

I also need to draw each boundary around the question being asked. Inspectable aircraft are too narrow for a claim about all aircraft that flew. Departmental comparisons belong inside an enquiry about what the pooled admissions figures mean.

How I would test this

The hardest part is separating one assumption from everything around it. A test confronts a bundle of assumptions at once, which philosophers discuss as the Duhem-Quine problem. A failed prediction alone cannot tell me which one failed.

I would therefore record the original explanation before changing it. Then I would name the assumption and explain why it was there. Chesterton’s fence is a useful warning here: understand an assumption’s purpose before removing it.

I would list what the analysis rests on, then mark the weak assumptions and test them. The CIA’s Tradecraft Primer describes this procedure as a Key Assumptions Check.

For my own test, I would give the original explanation to three readers independently. I would ask each to name the assumption carrying the conclusion. Agreement would give me a candidate to investigate. Disagreement would weaken my confidence that I had isolated something clear.

Then I would record different expectations before checking further evidence. The original explanation and its replacement must predict something distinguishable. Lakatos’s account makes a new prediction the test of a change worth keeping.

For admissions, I can spell out the competing expectations. Carrying the pooled judgement into departments predicts that the appearance of bias will persist there. Questioning that assumption lets me propose the opposite prediction: the appearance changes at department level.

The standard summary favours that alternative. Knowing the result already prevents me counting this illustration as a successful forecast.

If both explanations predict the same observations, this test cannot choose between them. If the original predicts better, I should keep it. Repeated failures to improve predictions would make this method less useful than I currently hope.

One worry is my own. AI could suggest a plausible buried assumption for almost anything. That could give me stories faster than tests.

The comparison itself has a separate limit. Evidence that the claimed inference never depended on the named assumption would break the match. It would leave the other case to stand on its own.

These are proposed tests. I have not yet logged a forecast that tests this method. The familiar success stories cannot supply that result.

Where I land

When an explanation feels convincing, I want to make it harder to keep. Before asking AI for another possible assumption, I will write down what would change my mind.

I can keep generating alternatives indefinitely. The next useful step is choosing an observation that could cost me the explanation I prefer.

Sources and claim map

• Mangel and Samaniego on Wald: returning-aircraft case, used only as the standard reading. The DOI was checked; the paper was not read.
• American Mathematical Society account: opened and read. Supports the warning about the embellished picture and story.
• Bickel, Hammel and O’Connell: pooled and departmental admissions findings, used only as the standard summary. The DOI was checked; the paper was not read.
• Duhem-Quine overview: opened and read. Supports the difficulty of testing one assumption separately.
• Chesterton’s fence: primary text opened and read. Supports understanding an assumption’s purpose before removal.
• CIA Tradecraft Primer: text extracted and read. The Key Assumptions Check is listed at page 7 of the PDF.
• Lakatos overview: opened and read. Supports requiring a new prediction.
• The shared inference, boundary choices and comparison limits are my working reading. The reader check and prediction comparison are proposals, without a completed test.

This idea was developed through an ongoing dialogue with AI. AI was used to challenge the argument, locate related research and test alternative explanations. The conclusions are mine unless otherwise stated.