“Please provide a post-hoc power analysis” is one of the most common reviewer requests after a null result — and the standard way of answering it is a recognized statistical error. Here is why, and what to report instead.
The request usually arrives after a primary result that did not reach significance: the reviewer wants to know whether your null is informative — a genuine absence of effect within useful precision — or just a failure to detect. That is a fair question, and it deserves a real answer. The trap is the default way authors answer it: computing observed power, by plugging the effect they observed and the sample they had into a power formula. It feels responsive. It produces a number. And the number carries no information the manuscript did not already contain.
For the common test statistics, observed power is — to a close approximation — a deterministic transform of the p-value. Once p is known, observed power is known too; the two numbers are one piece of information wearing two costumes. A result at exactly p = 0.05 corresponds to observed power of roughly 50% under the usual normal approximation, and larger p-values map to lower observed power, mechanically. So “we performed a post-hoc power analysis and power was low” restates “the result was not significant” in different units, and a null result with high observed power computed this way is close to arithmetically impossible. This is not a minority view; it is the argument of Hoenig & Heisey's “The Abuse of Power” ↗ , the standard citation on the point, and it is why careful reviewers treat an observed-power calculation as a red flag rather than an answer.
The intuition without the algebra: the observed effect is the noisiest quantity in the study. Treating it as if it were the true effect, in order to grade the very study that produced it, is circular — the calculation assumes the answer to the question it claims to address.
1. The confidence interval, interpreted. The direct answer to “is this null informative?” is the interval: report it and state plainly which effects it still admits. If the CI around a null odds ratio runs from 0.76 to 1.38, the data have not excluded a clinically meaningful difference, and the wording of the conclusion should say so. The model response language for this move is worked through in responding to Reviewer 2's statistics comments.
2. The minimum detectable effect of the design. Unlike observed power, the smallest true effect your study as sized could reliably detect at conventional power is a fixed property of the design — it does not launder the observed result back through a formula, and it can be stated honestly before or after the data. Anchor it against the minimal clinically important difference: if the effect that would change practice is smaller than what the design could detect, that is a limitation worth stating in exactly those words. Our free minimum detectable effect calculator shows the arithmetic (and its FAQ explains why it refuses to compute observed power).
3. If the claim is equivalence, a margin — and margins are pre-specified. A null difference is not a demonstration of equivalence, and a margin invented after the results are known convinces no one. Without a pre-specified margin, the honest move is to describe the interval and stop short of an equivalence claim.
4. The a priori calculation, if one existed. If the protocol had a sample-size justification, restate it — the assumed effect, the planned power, what enrollment actually delivered. That is a legitimate power statement because it grades the plan, not the result.
You do not have to give the reviewer the calculation they named to resolve the comment they raised — editors read a well-reasoned substitution as competence, not evasion. A model response: “We agree the study's precision needs to be quantified, but post-hoc power computed from the observed effect is uninformative (Hoenig & Heisey, 2001), so we have addressed the underlying concern directly: we now report the 95% CI and the effects it does not exclude, state the minimum detectable effect of the design (a risk difference of 8 percentage points at 80% power), and have tempered the conclusion wherever the null appeared (p. 9, lines 4–11).” The same three-move playbook — concede, requantify, recalibrate the wording — runs through the rest of the statistical comments too.
The pattern this comment punishes — a null read as stronger than its precision supports — is visible in a finished draft before submission. It is one of the calibration questions a pre-submission statistical review keeps returning to: RigorMD's two engines read whether the conclusions track the reported precision, and the report grounds each finding in your own sentences with suggested recalibrated wording. It flags for your judgment; it does not certify, and it is not a substitute for a statistician's input on design or analysis. See a full sample report → or review pricing — the pre-submission review is $30.