Responding to Reviewer 2's statistics comments
A revise-and-resubmit is an invitation, not a rejection. The statistical comments are usually the hardest to answer — and the ones where a defensive response does the most damage. Here is how to answer them cleanly.
§01 The posture that earns a fair reading
Reviewer 2 is a stand-in for the skeptical reader you will never meet. The goal of a response letter is not to win every point; it is to show the editor that each comment was taken seriously and resolved — by a change, by an added analysis, or by a reasoned, evidenced disagreement. Three moves carry almost every statistical response: concede and fix, concede and reframe, or disagree with evidence. What sinks a revision is a fourth move — deflect — where a real limitation is met with a paragraph that talks around it. Reviewers read deflection as a tell.
Quote each comment verbatim, answer directly beneath it, and point to the exact revised location (page and line). What follows are the statistical critiques that recur, and language that answers each without spin. One comment is a case of its own — the letter that asks outright for a statistician; the options and the honest response language for that one are in a reviewer asked for a statistician — what now?
§02 “The study is underpowered / your null is not evidence of no effect”
A nonsignificant result can still permit a clinically important effect. Repeating that the result was “not significant” does not address the reviewer's concern about precision.
How to answer. Concede the imprecision and requantify. Report the confidence interval and state what effects it still admits; add a minimum detectable effect or, if an equivalence claim is intended, a pre-specified margin — but not an observed-power calculation, which restates the p-value and reads as a statistical error. Then soften the wording everywhere the null appears. A model response: “The reviewer is correct that the study was not powered to establish equivalence. We now report the 95% CI (reoperation OR 1.03, 0.76–1.38), note that it does not exclude a clinically meaningful difference, and have changed ‘was not associated’ to ‘no significant association was detected’ throughout (p. 9, lines 4–11).” The mechanics of this fix are worked through in why manuscripts get rejected for statistics.
§03 “You ran many comparisons without adjustment”
How to answer. You have two honest routes. Either apply a correction and report what survives, or — often better — declare the analyses exploratory and label them as hypothesis-generating rather than deleting them. What you cannot do is keep presenting a handful of p < 0.05 results from twenty tests as confirmed findings. A model response: “We agree. The primary endpoint remains as pre-specified; the fifteen secondary comparisons are now explicitly labeled exploratory, and we have added that with α = 0.05 roughly one significant result is expected by chance (Methods, p. 6).”
§04 “How were missing data handled?” and “this is confounded”
Missing data. State how missing data were handled, the amount missing, and why the approach fits the study design, estimand, and assumptions about missingness. Where appropriate, use a prespecified or clearly labeled sensitivity analysis to examine dependence on those assumptions. Choose the method with a statistician and report changes in the conclusion rather than aiming to preserve it. A missing-indicator method can bias association estimates; it is not a generic alternative to multiple imputation.
Confounding. For an observational study, you usually cannot randomize your way out in revision. What you can do is add whatever risk adjustment the data support, report an e-value for how strong an unmeasured confounder would need to be to explain away the effect, and state the direction of likely residual bias. A model response: “We cannot fully exclude confounding by indication. We have added adjustment for a comorbidity index, report an e-value of 1.7, and now state that residual confounding would most plausibly bias the association toward the null (Discussion, p. 12).”
§05 “Your conclusion overreaches”
This comment is nearly always cheaper to satisfy than it looks, because the fix is wording, not re-analysis. If the primary endpoint was negative and the abstract leads with a secondary one, that is spin and a reviewer will name it. Pull the conclusion back to the primary result and let the abstract say plainly what was and was not shown.
How to answer. “We have revised the abstract and conclusion so the primary (negative) endpoint leads, and reframed the secondary finding as hypothesis-generating (Abstract; Discussion, p. 13).” Conceding an over-claim costs you nothing an editor values and buys you the credibility to hold your ground elsewhere.
§06 Getting ahead of Reviewer 2
The strongest position in a revision is having already named the limitation the reviewer raises. Every comment above is one you can anticipate before submission — the underpowered null, the unadjusted multiplicity, the silent missing data, the conclusion that drifts. Naming them yourself, in the limitations and in calibrated wording, is far stronger than having a reviewer find them unnamed.
RigorMD examines the manuscript's methods, analysis, and conclusions, recalculates applicable statistics when the reported inputs allow, and returns a severity-scored report with supporting manuscript passages, calculations, and reference records where applicable. It can suggest wording for revision — and, for a manuscript already reviewed, a $39 re-review of the revised draft with a finding-by-finding comparison against the first report, as a reading aid. Supply the decision letter for a point-by-point assessment and draft responses to edit against the changes you made. It flags for your judgment; it does not write your response letter or guarantee acceptance. See a full sample report →, read how the engine works, or review pricing — the pre-submission review is $99, and a re-review of a revised manuscript already reviewed here is $39.