Sample report · real appraisal, de-identified manuscript · for demonstration only
Three real samples:Colectomy cohort · ModeratePE risk model · SeriousBariatric benchmarks · Moderate
RigorMD Validation Report

Surgeon variation and the cost–outcome relationship in colectomy: a multi-hospital enterprise analysis

A retrospective multi-hospital cohort study (5,831 colectomies, 194 surgeons)
Design · Retrospective multi-hospital cohortClaim · Association (descriptive)Guideline · STROBE / RECORD
RM-2026-0188
generated 2026-06-23
report v1.0
2 engines + deterministic
overall · MODERATE
Conclusion calibration

Conclusions are stated a little more confidently than the evidence supports

The authors state the central negative with moderate confidence; the reported results provide low support for it — the key cost–outcome results are imprecise, wide-interval nulls with no pre-specified equivalence margin, and the analysis carries no adjustment for how sick or complex each patient was. The study is careful and the deterministic checks are clean; the gap is one of framing, not error.

01 Design / claim fitModerate
02 Results / conclusion alignmentModerate
03 Statistical appropriatenessModerate
04 Reporting guideline adherenceMild
05 Numerical / statistical consistencyNo findings
06 Clinical interpretability / verdictModerate
07 Contribution & literature positioningModerate

§00 Executive summary

This manuscript asks what an enterprise colectomy dashboard can honestly tell a surgeon or service line about “value.” Drawing on 5,831 colectomies by 194 surgeons across 31 hospitals, it reports that the routine measures (operating time, supply cost, complications, length of stay, conversion) do not move together, and that higher disposable supply cost is not associated with fewer short-term events. The cautionary message — do not judge value from a single metric — is well-matched to the design, and the reported numbers are internally consistent.

Two issues bear on how strongly the central negative can be stated. First, the cost–outcome results are imprecise nulls: confidence intervals (for example, reoperation OR 1.03, 95% CI 0.76–1.38) still admit clinically meaningful effects, and no equivalence margin is set, so “not associated” overstates a “none detected.” Second, the comparison carries no adjustment for case complexity (diagnosis, severity, comorbidity were unavailable), so confounding by indication can mask a real relationship. Both are disclosed by the authors. Four further findings are summarized below. The deterministic forensic layer found no numerical inconsistencies.

§01 Claim map

What the manuscript states, against what the evidence can support.

Stated claim
“Common colectomy value measures do not move together, and higher disposable supply cost is not associated with fewer short-term adverse events.”
Supportable claim
“In this enterprise cohort the measures were weakly correlated and no cost–outcome association was detected; imprecision and unmeasured case complexity preclude concluding the domains are independent.”

§02 Domain severity scorecard

Seven-domain assessment2-engine consensus + literature layer
DomainSeverityPrincipal finding
01Design / claim fitModerateCost–outcome inference drawn without adjustment for diagnosis, severity, or comorbidity.
02Results / conclusion alignmentModerateAn imprecise, wide-CI null is read as a demonstrated cost–outcome dissociation.
03Statistical appropriatenessModerateSurgeon-level estimates fragile on sparse volume; observed-subset missingness.
04Reporting guideline adherenceMildSTROBE cited; RECORD extension for routinely-collected data not mapped.
05Numerical / statistical consistencyNo findingsCohort counts and repeated numeric values reconcile where checked.
06Clinical interpretability / verdictModerateCautious, well-hedged thesis; the central nulls are limited by precision.
07Contribution & literature positioningModerateThe “first enterprise-scale” framing overlaps a retrieved 2023 network study; a real local contribution, but the primacy claim outruns the record RigorMD retrieved.
Overall severity Moderate2 central findings · well-conducted, hedged conclusions

§03 Major findings

Language calibration: 1 must-change wording · 2 precision polish. Analytic work: 2 need source-data or analytic work · 1 optional sensitivity check.

SeverityDomainFindingAuthor actionEvidenceLocus
Moderate01 · DesignThe cost–outcome conclusion is drawn without adjustment for diagnosis, disease severity, ASA class, or comorbidity (confounding by indication on the central inference).New analysis neededQuoteCentral
Moderate02 · AlignmentWide-CI, non-significant cost–outcome results (e.g. reoperation OR 1.03, 95% CI 0.76–1.38) are presented as a demonstrated “dissociation,” with no equivalence margin.Must-change wordingQuoteCentral
Moderate03 · StatisticsSurgeon-level variance (ICC) and correlation estimates rest on sparse volume (median 9.5 cases; 65 of 194 surgeons with 1–4) and one cost model did not converge.New analysis neededQuoteCentral
Moderate03 · StatisticsLength of stay (57.6%) and the readmission proxy (29.9%) are analyzed on a nonrandom observed subset, with no imputation or sensitivity analysis.Optional sensitivity analysisQuotePeripheral
Mild03 · StatisticsMany models and 30 subgroup analyses are run without multiplicity adjustment; mitigated by explicitly exploratory framing.Statistical precisionQuotePeripheral
Mild04 · ReportingA study built entirely from routinely-collected enterprise data is not mapped to the RECORD reporting extension.Statistical precisionChecklistPeripheral

§04 Detailed domain review

01

Design / claim fit

Moderate

This study can show cost and early complications did not track each other in these data; it cannot rule out that the costlier cases were simply the harder ones. Clinically, do not treat “cost does not predict outcomes” as the last word — the cases that cost more may also be the ones more likely to have problems.

Why this matters statistically

Finding
The central cost–outcome association and the by-approach comparisons adjust only for age, sex, BMI, stoma, division, and year, because diagnosis, disease severity, ASA class, and comorbidity were unavailable. Both engines raised this independently.
Bias direction
Confounding by indication — costlier cases are plausibly the more complex ones, which can pull the cost–outcome association toward the null and hide a real relationship.
Evidence
“Diagnosis, disease severity, ASA class, and comorbidity data were unavailable…” (Limitations)

Confounding by indication (ROBINS-I: confounding). The adjustment set omits diagnosis, severity, ASA, and comorbidity; disposable cost proxies case complexity, so the cost–outcome estimate is open to residual confounding that biases toward the null. Disclosed and tempered by the authors.

Clinical consequence: a reader could relax scrutiny of high-cost practice on the strength of a null that is partly a complexity artifact.

Technical details
Named bias
confounding by indication · ROBINS-I: confounding
GRADE
risk of bias · remedy: clinical risk adjustment + e-value
02

Results / conclusion alignment

Moderate

The study found no clear link between higher supply cost and fewer early complications, but the result’s range still allows a real effect it was too small to detect. Clinically, read this as “not shown here,” not “proven absent.”

Why this matters statistically

Finding
The load-bearing negative (“higher disposable supply cost was not associated with fewer short-term events”) rests on non-significant results with wide intervals (reoperation OR 1.03, 95% CI 0.76–1.38) and no equivalence margin — an imprecise null read as a demonstrated dissociation.
Bias direction
Overstates the negative — an imprecise “none detected” is presented as a true “no effect.”
Clinical consequence
Readers may conclude supply spending is irrelevant to outcomes when the study simply could not detect a moderate effect.
Evidence
“…even a doubling of disposable supply cost was associated with no meaningful change in 30-day reoperation odds.” (Results)

Absence of evidence treated as evidence of absence. Key null ORs near 1.03 with wide CIs; uncommon binary events limit power; no pre-specified equivalence margin or TOST/ROPE. The fix is wording plus a minimum detectable effect or equivalence bound, not a re-run.

Technical details
Named bias
interpreting non-significance as no effect · reporting / interpretation
GRADE
imprecision · remedy: report MDE or equivalence bounds; precise wording

§05 Forensic checks

Recomputed directly from the manuscript’s reported values — no numerical inconsistencies were found.

Quoted — Results
“…not associated with lower 30-day reoperation (OR 1.03; 95% CI, 0.76–1.38; p = 0.87).”
Odds ratio with 95% CI and p-value
Precision review
reported OR
1.03
95% CI
0.76–1.38
reported p
0.87
interpretation
imprecise null; clinically meaningful effects not excluded
No arithmetic flag — interpretation still needs qualification for imprecision
Quoted — Results
Operative approach: robotic 1,799, laparoscopic 2,216, converted 313, open 1,503.
Cohort stated as N = 5,831
Denominator check
stated N
5,831
approach sum
5,831
difference
0
Consistent — counts reconcile to the analytic N

§06 Revision priority

In order. The cheap wording fix is also the most important.

  1. Reframe the central result from “not associated” to “no significant association detected (intervals do not exclude meaningful effects),” and add a minimum detectable effect or equivalence range for the key endpoints.Moderate
  2. Re-estimate the cost–outcome and by-approach models with whatever clinical risk adjustment the data can be linked to (diagnosis, severity/stage, a comorbidity index), and report an e-value.Moderate
  3. Add a multiple-imputation or weighting sensitivity analysis for length of stay and readmission.Moderate
  4. Report confidence intervals for each ICC and promote the ≥10-case model to the primary variance decomposition.Moderate
  5. Label the subgroup analyses exploratory, and add a RECORD-mapped methods paragraph for the routinely-collected data.Mild

§07 Language calibration

Suggested wording is triaged by author action. Some wording overstates the evidence and should change; some is recommended risk reduction; some is precision polish; some is left to author discretion.

As written

“Higher disposable supply cost was not associated with fewer short-term events.”

Must change Must-change wording

“No significant association was detected between disposable supply cost and short-term events; the confidence intervals do not exclude clinically meaningful effects (reoperation OR 1.03, 95% CI 0.76–1.38).”

The current wording makes a claim the design or results cannot support.
As written

“Colectomy value measures did not move together — a dissociation of cost and outcomes.”

Recommended Recommended wording

“In these data the value measures were weakly correlated at most; the study was not powered to establish that the domains are truly independent.”

The wording is directionally defensible, but softer wording would reduce reviewer risk.
As written

“The surgeon intraclass correlation for operating-room time was 0.094.”

Statistical precision Statistical precision

“The surgeon ICC was 0.094 (report a 95% confidence interval; with a median of about nine cases per surgeon these estimates are imprecise).”

The sentence is acceptable, but could be made more statistically exact.
As written

“These data should not be used for surgeon rankings, compensation, or platform decisions.”

Author discretion Author discretion

“These data should not be used for surgeon rankings, compensation, or platform decisions.” The current wording is well-judged; this is a conservative clarity note, not a required change.

A conservative phrasing option; the current wording is defensible.

§08 Journal compliance

Checked against RigorMD’s journal registry. Compliance items reflect the journal’s formatting and submission rules; they do not affect the methodological severity grade above.

§09 Reference identifiers

Cited DOI and PMID identifiers, resolved against the public registries — Crossref, the DOI handle registry, and PubMed — as of 2026-06-23. A ✓ means the registry record exists and is consistent with the citation as printed; it does not assess whether the cited work supports the claim it is attached to. An identifier the check could not reach is listed as not checked, never assumed to resolve. Problems found here also appear as findings above. 4 of 5 cited identifiers were checked: 1 resolves to a different work · 1 does not resolve · 2 resolve · 1 not checked.

IdentifierOutcomeRegistryNotes
DOI 10.1016/j.jamcollsurg.2026.01.118✗ Resolves to a different workCrossrefDOI 10.1016/j.jamcollsurg.2026.01.118 resolves at Crossref to "Hospital volume and outcomes after elective colectomy" (2019), which does not match the citation as printed (checked 2026-06-23)
DOI 10.1007/s00384-026-04517-3✗ Does not resolveDOI handle registryDOI 10.1007/s00384-026-04517-3 does not resolve at Crossref or the DOI handle registry as of 2026-06-23
DOI 10.1097/SLA.0000000000091214— Not checkedThe registry could not be reached.
DOI 10.1097/DCR.0000000000002841✓ ResolvesCrossref
PMID 37412905✓ ResolvesPubMed

§10 Contribution & literature positioning

Findings — see below

The prior literature RigorMD retrieved into an evidence pack and compared with this manuscript's positioning as of 2026-06-23. This is a positioning-risk check, not a novelty score: it flags where a claim may overlap, understate, or be contradicted by retrieved prior work. It does not assert that a contribution is novel or first — a clean result means only that no directly overlapping prior study was found in this evidence pack. The retrieval is bounded and time-stamped; treat it as a starting point for your own literature review, not a replacement for it. Positioning risks found here also appear as findings above. One retrieved prior study reports the same enterprise-scale surgeon-variation cost–outcome analysis the abstract frames as the first of its kind.

Priors we compared you against. The prior work RigorMD retrieved and compared this manuscript against — disclosed so you can see the evidence pack behind the assessment. Listing a work here is not an instruction to cite it; it is the basis on which the positioning was checked.

Prior workYearIdentifierIn your references
Surgeon-level variation in the cost–outcome relationship for colorectal resection across a hospital network2023PMID 37021884Not in your references
Do value measures move together? A multi-hospital analysis of elective colectomy2021DOI 10.1097/DCR.0000000000002119In your references
Disposable supply cost and short-term outcomes after major abdominal surgery: an enterprise cohort2024PMID 38455120Not in your references
07

The “first enterprise-scale” framing overlaps a retrieved prior study

Moderate

The manuscript positions itself as the first enterprise-scale look at surgeon variation in the cost–outcome relationship for colectomy, but a 2023 network study RigorMD retrieved reports a closely overlapping analysis. This is a positioning risk, not an error in the results — the claim of primacy is stronger than the retrieved record supports.

Positioning risk
Overstated novelty · Abstract and Introduction (final paragraph)
Evidence pack
pack-004, pack-019
Across 28 hospitals we found that surgeon-level variation in disposable supply cost was not associated with differences in 30-day complications, challenging the assumption that higher spend reflects safer practice.From the prior work RigorMD retrieved and compared

Literature assessed as of 2026-06-23. Bounded PubMed retrieval on the manuscript's own concept pair, not a systematic review; a work not surfaced here was not necessarily absent from the literature. Listing a prior is disclosure of what was compared, not an instruction to cite it.

§11 Technical appendix

What could be checked from the submitted files — and what could not. Check your own paper →

Checked from submitted files

  • ✓ passed Cohort counts vs analytic N
  • ✓ passed Volume-category counts vs surgeon total
  • ✓ passed Abstract / results numeric agreement
  • ⚑ flag Multiplicity across many models

Not checkable from submitted files

  • — n/a Patient-level reanalysis (no dataset provided)
  • — n/a Risk adjustment for case complexity (no diagnosis/comorbidity)
  • — n/a Missing-data mechanism for length of stay / readmission
  • — n/a Random-effects model diagnostics (not reported)
Scope. This report provides methodological and statistical guidance based on the submitted materials. It is not a substitute for peer review and is not clinical treatment advice. Findings marked deterministic are recomputed from the manuscript's own reported values; findings marked quote are traceable to the quoted text. This sample is a real RigorMD appraisal of a de-identified manuscript. The executive summary, the before/after language revisions, and the passed-check panels are illustrative — the delivered report does not yet present them in this form; the findings, severity scorecard, conclusion calibration, journal-compliance, and reference-identifier sections reflect what the delivered report contains. The consensus artifact is downloadable as JSON alongside the PDF on your report page.

Get this report on your manuscript.

The same severity-scored review with deterministic checks — $30 per report, processed automatically and emailed when ready. Delivery time varies by package.

Tested against the public record — the PREDIMED concordance benchmark →