Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Correspondence
  • Published:

Getting over ANOVA: estimation graphics for multi-group comparisons

Nature Methods volume 23, pages 1924–1926 (2026) Cite this article

Save article
View saved research

Generations of statisticians have voiced concern about null-hypothesis significance testing for reducing research decision-making to dichotomous thinking, distancing researchers from the critical task of effect quantification1,2,3. As an alternative, estimation statistics focuses on effect size estimation, confidence intervals, data graphics and meta-analysis1,4. Realizing a transition to widespread use of estimation will require the implementation of estimation methods, accessible software (Supplementary Note 1) and data graphics appropriate for all experimental types, including complex designs.

Multi-group experiments present even greater challenges than two-group designs. Conventional multi-group analysis uses omnibus testing via analysis of variance (‘ANOVA’) and its variants5. Since ANOVA tests an unfocused null hypothesis of no effects, assuming all group means are equal, researchers must perform post hoc multiple comparisons on the relevant individual comparisons, often resulting in a web of subsidiary tests (Supplementary Note 2)6. Estimation can simplify multi-group analysis by focusing on relevant effect sizes while ignoring unrelated hypotheses. Multi-group estimation graphics show effect magnitudes and precision estimates — the information most relevant for understanding biological systems (Supplementary Note 3). Furthermore, interpreting data with effect sizes mitigates issues raised by corrections for multiple comparisons (Supplementary Note 4).

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to the full article PDF.

USD 39.95

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Estimation graphics for multiple group experimental designs.

Subjects

Code availability

DABEST is available in both Python and R. The Python package, DABEST-python, is hosted on PyPI, and its source code can be found on GitHub at https://github.com/ACCLAB/DABEST-python. A tutorial for the Python package can be found at https://acclab.github.io/DABEST-python/tutorials/. The R variation, dabestr, is available on CRAN (https://cran.r-project.org/web/packages/dabestr) and similarly has its source code accessible on GitHub (https://github.com/ACCLAB/dabestr). A tutorial for the R package is available at https://acclab.github.io/dabestr/. Both repositories are released under the Apache-2.0 license. In addition, https://www.estimationstats.com provides an online user interface and performs all calculations using DABEST-Python. The Python notebook used to generate Fig. 1 and Supplementary Figs. 1–3 has been deposited to Zenodo (https://doi.org/10.5281/zenodo.18426513).

References

  1. Cumming, G. & Calin-Jageman, R. Introduction to the New Statistics: Estimation, Open Science, and Beyond (Routledge, 2017).

  2. McShane, B. B., Gal, D., Gelman, A., Robert, C. & Tackett, J. L. Am. Stat. 73, 235–245 (2019).

    Article  Google Scholar 

  3. Wasserstein, R. L., Schirm, A. L. & Lazar, N. A. eds. Am. Stat. https://www.tandfonline.com/toc/utas20/73/sup1 (2019).

  4. Altman, D., Machin, D., Bryant, T. & Gardner, S. Statistics with Confidence: Confidence Intervals and Statistical Guidelines (BMJ Books, 2000).

  5. Fisher, R. A. Statistical Methods for Research Workers (Oliver & Boyd, 1925).

  6. Greenland, S. Paediatr. Perinat. Epidemiol. 35, 8–23 (2021).

    Article  PubMed  Google Scholar 

  7. Fisher, R. A. The Design of Experiments (Oliver & Boyd, 1935).

  8. Kim, H.-Y. Restor. Dent. Endod. 42, 152–155 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  9. Kosara, R. In Beautiful Visualization (ed. Steele, J. & Iliinsky, N.) 193–204 (O’Reilly Media, 2010).

  10. Goh, J. X., Hall, J. A. & Rosenthal, R. Soc. Personal. Psychol. Compass 10, 535–549 (2016).

    Article  Google Scholar 

Download references

Acknowledgements

We thank the users of DABEST-python and dabestr for feedback and J. Tamanini and Insight Editing London for critical reviews of the manuscript before submission. We thank F. Low for programming help.

Funding

This study was funded by grants from the Ministry of Education of Singapore (MOE): Z.L., R.Z., K.L., S.H., L.Z.W., Y.L., F.L. and A.R.C.G. were supported by 2022-MOET1-0001; J.A. was supported by FY2023-MOET1-0001; Y.M. was supported by a President’s Graduate Fellowship, MOE-T2EP30222-0018 (Research Scholarship) and MOE-T2EP30223-0009; N.M.L. was supported by Research Scholarships MOE2019-T2-1-133 and MOE-T2EP30222-0018; H.C. was supported in part by MOE (T2EP20223-0010) and the National Medical Research Council of Singapore (NMRC, CG21APR1008); S.X. was supported by the A*STAR Scientific Scholars Fund and NMRC (MOH-OFYIRG20nov-0051); A.C.-C. was supported by MOE (FY2022-MOET1-0001); A.C.-C. was also supported by NMRC (MOH-001397-01). The authors were supported by a Biomedical Research Council block grant to the Institute of Molecular and Cell Biology and a Duke-NUS Medical School grant to A.C.-C.

Author information

Author notes

  1. These authors contributed equally: Zinan Lu, Jonathan Anns, Yishan Mai, Sangyu Xu.

Authors and Affiliations

  1. Program in Neuroscience and Behavioural Disorders, Duke-NUS Medical School, Singapore, Singapore

    Zinan Lu, Jonathan Anns, Yishan Mai, Rou Zhang, Nicole MynYi Lee, Lucas Zhuoyu Wang, Yixuan Li, A. Rosa Castillo Gonzalez & Adam Claridge-Chang

  2. Department of Statistics and Data Science, National University of Singapore, Singapore, Singapore

    Zinan Lu, Rou Zhang, Kahseng Lian, Lucas Zhuoyu Wang & Yixuan Li

  3. Institute for Molecular and Cell Biology (IMCB), Agency for Science, Technology and Research (A*STAR), Singapore, Singapore

    Jonathan Anns, Joses Ho, Sangyu Xu & Adam Claridge-Chang

  4. School of Computing, National University of Singapore, Singapore, Singapore

    Shan Hashir

  5. Department of Medicine, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore

    Hyungwon Choi

  6. Singapore Lipidomics Incubator, Life Sciences Institute, National University of Singapore, Singapore, Singapore

    Hyungwon Choi

  7. Department of Physiology, National University of Singapore, Singapore, Singapore

    Adam Claridge-Chang

Authors
  1. Zinan Lu
    View author publications

    Search author on:PubMed Google Scholar

  2. Jonathan Anns
    View author publications

    Search author on:PubMed Google Scholar

  3. Yishan Mai
    View author publications

    Search author on:PubMed Google Scholar

  4. Rou Zhang
    View author publications

    Search author on:PubMed Google Scholar

  5. Kahseng Lian
    View author publications

    Search author on:PubMed Google Scholar

  6. Nicole MynYi Lee
    View author publications

    Search author on:PubMed Google Scholar

  7. Shan Hashir
    View author publications

    Search author on:PubMed Google Scholar

  8. Lucas Zhuoyu Wang
    View author publications

    Search author on:PubMed Google Scholar

  9. Yixuan Li
    View author publications

    Search author on:PubMed Google Scholar

  10. A. Rosa Castillo Gonzalez
    View author publications

    Search author on:PubMed Google Scholar

  11. Joses Ho
    View author publications

    Search author on:PubMed Google Scholar

  12. Hyungwon Choi
    View author publications

    Search author on:PubMed Google Scholar

  13. Sangyu Xu
    View author publications

    Search author on:PubMed Google Scholar

  14. Adam Claridge-Chang
    View author publications

    Search author on:PubMed Google Scholar

Contributions

Conceptualization: A.C.-C.; methodology: A.C.-C., H.C., Z.L., J.A., Y.M. and S.X.; software: Z.L. (Python), J.A. (Python), Y.M. (Python), R.Z. (Python), K.L. (Python, R), S.H. (Web), L.Z.W. (Python, R), Y.L. (Python), A.R.C.G. (Python, R) and S.X. (Python); data analysis: Y.M., J.A., Z.L., S.X. and N.M.L.; investigation: N.M.L., S.X.; writing — original draft: Y.M., J.A., Z.L., S.X., N.M.L. and A.C.-C.; writing — revision: Y.M., J.A., Z.L., S.X., N.M.L., A.C.-C. and H.C.; visualization: J.H., Y.M., J.A., Z.L., S.X. and N.M.L.; supervision: A.C.-C. and H.C.; project administration: A.C.-C.; funding acquisition: A.C.-C., H.C. and S.X.

Corresponding authors

Correspondence to Sangyu Xu or Adam Claridge-Chang.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Methods thanks Lewis Halsey and the other, anonymous reviewer(s) for their contribution to the peer review of this work.

Supplementary information

Supplementary Information (download PDF )

Supplementary Figs. 1–3, Notes 1–9, Tables 1–5, Methods and References

Supplementary Data (download XLSX )

Source data for Supplementary Figs. 1–3. Supp_Fig_1 contains repeated measures total sleep time (TST) data for 20 subjects across baseline and 5 treatment days. Supp_Fig_2 contains survival and tumor size data for a 2×2 factorial design with genotype (W, M) and treatment (Placebo, Drug) for 200 observations. Supp_Fig_3 contains binary seizure occurrence data for 40 subjects across 4 days under placebo and drug conditions.

Source data

Source Data Fig. 1 (download XLSX )

Statistical Source Data. Contains four tabs: Fig_1_A (repeated measures TST data), Fig_1_B (2×2 factorial survival data), Fig_1_C (proportion data for drug group), Fig_1_D (mini-meta analysis sleep data).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, Z., Anns, J., Mai, Y. et al. Getting over ANOVA: estimation graphics for multi-group comparisons. Nat Methods 23, 1924–1926 (2026). https://doi.org/10.1038/s41592-026-03187-7

Download citation

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1038/s41592-026-03187-7

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics