Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • NEWS FEATURE

ChatGPT broke the Turing test — the race is on for new ways to assess AI

Large language models mimic human chatter, but scientists disagree on their ability to reason.
By
  1. Celeste Biever
Save article
View saved research
Stylised illustration showing a human and a robot head attempting to solve simple shape based problems.

Illustration by The Project Twins

The world’s best artificial intelligence (AI) systems can pass tough exams, write convincingly human essays and chat so fluently that many find their output indistinguishable from people’s. What can’t they do? Solve simple visual logic puzzles.

Access options

Access through your institution

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Nature 619, 686-689 (2023)

doi: https://doi.org/10.1038/d41586-023-02361-7

References

  1. Moskvichev, A., Odouard, V. V. & Mitchell, M. Preprint at https://arxiv.org/abs/2305.07141 (2023).

  2. Turing, A. M. Mind LIX, 433–460 (1950).

    Article  Google Scholar 

  3. Jannai, D., Meron, A., Lenz, B., Levine, Y. & Shoham, Y. Preprint at https://arxiv.org/abs/2305.20010 (2023).

  4. OpenAI. Preprint at https://arxiv.org/abs/2303.08774 (2023).

  5. Bubeck, S. et al. Preprint at https://arxiv.org/abs/2303.12712 (2023).

  6. Chollet, F. Preprint at https://arxiv.org/abs/1911.01547 (2019).

  7. Johnson, A., Vong, W. K., Lake, B. M. & Gureckis, T. M. Preprint at https://arxiv.org/abs/2103.05823 (2021).

  8. Xu, Y., Li, W., Vaezipoor, P., Sanner. S. & Khalil, E. B. Preprint at https://arxiv.org/abs/2305.18354 (2023).

  9. Li, K. et al. Proc. Eleventh Int. Conf. Learn. Represent. https://openreview.net/forum?id=DeG07_TcZvT (2023).

Download references

Related Articles

Subjects

Latest on:

Nature Careers

Jobs

  • Director General

    Lead ELI ERIC, Europe’s flagship high-power laser research infrastructure, driving scientific excellence, strategy and international collaboration.

    ELI Beamlines in Dolní Břežany (Czech Republic) and ELI ALPS in Szeged (Hungary)

    The Extreme Light Infrastructure ERIC

  • Commissioning Editor

    Title: Commissioning Editor - Collections Management and Acquisition Location: Beijing, Shanghai, Nanjing, China - hybrid working model Application...

    Beijing, Shanghai, Nanjing, China - hybrid working model

    Springer Nature Ltd

  • Scientific Program Director (PhD) - Genetics & Genomics Research

    The Scientific Program Director provides high-level scientific, regulatory, and operational leadership in support of science research programs.

    New York City, New York (US)

    Icahn School of Medicine at Mount Sinai - Genetics and Genomic Sciences

  • Faculty Positions at the Center for Machine Learning Research (CMLR), Peking University

    CMLR's goal is to advance machine learning-related research across a wide range of disciplines.

    Beijing (CN)

    Center for Machine Learning Research (CMLR), Peking University

  • Assistant or Associate Professor of Cell Biology

    Department of Cell Biology at the University of Virginia School of Medicine invites applications for Assistant or Associate Professor of Cell Biology

    Charlottesville, Virginia

    University of Virginia

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics