AI Chatbots Made at Least One Clinically Relevant Error in Half of Endodontic Cases

Edited by Dentaltown staff

Two widely available large language models performed acceptably on individual diagnostic tasks for teeth needing root canal treatment. However, both made at least one clinically relevant error in about half of cases when all endpoints were assessed together, according to a multicenter retrospective study published online Oct. 13 in the Journal of Endodontics.

Researchers compared ChatGPT-5.2 and Gemini 3.0 with calibrated endodontists across 645 teeth requiring initial root canal treatment. The endodontists and both models received the same standardized clinical summaries and radiographs under a fixed single-pass protocol. Each was assessed on pulpal diagnosis, periapical diagnosis, treatment planning, and case difficulty.

Accuracy on individual endpoints ranged from 74% to 88%. Composite accuracy across the three clinical endpoints was 61.1% for ChatGPT-5.2 and 59.4% for Gemini 3.0. When case difficulty was added, four-endpoint accuracy fell to 47.3% and 46.2%, respectively.

Periapical diagnosis was the strongest endpoint by macro F1 score, at 0.724 for ChatGPT-5.2 and 0.719 for Gemini 3.0. Treatment planning was the weakest, at 0.385 and 0.550, and both models performed worse in less common categories.

Treatment planning was the only endpoint on which the two models differed significantly after correction for multiple comparisons. Gemini 3.0 performed better on that endpoint (OR, 1.457; adjusted P = .003).

The authors concluded that performance was endpoint-specific and substantially reduced on integrated assessment and low-prevalence categories. They wrote that the findings support using large language models in endodontic diagnosis and treatment planning as adjunctive tools, but not without expert supervision.

Mahmoud Torabinejad of Loma Linda University School of Dentistry led the study with Patricia A. Tordik of the University of Maryland School of Dentistry and colleagues in the United States, Kuwait, and Iran. The case difficulty instrument used in the study is proprietary to GENERDENT LLC. The authors reported that none of them holds a financial interest in the company.

The models were tested on standardized written summaries and radiographs rather than in live clinical use. The authors described the findings in minority diagnostic categories as suggestive because of small subgroup sizes.

Sources:
Journal of Endodontics, “Comparing Diagnostic Ability of Two Off-the-Shelf Large Language Models with That of Endodontists for Teeth Requiring Initial Root Canal Treatment: A Multicenter Retrospective Investigation,” published online Oct. 13, 2026: doi.org/10.1016/j.joen.2026.10.003
PubMed, PMID 42869378: pubmed.ncbi.nlm.nih.gov/42869378