Figure and Table | Beyond the Pass Mark: Accuracy of ChatGPT and Bing in the National Medical Licensure Examination in Japan

Table 1. Scores by Large Language Models.

Characteristic	Compulsory questions, N = 81¹	Others, N = 200¹	Total, N = 281
Bing score	70 (86%)	149 (74%)	219 (78%)
ChatGPT score	34 (42%)	72 (36%)	106 (38%)

Table 2. Reasons for Wrong Answers by Bing.

Characteristic	Normal questions, N = 31¹	Clinical questions, N = 31¹
Misinterpretation of the meaning of the question statement	10 (32%)	7 (22.6%)
Wrong diagnosis	0 (0%)	3 (9.7%)
Wrong information with a correct diagnosis	0 (0%)	4 (13%)
Wrong information	21 (68%)	17 (55%)