AI Tools as a “Second Opinion” in Medicine: Nimrod Harel – May 2025

Mr. Nimrod Harel, in his lecture “AI Tools as a ‘Second Opinion’ in Medicine,” delivered at the national conference of the Responsible AI in Medicine Initiative at Tel Aviv University, held in collaboration with the Ministry of Innovation, Science and Technology and the Steinhardt Museum of Natural History on May 5, 2025, described a fundamental problem with large language models (LLMs) in medicine: they are often not calibrated — meaning the confidence they report in their answers does not match the actual probability of correctness.

He explained that while these models demonstrate impressive capabilities, their lack of calibration makes them less reliable in critical settings such as healthcare. As a partial response to this problem, he proposed a “second opinion” approach, where the model is asked the same question repeatedly, with irrelevant external information added, and the level of agreement between the different answers is analyzed relative to the model’s stated confidence compared to its original answer.