I found this article at Nature by Mariana Lenharo interesting:
An artificial intelligence (AI) system trained to conduct medical interviews matched or even surpassed human doctors’ performance at conversing with simulated patients and listing possible diagnoses on the basis of the patients’ medical history1.
The chatbot, which is based on a large language model (LLM) developed by Google, was more accurate than board-certified primary-care physicians in diagnosing respiratory and cardiovascular conditions, among others. Compared with human doctors, it managed to acquire a similar amount of information during medical interviews and ranked higher on empathy.
“To our knowledge, this is the first time that a conversational AI system has ever been designed optimally for diagnostic dialogue and taking the clinical history,†says Alan Karthikesalingam, a clinical research scientist at Google Health in London and a co-author of the study1, which was published on 11 January in the arXiv preprint repository. It has not yet been peer reviewed.
I had several reactions. First, this is exactly what I have been saying for decades. If the skills required are rote memorization and using clinical findings to diagnose conditions, a properly designed algorithm will beat human physicians. But the finding that the the LLM chatbot had greater empathy is the key one.
Which leads to my second observation. I don’t know whether physicians will fight the encroachment of machines on their turf but they shouldn’t. The effect of AI on medicine will be to make human physicians more productive and accurate. But that’s not all.
AI in medicine will change how prospective physicians are selected. We won’t need rote memorization and using clinical findings to diagnose conditions as much anymore. Gone will be the days when you doctor is frozen to a computer screen. What will be needed is human physicians who can work in collaboration with these chatbots to achieve better outcomes more quickly than human physicians alone.
Having read the studies and talked with some people doing these studies what they have looking at it is responses in writing. Computers are faster so they can easily edit in more “empathetic” statements. This is, according to the article you cite, the first time that AI worked conversationally. They did it with actors. People who were given the correct prompts to lead to a diagnosis. AFAICT they weren’t time limited. In real life, there are time limits, patients lie and forget stuff, they argue with you. Also, who does the physical exam?
I think AI isn’t quite ready for prime time yet. It’s already used in some areas but I think it will find a lot of use soon as an aid or adjuvant. In the example of the interview you could have the AI listen in and provide prompts. When writing orders an AI could provide prompts and the AI could basically write the history and offer differentials. That said, I think they will take a while to fly solo.
You do realize that rote memorization is already de-emphasized in practice. You dont remember the details of rare syndromes since you can look them up. The problem is testing in the education process. That emphasizes memorization too much.
That’s largely my point.