So will the AI scribe know to omit the guacamole discussion in the summary? That would be impressive but if it doesn't my concern is that the physician wouldn't notice because they hardly bothered to read it. Getting docs to read and edit their dictations has always been challenging (and let's not get started on cut and paste), much less an AI generated summary. I retired before AI and LLM's came on the scene, and always being a little techie, I am sure I would have worked with it, but your concerns are real. On the other hand, I have some experience with AI listening to and generating meeting minutes, and it's impressive. Maybe it will finally make EHR's user friendly and efficient . . . we can only hope.
Hi Timothy - I agree with all of your commentary. A colleague of mine who has been trying to integrate ambient AI into visits for note writing says that the process of editing and revising consumes as much time and effort as generating documentation herself. Except for same day "sick visits" which are pretty focused, in which case the AI does well and the balancing acts are much easier. I would estimate that >95% of my visits these days are complex level 4/5, hospital follow ups, preops, etc and the problem list usually hits the double digits. I don't think AI would document that synergistic juggling act the way a human mind can, and a big part of the assessment and plan is formulated as I "tell the story." This also helps me pick up the story where we left off when I next see the patient. So far it's been a decent copilot at times, but not a co-author!
I think loneliness and isolation is killing people. Humans need that connection-- a genuine, organic, biological exchange of energy, ideas, and eye contact. Pushing that precious connection off on AI so humans can get our numbers up for our private equity overlords is not the answer.
Well, its a tool and can be useful but also could not as well. In my early days of FORTRAN programing, AI has always been a goal of computer scientists. However, much caution are needed. Translating organic reasoning is to be trended lightly. My real concern is AI and Data Centers may not be sustainable as they required huge amount of resources. Hell, they're restarting reactor one at Three Miles Island for Microsoft AI program. We still haven't figured out what to do with the waste! https://stateimpact.npr.org/pennsylvania/2024/09/25/fear-hope-among-mixed-reactions-as-three-mile-island-plans-to-restart/
Totally share that concern - the amount of energy and water being consumed by AI is incredible... not to mention the game of mining crypto. I grew up for part of my childhood 50 miles from 3 mile island and was going on 4 yo when the partial meltdown happened. Now I live in Philly, about 95 miles away and downwind. Nuclear is a tough coin to flip as you know - minimal greenhouse emissions but nuclear waste and potential for disaster/sabotage. Thanks for sharing, I did not know about the Microsoft deal
Bravo Dr. McCormick! I'm so glad to hear that there are still humans that prefer to do things the old fashion way, by doing it themselves. I'm really worried about all the damage this AI take-over is about to unleash. I know it's capable of doing many wondrous things, but, like everything else, it is a double sided blade and I fear the bad side will far outweigh the good it can do.
Thanks Edwina :) I share a lot of your concern and optimism at the same time, but I do think AI stands to hollow out the middle of our workforce. Skilled laborers (electricians, plumbers, etc), creatives, and professionals who work with humans using obligate interpersonal skills (healthcare?) will still have meaningful work... but if you read Yuval Noah Harari's "21 Lessons for the 21st Century" it leaves you with great apprehension about the eventual "uselessness" of humans.
Another beautifully written reflection on an important topic. (Assuming it wasn't written by AI!)
I have virtually no experience with AI in my current situation. My days are filled with humans who care for me physically, emotionally, spiritually and mentally and for whom I care as best I can. I'm temporarily out of the AI world - as far as I know. There is this phone in my hand that, I suppose, could be doing all kinds of AI-ish things of which I am blissfully unaware!
I do think there is a place for AI but I am convinced that it will never be superior to the human brain. The challenge will be to use it well, as a tool, and to always maintain a high degree of oversight.
Hopefully not easier said than done.
There are also those who see AI as being a key ingredient to the next big leap in human evolution...for now I'll hang out in my privileged world of healing and caring for others.
Hi Gemma! So wonderful to hear from you! You are definitely one of the worlds mensches and we need more people like you to keep this whole fabric of community and civilization together. I don't think there is any stopping of progress, but I do hope we slow it down long enough to cooperate with other nations and establish guardrails. But with the world being an angry place right now, and with so many desperate challenges on the horizon like climate change, AI seems like another Hail Mary to save us from our current problems that feel too big to fix.
Wishing you continued care and recovery - physically, emotionally, and spiritually as you've said. We all deserve such love.
There is a very interesting new Netflix series called “What’s Next, The Future With Bill Gates” . The first episode is all about AI, its development and future possibilities. I think you would find it very interesting. ( And frightening).
OK, will definitely add that to my queue, thank you. There is also a documentary about the gut microbiome that I intend to watch as recommended to me. So much fascinating stuff going on, it's important to give all this at least equal time as the politics/intrigue :
Reading about these AI reports with citations on MedPage amused me when I read that the programs sometimes INVENTED citations! I had a colleague I wanted to accuse of that but he was too honest. I can well believe it of AI. I think uploading diagnostics schematics would be helpful especially when you’re tired and could use a second ear but all the double checking would be exhausting, too. Interesting article. I’m glad you’re beating back burnout by writing, Doc.(1)
😆excellent reference and source quotation! AI is awful with citing sources. I’ve asked some clinical questions and included a request to cite specific journal articles and Claude.ai and ChatGPT absolutely make up nonsense. Perplexity does cite real sources but it’s often stuff like webmd and commercial sites rather than academic. A real blind spot with using these, and if I didn’t have clinical/medical background I would not risk it.
Yikes, AI is listening when I talk to my doctor?! Are there any HIPA protections? Or can the corporation that owns the AI take what I tell my doctor and share it with other corporations? This is very concerning.
Really good points, and LLM developers are always looking for more data to train their models with… so I’m sure the doctor patient conversations are a huge potential mine. Supposedly Dragon Ambient will get smarter and match your documenting style the more you work with and correct it. I wouldn’t mind trying it for simple reasons, privacy and HIPAA aside, but most visits these days are not simple.
Here is what was released about HIPAA compliance when these rolled out a year ago+
“… strict data agreements with its customers, and the data is fully encrypted and runs in HIPAA-compliant environments.”
Thank you for this reveal of AI in medicine and how you have balanced the benefits and risks with the choices you make in your practice. As a physician it makes me feel glad to be retired. I admire the choices you have made. Marty
Thanks Marty, and everyone I speak with who has retired in healthcare or education feels the same. So much changing, and so many technologies intruding on the core aspects of each service profession - caring, humanism, and autonomy. And yet things are improving in other ways with tech. May we live in interesting times…
Eloquent, insightful article. So many insights, it’s hard to choose, but I’ll pick this one: “I believe that the synthesis, reflection, intuition, and partly subconscious construction of a patient narrative is a laborious but essential way to make sense of chaos while trying to help another human being.”
Related to this, there is almost always going to be a gap in knowledge between what the medical professional can see on examining and conversing with the patient and what the patient knows and is able to understand. That gap can only be filled through conversation in which the medical professional teases out together with the patient where the gaps are and finds language in which to explain in a way that enables them to understand one another. I have also found it invaluable to review visit notes that capture those conversations. For one, it helps me to see whether gaps remain and therefore what might need to be discussed in a subsequent visit. It is hard to envision an AI solution to this that would capture and adequately record what the medical professional and patient take away from their interactions.
Also, I don’t know whether this is AI related or not, but I have observed recently an increase in physician visit notes that appear to be automated—reams of boilerplate one can easily find online, but set forth as if each item was discussed with the patient in a physician visit, when that is not remotely true (eg a recommendation for PT, which was not discussed, but absolutely should have been). This not only has the potential to degrade confidence in the patient-physician relationship, but also to distort the medical record so much that it cannot be relied upon. That seems very worrisome to me.
Eloquent, insightful comment! So many insights, it’s hard to choose, but I’ll pick this one 😉 you are absolutely correct about the bloat from boilerplate templates and data dumps into patient notes that are geared towards fulfilling minimum billing requirements for complexity of each visit. Many physicians think EMRs are billing platforms first, clinical visit documents second. I also resist the templates and do my notes in a narrative fashion as much as possible. Not perfect or absolutely achieved, but to break my mind and someone’s complaints into a million shattered checkboxes destroys much of the nuance and effectiveness of the history… and most of the diagnosis and plan are still built on the old fashioned history even before we get data.
“Many physicians think EMRs are billing platforms first, clinical visit documents second.” Fascinating, that. All so very backward when it comes to patient care! Good for you for resisting!
So will the AI scribe know to omit the guacamole discussion in the summary? That would be impressive but if it doesn't my concern is that the physician wouldn't notice because they hardly bothered to read it. Getting docs to read and edit their dictations has always been challenging (and let's not get started on cut and paste), much less an AI generated summary. I retired before AI and LLM's came on the scene, and always being a little techie, I am sure I would have worked with it, but your concerns are real. On the other hand, I have some experience with AI listening to and generating meeting minutes, and it's impressive. Maybe it will finally make EHR's user friendly and efficient . . . we can only hope.
Hi Timothy - I agree with all of your commentary. A colleague of mine who has been trying to integrate ambient AI into visits for note writing says that the process of editing and revising consumes as much time and effort as generating documentation herself. Except for same day "sick visits" which are pretty focused, in which case the AI does well and the balancing acts are much easier. I would estimate that >95% of my visits these days are complex level 4/5, hospital follow ups, preops, etc and the problem list usually hits the double digits. I don't think AI would document that synergistic juggling act the way a human mind can, and a big part of the assessment and plan is formulated as I "tell the story." This also helps me pick up the story where we left off when I next see the patient. So far it's been a decent copilot at times, but not a co-author!
I think loneliness and isolation is killing people. Humans need that connection-- a genuine, organic, biological exchange of energy, ideas, and eye contact. Pushing that precious connection off on AI so humans can get our numbers up for our private equity overlords is not the answer.
Thank you Jill, and glad to have you in "La Résistance!"
Loneliness and isolation are definitely harming us.
Even though I am still more cautious than most since Covid, I feel euphoric sometimes getting together with friends and family.
Well, its a tool and can be useful but also could not as well. In my early days of FORTRAN programing, AI has always been a goal of computer scientists. However, much caution are needed. Translating organic reasoning is to be trended lightly. My real concern is AI and Data Centers may not be sustainable as they required huge amount of resources. Hell, they're restarting reactor one at Three Miles Island for Microsoft AI program. We still haven't figured out what to do with the waste! https://stateimpact.npr.org/pennsylvania/2024/09/25/fear-hope-among-mixed-reactions-as-three-mile-island-plans-to-restart/
Sigh
Totally share that concern - the amount of energy and water being consumed by AI is incredible... not to mention the game of mining crypto. I grew up for part of my childhood 50 miles from 3 mile island and was going on 4 yo when the partial meltdown happened. Now I live in Philly, about 95 miles away and downwind. Nuclear is a tough coin to flip as you know - minimal greenhouse emissions but nuclear waste and potential for disaster/sabotage. Thanks for sharing, I did not know about the Microsoft deal
Hey, check this out
THE LOST DATA: HOW AI
SYSTEMS CENSOR LGBTQ+
CONTENT IN THE NAME OF
SAFETY
Many AI companies implement safety systems to protect users from offensive or inaccurate
content. Though well intentioned, these filters can exacerbate existing inequalities, and data
shows that they have disproportionately removed LGBTQ+ content. By Sophia Chen
https://www.nature.com/articles/s43588-024-00695-4.epdf?sharing_token=2hwj5nAwbUDqvXcRlAGkgNRgN0jAjWel9jnR3ZoTv0MAKklQkco3YJORrMqrz0TGLFmtZH_bXIoCJzHy0rNGdStYS_KiMUhg3yTGPyBbYLOWWTfC47k94SpzR8dBybf_RBZoTMxdQAo62B03TcqmGSlsxPP4Q1Jt_UCDQKGBMzg%3D
Bravo Dr. McCormick! I'm so glad to hear that there are still humans that prefer to do things the old fashion way, by doing it themselves. I'm really worried about all the damage this AI take-over is about to unleash. I know it's capable of doing many wondrous things, but, like everything else, it is a double sided blade and I fear the bad side will far outweigh the good it can do.
Thanks Edwina :) I share a lot of your concern and optimism at the same time, but I do think AI stands to hollow out the middle of our workforce. Skilled laborers (electricians, plumbers, etc), creatives, and professionals who work with humans using obligate interpersonal skills (healthcare?) will still have meaningful work... but if you read Yuval Noah Harari's "21 Lessons for the 21st Century" it leaves you with great apprehension about the eventual "uselessness" of humans.
Will a Woebot ever give me a steep discount because I told him a new joke he could tell his MD friends? (True story!).
Ha! I have an older man in his 80s who always ends out visits with a great joke - part performance, part cognitive exercise, and always a joy!
Another beautifully written reflection on an important topic. (Assuming it wasn't written by AI!)
I have virtually no experience with AI in my current situation. My days are filled with humans who care for me physically, emotionally, spiritually and mentally and for whom I care as best I can. I'm temporarily out of the AI world - as far as I know. There is this phone in my hand that, I suppose, could be doing all kinds of AI-ish things of which I am blissfully unaware!
I do think there is a place for AI but I am convinced that it will never be superior to the human brain. The challenge will be to use it well, as a tool, and to always maintain a high degree of oversight.
Hopefully not easier said than done.
There are also those who see AI as being a key ingredient to the next big leap in human evolution...for now I'll hang out in my privileged world of healing and caring for others.
Every day is a bit of a challenge...
Hi Gemma! So wonderful to hear from you! You are definitely one of the worlds mensches and we need more people like you to keep this whole fabric of community and civilization together. I don't think there is any stopping of progress, but I do hope we slow it down long enough to cooperate with other nations and establish guardrails. But with the world being an angry place right now, and with so many desperate challenges on the horizon like climate change, AI seems like another Hail Mary to save us from our current problems that feel too big to fix.
Wishing you continued care and recovery - physically, emotionally, and spiritually as you've said. We all deserve such love.
There is a very interesting new Netflix series called “What’s Next, The Future With Bill Gates” . The first episode is all about AI, its development and future possibilities. I think you would find it very interesting. ( And frightening).
OK, will definitely add that to my queue, thank you. There is also a documentary about the gut microbiome that I intend to watch as recommended to me. So much fascinating stuff going on, it's important to give all this at least equal time as the politics/intrigue :
https://www.netflix.com/title/81436688
Reading about these AI reports with citations on MedPage amused me when I read that the programs sometimes INVENTED citations! I had a colleague I wanted to accuse of that but he was too honest. I can well believe it of AI. I think uploading diagnostics schematics would be helpful especially when you’re tired and could use a second ear but all the double checking would be exhausting, too. Interesting article. I’m glad you’re beating back burnout by writing, Doc.(1)
1- Gummere S and Marren S 2024
😆excellent reference and source quotation! AI is awful with citing sources. I’ve asked some clinical questions and included a request to cite specific journal articles and Claude.ai and ChatGPT absolutely make up nonsense. Perplexity does cite real sources but it’s often stuff like webmd and commercial sites rather than academic. A real blind spot with using these, and if I didn’t have clinical/medical background I would not risk it.
Yikes, AI is listening when I talk to my doctor?! Are there any HIPA protections? Or can the corporation that owns the AI take what I tell my doctor and share it with other corporations? This is very concerning.
Really good points, and LLM developers are always looking for more data to train their models with… so I’m sure the doctor patient conversations are a huge potential mine. Supposedly Dragon Ambient will get smarter and match your documenting style the more you work with and correct it. I wouldn’t mind trying it for simple reasons, privacy and HIPAA aside, but most visits these days are not simple.
Here is what was released about HIPAA compliance when these rolled out a year ago+
“… strict data agreements with its customers, and the data is fully encrypted and runs in HIPAA-compliant environments.”
https://www.cnbc.com/amp/2023/03/20/microsoft-nuance-announce-clinical-notes-application-powered-by-openai.html
Hello Ryan,
Thank you for this reveal of AI in medicine and how you have balanced the benefits and risks with the choices you make in your practice. As a physician it makes me feel glad to be retired. I admire the choices you have made. Marty
Thanks Marty, and everyone I speak with who has retired in healthcare or education feels the same. So much changing, and so many technologies intruding on the core aspects of each service profession - caring, humanism, and autonomy. And yet things are improving in other ways with tech. May we live in interesting times…
Eloquent, insightful article. So many insights, it’s hard to choose, but I’ll pick this one: “I believe that the synthesis, reflection, intuition, and partly subconscious construction of a patient narrative is a laborious but essential way to make sense of chaos while trying to help another human being.”
Related to this, there is almost always going to be a gap in knowledge between what the medical professional can see on examining and conversing with the patient and what the patient knows and is able to understand. That gap can only be filled through conversation in which the medical professional teases out together with the patient where the gaps are and finds language in which to explain in a way that enables them to understand one another. I have also found it invaluable to review visit notes that capture those conversations. For one, it helps me to see whether gaps remain and therefore what might need to be discussed in a subsequent visit. It is hard to envision an AI solution to this that would capture and adequately record what the medical professional and patient take away from their interactions.
Also, I don’t know whether this is AI related or not, but I have observed recently an increase in physician visit notes that appear to be automated—reams of boilerplate one can easily find online, but set forth as if each item was discussed with the patient in a physician visit, when that is not remotely true (eg a recommendation for PT, which was not discussed, but absolutely should have been). This not only has the potential to degrade confidence in the patient-physician relationship, but also to distort the medical record so much that it cannot be relied upon. That seems very worrisome to me.
Eloquent, insightful comment! So many insights, it’s hard to choose, but I’ll pick this one 😉 you are absolutely correct about the bloat from boilerplate templates and data dumps into patient notes that are geared towards fulfilling minimum billing requirements for complexity of each visit. Many physicians think EMRs are billing platforms first, clinical visit documents second. I also resist the templates and do my notes in a narrative fashion as much as possible. Not perfect or absolutely achieved, but to break my mind and someone’s complaints into a million shattered checkboxes destroys much of the nuance and effectiveness of the history… and most of the diagnosis and plan are still built on the old fashioned history even before we get data.
“Many physicians think EMRs are billing platforms first, clinical visit documents second.” Fascinating, that. All so very backward when it comes to patient care! Good for you for resisting!
You seem to be engaged with your patients, which is admirable, so a certain amount of skepticism about LLMs is appropriate for you. Well done.
However in other contexts, where doctors have to process patients much the same way that abattoirs process pigs, AI might definitely be a help.