Big language models like ChatGPT-4 and Gemini can generate advice based on large amounts of data, and that's cool. But can they really be trusted with something as serious as protecting someone's brain after a surgery?

        Without proper training, just like a doctor, these models might give the best advice they have, but they might miss something important. Before we trust these algorithms, let's face some reality first.

        Picture this: PNDs, or Perioperative neurocognitive disorders, are common in older folks after surgery and anesthesia. They can make people sicker, and even lead to more deaths and big bills for healthcare. That's why big medical groups created some guidelines to prevent and treat them.

        These guidelines help doctors take steps to keep your brain safe , like keeping older people at a specific temperature and avoiding certain types of anesthesia. But if we put everyone's lives in the hands of these models, would they make the same recommendations? That's exactly the experiment! .

        We need to assess how well these large language models do at making recommendations. One critical question is can a model be trusted with complex decisions?  . Do these models actually follow the standards?

        These LLMS are awesome at generating really convincing text, but they might not be able to give the exact same medical answers you would get from a doctor following the latest guidelines

        So, we turned to a cross-sectional web-based analysis to find more.  Did ChatGPT-4 and Gemini know what they were talking about? The short answer is yes, they do pretty well. How? The models got the basics down, but there were some mistakes. And medical guidance is not a place where you want to make mistakes.  The greatest risk of using models is that we may act on incomplete or incorrect data.

        If you're going to rely on technology, you need to make sure the algorithms have been trained on the right information. The input and output needs to be testable.

        It is important to think critically about how we use technology. They are powerful tools, but computers shouldn't be replacing medical experts just yet. Trusting the wrong input will produce the wrong information.

        To do that, we really need to consider how a model is trained, what kind of information it has learned from, and what standards they have to live up to.  We need the entire medical community to trust them first.

How do Large Language Models Measure Up to Real Guidelines on Brain Health after Surgery

Actions