AI Health Grades: Fun or Frightening?

Inconsistent Grades Raise Concerns

AI doctors are making headlines, but are they ready for prime time? A recent test showed that these digital health advisors can give wildly different grades for the same data.

• AI Grades:
• One AI gave an F
• Another a B

• A third a C

• Real Doctors' Verdict: The data showed excellent health.

This inconsistency raises serious questions about the reliability of AI health tools.

The Test

The test involved feeding a decade's worth of Apple Watch data into two AI health tools:

• ChatGPT Health
• Claude for Healthcare

The goal was to see if these AI doctors could make sense of the data and provide a clear picture of the user's health. But the results were far from clear. The AI tools not only gave different grades but also made some pretty scary conclusions.

Issues with AI Doctors

Reliability Concerns

• Estimated Data: AI tools rely on estimated data like VO2 max, which is not accurately measured by an Apple Watch.
• Forgetfulness: They tend to forget basic information like age or gender during the conversation.

Data Privacy

• Encryption Promises: Companies assure users that their data will be encrypted.
• Privacy Standards: They are not held to the same privacy standards as real doctors.
• Trust Issues: Users are essentially trusting these companies with their most personal data based on little more than a promise.

Should You Trust a Digital Doctor?

• For Fun: If you enjoy looking at pretty charts of your health data, these AI tools can be fun to play around with.
• For Real Advice: If you need real medical advice, it's best to stick with a real doctor.

While AI health features may sound exciting, they are no replacement for the real thing.