We Tested ChatGPT's ECG Reader. Here's What It Missed.

Hello heart hero,

You've probably used ChatGPT to interpret your blood work or get insights on symptoms. It's surprisingly helpful for a lot of health questions. But ECGs? That's where things get tricky.

So we decided to put ChatGPT's ECG Reader to the test. Can it handle basic rhythm checks? What about measuring PR and QTc intervals? And most importantly, can it catch life-threatening rhythms like atrial fibrillation or ventricular tachycardia?

We tested it against real ECGs, running each one multiple times to check for consistency. Here's what we found.

‍

The Test: 7 Real ECGs

We selected seven ECGs covering common and critical rhythms:

• Sinus Tachycardia with PVCs
• Sinus Rhythm
• SVT Sustained
• Sinus Arrhythmia with 1st Degree Block
• Atrial Fibrillation with Normal Heart Rate
• Atrial Fibrillation with High Heart Rate
• Ventricular Tachycardia

Our methodology: For each ECG, we first reviewed it using the Qaly app (with Bobby the Bot, verified by certified cardiac technicians), then asked ChatGPT to "Calculate my PQRST values and review my ECG."

‍

ECG 1: Sinus Tachycardia with PVCs

Correct Analysis:

• Heart rate: 113 bpm
• 6 PVCs, with 5 in a trigeminy pattern
• PQRST Intervals:
  • PR: 132 ms
  • QRS: 85 ms
  • QT: 337 ms (QTc 462 ms)

ChatGPT's Performance: Not great. ChatGPT failed to identify the PVCs and provided PR and QTc values significantly different from the actual measurements.

‍

ECG 2: Sinus Rhythm

This was the easiest recording in our test - normal sinus rhythm with these intervals:

• PR: 132 ms
• QRS: 85 ms
• QT: 337 ms (QTc 462 ms)

ChatGPT's Performance: Acceptable. ChatGPT correctly identified the rhythm. QRS and PR fell within the ranges it provided (though those ranges were quite broad). It declined to give a specific QTc value but correctly stated it "appears normal," albeit with low confidence.

‍

ECG 3: SVT Sustained

Correct Analysis:

• Rhythm: SVT Sustained
• PQRST Intervals:
  • PR: Not measurable (no visible P waves)
  • QRS: 85 ms
  • QT: 337 ms (QTc 462 ms)

ChatGPT's Performance: Failed. ChatGPT misread it as Sinus Tachycardia (though it did suggest SVT as a possibility). It attempted to provide a PR interval for a recording with absent P waves. It did correctly note that QRS was normal.

‍

ECG 4: Sinus Arrhythmia with 1st Degree AV Block

Correct Analysis:

• Rhythm: Sinus Arrhythmia with 1st Degree AV Block
• PQRST Intervals:
  • PR: 220 ms (Wide)
  • QRS: 144 ms (Wide)
  • QT: 435 ms (QTc 456 ms)

ChatGPT's Performance: Complete failure. It missed both the sinus arrhythmia and the 1st degree block. Additionally, it couldn't accurately measure the QRS intervals.

ECG 5: Atrial Fibrillation with Normal Heart Rate

Correct Analysis:

• Rhythm: Atrial Fibrillation with heart rate of 80 BPM
• PQRST Intervals:
  • PR: Not measurable
  • QRS: 91 ms
  • QT: 378 ms (QTc 436 ms)

ChatGPT's Performance: Pretty well! ChatGPT provided the correct diagnosis. PQRST values were within the provided ranges, and it correctly flagged atrial fibrillation.

‍

But here's the problem: We ran the same ECG through ChatGPT again with the identical prompt, and got different results. We tested this 10 times total. The results ranged from "No sign of afib" to "Definitely afib."

Out of 10 attempts:

• 2 times: Didn't detect afib
• 2 times: "Possible afib"
• 6 times: Confidently stated afib

Below you can see answers from the 2nd and 3rd checks.

‍

‍

ECG 6: Atrial Fibrillation with High Heart Rate

‍

Correct Analysis:

• Rhythm: Atrial Fibrillation with heart rate of 140 BPM
• PQRST Intervals:
  • PR: Not measurable
  • QRS: 82 ms
  • QT: 304 ms (QTc 464 ms)

ChatGPT's Performance: Failed to provide the correct rhythm. Didn't notice the absence of P waves or the irregular RR intervals, both hallmark features of atrial fibrillation.

‍

ECG 7: Ventricular Tachycardia

Correct Analysis:

• Rhythm: Ventricular Tachycardia (sustained)
• Heart rate of 154 BPM
• PQRST Intervals:
  • PR: Not measurable
  • QRS: 164 ms
  • QT: 289 ms (QTc 463 ms)

‍

ChatGPT's Performance: Missed the diagnosis entirely. While it correctly noted that PR intervals couldn't be measured, it failed to identify the wide QRS complexes - a critical finding in ventricular tachycardia.

‍

Conclusion

Is ChatGPT a reliable ECG reader? No.

Out of 7 ECGs tested, ChatGPT struggled significantly:

• Missed critical diagnoses: Failed to identify PVCs, SVT, sinus arrhythmia, 1st degree block, high-rate atrial fibrillation, and ventricular tachycardia.
• Inaccurate measurements: Provided incorrect PR and QTc values that differed significantly from actual measurements.
• Inconsistent results: When we ran the same atrial fibrillation ECG 10 times with the identical prompt, we got 10 different answers - ranging from "no afib" to "definitely afib."
• Inconsistent formatting: Beyond the diagnostic inconsistencies, ChatGPT presented results in different formats each time - sometimes using bullet points, sometimes paragraphs, sometimes tables. This lack of standardization makes it difficult to quickly compare results or extract key information reliably.
• Fundamental errors: Attempted to measure PR intervals when P waves weren't present, missed wide QRS complexes in ventricular tachycardia, and failed to recognize irregular RR intervals in atrial fibrillation.

The one bright spot? ChatGPT showed reasonable performance on atrial fibrillation with a normal heart rate.

The bottom line: ChatGPT may be helpful for general health questions, but ECG interpretation isn't one of them.