Apple promoted the ability of its watch to detect atrial fibrillation. The truth does not live up to the promise altogether.

As a smartwatch, the Apple Watch was quite successful. As a medical device, the company would also like it to succeed. The recent results published in the New England Journal of Medicine from the Apple Heart Study Notice that a long way to go is still to go.

An estimated six million people in the U.S .— nearly 2 percent — have atrial fibrillation, a type of heartbeat that increases the risk of heartbeat incidents such as clots, heart attacks, and strokes.

A watch selling point is a sensor capable of measuring the pulse of a wearer and possibly detecting atrial fibrillation.

A group of researchers has recruited approximately 420,000 Apple Watch wearers in a study to assess the device’s ability to assist in diagnosis. (Some participants were employees of Apple, and Apple funded the study.) Participants were tracked for about four months. Over that time, 2,161 study participants were informed of an irregular pulse, accounting for just 0.5% of the survey.

Telemedicine appointments were given to these people, and if their symptoms were mild, electrocardiogram patches were offered to wear for up to a week to help confirm an atrial fibrillation diagnosis. Participants returned the spots and were immediately contacted and advised to receive care if the results indicated an emergency. If the tests for atrial fibrillation were positive but did not require immediate medical attention, a second telemedicine appointment was given to the participants and advised to see their regular doctor.

But for review, only 450 of the 2,161 people informed of having an irregular pulse returned their sensor patches. This indicates that it was overlooked by almost 80 percent among those who signed up for the report, wore the watch and got a health warning.

Of the 450 participants returning patches, 34 percent or 153 people reported atrial fibrillation. Of the 420,000 participants, those 153 are about 0.04 percent.

This does not mean a failure of the Apple device. This probably led to the diagnosis of some participants earlier than they might have. How many, though, are debatable, and how much of a difference it made in their wellbeing.

A topline finding was reported by many news outlets reporting on the study: an 84 percent “positive predictive value.” The figure refers to the probability that if a positive test result is obtained, somebody will eventually have the disorder.

Yet from any of the numbers above, this result was not determined. This specifically refers to the group of patients who, when wearing their confirmatory patch, had an irregular pulse warning. It’s a tiny minority of applicants. Of the 86 who received a patch update, 72 had reported signs for atrial fibrillation. (Dividing 72 by 86 yields 0.84, so you get an 84 percent positive predictive value.)

Positive predictive values are not always a good measure of the success of a test, although they are helpful when talking to patients. Of example, all positive results are right when you test a system on a population where everyone has a disease.

Other monitoring features such as sensitivity (if you have a disease, how likely the test is to be positive) and precision (if you do not have an infection, how possible the test is to be negative) are more useful in determining the overall quality of a study. However, this analysis was not designed to determine these characteristics.

There are other methods available for screening and diagnosing persons with atrial fibrillation. A systematic review of atrial fibrillation mobile health apps found 22 studies published on many of them between 2014 and 2019. Some had sensitivities and features that were very similar to the 100 ideal. Few, however, are close to the size of this study.

Only blood pressure monitoring will check for atrial fibrillation, which is omnipresent in doctor’s offices. A systematic review of them showed that they had more than 85% sensitivity, and more than 90% specificity.

But here’s the thing. Screening is an excellent idea to start with, but experts aren’t sure.

After all, if we felt strong enough to spot asymptomatic people who might have atrial fibrillation, we might screen with electrocardiograms for everyone. The Task Force on Preventive Care in the U.S. considered this among adults 65 and older who are at higher risk of stroke. The group found the evidence to be inadequate to support this, as it is not clear that this screening standard is better than the current treatment. Only getting a pulse is a pretty good indicator all by itself as part of a checkup.

There is also concern that electrocardiogram screening may result in a lot of false positives, leading to misdiagnosis and unnecessary additional testing, which involves its risks. Remember that most people who got warnings, even with the Apple Watch, did not have atrial fibrillation.

Besides, the task force concentrated on a demographic in which we could intervene: the elderly. Anticoagulation may be used to treat patients at high risk of stroke who have atrial fibrillation (i.e., older people). It’s not immediately clear how we’d handle them for younger people at lower risk, or if we would.

And it is more likely that younger people will have a smartwatch.

It should be clear that it concentrates on atrial fibrillation, which patients or physicians do not consider otherwise. Physicians will completely handle those who are already infected and those who are symptomatic, and many will be treated with medications or procedures. It is not necessary to mitigate or neglect the diagnosed and symptomatic illness.

This research provides positive messages. Automated tools can be used to track and assess individuals outside the clinical setting, and there is a demand for it as well. But for now, and based on these findings, though Apple Watch may have reasons to own, it is probably not one to use it as a full screen for atrial fibrillation.