Accuracy of smartphone apps for heart rate measurement

Mobile electronic devices such as smartphones or tablets are gaining increasing popularity worldwide. Recent estimates suggest 7.7 billion mobile broadband subscriptions in 2020, while the number of smartphone subscriptions is expected to equal about 70% of the world's population.1 The ubiquitous use of smartphones coupled with expanding mobile broadband connectivity could change the way healthcare is accessed, monitored and delivered ('mobile health technologies', mHealth). mHealth is defined by the practice of medicine supported by portable diagnostic devices.2 For healthcare systems, the importance of mHealth strategies has been demonstrated as they may play an important role in the control of epidemic disease such as cholera3 or Ebola.4 On an individual patient level, mHealth technology can be used, for example, to increase medication adherence5,6 or control of blood glucose levels and insulin doses in patients with diabetes.7 For cardiac patients, a wide array of smartp hone-connected cardiac monitoring devices and health apps for the diagnosis and prevention of cardiovascular disease is available.8

Overall, it is estimated that the number of health-related wearable sensors will reach 80 million in 2017.9 To reach the transformative potential of mHealth, validation of the technical capabilities and accuracy as well as the clinical impact of these technologies is needed.10 As heart rate monitoring is an essential component of almost all clinical situations, in the present study we evaluated the usability and accuracy of four different heart rate measuring apps. The accuracy of the apps was compared to the electrocardiogram (ECG) and pulse oximeter-derived heart rate using medically approved professional devices.

Basic principles

Photoplethysmography (PPG) is based on the principle that blood absorbs more light than the surrounding tissue. In addition, variations in blood volume (i.e. in systole and diastole) affect the transmission or reflectance of light.11 These two principles can be used to detect blood flow. The PPG technique is commercially used in pulse oximeters for the determination of arterial blood oxygen saturation (SaO2), achieved by computing the differences of light absorption in the red and infrared range by oxygenated and deoxygenated haemoglobin.12 In contrast, heart rate detection (but not SaO2 measurement) based on PPG is feasible using only light in the visible range.11

Two different concepts of measuring heart rate by PPG are known: contact and non-contact PPG. In contact PPG, the subject places a finger on the built-in camera of the phone. The camera is placed directly on the skin and the built-in flash provides the necessary light source in the visible range for reflection by blood cells (Figure 1(a)). In non-contact PPG, the camera is used in the classic way by holding the camera in front of the patient's face (up to 1.5 m away) without the need for direct skin contact (Figure 1(b)). There is no need for a dedicated light source, ambient light is sufficient.

Figure 1. Two different concepts of measuring heart rate by PPG are know: In contact PPG, the subject places a finger on the built-in camera of the phone directly on the skin and the built in flash provides the necessary light source in the visible range for reflection by blood cells (Figure 1a). In non-contact PPG, the camera is used in the classical way by holding the camera in front of the patients face without the need for direct skin contact (Figure 1b). (a) Contact photoplethysmography; (b) non-contact photoplethysmography.

Technology

Applications (apps)

Overall, four commercially available apps have been tested (downloaded from iTunes store). For contact PPG, we used 'Instant Heart Rate' (IHR) (version 3.0.1; Azumio Inc., USA) and 'Heart Fitness' (HF) (version 2.0.3; Senscare SAS, France). For non-contact PPG, we tested 'Whats My Heart Rate' (WMH) (version 2.0; Vitrox Technologies, Malaysia) and 'Cardiio' (CAR) (version 2.0; Cardiio Inc., USA).

Mobile devices

We tested the four different apps on two different portable phones (iPhone 4, iPhone 5). The exact camera specifications of the phones are shown in Table 1. The primary camera was used for the contact PPG-based measurements, while the secondary camera was used for all non-contact measurements.

Table 1. Camera specifications.

Table 1. Camera specifications.

Study population

From April to December 2013 we randomly included adult patients requiring heart rate monitoring on the chest pain unit or the emergency room of the University Hospital Zurich, Switzerland. All patients willing to participate in the study were eligible; however, patients in critical medical conditions were excluded from this study. Patients participating in this study gave written informed consent. The study was approved by the ethics committee of the Canton of Zurich, Switzerland (KEK-ZH-NR: 2012-0524).

After recruitment, initial heart rate and rhythm was measured using a 12-lead ECG (Schiller AT-104 PC/SDS 101 and Schiller AT 102 Plus; Schiller AG, Baar, Switzerland). Heart rate was then measured simultaneously by pulse oximetry (Draeger Infinity Delta XL; Draegerwerk AG & Co. KG, Luebeck, Germany), an ECG-based monitor (Philips Intellivue X2; Koninklijke Philips NV, Amsterdam, the Netherlands) and by each app. The patient history was taken from medical records, and additional clinical characteristics were collected (age, body temperature, blood pressure and blood oxygen saturation). The graphic conceived by Jubran13 was used for grading of the pulse oximetry curve quality.

Statistics

Continuous data are expressed as medians and interquartile ranges (IQRs) or as mean ± standard deviation (SD) as appropriate, and categorical data as number and percentage (%). Categorical data were analysed using the chi-square test. To compare the values provided by the four apps to the ECG and oximeter-derived heart rate we used Pearson's correlation and Bland–Altman analysis. To assess the level of agreement between methods we computed the mean absolute error.14 Linear and multiple regression analyses were performed to test the influence of the defined factors on the mean absolute error of the four different apps. A P value of less than 0.05 was considered statistically significant. Statistical analyses were performed using Stata SE, version 13.

Clinical characteristics

We included a total of 108 patients. In total, 432 datasets (each consisting of an app-based measurement, a simultaneous ECG-based measurement and simultaneous pulseoxymetry-based measurement for each patient) were obtained. Nineteen datasets were excluded due to missing values (due to either the app-based measurement or ECG being unable to give a value). Median patient age was 65 years (range 19–99 years, IQR 52–76 years); 31% of patients were women. During the measurements, median systolic blood pressure was 126 mmHg (range 76–189 mmHg, IQR 112–142 mmHg), body temperature ranged from 35.0℃ to 37.5℃ (median 36.0℃). Median arterial oxygen saturation measured by pulse oximetry was 96% (range 62–100%, IQR 94–98%). Sinus rhythm was present in 85 patients (79%), atrial fibrillation in 11 patients (10%), ventricular pacemaker stimulation in five patients (5%) and atrial tachycardia/third-degree AV-block/junctional escape rhythm each in one patient (1%). The base line characteristics of the study population are given in Table 2. In a total of 413 measurements by ECG we recorded a wide heart rate range from 31 to 124 bpm (median 75 bpm, IQR 62–90 bpm). The quality of the oximetry pulse curve was judged to be satisfactory in 86% (IHH measurements), 96% (HF measurements), 80% (WMH measurements) and 69% (CAR measurements) of the cases.

Table 2. Baseline characteristics of patients (n = 108).

Table 2. Baseline characteristics of patients (n = 108).

ECG measurements compared to pulse oximetry measurements with approved medical devices

ECG-based measurements compared to heart rate measurement using the pulse oximetry device showed a very high level of agreement (Figure 2, Table 3). Pearson correlation r between the ECG determined heart rate and the oximetric heart rate was 0.92 (P < 0.001), showing a mean absolute error of 2 ± 0.35 bpm.

Figure 2. Heart rate measurement by electrocardiogram compared to pulse oximetry device.

Table 3. Comparison of heart rate assessed by the four different apps and medically approved pulse oximetry compared to electrocardiogram.

Table 3. Comparison of heart rate assessed by the four different apps and medically approved pulse oximetry compared to electrocardiogram.

ECG measurements compared to app-based measurements

The accuracy of heart rate measured by apps as compared to ECG, reported as mean absolute error (in bpm ± standard error) was 4.5 ± 1.1 (IHR), 2.0 ± 0.5 (HF), 7.1 ± 1.4 (WMH) and 8.1 ± 1.4 (CAR). The ECG-derived heart rate correlated well with IHR (r = 0.83) and HF (r = 0.96), but less with WMH (r = 0.62) and CAR (r = 0.60) (see Table 3). The Bland–Altman plots (Figure 3) show considerable differences between the tested apps. Non-contact PPG measurements performed significantly worse compared to fingertip-based (contact) measurements. Both non-contact PPG-based apps performed significantly worse at higher heart rates. They also have a tendency to underestimate higher heart rates.

Figure 3. The Bland-Altman plots show considerable differences between the tested apps. Non-contact PPG measurements performed significantly worse compared to contact measurements. Both non-contact PPG based apps performed significantly worse at higher heart rates. They also have a tendency to underestimate higher heart rates. (a) Contact photoplethysmography; (b) non-contact photoplethysmography.

Influencing factors in app-based measurements

Using multiple regression analyses, we were unable to demonstrate any relationship between influencing factors and mean absolute error for the contact PPG apps IHR and HF (Table 4). However, the two non-contact PPG apps performed worse, with declining body temperature and increasing heart rate. Furthermore, camera technology tended to influence the performance of the non-contact PPG apps.

Table 4. Influencing factors in App-based measurements.

Table 4. Influencing factors in App-based measurements.

The widespread use of smartphones, expanding mobile broadband connectivity and the emergence of new mHealth technologies will result in an increase of collection (and transmission) of individualised health-related data. The potential to increase patient engagement, to reduce healthcare costs and to improve outcomes is remarkable. Nevertheless, establishing reliable and clinically meaningful digital health data is problematic: in the present study, the performance of different apps for heart rate measurements by smartphone (without any external connectable sensor device) was very heterogeneous. While one app (HF) measured heart rate with comparable accuracy to pulse oximetry, other apps did not perform as well. In some apps, absolute differences of over 20 bpm (or even more) occurred in more than 20% of all measurements (see Table 5). The precise reason for WMH and CAR performing slightly worse is unclear. Theoretically they should be performing similarly. We could not reliably ide ntify an influencing factor. The two non-contact PPG apps performed significantly worse compared to fingertip-based (contact PPG) apps. This may be to due to inferior camera technology (in both apps the front camera with less advanced specifications was used) as well as generally a more demanding measurement environment. Uncontrolled ambient light may make it more difficult to discriminate between two distinct peaks in the pulse wave. Furthermore, the accuracy of non-contact PPG apps was significantly worse at higher heart rates, which is in line with previously published data.15,16 In a different more recent study comparing the accuracy of wrist-worn heart rate monitors using contact PPG technology, no such effect was observed.16 One possible explanation for the worse accuracy of non-contact PPG could be the inferior resolution of the secondary camera. The trend to more reliable performance in next generation devices (iPhone5 vs. iPhone4) observed in the present study could prognos ticate better performance in the future using more advanced camera technology.

Table 5. App-performance grouped by differences (bpm) to ECG.

Table 5. App-performance grouped by differences (bpm) to ECG.

While some apps are developed using evidence-based guidelines and are continuously evaluated,17 most remain unevaluated. The sheer number of unregulated apps can lead to consumer confusion, reluctant promotion by healthcare providers and unnecessary consumption of healthcare resources. It is largely unclear how we should develop the resources necessary for administrating digital health services and the requirement for healthcare personnel to monitor the wave of incoming patient-generated data. Besides these logistic aspects, several other issues are problematic and need to be addressed (medico-legal liability, personal data safety, health insurance collaborations, reimbursement, incorporation in governmental healthcare systems/programmes, etc.).

Nevertheless, mHealth technologies potentially offer amazing opportunities for patients, healthcare providers, researchers and healthcare delivery systems. To identify the most effective and robust technologies for clinical use it is mandatory to create an evidence base that validates generated measurements and assesses the impact of specific mHealth products and concepts on healthcare quality, cost and outcomes.

We found substantial performance differences between the four studied heart rate measuring apps. The two contact PPG-based apps had higher feasibility and better accuracy for heart rate measurement than the two non-contact PPG-based apps. Careful analysis of app accuracy is warranted before using these apps in clinical practice.

TC, CT, CAW, AB, PB, JH and TFL contributed to the conception of the work. TC, CT, CAW, AB, AAT, FS and SM contributed to the acquisition, analysis, or interpretation of data for the work. TC, CAW and PB drafted the manuscript. All authors critically revised the manuscript, gave final approval and agree to be accountable for all aspects of work ensuring integrity and accuracy.

Acknowledgements

The use of the phones in the study was provided by Swisscom for free. At the end of the study, all data were deleted on the phones, and all phones were returned to Swisscom. Measurements were performed by the medical and nursing staff of the cardiology clinic at the University Hospital of Zurich.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

The authors received no financial support for the research, authorship, and/or publication of this article.

1. The Broadband Commission for Digital Development. The State of Broadband 2015. 2015; http://www.broadbandcommission.org/doccuments/reports/bb-annualreport2015.pdf. 2. Bhavnani SP, Narula J, Sengupta PP. Mobile technology and the digitization of healthcare. Eur Heart J 2016; 37: 1428–1438–1428–1438. Google Scholar CrossRef 3. Bengtsson L, Lu X, Thorson A, . Improved response to disasters and outbreaks by tracking population movements with mobile phone network data: a post-earthquake geospatial study in Haiti. PLoS Med 2011; 8: e1001083–e1001083. Google Scholar CrossRef, Medline 4. O'Donovan J, Bersin A. Controlling Ebola through mHealth strategies. Lancet Global Health 2015; 3: e22–e22. Google Scholar CrossRef, Medline 5. Anglada-Martinez H, Riu-Viladoms G, Martin-Conde M, . Does mHealth increase adherence to medication? Results of a systematic review. Int J Clin Pract 2015; 69: 9–32. Google Scholar CrossRef, Medline 6. Dale LP, Dobson R, Whittaker R, . The effectiveness of mobile-health behaviour change interventions for cardiovascular disease self-management: a systematic review. Eur J Prev Cardiol 2016; 23: 801–817. Google Scholar Link 7. Årsand E, Muzny M, Bradway M, . Performance of the first combined smartwatch and smartphone diabetes diary application study. J Diabet Sci Technol 2015; 9: 556–563. Google Scholar Link 8. Nguyen HH, Silva JN. Use of smartphone technology in cardiology. Trends Cardiovasc Med 2016; 26: 376–386. Google Scholar CrossRef, Medline 9. Swan M. Sensor mania! the internet of things, wearable computing, objective metrics, and the quantified self 2.0. J Sensor Actuator Netw 2012; 1: 217–253. Google Scholar CrossRef 10. Bruining N, Caiani E, Chronaki C, . Acquisition and analysis of cardiovascular signals on smartphones: potential, pitfalls and perspectives. Eur J Prev Cardiol 2014; 21(2 Suppl): 4–13. Google Scholar Link 11. Verkruysse W, Svaasand LO, Nelson JS. Remote plethysmographic imaging using ambient light. Optics Express 2008; 16: 21434–21445. Google Scholar CrossRef, Medline 12. Mendelson Y. Pulse oximetry: theory and applications for noninvasive monitoring. Clin Chem 1992; 38: 1601–1607. Google Scholar Medline 13. Jubran A. Pulse oximetry. Crit Care 1999; 3: R11–R17. Google Scholar CrossRef, Medline 14. Willmott CJ, Matsuura K. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate Res 2005; 30: 79–79. Google Scholar CrossRef 15. Wackel P, Beerman L, West L, . Tachycardia detection using smartphone applications in pediatric patients. J Pediatr 2014; 164: 1133–1135. Google Scholar CrossRef, Medline 16. Wang R, Blackburn G, Desai M, et al. Accuracy of wrist-worn heart rate monitors. JAMA Cardiol 2017; 2: 104–106. 17. McManus DD, Chong JW, Soni A, . PULSE-SMART: Pulse-based arrhythmia discrimination using a novel smartphone application. J Cardiovasc Electrophysiol 2016; 27: 51–57. Google Scholar CrossRef, Medline 18. Lal Shimpi A, Klug B and Gowri V. The iPhone 5 Review. AnandTech; 2012. http://www.anandtech.com/show/6630/the-iphone-5-review (accessed 20 March 2017).
Source: Accuracy of smartphone apps for heart rate measurement