Advertisement
Mayo Clinic Proceedings Home
MCP Digital Health Home

The Digitization and Decentralization of Clinical Trials

  • David M. Harmon
    Correspondence
    Correspondence: Address to David M. Harmon, MD, Department of Internal Medicine, Mayo Clinic, 200 First St, Rochester, MN 55905 USA.
    Affiliations
    Department of Internal Medicine, Mayo Clinic School of Graduate Medical Education, Mayo Clinic, Rochester, MN, USA

    Department of Cardiovascular Medicine, Mayo Clinic College of Medicine, Mayo Clinic, Rochester, MN, USA
    Search for articles by this author
  • Peter A. Noseworthy
    Affiliations
    Department of Cardiovascular Medicine, Mayo Clinic College of Medicine, Mayo Clinic, Rochester, MN, USA
    Search for articles by this author
  • Xiaoxi Yao
    Affiliations
    Department of Cardiovascular Medicine, Mayo Clinic College of Medicine, Mayo Clinic, Rochester, MN, USA

    Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, USA
    Search for articles by this author
Published:January 18, 2023DOI:https://doi.org/10.1016/j.mayocp.2022.10.001

      Abstract

      Now, more than ever, digital technology has made its way into the daily lives of billions across the globe, and the widespread use of this technology has also allowed a digital window into consumers’ and patients’ daily lives, respectively. In a similar way, the practice of medicine has digitally evolved with the application of electronic health records and development of wearable/portable consumer-based medical devices (eg, Apple Watch ECG and Kardia Mobile by AliveCor). Alongside the increased use of digital technology in clinical care (eg, telehealth and wearable arrhythmia detection), clinical investigators have harnessed this powerful stockpile of data to gain insight into what happens to patients beyond the clinic walls. In this thematic review, we show the impact of digital advancements on the clinical trial process from recruitment and enrollment to interventions and data collection. We also show the pragmatism of this decentralized process and how it will mitigate the limitations of conventional randomized controlled trials. Finally, while pushing the boundaries of tech, we also describe a few limitations of this rapidly growing field to understand better what gaps need to be bridged in the future.

      Abbreviations and Acronyms:

      AI (artificial intelligence), ECG (electrocardiogram), EHR (electronic health record), HFrEF (heart failure with reduced ejection fraction), mHealth (mobile health), RCT (randomized controlled trial)
      Now, more than ever, digital technology has made its way into the daily lives of billions across the globe.
      United Nations High-Level Panel on Digital Cooperation
      The Age of Digital Interdependence: Report of the UN Secretary-General’s High-Level Panel on Digital Cooperation. Digital Cooperation Report.
      ,
      • O’Dea S.
      Number of Smartphone Subscriptions Worldwide From 2016 to 2021, With Forecasts From 2022 to 2027.
      In the United States alone, 85% of citizens own a smartphone, and one in five Americans own some type of wearable digital tech, with expected consumer increases in coming years.
      Pew Research Center
      Mobile Fact Sheet. April 7, 2021.
      • Vogels E.A.
      About one-in-five Americans use a smart watch or fitness tracker. Pew Research Center.
      • Samet A.
      The top medical monitoring and healthcare wearable device trends of 2022. Insider Intelligence.
      The practice of medicine has also experienced its own technologic revolution with the telemedicine boom during the coronavirus disease 2019 (COVID-19) pandemic, enabling medical care within the privacy of patients’ own homes.
      • Bestsenny O.
      • Gilbert G.
      • Harris A.
      • Rost J.
      Telehealth: A quarter-trillion-dollar post-COVID-19 reality? McKinsey & Company.
      The widespread use of this technology has also allowed a digital window into consumers’ and patients’ daily lives, respectively. Whether by active recording of a watch-based electrocardiogram (ECG) or passive monitoring of adequate sleep, the accumulation of daily, real-world data continues to expand.
      Apple
      Take an ECG with the ECG app on Apple Watch.
      Apple
      Using Apple Watch for Arrhythmia Detection.
      Apple
      Track your sleep with Apple Watch. Apple Watch User Guide.
      Alongside the increased use of digital technology in clinical care (eg, telehealth and wearable arrhythmia detection), clinical investigators have harnessed this powerful stockpile of data to gain insight into what happens to patients beyond the clinic walls.
      • Rosa C.
      • Marsch L.A.
      • Winstanley E.L.
      • Brunner M.
      • Campbell A.N.C.
      Using digital technologies in clinical trials: current and future applications.
      Whereas traditional randomized controlled trials (RCTs) have standard protocols, measures, and safety, these conventional methodologies remain costly, time-intensive, and are frequently geographically inflexible. The unprecedented COVID-19 pandemic brought the limitations of this traditional RCT approach into full relief when ongoing clinical trials were delayed (eg, experimental cancer therapies), and the urgent COVID-19–focused investigations were hampered by the usual checks and balances of RCT execution.
      • Hashem H.
      • Abufaraj M.
      • Tbakhi A.
      • Sultan I.
      Obstacles and considerations related to clinical trial research during the COVID-19 pandemic.
      Digital tools in the clinic, such as the electronic health record (EHR), and owned by patients (eg, iPhone, Apple Watch, and similar technology) have allowed investigators to both assess the needs and well-being of their patients while evaluating the various impact of interventions on their health at a personal level.
      • Rosa C.
      • Marsch L.A.
      • Winstanley E.L.
      • Brunner M.
      • Campbell A.N.C.
      Using digital technologies in clinical trials: current and future applications.
      ,
      • Marquis-Gravel G.
      • Roe M.T.
      • Turakhia M.P.
      • et al.
      Technology-enabled clinical trials: transforming medical evidence generation.
      As a result, researchers have implemented these digital tools in multifaceted, pragmatic approaches, allowing robust clinical study unbound by the geographic, clinic-centric limitations of conventional RTCs at a fraction of the cost (Figure 1).
      • Jones W.S.
      • Mulder H.
      • Wruck L.M.
      • et al.
      Comparative effectiveness of aspirin dosing in cardiovascular disease.
      • Perez M.V.
      • Mahaffey K.W.
      • Hedlin H.
      • et al.
      Large-scale assessment of a smartwatch to identify atrial fibrillation.
      Figure thumbnail gr1
      Figure 1Descriptive visualization of the traditional and digital clinical trial pathways. Although an intervention can take place in both the traditional and digital clinical trial pathways, the intervention in digital trials may be entirely digital (eg, a mobile health [mHealth] app) or may have more limitations than interventions in traditional clinical trials (eg, medications/therapy in fully digital trials would not require in-person lab appointments or physical exam).
      The digitization and decentralization of clinical trials have enabled investigators to enhance current RCT methodology and conduct cutting-edge pragmatic studies with geographically diverse, real-world data collection.
      • Perez M.V.
      • Mahaffey K.W.
      • Hedlin H.
      • et al.
      Large-scale assessment of a smartwatch to identify atrial fibrillation.
      ,
      • Yao X.
      • Rushlow D.R.
      • Inselman J.W.
      • et al.
      Artificial intelligence-enabled electrocardiograms for identification of patients with low ejection fraction: a pragmatic, randomized clinical trial.
      In this thematic review, we show the impact of digital advancements on the clinical trial process from recruitment and enrollment to interventions and data collection. We also show the pragmatism of this decentralized process and how it will mitigate the limitations of conventional RCTs. Finally, while pushing the boundaries of technology, we describe a few limitations of this rapidly growing field to understand better what gaps need to be bridged in the future.

      Digital Tools Enabling Clinical Trials

      Electronic Health Record

      The ubiquitous application of the EHR has revolutionized and streamlined routine health care. In 2017, 94% of hospitals in the United States used an EHR to provide clinical care,
      • Sonal P.
      • Henry J.
      Hospitals’ Use of Electronic Health Records Data, 2015-2017.
      and the convergence towards a unified EHR system continues to grow (eg, 45% of the entire US population has a medical record in an Epic EHR).
      Why Epic. Epic at Hopkins Medicine. John Hopkins Medicine.
      Although the EHR was primarily intended as a tool for patient-focused care, clinical investigators have used its near-universal adoption in various aspects of studies.
      Although EHR use in retrospective studies is now commonplace,
      • Gianfrancesco M.A.
      • Goldstein N.D.
      A narrative review on the validity of electronic health record-based research in epidemiology.
      this tool has recently found utility in nearly all facets of clinical trials.
      • Marquis-Gravel G.
      • Roe M.T.
      • Turakhia M.P.
      • et al.
      Technology-enabled clinical trials: transforming medical evidence generation.
      In a few salient examples, the EHR was used as a tool for assessing patient eligibility and engagement (Table). The EHR took center stage in the landmark ADAPTABLE (Aspirin — Dosing a Patient-centric Trial Assessing Benefits and Long-term Effectiveness) and mSToPS (mHealth Screening to Prevent Strokes) trials as investigators used coded diagnostic information (International Classification of Diseases, Ninth Revision and Tenth Revision) hosted in local EHRs or insurance claims (ADAPTABLE via PCORnet) to identify eligible patients with specific inclusion and exclusion criteria. Investigators digitally contacted eligible patients for each trial, enabling a no-contact, pragmatic enrollment.
      • Jones W.S.
      • Mulder H.
      • Wruck L.M.
      • et al.
      Comparative effectiveness of aspirin dosing in cardiovascular disease.
      ,
      • Marquis-Gravel G.
      • Roe M.T.
      • Robertson H.R.
      • et al.
      Rationale and design of the Aspirin Dosing-a Patient-centric Trial Assessing Benefits and Long-term Effectiveness (ADAPTABLE) Trial.
      ,
      • Steinhubl S.R.
      • Mehta R.R.
      • Ebner G.S.
      • et al.
      Rationale and design of a home-based trial using wearable sensors to detect asymptomatic atrial fibrillation in a targeted population: the mHealth Screening to Prevent Strokes (mSToPS) trial.
      In a similar fashion, the ADAPTABLE and mSToPS investigators used the available EHR and insurance claims information for endpoint ascertainment. Whereas ADAPTABLE maintained a hybrid approach, including virtual follow-up visits, combinations of patient-reported outcomes with claims data, and nonstandardized requirement methods between participating centers, mSToPS did not use routine follow-up visits or patient-reported outcomes and relied on coded data.
      • Marquis-Gravel G.
      • Roe M.T.
      • Robertson H.R.
      • et al.
      Rationale and design of the Aspirin Dosing-a Patient-centric Trial Assessing Benefits and Long-term Effectiveness (ADAPTABLE) Trial.
      • Steinhubl S.R.
      • Mehta R.R.
      • Ebner G.S.
      • et al.
      Rationale and design of a home-based trial using wearable sensors to detect asymptomatic atrial fibrillation in a targeted population: the mHealth Screening to Prevent Strokes (mSToPS) trial.
      • Kochar A.
      • Summers M.B.
      • Benziger C.P.
      • et al.
      Clinician engagement in the ADAPTABLE (Aspirin Dosing: a Patient-centric Trial Assessing Benefits and Long-term Effectiveness) trial.
      These unique and successful studies established a framework of how to use existing data and infrastructure in trial methodologies, setting a precedent for future digital and decentralized study. This approach also allows investigators to maximize the pragmatism regarding the follow-up. Based on the Pragmatic Explanatory Continuum Indicator Summary (PRECIS-2), the most commonly used tool to guide pragmatic trial design, the most pragmatic approach would be to have no more follow-up with participants than would be the case in usual care, and to obtain outcome data by other means.
      • Loudon K.
      • Treweek S.
      • Sullivan F.
      • Donnan P.
      • Thorpe K.E.
      • Zwarenstein M.
      The PRECIS-2 tool: designing trials that are fit for purpose.
      TableComparison of Hybrid and Digital Trial Elements and Pitfalls
      AI, artificial intelligence; ASA, aspirin; ASCVD, atherosclerotic cardiovascular disease; BP, blood pressure; BEAGLE, Batch Enrollment for AI-Guided Intervention to Lower Neurologic Events in Unrecognized AF ; EAGLE, ECG AI-Guided Screening for Low Ejection Fraction ; ECG, electrocardiogram; EHR, electronic health record; HEART4U, name of the app utilized in the digital-health based heart healt trial: Usefulness of Cardiovascular Disease (CVD) Management Solution ; HF, heart failure; mHealth, mobile health app; MAFA, mobile atrial fibrillation application; mSTOPS, mHealth Screening to Prevent Strokes; NLP, natural language processing; PALM, Patient and Provider Assessment of Lipid Management; PPG, photoplethysmography; RCT, randomized controlled trial; TTE, transthoracic echocardiogram
      StudyStudy objectiveStudy typeKey digital trial elementsPitfalls of digital study
      ADAPTABLE
      • Marquis-Gravel G.
      • Roe M.T.
      • Robertson H.R.
      • et al.
      Rationale and design of the Aspirin Dosing-a Patient-centric Trial Assessing Benefits and Long-term Effectiveness (ADAPTABLE) Trial.
      Outpatient ASA dosing for ASCVD preventionPragmatic, open-label, patient-centered, randomized clinicalUse of EHR to screen for eligibility

      Novel use of PCORnet both for eligibility and event ascertainment

      Digitized consent

      Maintained hybrid approach
      Variability of recruitment between difference centers

      ± Accuracy of claims data

      ± Transferability of EHR protocol between centers
      Apple Heart
      • Perez M.V.
      • Mahaffey K.W.
      • Hedlin H.
      • et al.
      Large-scale assessment of a smartwatch to identify atrial fibrillation.
      Detection of atrial fibrillation by smart watch PPGProspective, single-group, open-label, site-less, pragmatic studyEntirely digital trial (recruitment, consent, patch monitor intervention)

      High-yield digital recruitment/enrollment

      Passive cardiac monitoring by patient-owned device
      Technologic barrier; use of patient-owned devices

      Total dependency on participants for process and completion of study

      Many did not return monitor patches
      mSTOPS
      • Steinhubl S.R.
      • Mehta R.R.
      • Ebner G.S.
      • et al.
      Rationale and design of a home-based trial using wearable sensors to detect asymptomatic atrial fibrillation in a targeted population: the mHealth Screening to Prevent Strokes (mSToPS) trial.
      Detection of atrial fibrillation by wearable ECG patchDecentralized RCT; fully remote enrollment and participationInsurance claims to identify high-risk patients for enrollment

      Digital consent

      Zio patches mailed to patients
      Lots of dropout (many steps)

      ± Accuracy of claims data

      No formal follow-up for new diagnosis
      Hauwei Heart
      • Guo Y.
      • Wang H.
      • Zhang H.
      • et al.
      Mobile photoplethysmographic technology to detect atrial fibrillation.
      Detection of atrial fibrillation by smart watchPragmatic, single-group, open-label, site-less studyUse of patient-owned technology to detect atrial fibrillation

      Hybrid approach with telehealth or in-person diagnostic visit if atrial fibrillation suspected

      Follow up treatment with MAFA app
      Technologic barrier; use of patient-owned device

      Had to connect with centralized healthcare system (MAFA network telehealth provider or hospital)

      Compatible tech issue (ie, 25% of watch owners did not have compatible phone for app)
      EAGLE
      • Yao X.
      • Rushlow D.R.
      • Inselman J.W.
      • et al.
      Artificial intelligence-enabled electrocardiograms for identification of patients with low ejection fraction: a pragmatic, randomized clinical trial.
      Implementation of AI-ECG to detect HF in clinical settingPragmatic, cluster, RCTEHR-based intervention

      NLP used to improve accuracy of patient data extraction

      Low-cost, novel application of AI in clinical practice
      Required in-person assessment to be included at single medical system

      Reliance on provider teams to respond to digital trial notifications
      BEAGLE
      • Yao X.
      • Attia Z.I.
      • Behnken E.M.
      • et al.
      Batch enrollment for an artificial intelligence-guided intervention to lower neurologic events in patients with undiagnosed atrial fibrillation: rationale and design of a digital clinical trial.
      Prospective detection of atrial fibrillation by AI-ECGProspective batch-enrolled, patient-centered, nonrandomized cohortEntirely site-less design from enrollment to follow-up

      Video-based enrollment and consent

      Novel investigation no of AI enhanced ECG and EHR-based NLP algorithms for previously undiagnosed atrial fibrillation in at-risk patients
      EHR-based recruitment, although <50% of patients use EHR portal

      Potential technology limitations for patients without cameras —mediated by phone calls/mail

      Although NLP is more accurate than insurance claims, still not infallible
      Mayo Apple Watch
      • Attia Z.I.
      • Harmon D.M.
      • Dugan J.
      • et al.
      Prospective evaluation of smartwatch-enabled detection of left ventricular dysfunction [published online ahead of print, 2022 Nov 14].
      Detection of heart failure by smart watch AI-ECGDecentralized, pragmatic, observational studyEntirely digital process

      Significant recruitment and active participation from enrollees

      App-based reminders for patient-activated data collection

      Streamlined patient care via centralized institution-based dashboard
      Technologic barrier; use of patient-owned device

      Limited “diseased” cohort with valid TTE in acceptable time frame

      Housed within a single medical system
      PALM
      • Fanaroff A.C.
      • Li S.
      • Webb L.E.
      • et al.
      An observational study of the association of video- versus text-based informed consent with multicenter trial enrollment: lessons from the PALM study (Patient and Provider Assessment of Lipid Management).
      Assessment of video-based informed consent versus plain textMulticenter, observational studyFaster speed to first patient enrolled at video-consent sites

      More easily able to enroll older and non-White participants to overarching PALM study
      No significant difference in overall enrollment number

      Difficult to account for confounding, as general population demographics were unknown

      Site-specific observations; not patient-to-patient observation reported
      HEART4U
      • Kang S.H.
      • Baek H.
      • Cho J.
      • et al.
      Management of cardiovascular disease using an mHealth tool: a randomized clinical trial.
      mHealth vs standard of care for patients with ASCVDProspective randomized, single-center, open-label trialGood follow-up at 6-mo endpoint for intervention/control

      More frequent app used trend with lower BP
      Minimal difference with ASCVD and other outcome measures mHealth application

      Highlights difficulty of mHealth tool impacting care beyond the interpersonal
      Sleep Health Web
      • Deering S.
      • Grade M.M.
      • Uppal J.K.
      • et al.
      Accelerating research with technology: rapid recruitment for a large-scale web-based sleep study.
      Large-scale recruitment for mHealth use as well as examine sleep quality and daytime functionProspective, single-group, open-label, site-less studyEntirely digital process for large-scale recruitment

      Allowed participants to use personal wearable (primarily FitBit) to track activity

      Web-based app rather than brand-specific
      Potential variability of performance between activity trackers

      Internet access needed

      Bias from cohort demographics (93% women, mostly white)
      a AI, artificial intelligence; ASA, aspirin; ASCVD, atherosclerotic cardiovascular disease; BP, blood pressure; BEAGLE, Batch Enrollment for AI-Guided Intervention to Lower Neurologic Events in Unrecognized AF ; EAGLE, ECG AI-Guided Screening for Low Ejection Fraction ; ECG, electrocardiogram; EHR, electronic health record; HEART4U, name of the app utilized in the digital-health based heart healt trial: Usefulness of Cardiovascular Disease (CVD) Management Solution ; HF, heart failure; mHealth, mobile health app; MAFA, mobile atrial fibrillation application; mSTOPS, mHealth Screening to Prevent Strokes; NLP, natural language processing; PALM, Patient and Provider Assessment of Lipid Management; PPG, photoplethysmography; RCT, randomized controlled trial; TTE, transthoracic echocardiogram
      Although these are only a few examples in a rapidly growing and evolving field of study, the EHR is not free of its limitations. Data validity, patient privacy, and variable EHR interoperability are a few barriers that could significantly impact EHR-based clinical trials at each level.
      • Marquis-Gravel G.
      • Roe M.T.
      • Turakhia M.P.
      • et al.
      Technology-enabled clinical trials: transforming medical evidence generation.
      In an attempt to overcome the limitations of secondary data, our team developed and used digital phenotyping algorithms that leverage both the structured data (eg, age, sex, lab tests, diagnosis codes, and procedure codes) and unstructured data, such as clinical notes abstracted via natural language processing.
      • Yao X.
      • Attia Z.I.
      • Behnken E.M.
      • et al.
      Batch enrollment for an artificial intelligence-guided intervention to lower neurologic events in patients with undiagnosed atrial fibrillation: rationale and design of a digital clinical trial.
      ,
      • Wen A.
      • Fu S.
      • Moon S.
      • et al.
      Desiderata for delivering NLP to accelerate healthcare AI advancement and a Mayo Clinic NLP-as-a-service implementation.
      In routine clinical practice, these EHR-based phenotyping algorithms can abstract patients’ medical history to provide individualized recommendations at the point of care,
      • Kaggal V.C.
      • Elayavilli R.K.
      • Mehrabi S.
      • et al.
      Toward a learning health-care system - knowledge delivery at the point of care empowered by big data and NLP.
      as well as capture patient subsequent outcomes from the EHR and feed the data into dashboards to track the quality of care. In clinical trials, the same set of tools can be used to determine trial eligibility and capture trial endpoints. These tools will facilitate the pragmatic conduct of clinical trials in terms of embedding research into practice, rather than using an expensive parallel research system separate from routine clinical practice to enroll and follow patients. When the investigators collect feedback via patient reports or clinician chart reviews, these phenotyping algorithms can be further refined, thereby fostering a continuous cycle of learning and improvement.

      Digitization of the Consent Process

      In-person recruitment and consent are arduous parts of clinical trials. Digital technologies (eg, smart tablets and smartphones) and platforms (eg, EHR, World Wide Web, and web-based smartphone applications) have allowed for the digitization of the consent process mediating the burden of research efforts while facilitating a consent process both inside and out of the clinic walls. This seemingly simple step of going paperless for the consent process is a digital leap enabling “site-less” trials to enroll patients from the comfort of their homes across the globe at dramatic rates.
      • Perez M.V.
      • Mahaffey K.W.
      • Hedlin H.
      • et al.
      Large-scale assessment of a smartwatch to identify atrial fibrillation.
      ,
      • Deering S.
      • Grade M.M.
      • Uppal J.K.
      • et al.
      Accelerating research with technology: rapid recruitment for a large-scale web-based sleep study.
      Many teams have used web- or app-based study sites to host educational material and the consent process (Table).
      • Marquis-Gravel G.
      • Roe M.T.
      • Robertson H.R.
      • et al.
      Rationale and design of the Aspirin Dosing-a Patient-centric Trial Assessing Benefits and Long-term Effectiveness (ADAPTABLE) Trial.
      ,
      • Steinhubl S.R.
      • Mehta R.R.
      • Ebner G.S.
      • et al.
      Rationale and design of a home-based trial using wearable sensors to detect asymptomatic atrial fibrillation in a targeted population: the mHealth Screening to Prevent Strokes (mSToPS) trial.
      ,
      • Deering S.
      • Grade M.M.
      • Uppal J.K.
      • et al.
      Accelerating research with technology: rapid recruitment for a large-scale web-based sleep study.
      Studies such as the ADAPTABLE and PALM (Patient and Provider Assessment of Lipid Management) trials have also uploaded pre-recorded videos as part of the preconsent education material.
      • Marquis-Gravel G.
      • Roe M.T.
      • Robertson H.R.
      • et al.
      Rationale and design of the Aspirin Dosing-a Patient-centric Trial Assessing Benefits and Long-term Effectiveness (ADAPTABLE) Trial.
      ,
      • Fanaroff A.C.
      • Li S.
      • Webb L.E.
      • et al.
      An observational study of the association of video- versus text-based informed consent with multicenter trial enrollment: lessons from the PALM study (Patient and Provider Assessment of Lipid Management).
      Interoperability between digital platforms facilitates a uniform consent process whether the patient is approached in a clinic (eg, use of tablet-based consent) or at home (eg, study website with education) because digitized education material can be made available on these platforms simultaneously. This flexible approach also allows patients who encounter a digital barrier (eg, lack of Internet access/smartphone ownership) to participate by other means, such as in-person recruitment using a study-owned tablet for preconsent education and digital consent.
      • Jones W.S.
      • Mulder H.
      • Wruck L.M.
      • et al.
      Comparative effectiveness of aspirin dosing in cardiovascular disease.
      ,
      • Marquis-Gravel G.
      • Roe M.T.
      • Robertson H.R.
      • et al.
      Rationale and design of the Aspirin Dosing-a Patient-centric Trial Assessing Benefits and Long-term Effectiveness (ADAPTABLE) Trial.

      Smartphone/Applications

      As smartphones have become an indispensable device for most Americans, clinical researchers identified this resource as a powerful investigative tool. From recruitment and enrollment to intervention and data collection, smartphones with customized study-based mobile health (mHealth) applications have quickly become focal to many pragmatic digital trials (Table). Furthermore, these portable technologies have allowed clinicians and researchers a window into study subjects’ daily lives.
      In the groundbreaking Apple Heart study, 400,000 participants were recruited and enrolled via iPhone application in just 9 months.
      • Perez M.V.
      • Mahaffey K.W.
      • Hedlin H.
      • et al.
      Large-scale assessment of a smartwatch to identify atrial fibrillation.
      Although only a fraction of the study participants who received “irregular heart beat” notifications completed patch monitoring for atrial fibrillation (n=450), this study highlighted the promise of massively scaled studies outside the clinic walls using readily available, widely used digital technology.
      • Turakhia M.P.
      • Desai M.
      • Hedlin H.
      • et al.
      Rationale and design of a large-scale, app-based study to identify cardiac arrhythmias using a smartwatch: the Apple Heart study.
      Many teams across the globe have implemented this similar approach, either recruiting and enrolling patients digitally into clinical studies (eg, Huawei Heart Study) or using mHealth applications/wearable devices for data collection (HEART4U, Sleep Health Web Study).
      • Guo Y.
      • Wang H.
      • Zhang H.
      • et al.
      Mobile photoplethysmographic technology to detect atrial fibrillation.
      ,
      • Kang S.H.
      • Baek H.
      • Cho J.
      • et al.
      Management of cardiovascular disease using an mHealth tool: a randomized clinical trial.
      ,
      • Deering S.
      • Grade M.M.
      • Uppal J.K.
      • et al.
      Accelerating research with technology: rapid recruitment for a large-scale web-based sleep study.
      Our own team used a similar approach, recruiting patients via Mayo Clinic Patient App for an analogous smartwatch study.
      • Attia Z.I.
      • Harmon D.M.
      • Dugan J.
      • et al.
      Prospective evaluation of smartwatch-enabled detection of left ventricular dysfunction [published online ahead of print, 2022 Nov 14].
      More than 2400 engaged participants were digitally enrolled and transmitted over 120,000 patient-recorded Apple Watch ECGs via study application in a short time frame of 6 months.
      • Attia Z.I.
      • Harmon D.M.
      • Dugan J.
      • et al.
      Prospective evaluation of smartwatch-enabled detection of left ventricular dysfunction [published online ahead of print, 2022 Nov 14].

      Wearable Tech Integration With EHR

      There is a gap, however, between consumer-owned products with medical applications (eg, Apple Watch and the AliveCore Kardia device) and clinician-accessible EHR data. For example, consumers must download a PDF of a manually recorded Apple Watch ECG and send a copy to their preferred medical provider via email/patient portal.
      Apple
      Take an ECG with the ECG app on Apple Watch.
      Ideally, integrating these two data sources to allow provider review in real-time could streamline patient care.
      Our recently presented Apple Watch study describes a proof-of-concept process in which real-time recorded Apple Watch data was securely and automatically transmitted via study app to an EHR-linked individualized ECG dashboard.
      • Attia Z.I.
      • Harmon D.M.
      • Dugan J.
      • et al.
      Prospective evaluation of smartwatch-enabled detection of left ventricular dysfunction [published online ahead of print, 2022 Nov 14].
      This dashboard was accessible to all providers within our institution and was updated with new patient-triggered ECG recordings within a matter of minutes, allowing for a near real-time review of patient-transmitted data (Figure 2).
      • Attia Z.I.
      • Harmon D.M.
      • Dugan J.
      • et al.
      Prospective evaluation of smartwatch-enabled detection of left ventricular dysfunction [published online ahead of print, 2022 Nov 14].
      It is our hope this proof-of-concept provides a framework for remote patient care by integrating these two valuable data sources.
      Figure thumbnail gr2
      Figure 2Case example for a patient with new atrial fibrillation on Apple watch electrocardiogram (ECG) without prior clinical history of atrial fibrillation. The streamlined digital care created during the Mayo Apple Watch study allowed providers to see patients’ ECG recordings in real time through an electronic health record (EHR)–linked ECG dashboard via use of study application. Data from Attia et al.
      • Attia Z.I.
      • Harmon D.M.
      • Dugan J.
      • et al.
      Prospective evaluation of smartwatch-enabled detection of left ventricular dysfunction [published online ahead of print, 2022 Nov 14].

      Decentralization of Clinical Trials

      With the advances in digital and medical technology, there has been a shift toward the decentralization of medical care with increased telemedicine visits, patient-to-care team portal messaging, and even remote-robotic procedures.
      • Bestsenny O.
      • Gilbert G.
      • Harris A.
      • Rost J.
      Telehealth: A quarter-trillion-dollar post-COVID-19 reality? McKinsey & Company.
      ,
      • Madder R.
      Robot surgery could be the future of health care in remote areas.
      Clinical trials, frequently limited by clinical setup and geographic location, have also followed this pathway of decentralization (Table). Using the digital tools described above (eg, EHR, digitized consent, and smartphones), patients can be recruited, enrolled, and followed-up in an entirely digitized, site-less fashion. On a similar note, wearable technologies (eg, smartwatch, wearable ECG patches, and handheld ECG devices) enable both passive (eg, recording triggered by device) and active (eg, patient-triggered recording) remote data collection.
      • Perez M.V.
      • Mahaffey K.W.
      • Hedlin H.
      • et al.
      Large-scale assessment of a smartwatch to identify atrial fibrillation.
      ,
      • Guo Y.
      • Wang H.
      • Zhang H.
      • et al.
      Mobile photoplethysmographic technology to detect atrial fibrillation.
      As these decentralized studies have become more popular, investigators have come up against barriers to implementation. For example, the Apple Heart and Huawei Heart studies identified consumers already familiar with the technology interface. Use of patient-owned devices for data collection allowed for efficient enrollment and data collection from large, geographically diverse populations.
      • Perez M.V.
      • Mahaffey K.W.
      • Hedlin H.
      • et al.
      Large-scale assessment of a smartwatch to identify atrial fibrillation.
      ,
      • Guo Y.
      • Wang H.
      • Zhang H.
      • et al.
      Mobile photoplethysmographic technology to detect atrial fibrillation.
      ,
      • Attia Z.I.
      • Harmon D.M.
      • Dugan J.
      • et al.
      Prospective evaluation of smartwatch-enabled detection of left ventricular dysfunction [published online ahead of print, 2022 Nov 14].
      ,
      • Deering S.
      • Grade M.M.
      • Uppal J.K.
      • et al.
      Accelerating research with technology: rapid recruitment for a large-scale web-based sleep study.
      Many of these individuals familiar with technology were younger (implying a more technology-savvy/tech-ownership cohort), and in studies identifying atrial fibrillation, the prevalence of disease was relatively low as a result (0.52% with an irregular pulse on Apple Heart; 0.23% in Huawei).
      • Perez M.V.
      • Mahaffey K.W.
      • Hedlin H.
      • et al.
      Large-scale assessment of a smartwatch to identify atrial fibrillation.
      ,
      • Guo Y.
      • Wang H.
      • Zhang H.
      • et al.
      Mobile photoplethysmographic technology to detect atrial fibrillation.
      Some of these decentralized processes require active participation from the study subjects, placing responsibility on the patient for personal data collection. Although it can be anticipated that participation may wane over the study period, push-notifications and app-based reminders may encourage continued, active participation during the study period.
      • Kang S.H.
      • Baek H.
      • Cho J.
      • et al.
      Management of cardiovascular disease using an mHealth tool: a randomized clinical trial.
      ,
      • Morawski K.
      • Ghazinouri R.
      • Krumme A.
      • et al.
      Association of a smartphone application with medication adherence and blood pressure control: the MedISAFE-BP randomized clinical trial.
      • Lee J.L.
      • Foschini L.
      • Kumar S.
      • et al.
      Digital intervention increases influenza vaccination rates for people with diabetes in a decentralized randomized trial.
      In our own institution-based Apple Watch study, our study app would remind participants to record an ECG every 14 days to assist with retention.
      • Attia Z.I.
      • Harmon D.M.
      • Dugan J.
      • et al.
      Prospective evaluation of smartwatch-enabled detection of left ventricular dysfunction [published online ahead of print, 2022 Nov 14].
      Although there are significant benefits and investigative implications of the decentralized study process, there are notable pitfalls. First, recruiting and enrolling patients is no small effort. Automated messages and pre-recorded material alleviate the burden placed on research coordinators in a traditional in-person approach; however, response rates from eligible patients to digital invitations and completion of digital consent with appropriate enrollment can be low (sometimes <10%).
      • Steinhubl S.R.
      • Mehta R.R.
      • Ebner G.S.
      • et al.
      Rationale and design of a home-based trial using wearable sensors to detect asymptomatic atrial fibrillation in a targeted population: the mHealth Screening to Prevent Strokes (mSToPS) trial.
      ,
      • Morawski K.
      • Ghazinouri R.
      • Krumme A.
      • et al.
      Association of a smartphone application with medication adherence and blood pressure control: the MedISAFE-BP randomized clinical trial.
      As this digital recruitment and enrollment process is rather pragmatic, patients may also incorrectly “qualify” themselves for a study when they would have otherwise been excluded (eg, patients with a known diagnosis of atrial fibrillation participating in a study to detect first-time atrial fibrillation).
      • Perez M.V.
      • Mahaffey K.W.
      • Hedlin H.
      • et al.
      Large-scale assessment of a smartwatch to identify atrial fibrillation.
      Even when patients develop interest and appropriately complete enrollment after receiving a recruitment email or portal message, participation may decline throughout the study period or, as exemplified in the mSToPS study, patients may not complete all the necessary steps for data collection (more than one-third of patients who received the Zio patch did not wear the patch during the study period).
      • Steinhubl S.R.
      • Waalen J.
      • Edwards A.M.
      • et al.
      Effect of a home-based wearable continuous ECG monitoring patch on detection of undiagnosed atrial fibrillation: the mSToPS Randomized Clinical trial.
      Finally, there are limitations regarding follow-up in these decentralized trials. Although some studies may have a centralized physician hub to complete virtual follow-up visits,
      • Jones W.S.
      • Mulder H.
      • Wruck L.M.
      • et al.
      Comparative effectiveness of aspirin dosing in cardiovascular disease.
      ,
      • Perez M.V.
      • Mahaffey K.W.
      • Hedlin H.
      • et al.
      Large-scale assessment of a smartwatch to identify atrial fibrillation.
      ,
      • Guo Y.
      • Wang H.
      • Zhang H.
      • et al.
      Mobile photoplethysmographic technology to detect atrial fibrillation.
      others have simply informed patients of a potential diagnosis with the recommendation to see a physician thus placing the burden back on the patient who may or may not have a primary care connection.
      • Morawski K.
      • Ghazinouri R.
      • Krumme A.
      • et al.
      Association of a smartphone application with medication adherence and blood pressure control: the MedISAFE-BP randomized clinical trial.
      Even when in contact with one of these virtual providers, there may be limited opportunity for novel treatment intervention beyond use of over-the-counter medication, commonly used prescriptions, or recommendations to follow-up with a local provider.
      • Jones W.S.
      • Mulder H.
      • Wruck L.M.
      • et al.
      Comparative effectiveness of aspirin dosing in cardiovascular disease.
      ,
      • Morawski K.
      • Ghazinouri R.
      • Krumme A.
      • et al.
      Association of a smartphone application with medication adherence and blood pressure control: the MedISAFE-BP randomized clinical trial.
      The Huawei Heart study not only detected atrial fibrillation by wearable photoplethysmography technology successfully, but also successfully initiated anticoagulation in 80% of high-risk patients, although this intervention required some secondary in-person evaluation by an affiliated provider to confirm atrial fibrillation diagnosis.
      • Guo Y.
      • Wang H.
      • Zhang H.
      • et al.
      Mobile photoplethysmographic technology to detect atrial fibrillation.
      In light of the growing popularity of mHealth studies, the MedISAFE-BP (Medication Adherence Improvement Support App for Engagement — Blood Pressure) study made a striking observation. The investigators noted potential contamination of results, where patients in the control, “non-app user” group downloaded the Medisafe app for personal use.
      • Morawski K.
      • Ghazinouri R.
      • Krumme A.
      • et al.
      Association of a smartphone application with medication adherence and blood pressure control: the MedISAFE-BP randomized clinical trial.
      This pragmatic, real-world approach to data collection and digital technology application comes with these risks of participants making unsupervised decisions that could potentially impact and skew results.

      Pragmatism in the Digital “Real-World”

      Pragmatic clinical trials have better-informed trial stakeholders (from patients to providers and clinical investigators) of the real-world impact and performance of specific interventions.
      • Loudon K.
      • Treweek S.
      • Sullivan F.
      • Donnan P.
      • Thorpe K.E.
      • Zwarenstein M.
      The PRECIS-2 tool: designing trials that are fit for purpose.
      Although the degree of pragmatism will vary across studies, a pragmatic design seems well poised to not only evaluate if an intervention is effective and impactful, but also if it can be widely applied and flexibly used within the real-world.
      • Loudon K.
      • Treweek S.
      • Sullivan F.
      • Donnan P.
      • Thorpe K.E.
      • Zwarenstein M.
      The PRECIS-2 tool: designing trials that are fit for purpose.
      ,
      • Thorpe K.E.
      • Zwarenstein M.
      • Oxman A.D.
      • et al.
      A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers.
      For example, our team has developed and used an artificial intelligence (AI) algorithm that, when applied to a 12-lead ECG, can detect heart failure with reduced ejection fraction (HFrEF).
      • Attia Z.I.
      • Kapa S.
      • Lopez-Jimenez F.
      • et al.
      Screening for cardiac contractile dysfunction using an artificial intelligence-enabled electrocardiogram.
      Although this AI-ECG tool showed promising results, it was created and tested on a retrospective cohort, and the effectiveness in routine practice remained largely unknown. As a result, our group designated a randomized, controlled pragmatic clinical trial that integrated the results of this AI-ECG for the detection of HFrEF into the routine clinical practice of medical providers at partnered sites across our institution.
      • Yao X.
      • Rushlow D.R.
      • Inselman J.W.
      • et al.
      Artificial intelligence-enabled electrocardiograms for identification of patients with low ejection fraction: a pragmatic, randomized clinical trial.
      Providers were notified of the result indicating an increased risk of HFrEF via email from a recently performed but routinely ordered ECG.
      • Yao X.
      • Rushlow D.R.
      • Inselman J.W.
      • et al.
      Artificial intelligence-enabled electrocardiograms for identification of patients with low ejection fraction: a pragmatic, randomized clinical trial.
      This study revealed the actual usefulness of this technology, with intervention-group providers successfully identifying HFrEF more frequently than their control-group counterparts.
      • Yao X.
      • Rushlow D.R.
      • Inselman J.W.
      • et al.
      Artificial intelligence-enabled electrocardiograms for identification of patients with low ejection fraction: a pragmatic, randomized clinical trial.
      Although this pragmatic RCT is a single example of a digital tool enhancing clinical care, there have been many digital trials that highlight pragmatism as a key value of the study.
      • Jones W.S.
      • Mulder H.
      • Wruck L.M.
      • et al.
      Comparative effectiveness of aspirin dosing in cardiovascular disease.
      ,
      • Perez M.V.
      • Mahaffey K.W.
      • Hedlin H.
      • et al.
      Large-scale assessment of a smartwatch to identify atrial fibrillation.
      ,
      • Guo Y.
      • Wang H.
      • Zhang H.
      • et al.
      Mobile photoplethysmographic technology to detect atrial fibrillation.
      ,
      • Attia Z.I.
      • Harmon D.M.
      • Dugan J.
      • et al.
      Prospective evaluation of smartwatch-enabled detection of left ventricular dysfunction [published online ahead of print, 2022 Nov 14].
      ,
      • Deering S.
      • Grade M.M.
      • Uppal J.K.
      • et al.
      Accelerating research with technology: rapid recruitment for a large-scale web-based sleep study.
      In many cases, pragmatic trial design is essential for understanding the clinical impact. Some digital interventions may hold great promise but do not show significant impact with first-time, real-world application among broad populations.
      • Kang S.H.
      • Baek H.
      • Cho J.
      • et al.
      Management of cardiovascular disease using an mHealth tool: a randomized clinical trial.
      Given the flexibility of pragmatic trials, results from these unexpected events can inform trialists on how to lead re-application efforts of digital tech via different methodologies to improve intervention delivery, participant adherence, and primary outcomes.
      • Perez M.V.
      • Mahaffey K.W.
      • Hedlin H.
      • et al.
      Large-scale assessment of a smartwatch to identify atrial fibrillation.
      ,
      • Yao X.
      • Attia Z.I.
      • Behnken E.M.
      • et al.
      Batch enrollment for an artificial intelligence-guided intervention to lower neurologic events in patients with undiagnosed atrial fibrillation: rationale and design of a digital clinical trial.
      ,
      • Kang S.H.
      • Baek H.
      • Cho J.
      • et al.
      Management of cardiovascular disease using an mHealth tool: a randomized clinical trial.
      It is also worth understanding that fully digital, decentralized observational trials and RCTs will have inherent elements of pragmatism as many of these digital tools are designed for use in daily life (eg, mHealth, wearable ECG, and smartphone application).

      The Hybrid Approach

      From this review, it is clear there are many tools enabling the digitization and decentralization of clinical trials. Although one may think of these trials and methodology as one or the other, the hybrid approach to clinical trials, incorporating elements of both these trial techniques, is common, and is likely more present than originally imagined. It would be most beneficial to imagine a continuum between fully traditional and fully digital clinical trials with steps of EHR use, digital consent, mHealth/wearable technology, and no requirement of in-person visits as steps between the two (Figure 3). From our examples in this review, ADAPTABLE, Huawei Heart, Apple Heart, PALM, and Mayo’s AI-ECG for HFrEF in the community all incorporated elements of both traditional clinical trials (ie, an in-person assessment or in-person enrollment) and digital trials (ie, digital consent and mHealth use) (Figure 3).
      • Perez M.V.
      • Mahaffey K.W.
      • Hedlin H.
      • et al.
      Large-scale assessment of a smartwatch to identify atrial fibrillation.
      ,
      • Yao X.
      • Rushlow D.R.
      • Inselman J.W.
      • et al.
      Artificial intelligence-enabled electrocardiograms for identification of patients with low ejection fraction: a pragmatic, randomized clinical trial.
      ,
      • Marquis-Gravel G.
      • Roe M.T.
      • Robertson H.R.
      • et al.
      Rationale and design of the Aspirin Dosing-a Patient-centric Trial Assessing Benefits and Long-term Effectiveness (ADAPTABLE) Trial.
      ,
      • Guo Y.
      • Wang H.
      • Zhang H.
      • et al.
      Mobile photoplethysmographic technology to detect atrial fibrillation.
      ,
      • Fanaroff A.C.
      • Li S.
      • Webb L.E.
      • et al.
      An observational study of the association of video- versus text-based informed consent with multicenter trial enrollment: lessons from the PALM study (Patient and Provider Assessment of Lipid Management).
      As technology in the medical field continues to advance, fully digital trials, as well as hybrid digital/traditional trials will be more commonplace.
      Figure thumbnail gr3
      Figure 3The continuum of the hybrid clinical trial. As innovations and technology revolutionize medical practice, elements of digital trials will frequently be incorporated to previously traditional clinical trials allowing for a hybrid approach to trials. EHR, electronic health record; mHealth, mobile health application.

      Limitations of the Technology

      Although the availability of technology continues to improve, the digital divide persists, particularly in the underrepresentation of minorities and rural residents in these digital studies (Figure 4) (Supplemental Table, available online at http://www.mayoclinicproceedings.org).
      • Rosa C.
      • Marsch L.A.
      • Winstanley E.L.
      • Brunner M.
      • Campbell A.N.C.
      Using digital technologies in clinical trials: current and future applications.
      There is evidence that varying forms of educational material for the consent process, such as tablet-based video modules, have allowed better inclusion of individuals typically marginalized by the digital divide.
      • Fanaroff A.C.
      • Li S.
      • Webb L.E.
      • et al.
      An observational study of the association of video- versus text-based informed consent with multicenter trial enrollment: lessons from the PALM study (Patient and Provider Assessment of Lipid Management).
      It has also been offered that the total cost of study participation and the interventions used in digital/decentralized trials come at a fraction of the cost both to the researchers and the participants (eg, frequent use of patient’s own technology, no travel to centralized trial site, etc). However, it must be considered that the technology itself (eg, digital tablet, wearable ECG devices, and wireless Internet cost) may remain unattainable or unaffordable to the medically underserved/underrepresented (Figure 4).
      • Rosa C.
      • Marsch L.A.
      • Winstanley E.L.
      • Brunner M.
      • Campbell A.N.C.
      Using digital technologies in clinical trials: current and future applications.
      ,
      • Noonan D.
      • Simmons L.A.
      Navigating nonessential research trials during COVID-19: the push we needed for using digital technology to increase access for rural participants?.
      Figure thumbnail gr4
      Figure 4Bridging the digital divide by identifying the patients frequently marginalized in the clinical trial process (eg, technologic and medical marginalization). Through use of flexible, digital, and remote technology, these groups may be able to participate in clinical trials which may otherwise be inaccessible to them.
      As previously mentioned, the types of interventions available also have their own set of limitations in digital trials (Supplemental Table). Although telemedicine visits may be coordinated and patients may be followed closely, medical intervention outside of over-the-counter medication or minimal-low risk prescriptions will frequently need some in-person diagnostic testing or physical assessment.
      • Jones W.S.
      • Mulder H.
      • Wruck L.M.
      • et al.
      Comparative effectiveness of aspirin dosing in cardiovascular disease.
      ,
      • Guo Y.
      • Wang H.
      • Zhang H.
      • et al.
      Mobile photoplethysmographic technology to detect atrial fibrillation.
      ,
      • Laggis C.W.
      • Williams V.L.
      • Yang X.
      • Kovarik C.L.
      Research techniques made simple: teledermatology in clinical trials.
      However, with the significant increase in telemedicine care and medical, technological advancement over the past few years, there will be a continued shift toward entirely digital/virtual clinical visits and enrollment in clinical drug trials or other interventions previously requiring in-person evaluation.
      • Mayfield J.J.
      • Chatterjee M.A.
      • Noseworthy P.A.
      • et al.
      Implementation of a fully remote randomized clinical trial with cardiac monitoring.

      Conclusion

      The digitization and pragmatic decentralization of clinical trials is a significant growing frontier in medicine. Using digital technology in various aspects of a clinical study, investigators can rapidly complete vital research with decreased cost and high participation. Further, using this technology to move trials entirely outside the clinic walls enables pragmatic study participation and provides impactful real-world data for investigators. While touting the decreased cost and high participation in such studies, we must take care to promote inclusion of those who are marginalized by the digital divide allowing flexibility in study design.

      Potential Competing Interests

      Dr Harmon has received support from the NIH StARR Resident Investigator Award (NIH 5R38HL150086-02). Dr Noseworthy has received research funding from National Institutes of Health (NIH, including the National Heart, Lung, and Blood Institute [NHLBI, R21AG 62580-1, R01HL 131535-4, R01HL 143070-2] the National Institute on Aging [NIA, R01AG 062436-1]), Agency for Healthcare Research and Quality (AHRQ, R01HS 25402-3), US Food and Drug Administration (FDA, FD 06292), and the American Heart Association (18SFRN34230146, AHA). Dr Noseworthy and Mayo Clinic have licensed an AI-ECG algorithm to AliveCor for measurement of the QT interval and have licensed several other AI-ECG algorithms to Anumana. Dr Yao reports no potential competing interests.

      Supplemental Online Material

      References

        • United Nations High-Level Panel on Digital Cooperation
        The Age of Digital Interdependence: Report of the UN Secretary-General’s High-Level Panel on Digital Cooperation. Digital Cooperation Report.
        2019
        • O’Dea S.
        Number of Smartphone Subscriptions Worldwide From 2016 to 2021, With Forecasts From 2022 to 2027.
        Statista. February 23, 2022;
        • Pew Research Center
        Mobile Fact Sheet. April 7, 2021.
        • Vogels E.A.
        About one-in-five Americans use a smart watch or fitness tracker. Pew Research Center.
        • Samet A.
        The top medical monitoring and healthcare wearable device trends of 2022. Insider Intelligence.
        • Bestsenny O.
        • Gilbert G.
        • Harris A.
        • Rost J.
        Telehealth: A quarter-trillion-dollar post-COVID-19 reality? McKinsey & Company.
        July 2021
        • Apple
        Take an ECG with the ECG app on Apple Watch.
        https://support.apple.com/en-us/HT208955
        Date: December 2021
        Date accessed: May 3, 2022
        • Apple
        Using Apple Watch for Arrhythmia Detection.
        • Apple
        Track your sleep with Apple Watch. Apple Watch User Guide.
        • Rosa C.
        • Marsch L.A.
        • Winstanley E.L.
        • Brunner M.
        • Campbell A.N.C.
        Using digital technologies in clinical trials: current and future applications.
        Contemp Clin Trials. 2021; 100106219
        • Hashem H.
        • Abufaraj M.
        • Tbakhi A.
        • Sultan I.
        Obstacles and considerations related to clinical trial research during the COVID-19 pandemic.
        Front Med (Lausanne). 2020; 7598038
        • Marquis-Gravel G.
        • Roe M.T.
        • Turakhia M.P.
        • et al.
        Technology-enabled clinical trials: transforming medical evidence generation.
        Circulation. 2019; 140: 1426-1436
        • Jones W.S.
        • Mulder H.
        • Wruck L.M.
        • et al.
        Comparative effectiveness of aspirin dosing in cardiovascular disease.
        N Engl J Med. 2021; 384: 1981-1990
        • Perez M.V.
        • Mahaffey K.W.
        • Hedlin H.
        • et al.
        Large-scale assessment of a smartwatch to identify atrial fibrillation.
        N Engl J Med. 2019; 381: 1909-1917
        • Yao X.
        • Rushlow D.R.
        • Inselman J.W.
        • et al.
        Artificial intelligence-enabled electrocardiograms for identification of patients with low ejection fraction: a pragmatic, randomized clinical trial.
        Nat Med. 2021; 27: 815-819
        • Sonal P.
        • Henry J.
        Hospitals’ Use of Electronic Health Records Data, 2015-2017.
        ONC Data Brief. No.46. 2019;
      1. Why Epic. Epic at Hopkins Medicine. John Hopkins Medicine.
        • Gianfrancesco M.A.
        • Goldstein N.D.
        A narrative review on the validity of electronic health record-based research in epidemiology.
        BMC Med Res Methodol. 2021; 21: 234
        • Marquis-Gravel G.
        • Roe M.T.
        • Robertson H.R.
        • et al.
        Rationale and design of the Aspirin Dosing-a Patient-centric Trial Assessing Benefits and Long-term Effectiveness (ADAPTABLE) Trial.
        JAMA Cardiol. 2020; 5: 598-607
        • Steinhubl S.R.
        • Mehta R.R.
        • Ebner G.S.
        • et al.
        Rationale and design of a home-based trial using wearable sensors to detect asymptomatic atrial fibrillation in a targeted population: the mHealth Screening to Prevent Strokes (mSToPS) trial.
        Am Heart J. 2016; 175: 77-85
        • Kochar A.
        • Summers M.B.
        • Benziger C.P.
        • et al.
        Clinician engagement in the ADAPTABLE (Aspirin Dosing: a Patient-centric Trial Assessing Benefits and Long-term Effectiveness) trial.
        Clin Trials. 2021; 18: 449-456
        • Loudon K.
        • Treweek S.
        • Sullivan F.
        • Donnan P.
        • Thorpe K.E.
        • Zwarenstein M.
        The PRECIS-2 tool: designing trials that are fit for purpose.
        BMJ. 2015; 350: h2147
        • Guo Y.
        • Wang H.
        • Zhang H.
        • et al.
        Mobile photoplethysmographic technology to detect atrial fibrillation.
        J Am Coll Cardiol. 2019; 74: 2365-2375
        • Yao X.
        • Attia Z.I.
        • Behnken E.M.
        • et al.
        Batch enrollment for an artificial intelligence-guided intervention to lower neurologic events in patients with undiagnosed atrial fibrillation: rationale and design of a digital clinical trial.
        Am Heart J. 2021; 239: 73-79
        • Attia Z.I.
        • Harmon D.M.
        • Dugan J.
        • et al.
        Prospective evaluation of smartwatch-enabled detection of left ventricular dysfunction [published online ahead of print, 2022 Nov 14].
        Nat Med. 2022; https://doi.org/10.1038/s41591-022-02053-1
        • Fanaroff A.C.
        • Li S.
        • Webb L.E.
        • et al.
        An observational study of the association of video- versus text-based informed consent with multicenter trial enrollment: lessons from the PALM study (Patient and Provider Assessment of Lipid Management).
        Circ Cardiovasc Qual Outcomes. 2018; 11e004675
        • Kang S.H.
        • Baek H.
        • Cho J.
        • et al.
        Management of cardiovascular disease using an mHealth tool: a randomized clinical trial.
        NPJ Digit Med. 2021; 4: 165
        • Deering S.
        • Grade M.M.
        • Uppal J.K.
        • et al.
        Accelerating research with technology: rapid recruitment for a large-scale web-based sleep study.
        JMIR Res Protoc. 2019; 8e10974
        • Wen A.
        • Fu S.
        • Moon S.
        • et al.
        Desiderata for delivering NLP to accelerate healthcare AI advancement and a Mayo Clinic NLP-as-a-service implementation.
        NPJ Digit Med. 2019; 2: 130
        • Kaggal V.C.
        • Elayavilli R.K.
        • Mehrabi S.
        • et al.
        Toward a learning health-care system - knowledge delivery at the point of care empowered by big data and NLP.
        Biomed Inform Insights. 2016; 8: 13-22
        • Turakhia M.P.
        • Desai M.
        • Hedlin H.
        • et al.
        Rationale and design of a large-scale, app-based study to identify cardiac arrhythmias using a smartwatch: the Apple Heart study.
        Am Heart J. 2019; 207: 66-75
        • Apple
        Take an ECG with the ECG app on Apple Watch.
        https://support.apple.com/en-us/HT208955
        Date: January 2022
        Date accessed: May 4, 2022
        • Madder R.
        Robot surgery could be the future of health care in remote areas.
        Fortune. February 2020;
        • Morawski K.
        • Ghazinouri R.
        • Krumme A.
        • et al.
        Association of a smartphone application with medication adherence and blood pressure control: the MedISAFE-BP randomized clinical trial.
        JAMA Intern Med. 2018; 178 ([Published correction appears in JAMA Intern Med. 2018;178(6):876]): 802-809
        • Lee J.L.
        • Foschini L.
        • Kumar S.
        • et al.
        Digital intervention increases influenza vaccination rates for people with diabetes in a decentralized randomized trial.
        NPJ Digit Med. 2021; 4: 138
        • Steinhubl S.R.
        • Waalen J.
        • Edwards A.M.
        • et al.
        Effect of a home-based wearable continuous ECG monitoring patch on detection of undiagnosed atrial fibrillation: the mSToPS Randomized Clinical trial.
        JAMA. 2018; 320: 146-155
        • Thorpe K.E.
        • Zwarenstein M.
        • Oxman A.D.
        • et al.
        A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers.
        J Clin Epidemiol. 2009; 62: 464-475
        • Attia Z.I.
        • Kapa S.
        • Lopez-Jimenez F.
        • et al.
        Screening for cardiac contractile dysfunction using an artificial intelligence-enabled electrocardiogram.
        Nat Med. 2019; 25: 70-74
        • Noonan D.
        • Simmons L.A.
        Navigating nonessential research trials during COVID-19: the push we needed for using digital technology to increase access for rural participants?.
        J Rural Health. 2021; 37: 185-187
        • Laggis C.W.
        • Williams V.L.
        • Yang X.
        • Kovarik C.L.
        Research techniques made simple: teledermatology in clinical trials.
        J Invest Dermatol. 2019; 139: 1626-1633.e1
        • Mayfield J.J.
        • Chatterjee M.A.
        • Noseworthy P.A.
        • et al.
        Implementation of a fully remote randomized clinical trial with cardiac monitoring.
        Commun Med. 2021; 1: 62