JMIR MEDICAL INFORMATICS Ouchi et al Original Paper Automatic Estimation of the Most Likely Drug Combination in Electronic Health Records Using the Smooth Algorithm: Development and Validation Study Dan Ouchi1,2, MSc; Maria Giner-Soriano1,2,3, PhD; Ainhoa Gómez-Lumbreras1,4, PhD; Cristina Vedia Urgell1,2,3, MD; Ferran Torres5, PhD; Rosa Morros1,2,3,6, PhD 1 Fundació Institut Universitari per a la recerca a l'Atenció Primària de Salut Jordi Gol i Gurina, Barcelona, Spain 2 Facultat de Medicina. Departament de Farmacologia, Toxicologia i Terapèutica, Universitat Autònoma de Barcelona, Bellaterra (Cerdanyola del Vallés), Spain 3 Unitat de Farmacia. Servei d’Atenció Primaria Barcelonès Nord i Maresme, Institut Català de la Salut, Barcelona, Spain 4 Department of Pharmacotherapy, College of Pharmacy, University of Utah, Salt Lake City, UT, United States 5 Unitat de Bioestadística Facultat de Medicina, Universitat Autònoma de Barcelona, Bellaterra (Cerdanyola del Vallès), Spain 6 Spanish Clinical Research Network Platform, Barcelona, Spain Corresponding Author: Dan Ouchi, MSc Fundació Institut Universitari per a la recerca a l'Atenció Primària de Salut Jordi Gol i Gurina Gran Via de les Corts Catalanes 587, àtic Barcelona, 08007 Spain Phone: 34 934824110 Email: douchi@idiapjgol.info Abstract Background: Since the use of electronic health records (EHRs) in an automated way, pharmacovigilance or pharmacoepidemiology studies have been used to characterize the therapy using different algorithms. Although progress has been made in this area for monotherapy, with combinations of 2 or more drugs the challenge to characterize the treatment increases significantly, and more research is needed. Objective: The goal of the research was to develop and describe a novel algorithm that automatically returns the most likely therapy of one drug or combinations of 2 or more drugs over time. Methods: We used the Information System for Research in Primary Care as our reference EHR platform for the smooth algorithm development. The algorithm was inspired by statistical methods based on moving averages and depends on a parameter Wt, a flexible window that determines the level of smoothing. The effect of Wt was evaluated in a simulation study on the same data set with different window lengths. To understand the algorithm performance in a clinical or pharmacological perspective, we conducted a validation study. We designed 4 pharmacological scenarios and asked 4 independent professionals to compare a traditional method against the smooth algorithm. Data from the simulation and validation studies were then analyzed. Results: The Wt parameter had an impact over the raw data. As we increased the window length, more patient were modified and the number of smoothed patients augmented, although we rarely observed changes of more than 5% of the total data. In the validation study, significant differences were obtained in the performance of the smooth algorithm over the traditional method. These differences were consistent across pharmacological scenarios. Conclusions: The smooth algorithm is an automated approach that standardizes, simplifies, and improves data processing in drug exposition studies using EHRs. This algorithm can be generalized to almost any pharmacological medication and model the drug exposure to facilitate the detection of treatment switches, discontinuations, and terminations throughout the study period. (JMIR Med Inform 2022;10(11):e37976) doi: 10.2196/37976 KEYWORDS electronic health records; data mining; complex drug patterns; algorithms; drug utilization; polypharmacy; EHR; medication; drug combination; therapy; automation; drug exposition; treatment; adherence https://medinform.jmir.org/2022/11/e37976 JMIR Med Inform 2022 | vol. 10 | iss. 11 | e37976 | p. 1 (page number not for citation purposes) XSL• FO RenderX JMIR MEDICAL INFORMATICS Ouchi et al complex treatment patterns that are challenging to analyze or Introduction interpret, can be blinded to researchers, and can be a source of The recent rise in the use of electronic health records (EHRs) misunderstanding as it is difficult to distinguish whether they has had a major impact on epidemiological research. These are real occurrences or recording errors. To address this, we databases provide a low-cost means of accessing longitudinal propose a novel algorithm called smooth to obtain the most data such as demographic, vital signs, administrative, medical likely therapy of one or more drugs over time. and pharmacy claims, clinical, and patient-centered data on large populations for epidemiologic research [1,2]. Methods However, in their current form, EHRs are complex and imperfect Data Sources data sets that can be enhanced in a dizzying number of often We used the Information System for Research in Primary Care ineffective ways. Although the challenges of working with (SIDIAP) [18] as our reference EHR platform for the algorithm EHRs in clinical trials have been identified [3-5], more research development. The SIDIAP includes information recorded by is needed to develop new and better ways to use them. health professionals during routine visits at 287 primary health From the data mining perspective, addressing these data is care centers from the Catalan Health Institute (Institut Català particularly challenging as the outcomes can be significantly de la Salut). affected depending on the quality, validity, completeness, and The platform includes information on disease diagnoses heterogeneity of the available data [6]. Besides technical (International Classification for Diseases, 10th Edition), drug perspective, researchers have their particular ways of addressing prescriptions and drug invoices in the primary care setting EHR, dealing with EHR complexity, and their decisions have (Anatomical Therapeutic Chemical [ATC] classification been shown to have a significant impact to the results [7,8]. system), and clinically relevant parameters (eg, weight, blood Approaching these challenges in a heterogeneous and biased pressure, laboratory tests) as well as sociodemographic way favors the emergence of inconsistencies between similar characteristics. It is also linked to a hospital discharge database studies [9]. for patients admitted to the Catalan Health Institute hospitals In studies with EHR-based databases, information on drug (30% of the SIDIAP population). The SIDIAP has exposure is usually obtained from electronic prescription, pseudonymized records for more than seven million people and electronic dispensation, or invoice of drugs. This information is representative of the Catalan population in terms of age, sex, is widely used and accepted in clinical research as the and geographic distribution [19]. availability of longitudinally recorded data allows for a detailed For the algorithm development and validation study, we used characterization of both the exposure to medication and the a subset of patients drawn from all Catalan Health Institute outcome of interest, and mining the data contained within EHRs primary care centers. From this population, we also obtained can potentially generate a greater understanding of medication sociodemographic characteristics: sex, age, country of origin, effects in the real world, complementing what we know from profession, socioeconomic index, smoking habits, alcohol intake, randomized control trials [10]. institutionalization in nursing homes, comorbidities, and Focusing on pharmacovigilance or pharmacoepidemiology when electronic prescriptions of pharmacological treatments. using EHRs, one of the main objectives is to characterize the In SIDIAP, electronical prescriptions and drug invoices are therapy in terms of duration [11], discontinuation [12,13], stored in longitudinal format. Each record comprises the changes [14], and adherence to pharmacological treatments pseudonymized patient identifier, ATC code, and prescription [15]. Although progress has been made in this area for or invoice date. The end of the prescription is determined by monotherapy [16], when we study treatment exposure in diseases the health professional, whereas in drug invoice records we only such as hypertension, diabetes, or chronic obstructive pulmonary have the month in which the invoice was made, and thus the disease, treatment often switches from monotherapy to end of the treatment is usually inferred based on the number of combinations of two or more drugs, which significantly packages collected. Each prescription or invoice is recorded increases the challenge of characterizing the treatment. In our independently of the health problem, and duplicate records or experience [17], polytherapy in EHR-based studies creates overlaps are common (Figure 1A). https://medinform.jmir.org/2022/11/e37976 JMIR Med Inform 2022 | vol. 10 | iss. 11 | e37976 | p. 2 (page number not for citation purposes) XSL• FO RenderX JMIR MEDICAL INFORMATICS Ouchi et al Figure 1. Overview of the smooth algorithm workflow with an illustrated example. (A) Horizontal axis is follow-up time in days, and the vertical axis is the different drugs prescribed. Box length indicates the period in which the prescription is active. (B) Patient profile after combining all active prescriptions in the same day, the first step of data process. (C) Example of complex patterns and 4 ways to overcome them. (D) Result obtained after passing the data through the smooth algorithm. the prescription or invoice records but frequently reveals Ethics Approval complex patterns that should be considered before conducting The study protocol was approved by the Research Ethics any analysis. Data are processed based on assumptions about Committee of Fundació Institut Universitari per a la recerca a those patterns by individual researchers and therefore give l'Atenció Primària de Salut Jordi Gol i Gurina (IDIAPJGol; nonhomogeneous results. In Figure 1C, we imagined 4 scenarios AR20/029, SIDIAP 386 on June 3, 2020). This is a database (but there could be more) to handle the same problem research study that has been conducted according to the (highlighted area during the first half of 2019). While some guidelines of the Declaration of Helsinki (Fortaleza, Brazil researchers may consider that the first therapy lasts until we 2013) and does not require consent from the people included observe a change in the treatment, others with different to participate or for publication. The need for consent was backgrounds may decide that a change from double to triple waived by the Research Ethics Committee of IDIAPJGol as it therapy can only be considered if the triple therapy lasts longer is deemed unnecessary according to European legislation than an arbitrary period (eg, 60 days). (Regulation [EU] 2016/679). The smooth algorithm is inspired by statistical methods using Finding the Most Likely Therapy Using the Smooth simple moving averages that calculate trends or smooth time Algorithm series [20]. For EHRs, we changed the concept of moving The process has 2 parts: data mining to look for treatments averages to a moving window from where we choose the most recorded daily and applying the smooth algorithm to the data frequent treatment. Thus, by moving the window one day at a (Figure 1). time, we identify the most frequent pattern over the study period (Figure 1D). For the first part, we looked at all drugs of interest that are active on the same day (Figure 1A and Figure 1B). This step simplifies In Figure 2, we can see a detailed description of the algorithm. It is an iterative process that works as follows: https://medinform.jmir.org/2022/11/e37976 JMIR Med Inform 2022 | vol. 10 | iss. 11 | e37976 | p. 3 (page number not for citation purposes) XSL• FO RenderX JMIR MEDICAL INFORMATICS Ouchi et al • Starting from the first prescription or dispensation at day • We then shifted the window forward one day at a time, ti, we opened a window of specific length in days (Wt) in repeating the process until the end of follow-up (Figure which we search for the most frequent treatment 2A). • That treatment was assigned to the whole window unless • After the first iteration, we had up to Wt possible treatments we had a draw (2 treatments are active for the same number (or candidates) for each day of days), in which case we carried the previous treatment • Finally, we chose the candidate most frequently observed (ti-1) forward. on that day (Figure 2B). Figure 2. Smooth algorithm in detail. In a more in-depth analysis, a simulation study was conducted Window Size using 7132 patients who were under long-term administration The length of Wt is the only parameter that needs to be defined of aspirin, statins, beta-blockers, and angiotensin-converting beforehand, and its value can modify the outcome (Figure 3). enzyme (ACE) inhibitors or angiotensin-receptor blockers The length of Wt determines the level of smoothing, and the between 2018 and 2020. The smooth algorithm was run on this value can go from 1 day to the total days of follow-up. Thus, data using 6 Wt values: 10, 20, 30, 45, 60, and 90 days. For each for Wt of 1 day, we are not changing the data while for a Wt Wt, we counted the number of patients with at least one change equal to the number of days of follow-up, we expect to reduce in the treatment pattern; out of these, we calculated the all records to the most frequent treatment. Therefore, small percentage of smooth as the ratio of the number of days changed values of the Wt parameter will not significantly modify the by the algorithm divided by the total number of days with active raw data, whereas increasing the size of the window is expected treatment. to have a larger impact on the data. https://medinform.jmir.org/2022/11/e37976 JMIR Med Inform 2022 | vol. 10 | iss. 11 | e37976 | p. 4 (page number not for citation purposes) XSL• FO RenderX JMIR MEDICAL INFORMATICS Ouchi et al Figure 3. Example of how the outcome changes according to the value of the Wt parameter. For the smooth algorithm, the Wt parameter was set to 60 days Validation Study in all 4 scenarios. To understand how the algorithm performs from a clinical or pharmacological perspective, we conducted a validation study. Before conducting the validation, we prepared a training session We identified 4 pharmacological scenarios where the algorithm with the 4 reviewers consisting in an introduction to the data, could be needed—combination of 3 or more drugs (platelet drugs of study (including the selected ATC codes), explanation aggregation inhibitors, beta-blockers, and ACE inhibitors), of the algorithms, and discussion of the common criteria to treatments likely to discontinue (antidepressants), long-term apply during the validation. From their feedback, and after the combination of 2 drugs (insulins and oral antidiabetics), and training, we decided to include all health problems related to short-term treatment (systemic antibiotics)—and asked to 4 the treatment as it may facilitate the evaluation and give more independent professionals with experience in databases and importance to clinical criteria (see validated sample in drug exposure studies to compare a traditional method with the Multimedia Appendix 1, Figure S1). smooth algorithm. Our primary objective was changes to the original data; we The traditional method is a more intuitive and simple approach, analyzed whether the algorithms improved, worsened, or made commonly observed in the literature, to address noise and no changes to the original data. In addition, we asked the variability in electronic drug records [21,22]. Briefly, it starts reviewers to choose between the traditional method and the with the first treatment observed and accepts a treatment change smooth algorithm and evaluate its value for detecting treatment only if the new one is longer than a certain period of time. This switches and/or discontinuations. period is generally arbitrary, an assumption done by the Each professional reviewed 100 patient records with one-quarter researcher based on the characteristics of the drug. For our of the records being assigned to all reviewers to analyze validation study, we set the period to 60 days except for consistency across validations. To reduce potential biases, antibiotics (the short-term treatment), with a period of 15 days. reviewers were blinded and they did not know which method or algorithm generated the results (Table 1). Table 1. Description of drugs and distribution of samples in the validation study. Treatment pattern of use Description (ATCa code) Prescriptions Samples analyzed, Samples repeated across re- in the data set total (per reviewer) viewers, total (per reviewer) Short-term drugs Systemic antibiotics (J01) 10,846,282 80 (20) 40 (10) Likely to discontinue Antidepressants (N06a) 3,859,496 80 (20) 40 (10) Long-term combinations of 2 drugs Insulins and oral antidiabetics (A10) 22,271,154 120 (30) 60 (15) Combination of 3 or more drugs Platelet aggregation inhibitors (B01Ac) 21,253,742 120 (30) 60 (15) a ATC: Anatomical Therapeutic Chemical. https://medinform.jmir.org/2022/11/e37976 JMIR Med Inform 2022 | vol. 10 | iss. 11 | e37976 | p. 5 (page number not for citation purposes) XSL• FO RenderX JMIR MEDICAL INFORMATICS Ouchi et al Statistical Methods monotherapy to a combination of 2 drugs (A+C and A+B) prior We determined that a sample size of 400 patients would be to switching to the A monotherapy. In contrast, with a Wt of 90 enough to ensure 80% power assuming a minimum effect size days (Figure 3C), the entire treatment pattern was simplified. of 0.3062 with two degrees of freedom for a Chi-square test During the first year of follow-up, we observed a monotherapy; under a significance level of 5%. during the most complex pattern, the algorithm smoothed the changes to a single combination of A+C before returning to a Categorical variables were described with relative and absolute monotherapy. frequencies, and results from numerical variables were reported using means and standard deviations. For the validation study, Results of the simulation study are represented in Figure 4. With we used the Chi-square test to evaluate differences between the a Wt of 10 days, 11.4% (814/7132) had their treatment pattern traditional method and smooth algorithm on performance changed, while with a 90-day window, 39.6% (2822/7132) had compared with the raw data. The algorithm was programmed their treatment pattern modified; 31.5% (2244/7132), 33.8% in R (version 4.1.0, R Foundation for Statistical Computing), (2413/7132), and 39.6% (2822/7132) of patients with Wt values and all analyses were performed in R. of 45, 60, and 90 days, respectively, saw at least 1 change. Thus, the effect of Wt was not linear as the expected progression of Results the number of patients being smoothed was different than the results from the simulation. Impact of Wt Parameter and Simulation Study As a relative measure, we reported the percentage of days In Figure 3, we show how the results changed according to the changed, ranging from 0.27 (IQR 0.09, 0.36) to 2.28 (IQR 1.09, Wt value. In the example, for the raw data we note that during 3.65). Thus, for each of the Wt values 10, 20, 30, 45, 60, and the first period of follow-up, the main treatment was a 90 days in a 1000-day period of follow-up, the algorithm monotherapy followed by a complex pattern during the first modified 2.7, 4.6, 6.4, 10, 14.6, and 22.8 days, respectively. At 6-month semester of 2019 (Figure 3A). To reduce noise in the windows from 10 to 30 days, the percentage of days changed pattern, we applied the smooth algorithm using Wt values of 30 always remain below the 5%, but as we increased the Wt to 45, and 90 days. With a Wt of 30 days (Figure 3B), we retained the 60, and 90 days, we started to observe patients with more than combination of 2 drugs during the early stages of follow-up; 5% of the data smoothed. during months with more changes, patient moved from Figure 4. Statistics of the simulation study using 6 Wt values on 7132 patients under treatment for cardiovascular disease between 2018 and 2020. *100 x Number of days changed / Total days under prescription. https://medinform.jmir.org/2022/11/e37976 JMIR Med Inform 2022 | vol. 10 | iss. 11 | e37976 | p. 6 (page number not for citation purposes) XSL• FO RenderX JMIR MEDICAL INFORMATICS Ouchi et al Validation Study observed changes in up to 40% of the patients, with 75% of Multimedia Appendix 2 includes the results of the validation those having at least 1% of the records smoothed. study. For the 400 samples, the smooth algorithm improved the Interestingly, the simulation shows that at a certain Wt value, raw data for 56.8% (227/400) of individuals, while 42.5% the number of individuals modified reaches a plateau. The data (170/400) benefited from using the traditional method. In 39% changed by the algorithm are less than expected, particularly (156/400) of the samples, the outcome provided by each when Wt is greater than 30 days, suggesting that independently algorithm did not change the patterns, and 4.2% (17/400) of of the parameter, some patients will never be changed by the cases reported worsening after being processed by the smooth algorithm. algorithm. The traditional method resulted in a worse outcome for 18.5% (74/400) of the samples, and the observed differences In the validation study, we observed that most times our between algorithms were statistically significative (P<.001). algorithm improved the data patterns. It was designed to improve polypharmacy exposure assessment, and we were interested in Significant differences were also observed between algorithms the results for combinations of 3 or more drugs. In this scenario, stratified by scenario. With a combination of 3 or more drugs both the traditional and smooth approaches demonstrated (platelet aggregation inhibitors, beta-blocker, and ACE usefulness, and the percentages of improved samples were inhibitors), drugs likely to be discontinued (antidepressants), similar, although the smooth algorithm performed significantly and the combination of 2 or more long-term drugs (insulins and better. These differences were also observed in the other antidiabetics), the smooth algorithm improved 69.2% (83/120), scenarios, and we believe that the smooth algorithm not only 61.3% (49/80), and 60.0% (72/120) of samples, respectively. improves the treatment pattern but also does not make it worse. In short-term treatment (systemic antibiotics), 90.0% (72/80) In addition, the performance was not affected by the window of samples did not show changes and 28.7% (23/80) were length, since for antibiotics and antidepressants (short- and improved by the smooth algorithm. long-term drug use, respectively) the smooth algorithm As for the samples validated by the 4 professionals (Table 1), performed well using the same Wt window. they decided in 88.0% (44/50) of patients to choose the smooth The traditional method proved to be a good approach, and algorithm over the traditional method (Multimedia Appendix similar versions are being used in other studies [21,22,29], but 2 and Multimedia Appendix 1, Figure S2). It was in the it differs according to drug or study characteristics and so is short-term treatment scenario where we observed less of a less generalizable. In fact, it has not worked well for systemic consensus, 70% (7/10), whereas for the rest of scenarios the 4 antibiotics even though we specifically changed it to fit for its reviewers agreed in 93.3% (14/15), 100% (10/10), and 86.7% characteristics. Overall, the validations for the smooth algorithm (13/15) of the samples, respectively. were consistent between scenarios and reviewers. The smooth algorithm performed better than the traditional Potential Uses and Strengths method in detecting discontinuations (350/366, 95.6%) and We believe that the smooth algorithm has significant potential treatment switches (138/230, 60.0%; see Multimedia Appendix to assess exposure for treatment combinations, especially in 1, Table S1). chronic treatments, since it allows us to have a time sequence of exposure to the treatment. This sequence allows us to better Discussion model the drug exposition and detect discontinuations, switches, Principal Findings and periods of interaction with other drugs of short duration. In addition, it can be of great help in estimating adherence to a Since the use of EHR databases began in combination treatment [30]. With the smooth algorithm, we can pharmacoepidemiologic studies, researchers have been trying easily calculate the exposure time using only electronic to establish algorithms to model drug exposure [23]. This prescriptions without the need to know the dosage posology. becomes even more challenging when trying to assess drug exposures with multiple pharmacologic treatments, which From a clinical point of view, the smooth algorithm has great happens quite often in older people, as they are prescribed with advantage when estimating polypharmacy adherence [31]. up to 5 drugs simultaneously [24]. Thus, we have developed an Patients affected by chronic conditions in need of polypharmacy automatic algorithm to model drug exposure through EHRs, may have differing levels of adherence to individual medications which standardizes the data mining process to obtain more within their regimen, and this could lead to varying health consistent and replicable results across studies. outcomes and misleading results if the methodological approach assumes as equivalent adherence to all medications. These The algorithm is inspired by time series forecasting methods patients also face other acute conditions requiring the addition and requires a parameter to be set beforehand. This is commonly of drugs or modification of doses while maintaining their actual observed in similar statistical methods such as autoregressive medication regimen. models or moving averages [25], and it is known that the value of the parameter can modify the outcome significantly [26-28]. Another research area in which our algorithm could be of utility The simulation study shows the impact of the Wt value. Small is the study of adverse events due to lack of effectivity values hardly change the original data, but as the parameter (antibiotics or hypertension treatments) and drug-drug value increases, the raw data can be affected to the point of interactions (anticoagulants and nonsteroidal anti-inflammatories losing clinical relevance. In the worst-case scenario, we combined for short time periods). https://medinform.jmir.org/2022/11/e37976 JMIR Med Inform 2022 | vol. 10 | iss. 11 | e37976 | p. 7 (page number not for citation purposes) XSL• FO RenderX JMIR MEDICAL INFORMATICS Ouchi et al Although we cannot ever know a patient’s true adherence, the is important to understand the effect on the outcome to use a smooth algorithm is an automated way to analyze EHR data good value. This is an inherent limitation of the algorithm and, offering methodological consistency across studies. In several as a guide, we recommend setting the Wt value within the range studies, assumptions were made prior to the data processing, of 30 to 60 days to reduce complex patterns without and this may have impacted the results of the analyses by compromising relevant information. Moreover, the Wt can be introducing bias on the final results [32]. Algorithms like smooth changed so the algorithm can be used in short (antibiotics) and can help standardize these assumptions and minimize long-term (antidepressants) treatments as well as in drug inconsistencies between studies with similar databases. combinations for chronic conditions such as diabetes and hypertension. Limitations Due to the nature of the algorithm eliminating complex patterns, Another limitation of the study is that the validation was done we may lose relevant exposures of a short period of time, and with the traditional method instead of other algorithms as a smooth is not recommended for all scenarios. Similarly, smooth comparator. Moreover, in our experience, we commonly see may not work well in long-acting drugs. the traditional method being used with some differences or criteria according to the framework or objectives of the study. We were not able to capture the posology with the SIDIAP For example, in projects with the European Medicines Agency, database, so we could not estimate the length of treatment we have never seen a method or an automatic approach to deal through the dose prescribed. The treatment doses can change with drug exposure other than the one we call traditional [33]. throughout the year (decreased use of diuretics during summertime, when traveling, etc). Conclusion The smooth algorithm is an automated approach to estimate the From a technical perspective, using the smooth algorithm is a most likely drug exposure pattern. We proved that it time-consuming process. To run the algorithm on big data sets standardizes, simplifies, and improves the data processing steps like EHRs, good information technology is needed. The time before performing the study analysis; can model the drug needed to complete the process may vary depending on the exposure to detect cotreatment, switches, discontinuations, and number of patients and follow-up time. treatment terminations; and facilitates adherence calculations In addition, before running the algorithm, we must set a throughout the study period. In future pharmacoepidemiological parameter, Wt, that allows us to choose between precision and studies, we aim to further validate the algorithm and analyze simplicity. Setting this parameter is not straightforward, and it the impact the algorithm can have on the main results. Authors' Contributions DO designed and developed the algorithm and performed the statistical analyses. MG and DO collected the data. All authors contributed to the validation study. DO, AG, MG, FT, and RM were involved in the interpretation of the data. DO wrote the manuscript and designed the tables and figures. All authors critically revised the report and approved the final version to be submitted for publication. The corresponding author confirms that he had full access to all the data in the study and had final responsibility for the decision to submit for publication. Conflicts of Interest None declared. Multimedia Appendix 1 Supplementary material. [DOCX File , 1022 KB-Multimedia Appendix 1] Multimedia Appendix 2 Results of the validation study. [DOCX File , 15 KB-Multimedia Appendix 2] References 1. Casey JA, Schwartz BS, Stewart WF, Adler NE. Using electronic health records for population health research: a review of methods and applications. Annu Rev Public Health 2016 Mar;37:61-81. [doi: 10.1146/annurev-publhealth-032315-021353] [Medline: 26667605] 2. Rogers JR, Lee J, Zhou Z, Cheung YK, Hripcsak G, Weng C. Contemporary use of real-world data for clinical trial conduct in the United States: a scoping review. J Am Med Inform Assoc 2021 Jan 15;28(1):144-154. [doi: 10.1093/jamia/ocaa224] [Medline: 33164065] 3. Cowie MR, Blomster JI, Curtis LH, Duclaux S, Ford I, Fritz F, et al. Electronic health records to facilitate clinical research. Clin Res Cardiol 2017 Jan;106(1):1-9 [FREE Full text] [doi: 10.1007/s00392-016-1025-6] [Medline: 27557678] https://medinform.jmir.org/2022/11/e37976 JMIR Med Inform 2022 | vol. 10 | iss. 11 | e37976 | p. 8 (page number not for citation purposes) XSL• FO RenderX JMIR MEDICAL INFORMATICS Ouchi et al 4. Lu Z. Information technology in pharmacovigilance: benefits, challenges, and future directions from industry perspectives. Drug Healthc Patient Saf 2009 Oct;1:35-45 [FREE Full text] [doi: 10.2147/dhps.s7180] [Medline: 21701609] 5. Menachemi N, Collum TH. Benefits and drawbacks of electronic health record systems. Risk Manag Healthc Policy 2011 May;4:47-55 [FREE Full text] [doi: 10.2147/RMHP.S12985] [Medline: 22312227] 6. Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. J Am Med Inform Assoc 2013 Jan 01;20(1):144-151 [FREE Full text] [doi: 10.1136/amiajnl-2011-000681] [Medline: 22733976] 7. Salas M, Lopes LC, Godman B, Truter I, Hartzema AG, Wettermark B, et al. Challenges facing drug utilization research in the Latin American region. Pharmacoepidemiol Drug Saf 2020 Nov 17;29(11):1353-1363. [doi: 10.1002/pds.4989] [Medline: 32419226] 8. Rudin RS, Friedberg MW, Shekelle P, Shah N, Bates DW. Getting value from electronic health records: research needed to improve practice. Ann Intern Med 2020 Jun 02;172(11_Supplement):S130-S136. [doi: 10.7326/m19-0878] 9. Wang SV, Schneeweiss S, Berger ML, Brown J, de Vries F, Douglas I, Joint ISPE-ISPOR Special Task Force on Real World Evidence in Health Care Decision Making. Reporting to improve reproducibility and facilitate validity assessment for healthcare database studies V1.0. Pharmacoepidemiol Drug Saf 2017 Sep 15;26(9):1018-1032 [FREE Full text] [doi: 10.1002/pds.4295] [Medline: 28913963] 10. Yao L, Zhang Y, Li Y, Sanseau P, Agarwal P. Electronic health records: implications for drug discovery. Drug Discov Today 2011 Jul;16(13-14):594-599. [doi: 10.1016/j.drudis.2011.05.009] [Medline: 21624499] 11. Støvring H, Pottegård A, Hallas J. Determining prescription durations based on the parametric waiting time distribution. Pharmacoepidemiol Drug Saf 2016 Dec 26;25(12):1451-1459. [doi: 10.1002/pds.4114] [Medline: 27670969] 12. Lyu H, Zhao S, Yoshida K, Tedeschi S, Xu C, Nigwekar S, et al. Delayed denosumab injections and bone mineral density response: an electronic health record-based study. J Clin Endocrinol Metab 2020 May 01;105(5):1 [FREE Full text] [doi: 10.1210/clinem/dgz321] [Medline: 31894244] 13. Mascarenhas J, Mehra M, He J, Potluri R, Loefgren C. Patient characteristics and outcomes after ruxolitinib discontinuation in patients with myelofibrosis. J Med Econ 2020 Jul 31;23(7):721-727 [FREE Full text] [doi: 10.1080/13696998.2020.1741381] [Medline: 32159402] 14. Sharman J, Kabadi SM, Clark J, Andorsky D. Treatment patterns and outcomes among mantle cell lymphoma patients treated with ibrutinib in the United States: a retrospective electronic medical record database and chart review study. Br J Haematol 2021 Feb 23;192(4):737-746. [doi: 10.1111/bjh.16922] [Medline: 33095453] 15. Li X, Cole SR, Westreich D, Brookhart MA. Primary non-adherence and the new-user design. Pharmacoepidemiol Drug Saf 2018 Apr 19;27(4):361-364 [FREE Full text] [doi: 10.1002/pds.4403] [Medline: 29460385] 16. Suchard MA, Schuemie MJ, Krumholz HM, You SC, Chen R, Pratt N, et al. Comprehensive comparative effectiveness and safety of first-line antihypertensive drug classes: a systematic, multinational, large-scale analysis. Lancet 2019 Nov;394(10211):1816-1826. [doi: 10.1016/s0140-6736(19)32317-7] 17. Giner-Soriano M, Sotorra Figuerola G, Cortés J, Pera Pujadas H, Garcia-Sangenis A, Morros R. Impact of medication adherence on mortality and cardiovascular morbidity: protocol for a population-based cohort study. JMIR Res Protoc 2018 Mar 09;7(3):e73 [FREE Full text] [doi: 10.2196/resprot.8121] [Medline: 29523501] 18. Bolíbar B, Fina AF, Morros R, Garcia-Gil MDM, Hermosilla E, Ramos R, et al. [SIDIAP database: electronic clinical records in primary care as a source of information for epidemiologic research]. Med Clin (Barc) 2012 May 19;138(14):617-621. [doi: 10.1016/j.medcli.2012.01.020] [Medline: 22444996] 19. García-Gil M, Hermosilla E, Prieto-Alhambra D, Fina F, Rosell M, Ramos R, et al. Construction and validation of a scoring system for the selection of high-quality data in a Spanish population primary care database (SIDIAP). Inform Prim Care 2011 Jun 01;19(3):135-145 [FREE Full text] [doi: 10.14236/jhi.v19i3.806] [Medline: 22688222] 20. Watson GS. Smooth regression analysis. Sankhyā Indian J Stat 1964;Series A(26):359-372. 21. Xu H, Aldrich M, Chen Q, Liu H, Peterson N, Dai Q, et al. Validating drug repurposing signals using electronic health records: a case study of metformin associated with reduced cancer mortality. J Am Med Inform Assoc 2015 Jan;22(1):179-191 [FREE Full text] [doi: 10.1136/amiajnl-2014-002649] [Medline: 25053577] 22. Hughey J, Colby J. Discovering cross-reactivity in urine drug screening immunoassays through large-scale analysis of electronic health records. Clin Chem 2019 Dec;65(12):1522-1531 [FREE Full text] [doi: 10.1373/clinchem.2019.305409] [Medline: 31578215] 23. Andrade SE, Kahler KH, Frech F, Chan KA. Methods for evaluation of medication adherence and persistence using automated databases. Pharmacoepidemiol Drug Saf 2006 Aug;15(8):565-574. [doi: 10.1002/pds.1230] [Medline: 16514590] 24. Pazan F, Wehling M. Polypharmacy in older adults: a narrative review of definitions, epidemiology and consequences. Eur Geriatr Med 2021 Jun 10;12(3):443-452 [FREE Full text] [doi: 10.1007/s41999-021-00479-3] [Medline: 33694123] 25. Hannan EJ, Rissanen J. Recursive estimation of mixed autoregressive-moving average order. Biometrika 1983;70(1):303. [doi: 10.1093/biomet/70.1.303-b] 26. Durbin J. Efficient estimation of parameters in moving-average models. Biometrika 1959 Dec;46(3/4):306. [doi: 10.2307/2333528] 27. Sandgren N, Stoica P, Babu P. On moving average parameter estimation. Eur Signal Process Conf 2012:2348-2351. https://medinform.jmir.org/2022/11/e37976 JMIR Med Inform 2022 | vol. 10 | iss. 11 | e37976 | p. 9 (page number not for citation purposes) XSL• FO RenderX JMIR MEDICAL INFORMATICS Ouchi et al 28. Żebrowska M, Dzwiniel P, Waleszczyk WJ. Removal of the sinusoidal transorbital alternating current stimulation artifact from simultaneous EEG recordings: effects of simple moving average parameters. Front Neurosci 2020 Jul 29;14:735 [FREE Full text] [doi: 10.3389/fnins.2020.00735] [Medline: 32848538] 29. van Staa T, Abenhaim L, Leufkens H. A study of the effects of exposure misclassification due to the time-window design in pharmacoepidemiologic studies. J Clin Epidemiol 1994 Feb;47(2):183-189. [doi: 10.1016/0895-4356(94)90023-x] 30. Vlacho B, Mata-Cases M, Mundet-Tudurí X, Vallès-Callol J, Real J, Farre M, et al. Analysis of the adherence and safety of second oral glucose-lowering therapy in routine practice from the mediterranean area: a retrospective cohort study. Front Endocrinol (Lausanne) 2021 Jul 14;12:708372 [FREE Full text] [doi: 10.3389/fendo.2021.708372] [Medline: 34335477] 31. Franklin JM, Gopalakrishnan C, Krumme AA, Singh K, Rogers JR, Kimura J, et al. The relative benefits of claims and electronic health record data for predicting medication adherence trajectory. Am Heart J 2018 Mar;197:153-162. [doi: 10.1016/j.ahj.2017.09.019] [Medline: 29447776] 32. Pye SR, Sheppard T, Joseph RM, Lunt M, Girard N, Haas JS, et al. Assumptions made when preparing drug exposure data for analysis have an impact on results: an unreported step in pharmacoepidemiology studies. Pharmacoepidemiol Drug Saf 2018 Jul 17;27(7):781-788 [FREE Full text] [doi: 10.1002/pds.4440] [Medline: 29667263] 33. Cid-Ruzafa J, Lacy BE, Schultze A, Duong M, Lu Y, Raluy-Callado M, et al. Linaclotide utilization and potential for off-label use and misuse in three European countries. Therap Adv Gastroenterol 2022 Jun 11;15:17562848221100946 [FREE Full text] [doi: 10.1177/17562848221100946] [Medline: 35706826] Abbreviations ACE: angiotensin-converting enzyme ATC: Anatomical Therapeutic Chemical EHR: electronic health record IDIAPJGol: Fundació Institut Universitari per a la recerca a l'Atenció Primària de Salut Jordi Gol i Gurina SIDIAP: Information System for Research in Primary Care Edited by C Lovis; submitted 14.03.22; peer-reviewed by E Urtnasan, G Yu; comments to author 19.08.22; revised version received 19.09.22; accepted 13.10.22; published 15.11.22 Please cite as: Ouchi D, Giner-Soriano M, Gómez-Lumbreras A, Vedia Urgell C, Torres F, Morros R Automatic Estimation of the Most Likely Drug Combination in Electronic Health Records Using the Smooth Algorithm: Development and Validation Study JMIR Med Inform 2022;10(11):e37976 URL: https://medinform.jmir.org/2022/11/e37976 doi: 10.2196/37976 PMID: ©Dan Ouchi, Maria Giner-Soriano, Ainhoa Gómez-Lumbreras, Cristina Vedia Urgell, Ferran Torres, Rosa Morros. Originally published in JMIR Medical Informatics (https://medinform.jmir.org), 15.11.2022. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on https://medinform.jmir.org/, as well as this copyright and license information must be included. https://medinform.jmir.org/2022/11/e37976 JMIR Med Inform 2022 | vol. 10 | iss. 11 | e37976 | p. 10 (page number not for citation purposes) XSL• FO RenderX