Powered by EHA Perspective Challenges and Solutions for Collecting and Analyzing Real World Data: The Eric CLL Database as an Illustrative Example Anastasia Chatzidimitriou1,2, Eva Minga1, Thomas Chatzikonstantinou1,3, Carol Moreno4, Kostas Stamatopoulos1,2, Paolo Ghia5, on behalf of ERIC, the European Research Initiative on CLL Correspondence: Anastasia Chatzidimitriou (e-mail: achatzidimitriou@certh.gr). ERIC, the European research on CLL Challenges of gathering high-quality Chronic lymphocytic leukemia (CLL) is an age-related real-world data malignancy of mature B lymphocytes.1 While the diagnosis of Collection and analysis of real world data (RWD) can prove CLL is relatively straightforward, the clinical course and outcome both effective and efficient for advancing precision medicine and are highly heterogeneous.2 Moreover, despite remarkable improving the quality and delivery of medical care, provided therapeutic advances achieved in recent years, the disease is these come along with data quality.6,7 The amount of biomedical mostly incurable. data continuously increases due to technological advances, thus ERIC, the European Research Initiative on CLL (http://www. raising the necessity for designing and developing standardized ericll.org) is a Scientific Working Group (SWG) of the European approaches and methodologies to be implemented in clinical Hematology Association (EHA) aimed at improved management practice.8 of CLL through collaborative research. Thanks to the active Data acquisition is usually a process distributed among participation of its members, now exceeding 1300 from all over different health professionals potentially leading to data quality Europe and beyond, ERIC engages in projects extending from problems across datasets, such as data redundancy (ie, repeated basic to (mainly) translational and clinical research. information), heterogeneity (eg, different date format) and Capitalizing on such initiatives but also on our expertise in the inconsistency (eg, a date of diagnosis after the date of treatment), collection, management and analysis of heterogeneous clinical mainly resulting from lack of standardization and data curation and biological data,3–5 we have developed and present here the processes. Such problems are particularly pertinent in the case of ERIC CLL database, a registry of clinical and biological data of multi-institutional efforts, where multilevel and multi-originated patients with CLL. data are collected. Furthermore, the rapid increase of data complexity captured during patient care, especially data produced by the application of novel methodologies (eg, next generation sequencing), poses challenges that cannot be addressed with standard computational approaches. 1 Institute of Applied Biosciences, Centre for Research and Technology Hellas, Thus, there is an imperative to improve real-world evidence Thessaloniki, Greece generation by optimizing the integration of the heterogeneous 2 Department of Molecular Medicine and Surgery, Karolinska Institutet, Stockholm, information through automated and thorough quality control Sweden and curation mechanisms; and, support analysis and compatibil- 3 Hematology Departments and Hematopoietic Cells Transplantation Unit, G. ity with established ontologies. This will provide unified and Papanikolaou Hospital, Thessaloniki, Greece 4 Hospital de la Santa Creu i Sant Pau, Autonomous University of Barcelona, standardized access to valid, accurate and comparable datasets. Barcelona, Spain Practical and feasible tools are required, capable of providing 5 Division of Experimental Oncology, Università Vita-Salute San Raffaele and easiness in use, flexibility and simplicity, in order to facilitate the IRCCS Ospedale San Raffaele, Milan, Italy. data entry procedure and encourage the registration and The authors have indicated they have no potential conflicts of interest to disclose. organization of clinically relevant data from the daily practice.9 The development of ERICdb is funded partially by an unrestricted grant from AbbVie and the European Initiative on CLL (ERIC). Copyright © 2020 the Author(s). Published by Wolters Kluwer Health, Inc. on Towards the development of a unified data behalf of the European Hematology Association. This is an open access article management framework distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible Harmonization of heterogeneous data is a prerequisite for to download and share the work provided it is properly cited. The work cannot gathering homogenized high-quality datasets and bridging the be changed in any way or used commercially without permission from the many forms of biological and medical information. journal. HemaSphere (2020) 4:5(e425). http://dx.doi.org/10.1097/HS9.0000000000000425. A common approach that can be adapted to local and project- Received: 27 April 2020 / Accepted: 26 May 2020 specific requirements, will inevitably facilitate biological, trans- 1 Chatzidimitriou et al Challenges and Solutions for Collecting and Analyzing Real World Data lational and clinical research, enabling multi-center projects on, is designed in expandable modules that allow the rapid for example, clinical association studies and translational introduction of additional categories and values when and if medicine at large.10 needed, based on the type of project run at any single moment in time, making it flexible and adjustable. All new information eventually ends up in the central dataset as stable asset of the Standardization database. Currently, the ERIC CLL database includes data from Agreement on common policies along with a user-friendly 9147 cases coming from 19 centres in 10 different countries. integration of ontologies, terminologies and standards, paves the The main objectives of the ERIC CLL database are to (1) collect way for standardized registration of RWD; thus, the collected and transform clinically relevant RWD into evidence and data can be seamlessly combined into a data integration correlate with biological data in order to provide accurate framework, achieving semantic interoperability. information about the state-of-the-art diagnosis; (2) generate hypotheses regarding important disease characteristics, labora- tory studies and therapies; and, (3) define relevant parameters Retrospective data integration influencing CLL impact on health systems. Development and use of reliable “Extract Transform Load” The data categories composing all the relevant information to (ETL) software enables and facilitates the massive import of form an accurate and complete representation of the disease datasets from external, diverse sources into a centralized course include basic demographic data, disease-related informa- repository.11 tion, treatment options and response, laboratory results and relevant outcome data. The data model has been designed in order to meet the requirements for an accurate description of Quality assessment diagnosis, prognostic assessment and management of patients The development of a semantic framework with data cleaning with CLL. processes12 capable of identifying quality problems in the A relational database developed in PostgreSQL has been collected data, is a prerequisite in order to minimize barriers designed in a way to provide data integrity and promote data to data sharing, availability and reusability for research purposes. correlations and statistical analysis. To the benefit of the scientific A standard-based cleaning, organization and integration community at large, open-source tools have been used for the approach can provide relevance and data accuracy while development of the ERIC CLL database. A standards-based guaranteeing long-term usability of the collected high-quality approach is used to determine efficacy of data registration and data.13 integration, providing useful, accurate and valid information, The definition of rules for syntactical and semantic errors, out increasing the availability of clinically relevant structured data of range values, missing data, unique value and functional and fulfilling the data quality assurance requirements. dependency violations, is a well-recognized objective of any A web-based user interface has been developed for prospective strategy aiming at improving quality control. data collection as part of patient’s routine care, designed to ensure data protection, security and availability. The interface allows for controlled database login, real-time registration with Building an integrated research infrastructure data validation mechanisms, data retrieval and management. Organizing clinical and translational RWD in a standardized Moreover, a retrospective data registration and import tool has and centralized data repository allows unified access for research been developed in order to efficiently and effectively load into the purposes, improving research efficiency and quality on multiple database retrospective patient data, collected in purpose-specific levels. That said, such integrated approaches must ensure data template registration spreadsheets. The tool deploys data security and privacy, in particular when aiming for multi- cleaning processes based on certain rules ensuring content institutional, transnational efforts. In this context, the preferred validation and detecting data inconsistency and redundancy systems for data collection and retrieval are web-based, with errors. A mapping mechanim is then applied using transforma- remote data entry, consisting of custom project-related electronic tion rules to convert data to predefined types and import them case report forms with simple, user-friendly interfaces to facilitate into the database in the appropriate form. Accordingly, the tool efficient data registration. can be configured and applied to transform exports of data Moreover, data collection, management and sharing must be coming from different sources, regardless the diversity of the conducted with standardized procedures ensuring compliance software currently utilized in each single institution (ie, different and adherence to ethical, regulatory and legal standards and databases), thus enabling interoperability. preventing unauthorized access and unintended disclosure. Personal identifying data are not requested or stored in the Personal data protection and conforming to EU regulations, at ERIC CLL database. Concerning the collection of retrospective least for member states of the European Union (EU),14 and the data, anonymization of data takes place during the registration principles of beneficence, non-maleficence, respect for autonomy and validation processes; anonymized datasets are then saved and and confidentiality must be guaranteed through the application imported into the ERIC CLL database, conforming to EU of anonymization methods in the captured data15 and the regulations. Moreover, a user management system has been development of access control and activity monitoring mecha- developed and configured for authentication and authorization nisms. of users to ensure data confidentiality and privacy, providing efficient and secure handling and exchange of information. Center-based, lab-based and role-based privileges are defined to The ERIC CLL database restrain access, control, monitor and facilitate data management The ERIC CLL database is a data management system that procedures, according to general and local requirements. Data is supports research and medical knowledge discovery in CLL stored in a secure dedicated database server, controlled by an implementing the aforementioned methodologies. The database ISMS which is ISO27001-2013 certified and abides to GDPR. 2 (2020) 4:5 www.hemaspherejournal.com The infrastructure includes system failover mechanisms, backup 5. Herishanu Y, Shaulov A, Fineman R, et al. Frontline treatment with the combination obinutuzumab +/- chlorambucil for chronic lymphocytic processes to prevent data loss and history management leukemia outside clinical trials: Results of a multinational, multicenter mechanisms providing information about data modifications. study by ERIC and the Israeli CLL study group. Am J Hematol. 2020;95:604–611. 6. US Food and Drug Administration. Framework for FDA’s real-world Concluding remarks evidence program. 2018. https://www.fda.gov/media/120060/down load. Accessed September 19, 2020. The uniqueness of CLL in terms of clinico-biological 7. Personalized Medicine Coalition. Personalized Medicine at FDA: The heterogeneity and rapidly evolving therapeutic paradigms Scope & Significance of Progress in 2019. http://www.personalized underlines the need for large-scale collaboration and multi- medicinecoalition.org/Userfiles/PMC-Corporate/file/PM_at_FDA_The_ disciplinarity aimed towards the realization of precision Scope_and_Significance_of_Progress_in_2019.pdf. Accessed September medicine in CLL. This essentially requires refined understand- 22, 2020. 8. Khozin S, Blumenthal GM, Pazdur R. Real-world data for clinical ing of CLL at the fundamental, pathophysiological level as well evidence generation in oncology. J Natl Cancer Inst. 2017;109: as integration of multiple layers and sources of biological data 10.1093/jnci/djx187. with information about disease trajectories and outcomes. The 9. Meystre SM, Lovis C, Burkle T, et al. Clinical data reuse or secondary ERIC CLL database is a concrete step in this direction, tailored use: current status and potential future progress. Yearb Med Inform. 2017;26:38–52. to user needs and aspiring to contribute to improved 10. Wilkinson MD, Dumontier M, Aalbersberg IJ, et al. The FAIR Guiding management of patients with CLL. Principles for scientific data management and stewardship. Sci Data. 2016;3:160018. References 11. Meyer J, Ostrzinski S, Fredrich D, et al. Efficient data management in a large-scale epidemiology research project. Comput Methods Programs 1. Kipps TJ, Stevenson FK, Wu CJ, et al. Chronic lymphocytic leukaemia. Biomed. 2012;107:425–435. Nat Rev Dis Primers. 2017;3:17008. 12. Rahm EDH. Data cleaning: Problems and current approaches. IEEE 2. Hallek M, Cheson BD, Catovsky D, et al. iwCLL guidelines for diagnosis, Data Eng Bull. 2000;23:3–13. indications for treatment, response assessment, and supportive 13. Noy NF. Semantic integration: a survey of ontology-based approaches. management of CLL. Blood. 2018;131:2745–2760. ACM Sigmod Record. 2004;33:65. 3. Baliakas P, Hadzidimitriou A, Sutton LA, et al. Clinical effect of 14. Rumbold JM, Pierscionek B. The effect of the general data protection stereotyped B-cell receptor immunoglobulins in chronic lymphocytic regulation on medical research. J Med Internet Res. 2017;19:e47. leukaemia: a retrospective multicentre study. Lancet Haematol. 2014;1: 15. Kushida CA, Nichols DA, Jadrnicek R, et al. Strategies for de- e74–e84. identification and anonymization of electronic health record data for 4. Baliakas P, Hadzidimitriou A, Sutton LA, et al. Recurrent mutations refine use in multicenter research studies. Med Care. 2012;50 Suppl:S82– prognosis in chronic lymphocytic leukemia. Leukemia. 2015;29:329–336. S101. 3