Perspective Open Access The Progress Test of the European Hematology Association: A New Tool for Continuous Learning Marielle J. Wondergem1, Antonio Almeida2, Janaki Brolin3, Carlos Fernandez de Larrea4, Nathalie van Havre5, Bas T. Hemker6, Wietske Hollegien5, José-Tomás Navarro7, Mahesh Prahladan8, Alicia Rovó9, Gunnar Birgegård10 Correspondence: Marielle J. Wondergem (m.wondergem@amsterdamumc.nl). T he European Hematology Curriculum, first launched training was needed. In the spring of 2020, EHA started offering in 2006, was created by the European Hematology a progress test: a longitudinal test based on equivalent evalua- Association (EHA) with the aim of harmonizing hema- tions given at fixed intervals, assessing developments in knowl- tology training in Europe. Its goals were to define the edge. The EHA Progress Test was inspired by an earlier version different areas hematologists are expected to cover during their developed by the Swedish Hematology Association in 2013, training, and to establish the minimum recommended levels of which has become widely used by specialist trainees and spe- competence that a hematology trainee should attain.1 The logical cialists in Sweden. Noticeable pedagogical effects, like targeted next step was the creation of the European Hematology Exam study efforts in weak knowledge areas, changes in clinical rota- in 2015,2 in the wake of several European medical associations tions, and more have been reported in personal questionnaires. that use similar examinations as part of their end of training However, due to the limited number of Swedish trainees, data certification. Since then, the number of candidates and countries have not been published. participating has increased. Switzerland has included the exam- In undergraduate settings, progress testing has been used ination as part of its final assessment toward specialization as a for a long time, first introduced at the University of Maastricht hematologist. During the COVID pandemic, participation in the (“The Maastricht progress test”) in 1977.3 It is now widely used examination continued unabated, reflecting its importance for in many undergraduate programs all over the world. Students hematologists across Europe and beyond. are given a test containing an extensive set of multiple-choice EHA’s education platform (EHA Campus) offers opportu- questions (MCQs) once or twice a year, serving both as an edu- nities for continuous learning for both trainees and specialists. cational tool and as formal examinations of knowledge.4 The Content is guided by the European Hematology Curriculum, information gathered can also be used for (re)development of which provides a structure for individual study and self-assess- the curriculum and, in the case of collaboration between med- ment. To complete this organized learning environment, a tool ical schools, comparison of results between institutions.5 In for objective assessment of knowledge during and after specialist specialist training, progress tests have rarely been implemented. Instead, the focus has been on learning through clinical experi- ence, which means that longitudinal knowledge testing has not 1 Department of hematology, Amsterdam University Medical Centers, the been explored in most specialties. Trainees are mostly expected Netherlands to build their knowledge base in a self-directed way by accumu- 2 Department of Hematology, Hospital da Luz, Faculdade de Medicina, lating clinical experience, often with the help of a clinical tutor. Universidade Católica Portuguesa, Lisbon, Portugal There is extensive research on progress testing in undergraduate 3 Department of Internal Medicine, Central Hospital Karlstad, Sweden medical training, and the usefulness of the approach has been 4 Department of Hematology, Hospital Clínic de Barcelona, Institut d’Investigacions Biomèdiques August Pi i Sunyer, Universitat de Barcelona, Spain well validated. In postgraduate settings, there is a striking pau- 5 Education Department, European Hematology Association, The Hague, the city of reports on such testing, even though attempts have been Netherlands made in a few specialties.6–9 6 Educational Measurement, Cito, the Netherlands EHA developed the Progress Test to serve both trainees and 7 Department of Hematology, Institut Català d’Oncologia-Hospital Germans Trias certified hematologists. It should lead to greater comprehen- i Pujol, Josep Carreras Leukaemia Research Institute, Universitat Autònoma de sion of their level of knowledge in the different sections of the Barcelona, Spain European Hematology Curriculum, so that everyone can work 8 University of Cambridge, Pathology Portal, Royal college of Pathology, East Suffolk and North Essex Foundation Trust, Suffolk, United Kingdom on improving their knowledge in the weaker sections aiming at 9 Department of Hematology and Central Hematology Laboratory, Inselspital, Bern intrapersonal development. University Hospital, Switzerland To assess progress over time, the level of the test needed to be 10 Department of Medical Sciences, Uppsala Unicversity, Uppsala, Sweden consistent from one session to the next. Hence, we decided to use Copyright © 2022 the Author(s). Published by Wolters Kluwer Health, Inc. the previous versions of the European Hematology Exam. The on behalf of the European Hematology Association. This is an open-access questions of the examination are written by a trained group of article distributed under the terms of the Creative Commons Attribution-Non question writers and rigorously peer reviewed according to the Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible guidelines formed by the Examination and Progress Test Working to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal. Group. The choice for MCQs allows for a large sample size. Even HemaSphere (2022) 6:12(e815). with limited testing time, it can assess knowledge in several areas http://dx.doi.org/10.1097/HS9.0000000000000815. with high reproducibility and can be used in a web based for- Received: October 11, 2022 / Accepted: November 13, 2022 mat.10,11 The examination questions represent all 8 sections of the 1 Wondergem et al The Progress Test, a New Tool for Hematologist’s Continuous Learning Figure 1. Representation of candidate performance. The white line is the expected end of training level. Curriculum and are written in English.2 To guarantee independent in a room with minimal distraction. They had access to a Unit judgment of the quality of the examination and the passing score Reference Card that could be opened in a separate window. definition, EHA collaborates with Cito, an internationally rec- After taking the test, each participant could immediately access ognized professional research institute in the field of testing and their results: correct answers, explanations, and suggested liter- educational measurement. Cito judged the European Hematology ature. A summary displaying the overall result and the result per Exam to be a robust method of knowledge testing.2 All 2017 section of the curriculum was sent to every participant within exam questions were reviewed for the 2020 Progress Test by their 3 weeks after closure of the test. Results were not expressed as authors to ensure they were still up to date, considering changes pass or fail, underscoring that the Progress Test is not an exam- in diagnostics, treatment, and practice guidelines, and adapted if ination but a formative assessment to be used as a continuous needed. Authors were also asked to provide a short explanation educational tool. Participants received a graph depicting their of why the answers were right or wrong with references to rele- ability and a separate table showing whether they scored below vant literature or learning materials as feedback. (red cross) or at/above (green V) the expected end-of-training The same Progress Test was made available to EHA members level for each section of the European Hematology Curriculum twice a year, thus expanding the opportunity for everyone to (Figure 1). In addition, since a significant group was expected to take the test every year. The test was accessible on the EHA use the Progress Test as a preparation for the examination, par- Campus (ehacampus.ehaweb.org) for one month, in March and ticipants were provided with the number of correct questions November each year. After this period, participants were able that would have been a pass in the EHA European Exam in the to access their results and feedback for 3 months to guide their year the Progress Test was based on. All candidates were asked studies. The first Progress Test was made available in spring to provide demographic and training background information 2020, using the exam of 2017. The same test was again made using an online survey. The candidates were also asked why they available in fall 2020. A new Progress Test was created using the chose to take the Progress test and what their experience was. 2018 examination and made available spring and fall of 2021. Participation increased with each test, from 141 in spring and To make results comparable over the years, participants were 156 in autumn 2020 to 185 in spring 2021. Two hundred fif- advised to take the test in a continuous time period (2.5 hours), ty-eight participants took one test, 68 took two, and 19 took all 2 (2022) 6:12www.hemaspherejournal.com three; 207 of them answered the survey. There were participants AUTHOR CONTRIBUTIONS from 44 countries with 70% of the participants from Europe. MJW and GB participated in design and writing of the paper, and data Most were in specialty training and 26–35 years old. The par- analysis. AA, JB, CFL, JTN, MP, AR participated in writing of the paper. ticipants found the Progress Test to be relevant to their hema- NvH and WH participated in writing of the paper and data analysis. BTH tology training, the vast majority being clinicians (86%–93%). participated in data analysis. Participants mentioned that some domains were beyond the scope of their practice: some only treated malignant diseases, and DISCLOSURES in other countries, disorders of thrombosis and hemostasis are not treated by hematologists. The European Hematology Exam The authors have no conflicts of interest to disclose. and the Progress Test are made for European hematologists and based on the European Hematology Curriculum. Even so, writ- REFERENCES ing questions that apply to hematology practice in the whole of Europe is a challenge. It is reassuring that such a high percentage 1. Almeida AM, Ar C, Hellstrom-Lindberg E, et al. The European hematol- of participants considers the Progress Test relevant to their train- ogy curriculum: an electronic passport promoting professional compe- ing. The many participants from countries outside of Europe tence and mobility. HemaSphere. 2018;2:e49. report the same experience. Half of the participants used it “to 2. Navarro JT, Birgegard G, Strivens J, et al. The European hematology exam: the next step toward the harmonization of hematology training prepare for the specialist exam” and 54% “to use it as a learning in Europe. HemaSphere. 2019;3:e291. tool.” Both reasons for participation are what EHA aimed for 3. Schuwirth LW, van der Vleuten CP. The use of progress testing. Perspect when constructing the Progress Test. The overall evaluation of Med Educ. 2012;1:24–30. the experience of the test scored 4.5 on a scale of 1–5. Most of 4. Wrigley W, van der Vleuten CP, Freeman A, et al. A systemic framework the participants would have passed the examination with their for the progress test: strengths, constraints and issues: AMEE Guide No. scores on the Progress Test: 75.7%, 68.1%, and 66% in spring 71. Med Teach. 2012;34:683–697. 2020, fall 2020, and spring 2021, respectively. Thrombosis and 5. van der Vleuten CPM, Schuwirth LWT. Assessment in the con- hemostasis were the section with the lowest scores. text of problem-based learning. Adv Health Sci Educ Theory Pract. The Progress Test can be used in different ways. It could be 2019;24:903–914. 6. Dijksterhuis MGK, Scheele F, Schuwirth LWT, et al. Progress testing in used by educators and hematologists in training as a basis for postgraduate medical education. Med Teach. 2009;31:e464–e468. evaluation and to plan for future training goals, targeting areas 7. Van Mook WN, Arbous SM, Delwig H, et al. Progress testing in inten- where knowledge is below the expected level. For trained hema- sive care medicine training: useful and feasible?! Minerva Anestesiol. tologists, this information could influence decisions on which 2016;82:711–719. courses or conferences one might choose to attend the follow- 8. Joiner A, Thomas G, Hackett L, et al. Written progress tests in postgrad- ing year. There are even examples of the test being used in a uate psychiatry. Ir J Psychol Med. 2020;37:152–153. trivia-like setting, for example as a quiz. EHA can use (under) 9. Ravesloot C, van der Schaaf M, Haaring C, et al. Construct validation achievement in different sections of the Curriculum to guide the of progress testing to measure knowledge and visual skills in radiology. development of new educational materials. Using participants’ Med Teach. 2012;34:1047–1055. 10. Little JL, Bjork EL, Bjork RA, et al. Multiple choice tests exonerated evaluations, feedback can be adapted. at least of some charges: fostering test-induced learning and avoiding Whichever way the Progress Test is used, individually or test-induced forgetting. Psychol Sci. 2012;23:1337–1344. in groups, the objective of improving knowledge will still be 11. Pham H, Trigg M, Wu S, et al. Choosing medical assessments: does achieved. Its increasing popularity will most likely enable more the multiple-choice question make the grade? Educ Health. 2018; detailed analyses in the future. 31:65–71. 3