The radiation oncology in-training examination: an appeal for better testing.

Published on Nov 1, 2013in International Journal of Radiation Oncology Biology Physics7.038
· DOI :10.1016/J.IJROBP.2013.07.001
Ann Morris3
Estimated H-index: 3
(UIHC: University of Iowa Hospitals and Clinics)
Source
Abstract
Throughout medicine, many residency programs offer annual intraining examinations to gauge the progress of their trainees. As of 1992, 20 of 24 medical specialties offered such an examination (1), including internal medicine, surgery, radiology, and pediatrics. In 1984 the American College of Radiology (ACR) developed a similar examination for radiation oncology, the TXIT. Consisting of approximately 450 questions, this examination addressed the topics of radiation biology, radiation physics, and clinical oncology. Per the authors, the initial goal was to design an examination that “test[ed] the cognitive bases required of radiation oncologists.” At that time, Wilson and Diamond (2) stated that the items “were not chosen to discriminate among training levels but rather to reflect basic core knowledge in radiation oncology.” Unfortunately, it was not articulated at that time how training programs were intended to use the examination, nor were trainees instructed in how to interpret their results. Of note, no indication was given that the examination was intended as a predictive or punitive benchmark. This issue was clarified to some degree in 1992 when a later analysis of the examination stated that the American College of Radiology advised residency program directors not to base their resident evaluations on the examination (1). A 2004 report issued by the Association of Residents in Radiation Oncology further advocated that the examination be used only as a self-assessment measure. They believed that the examination should not be used as a “means of comparing residents within or between programs” (3). Nonetheless, a national survey of radiation oncology residents in 1990 found that 28% of residents felt that the TXIT had a moderate or strong impact on their annual evaluation (1). A similar survey, in 2004, showed that 33% of residents still felt that the TXIT had a moderate to significant impact on their annual review (3). Given this perception, it is important to assess what, if any, evidence exists to support the use of the ACR in-training examination as a formative assessment of radiation oncology residents.
📖 Papers frequently viewed together
2006
7 Authors (Shilpen Patel, ..., Eric Hansen)
References10
Newest
#1Keith PunchH-Index: 1
#2Alis OanceaH-Index: 1
Introduction Theory and Method in Education Research The Contexts for Education Research Ethics in Educational Research Research Questions From Research Questions to Data Literature Searching and Reviewing Qualitative Research Design Collecting Qualitative Data The Analysis of Qualitative Data Quantitative Research Design Collecting Quantitative Data The Analysis of Quantitative Data Mixed Methods Research Research Writing
Purpose To report the results, demographic data, and resident evaluation of the 2007 ACR radiation oncology in-training (TXIT) examination. Methods The 2007 TXIT examination consisted of 360 multiple-choice questions covering 13 different subject areas. It included 9 demographic questions and 7 evaluation items. Five hundred seventy-two residents from 85 institutions took the 2007 TXIT examination. Results The median raw score was 218.3 ± 29 (range, 140-295). The mean item difficulty was 60.7%, ...
Source
#1Arnold dela Cruz Paulino (BCM: Baylor College of Medicine)H-Index: 38
#2Elizabeth Kurtz (American College of Radiology)H-Index: 1
Purpose To review the results of the recent American College of Radiology (ACR) in-training examinations in radiation oncology and to provide information regarding the examination changes in recent years. Methods and Materials A retrospective review of the 2004 to 2007 ACR in-training examination was undertaken. Results The number of residents taking the in-training examination increased from 2004 to 2007, compatible with the increase in the number of radiation oncology residents in the United S...
Source
#1Mia A. Swartz (UW: University of Washington)H-Index: 4
#2Michael P. Porter (UW: University of Washington)H-Index: 31
Last. Noel S. Weiss (UW: University of Washington)H-Index: 125
view all 4 authors...
Abstract Objectives Primary urethral carcinoma is rare, and the demographic correlates of its incidence have not been examined using population-based data. Methods The National Cancer Institute Surveillance, Epidemiology, and End Results database was used to identify persons diagnosed with primary urethral carcinoma from 1973 to 2002. During this period, the Surveillance, Epidemiology, and End Results program included population-based tumor registries in nine geographic areas that represented ap...
Source
#1Shilpen Patel (UMB: University of Maryland, Baltimore)H-Index: 2
#2Reshma Jagsi (Harvard University)H-Index: 79
Last. Eric Hansen (UCSF: University of California, San Francisco)H-Index: 9
view all 6 authors...
Purpose: The aim of this study was to document adequacy of training, career plans after residency, use of the in-service examination, and motivation for choice of radiation oncology as a specialty. Methods and Materials: In 2004, the Association of Residents in Radiation Oncology (ARRO) conducted a nationwide survey of all radiation oncology residents in the United States. Results: The survey was returned by 297 residents (response rate, 54%). Of the respondents, 29% were female and 71% male. Th...
Source
#1Steven M. Downing (UIC: University of Illinois at Chicago)H-Index: 36
Source
#1Thomas M. HaladynaH-Index: 27
Contents: Introduction. Part I: A Foundation for Multiple-Choice Testing. The Importance of Item Development for Validity. Content and Cognitive Processes. Item Formats. Part II: Developing MC Test Items. MC Formats. Guidelines for Developing MC Items. A Casebook of Exemplary Items and Innovative Item Formats. Item Generation. Part III: Validity Evidence Arising From Item Development and Item Response Validation. Validity Evidence Coming From Item Development Procedures. Validity Evidence Coming...
#1Lawrence R. Coia (UPenn: University of Pennsylvania)H-Index: 11
#2J. Frank Wilson (MCW: Medical College of Wisconsin)H-Index: 26
Last. James J. Daimond (Thomas Jefferson University Hospital)H-Index: 1
view all 4 authors...
There have been eight in-training examinations for residents in radiation oncology since the American College of Radiology (ACR) administered its first in-training examination in 1984. In-training examinations are presently administered in 20 of 24 medical specialties (1). Sponsorship of the in-training examination is through organizations other than the specialty board in over half of the specialties. A previous paper summarized the results of the ACR experience with an in-training examination ...
Source
#1J. Frank Wilson (MCW: Medical College of Wisconsin)H-Index: 26
#2James J. Diamond (American College of Radiology)H-Index: 9
Abstract In 1984, the American College of Radiology administered its first in-training examination to residents in radiation oncology. This paper provides a summary of the overall results of the first 4 years of experience with this evaluatory instrument.
Source
#1Samuel Messick (Princeton University)H-Index: 52
Questions of the adequacy of a test as a measure of the characteristic it is interpreted to assess are answerable on scientific grounds by appraising psychometric evidence, especially construct validity. Questions of the appropriateness of test use in proposed applications are answerable on qthical grounds by appraising potential social consequences of the testing. The first set of answers provides an evidential basis for test interpretation, and the second set provides a consequential basis for...
Source
Cited By5
Newest
#1Hyun Ji Kim (Thomas Jefferson University)H-Index: 13
#2Voichita Bar Ad (Thomas Jefferson University)H-Index: 9
Last. Adam P. Dicker (Thomas Jefferson University)H-Index: 77
view all 4 authors...
Abstract Purpose The yearly radiation oncology in-training examination (ITE) by the American College of Radiology is a widely used, norm-referenced educational assessment, with high test reliability and psychometric performance. We distributed a national survey to evaluate the academic radiation oncology community's perception of the ITE. Methods and Materials In June 2014, a 7-question online survey was distributed via e-mail to current radiation oncology residents, program directors, and atten...
Source
#1Sandra S. Hatch (UTMB: University of Texas Medical Branch)H-Index: 11
#2Neha Vapiwala (UPenn: University of Pennsylvania)H-Index: 27
Last. Marie E. Taylor (Baptist Memorial Hospital-Memphis)H-Index: 1
view all 8 authors...
Radiation Oncology Resident In-Training Examination Sandra S. Hatch, MD, FACR,* Neha Vapiwala, MD,y Seth A. Rosenthal, MD, FACR,z John P. Plastaras, MD, PhD,y Albert L. Blumberg, MD, FACR,x William Small Jr, MD, FACR,jj Matthew J. Wenger, BBA,{ and Marie E. Taylor, MD, FACR, *University of Texas Medical Branch, Galveston, Texas; yUniversity of Pennsylvania, Philadelphia, Pennsylvania; zDepartment of Radiation Oncology, Sutter Cancer Center, Sacramento, California; xGreater Baltimore Medical Cent...
Source
#1Marie E. Taylor (Baptist Memorial Hospital-Memphis)H-Index: 1
#2Sandra S. HatchH-Index: 11
Last. Albert L. Blumberg (Greater Baltimore Medical Center)H-Index: 4
view all 7 authors...
Source
This website uses cookies.
We use cookies to improve your online experience. By continuing to use our website we assume you agree to the placement of these cookies.
To learn more, you can find in our Privacy Policy.