Protocol Analysis and Expert Thought: Concurrent Verbalizations of Thinking during Experts' Performance on Representative Tasks

Published on Jun 1, 2006
· DOI :10.1017/CBO9780511816796.013
K. Anders Ericsson82
Estimated H-index: 82
The superior skills of experts, such as accomplished musicians and chess masters, can be amazing to most spectators. For example, club-level chess players are often puzzled by the chess moves of grandmasters and world champions. Similarly, many recreational athletes find it inconceivable that most other adults – regardless of the amount or type of training – have the potential ever to reach the performance levels of international competitors. Especially puzzling to philosophers and scientists has been the question of the extent to which expertise requires innate gifts versus specialized acquired skills and abilities. One of the most widely used and simplest methods of gathering data on exceptional performance is to interview the experts themselves. But are experts always capable of describing their thoughts, their behaviors, and their strategies in a manner that would allow less-skilled individuals to understand how the experts do what they do, and perhaps also understand how they might reach expert level through appropriate training? To date, there has been considerable controversy over the extent to which experts are capable of explaining the nature and structure of their exceptional performance. Some pioneering scientists, such as Binet (1893 / 1966), questioned the validity of the experts’ descriptions when they found that some experts gave reports inconsistent with those of other experts. To make matters worse, in those rare cases that allowed verification of the strategy by observing the performance, discrepancies were found between the reported strategies and the observations (Watson, 1913). Some of these discrepancies were explained, in part, by the hypothesis that some processes were not normally mediated by awareness/attention and that the mere act of engaging in self-observation (introspection) during performance changed the content of ongoing thought processes. These problems led most psychologists in first half of the 20th century to reject all types of introspective verbal reports as valid scientific evidence, and they focused almost exclusively on observable behavior (Boring, 1950). In response to the problems with the careful introspective analysis of images and perceptions, investigators such as John B.
📖 Papers frequently viewed together
1,600 Citations
5,023 Citations
5,010 Citations
#1Michelene T. H. ChiH-Index: 67
#2Robert GlaserH-Index: 52
Last. Marshall J. FarrH-Index: 2
view all 3 authors...
Contents: M.T.H. Chi, In Memoriam. R. Glaser, M.T.H. Chi, Overview. M.I. Posner, Introduction: What Is It to Be an Expert? Part I:Practical Skills. D.R. Gentner, Expertise in Typewriting. K.A. Ericsson, P.G. Polson, A Cognitive Analysis of Exceptional Memory for Restaurant Orders. J.J. Staszewski, Skilled Memory and Expert Mental Calculation. Part II:Programming Skills. E. Soloway, B. Adelson, K. Ehrlich, Knowledge and Processes in the Comprehension of Computer Programs. J.R. Anderson, P. Piroll...
2,013 CitationsSource
#1John R. AndersonH-Index: 132
1,236 CitationsSource
#1Nicola J. HodgesH-Index: 43
#2A.M. Williams (UBC: University of British Columbia)H-Index: 15
Preface - NICOLA HODGES and MARK WILLIAMS I. PRESENTING INFORMATION 1. The roles and uses of augmented feedback in motor skill acquisition - RICHARD MAGILL and DAVID ANDERSON 2. Mixing it up a little: How to schedule observational practice - NICOLE ONG and NICOLA HODGES 3. Attentional focus affects movement efficiency - KEITH LOHSE, GABRIELE WULF and REBECCA LEWTHWAITE 4. Advances in implicit motor learning - RICHARD S. W. MASTERS and JAMIE POOLTON II: OPTIMIZING PRACTICE CONDITIONS 5. Schedulin...
315 CitationsSource
#1Robert R. Hoffman (Adelphi University)H-Index: 39
Experts, who were the sole active dispensers of certain kinds of knowledge in the days before A1, have now often assumed a rather passive role. They relay their knowledge to various novices, knowledge engineers, experimental psychologists or cognitivists - or other experts - involved in the development and understanding of expert systems. This book aims to achieve a balance between experimentalists and theoreticians who deal with expertise. It tries to establish the benefits to society of an adv...
122 Citations
#1John B. Watson (Johns Hopkins University)H-Index: 27
1,504 Citations
#1Han L. J. van der Maas (UvA: University of Amsterdam)H-Index: 47
This study introduces the Amsterdam Chess Test (ACT). The ACT measures chess playing proficiency through 5 tasks: a choose-a-move task (comprising two parallel tests), a motivation questionnaire, a predict-a-move task, a verbal knowledge questionnaire, and a recall task. The validity of these tasks was established using external criteria based on the Elo chess rating system. Results from a representative sample of active chess players showed that the ACT is a very reliable test for chess experti...
175 Citations
#1K. Anders Ericsson (FSU: Florida State University)H-Index: 82
#2Peter F. Delaney (UF: University of Florida)H-Index: 18
Last. Rajan Mahadevan (UT: University of Tennessee)H-Index: 3
view all 4 authors...
After extensive laboratory testing of the famous memorist Rajan, Thompson, Cowan, and Frieman (1993) proposed that he was innately endowed with a superior memory capacity for digits and letters and thus violated the hypothesis that exceptional memory fully reflects acquired ?skilled memory.? We successfully replicated the empirical phenomena that led them to their conclusions. From additional analyses and new experiments, we found support for an alternative hypothesis, namely that Rajan’s superi...
131 CitationsSource
#1K. Anders Ericsson (FSU: Florida State University)H-Index: 82
The factors that cause large individual differences in professional achievement are only partially understood. Nobody becomes an outstanding professional without experience, but extensive experience does not invariably lead people to become experts. When individuals are first introduced to a professional domain after completing their education, they are often overwhelmed and rely on help from others to accomplish their responsibilities. After months or years of experience, they attain an accepta...
2,597 CitationsSource
#1Paul WardH-Index: 63
#2Nicola J. HodgesH-Index: 43
Last. Janet L. StarkesH-Index: 38
view all 4 authors...
104 CitationsSource
#1John B. WatsonH-Index: 27
96 Citations
Cited By280
Abstract Better understanding respondents' cognitions as they respond to situational judgment test (SJT) items and isolating which elements of knowledge they measure may allow psychologists to develop more predictive SJT items with greater ease. Consequently, we present a theoretical framework outlining the thought processes individuals engage in as they respond to SJT items. We review interactionist theories explaining how these models have shaped the understanding of how personality traits aff...
#1Cheryl L. Eames (SIUE: Southern Illinois University Edwardsville)H-Index: 3
#2Edith Aurora Graf (Princeton University)H-Index: 6
Last. Tammy Voepel (SIUE: Southern Illinois University Edwardsville)H-Index: 1
view all 5 authors...
Abstract In this paper we report validation efforts around the finite-to-finite strand of a provisional learning progression (LP) for the concept of function. We regard an LP as an empirically-verified account of how student understandings form over time and in response to instruction. The finite-to-finite strand of the LP was informed by literature on students’ thinking and learning related to functions as well as the Algebra Project’s curricular approach, which is designed for students who are...
1 CitationsSource
#1Matthew J. Liberatore (Villanova University)H-Index: 28
#2William P. Wagner (Villanova University)H-Index: 11
Abstract This paper explores the ways in which higher education students search for information in fragile and conflict-affected contexts. Data for this study was drawn from verbal reports of nine participants engaged in retrospective think-aloud sessions to solve ten tasks each. The results of the thematic analysis revealed that the participants followed the pattern outlined in literature of connectivism and literacy frameworks. Namely, the participants proceeded with four interrelated steps: l...
#1Pmgm Kop (LEI: Leiden University)H-Index: 1
#1Peter M.G.M. Kop (LEI: Leiden University)H-Index: 4
Last. J.H. van Driel (University of Melbourne)H-Index: 10
view all 4 authors...
Student insight into algebraic formulas, including the ability to identify the structure of a formula and its components and to reason with and about formulas, is an issue in mathematics education....
1 CitationsSource
#1Sajad Ahmad Najar (IITI: Indian Institute of Technology Indore)H-Index: 2
#2Premjit K. Sanjram (IITI: Indian Institute of Technology Indore)H-Index: 3
AbstractDriving is a complex task performed in a dynamic environment whereby visuospatial information is a critical aspect of driver cognition. Drivers’ interaction with the in-vehicle interfaces challenges their ability to respond to the dynamic traffic scenes efficiently. This study examines the effect of in-vehicle object and spatial distractions on drivers’ gaze behavior and driving performance. An experimental paradigm named as direction following in distracted driving – object and spatial ...
1 CitationsSource
#1E. Brumberger (ASU: Arizona State University)H-Index: 10
#1Lauren R. Tufton (Lboro: Loughborough University)
#2Sophia Jowett (Lboro: Loughborough University)H-Index: 51
This qualitative study explored how riders perceive and understand the relationship with their horse. Participants included ten elite female riders with a mean age of 40.6 years, five of whom compe...
#1Maria KramerH-Index: 3
#2Christian FörtschH-Index: 7
Last. Birgit J. NeuhausH-Index: 14
view all 4 authors...
Abstract Different constructs describe and analyze teachers’ diagnostic processes, in which situation-specific diagnostic skills are utilized to solve challenging situations of teaching and learning in class. This study compares two constructs—professional vision and diagnostic activities—with regard to their suitability for coding and analyzing data collected from interviews using think-aloud protocols with five in-service teachers, who worked in the video-based assessment tool DiKoBi Assess. F...
#1Rebekka Stahnke (Technical University of Dortmund)H-Index: 2
#2Sigrid Blömeke (University of Oslo)H-Index: 33
Abstract The study investigates 39 novice and expert teachers’ perception, interpretation and decision-making skills with respect to classroom management events which they observed in two video clips. Their retrospective comments were analyzed with a multi-category coding scheme. Experts interpreted more and suggested more alternative courses of action than novices. They also focused more on student learning and the context of instruction. Concerning the relation of skills and focus, experts per...