The Design and Development of Serious Games Using Iterative Evaluation

Published on May 1, 2017in Games and Culture
· DOI :10.1177/1555412016673262
Carl Symborski4
Estimated H-index: 4
Meg Barton2
Estimated H-index: 2
+ 3 AuthorsCarey K. Morewedge27
Estimated H-index: 27
(BU: Boston University)
In this article, we report on a serious game development approach, characterized by combining theory-based design with an iterative development strategy guided by experimental test and evaluation. We describe two serious games that teach the mitigation of cognitive biases (human tendencies to commit systematic errors in thinking that lead to irrational judgments). Cognitive biases tend to be deeply ingrained and early attempts to reduce biases with training have met with little success. We address this training challenge using bias mitigation theory derived from the literature and an instructional framework to establish the educational content of each game. The mitigation effects of the games were measured through multiple experiment cycles, and multiple play-testing campaigns were conducted to inform instructional model and game design revisions. The final game versions achieved a medium-to-large training effect following a single play session.
#1Irene Scopelliti (City University London)H-Index: 8
#2Carey K. Morewedge (BU: Boston University)H-Index: 27
Last. Karim S. Kassam (CMU: Carnegie Mellon University)H-Index: 17
view all 6 authors...
People exhibit a bias blind spot: they are less likely to detect bias in themselves than in others. We report the development and validation of an instrument to measure individual differences in the propensity to exhibit the bias blind spot that is unidimensional, internally consistent, has high test-retest reliability, and is discriminated from measures of intelligence, decision-making ability, and personality traits related to self-esteem, self-enhancement, and self-presentation. The scale is ...
36 CitationsSource
#1Carey K. Morewedge (BU: Boston University)H-Index: 27
#2Haewon Yoon (BU: Boston University)H-Index: 5
Last. Karim S. Kassam (CMU: Carnegie Mellon University)H-Index: 17
view all 6 authors...
From failures of intelligence analysis to misguided beliefs about vaccinations, biased judgment and decision making contributes to problems in policy, business, medicine, law, education, and private life. Early attempts to reduce decision biases with training met with little success, leading scientists and policy makers to focus on debiasing by using incentives and changes in the presentation and elicitation of decisions. We report the results of two longitudinal experiments that found medium to...
99 CitationsSource
#1Mary Magee QuinnH-Index: 3
#2Carl SymborskiH-Index: 4
Last. Stephanie GranatoH-Index: 1
view all 6 authors...
This paper describes a case study using a mixed method approach to playtesting a serious computer game designed to identify and mitigate cognitive bias. This approach incorporated in-depth data collection through the use of recordings of the screen and voice of the player while testing, a detailed survey, and a follow-up focus group. The data collection methodology served to provide detailed feedback from playtesters that was analyzed and used to make continuous and timely changes to the game.
2 CitationsSource
#1Pat Croskerry (Dal: Dalhousie University)H-Index: 32
#2Geeta Singhal (BCM: Baylor College of Medicine)H-Index: 8
Last. Sílvia Mamede (EUR: Erasmus University Rotterdam)H-Index: 25
view all 3 authors...
In a companion paper, we proposed that cognitive debiasing is a skill essential in developing sound clinical reasoning to mitigate the incidence of diagnostic failure. We reviewed the origins of cognitive biases and some proposed mechanisms for how debiasing processes might work. In this paper, we first outline a general schema of how cognitive change occurs and the constraints that may apply. We review a variety of individual factors, many of them biases themselves, which may be impediments to ...
178 CitationsSource
#1Tanner Olsen (UCF: University of Central Florida)H-Index: 2
#2Katelyn Procci (UCF: University of Central Florida)H-Index: 8
Last. Clint A. Bowers (UCF: University of Central Florida)H-Index: 45
view all 3 authors...
Usability testing is an important, yet often overlooked, aspect of serious game development. Issues in usability can drastically impact user experience and thus the learning outcomes associated with serious games. The goal of this paper is to provide serious game developers with an approach to efficiently and effectively apply usability testing into their development process. We propose a three-tiered approach to the assessment of game usability with the addition of assessments playability and l...
37 CitationsSource
May 4, 2011 in VS-GAMES (International Conference on Games and Virtual Worlds for Serious Applications)
#1Corrado Grappiolo (ITU: IT University of Copenhagen)H-Index: 5
#2Yun-Gyung Cheong (ITU: IT University of Copenhagen)H-Index: 13
Last. Georgios N. Yannakakis (ITU: IT University of Copenhagen)H-Index: 55
view all 5 authors...
We present a technology demonstrator for an adaptive serious game for teaching conflict resolution and discuss the research questions associated with the project. The prototype is a single-player 3D mini-game which simulates a resource management conflict scenario. In order to teach the player how to resolve this type of conflict, the underlying system generates level content automatically which adapts to player experience and behaviour. Preliminary results demonstrate the efficiency of the proc...
21 CitationsSource
#1Daniel Kahneman (Princeton University)H-Index: 149
Daniel Kahneman, recipient of the Nobel Prize in Economic Sciences for his seminal work in psychology challenging the rational model of judgment and decision making, is one of the world's most important thinkers. His ideas have had a profound impact on many fields - including business, medicine, and politics - but until now, he has never brought together his many years of research in one book. In "Thinking, Fast and Slow", Kahneman takes us on a groundbreaking tour of the mind and explains the t...
9,287 Citations
#1Carey K. Morewedge (CMU: Carnegie Mellon University)H-Index: 27
#2Daniel Kahneman (Princeton University)H-Index: 149
Dual-system models of reasoning attribute errors of judgment to two failures: the automatic operations of a ‘System 1' generate a faulty intuition, which the controlled operations of a ‘System 2' fail to detect and correct. We identify System 1 with the automatic operations of associative memory and draw on research in the priming paradigm to describe how it operates. We explain how three features of associative memory – associative coherence, attribute substitution and processing fluency – give...
398 CitationsSource
#1Ivan L. Beale (UNSW: University of New South Wales)H-Index: 20
#2Pamela M. Kato (Stanford University)H-Index: 18
Last. Steve W. Cole (UCLA: University of California, Los Angeles)H-Index: 75
view all 5 authors...
Abstract Purpose Adolescents with chronic illnesses have been shown to have management and treatment issues resulting in poor outcomes. These issues may arise from non-interest in self care and illness knowledge. A video game, “Re-Mission,” was developed to actively involve young people with cancer in their own treatment. Re-Mission provides opportunities to learn about cancer and its treatment. Method The efficacy of Re-Mission was investigated in a multi-site, randomized, controlled study with...
290 CitationsSource
#1W. Lewis Johnson (SC: University of Southern California)H-Index: 31
The Tactical Language and Culture Training System (TLCTS) helps learners acquire basic communicative skills in foreign languages and cultures. Learners acquire communication skills through a combination of interactive lessons and serious games. Artificial intelligence plays multiple roles in this learning environment: to process the learner's speech, to interpret and evaluate learner actions, to control the response of non-player characters, to generate hints, and to assess the trainee's mastery...
146 CitationsSource
Cited By4
#1Haewon Yoon (IU: Indiana University)H-Index: 5
#2Irene Scopelliti (City University London)H-Index: 8
Last. Carey K. Morewedge (BU: Boston University)H-Index: 27
view all 3 authors...
Abstract Observational learning can debias judgment and decision making. One-shot observational learning-based training interventions (akin to “hot seating”) can produce reductions in cognitive biases in the laboratory (i.e., anchoring, representativeness, and social projection), and successfully teach a decision rule that increases advice taking in a weight on advice paradigm (i.e., the averaging principle). These interventions improve judgment, rule learning, and advice taking more than practi...
#1Sam Snodgrass (NU: Northeastern University)H-Index: 10
#2Omid Mohaddesi (NU: Northeastern University)H-Index: 3
Last. Casper Harteveld (NU: Northeastern University)H-Index: 5
view all 6 authors...
Personalization has been explored in the context of games in many forms (e.g., dynamic difficulty adjustment, affective video games, adaptive systems, experience-driven PCG, etc.). The majority of techniques used in these fields have relied on data-driven or manual methods for identifying game components to modify for personalization. We propose a theoretical framework for identifying and categorizing low-level components of games that can be personalized. In this paper we first perform a review...
4 CitationsSource
Jul 26, 2019 in INTERACT (International Conference on Human-Computer Interaction)
#1Kelly J. NevilleH-Index: 3
#2Larry FlintH-Index: 1
Last. Amy Bolton (ONR: Office of Naval Research)H-Index: 2
view all 6 authors...
The rapidly increasing sophistication of cyber threats occurring in parallel with our growing reliance on networked systems for everything from shopping to managing critical infrastructure is not a coincidence. Ransomware events, compromise of personal financial information, and hacking into critical infrastructure systems make the headlines on a seemingly daily basis. Still, the average system user continues to operate in a mode that signals belief those events will happen to someone else. This...
1 CitationsSource
#1Anne-Laure Sellier (HEC Paris)H-Index: 7
#2Irene Scopelliti (City University London)H-Index: 8
Last. Carey K. Morewedge (BU: Boston University)H-Index: 27
view all 3 authors...
The primary objection to debiasing-training interventions is a lack of evidence that they improve decision making in field settings, where reminders of bias are absent. We gave graduate students in...
10 CitationsSource