Methodology for playtesting serious games A case study using a mixed method approach

Published on Nov 11, 2013
· DOI :10.1109/IGIC.2013.6659130
Mary Magee Quinn3
Estimated H-index: 3
,
Carl Symborski4
Estimated H-index: 4
+ 3 AuthorsStephanie Granato1
Estimated H-index: 1
Sources
Abstract
This paper describes a case study using a mixed method approach to playtesting a serious computer game designed to identify and mitigate cognitive bias. This approach incorporated in-depth data collection through the use of recordings of the screen and voice of the player while testing, a detailed survey, and a follow-up focus group. The data collection methodology served to provide detailed feedback from playtesters that was analyzed and used to make continuous and timely changes to the game.
📖 Papers frequently viewed together
1 Citations
2019CHI: Human Factors in Computing Systems
2 Citations
2012ICIS: International Conference on Computer and Information Science
3 Authors (Sarmad Soomro, ..., Suziah Sulaiman)
17 Citations
References9
Newest
#1Tanner Olsen (UCF: University of Central Florida)H-Index: 2
#2Katelyn Procci (UCF: University of Central Florida)H-Index: 8
Last. Clint A. Bowers (UCF: University of Central Florida)H-Index: 45
view all 3 authors...
Usability testing is an important, yet often overlooked, aspect of serious game development. Issues in usability can drastically impact user experience and thus the learning outcomes associated with serious games. The goal of this paper is to provide serious game developers with an approach to efficiently and effectively apply usability testing into their development process. We propose a three-tiered approach to the assessment of game usability with the addition of assessments playability and l...
44 CitationsSource
#1Regina BernhauptH-Index: 20
89 CitationsSource
#1Christina KoeffelH-Index: 3
Last. Manfred TscheligiH-Index: 37
view all 6 authors...
This chapter describes an approach to evaluating user experience in video games and advanced interaction games (tabletop games) by using heuristics. We provide a short overview of computer games with a focus on advanced interaction games and explain the concept of user-centred design for games. Furthermore, we describe the history of heuristics for video games and the role of user experience of games in general. We propose a framework consisting of three sets of heuristics (game play/game story,...
42 CitationsSource
#1Michael C. MedlockH-Index: 1
#2Dennis WixonH-Index: 24
Last. Ramon L. RomeroH-Index: 3
view all 4 authors...
This paper defines and evaluates a method that some practitioners are using but has not been formally discussed or defined. The method leads to a high ratio of problems found to fixes made and then empirically verifies the efficacy of the fixes. We call it the Rapid Iterative Testing and Evaluation method – or RITE method. Application to the tutorial of a popular game, Age of Empires II, shows this method to be highly effective in terms of finding and fixing problems and generating positive indu...
144 Citations
#1Wolmet Barendregt (TU/e: Eindhoven University of Technology)H-Index: 19
#2Mathilde M. Bekker (TU/e: Eindhoven University of Technology)H-Index: 18
Last. Ester Baauw (TU/e: Eindhoven University of Technology)H-Index: 6
view all 4 authors...
This paper describes an experiment to discover the change in the types of detected problems and the attitude of children towards a game when user testing a computer game for young children during first use and after they have practiced with a game. Both the numbers of different types of identified problems and the severity of the problems are investigated. Based on this knowledge, practitioners could adapt the set up of their user tests to effectively find as many aspects of the game as possible...
57 CitationsSource
#1Anker Helms Jørgensen (ITU: IT University of Copenhagen)H-Index: 8
The fields HCI/usability and computer games have existed for a few decades with virtually no mutual interaction. However, in recent years, a number of exchanges have appeared, both in academia and in practice. This paper presents a preliminary account of this development. Exchanges in both directions seem viable: evaluation methods from HCI/usability towards games and interaction techniques and supporting user communication from games towards HCI/usability. The paper concludes with a discussion ...
81 CitationsSource
Apr 24, 2004 in CHI (Human Factors in Computing Systems)
#1Heather DesurvireH-Index: 10
#2Martin CaplanH-Index: 1
Last. Jozsef A. TothH-Index: 1
view all 3 authors...
Heuristics have become an accepted and widely used adjunct method of usability evaluation in Internet and software development. This report introduces Heuristic Evaluation for Playability (HEP), a comprehensive set of heuristics for playability, based on the literature on productivity and playtesting heuristics that were specifically tailored to evaluate video, computer, and board games. These heuristics were tested on an evolving game design to assess their face validity and evaluation effectiv...
453 CitationsSource
Mar 31, 2001 in CHI (Human Factors in Computing Systems)
#1Jared M. SpoolH-Index: 9
#2Will SchroederH-Index: 5
We observed the same task executed by 49 users on four production web sites. We tracked the rates of discovery of new usability problems on each site and, using that data, estimated the total number of usability problems on each site and the number of tests we would need to discover every problem. Our findings differ sharply from rules-of-thumb derived from earlier work by Virzi[1] and Nielsen[2,3] commonly viewed as "industry standards." We found that the four sites we studied would need consid...
141 CitationsSource
#7Dave ThomasH-Index: 17
2,387 Citations
Cited By3
Newest
Source
#1Pratheep Kumar Paranthaman (NU: Northeastern University)H-Index: 5
#2Seth Cooper (NU: Northeastern University)H-Index: 19
Iterative playtesting of games is crucial, but remains a time consuming and ad-hoc process. In this work, we streamlined and systematized the playtesting process by adding support for crowdsourced, on-demand playtesting directly into a game development environment to test and evaluate platform games. Our system, called ARAPID (As Rapid As Possible Iterative Design), manages playtesting from a control panel in Unity and recruits players from Mechanical Turk. With this system, game designers can l...
2 CitationsSource
#1Carl Symborski (Leidos)H-Index: 4
#2Meg Barton (Leidos)H-Index: 2
Last. Carey K. Morewedge (BU: Boston University)H-Index: 27
view all 6 authors...
In this article, we report on a serious game development approach, characterized by combining theory-based design with an iterative development strategy guided by experimental test and evaluation. We describe two serious games that teach the mitigation of cognitive biases (human tendencies to commit systematic errors in thinking that lead to irrational judgments). Cognitive biases tend to be deeply ingrained and early attempts to reduce biases with training have met with little success. We addre...
6 CitationsSource