Software system

Usability

Engineering

Pure mathematics

Usability engineering

Software construction

Field (mathematics)

Non-regression testing

Systems engineering

Operating system

Usability inspection

Computer science

Software

Acceptance testing

Task (project management)

Mathematics

System usability scale

Workload

Software engineering

Usability lab

Human–computer interaction

Applied Ergonomics

The present article examined the effects of using different extra-laboratorial testing procedures in usability testing. Three experiments were conducted using different artefacts (website, computer-simulated mobile phone, fully operational smartphone) to compare different methodological approaches in field testing (synchronous and asynchronous remote testing, classical field testing) to lab-based testing under different operational conditions (dual task demands, poor product usability). Typical outcome variables of usability testing were measured, including task completion time, click rate, perceived usability and workload. Overall, the results showed no differences between field and lab-based testing under favourable operational conditions. However, under difficult operational conditions (i.e. dual task demands, poor product usability) differences between field and lab-based testing emerged (corresponding to small and medium effect sizes). The findings showed a complex pattern of effects, suggesting that there was no general advantage of one testing procedure over another.

Extra-laboratorial usability tests: An empirical comparison of remote and classical field testing with lab testing