Examining the Neurocognitive Validity of Commercially Available, Smartphone-Based Puzzle Games

doi:10.4236/psych.2012.37076

Paper Menu >>

Journal Menu >>

Psychology

2012. Vol.3, No.7, 525-526

Published Online July 2012 in SciRes (http://www.SciRP.org/journal/psych) http://dx.doi.org/10.4236/psych.2012.37076

Examining the Neurocognitive Validity of Commercially

Available, Smartphone-Based Puzzle Games

Oonagh Thompson, Suzanne Barrett, Christopher Patterson, David Craig

Centre for Public Health, Qu e e n ’s University Belfast, Belfast, U n i t e d K i n g d o m

Email: [email protected]

Received April 6th, 2012; revised May 2nd, 2012; accepted Ju n e 1st, 2012

Cognitive assessment typically involves assessing a person’s cognitive performance in unfamiliar and ar-

guably unnatural clinical surroundings. User-centred approaches to assessment and monitoring, driven by

issues such as enjoyability and familiarity, are largely absent. Everyday technologies, for example,

smartphones represent an opportunity to obtain an objective assessment of a person’s cognitive capabili-

ties in a non-threatening, discreet and familiar way, e.g. by everyday puzzle games undertaken as a leisure

activity at home. We examined the strength of relationships that exist between performance on common

puzzle games and standard measures of neuropsychological performance. Twenty-nine participants, aged

50 - 65 years, completed a comprehensive neuropsychological test battery and played three smart-

phone-based puzzle games in triplicate: a picture puzzle [Matches Plus], a word puzzle [Jumbline] and a

number puzzle [Sudoku]. As anticipated, a priori, significant correlations were observed between scores

on a picture puzzle and visual memory test (r = 0.49; p = 0.007); a word puzzle and estimated verbal IQ (r

= 0.53; p = 0.003) and verbal learning (r = 0.30; p = 0.039) tests; and a number puzzle and reason-

ing/problem solving test (r = 0.42; p = 0.023). Further analyses making allowance for multiple compari-

sons identified a significant unanticipated correlation (r = 0.49; p = 0.007) between number puzzle scores

and a measure of nonverbal working memory. Performance on these smartphone-based games was in-

dicative of relative cognitive ability across several cognitive domains at a fixed time point. Smart-

phone-based, everyday puzzle games may offer a valid, portable measure of assessing and monitoring

cognition in older adults.

Keywords: Cognitive Assessment; Alzheimer’s Disease; Information & Communication Technology

(ICT); Smartphone

Introduction

Current methods for assessing and monitoring Alzheimer’s

disease (AD) can be l engthy and obtrusive and rely upon testing

cognitive performance at fixed time-points, often in arguably

unnatural clinical settings. The majority of assessment method-

ologies are developed with neurocognitive assessment princi-

ples as the starting point.

Clinical assessment and diagnosis of dementia in the UK is

based upon a clinical interview and assessment of activities of

daily living, often involving the input of a relative or friend; the

administration of standardised assessment scales to assess the

severity of impairment in the individual; and a physical exami-

nation and/or brain scan where considered appropriate by the

physician. The most commonly used assessment scale is the

30-item Mini-Mental State Examination [MMSE] (Folstein et

al., 1975). The MMSE includes key domains of cognitive func-

tioning: orientation, registration, attention, recall, visuospatial

capabilities and language delivered in a structured setting. Pa-

tient monitoring usually takes the form of follow-up clinic ap-

pointments with the physician or nurse, including further clini-

cal interviews and formal cognitive assessment to identify

change.

Interest is growing in new assessment methods that combine

assessment and monitoring and which reduce the need for mul-

tiple, fixed time-point clinic visits while offering a more con-

venient, home-based alternative. Such user-centred approaches

to assessment and monitoring, driven by issues such as enjoy-

ability, familiarity, convenience and availability, are largely

absent.

Information and communication technology (ICT), particu-

larly the smartphone, represents an opportunity to obtain an

objective assessment of a person’s cognitive capabilities in a

non-threatening, discreet and familiar way, e.g. by everyday

puzzle games undertaken as a leisure activity. The mobile

phone is an accessible, portable, everyday technology with 77%

uptake among older adults in the UK, and smartphone market

penetration is rising (Ofcom, 2011). If performance on every-

day puzzles maps to scores attained on conventional neuropsy-

chological tests, then smartphone-based puzzles may offer a

valid portable measure of cognition. To investigate this idea

further, we sought to examine the strength of relationships that

exist between performance on three widely available puzzle

games and standard measures of neuropsychological perform-

ance.

Twenty-nine older adults (gender [f:m] = 19[66%]:10[34%];

mean age = 56.2; range = 50 - 65 years) were recruited through

local ageing networks to participate in this study. During single

researcher-supervised sessions participants completed a com-

prehensive neurocognitive assessment and played three time-

limited, smartphone-based puzzles in triplicate (tests and games

listed in Table 1). A standard operating procedure was used to

ensure consistency of test adminstration and game instruction. i

O. THOMPSON ET AL.

Table 1.

Relationship between puzzle game scores and neurocognitive test performance.

Puzzle Games

Word Game Jumbline Number Game Sudoku Picture Game Matches Plus

Neurocogn it ive Doma in : Neurocognitive Te st

r r r

Verbal IQ Estimate: National Adult Reading Test [NART] 0.53**1 0.22 –0.04

Verbal At tention/Work i ng Memory Span: Letter-Number Spa n [LNS] 0.27 0.23 0.29

Verbal Memory-Immediate: HVLT-R™ 0.271 0.27 0.40*

Verbal Memory-Delayed: HVLT-R™ 0.39*1 0.16 0.22

Visuospatial Attention/Working Memory Span: WMS®-III 0.21 0.49** 0.25

Visual Memory-Immediate: BVMT-R™ 0.32 0.27 0.49**1

Visual Memory-Delayed: BVMT-R™ 0.26 0.15 0.341

Attention/Speed of Processing: Trail Ma ki ng Test—Part A –0.35 –0.39 –0.21

Attention/Vigilance: Continuous Perform ance Test—Identical Pairs 0.1 0.24 0.16

Executive Function/Speed of Processing: BACS—Symbol Coding 0.09 0.28 0.42*

Executive Fu nction/Reasoning & Problem Solving: NAB®—Mazes 0.14 0.42*1 0.35

Verbal Flue ncy/Speed of Processing: Controlled Oral Word Association0.26 0.16 0.28

Pearson’s correlations: moderat e effect size = r > 0.3 ; large effect size = r > 0.5; St atistical si gnificance (2-tailed): *p < 0.05; **p < 0.01; 1a priori predictions; BACS: Brief

Assessment of Cognition in Schizophrenia; HVLT-R™: Hopkins Verbal Learning Test™-Revised; WMS®-III: Wechsler Memory Scale®-Third Edition; NAB®: Neuro-

psychological Assessment Battery® mazes; BMVT-R™: Brief Visuospatial Memory Test-Revised.

Neurocognitive tests were selected on the basis of having lim-

ited floor and ceiling effects, high tolerability for participants

and applicability in clinical trials (Nuechterlein et al., 2008).

The three games were selected on the basis of having good face

validity with regards to verbal memory [Jumbline], problem

solving [Sudoku] and visual memory [Matches Plus], and sat-

isfactory user ratings (minimum 3/5 stars) on the online “App

Store”.

Jumbline is a word puzzle that involves finding as many

words as possible within a set of given letters. Each level of the

game presents a new set of letters and players work against a

timer to try to complete each level in order to progress to the

next. Sudoku is a number-placement puzzle that involves in-

serting numbers into a partially filled 9 × 9 grid. The player’s

objective is to complete the grid so that the digits 1 to 9 are

contained within each row, each column and each 3 × 3 sub-

grid without duplication. Matches Plus is a matching game that

involves matching pairs of identical symbols as quickly as pos-

sible. Players work against a timer to try to match all pairs to

progress to the next level.

Based on game characteristics, we anticipated that the fol-

lowing would show moderate to large correlations: 1) Word

game [Jumbline] scores and tests of verbal memory; 2 Number

game [Sudoku] scores and tests of problem solving and logical

reasoning; 3) Picture game [Matches Plus] scores and tests of

visual memory.

The relationships between puzzle game and neuropsy-

chological test performance are reported in Table 1. In line

with a priori predictions, significant correlations were observed

between: word game scores and performance on the National

Adult Reading Test [NART] measure of reading ability/esti-

mated IQ and the Hopkins Verbal Learning Test™-Revised

[HVLT-R™] measure of verbal memory; number game scores

and the Neuropsychological Assessment Battery® [NAB®] test

of reasoning and problem solving; and picture game scores and

the Brief Visuospatial Memory Test-Revised [BVMT-R™]

measure of visual memory. Post-hoc exploratory analyses at the

1% significance level identified an additional, significant mod-

erate correlation between performance on the number game and

the Wechsler Memory Scale®-Third Edition [WMS®-III] mea-

sure of visuospatial working memory.

To our knowledge, this is the first study to examine the util-

ity of everyday, smartphone-based puzzles for validly assessing

cognitive abilities in this age group. Of note, in this middle-

aged, cognitively intact cohort, we were able to identify moder-

ately sized associations between conventional cognitive tests

and smartphone-based games in keeping with their face validity.

Further work will explore the ability of smartphone-based puz-

zles to detect cognitive impairment, as well as the level of im-

pairment that might limit smartphone use.

REFERENCES

Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). Mini-mental

state. A practical method for grading the cognitive state of patients

for the clinician. Jou rnal of Psychiatric Research, 12, 189-198.

doi:10.1016/0022-3956(75)90026-6

Nuechterlein, K. H., G reen, M. F., Kern, R. S., Baade, L. E., Barch , D.

M., Cohen, J. D., & Marder, S. R. (2008). The MATRICS consensus

cognitive battery, part 1: Test selection, reliability, and validity.

American Journal of Psychiatry, 165, 203-213.

doi:10.1176/appi.ajp.2007.07010042

Ofcom. (2011). The Communications Market Report: United Kingdom.

A nation addicted to smartphones. UK.

526