Brain-Computer Interfaces (BCIs) are systems that are primarily developed for use of paralyzed people. Although their main aim of use has a medical point of view, they can also be used for different aims such as entertainment and cognitive researches. Since BCI systems have specific brain potentials (P300, steady state evoked potential) and ERD/ERS (Motor Imagery), they are also flexible tools for cognitive science. In this study, an experiment was conducted with 30 participants. Each participant completed two tasks through a BCI and filled NASA-TLX forms. The results were analyzed using paired t-tests to see whether BCI tasks are significantly different in terms of creating cognitive load. The results showed that NASA-TLX scores of the BCI tasks were significantly different and these systems can be considered for estimating cognitive states studies.
Cognitive Load BCI NASA-TLX1. Introduction
Human error can be defined as some undesired behaviors caused by some improprieties such as lack of knowledge, lack of communication, lack of attention, excessive stress, mental fatigue, and environmental factors. Monitoring cognitive state is a useful tool to reduce these errors. Therefore, many studies about cognitive load have been conducting in several research areas.
According to cognitive load theory, two persons may complete the same task under the same conditions, at the same time and with the same level of success. However, it doesn’t guarantee that both people have the same level of cognitive load. It depends on many different factors, such as personal qualifications, attributes, cognitive load, and cognitive state. Although it is important to define the level of cognitive load, there is no available certain metrics to measure. The current methods to measure cognitive load in the literature just try to give some efficient indicators to estimate cognitive load.
Tracy and Albers [1] state that when the cognitive load increases, the ability to perform effectively decreases slowly until the person reaches a point of cognitive overload. According to Hussain et al. [2] , monitoring cognitive load is crucial for developing adaptive systems aware of the user’s mental workload. Because such systems can reduce error related risks during task-critical operations. Cain [3] states that a commonly accepted definition of workload does not exist. However, it is possible to characterize workload as a mental structure that reflects the mental strain caused by performing a task under specific environmental and operational conditions, besides the capability of the operator to respond to demands of the task.
Methods for measuring cognitive load can be grouped under three titles: subjective methods, behavioral methods, and physiological methods. Subjective methods are based on self-evaluations of people who completed the relevant tasks. Behavioral methods deal with task completion time and task completion success. These methods usually include secondary task implementations that require completing two tasks concurrently. Physiological methods are used via monitoring some indications from autonomic nerve system and brain activities. Pupil responses, galvanic skin response, and heart rate variability are some of the most common indicators used to estimate cognitive load. EEG and fMRI systems are preferred in studies based on brain activities to estimate cognitive load by monitoring cognitive state.
Antonenko et al. [4] study on comparing NASA-TLX and EEG data and state using EEG is a more objective approach to estimate cognitive load. Haapalainen et al. [5] explain using more than one physical symptom makes the cognitive load estimation more reliable. The study has an experimental part that considers 20 participants. The parameters below were collected from the participants during the experiment:
・ Changes on pupil sizes,
・ Eye movements,
・ Number and duration of eye blinking,
・ Heart rate variability,
・ Respiration, and
・ Monitoring brainwaves.
Walczyk et al. [6] explain that when somebody is telling a lie, his/her pupil sized shows some changes. They conducted a scenario based experiment with 145 participants to identify pupil size differences between telling truth and lie.
BCIs mainly have medical aims but they also have huge potential to be used in different areas of life. Since they are being activated via specific brain activities brain based cognitive researches. They usually offer three kinds of applications that are activated through P300 potential, steady state visual evoked potential, and event related desynchronization/synchronization. In this study, one of the recently developed BCI system’s letter matrix applications was utilized. Different difficulty levels and cognitive load of participants was measured via a well-known subjective method: NASA-TLX.
1.1. NASA-TLX
Since subjective methods are easy to use, low cost and provide simple quantitative results, they still keep their popularity. One of the widely used subjective methods is NASA-TLX. This method was developed in Hart and Staveland [7] and its application areas were investigated by Hart [8] after 20 years development. There are many other numbers of studies that used NASA-TLX in the relevant literature. Rubio et al. [9] conducted a comparison with NASA-TLX and another subjective method SWAT. Noyes and Bruneau [10] searched differences between use of pencil and computer versions of NASA-TLX. Alm and Nilsson [11] investigated negative effects of using mobile telephone during driving via NASA-TLX method. Yurko et al. [12] stated that higher mental workload caused poorer laparoscopic performance. A paper published by Miyake [13] offered a combined workload estimating method including physiological and subjective methods.
NASA-TLX has six sub-scales which reflect participant’s feelings and opinions about the task. The sub-scales are listed below:
・ Mental demand,
・ Physical demand,
・ Temporal demand,
・ Performance,
・ Effort,
・ Frustration.
Participants evaluate the task with these scales from 0 to 100. It is also possible to decide weights of the scales with participants’ evaluations. Fifteen paired comparisons are available and the participant decides more considerable scales for each comparison. Finally it provides an overall score between 0 and 100 to evaluate the difficulty level of the task.
1.2. Brain Computer Interfaces
BCIs are technological systems including a computer, relevant software, EEG components (bonnet, electrodes, etc.), and an amplifier. The system works based on EEG and simply brain activities by converting them to comments for the computer using interfaces [14] . Figure 1 illustrates working steps of BCI systems. The main goal of BCI research for the last thirty years is to help paralyzed people for their daily life. However, it is also possible to use these interfaces for other cognitive researches related with ERD and other brain activities.
Karagoz et al. [15] state that BCI systems utilize three main neurophysiological facts. One of them is P300 potential that appears when a problem is solved, a decision is made, or an fogginess disappears in 300 ms. Alkaç [16] stated that P300 doesn’t appear with the standard stimulates. It requires specific target stimulates. That’s why this potential is used in the studies about gathering attention and focusing. The relevant interface of P300 is usually a letter matrix that columns and rows flash and the participant tries to focus on his/her target letter to type it on the screen.
Since P300 potential has a close relationship with decision making, problem solving and focusing on target stimulus, it is also an attractive brain fact for cognitive load researchers. Schultheis and Jameson [17] designed an experiment with a task including reading written materials with various level of difficulty. They used an EEG, remote eye tracker, and subjective scale to measure the cognitive load. Based on the results, it can be satated that more difficult texts led to lower reading speed, higher subjective load ratings, and a reduced P300 amplitude.
SSVEP is an another fact that is being used by BCI systems. BCIs generate this potential with flashing controls. The direction indicators flash with various frequencies and the participant tries to focus on the target indicator to give the comment. This application makes possible to control some electronic systems [18] . Also, It requires a high level of attention and concentration to be able to catch quick flickers. Further, SSVEP is an useful fact for cognitive load researchers. Perego et al. [19] conducted an experimental study and they stated that the test showed SSVEP based BCI can be used for psychometric test.
The third neurophysiologic fact used by BCIs is changes on motor zones of the brain when participant focuses on thinking amovement to a specific direction [18] . When we think to move our right hand, an activity appears on the left side of our brain. BCI system converts it to a comment for the computer to move cursor on the screen.
2. Method
In this study, an experiment was designed using a BCI system developed by G-TEC Company. The system includes a letter matrix (Figure 2) to type letters via P300 potential. Thirty 18 - 27 aged participants (15 female,
Operation Diagram of BCI [<xref ref-type="bibr" rid="scirp.53279-ref20">20</xref>]
15 male) were recruited for this experiment. Each participant completed two tasks using the letter matrix. All the experiments were conducted in the same silent room with same climatic, illumination conditions. For running BCI, the same computer (Core i5, 8 GB Ram, 15.6 inch screen) was used.
Before executing the task EEG, electrodes were mounted. Figure 3 illustrates the locations for electrode placement for using the matrix.
Task 1: The first task was defined as typing “ABCDE” on the screen. Letter matrix’s rows and columns started flashing and participants must have focused on the target letters one by one starting from the first letter. Each target letter flashed 15 times for 75 ms. Besides each target letter had flashed for a second right before the rows and columns’ flashes started to make the participant realize its position on the matrix.
Task 2: After Task 1 had been completed, participants filled a paper version NASA-TLX form and took a rest for 5 minutes. For Task 2, participants were asked to type “ESOGU2014” on the screen. For this task, consecutive target letters were not assigned side to side. There was no flashing to show letter’s initial position. Therefore, participants had to find the letter’s location by themselves during 2 seconds that is a constant time period between typing a letter and starting flashing for the next one. In this task, each target letter flashed 5 times for 50 ms. After completing Task 2, participants filled NASA-TLX form again. It was expected that Task 2’s performances would be lower and mental workload score would be higher.
P300 Letter Matrix
P300 Electrode Placement Locations
3. Results
All participants’ task performances were recorded (number of correct letters/total number of letters) and NASA- TLX scores were calculated for both tasks. Since Task 2 was more difficult, it was expected that NASA-TLX scores for Task 2 would be dramatically lower. Appendix A and Appendix B illustrate scale weights and overall NASA-TLX scores and Figure 4 summarizes task performances of all participants.
Due to longer word, no help for finding letter on the matrix, quicker, and shorter flashings a huge decrease is recorded in Task 2 performances. Average success levels are 64.00 and 34, 81 for task 1 and task 2 respectively.
As seen in Appendices A and B, better overall NASA-TLX scores was obtained for task 1. Difference between difficulty levels of the tasks can be seen from the average values. The most dramatic change was provided for mental demand in terms of raw scores. When the weight of the scales is investigated, the largest change belongs to weight of temporal demand due to quicker flashings and trying to find the next target letter during Task 2.
4. Analysis
A paired t-test was conducted to assess whether the difference between overall NASA-TLX scores is significant or not. Before conducting paired t-test, normality of the overall scores was checked with Anderson-Darling test in Minitab 16. Both tasks’ NASA-TLX scores distributed normally (Figure 5 and Figure 6).
Result of paired t-test is presented below. Since P-value is less than 0.05, we can say that mental workload levels of the tasks are statistically significant (Table 1).
5. Conclusion and Future Work
In this study, it aims to see how possible it is to use BCI systems for cognitive load measurement and cognitive state monitoring. Results show that it is possible to use these systems with at least one brain potential including
Task Performances
Normality Test of Overall NASA-TLX Score for Task 1
Normality Test of Overall NASA-TLX Score for Task 2
Paired t-test result for overall NASA-TLX scores
N
Mean
StDev
SE Mean
NASA-TLX 1
30
40.44
16.29
2.97
NASA-TLX 2
30
58.40
14.76
2.69
Difference
30
−17.96
19.87
3.63
95% CI for mean difference: (−25, 38; −10, 54); t-test of mean difference = 0 (vs not = 0); T-value = −4.95; P-value = 0.000.
well-determined task parameters. Obtaining significant differences in terms of subjective mental load assessment indicates that these systems can be used as a member of combined methods of estimating cognitive state or measuring cognitive load. With the support of some physiological parameters, these systems can become a wide task designing and application platforms. In the future studies, this BCI system will be combined with a remote eye tracker and Galvanic Skin Response device and a new experiment can be designed with more participants.
Acknowledgements
This Study is a part of the project (No. 201315A208) which is supported by Eskisehir Osmangazi University Scientific Research Projects Office.
Appendix A: NASA-TLX Results of Task 1Appendix B: NASA-TLX Results of Task 2ReferencesTracy, J.P. and Albers, M.J. (2006) Measuring Cognitive Load to Test the Usability of Web Sites. Society for Technical Communication 53rd Annual Conference, Las Vegas, 7-10 May 2006, 256-260.Hussain, S., Chen, S.Y., Calvo, R.A. and Chen, F. (2011) Classification of Cognitive Load from Task Performance & Multichannel Physiology during Affective Changes. MMCogEmS: Inferring Cognitive and Emotional States from Multimodal Measures, ICMI 2011 Workshop, 17 November 2011, Alicante.Cain, B. (2007) A Review of the Mental Workload Literature. Defence Research and Development Canada, Toronto.Antonenko, P., Paas, F., Grabner, R. and van Gog, T. (2010) Using Electroencephalography to Measure Cognitive Load. Educational Psychology Review, 22, 425-438. http://dx.doi.org/10.1007/s10648-010-9130-yHaapalainen, E., Kim, S., Forlizzi, J.F. and Dey, A.K. (2010) Psycho-Physiological Measures for Assessing Cognitive Load. Proceedings of the 12th ACM International Conference on Ubiquitous Computing, Copenhagen, 26-29 September 2010, 301-310. http://www.ubicomp.org/ubicomp2010/Walczyk, J.J., Griffith, D.A., Yates, R., Visconte, S.R., Simoneaux, B. and Harris, L.L. (2012) Lie Detection by Inducing Cognitive Load Eye Movements and Other Cues to the False Answers of “Witnesses” to Crimes. Criminal Justice and Behavior, 39, 887-909. http://dx.doi.org/10.1177/0093854812437014Hart, S.G. and Staveland, L.E. (1988) Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Advances in Psychology, 52, 139-183. http://dx.doi.org/10.1016/S0166-4115(08)62386-9Hart S.G. ,et al. (2006)NASA-Task Load Index (NASA-TLX); 20 Years Later Proceedings of the Human Factors and Ergonomics Society Annual Meeting 50, 904-908.Rubio, S., Díaz, E., Martín, J. and Puente, J.M. (2004) Evaluation of Subjective Mental Workload: A Comparison of SWAT, NASA-TLX, and Workload Profile Methods. Applied Psychology, 53, 61-86. http://dx.doi.org/10.1111/j.1464-0597.2004.00161.xNoyes, J.M. and Bruneau, D.P. (2007) A Self-Analysis of the NASA-TLX Workload Measure. Ergonomics, 50, 514-519. http://dx.doi.org/10.1080/00140130701235232Alm, H. and Nilsson, L. (1995) The Effects of a Mobile Telephone Task on Driver Behaviour in a Car Following Situation. Accident Analysis & Prevention, 27, 707-715. http://dx.doi.org/10.1016/0001-4575(95)00026-VYurko, Y.Y., Scerbo, M.W., Prabhu, A.S., Acker, C.E. and Stefanidis, D. (2010) Higher Mental Workload Is Associated with Poorer Laparoscopic Performance as Measured by the NASA-TLX Tool. Simulation in Healthcare, 5, 267-271. http://dx.doi.org/10.1097/SIH.0b013e3181e3f329Miyake, S. (2001) Multivariate Workload Evaluation Combining Physiological and Subjective Measures. International Journal of Psychophysiology, 40, 233-238. http://dx.doi.org/10.1016/S0167-8760(00)00191-4Ozkan, N.F. and Kahya, E. (2014) Measurement of Cognitive Workload by Use of Combined Methods Including Brain-Computer Interfaces. Advances in Social and Organizational Factors, 12, 458-468.Karagoz, M., Alkac, U., Ergen, N., Erdamlar, N. and Alpkan, L. (2005) Psikiyatrik Hastaliklarda Elektrofizyolojik (P300) Yontemler. Düsünen Adam, 18, 210-216.Alkac, U. ( 2009) Beyin Arastirmalari Tarihinde Bir Gezinti: Elektronorofizyoloji. Klinik Gelisim, 3, 14-19Schultheis, H. and Jameson, A. (2004) Assessing Cognitive Load in Adaptive Hypermedia Systems: Physiological and Behavioral Methods. In: De Bra, P.M.E. and Nejdl, W., Eds., Adaptive Hypermedia and Adaptive Web-Based Systems, Springer Berlin, Heidelberg, 225-234. http://dx.doi.org/10.1007/978-3-540-27780-4_26Gencer, N., Usakli, A.B., Erdogan, H.B., Akinci, B. and Acar, E. (2010) ODTü Beyin Arastirmalari Laboratuvari: Beyin-Bilgisayar Arayüzü calismalari. ODTüLü, 45, 20-21Perego, P., Turconi, A.C., Andreoni, G., Maggi, L., Beretta, E., Parini, S. and Gagliardi, C. (2011) Cognitive Ability Assessment by Brain-Computer Interface: Validation of a New Assessment Method for Cognitive Abilities. Journal of Neuroscience Methods, 201, 239-250. http://dx.doi.org/10.1016/j.jneumeth.2011.06.025www.wired.com.