Nowadays, there is a great need to investigate the effects of fatigue on physical as well as mental performance. The issues that are generally associated with extreme fatigue are that one can easily lose one’s focus while performing any particular activity whether it is physical or mental and this decreases one’s motivation to complete the task at hand efficiently and successfully. In the same line of thought, myriads of research studies posited the negative effects of fatigue on mental performance, and most techniques to induce fatigue to require normally long-time and repetitive visual search tasks. In this study, a visual search algorithm task was devised and customized using performance measures such as d’ ( d-prime) and Speed Accuracy Trade-Off ( SATF) as well as ROC analysis for classifier performance. The visual search algorithm consisted of distractors ( L) and a target ( T) whereby human participants had to press the appropriate keyboard button as fast as possible if they notice a target or not upon presentation of a visual stimulus. It was administered to human participants under laboratory conditions, and the reaction times, as well as accuracy of the participants, were monitored. It was found that the test image Size35Int255 was the best image to be used in terms of sensitivity and AUC (Area under Curve). Therefore, ongoing researches can use these findings to create their visual stimuli in such a way that the target and distractor images follow the size and intensity characteristics as found in this research.
Fatigue is defined as a condition or phenomenon of decreased ability and efficiency of mental and/or physical activities caused by excessive mental and/or physical activities, or illness. It is characterized by decreased energy, motivation and difficult concentrating or difficulty in maintaining sustained attention and remaining alert to relevant stimuli appearing at unpredictable time points over long periods of time. Fatigue is usually accompanied by a sense of discomfort, desire to rest and reduced motivation [
To date, in most fatigue studies, the methods used to induce fatigue are either through sleep deprivation or the performance of long-term, tedious and repetitive tasks. Moreover, there are many researches study which employ and support long-time repetitive visual search tasks to induce mental fatigue [
The visual search algorithm was developed using GNU Octave programming language running on Linux (Debian8). The visual search algorithm displays a number of rotated distractors (L) and a rotated target (T) presented randomly on a Cathode Ray Tube (CRT) monitor’s screen with width 600 and height 480. The number of distractors including the target (N) was varied from 12 to 28 in increments of 4. The image size of the distractor L and that of the target T were varied from [25 × 25] pixels to [40 × 40] pixels in increments of 5 to produce 4 different image sizes. Then, the image intensity for each particular distractor image size as well as the target image size was varied in the following manner: 220, 230, 240 and 255. This produces in total 16 types of grayscale test images for the distractor L and 16 types of grayscale test images for the target T. Some examples of the images are shown in Figures 1-3. The distance between the participants and the monitor screen was kept at constant at 60 cm.
The whole research study ethics was approved by Tubitak’s Ethics committee. Prior to performing the visual search task, an informed consent form was provided to each participant to demonstrate their willingness to take part in this psychophysics experiment. The psychophysics experiment took about 20 minutes, and each participant was required to press the right-arrow key on the keyboard when he or she sees a target T among randomly shown distractors L on the screen or to press the left-arrow key on the keyboard when he or she do not see a target T among randomly shown distractors L on the screen. The participants were required to press the appropriate key as quickly and accurately as possible. Participants could withdraw from the experiment at any time while performing the psychophysics experiment if they did not feel comfortable to continue, and their results were eventually withdrawn from the recorded database. All participants’ responses data were coded and no names were be divulged during this dissemination. A total number of 18 participants took part in this study of which 10 were males and 8 were females. The healthy participants comprised of medical students of different years of study (1st year to 6th year of their medical studies), and also research staff of the biophysics, physiology and biology department of Necmettin Erbakan University. Their ages range from 19 to 51 years old (24.8 ± 8.2).
Both reaction times (response times) as well as the responses of the participants were recorded. The responses of the participants were compared to the presence or absence of target in a particular screen. Moreover, the different set sizes of the number of distractors including target for each screen were recorded.
A Receiver Operating Characteristics (ROC) graph is a technique for visualizing, organizing and selecting classifiers based on their performance. ROC graphs have been employed in signal detection theory to depict the tradeoff between hit rates and false alarm rates of classifiers. ROC analysis has been extended for use in visualizing and also analyzing the behavior of diagnostic systems. One of the earliest adopters of ROC graphs in machine learning was Spackman (1989), who demonstrated the value of ROC curves in evaluating and comparing algorithms.
Classifier PerformanceA classification model or classifier is a mapping from instances to predicted classes. Given a classifier and an instance, there are four possible outcomes. If the instance is positive, and it is classified as positive, it is counted as a true positive; if it is classified as negative, it is counted as a false negative. If the instance is negative and it is classified as negative, it is counted as true negative; if it is classified as positive, it is counted as a false positive. Given a classifier and a set of instances (the test set), the following terms are defined [
The true positive rate (also called the hit rate and recall) is estimated as:
1) Tp-rate = positives correctly classified/total positives
The false positive rate (also called the false alarm rate) of the classifier is:
2) Fp-rate = Negatives incorrectly classified/Total negatives
Other terms derived from these two rates are:
3) Specificity = 1 − fp rate
Specificity measures the proportion of negatives that are correctly identified.
4) Sensitivity = tp rate
Sensitivity measures the proportion of positives that are correctly identified.
In this research, the Receiver Operator Characteristics (ROC) analysis was applied to the reaction times of the participants that were affected by the different image size and image intensity as well as the number of distractors on a particular screen. In this psychophysics experiment, for each participant, the target was present at 50% of the screen presentations in order to observe how well the reaction times could be used to predict the two different classes of presence of target on a screen and the absence of target on a particular screen. The Area under Curve (AUC) of an ROC curve will pinpoint the best test image(s) for this particular visual task in terms of specificity and sensitivity.
There is normally a negative relationship between reaction times and accuracy of responses. It means that the faster one reacts in a particular task, the worse is one’s performance (accuracy). The confusion matrix summarizes the different types of true positives, false positives, true negatives as well as false negative that might occur during an experiment (
| Target present (P) | Target absent (N) | |
|---|---|---|
| Response (Yes) | True positives | False positives |
| Response (No) | False negatives | True negatives |
Based on the SATF function, low constant d’ for low reaction times mean the Task is too difficult and if high constant d’ for high reaction time, this means performance saturates.
A detailed summary of the results produced from the ROC analysis, d’, the mean reaction times of the participants for each test image as well as the corresponding responses’ accuracy are shown in
SATF analysis was conducted on the test image Size35Int255 based on its high d’ and highest AUC values to observe the change in sensitivity with increasing reaction times of the participants (see
| Test images | Area under curve from ROC analysis | Dprime [d’ = z(H) − z(F)] | Mean reaction time (s) | Percentage accuracy (%) |
|---|---|---|---|---|
| Size40Int255 | 0.7810 | 1.683 | 1.642 | 90.0 |
| Size40Int240 | 0.8574 | 3.002 | 1.984 | 96.1 |
| Size40Int230 | 0.8441 | 3.002 | 1.964 | 96.7 |
| Size40Int220 | 0.8430 | 3.184 | 1.842 | 97.2 |
| Size35Int255 | 0.8631 | 3.403 | 1.894 | 96.1 |
| Size35Int240 | 0.8220 | 2.840 | 1.922 | 96.1 |
| Size35Int230 | 0.7996 | 3.667 | 2.090 | 98.3 |
| Size35Int220 | 0.8344 | 3.186 | 1.962 | 96.7 |
| Size30Int255 | 0.8491 | 3.002 | 2.059 | 95.6 |
| Size30Int240 | 0.8321 | 2.328 | 1.989 | 92.7 |
| Size30Int230 | 0.8094 | 3.668 | 2.019 | 98.3 |
| Size30Int220 | 0.8275 | 2.693 | 2.012 | 95.6 |
| Size25Int255 | 0.7775 | 2.327 | 2.292 | 93.3 |
| Size25Int240 | 0.8270 | 2.441 | 2.241 | 91.7 |
| Size25Int230 | 0.7967 | 2.441 | 2.244 | 94.4 |
| Size25Int220 | 0.8311 | 2.121 | 2.084 | 92.7 |
ROC analysis proved to be a useful tool in distinguishing the effect of the different types of images (distractors and targets) on the subjects’ reaction times. The highest value of AUC was the test image with size of 35 × 35 pixels and with an image intensity of 255 and it represented the best test image in terms of specificity and sensitivity. Moreover, the results based on the d-prime alone showed that the test image Size35Int255 ranked among the first three best images to represent a readily detected visual stimulus. Speed-accuracy trade-off function analysis on the test image Size35Int255 showed that the sensitivity increases sharply between the reaction times of 0.8 to 1.1 seconds, then sensitivity rises gradually until 2.8 seconds and then sensitivity remains constant beyond this point which insinuates that performance saturates at this point onward. The ongoing analysis is also focusing on the effect of a varied number of targets and distractors on the participants’ performance.
This research work is supported by The Scientific and Technological Research Council of Turkey (TÜBİTAK) and the Marie Curie Action Cofund of the 7th Framework Program (FP7) for me to pursue my postdoctoral research work. I would like to sincerely thank the medical students as well as the various research staffs from the Biophysics, Biology and Physiology Departments of the Necmettin Erbakan University for their precious participation as well as for helping me in the organization (scheduling and translation) process of this psychophysics experiment. My sincere thanks go also to my mentors Assoc. Professor Barkin Ilhan for his fruitful advice and guidance about performance algorithms, and the concept of the rotation of the targets and distractors, and Prof. Nizamettin Dalkılıç about the support of this work.
The authors declare no conflicts of interest regarding the publication of this paper.
Chuckravanen, D., Ilhan, B. and Dalkılıç, N. (2021) ROC and SAT Analysis of Different Grayscale Test Images (Distractors L and Target T) to Customize a Visual-Search Attention Task. Open Journal of Biophysics, 11, 407-414. https://doi.org/10.4236/ojbiphy.2021.114017