Travel Theory

Travel therapy is not a particularly popular type of intervention, but neither is it particularly explored. Quick research on the topic shows that travel is used for relationship therapy [10] [11], but there is little to no literature on the uses of travel therapy for neurological disorders such as AD other than indoor traveling [12] [13]. We found only one exception; the study carried out by Biamonti and colleagues [14].

In their study, researchers explored the psychological and behavioral impacts that virtual travel could have on AD patients in two contexts: the first being a mediocre replicate of a physical train, and the second being a more realistic physical train replicate. Results of the study showed that virtual travel by means of a computerized train increased well-being in individuals with AD. More precisely, participants experienced positive changes in wandering, agitation, anxiety, apathy and sleep [14]. These improvements in psychological and behavioral states are valuable, as these are typical symptoms of AD which significantly diminish the quality of life of patients and even caretakers.

We believe that virtual travel influences behavior and psychological states by first influencing emotional states. Indeed, individuals with AD often require specialized care which generally causes distance between patients and their loved ones. In such situations, feelings of loneliness and isolation are not uncommon. It is well known that loneliness can have negative effects on individuals [15] [16] [17]. Through virtual travel, participants can break the cycle of loneliness and isolation. We believe that virtual travel positively influences emotional states of participants which in turn promote well-being.

Music Theory

Music as a therapeutic intervention is commonly used in different health conditions, such as autism, brain injury, depression, anxiety and dementia [18] [19] [20] [21]. It has repeatedly shown to be effective in improving psychological and behavioral states.

Effects of music therapy (MT) on psychological state include decreased stress, anxiety and depression, improved emotional states and relationships with caregivers [22] [23] [24] [25]. Studies often report that the participants seem generally happier following the MT intervention, suggesting an increased well-being [26]. Behavioral changes following these interventions include decreased agitation and wandering, two common and potentially dangerous symptoms of the disease. Finally, MT can also be beneficial on a cognitive level. Studies show slowed cognitive decline and improved orientation, memory and other cognitive functions in patients following a MT session [22] [27] [28] [29].

Although the mechanisms by which MT generates positive changes in well-being and potential neurological rehabilitation are not clear, its effects are unmistakable. We believe that some of these effects can be attributable to music’s capacity to easily influence emotional states. By increasing positive emotions, we believe that participants become in a favorable cognitive environment.

Savannah Theory

The term savannah refers here to open environments with short grass, deciduous green trees, water, and animal life. Ancient survival challenges led to a correlation between natural landscapes and positive feelings. Savannah as a natural environment brings back the feeling of tranquility and peace [30].

Spreading trees in savannah gives a feeling of security because they provided early humans with a place to observe or hide from predators. The savannah is an ideal place to travel and explore, it attracts attention and relaxes thanks to its varied and pleasant landscapes.

Wilson introduced the world biophilia in 1984, referring to an innate affinity that people have with nature [31]. Savannah landscapes are preferred by modern humans since the first humans lived for thousands of years in the African savannah [32].

In addition, Meidenbauer K. L. et al. [33] demonstrated in their research that adults have aesthetic preferences for natural environments over urban ones which influenced theories like Biophilia to explain why nature is beneficial for mood improvement. They showed that children demonstrated preferences for urban over natural environments and those urban preferences decreased significantly with age. Both adults and children showed cognitive and affective benefits after nature exposure.

We believe that savannah and natural environments are appreciated and help relax AD patients which will increase positive emotions and decrease negative emotions.

The main characteristics of VR are immersion, sense of presence, and interactivity [34]. These characteristics can be very beneficial and thus VR is rapidly gaining interest in many fields. Biocca et al. defined it as a sum of hardware and software systems that attempt to generate a sensory illusion of being present in another environment different from the one where the user is present physically [35].

VR is used as a therapeutic tool in psychology and psychiatry. Dana et al. [36] have used VR to create virtual environments as treatments for certain types of phobias. They created objects, environments and situations simulating exposure therapy which helps patients face their fear. Moreover, Lorenzo G. et al. [37] have used VR systems with children diagnosed with autism spectrum disorders in order to enhance their emotional skills.

VR interventions are also used to treat pain. Tennant M. et al. [38] investigated the treatment efficacy difference between immersive VR and an iPad on oncology patients’ physical and emotional states. Participants who had used VR reported greater improvements in their mood states and reductions in their negative symptoms. The preliminary results of this study support the use of VR in clinical oncology in order to improve patient well-being.

Moreover, many studies revealed that VR is beneficial with AD patients. In fact, some studies have indicated that VR intervention could improve cognitive functions of individuals with mild cognitive impairment (MCI) or AD [39] [40].

Adaptative systems are used in many fields to provide personalized experiences for each user. There are many examples of such instances. For instance, in the learning domain, Chaouachi et al. [41] used adaptive systems in order to maintain users in a positive mental state while learning. In the video games domain, Ghali et al. [42] used adaptive systems to improve players’ intuitive reasoning in a physics VR game by changing assistance strategies according to players’ emotions.

Moreover, many researchers use intelligent agents in order to adapt their systems. For example, Dayong Ye et al. [43] used an agent-based approach in order to adapt their system to alleviate the runtime quality management problem of distributed service-based systems. They claim that multi-agent systems are ideal to model service-based systems and then solve service adaptation problems. Also, Spencer Rugaber et al. [44] used intelligent agents to create a model-based technique for self-adaptation in game-playing agents. Additionally, Ben Abdessalem and Frasson [45] used intelligent agents to adapt video games to players according to their cognitive and emotional states. In their work, they proposed to track and adapt in real-time the parameters of the video game according to players emotions. Similarly, Peña et al. [46] used intelligent agents to adapt their system to learners. The adaptation technique they used is focused on the didactic contents, navigation tools and navigation strategies of students’ learning styles. Likewise, Doctor et al. [47] used intelligent agents to monitor dementia patients living in their own homes. Adaptation rules are applied to facilitate short-term tuning of learned behaviors and long-term tracking of behavioral changes.

Physiological sensing approaches are used in several studies in the brain assessment field. Electroencephalography (EEG) is one of these approaches which is used to detect and analyze emotional states. Since EEG is non-invasive and inexpensive, it is a simple and effective way to detect, analyze and evaluate emotions and mental states. In fact, researchers can analyze the brain’s electrical activity by using EEG in order to understand the brain’s inner working. Several works in different fields relied on EEG in the brain assessment [45] [48].

In the field of serious games, Ghali et al. [42] assessed participants' mental states by considering their engagement and frustration in a physics-related game using EEG. They have proposed to improve intuitive reasoning through help strategies based on these two emotional states.

Furthermore, in the field of learning, studies realized has shown that students’ learning skills are improved when they are in a positive mental state. Considering this, Chaouachi et al. [41] developed a system that aims to help students in their learning by keeping them in a positive mental state. The study was based on students’ engagement and workload extracted from EEG signals.

In order to assess emotions, some researchers used EEG data. There is a study, in the field of emotions’ detection, designed to detect participants’ valence (positive or negative emotion) in real-time while they were invited to watch different videos [49].

Considering the related works presented in this section, we propose the use of VR to create relaxing immersive environments as well as the use of Intelligent Agents with EEG as input in order to adapt these environments and optimize their effects to each person.

As mentioned in Section 2.1, Music Therapy (MT) can positively influence psychological states and can help reduce negative emotions. Some studies have shown that VR can be beneficial for individuals suffering from SCD in addition to increasing its positive effect by isolating the participants from external distraction. With this in mind, we designed a Virtual MT therapeutic environment. This environment consists of a music theater in which the patient is immersed, placed in the center and facing the stage up front (see Figure 4).

The environment presents a series of eight songs in the following order: Ave Maria by Franz Schubert, Eine Klein Nachtumusik by Mozart, Clair De Lune by Debussy, La Vie En Rose by Edith Piaf, Everyday by Buddy Holly, La Bamba by Ritchie Valens, and finally What A Wonderful World by Louis Armstrong. These songs were meticulously selected based on music and psychology theories. For example, some songs were chosen due to their structural features; slow tempos and low pitches are generally associated with anxiety reduction [50]. Other songs were selected because they had been popular during years coinciding with the participant’s reminiscence bump; the interval between ages 10 and 30 where memories are generally more remembered [51].

A 30 seconds portion of the songs were chosen because they contained melodies with structural features associated with anxiety reduction. Each song is separated from the previous by red curtains which open and close as different songs are presented on stage. For each song, an appropriate selection of instruments is presented on stage, each instrument is animated with the music.

In addition, visual effects are presented above the instruments and the intensity of the light as well as their colors were chosen as a function of the emotional purpose of the song. Scenes were designed with specific shades of red, purple, blue, green and yellow, as they are suggested to be the most pleasant and arousing hues [52]. Figure 4 illustrates two songs with their respective selection of instruments and visual effects.

Combining VR with MT could lead to a promising therapy; however, a static environment could have different impacts for each participant. Indeed, the intended effect of this environment is to increase emotional wellbeing. Consequently, it is important to adapt the environment to each participant in order to provide the most beneficial therapeutic experience for each individual.

In order to adapt the virtual MT environment to each participant, it is important to track and analyze the emotional impact of each song in real-time. Consequently, we created a Music Adaptation Agent able to record and analyze the users’ emotions using Emotiv electroencephalograph (EEG) headset data in real-time. This agent tracks the emotions of the participants while they listen to the eight songs, analyzes the emotional impact of each song while they are playing, and calculates a score for each song according to emotion recordings. Subsequently, based on the scores, the agent detects the song which provokes the best emotional impact and adapts the virtual MT in such a way as to play this song once again for a longer period of time.

As mentioned in Section 2.1, savannahs bring the feeling of tranquility and peace, especially for older adults suffering from SCD, which can help relax and reduce negative emotions. Furthermore, VR in beneficial also for SCD patients and it increases the positive effect of savannah, hence we propose to combine them to create Savannah VR [53].

Savannah VR is a therapeutic environment designed to relax SCD patients. In this environment, participants follow an avatar walking through a savannah. This environment requires only a virtual reality headset to control of the view angle. The dominant colors are warm, the animals are calm and their movement is slow. A soothing piano tune is played in the background at a volume low enough to appreciate the sound of each animal. Figure 5 illustrates screen captures of the environment.

Participants follow a gazelle that moves along a path with breaking points. The gazelle moves slowly to avoid any possible nausea caused by movements within a virtual reality environment. Throughout the experience, the gazelle

remains in front of the participant in such a way as to imitate a third-person view, which is less likely to cause motion sickness [54]. At any moment, participants can look around in 360 degrees by rotating their heads.

Savannah VR could be more beneficial if we made it a dynamic environment which could be changed and adapted to each participant. Integrating an agent capable of adapting the environment could provide an optimized therapeutic experience to each individual.

In order to make the environment adaptable for each participant, we implemented functions that change the environment’s parameters in real-time. For instance, one of the adjustable parameters is the color and intensity of the light, since light influences perception and decision-making [55].

In order to personalize the environment, we propose to use a Neural Agent [45] able to modify the parameters of Savannah VR based on intervention rules triggering a biofeedback approach and more precisely a neurofeedback. Neurofeedback is a type of biofeedback that measures brain waves to produce a signal that can be used as feedback [56].

The Neural Agent consists of an intelligent agent which tracks the emotions of the participants and intervenes on Savannah VR accordingly. This agent has an intervention rules base which possesses multiple intervention rules. Giving the main emotion of the participant, the agent chooses a rule from this rules base and modifies the VR environment to adapt it to the participants. After each intervention, the agent tracks the emotional reactions of the participant to check whether the optimization of participants’ emotion was a failure or a success and, if needed, intervenes again on the VR environment.

The Neural Agent is composed of two modules: “Analysis, decision and action” and “Rules base Manager” as well as two databases: “Decisions base” and “Rules base”. Figure 6 illustrates the architecture of the Neural Agent and its interactions with the EEG module and Savannah VR. This architecture allows real-time interaction with the user in a neurofeedback loop.

The Analysis, Decision and Action module has four components which handle reception of data (emotions and information about the status of the environment) and data analysis in order to identify the main emotion and detect whether it corresponds to the expected emotion generated by the environment or not. After that, the module consults the rules base to extract the rule that can be applied and send the action to the environment and stores the decision in the “Decisions base”.

After each intervention, the agent observes the emotional reactions of the participant to check whether or not it does modify the emotional state as desired, and updates the weight of the intervention rule accordingly (increase the weight if it worked and decrease it if it doesn’t).

The rules base manager module allows for the management of the intervention rules. The three main functionalities are adding, modifying, and deleting of an intervention rule. This module stores the rules in the “Rules base”.

Exercise 1.

In this exercise, we present a series of 5 numbers in auditory form, then a numerical pad is displayed, and the participant is asked to repeat the same sequence; then we hide the pad and present another sound sequence of 3 numbers and the participant is asked to report the sequence in the reverse order.

Exercise 2.

In the second attention exercise, we present a sound sequence of different letters at a rate of one per second and the participant must click the space bar each time they hear the letter “A”.

Exercise 3.

The third attention exercise is about naming different objects. For every object, we show an image for 4 seconds, hide it, and then show four letters. Then, the participant is asked to select the first letter of the object’s name. Figure 8 illustrates how the participants see an image and then four letters as a suggestion.

Exercise 4.

In this exercise the participants are asked to memorize a series of different objects visually or aurally. Then, we present a series of objects and the participants are asked to select whether they saw the object, heard its name, or whether it was never presented. Figure 9 illustrates a case where the participant correctly answered that they saw the image of the dog.

Exercise 5.

In the second memory exercise, we present ten circles, and then a series of circles are highlighted one by one in order to create a sequence. The participants are then asked to reproduce the same sequence. We present two sequences with two different levels of difficulty.

Exercise 6.

In the third and final memory exercise, we present a set of three pictures for a short period of time. Then, we present four sets of three pictures and the participant is asked to identify which set was presented. Figure 10 shows how we present the pictures and a case where the participant chooses the correct answer.

In order to track and analyze participants’ emotions, we used Emotiv Epoc+ EEG headset technology. The headset contains 14 electrodes spatially organized according to the International 10 - 20 system, moist with a saline solution. The electrodes are placed at antero-frontal (AF3, AF4, F3, F4, F7, F8), fronto-central (FC5, FC6), parietal (P7, P8), temporal (T7, T8) and occipital (O1, O2) regions with two additional reference sensors placed behind the ears. The detailed position of the measured regions is shown in Figure 11.

The system uses internal algorithms to measure the following mental states: meditation, frustration, engagement, excitement and valence. They were used to assess the effect of the Virtual Reality Train, the Adaptive Music Therapy and Intelligent Savannah Therapy on the emotions of participants and were used as input for the Music Adaptation Agent and The Neural Agent.

Even though we don’t have access to Emotiv system proprietary algorithms to infer these mental states, several studies have established the reliability of the output.

In order to verify our first hypothesis that relaxing environments reduce negative emotions, we proceeded by analyzing the participants’ negative emotions before and after the VR Train. Results show that the mean frustration of participants before the VR Train was 0.71 (minimum 0.41 and maximum 0.96). The participants’ mean frustration during the VR Train was 0.51 (minimum 0.24 and maximum 0.94). After the VR Train, the mean frustration was 0.53 (minimum 0.17 and maximum 0.79). Figure 12 shows a boxplot of the mean frustration before, during and after the travel in the VR Train.

In addition, we conducted a repeated measure ANOVA with participants’ frustration level as dependent variable. Results revealed a significant decrease of the frustration during the VR Train and the effect was still observed after the VR Train immersion (p = 0.0011 and F = 7.7218). Table 1 details the ANOVA results and shows that the mean frustration level decreased from 0.705 to 0.513 and kept the positive effect after the VR train by increasing only to 0.534.

Thus, the frustration decreased when the participants were in the VR Train and the positive effect on frustration was preserved following the immersion.

In order to verify our second hypothesis which is: relaxing environments improve attention and memory performance, we started by comparing the performance improvement of the attention and memory exercises by calculating the average mean performance of all participants before and after the VR train. Figure 13 shows the obtained results in which we can notice a clear difference between the improvement of performance on the attention exercises versus the memory exercises.

These results show a clear increase in memory performance following the VR Train and in some cases an improvement in attention abilities.

In order to have more detailed results, we also compared the participants’ performances before and after the VR train for each exercise. The mean improvement results of each exercise are shown in Table 2.

The biggest performance improvement is observed in exercise 5 and corresponds to a mean 20% improvement. In addition, we performed a paired t-test statistical test with participants’ scores for exercise 5 before and after the VR Train as parameters. The obtained result is t = 2.293 and p = 0.017, this result is significant at p < 0.05.

For the second system, we also analyzed the negative emotions before and after Adaptive Music Therapy. We analyzed the level of frustration of the participants and results show that the mean frustration before AMT was 0.691 (minimum 0.377 and maximum 1.0). Their level of frustration during AMT was 0.457 (minimum 0.213 and maximum 0.824). After AMT, their level of frustration was 0.517 (minimum 0.238 and maximum 0.927). Figure 14 shows a boxplot of the mean frustration before, during and after the Adaptive Music Therapy immersion.

In addition, we conducted a repeated measure ANOVA with participants’ frustration level as dependent variable. Results revealed a decrease of the frustration during AMT and the effect was still observed after the immersion (p = 0.0011 and F = 8.26316). Table 3 details the ANOVA results and shows that the mean frustration level decreased from 0.691 to 0.457, and that the positive effect was kept after AMT and increased only to 0.517.

In order to verify our second hypothesis for the second system, we compared the improvement of attention and memory exercises by calculating the average mean performance of all participants before and after AMT. Figure 15 shows the obtained results.

These results show a difference between the improvements of performance on the attention exercises versus the memory exercises. We notice a clear increase of memory performance after AMT for most participants and an increase in attention performances for nearly half the participants.

We also compared the performance of the participants before and after AMT for each exercise. The mean improvement results of each exercise are shown in Table 4.

The most noticeable improvement is observed in exercise 6, with a mean improvement of 36.84%. In addition, we performed a paired t-test statistical test with participants’ scores for exercise 6 before and after the AMT as parameters. The obtained result is t = 3.440 and p = 0.0029, this result is significant at p < 0.05.

As we mentioned is Section 5.2, the Music Adaptation Agent intelligently adapts the environment by choosing the most efficient song as a function of the participant’s emotional response. Therefore, we analyzed the mean frustration during AMT before and after the agent’s adaptation. Results showed that the general mean frustration for all participants, before the agent’s adaptation, was 0.48 (minimum 0.29 and maximum 0.84) and dropped to 0.41 (minimum 0.18 and maximum 0.77) after the agent’s adaptation. Individual results are shown in Figure 16.

We can observe a frustration decrease after the adaptation for most participants, except for two; only participants 5 and 16 failed to show the effect.

In order to further analyze and support these results, we conducted a t-test for 2 dependent means where the parameters were the mean frustration before and after the adaptation. The obtained results are t = −2.2082 and p = 0.040. This result is significant at p < 0.05.

These results support our third hypothesis which is: intelligent agents can adapt relaxing environments in order to optimize its effect, and shows that it is possible to intelligently optimize the environment in order to reduce negative emotions.

In addition to the results obtained in the previous section, and in order to support our first hypothesis, we analyzed the negative emotions before and after the Intelligent Savannah Therapy. We analyzed the participants’ level of frustration and results show that the mean frustration of participants before IST was 0.68, (minimum 0.24 and maximum 0.98). The participants’ mean frustration during IST was 0.57 (minimum 0.31 and maximum 0.88). After IST, the mean frustration was 0.55 (minimum 0.28 and maximum 0.91). Figure 17 shows a boxplot of the mean frustration before, during and after IST.

In addition, we conducted a repeated measure ANOVA with participants’ frustration level as dependent variable. Results revealed a decrease of the frustration during IST and the effect was still observed after the immersion (p = 0.0345 and F = 3.7018). Table 5 details the ANOVA results and shows that the mean frustration level decreased from 0.685 to 0.579 and further decreased after IST to 0.555.

In order to support the results obtained in Sections 8.1 and 8.2 for our second hypothesis, we compared the improvement of attention and memory exercises by calculating the average mean performance of all participants before and after IST. Figure 18 shows the obtained results.

These results show a clear increase of memory performance after IST for most participants (17) and an increase in attention performance for almost half the participants (8).

We also compared the performance of the participants before and after IST for each exercise. The mean improvement results of each exercise are shown in Table 6.

The largest improvement is observed in exercise 6 by 34.21%. In addition, we performed a paired t-test statistical test with participants score of exercise 6 before and after the IST as parameters. The obtained result is t = 3.9804 and p = 0.0008. This result is significant at p < 0.05.

The performance improvement for exercise 5 was also considerable, with a mean of 16.84%. This led us to perform another paired t-test with participants score of exercise 5 before and after the IST as parameters. The obtained result is t = 2.9682 and p = 0.0082, which is significant at p < 0.05.

As mentioned, in Section 6.2, the Neural Agent adapts the environment to personalize its effect to each participant. Thus, in order to confirm our third hypothesis and support the results found in Section 8.2, we analyzed the mean frustration level of each participant before and after the agent’s adaptation and personalization of the environment. The Neural Agent intervenes in the

environment and then checks whether its changes modify the participant’s emotion in the expected way. We analyzed the mean frustration before and after each time the agent intervened and mark the intervention as a success.

Results show that the mean frustration for all participants before the agent’s successful interventions was 0.57 (minimum 0.31 and maximum 0.88) and dropped to 0.53 (minimum 0.28 and maximum 0.81) after the agent’s adaptation. Individual results are shown in Figure 19.

We can observe a frustration decrease after the adaptation for most participants except for two. Only participants 5 and 8 failed to show the effect.

In order to support these results, we conducted a t-test for 2 dependent means where the parameters were the mean frustration before and after the agent’s intervention. The obtained results are t = −4.5815 and p = 0.0002, which is significant at p < 0.05. These results confirm that we can intelligently adapt the relaxing environment in order to optimize its effect.

The results presented in the previous subsection support our first hypothesis and show that our three relaxing systems successfully reduce participants’ frustration. This effect varies with the level of intelligence of the system. In Table 7, we present the participants’ mean frustration before, during and after the three relaxing systems. We note that for our three systems, the frustration drops during all systems, but the effect of relaxation stays longer with more adaptation and more intelligence. The system with the Neural Agent is the only one which makes the effect of relaxation persist longer even after the immersion and does not exceed the initial level of frustration. These results support our third hypothesis and show that intelligent agents can optimize the effect of relaxing systems.

In addition, the results presented in the previous subsection, support our second hypothesis, and show that our three systems improve participants’ memory performance and in some cases attention performance too. In Table 8, we present a comparison of the mean performance improvement for memory exercises. We note that for our three systems, the memory performance increases, but the level of improvement is more important when the system is more intelligent. The largest mean improvement is observed in the system with the Neural Agent. This therefore supports our third hypothesis shows that intelligent agents can optimize the effect of our therapeutic systems.

The results presented in the previous section show that our approach has a great potential to reduce negative emotions and improve memory performances. However, our work has some limitations; First, we could not perform experiments on more than 19 participants per system. Future lab research will require additional experiments in order to verify the effect on our environments is a larger scale and consequently confirm the obtained effect. Second, our approach requires the use of two headsets (EEG and VR headsets) in order to collect experiments’ data and some participants were not very comfortable using both headsets for a long period of time. This could explain the decrease of performance of some participants.

In terms of the environments themselves, each participant had a different preference. Although most of the participants appreciated the environments, which led to a general positive effect detailed in the previous section, their effects were not unanimous, as some participants did not benefit from the environment’s therapeutic effect as much as others. For example, in Train VR system, some participants experienced motion sickness problems. For AMT, some songs could have triggered the recollection of negative memories, ultimately negatively influencing their emotional state. Finally, for IST, some participants did not enjoy or were uncomfortable in the presence of animals. By influencing emotional states, all of these factors could have contributed to a decrease in the participants’ performance.

In addition, we also note that this approach would not be feasible for blind individuals since the VR headset is required to the immersive effect. However, two of our environments, namely Train VR and IST, could work for deaf individuals, but the result could be different due to the absence of relaxing background music for Train VR and Volume/Music controlled by the agent in IST.