The developed UI for sensory impaired users was based on the most common approach for designing inclusive interfaces - multimodality. According to research, the optimal combination for persons with reduced vision and/or hearing is to use visual elements combined with auditory messages and tactile cues (female voice is preferred) [17] . Therefore, a visual interface was designed as an app for in-vehicle communication containing not only screen options, but also an option for voice control with auditory (natural language) feedback. In addition, HUD messages were designed as a combination of notification icons and sounds intended to be used as additional features to grab the passenger’s attention. General and specific guidelines for designing the visual and sound interface elements for persons with reduced vision and hearing were combined to generate the final solution [18] .

The final interface was designed using Adobe XD. Initially, three variants were generated which underwent a first simulation that provided important feedback on how to optimize the final UI version [19] . The optimized UI version was then used in the driving simulator. The final design consisted of a simple wireframe containing a header, menu bar and options section which allowed the most important content to be positioned centrally on the screen. The navigation items were not designed to be constantly displayed on the screen in order to free space for larger buttons and text. Excessive text was avoided and replaced with universal, filled icons. The UI was designed with a high “AAA” contrast between the icons/text and the background. The high level of contrast “AAA” means the used contrast ratio is 7:1 to allow maximum inclusivity (the minimum required level of contrast ratio is 3:1) [20] . In addition, pre-determined choice options were included as selection buttons in order to simplify the use of the available options. Samples of the designed UI are provided in Section 4.2, Table 1 .

In total, 32 participants (4 females), from 24 to 52 years old (Мage = 31.75, SD = 6.46) without reported sensory impairment, were involved in the experiment. To simulate a vision acuity loss, participants were given glasses equipped with specially designed lenses to induce a blurred vision (Inclusive Design Toolkit, University of Cambridge) ( Figure 1(a) ). The study participants were given three pairs of the simulation glasses, which made their vision about 0.49 logMAR worse [21] . To imitate the reduced hearing capacity, the audio files of the user interface that represented the virtual in-vehicle assistant were processed with a filter based on the approximate hearing curve an 80-year-old person with typical age-related sensor neuronal hearing loss [22] [23] .

To recreate the ride in an AV a driving simulator was used. It consists of a section of a vehicle cabin (including front seats, cockpit, dashboard, windscreen and mirrors, steering wheels and pedals, spatial audio) with a cylindrical projection screen (5m diameter) in front of it ( Figure 1(b) ). The use of the simulator helped for evaluation of the designed UI and was chosen because previous research has shown it is a valid tool for investigating human factors in the evaluation of in-vehicle systems [24] .

A portable device (10-inch) tablet was included in the simulator, through which the developed UI (app) was presented to the participants ( Figure 1(b) and Figure 2 ). Two apps were used in this study - one being the inclusive variant and the other one being a regular interface designed with no inclusive features. The voice messages were implemented only in the inclusive UI as an additional accessibility feature. The two UI variants were included in order to capture the difference between using inclusive features and the possible benefits of its use for persons with sensory impairments. Differences between the standard and inclusive UI are given in Table 1 . The screens in front of the cabin were also used as a virtual HUD displaying additional information regarding the ride and notification messages ( Figure 2 ).

The experimental design consisted of a 4x2 factorial design with two independent variables: “simulated sensory condition” and “UI design”. The sensory condition was treated as a between-subject factor, with 4 levels: normal vision and hearing, normal vision with hearing impairment, impaired vision with normal hearing, impaired vision and hearing. The UI design was treated as a within-subject factor, with two levels: standard graphics, inclusive graphics. This design resulted in 8 experimental conditions, as described in Table 2 . Each participant was randomly assigned to one of the four groups defined by the sensory conditions, and had to interact with both UI designs.

Before beginning the experiment, each participant was provided with instructions and asked to fill out an informed consent form. An internal review board had previously approved the ethical aspects of the study. Our study adheres to the guidelines of the Declaration of Helsinki. The ethical aspects of the study were reviewed and approved by Virtual Vehicle Research IRB with protocol n. 22001. The participants were asked to imagine a situation in which they are a person with reduced visual and/or hearing acuity, i.e., the ability to discriminate details, on an AV for shared rides (an explanation was provided to them regarding what an AV is). Participants from the vision-impairment and Vision & hearing - impairment group were provided with the simulation glasses and were asked to keep them on during the whole experiment. Participants from the Hearing-impairment group and Vision & hearing - impairment group were also explained that in order to simulate their hearing acuity loss they will be listening to modified sound files during the whole experiment. Finally, they were required to enter the simulator and take a seat.

Participants were encouraged to use the UI displayed on the tablet in the cabin and the options of the interface which included: navigation, comfort settings, seat adjustment, climate control, light adjustments, and multimedia for listening to music. Additional information regarding the options were not provided in the participants’ instructions in order to determine if despite their “sensory impairment” they can understand and recognize all the information without having previous knowledge about them. It was emphasized to the participants that to achieve the goal it is very important for them to explore and use all the available features of the interface to get a good impression of it. They were encouraged to use the clickable (i.e., touch) buttons and, if possible, use the voice control option. They were also informed that messages are to appear on the screens in front of them as if they were projected onto the windshield to simulate a HUD. This approach was used to allow a free and spontaneous exploration of the UI functions by the participants and trigger realistic, natural reactions throughout the experiment. The choice was to exclude specific task-related performance measures like, e.g., completion time, since the main goal was to receive feedback about the overall user satisfaction in self-initiated interactions with the interface.

After entering the simulator, the participants were guided by the UI provided on the portable device ((10-inch) tablet) in front of them. Two rides were performed per participant and each lasted about 8 minutes. The route and scenario were the same for all groups and rides. The simulation began after every participant “started the ride” through the UI and the driving environment appeared on the screens in front of the cabin ( Figure 2 ).

The participants’ opinions, impressions and experiences during and after each simulation were collected through 1) a 3-part questionnaire and 2) an interview. The questionnaire consisted of 3 different Likert scales:

After the drive, participants exited the simulator and were interviewed in order to understand their impressions during the whole interaction with the UI in greater qualitative detail. An experimenter annotated their comments.

The collected results are presented and discussed in the following section.

The collected data from the first two sets of Likert questions (“Checklist for trust between people and automation” and “System Usability Scale”) were analyzed according to the proposed methods from the relevant sources stated above. Both scales provided a model for assessing trust and usability of the UI based on statements with both a positive and negative tone that measure polar opposites along a single dimension. The results are shown in Figure 3 and Figure 4 .

Based on the charts, we can see that the inclusive UI received the highest score for both trust and usability from the “vision-impairment group” compared to all other groups with simulated sensory impairment. For the “Vision-impairment group”, the “Checklist for trust between people and automation” score increased from 16 (while using the non-inclusive, standard version) to 20.125 (while using the inclusive version), which is the highest score compared to all other groups. The SUS score for this group increased from 60.625/100 (non-inclusive) to 69.6875/100 (inclusive). This SUS score was second-highest after the SUS score of the non-inclusive UI from the “No-impairment group” (71.5625). Since the non-inclusive UI for the “No-impairment group” equals a standard UI for a standard user, i.e., a reference score (or baseline), the fact that the inclusive UI scores very near that value with visually impaired users indicates a great achievement, that almost entirely compensates for the loss of usability induced by the sensory impairment.

The “No-impairment group” preferred the non-inclusive UI since they rated it as more usable, based on the SUS score, which was expected. However, it was

interesting to find out that the participants from the “No-impairment group” felt they trust the autonomous system slightly more during the ride when interacting with the inclusive graphics. The regular UI did score higher on the SUS, but the results from the checklist for trust showed an opposite result. This means that even though the non-inclusive UI did seem more appealing and useable for this category of users that had no sensory disabilities, they rated the inclusive graphics as more trustworthy.

The “Hearing-impairment group”, interestingly, showed a preference of the standard UI once more. The standard UI with no voice messages received a higher SUS and trust between people and automation score.

The results from the “Vision & hearing - impairment group” showed more usability and trust of the inclusive variant, but the data was not as evident as it was with the “Vision-impairment group”. For example, the SUS score, in this case, increased from 58.125/100 (non-inclusive) to 63.75/100 (inclusive), but this usability score (in the inclusive case) is lower in comparison to the total SUS score of nearly 70/100 (inclusive) calculated from the answers of the participants in the “Vision-impairment group”.

Data collected from the additional Likert questions are shown in Figure 5 to Figure 10 , with statistical parameters (minimum, lower quartile, median, upper quartile, and maximum). Data was analyzed by a two-way repeated measures ANOVA to compare means between groups, interfaces and their interactions, and post-hoc multiple comparison Tukey tests were used to compare the means of observations between interfaces from the same impairment groups.

The main result is that the inclusive UI received the highest usability rate from the “Vision-impairment group”.

For the other impairment groups there was no significant improvement in the usability when interacting with the inclusive UI. The mean scores to the question “How well were you able to see and select the graphics?” regarding the inclusive UI were 4 (“Hearing-impairment group”) and 3 (“Vision and hearing impairment group”) showing no improvement compared to using the standard UI ( Figure 5 ). Different impairment groups showed no significant difference between interfaces, F = 1.65, p > 0.05 (p = 0.18). There was no significant difference between the two interfaces, F = 0.8, p > 0.05 (p = 0.37). There was also no interaction effect present, F = 1.13, p > 0.05 (p = 0.34).

However, based on the multiple comparisons test, (adjusted p value = 0.049) we can see a significant evidence that the UI with inclusive graphics were more useable to persons with a simulated vision impairment compared to the non-inclusive graphics. The result to the same question for the “Vision-impairment group” ( Figure 5 ) increased from 3.125/5 (non-inclusive) to 4.25/5 (inclusive). The mean score to this question was the highest in this group in comparison to all other groups with a simulated impairment ( Figure 5 ).

In addition, the feeling of safety during the ride was highest in the “Vision-impairment group” when interacting with the inclusive UI ( Figure 6 ). There was a significant difference between impairment groups (F = 28, p < 0.01). The overall feeling of safety during the ride while using the inclusive UI variant in this group was quite high – 4.875/5 ( Figure 6 ) which is another positive indicator that the inclusively designed UI is suitable for persons with impaired vision helping them feel safe while traveling.

Regarding the effectiveness of the HUD messages, the scores to the question “Were the messages that appeared outside the vehicle (on the big screen) attracting your attention?” were average (around 3) showing no major difference between groups and interfaces ( Figure 7 ).

The effectiveness of the HUD messages that appeared on the screen in front of the simulator received a mean score of around 3.5/5 from the “No-impairment group” for being good for attracting the passengers’ attention.

The effectiveness of the HUD messages decreased in the scenario where degraded audio messages were included (“Hearing-impairment group”) indicating that the participants with a simulated hearing impairment experienced a cognitive overload. While focusing on hearing the audio, they did not receive these pop-up messages as well as when they were shared with them only as graphical symbols on the screen. The result was similar with the group with a simulated vision and hearing impairment (“Vision & hearing impairment group”).

The HUD messages received a highest mean score from the “Vision-impairment group” ( Figure 7 ). The resulting score for the question “Were the messages that appeared outside the vehicle (on the big vertical screen) attracting your attention?” was 4/5, which is higher than the other cases.

The results related with the voice messages are given in Figure 8 and Figure 9 . The answers to the questions “Was the content of the voice messages understandable?” and “How easy was it for you to hear the voice messages?” are relatively low, especially from the groups where a hearing impairment was simulated.

The received mean score from the “Hearing-impairment group” to the question “Was the content of the voice messages understandable?” was 3.125/5 ( Figure 8 ) and to the question “How easy was it for you to hear the voice messages?” – 3.25/5 ( Figure 9 ). These results matched the obtained scores from the first two Likert-scales where this group showed a preference of the standard UI with no voice messages. Later, the answers from the interview questions provided the reasons for these results and explained that persons were annoyed by the voice messages because they were not able to hear them well – “acoustic was irritating because I did not understand properly”, “audio was bad, but I understand that I was in hearing impairment group”, etc. The effect was similar with the “Vision & hearing - impairment group”. Based on the previously analyzed answers from the “Hearing-impairment group” where it became obvious that some of the voice messages have a negative effect that decreases the usability of the interface and the feeling of trust, it was concluded that the voice messages have the same effect in this case as well. As a result, the overall feeling of safety and trust was low despite the inclusive design of the interface – 2.125/5 ( Figure 6 ).

On a positive note, the graphical design of the UI on the tablet received a high mean score of 4.5/5 for being useful to compensate the lack of information due to the hearing impairment ( Figure 10 ).

The participants’ comments from the interview session were very beneficial to pinpoint all advantages and disadvantages of the UI and gather ideas for further improvement and development.

Answers to the open-type questions indicated an overall positive impression of the inclusive UI. Most of the participants stated that the easiest tasks for them were to use the menu items due to the clear layout and interactive structure. They felt the design was familiar and therefore easy to use. Participants also liked that they can use all the options with only 1 - 2 clicks. Some of the participants from the “No-impairment group” and “Vision-impairment group” stated that they enjoyed the voice messages and the option to use voice control as an alternative to clicking the buttons because they found it practical. Participants with a with simulated vision impairment explained that the UI provided on the tablet was easy to use and the options were visible.

When participants were asked what was the hardest task and what they disliked about the UI the most common answers were related with the voice messages. The participants explained that the voice messages were sometimes too long and by the end of the message, they tended to forget what type of input or keywords are required from them in order to use the options. This was especially emphasized when there was a hearing impairment simulated. Participants from the groups with a hearing impairment felt a bit frustrated that they were not able to hear the messages well and this interfered with their driving experience – “the options were read aloud to support the driver but took concentration and attention instead of supporting because of the hearing impairment”. Participants stated they would have preferred shorter messages and an option to repeat them.