E-Service Accessibility Evaluation for Saudi Arabia’s Ministry of Human Resources and Social Development ()
1. Introduction
In e-government, information technologies make it easier for connected parties to share data and information [1]. As a result, the majority of e-government projects worldwide use software to provide their services electronically. Many government entities, including those in Saudi Arabia, have worked diligently to provide their services electronically to achieve e-government goals. The main electronic outlet used was the entity’s website. Furthermore, due to the increased dependence on mobile devices, government entities started providing their services through mobile applications. However, issues of exclusion for people with disabilities have arisen. Therefore, accessibility of these websites and mobile applications need to be checked and verified to cater for people with disabilities.
HTML, CSS, and other international Web standards are developed by the World Wide Web Consortium (W3C). One of the standards developed is the Web Content Accessibility Guidelines (WCAG), the latest version of it is 2.2, and version 3.0 is in the making. WCAG documents outline how to make web content more accessible to people with disabilities. WCAG is made up of 13 guidelines, organized under 4 principles: perceivable, operable, understandable, and robust. WCAG aims to offer a single, common standard for web content accessibility that satisfies the requirements of people, businesses, and governments around the world.
Automated accessibility tools—such as IBM’s Equal Access Toolkit [2] (which covers the entire lifecycle), ARC Toolkit [3] (which validates multiple standards), and WAVE [4] (which performs ARIA diagnostics), facilitate the process of checking compliance with WCAG 2.2. All of these tools support WCAG 2.2 through browser extensions available for Chrome, Firefox, and Edge, and they were chosen for the purpose of cross-validation. The 13 guidelines of WCAG 2.2, organized under 4 principles (Perceivable, Operable, Understandable, Robust), serve as the technical basis for the standards set by Saudi CSTC/DGA [5] [6].
Clearing the way for Saudi Arabian government agencies to successfully and efficiently provide their services digitally is the main duty of the Digital Government Authority (DGA) [7], the accountable government organization in charge of the electronic government initiative. Developing the laws, rules, and regulations required to support Saudi Arabia’s digital transformation is another responsibility of the DGA. DGA also keeps an eye on, evaluates, and quantifies the performance and digital capabilities of Saudi government organizations. In 2005, the Ministry of Finance (MoF) [8], the Ministry of Communications and Information Technology (MCIT) [9], and the Communications, Space & Technology Commission (CSTC) [10], formerly the Communication and Information Technology Commission (CITC), collaborated to create the e-government program, which was formerly known as Yesser. After the DGA was established by a royal decree in Saudi Arabia in March 2021, all of Yesser’s duties and initiatives were moved to DGA. At the moment, MCIT is in charge of DGA.
Saudi Arabia’s DGA E-Participation Controls [6] clearly mandate the inclusion of individuals with disabilities (Control 4-101-23), and CSTC’s Arabic accessibility standards [5] apply to all governmental platforms. However, in spite of these regulations, HRSD services exhibit significant deficiencies, which jeopardize the inclusion of over 112,000 visually impaired individuals in Saudi Arabia [11].
The General Authority for Statistics in Saudi Arabia (GASTAT) [11], conducted the Saudi Arabian census in 2022. In 2023, GASTAT issued the disability statics report, which included the prevalence percentage of disability in Saudi Arabia is 1.8%, which equals to (588,680) individuals of the total population; of this percentage there are 19.9% with vision disabilities 6.9% with hearing disabilities, and 60.6% have Mobility disabilities which include: walking or climbing stairs, hands & fingers, and arm with the percentage values 56.2%, 0.6%, and 3.8% consecutively. The field survey conducted was part of the 2022 census, with the objective of presenting information on the many forms and causes of impairment, as well as the demographic and practical traits of people with disabilities and their geographic distribution.
In Saudi Arabia, the Ministry of Human Resources and Social Development (HRSD) [12] is the government entity that handles all social affairs and development initiatives that align with goals and policies of the government. In addition, it is responsible for creating general labor and worker laws in both the public and private sectors. Currently, HRSD work is divided into three sectors, the Social Development sector, the Business Sector, and the Civil Service Sector. There are several online services offered to beneficiaries of each sector, 92 services in total. Of these 92 services, there are 41 services dedicated for the Social Development Sector, nine of which are provided to people with disabilities. Since HRSD is tasked with providing services to people with disabilities in general, the accessibility of these nine services (listed below) are the target of this study.
1) Disability evaluation
2) Financial subsidy service for people with disability
3) Medical Equipment Subsidy Service
4) Digital certificates service for traffic facilities for people with disabilities
5) Digital certificate service to reduce passenger wages for persons with disabilities and their escort
6) Day-care Centers
7) Issue disability certificate
8) Digital certification service for autism disorder
9) Visa exemption service for people with disabilities
Saudi Arabia is positioned 6th in the world according to the 2024 UN E-Government Development Index (EGDI) [13], indicating a sophisticated digital infrastructure. Nevertheless, this research uncovers a significant shortcoming: in spite of elevated EGDI ratings, crucial services tailored for individuals with disabilities do not meet WCAG 2.2 standards, thereby obstructing inclusive access.
Despite the E-Participation Controls set forth by DGA [6], which require the inclusion of individuals with disabilities, and Saudi Arabia’s notable #6 ranking in the EGDI [13], the disability services provided by HRSD (essential for 588,680 citizens [11]) exhibit significant non-compliance with WCAG 2.2 standards. This situation highlights a disconnect between policy and implementation in the context of Vision 2030’s goal for an inclusive digital transformation.
This study represents the inaugural assessment of HRSD’s services aimed at individuals with disabilities, utilizing WCAG 2.2 alongside three automated tools. This research progresses beyond earlier studies conducted in Saudi Arabia [14], which primarily concentrated on WCAG 2.1 or general e-government systems.
This study focuses on how accessible are the online services provided by HRSD to individuals with disabilities based on the WCAG 2.2 standard. The IBM’s Equal Access Toolkit, the ARC Toolkit, and the WAVE Tool are used to assess the accessibility of the websites of nine selected services. Furthermore, results from all three tools are analyzed to provide a detailed overview of the service accessibility.
The remainder of this study is arranged as follows: The following section provides background information and related work. Section 3 provides a description of the accessibility evaluation approach, its results, and recommendations. Section 4 presents the conclusion and future work.
2. Background and Related Work
The Saudi Arabian Ministry of Health (MOH) defines disabilities as complete or partial, temporary or permanent, physical, sensory, mental, communicative, educational, or psychological impairments that significantly and persistently impair a person’s capacity to perform daily activities, rendering the person to rely on others or requiring special tools, training, and rehabilitation to effectively utilize them [15]. Furthermore, Visual impairment is a type of disability that ranges from blurred vision to total blindness.
In its “Digital Accessibility, Breaking Down Barriers, Building Digital Inclusion” document [5], CSTC defines digital accessibility as creating and providing digital services that are accessible to all users, including those with disabilities, via computers, mobile devices, web platforms, etc. Through digital accessibility, person with a range of disabilities, such as hearing loss, vision impairments, cognitive limits, etc. can take advantage of the digital tools and services that are available in their surroundings. For more specialized person’s needs, it is usually accomplished by using a wide range of software tools (text-to-speech, voice commands, closed captioning, etc.) and other forms of assistive technology, which is usually a mix of hardware and software solutions.
Since 2001, the UN e-government survey has been published by the UN Department of Economic and Social Affairs [13], the survey’s outcome is the Electronic Government Development Index (EGDI). The survey’s objective is to assess how e-government programs are progressing among all United Nations (UN) members. UN members can use the results as a benchmark to assess their progress in e-government-related areas and make improvements.
Saudi Arabia received a score of 0.9602, placing it in the “Very High” category and sixth out of all UN nations in the most current EGDI, which was published in 2024. According to surveys conducted in 2020 and 2022, Saudi Arabia received a rating class of V2, placing it in the “Very High” group, and it was ranked thirty-first in 2022 and forty-third in 2020 with its EGDI scores of 0.8539 in 2022 and 0.7991 in 2020.
The Advancement of Saudi Arabia in the EGDI is contributed in part to DGA’s effort, such as the issuance of several policies, controls, standards, and guidelines. one of which is the E-Participation Controls [6]. In section 7.2: “Controls for enabling beneficiaries to participate in e-Participation” control number 4-101-23 states: “Use all available means to enable the participation and inclusion of different categories of beneficiaries, such as: people with disabilities, women, children, the elderly, and residents, and comply with what the Authority and the competent authorities issue in this regard.”
Furthermore, the National Information Center (NIC) in Saudi Arabia [16], developed a service called Nafath [17], which is an application that uses identity authentication to enable centralized, uniform access to all service providers in Saudi Arabia
Kleynhans and Fourie [18] explained that visually impaired, visually disabled, blind, partially sighted, non-sighted, and other terms are used in literature. Rarely are these terms thoroughly defined to provide a clear distinction between the finer nuances, and they are frequently neither clarified nor explained. The significance of defining the meaning of “visually impaired” and related phrases precisely before starting accessibility research is discussed in their work, where they found that based on their degree of visual acuity and visual field, those with low vision or blindness are classified as having visual impairments.
Rakhmawati et al. [19] pointed out that research shows that educational websites have substantial accessibility issues that prevent people with disabilities from having equitable access. By creating an accessible counseling platform in accordance with WCAG, their work fills these gaps and guarantees inclusive access for all students. In their work, they intended to develop a technically sophisticated, user-centered application that improves usability and independence for students with disabilities by utilizing the structured phases of Analysis, Design, Development, Implementation, and Evaluation found in the ADDIE model. A high satisfaction percentage of 89.33% was found in user acceptance testing with 11 participants, indicating that the platform successfully satisfies users’ needs and greatly enhances accessibility and usability in educational counseling services. Results of their work emphasizes how crucial it is to incorporate accessibility guidelines in order to promote equity and diversity in the digital spectrum.
Most websites are considered difficult to use by those with vision problems. Therefore, Ram et al. [20] presented WebSight, a cutting-edge web plugin for an AI-based image description generator, to solve such issues. The proposed browser extension systematically improves web accessibility by replacing the “alt” properties of images with well crafted descriptions using a smooth interface with a Flask server, Optical Character Recognition, and a specialized Image Description Generator model. By giving visually impaired users accurate and contextually rich information, there method greatly promotes web inclusion and gives them a more personalized and improved surfing experience. Utilizing Tesseract-OCR to extract text from images that contain embedded text, creating a web extension that effortlessly retrieves the alternate text of images from the image captioning model, and developing two image description generator models based on the encoder-decoder approach and comparing their performances to choose the best model for use in WebSight are the methods by which they accomplish their work.
Nicoson and Ha-Brookshire [21] investigated the difficulties that people with visual impairments encounter when trying to access informational content via Assistive technology in clothing websites. Assistive technology, which offers a human interface to the website’s alternative text in a variety of modalities, including screen readers and linear navigation, provides people with visual impairments an alternate method of accessing information on clothing websites. In their work, they described the necessity of critically examining clothing websites to make sure that everyone, especially those with visual impairments can access them online. In addition, they found that the WCAG are not followed by many clothing websites, which restricts people with visual impairments access to clothing. Moreover, they assured that access to published digital content was deemed a fundamental human right by the UN in 2021.
Raymond et al. [22] explained that persons who are visually impaired depend on inclusive design, which incorporates alternate text (alt text) for digital interpretation of images, videos, and communications. Due to its many subtleties, alt text might contain a lot of technical errors as well as erroneous media interpretations. Furthermore, Social networking sites and other digital media have grown to be important information sources and the entry point to online shopping. Due to insufficient accessibility features and inclusive design, many vulnerable consumers with visual impairments were left out and unable to learn about opportunities, services, and products. Their research concentrated on how accessible and inclusive company advertisements were on social media for customers with visual impairments. The study specifically looked at auto alt text (using AI) and alt text inclusion and accuracy on social media platforms. In addition, they looked at the accuracy of alt text in social media postings. They also examined the challenges faced by marketers when implementing alt text, which restricts accessibility and inclusion for customers who are more susceptible, particularly those who are visually challenged. According to found results, more than 95% of the posts were not inclusive since they either lacked accessibility or used it incorrectly.
Al Saeed et al. [14] employed automated tool evaluation and usability testing to examine how challenging it is for teachers with visual impairments to access three e-government educational systems. Four teachers with vision impairments participated in the usability assessment. Furthermore, four automatic programs were also used: SortSite, HTML-CodeSniffer, AChecker, and Total Validator. As the primary concern raised by the participants, the results demonstrated that none of the three systems adequately supported screen readers. However, the automated assessment tools assisted in determining the most significant accessibility problems with these systems. The three systems most frequently broke the perceivable principle, which was followed by operable and robust. Furthermore, additional investigation showed that even the lowest level of accessibility conformance was broken by the majority of the faults found.
Bondok et al. [23] suggested that enhancing the accessibility of ophthalmology websites can assist people with vision access needs in obtaining necessary appointment details, instructional materials, and information about accessible ophthalmic services. The accessibility of these webpages for patients with color vision deficits and visual impairment was assessed by their work. In their work, AChecker, ARC Toolkit, and WAVE Tool were the three evaluation tools utilized to evaluate the accessibility of the homepages of the top-ranked ophthalmology institutes in the United States using WCAG 2.0. The websites of the 38 ophthalmic facilities that were ranked first in the 2023 US News & World Report were assessed. Every website that was evaluated had accessibility problems. Utilized tools found that it is far more difficult to navigate web pages. The majority of problems were related to difficulty understanding content and interface elements, and websites were challenging to use or traverse. Missing directions for user data input, hyperlinks without destination information, and the lack of alternate language for images were common problems that affected people who are blind or have limited vision and use screen readers. They came to the conclusion that in order to make ophthalmology websites more accessible to those with vision access needs, they should be updated to meet WCAG guidelines.
Phochai et al. [24] examined the environmental and sociodemographic factors that affect how visually impaired people in Thailand use the Internet. Data from 5621 visually handicapped respondents were subjected to logistic regression analysis, chi-square testing, and descriptive statistics. Their results show that about 26.88% of people with visual impairments use the Internet, men are less likely than women to use it, younger people are more likely to use it, use decreases with age, and there are regional differences. The chance of using the Internet was positively correlated with wealth index, income, and educational achievement. Internet use is more common among those who are employed. They highlight the necessity of all-encompassing digital inclusion programs that take into account a number of variables, including developments in assistive technology.
Zombron et al. [25], investigated enhancing accessibility in STEM education for those with visual disabilities through the use of Generative Artificial Intelligence (GenAI). Their research offers a proof-of-concept for producing descriptions of the visual representations utilized in Systems Thinking, a technique that can be difficult for visually challenged people to adopt due to its reliance on intricate diagrams and visual models. In their work, they aim to increase the accessibility of dynamic systems and support the moral application of GenAI in accordance with WCAG 2.1 AA standards. In addition, they were successful in creating a ChatGPT 4 GenAI prompt that can identify and explain fundamental Systems Thinking representations, which is a first step toward full accessibility for visually impaired people when it comes to interpreting visual system representations.
Ashfin [26] examined the opportunities, advantages, and difficulties of employing AI to help visually impaired students in classrooms. The study, that utilized a mix methodology approach, assessed the efficacy of AI-driven solutions and offered ways for fair deployment. The results demonstrate how AI may be used to build inclusive classrooms and enable visually impaired students to reach their greatest potential.
Maulana [27], claimed that, especially for those with disabilities, web accessibility is essential to establishing an inclusive online environment. The purpose of the work was to create a prototype website that incorporates accessibility features as best as possible and to examine how the WCAG principles are applied on the Tokopedia e-commerce platform. The work used a qualitative approach and a case study methodology, which includes developing a prototype website using Tailwind CSS, testing assistive technology using the NVDA screen reader, and website analysis. The findings showed that the prototype website improved usability for the general public in addition to being accessible to visually impaired people. Furthermore, the work comes to the conclusion that everyone’s comfort and inclusiveness can be enhanced by web accessibility.
Hamideh Kerdar et al. [28], researched the first-hand experiences and difficulties that people with visual impairment and blindness have when utilizing digital technology, with the goal of enhancing knowledge of critical elements in digital accessibility for these persons. 49 articles that satisfied the inclusion requirements were chosen for assessment based on the WCAG rules, highlighting specifics for the guidelines’ consideration and improvement. The findings showed that inaccessibility negatively impacted consumers in a number of ways. For instance, a lot of websites are made for sighted users and only use visual content to convey information; assistive technology users are not given any other options. Furthermore, a persistent issue is the absence of keyboard accessibility, shortcuts, or interoperability with other assistive technology. Additionally, basic accessibility guidelines—such as labeling buttons and links or providing alternative language for images—are not sufficiently or regularly adhered to. In their work, they explained the factors that should be taken into account while developing digital technologies, highlighted the difficulties and repercussions of their inaccessibility. Finally, they advised that people with disabilities participate in the development of technology and the evaluation of its accessibility.
Research Gaps
Although initiatives in Saudi Arabia, such as Nafath [17] and the E-Participation Controls by DGA [7], reflect a dedication to digital inclusion, previous research on accessibility indicates ongoing challenges:
1) Excessive dependence on outdated or singular automated tools: Research conducted by AlSaeed et al. [14] (pertaining to Saudi e-education systems) and Bondok et al. [23] (related to US hospital portals) employed tools aligned with WCAG 2.0/2.1 (for instance, AChecker, SortSite) without cross-validation, which poses a risk of incomplete diagnostics.
2) Oversight of contextually significant elements: Prior studies have either assessed superficial pages [14] or services not specifically designed for individuals with disabilities [23], neglecting to consider:
- Interfaces in Arabic (which are essential for Saudi users),
- Authenticated processes (such as form submissions),
- Services that are directly associated with national disability policies.
3) A culture of reactive evaluation: Accessibility assessments are generally conducted after development [28], thereby overlooking chances for the integration of inclusive design.
4) Uniform reporting of severity: Violations are reported in a standardized manner [14] [23] without distinguishing between WCAG 2.2 Level A (must-fix) failures.
This research addresses these deficiencies by:
1) Triangulating three tools aligned with WCAG 2.2 (IBM Equal Access, ARC Toolkit, WAVE) to corroborate findings across various platforms,
2) Concentrating solely on nine services critical to disability under HRSD—the Saudi organization responsible for disability support,
3) Implementing rapid manual validation (including keyboard and screen reader tests) for high-traffic services to identify experiential barriers,
4) Categorizing violations by severity (WCAG 2.2 A/AA) to facilitate policy-enforceable remediation,
5) Suggesting proactive compliance frameworks that align with the mandates of the Saudi Digital Government Authority (DGA) [6] [7].
This research introduces innovation in three distinct aspects. Firstly, it employs a triangulated evaluation methodology that utilizes three automated tools aligned with WCAG 2.2, supplemented by manual assessments, thus addressing the shortcomings of previous studies conducted in Saudi Arabia and internationally that depended on singular or outdated tools. Secondly, it concentrates on nine HRSD services specifically tailored for individuals with disabilities, ensuring that the results directly enhance the inclusivity of essential citizen-facing services rather than merely general government websites. Lastly, it delivers actionable compliance insights by classifying violations based on WCAG conformance levels (A and AA) and explicitly linking them to the mandates of the Saudi Digital Government Authority. Collectively, these contributions create a Saudi-specific evaluative framework that transcends merely identifying known accessibility issues by providing a replicable and policy-relevant methodology.
3. Accessibility Evaluation Approach
3.1. Research Design
As mentioned previously, many governments strive to provide their services electronically to cater for their citizens, including Saudi Arabia. On the other hand, citizens with disabilities may encounter exclusion when using e-government services.
3.2. Selection of Services
This study’s approach is to evaluate the aforementioned nine online services that are provided by HRSD to people with disabilities. Each e-government service in Saudi Arabia must have a dedicated main web page that contains information about the service, such as: service name, service description, service requirements, service fees, service steps, service time, service language, service channels, and target beneficiaries. Most e-government services require logging in either through a username and password acquired by the service provider or through NAFATH. As for HRSD services that are within the scope of this study, all require logging in. Thus, the evaluation was conducted on the English version only of the service’s main web page.
Although this research utilized automated tools for accessibility evaluation alongside limited manual testing (specifically keyboard navigation), it did not engage participants with disabilities directly. This omission is acknowledged as a limitation, given that insights derived from users’ experiences are crucial for validating technical results and identifying barriers that automated scans may overlook. The current study is framed as a benchmarking phase that determines the systemic accessibility status of HRSD services, which will provide a basis for organized user testing in subsequent research. Future plans include executing usability studies with individuals who have visual impairments, utilizing Arabic-compatible screen readers like JAWS, in partnership with local advocacy groups. By positioning automated evaluation as a preliminary step rather than a replacement for user testing, this research establishes an empirical foundation for improvements in accessibility based on evidence.
3.3. Testing Environment
To evaluate web accessibility a well-known and reliable tool needs to be handy. In addition, having a free web extension of the tool is advantageous. Some of the reviewed literature in this study utilized tools that aid in accessibility evaluation, such as: SortSite, HTML-CodeSniffer, AChecker, and Total Validator [14], and AChecker, ARC Toolkit, and the WAVE Tool [23]. After thorough examination of these tools, it was found that SortSite, HTML-CodeSniffer, and AChecker only evaluate against the previous version of WCAG (2.1). As for the Total Validator tool, despite having a browser extension, using it requires installing software on a local computer and a one time purchase for the basic version, which only validates one page at a time.
Language Limitation Note: This research focused solely on the English interfaces of HRSD services. Subsequent evaluations should incorporate validation in Arabic, utilizing tools such as JAWS Arabic [29] (the industry-standard screen reader for Arabic RTL support) to identify language-specific obstacles (e.g., semantic markup errors, screen reader incompatibility) that impact Saudi users.
Expanded Validation Framework
In order to overcome the constraints related to language coverage and the validation by actual users, this research suggests methodological improvements for forthcoming implementations:
1) Arabic Interface Testing: JAWS Arabic [29] will assess semantic markup, support for right-to-left (RTL) text, and compatibility with screen readers in authenticated workflows (for instance, form submissions in the services).
2) User-Centered Protocols: Following the ethical guidelines supervised by the Authority for the Care of People with Disabilities (APD) [30], task-oriented tests (comprising 3 tasks per service: login, form submission, and status check) will evaluate success rates among 5 - 10 disabled participants.
3.4. Evaluation Tools
The evaluation process will be based on the WCAG 2.2 standard [31] using IBM’s Equal Access Toolkit, the ARC Toolkit, and the WAVE Tool.
The attributes used in the measurement for the aforementioned tools including their descriptions are shown in Tables 1-3. It is worth mentioning that Accessible Rich Internet Applications (ARIA), part of the WAVE Tool results, outlines a method for improving the usability of Web applications and content for those with disabilities. It is particularly useful for complex UI controls and dynamic content created with HTML, JavaScript, and related technologies.
Table 1. IBM’s Equal Access Toolkit attributes used in the evaluation.
Attribute |
Description |
Violations |
Issues with accessibility that must be fixed. |
Needs review (NR) |
An evaluation by an evaluator is necessary, these issues may be a violation. |
Recommendations (Rec.) |
Opportunities to further enhance accessibility by implementing best practices. |
Table 2. ARC Toolkit attributes used in the evaluation.
Attribute |
Description |
Errors |
Findings where problems with accessibility are discovered. |
Alerts |
Possible problems that have been identified but need to be manually confirmed. |
Best Practice (BP) |
Outcomes that may affect users of assistive technology and the site’s usability but are not covered by the guidelines. |
Table 3. WAVE Tool attributes used in the evaluation.
Attribute |
Description |
Errors |
Indicates issues that will impact certain users with disability, and failure to meet WCAG requirements. |
Contrast Errors (CE) |
Instances of text that does not meet WCAG contrast requirements. |
Alerts |
Indicate elements of the page that may cause accessibility issues, an evaluator intervention is required. |
Features |
indicates elements that improve accessibility when implemented correctly. |
Structural Elements (SE) |
defines elements of the web page, an evaluator review is required to approve its correctness. |
ARIA |
Items in this attribute refer to issues related to ARIA. |
3.5. Testing Procedure
In this research, an initial manual testing phase was performed on a selected group of services to investigate the various types of accessibility violations that might arise. This exploratory stage, which utilized fundamental keyboard navigation assessments, yielded valuable insights into possible obstacles and guided the following automated evaluation. After this preliminary assessment, the main testing procedure utilized three automated tools—ARC Toolkit, WAVE, and IBM’s Equal Access toolkit to methodically evaluate the accessibility of all chosen HRSD services in accordance with WCAG 2.2 standards.
The manual keyboard navigation testing was conducted on three services with significant violations:
1) Disability evaluation.
2) Financial subsidy service for people with disability.
3) Day-care Centers.
The testing protocol used was as following:
1) Tab Navigation: Pressed Tab/Shift + Tab to navigate through all interactive components (e.g. links, buttons).
2) Focus Visibility: Verified visual focus indicators (e.g. outline, color).
3) Operability: Ensured functionalities could be completed through the use of a keyboard.
These preliminary findings validated the existence of accessibility obstacles and underscored the necessity for systematic verification. As a result, the following automated assessment delivered a thorough and scalable evaluation of the HRSD services’ adherence to WCAG 2.2.
3.6. Data Analysis
Based on the insights gained from the exploratory manual testing and the systematic scans performed using ARC Toolkit, WAVE, and IBM’s Equal Access toolkit, the outcomes were compiled and examined to discern patterns, categorize violations, and evaluate overall adherence to WCAG 2.2.
For clarity, the following terminology was applied in categorizing issues:
1) Violations/Errors: Failures at WCAG 2.2 Level A [31] that obstruct fundamental access (for instance, absent labels).
2) Alerts: Possible concerns that require human assessment (such as duplicate links).
3) Best Practices: Improvements that exceed the minimum requirements of WCAG 2.2 (for example, skip navigation).
The sampling methodology was derived from the catalogue of ninety-two electronic services provided by the Ministry of Human Resources and Social Development. From this catalogue, nine services were specifically chosen as they are tailored for individuals with disabilities, making them essential for achieving national inclusion objectives. The evaluation concentrated on English-language interfaces, as the authenticated Arabic workflows necessitated beneficiary login, which fell outside the parameters of this preliminary study. Automated scans were performed on July 10, 2025, utilizing Google Chrome on a Windows 11 platform. The tools employed included the IBM Equal Access Toolkit (2025 release), the ARC Toolkit (2025 Chrome extension), and the WAVE browser extension (2025 WebAIM release). In addition to the automated scans, a swift manual protocol was implemented on three services that exhibited high violation counts. This protocol involved navigation using Tab/Shift + Tab, checks for focus visibility, and validation of operability. These specifics are provided to facilitate replication by other research teams.
4. Results, Analysis, and Recommendations
4.1. Results
The three tools utilized in this study were executed on the main web page of each selected service on July 10, 2025 using Microsoft Windows 11 Home as the operating system and Google Chrome as the web browser. In addition, as mentioned previously all these services require logging in, thus only the English version of each service’s main web pages was evaluated; other pages requiring login were excluded. Results of running the evaluations tools are shown in Tables 4-6.
Table 4. Results of executing the IBM Equal Access Toolkit on the selected services.
# |
Service Name |
Violations |
NR |
Rec. |
WCAG Conformance Level |
1 |
Disability evaluation |
86 |
119 |
6 |
|
2 |
Financial subsidy service for people with disability |
91 |
124 |
6 |
|
3 |
Medical Equipment Subsidy Service |
86 |
103 |
6 |
|
4 |
Digital certificates service for traffic facilities for people with disabilities |
87 |
123 |
6 |
|
5 |
Digital certificate service to reduce passenger wages for persons with disabilities and their escort |
85 |
120 |
6 |
Mostly A, some AA |
6 |
Day-care Centers |
72 |
119 |
7 |
|
7 |
Issue disability certificate |
87 |
106 |
6 |
|
8 |
Digital certification service for autism disorder |
85 |
91 |
6 |
|
9 |
Visa exemption service for people with disabilities |
90 |
120 |
6 |
|
Table 5. Results of executing the ARC Toolkit on the selected services.
# |
Service Name |
Errors |
Alerts |
BP |
WCAG Conformance Level |
1 |
Disability evaluation |
101 |
109 |
4 |
|
2 |
Financial subsidy service for people with disability |
102 |
117 |
5 |
|
3 |
Medical Equipment Subsidy Service |
102 |
107 |
5 |
|
4 |
Digital certificates service for traffic facilities for people with disabilities |
102 |
112 |
12 |
Majority A, some AA |
5 |
Digital certificate service to reduce passenger wages for persons with disabilities and their escort |
102 |
107 |
11 |
|
6 |
Day-care Centers |
89 |
85 |
0 |
|
7 |
Issue disability certificate |
99 |
106 |
0 |
|
8 |
Digital certification service for autism disorder |
100 |
102 |
16 |
|
9 |
Visa exemption service for people with disabilities |
101 |
108 |
0 |
|
Table 6. Results of executing the WAVE Tool on the selected services.
# |
Service Name |
Errors |
CE |
Alerts |
Features |
SE |
ARIA |
WCAG Conformance Level |
1 |
Disability evaluation |
5 |
11 |
68 |
36 |
36 |
94 |
|
2 |
Financial subsidy service for people with disability |
5 |
12 |
67 |
36 |
43 |
97 |
|
3 |
Medical Equipment Subsidy Service |
5 |
12 |
70 |
36 |
32 |
92 |
|
4 |
Digital certificates service for traffic facilities for people with disabilities |
5 |
12 |
68 |
36 |
33 |
93 |
Errors & ARIA: A; Contrast: AA |
5 |
Digital certificate service to reduce passenger wages for persons with disabilities and their escort |
5 |
12 |
69 |
36 |
28 |
91 |
|
6 |
Day-care Centers |
4 |
7 |
76 |
36 |
25 |
91 |
|
7 |
Issue disability certificate |
5 |
11 |
68 |
36 |
40 |
95 |
|
8 |
Digital certification service for autism disorder |
5 |
12 |
67 |
36 |
27 |
91 |
|
9 |
Visa exemption service for people with disabilities |
5 |
13 |
67 |
36 |
36 |
94 |
|
It is apparent from the results of the IBM Equal Access Toolkit in Table 4 that all evaluated services generated a large number of violations, the least violating service was service number six (Day-care Centers), the highest violating service was service number two (Financial subsidy service for people with disability) with a violation count of 91. In addition, all services returned a big number in the “Needs Review” attribute, above 90, the least service was number eight (Digital certification service for autism disorder) that had 91 issues, and the highest service was service number two (Financial subsidy service for people with disability) which had 124 issues, all these issues need to be evaluated by an evaluator to assess its validity. Moreover, all services were equal in the “Recommendations” attribute, where there were six recommendations for each service.
Examining the ARC Toolkit results displayed in Table 5 shows that all services generated more than 88 errors, the least was the sixth service (Day-care Centers) with 89 errors, and the highest was services two (Financial subsidy service for people with disability), three (Medical Equipment Subsidy Service), four (Digital certificates service for traffic facilities for people with disabilities), and five (Digital certificate service to reduce passenger wages for persons with disabilities and their escort). In addition, all services generated more than 84 alerts, the least service was service number six (Day-care Centers) with 85 alerts, and the highest was the service number two (Financial subsidy service for people with disability). Moreover, in the best practice attribute, there were three services with zero suggestions for best practices, service number six (Day-care Centers), seven (Issue disability certificate), and nine (Visa exemption service for people with disabilities). The highest service with best practice suggestions was service number eight (Digital certification service for autism disorder) with 16 best practices suggested.
The WAVE Tool results shown in Table 6 provide more detailed information about web page accessibility. In general, all services generated 5 errors except for service number six (Day-care Centers), only four. In the “Contrast Errors” attribute, the count ranged from 7, generated by service number six (Day-care Centers), all the way to 13 generated by service number nine (Visa exemption service for people with disabilities). As for the “Alerts” attribute, three services generated the least number of alerts, service number 2 (Financial subsidy service for people with disability), eight (Digital certification service for autism disorder), and nine (Visa exemption service for people with disabilities), 67 alerts. The highest alert generating service was service number six (Day-care Centers), 76 alerts. In the features attribute, all services were equal, were they generated 36 features that may improve accessibility. In the “Structural Elements” attribute, the least service that generated remarks was service number six (Day-care Centers), 25 remarks, and the most service that generated remarks was service number two (Financial subsidy service for people with disability), 43 remarks. Finally, in the “ARIA” attribute, the service that generated the least number of issues were three services, service number five (Digital certificate service to reduce passenger wages for persons with disabilities and their escort), six (Day-care Centers), and eight (Digital certification service for autism disorder), 91 issues, and the highest service with issues was service number two (Financial subsidy service for people with disability), 97 issues.
Table 7 illustrates the average count of WCAG 2.2 accessibility violations categorized by principle (Perceivable, Operable, Understandable, Robust) and conformance level (A, AA) across the assessed e-services. This overview emphasizes the principles and conformance levels that exhibit the greatest number of violations, offering a concise representation of overall accessibility performance and informing areas for enhancement.
Table 7. Average WCAG 2.2 accessibility violations by principle and conformance level.
Tool |
Mean Violations/Errors |
Key Issues |
Level A (Critical) |
Level AA (Usability) |
IBM Equal Access |
85.8 |
ARIA misuse, unlabeled fields, missing alt text |
≈75% - 80% |
≈20% - 25% |
ARC Toolkit |
98.1 |
Keyboard focus, ARIA roles, form labeling |
≈75% - 80% |
≈20% - 25% |
WAVE |
4.7 |
ARIA misuse, low contrast, structural navigation |
Errors & ARIA: A |
Contrast issues |
Overall Mean (All Tools) |
62.9 |
Systemic Level A issues dominate;
contrast = main AA issue |
≈245.7 total |
≈47.3 total |
When considering the data from the nine services collectively, the average number of violations was roughly 85 when utilizing the IBM Equal Access Toolkit, 100 with the ARC Toolkit, and 5 direct errors identified through the WAVE tool. Nevertheless, the WAVE findings also indicated an average of 11 contrast errors, 33 structural problems, and 93 ARIA-related issues per service, all of which constitute considerable accessibility obstacles. The most prevalent issues identified across the various tools included unlabeled form elements, inadequate color contrast, misuse of ARIA attributes, and keyboard traps. When categorized by severity, the majority of these issues were classified as WCAG 2.2 Level A violations, which impede essential access, while others were identified as Level AA issues (such as those related to contrast). Regarding user impact, unlabeled form fields hinder screen reader users from successfully completing submissions, low-contrast text prevents users with low vision from reading instructions, and elements that require mouse interaction obstruct users with motor impairments. These results underscore that technical non-compliance directly results in barriers for individuals with disabilities when accessing vital HRSD services.
In order to enhance understanding of the seriousness of accessibility failures, each type of violation was aligned with the WCAG 2.2 conformance levels (A or AA). Throughout all tools, the vast majority of violations (around 75% - 80%) were associated with Level A, indicating critical accessibility obstacles that need to be addressed to guarantee fundamental functionality for individuals with disabilities. Instances of these violations include absent form labels, erroneous ARIA roles, and elements that are not accessible via keyboard. The remaining 20% - 25% pertained to Level AA issues, mainly characterized by inadequate contrast ratios and inconsistencies in visual presentation. Level AAA criteria were not evaluated, as they exceed the compulsory compliance requirements for most international and Saudi accessibility regulations. This classification emphasizes that HRSD services demonstrate systemic shortcomings at the most fundamental levels of accessibility, highlighting the pressing need for remedial measures.
4.2. Analysis
The assessment findings distinctly reveal considerable accessibility shortcomings in all nine evaluated services. Although the three tools employed—IBM Equal Access Toolkit, ARC Toolkit, and the WAVE Tool—vary in their analytical methodologies, numerous patterns and consistencies emerged:
1) Under-performing Services: The “Financial Subsidy Service for People with Disability” has consistently shown the highest incidence of violations among all three assessment tools. It recorded 91 violations (IBM), 102 errors (ARC), and the most significant number of ARIA issues (97) and structural elements (43) in the WAVE Tool. This indicates that, although it serves as a fundamental benefit for individuals with disabilities, it may be the least accessible service, raising significant concerns regarding its intended users.
2) Better-Performing Services: The “Day-care Centers” service demonstrated the fewest violations among the tools utilized. 72 violations in the IBM Equal Access Toolkit, 89 errors in the ARC Toolkit, 4 WAVE Tool errors, along with the least number of ARIA and structural issues. Although it remains not entirely compliant, this service could serve as a standard for relatively superior design practices.
3) Significant Number of Ambiguous or Unverified Issues: The “Needs Review” (NR) attribute in IBM results exhibited a significantly elevated level across all services, with values ranging from 91 to 124. This underscores a pressing necessity for human evaluators to intervene and verify numerous flagged issues. Additionally, it indicates potential constraints of relying solely on automated tools, particularly in dynamic or ARIA-rich environments.
4) Discrepancies and Notification Concerns in the WAVE Tool: All services exhibited contrast errors along with a significant number of alerts (ranging from 67 to 76). Although these alerts do not constitute violations in themselves, they indicate possible accessibility pitfalls, especially for users with low vision. The elevated number of alerts implies a lack of consistency in the application of color schemes, landmarks, or semantic markup.
5) ARIA Complexity: Issues related to ARIA were evident across all services, with prevalence rates between 91 and 97. This indicates a potential overdependence on JavaScript-driven dynamic components or inadequate application of ARIA roles and states. Such findings underscore the necessity for training developers in semantic HTML and the principles of accessible interaction design.
6) Absence of Best Practice Adoption: Based on the findings from ARC, numerous services did not fulfill the criteria for “best practice.” While these shortcomings do not constitute outright violations, they do affect usability and the support provided for assistive technologies. For instance, the service titled “Digital Certification for Autism Disorder” received 16 warnings regarding best practices, indicating that there are acknowledged issues that remain unresolved.
4.2.1. User Impact Analysis
The identified accessibility violations have significant and detrimental effects on individuals with disabilities:
1) Critical Barriers in Core Services:
- Financial Subsidy Service: Unlabeled fields (e.g. income input) violating WCAG 4.1.2 (A), hindering screen reader users.
- Visa Exemption Services: Low-contrast text (3.8:1) violating WCAG 1.4.3 (AA), excluding low vision users.
2) Contrast Errors (WAVE):
- There are 13 contrast errors present in the Visa Exemption Service (refer to Table 6), the text-background contrast ratio is less than 4.5:1.
- User Impact: Users with low vision are unable to read the application instructions, which results in their reliance on others for assistance.
3) Missing ARIA Labels (IBM/ARC):
- The Financial Subsidy Service has 97 ARIA-related issues (see Table 6), reflecting that there are form fields that lack labels.
- User Impact: Users of screen readers encounter the phrase “unlabeled button” instead of “Submit Application,” which leads to the abandonment of the form by visually impaired users.
4) Keyboard Traps (Manual Validation):
- Disability Evaluation service necessitated the use of a mouse for modal dialogs.
- User Impact: Motor impaired users are unable to submit requests on their own.
4.2.2. Synthesis of Systemic Issues
Overall, these findings underscore notable shortcomings in ARIA consistency, adherence to WCAG 2.2-A standards, and validation prior to launch. Services that are crucial for individuals with disabilities (such as Financial Subsidy) show the lowest levels of accessibility, which directly impacts beneficiaries (Sec 3.3.1) and reveals a gap between the policies of the Saudi DGA [6] [7] and their technical implementation.
It is essential to incorporate accessibility considerations throughout the design process, rather than solely relying on evaluations conducted after development. As evidenced by manual validation (Sec 3.1.2), employing hybrid approaches (combining automated tools with user testing) is vital for ensuring accurate compliance.
4.3. Recommendations
Services evaluated in this study were based on their relevance to the government entity that is serving, supporting, and overseeing people with disabilities in Saudi Arabia. It does not imply that these are the only services provided to people with disabilities; however, they are the most relevant.
Based on the evaluation results generated by the utilized three tools, the following recommendations aim to improve the accessibility of e-government services’ web pages in HRSD and other government entities in Saudi Arabia. The recommendations are grouped by three themes: Policy, Technical, and Community. In addition, executing these multi-tiered recommendations has the potential to greatly enhance the inclusiveness and usability of e-government services in Saudi Arabia.
Every suggestion made in this research is clearly aligned with both international and national accessibility standards. For example, the adjustment of ARIA attributes aligns with the WCAG 2.2 criterion 4.1.2 (“Name, Role, Value”), while the provision of adequate color contrast pertains to WCAG 2.2 criterion 1.4.3. The operability via keyboard corresponds to WCAG 2.2 criterion 2.1.1 and directly supports the Saudi Digital Government Authority’s E-Participation Control 4-101-23, which requires inclusive engagement for persons with disabilities. Likewise, the implementation of compulsory accessibility audits prior to launch addresses both the conformance requirements of WCAG 2.2 and the oversight duties of the DGA. By associating each recommendation with a particular criterion or national control, this study offers a practical framework for enforcement instead of merely restating broad policy commitments.
4.3.1. Policy Recommendations
1) Establish a DGA-led Accessibility Taskforce:
- Conduct an audit of all 3515 e-services for individuals (according to DGA [7]), utilizing WCAG 2.2 AA standards as benchmarks.
- Implement a pre-launch certification process that necessitates:
* Automated scans (IBM/ARC/WAVE),
* Manual assessments (JAWS Arabic, keyboard/screen reader),
* User trials with APD [30].
- Suspend updates/services for entities that do not comply.
2) Revise E-Participation Controls [15]:
- Incorporate WCAG 2.2 AA into the Saudi Digital Government Standards.
- Mandate accessibility approval at every phase of the Software Development Life Cycle (SDLC) (Design/Develop/Verify).
3) Initiate National Accessibility Certification:
- Grant “Saudi Web Accessibility Compliance” seals to services that successfully pass DGA audits.
- Release quarterly rankings of government entities on DGA’s [7] website.
4.3.2. Technical Improvements
1) AI-Driven Remediation:
- Incorporate generative AI technologies (such as WebSight [20] for alternative text; generative ai [25] for translating workflows) into the HRSD’s Content Management System (CMS).
- Create ARIA labels that are compatible with right-to-left (RTL) languages, following CSTC’s Arabic guidelines [5].
2) Developer Capacity Building:
- Collaborate with DGA and CSTC to develop mandatory quarterly workshops on WCAG 2.2 Level A/AA compliance and ARIA implementation for public sector developers.
- Create an open-source repository containing accessible UI components in Arabic.
4.3.3. Community Engagement
1) Engage charitable and disability advocacy organizations in the processes of testing, reviewing, and co-designing e-services to guarantee accessibility from the user’s viewpoint.
2) DGA, CSTC, SDAIA, and other concerned entities need to promote web accessibility awareness in government entities and for the general public.
5. Conclusion and Prospective
This study uncovers significant accessibility deficiencies in HRSD’s disability services, even with Saudi Arabia’s leading e-government ranking (EGDI 2024 [13]). The Financial Subsidy Service—a vital benefit—demonstrated the most substantial violations (for instance, 97 ARIA errors), which directly obstruct visually impaired users (Sec 3.3.1).
These initiatives put into action the inclusivity objectives of Saudi Vision 2030 by closing the divide between DGA policy [6] [7] and its technical execution. The suggested Taskforce and the integration of AI specifically tackle the fundamental issues highlighted in this research: disjointed compliance assessments and a lack of adequate developer skills.
The government of Saudi Arabia provides more than 2500 electronic services from different ministries directorates and authorities. In this study, focus was on assessing nine essential services provided by the Ministry of Human Resources and Social Development to individuals with disabilities through three tools aligned with the WCAG 2.2 standards. The findings reveal significant accessibility deficiencies that hinder equitable access. The study recommends incorporating accessibility from the initial design phases, enforcing national compliance frameworks, and leveraging AI to facilitate improvements in accessibility.
5.1. Ethics Statement
This research did not engage human participants, collect personal data, or intervene with users. It solely assessed publicly accessible government e-services via both automated and manual accessibility evaluations. Consequently, there was no necessity for formal ethics approval. The research adhered to the principles of responsible research conduct, which encompass transparency, reproducibility, and recognition of limitations.
5.2. Limitations
Limitations of this research work are described below:
1) This research has revealed systemic failures in accessibility; however, the omission of Arabic interfaces and authenticated workflows (such as post-login forms) restricts the potential for actionable insights for the primary user demographic in Saudi Arabia.
2) Automated Tool Dependency: Automated tools are unable to identify all accessibility concerns (for instance, the quality of meaningful alt-text and contextual usability). Thus, there is a possibility of encountering false positives and negatives.
3) Limited Manual Testing: Some services were subjected to manual validation using keyboard navigation. Comprehensive usability testing involving actual individuals with disabilities was not performed (this is planned for future work).
4) Scope Restriction: The focus was solely on nine HRSD services related to disabilities, Mobile applications and other HRSD digital platforms were excluded from this evaluation.
5) Time-Bound Evaluation: The findings reflect the accessibility status at the time of testing (for example, July 2025). Services may change due to ongoing updates.
6) Language Coverage: The primary evaluation was conducted in English. There may be some variations in accessibility across different languages.
5.3. Future Work
Future research will:
1) Arabic-First Validation: Conduct tests on authenticated workflows in Arabic utilizing JAWS Arabic [29] across the nine services, assessing task success rates with 5 participants who are visually impaired and are affiliated with APD.
2) National Accessibility Audit: Assess Saudi Arabian e-government services prioritized services targeting high-disability-impact sectors (e.g. healthcare) alongside user volume metrics.
3) Stakeholder Integration: Conduct interviews with 10 - 15 developers from HRSD and other government entities to uncover barriers to compliance (such as training issues and limitations of the CMS).
4) Automated Compliance Dashboards: Create AI-powered monitoring systems utilizing IBM’s Equal Access Toolkit to:
- Track WCAG 2.2 violation trends across various services;
- Produce public accessibility scorecards (A-AA-AAA ratings);
- Publish quarterly government entity compliance reports on DGA’s [7] website.
Statements and Declarations
Author Contribution
The Author is responsible for the conceptualization of this research work. He developed the questionnaire, prepared the figures of the manuscript, wrote the main manuscript, and reviewed the manuscript draft.
Data Availability
The author declares that the data used in this study are available in the article. In addition, detailed questionnaire responses are available from the corresponding author on reasonable request after removing any sensitive information based on data privacy laws in the Kingdom of Saudi Arabia.
Funding
No funds, grants, or other support was received for this work.