Building an AI-Integrated Braille Recognition Model for Blind Education: From Mechanical Devices to Personalized Learning Systems ()
1. Introduction
Inclusive education aims to ensure that all children, including students with visual impairments, have access to the mainstream curriculum and appropriate learning resources. In Vietnam, the learning needs of the visually impaired are extensive. However, there are still many barriers in terms of teaching materials and media, especially when Braille books are expensive to print, have limited durability, and have a limited content base compared to regular text and digital resources. These realities are recorded in inclusive education facilities such as the Da Nang Inclusive Education Development Support Center, where traditional equipment still dominates due to costs and limited access to technology [1].
In this context, the combination of assistive technology (AT) and artificial intelligence (AI) presents a promising new approach. It has the potential to transform plain text documents into multi-sensory learning experiences (tactile, audio) that are personalized to the pace and needs of each learner. The solution framework we are exploring is a comprehensive system that spans from device to software: sensors and cameras capture text, AI processors recognize and convert it, then map it to tactile channels (vibration dot matrix) and voice (TTS). Learning data is saved and analyzed in a learning management system (LMS) to suggest appropriate content [2] [3].
Previous efforts have primarily focused on mechanical improvements to read and print Braille, such as the Braille Master Wheel (BMW) model. This model uses a rotating mechanism and electromagnets to display continuous raised characters, integrating additional calculations and printing. Despite the many advantages, the experimental versions still have significant limitations in size, bulkiness, speed, and the scalability of multimodal sensory content. These limitations underscore the urgent need for tighter integration between haptic hardware and recognition algorithms, language processing, and learning platforms to enhance the learning experience and efficiency [4].
The research gap therefore lies in a unified “mechanical + AI + LMS” model: (i) recognize lowercase text with layout using visual AI (segmented OCR), (ii) convert, translate to Braille and/or haptic signals according to appropriate perception rules, (iii) synchronize with LMS to measure speed, accuracy, detect weaknesses and adapt training parameters (vibration speed, phrase division, reading suggestions). Such architecture overcomes the limitations of single-function devices, while leveraging the lowercase resources that are dominant in schools [5].
Based on the above observations, this study aims to build an AI-integrated Braille recognition and conversion model, aiming at a personalized learning ecosystem for the visually impaired. Specifically, we propose: (1) an end-to-end architecture consisting of a camera/vibration matrix device, an AI Engine (OCR image processing, geometry/color segmentation; tactile mapping), and an LMS (progress analysis, learning suggestions); (2) a real-world image preprocessing and recognition process “camera close to paper”; (3) tactile mapping rules (frequency, intensity, pattern) for letters, geometry, and color; (4) an adaptive mechanism based on learner performance data [6].
The expected contributions of the work include: (i) a feasible technical framework for converting plaintext documents into near-real-time Braille-tactile-audio experiences; (ii) an integrated design that reduces the cost of Braille content and expands the learning resource; (iii) a learning data platform for personalization and early intervention by teachers/parents in inclusive education. The following section will present the theoretical basis, system architecture, and assessment implementation process [7].
2. Materials and Methods
2.1. Theoretical Basis and Proposed Model
Braille was invented by Louis Braille, using different arrangements of 3 rows and two columns numbered from 1 to 6, to replace characters used in language such as letters, tones. Braille is for the blind, felt through the sense of touch with the fingertips. The letters are written on the surface of a special type of paper. To write Braille, a writing board or a specialized typewriter is required (Figure 1).
Figure 1. Braille character encoding structure.
The display device consists of a matrix of dot-shaped actuators placed under the fingertip or at the tip of the pen. A sensor or camera captures text, shapes, and colors at close range to the paper surface. The controller generates pulses in a width modulation pattern to create a stable vibration for each point in the matrix [8].
The system adopts two levels of vibration intensity, called level A and level B. Dots are considered raised when at level A and not raised when at level B. Six points in a cell can vibrate simultaneously at different levels, through which the learner distinguishes dot combinations to recognize characters [9].
This principle extends to geometry by vibrating along the object’s contour and to color perception by reducing it to twelve primary colors and assigning characteristic pulse patterns to each color.
Braille’s six-dot structure allows the dot matrix and vibration mechanism to reproduce characters in near real time, with compact hardware. The expansion into shape and color recognition helps support visual teaching for the blind (Figure 2).
Figure 2. Braille character encoding table.
The model is designed with a three-layer architecture including terminals, artificial intelligence engines and learning management systems. This structure aims to ensure a closed processing chain from field signal acquisition to creating synchronous tactile and audio experiences, while also storing data to serve personalized learning [10]. The three-layer framework and the role of each component are clearly defined in the research proposal on Braille recognition integrated with artificial intelligence for education of blind people (Figure 3).
Figure 3. Order of encoding Vietnamese syllables in Braille.
At the terminal layer, the system uses a camera placed near the paper surface to capture text, shapes, and colors in the learning context; it can also record touch or tap gestures to support segmentation by word or by phrase. The output signal is displayed via a dot matrix and vibrating mechanism to reproduce the feel of raised dots. The processing chain at this layer is coordinated to convert raw images into learning information, then create a tactile and audio experience before synchronizing the data to the learning platform.
The AI engine layer performs image preprocessing and recognition. The input image is cropped for the region of interest near the cursor, contrast is balanced using local enhancement, noise reduction, adaptive thresholding for white paper backgrounds, and then skew and slight distortion correction is performed to stabilize the text [11]. From here, the system segmentes hierarchically from line to word to character or extracts object contours for basic geometries. The text recognition block uses optics with a lightweight and scalable tool using deep learning models deployed through an optimized execution environment when encountering small or complex text. For geometry and color, the system extracts contours and reduces them to twelve basic colors for tactile mapping [12] [13]. The recognition results are translated into vibration rules based on two intensity levels for raised and unraised dots, controlled by pulse width modulation appropriate to the dot matrix.
The learning management system layer takes in all sessions and interaction events for analysis and adaptation. It manages the identities and roles of learners, teachers, and administrators, provides a content repository with authoring tools, and synchronized vibrating and speech previews. Based on historical performance data such as accuracy, reading speed, and common errors, the adaptive module recommends the next lesson and operational parameters such as vibrating speed, field size, and speech cadence using threshold rules and lightweight machine learning models such as nearest neighbor and LightGBM. The system supports offline first and secure sync when connected, allowing teachers to monitor, intervene, and report on a session-by-session or weekly basis.
Overall, the three-layer architecture transforms paper images into near-real-time tactile and audio signals, forming a data loop to optimize individual learning paths. This approach inherits the advantages of Braille displays but overcomes the limitations of bulkiness and content scope by directly connecting to machine learning-based recognition and adaptive learning platforms.
2.2. Technology Design
The system uses an 8GB Raspberry Pi 5 embedded platform as the central processor, connecting a camera placed near the paper surface [14]. Two camera options considered include the OV9281 monochrome camera with full shutter resolution of 1280×800 and the IMX708 12MP supporting high dynamic range, both with fixed M12 lens versions suitable for close-up shooting scenes. For the prototype tested with students, the IMX708 was ultimately selected. The rationale for this choice was its high dynamic range (HDR) capability, which provided more stable and accurate image captured under the variable lighting conditions commonly found in a real classroom environment.
The images captured from the camera are normalized through preprocessing steps using the OpenCV library, including cropping the area of interest close to the cursor, balancing the contrast using the local enhancement method, denoising using a median or bilateral filter, adaptive thresholding for white paper backgrounds, and then skew and slight distortion correction to stabilize the text lines. Next, the system is segmented hierarchically from lines to words to characters or contour extraction for basic geometries. The text recognition block applies to Tesseract optics for fast scenarios. The system uses a confidence threshold to determine the recognition strategy: if Tesseract returns a confidence score below 90% for a recognized character or word, the system automatically escalates the task to deep learning models (CRNN or DBNet) deployed via ONNX Runtime, which are reserved for these more “difficult conditions” such as small letters or unusual fonts [15]. For geometry and color, the system extracts object contours and normalizes them to twelve basic colors as input for the tactile mapping part [16].
The recognition results are converted to Braille characters and represented by the vibration patterns of the dot matrix. The control library uses pulse width modulation to generate a stable vibration sequence according to the hardware characteristics [17]. The intensity convention has two levels, the points are raised when at level A and not raised when at level B, six points in a cell can vibrate simultaneously for the learner to recognize the dot combination. For geometry, the points corresponding to the contour are activated in a predetermined sequence. For color, the homogeneous areas are reduced to twelve basic colors and assigned the appropriate pulse hash pattern [18] (Figure 4).
Figure 4. Schematic diagram of the ESP32-C3-based control circuit.
A learning management system with a simplified user interface, incorporating an AI-based conversational assistant to explain content and guide operations for visually impaired students [19]. An identity and role management system for students, teachers, and administrators, providing a content repository and lesson authoring tools with synchronized vibrating and speech previews [20]. An adaptive learning module that uses historical data such as reading speed accuracy and standard errors, applies threshold rules and lightweight machine learning models such as nearest neighbor and LightGBM to recommend the next lesson as well as operating parameters including vibrating speed, field of view size, and speech tempo. An offline-first architecture is adopted for caching and synchronization when the network is available, along with security and privacy requirements such as transmission encryption and anonymization of sensitive data [21].
2.3. Testing Procedure
The test was conducted at the Da Nang Inclusive Education Center, where the research team conducted a survey and interviews to understand the physical context and learning needs of blind students [22].
The participants included 15 blind students in primary and secondary schools. The sample was selected to represent the reading and math proficiency levels of students at the center.
The assessment tool was a set of tasks developed by the team, including three groups of functions: letter recognition, reading passages, and basic math practice. The tasks were designed based on the popular curriculum at the center, ensuring an increasing level of difficulty to observe progress over a short period of time.
The research methodology uses a mixed approach. The quantitative component uses correct recognition rate and task time index to measure the effectiveness of the system. The qualitative component includes structured interviews and classroom observations to record the perceptions of students and teachers, and to detect obstacles in the operation and usage environment [23]. The interviews and observations were built on the questionnaire prepared during the pre-test survey phase at the center.
The procedure for each session was as follows first, a brief tutorial on hand placement and interaction with the vibration matrix and voice reading. Second, learners completed three sets of tasks in the correct order: reading single letters, reading paragraphs, and basic math, with the system automatically recording the results and timing. Third, learners and teachers completed a short interview about ease of use, appropriateness of vibration and voice speed, and difficulties encountered. Data from the device and interview transcripts were compiled for quantitative and qualitative analysis in the results section. The above steps followed the principles of ensuring safety and consent of participants at the site. This organization of the process allowed for the linking of the assessment of the system’s recognition capabilities with actual classroom usage experiences, thereby creating reliable data for the discussion of effectiveness and pedagogical implications in the following sections (Figure 5).
![]()
Figure 5. System recognition process.
3. Results and Discussion
3.1. Braille Recognition Accuracy
On the test set of basic Braille characters, the system achieved a correct recognition rate of more than 90%. This metric refers to the accuracy calculated as (1—Character Error Rate (CER)). The “basic Braille characters” test set used for this result comprised the 26 letters of the English alphabet and the 10 Arabic numerals (0-9). This result confirms the feasibility of the integration approach between sensors, image processing and machine learning on the selected embedded device configuration [24].
Methodologically, the above accuracy level is consistent with the described architecture: the input image is cropped for the region of interest, contrast balanced, denoised, adaptively thresholded, skewed and distorted; then hierarchically segmented and recognized using optical tools or deep learning models when encountering small or difficult characters. The resulting sequence is mapped to a dot matrix in two intensity levels to reproduce the raised dots for the user [25]. Normalizing the process according to these steps helps reduce noise, stabilize the character structure and thus increase accuracy in experimental situations.
Although the system achieves a high level of accuracy on the basic character set, further expansion of the data and evaluation scenarios is needed to test the system with long texts, accented characters, and varying classroom lighting conditions [26]. This is a priority improvement in the following stages.
3.2. Learning Applications
Use either SI (MKS) or CGS as primary units. (SI units are encouraged.) English units may be used as secondary units (in parentheses). An exception would be the use of English units as identifiers in trade, such as “3.5-inch disk drive”.
Avoid combining SI and CGS units, such as current in amperes and magnetic field in oersteds. This often leads to confusion because equations do not balance dimensionally. If you must use mixed units, clearly state the units for each quantity that you use in an equation.
Do not mix complete spellings and abbreviations of units: “Wb/m2” or “webers per square meter”, not “webers/m2”. Spell out units when they appear in text: “... a few henries”, not “... a few H”.
Use a zero before decimal points: “0.25”, not “.25”. Use “cm3”, not “cc”.
3.3. Comparison with Previous Studies
Previous mechanical systems such as BMW mainly relied on rotating mechanisms and electromagnets to create raised edges. This product line recorded some advantages in terms of cost and continuous display capability. Still, it was limited in terms of bulky size, large volume and limited number of characters displayed simultaneously, so the reading speed did not meet the needs of today’s inclusive classrooms [27] [28].
The proposed model differs in three core points. First, the device layer uses a camera placed near the paper surface combined with a vibrating dot matrix, thereby moving from a heavy mechanical architecture to a more compact sensor and control architecture, allowing the problem to be extended from Braille to shape and color recognition through tactile and speech mapping. Second, the artificial intelligence layer performs a series of image processing and recognition using machine learning algorithms, which helps standardize the input and increase accuracy under real-world conditions. Third, the learning management system layer integrates the principle of personalized learning: collecting performance history and using lightweight algorithms such as nearest neighbor and LightGBM to suggest the next lesson and adjust operating parameters [29]. This approach moves the device from a single-function display to a personalized adaptive learning platform.
4. Limitations Discussion
Although the initial results are promising, the model still has some limitations. In terms of hardware, the device is limited by its lack of compactness. Specifically, the current prototype (including the battery pack) measures approximately 15 cm × 10 cm × 5 cm and weighs around 450 g. This size, along with the dependence on existing components, makes the device suboptimal for long-term use in all classroom contexts. This provides a concrete baseline for the goals of future hardware optimization; the size and dependence on existing components make the device not really compact for long-term use in all classroom contexts. Furthermore, while the Introduction noted the high cost of traditional Braille books, the material cost for our prototype (including the Raspberry Pi 5, IMX708 sensor, and vibrating dot matrix) is estimated at approximately $120. Although this is a non-trivial initial investment, the device is highly scalable. A single unit can process a virtually unlimited library of digital texts, offering significant long-term cost-effectiveness compared to the recurring expenses of printing, binding, and distributing specialized, single-use Braille volumes. In terms of data, the training repository still focuses on.
In the following stages, the research team will prioritize optimizing the hardware design towards being lighter and more durable, while expanding the data set by collecting text and interaction samples under different lighting conditions and reading speeds. Standardization of the synchronization protocol and evaluation criteria will also be implemented to ensure the repeatability and comparability of results between test batches.
5. Conclusions
This study validates the feasibility of an integrated model of AI-based recognition and tactile display for Braille. On a basic character test set, the system achieves high accuracy and stable operation on an embedded device configuration, demonstrating that the processing chain from image acquisition to tactile mapping can be deployed in an inclusive classroom setting [30]. These results are consistent to design an AI-based Braille recognition model to personalize learning for the visually impaired.
The key contribution of the work is to move from a discrete mechanical device approach to a three-layer personalized learning system consisting of a terminal, an AI engine, and a learning management system. This architecture integrates sensor acquisition, machine learning recognition, and vibration and speech coordination, and creates a data loop for adaptive suggestions based on the progress of each learner [31]. As a result, the device goes beyond just a Braille display to become a user-oriented, intelligent learning platform.
In the future, the system could expand its applications to general education, higher education, and vocational training for visually impaired individuals. Key areas for development include collecting multilingual and multi-classroom data, optimizing hardware design for compactness and durability, and standardizing synchronization protocols and evaluation criteria to ensure consistent results across testing rounds. These goals align with the vision of broadly implementing the model in inclusive education [32].