As the demand for intelligent vision-based human and computer/machine interfaces continues to grow, more intuitive and cost-efficient alternatives to conventional human-machine interfaces such as mice, keyboards, and touch-screens are required. Examples of such interfaces are vision-based hand gesture recognition system and handwriting recognition systems [1] - [3] . In sign language recognition, hand gestures are interpreted as symbols and words. On-line handwriting recognition systems provide a natural, convenient, and touch-free interface for human-computer interaction. This opens the door to new wireless character-inputting methods. Human fingertip can be used as a pointer input interface in place of a pen or a mouse, making the interaction/interface more user- friendly. A fingertip writing recognition system has applications, such as virtual mouse, signature input device and application selector.

In general, a visual fingertip/handwriting recognition system consists of 3 modules: visual data acquisition, feature extraction, and handwriting recognition. Based on a user’s body position, 3D trajectories of the human body, arm and/or hand can be reconstructed [4] [5] . An easier way for human hand or fingertip movement detection is based on image segmentation methods, such as skin color segmentation, background subtraction, etc. [6] - [8] . The connecting vectors between sampled points of the trajectory serve as features for the handwriting recognition system. The key-features selected are trajectory points, tangent angles, curvatures, etc. [8] [9] . Commonly used handwriting recognition techniques include template matching, DTW (dynamic time wrap) based KNN classifiers, hidden Markov models (HMM), neural networks, etc. [5] [7] [9] [10] .

Most earlier handwriting or fingertip-writing character recognition interfaces are built on PC-based systems with image input from USB cameras or Microsoft Kinect RGB-D sensors. This work focuses on an entirely FPGA based implementation with input from a CMOS image sensor. The proposed system achieved recognition accuracy of 100% for digits and lower case alphabets, and it could deal with scaled finger motion. One requirement of the system is that a fingertip blue cover is mounted on the user’s finger and is visible to the camera.

The fingertip detection image processing module captures real-time images with a CMOS image sensor, image capture is followed by fingertip detection, character stroke tracking, and feature points recording. Figure 4 shows the functional block diagram of fingertip detection image processing module. The processing pipeline includes color space transformation, histogram equalization, color detection, filtering, object tracking and recording.

The RGB to YCbCr color transform is defined as

where and. Color space transform is followed by histogram equalization for the component. Let histogram density function of be denoted by

where is the total number of pixels, then the histogram equalization function is

Then, the fingertip blue-cover color segmentation is performed as

where denotes the component after histogram equalization. Finally, a rank order filter in a window,

is applied twice to generate binary image, where pixels in the window are numbered from 1 to 25 from left to right and then top to bottom, with being the center pixel. True pixels indicate potential fingertip positions. The detected fingertip position is recoded and tracked at a frequency of 7 times per seconds.

The proposed fingertip-writing visual character recognition system is implemented as a dedicated logic circuit on a FPGA chip. Real-time image input is feed to the FPGA chip line by line. Up to 5 rows of a image are stored in line-buffers (FIFO) in a pipelined fashion. The image processing pixel clock is 96 MHz. The raw image data is captured by a color camera with resolution and a frame rate of 7 fps (frames per second). The raw image is converted to a 24-bit RGB image (by a local neighborhood of every four scanned pixels). Figure 5 shows the image processing pipeline for fingertip detection and tracking.

Two mathematical functions, arccos(x) and arctan2(y,x), are required when computing a character stroke's feature angles. The arccosine function is implemented using a lookup-table with a resolution of 1.0 degrees. CORDIC algorithm [11] with 12-bit input data is applied to compute tangent angles. Figure 6 shows the pipelined CORDIC design for computing first-quadrant inverse tangent on the FPGA chip. The four-quadrant inverse tangent function can then be obtained from Equation (3). The synthesis results for all character recognition system architectures are shown in Table 1. The overall percentages of total FPGA resource are enclosed in parenthesis.

Figure 7 shows a typical fingertip detection and character recognition execution. The start of the fingertip writing process is triggered by moving the fingertip to a start button box shown on the LCD screen. The recognition system then starts to track and record a character stroke after 4 seconds. The end of a stroke is signaled by holding the fingertip stationary for 2 seconds. The recognition result is shown on the LCD screen thereafter. The entire process takes about a few seconds. Figure 8 shows a 2-character writing recognition experiment, in which the recognition of the second character starts right after the first character is recognized by 3 seconds.

The following character stroke pairs are similar and are more difficult to distinguish, hence the requirement for special distinction mechanisms.

(1) Digit 0 and alphabet o: digit 0 is written clock-wise, and alphabet o counter clock-wise.

(2) Digit 2 and alphabet z: alphabet z is written with an additional up-tick stroke.

(3) Digit 5 and alphabet s: alphabet s is written with a smoother curvature.

(4) Digit 9 and alphabet g: alphabet g is written with an additional up-tick stroke.

(5) Alphabets f and t: alphabet f is finished with an left-up tick stroke.

Figure 9 shows the experimental sample character strokes of digits 0 - 9 and lower case alphabets a - z. The

proposed system demonstrated 100% recognition accuracy in our experiments. We tested each sample character 10 times (a total of 360 character strokes), and no recognition error occurred.

We have presented a simple but effective vision-based fingertip writing character stroke recognition system. The proposed system is implemented on a single FPGA chip with input from a CMOS image sensor. The recognition system achieved accuracy 100% for digits and lower case alphabets, and it could deal with scaled finger motion. One additional requirement of the system is that a blue cover is mounted on the user’s fingertip and is visible to the camera. In future work, we would like to perform recognition of words with a few characters.

This work is supported by Taiwan Ministry of Science and Technology grants MOST-103-2221-E-011-101 and MOST 104-2221-E-011-035.

Ching-Long Shih,Wen-Yo Lee,Yu-Te Ku, (2016) A Vision-Based Fingertip-Writing Character Recognition System. Journal of Computer and Communications,04,160-168. doi: 10.4236/jcc.2016.44014