Combination of the Kinect with Virtual Reality in Balance Training for the Elderly

doi:10.4236/eng.2013.510B037

Paper Menu >>

Journal Menu >>

Engineering, 2013, 5, 171-175

http://dx.doi.org/10.4236/eng.2013.510B037 Published Online October 2013 (http://www.scirp.org/journal/eng)

Combination of the Kinect with Virtual Reality in

Balance Training for the Elderly

Wei-Min Hsieh1,2, Chih-Chen Chen3, Shih-Chuan Wang4, Yu-Luen Chen4,5*,

Yuh-Shyan Hwang6, Jin-Shin Lai7

1Graduate Institute of Computer and Communication Engineering, National Taipei University

of Technology, Taipei, Chinese Taipei

2Department of Electronic Engineering, Hwa Hsia Institute of Technology, Taipei, Chinese Taipei

3Management Information System, Hwa Hsia Institute of Technology, Taipei, Chinese Taipei

4Department of Computer Science, National Taipei University of Education, Taipei, Chinese Taipei

5Department of Information Management, St. Mary ’s Medicine, Nursing and Management College, Yilan, Chinese Taipei

6Department of Electronic Engineering & Institute of Computer and Communication,

National Taipei University of Technology, Taipei, Chinese Taipei

7Department of Physical Medicine and Rehabilitation, National Taiwan University Hospital,

Taipei, Chinese Taipei

Email: *[email protected]

Received May 2013

ABSTRACT

Daily life movements such as standing, walking, and jumping need balance ability to achieve. Good balance control is

closely related to the body stability and its development. According to medical research, people showing dizziness after

taking drugs may have their balance ability substantially affected, and are more likely to fall, especially true to the el-

derly. This study proposes the combination of Kinect with vir tual reality to build an information platform of interactive

scenarios, for practices and evaluation on balance ability. Based on the indicators of balance ability, this platform sets

out various training activities to improve the balance ability, making the supposedly boring process fun and vivid for a

much better training purposes. Also, according to literatures, training with gaming patterns results in 30% reduction of

falls. What’s more, this type of training can have the data, like time of use, scores, and joint postures, recorded and sent

through network to databases on remote computer servers. The data collected from this platform can be sorted and an a-

lyzed, and the results can then be used to evaluate the performance of the balance training, and referenced for follow-up

planning in the future.

Keywords: Balance; Kinect; Fall; Virtual Reality

1. Introduction

Balancing means the ability to maintain stability in all

kinds of movements and postures. Most human move-

ments rely on the foundation of balance, and keeping

body balanced is critical to human movements. Body

movements are pulled by gravity; a movement can be

done only after the balancing point is achieved. Although

normal people have decent balance ability to deal with

daily needs, dizziness caused by diseases or drugs—es-

pecially taking multiple drugs, such as sleeping pills and

tranquilizers, can bring about fall incidents.

Bureau of Health Promotion, Department of Health,

points out in its Elderly Health Promotion Plan (2009-

2012) that falling is one of the fetal causes of the elderly

illness and death (51.0%). Elderly people over the age of

65 are the group with highest risk of fall death. People

prone to fall are handicapped in activities of daily living

(ADL) and instrumental activities o f daily livin g (IADL).

Researches also point out that 42.3% of the elderly

people who suffer fall injuries need hospitalization, and

37.0% of them need to stay at nursing homes after the

clinic therapies. Also, statistics show that in 2008 there

are about 125,000 patients hospitalized for fall injuries.

All the evidences tell the tru th that aging does make sub -

stantial impact on the ability of balance control [1]. This

study proposes the combination of Kinect with virtual

reality to build an information platform of interactive

scenarios, for practices and evaluation on balance ability.

Based on the indicators of balance ability, this platform

sets out various training activities to improve the balance

ability, making the supposedly boring process fun and

vivid for a much better training purpose.

*Corresponding a uthor.

W.-M. HSIEH ET AL.

172

In his studies on older people’s falling, Robbin et al.

(1989) argues that abnormal gaits are in connection with

the falls [2].

Daubney et al. (1999) studied 50 elderly people over

the age of 65, and analyzed the relationship between their

lower limb muscle strengths and balances, and the results

showed that the ones who fell without obvious reasons

obviously had less balance ability than the ones who

never fell, and the lower limb muscle strength was

closely related to the balance ability; therefore, the mus-

cle strength measurement index provides a very high

prediction performance [3].

Robertson et al. (2002)’s research for a community

showed that the elderly an d people who fell before could

go through exercise training to reduce the incidents of

fall and alleviate fall injuries as well [4]. Ledin et al.

(1990)’s research also showed that the elderly undertak-

ing a period of time of balance training could have their

balance ability improved to prevent fall and bone fracture

[5].

In his study, Belgen et al. (2006) argues that individual

body’s balance ability is related to the probability of fall

incidence, and people who suffer more fall incidents

have less balance abilities [6].

D. Taylor et al. (2004) point out certain programmes

of exercise such as the Falls Management Exercise

(FaME) and Otago Exercise (OEP) programmes are ef-

fective in returning falls patients to normal functional

movement and can reduce the risk of falling by as much

as 30% [7].

R. Y. Wang et al. point out in their research projects

(NSC100-2314-B010-022-MY2): falls among the elderly

often lead to injury and thus lead to other complications.

Exercise can prevent falls in older people. Previous stu-

dies also suggested integrated exercise training program

was better than a single exercise. For healthy older adults,

combining virtual reality (VR) with integrated exercise

training program can enhance their motivation [8].

Y. R. Yang et al. point out in their research projects

(NSC100-2314-B010-021-MY2): Exercise has beneficial

physiologic effects in older adults, including effects on

strength, aerobic capacity, and balance. These effects can

reduce the risk and rate of falling among older adults. To

effectively improve physical activities and to reduce risk

of fall, appropriate exercise programs for specific fallers

are needed [9].

2. Material and Method

Figure 1 is an overall schematic illu s tration, where Unity-

3D and Windows SDK software tools in combination

with Kinect sensors are used to show body movements

on a notebook computer at the other side.

Figure 2 shows the system structure of this study,

Figure 1. Overall schematic diag ra m.

Figure 2. System structure diagram.

where the development process includes the hardware

connections, signals receiving and analyses, movement

identifying, etc. The hardware used is Kinect (for Win-

dows) developed by Microsoft. The software application

is developed with the Microsoft officially released Win-

dows SDK software tool along with Unity3D, a 3D-

enabled game development software tool. The Windows

SDK tool includes all the needed documents and API

libraries and header files, while Unity3D is a compre-

hensive, 3D-game and 3D interactive tool for creativity

development.

Kinect is used to detect human skeleton, and the de-

tected 3D in-depth images are converted to the skeleton

tracing system, which is capable of tracing up to 20 ske-

leton joints of the human body. The coordinates of the

detected skeleton joints are then reflected in the 3D

scenes developed with Windows SDK programs using

Unity3D’s libraries to visualize the body movements and

present the visual reality simulation.

This study mainly focuses on fall prevention, and the

proposed training and test items are designed by taking

references to the Four Square Step Test (FSST) method

often used for examining balance ability; one of the de-

signed activities is using the hand or foot or the combina-

tion of foot and hand to touch the random balls for the

balance training or test. The lead preparation for the

W.-M. HSIEH ET AL.

173

training and test activities is shown in Figure 3, where

the medical personnel needs to pre-select the difficulty

level, applicable module, evaluation method, and suitable

postures.

The degree of difficulty of the stretching movements is

defined by the participant’s skeleton. To take into ac-

count safety during the activities, Figure 4 demonstrates

the scope of the stretching movements, where the 50%,

75%, and 100% degree between SHOULDER CENTER

and WRIST are easy, medium, and difficult stretching,

respectively, which are corresponded with the random

balls appearing at specific spots designed by the software

programs.

The program needs to be initialized before the training

or test starts, i.e. benchmarking the participant’s position

and skeleton, followed by limb stretching warm-up be-

fore entering the training procedure. Figure 5 is a block

diagram showing the initialization, warm-up and training.

The activities are conducted in a game-like manner in

order to increase the participant’s interest in doing the

activities. This system also provides the training data that

can be used for evaluation on the training effectiveness.

Figure 6 is a block diagram of the test, which r equires

the lead process and initialization before being com-

menced. The result can be based on fixed time to calcu-

late the scores or on fixed scores to calculate the time.

3. Results

This study takes an empirical test on Mr. A of 67 years of

age, who is 178 cm high, and weighted 83 kg. With the

developed virtual-reality training program, the tested

male uses his hand or foot to touch the random balls, and

the time used to finish the given number of touches on

the balls is measured for the balance training.

Figure 3. Lead preparation for the training and test activi-

ties.

Figure 4. Th e scope of the st ret ch i n g m o v em ents .

Figure 5. The initialization, warm-up and training block

diagram.

Figure 6. Test block diagram.

Figure 7 shows the screen of the hand touching the

ball, where the ball appears on the upper right corner,

and the tester sc ore s with his ri g ht hand touc hi ng the ball .

On this screen, (a) shows the right hand has not yet

touched the ball, and (b) shows the right hand are touch-

ing the bal l.

A training program was set as a six-week session, and

the difficulty level was easy palm or foot exercise. Each

of the exercises was given 3 games, with a 2-minute in-

terval between games. There was a 5-minute break be-

fore switching to a different touch method. In each game,

the random ball showed up 5 times, and each of which

was to be touched by a specific limb. The time used for

the games was calculated. And the times used were av-

eraged by each different touch method. Table 1 lists the

results from the elderly Mr. A’s training, which shows

the times used for these two training methods, and the

W.-M. HSIEH ET AL.

174

Table 7. The screen of the hand touching the ball.

Table 1. Lists of the training results.

Week/Limb Week 1 Week 2 Week 3 Week 4 Week 5 Week 6

Hand 35 sec 34 sec 32 sec 32 s ec 31 sec 29 s ec

Foot 37 sec 36 sec 36 s ec 35 sec 33 sec 32 sec

results from adjacent weeks show either improved or

unchanged. Also, Mr. A’s training records of the times

used by his palm and foot show that his foot reaction is

weaker than his palm. An interview with elderly Mr. A

reveals that his foot received medical treatment because

of sports injuries, and that’s why his foot is weaker in

reaction. Figure 8 shows the averaged weekly time used

for his hand or foot to touch the random balls.

4. Conclusion

This study proposes the combination of Kinect with vir-

tual reality to build an information platform of interactive

scenarios, for practices and evaluation on balance ability.

A clinical assessment test was given to another elderly

person, and it was also proved that after the 6-week

training, the tester obviously made progress in reactions.

From the training results of the participants, we know

that the information platform developed for enhancing

the balance ability is in line with the practical needs, and

the fun and interesting game-like exercises it introduces

are much helpful to the improvement of balance ability,

and certainly to the prevention of falling. In the future,

more tests w ill be given on the elderly in order to collect

(a)

(b)

Figure 8. The averaged weekly time used for hand or foot.

(a) theaveraged weekly time of hand; (b) the averaged

weekly time of foot.

more effective and comprehensive quantitative data for

the system’s fine tuning and modifications.

5. Acknowledgements

The authors wish to thank the National Science Council

(NSC) in Taiwan (Grant Number: NSC 101-2221-E-146

-001-, NSC101-2221-E-152-001-, NSC101-2627-E-002-

006-) for support this research.

REFERENCES

[1] Bureau of Health Promotion, Department of Health, Chi-

nese Taipei.

http://www.bhp.doh.gov.tw/bhpnet/portal/PressShow.asp

x?No=200911060002

[2] A. S. Robbin and L. Z. Rubenstein, “Predictors of Fall

among Elderly People,” Archives of Internal Medicine,

Vol. 49, 1989, pp. 1628-1633.

[3] M. E. Daubney and E. G. Culham, “Lower-Extremity

Muscle Fore and Balance Performance in Adults Aged 65

Years and Older”, Physical Therapy, Vol. 79, No. 12,

1999, pp. 1177-1185.

[4] M. C. Robertson, A. J. Campbell, M. M. Gardner and N.

Devlin, “Preventing Injuries in Older People by Prevent-

ing Falls: A Meta-Analysis of Individual-Level Data,”

Journal of the American Geriatrics Society, Vol. 50, 2002,

pp. 905-911.

[5] T. Ledin, A. C. Kronhed, C. Moller, M. Moller, L. M.

Odkvist and B. Olsson, “Effects of Balance Training in

Elderly Evaluated by Clinical Tests and Dynamic Petro-

graphy,” Journal of Vestibular Research, Vol. 1, No. 2,

1990, pp. 129-138.

[6] B. Belgen, M. Beninato, P. E. Sullivan and K. Nariel w al-

la, “The Association of Balance Capacity and Fall s Self- -

efficacy with History of Falling in Community-Dwelling

People with Chronic Stroke,” The American Congress of

Rehabilitation Medicine and the American Academy of

W.-M. HSIEH ET AL.

175

Physical Medicine and Rehabilitation, Vol. 87, 2006, pp.

554-561.

[7] D. Taylor and C. Stretton, “The Otago Exercise Program,

An Evidence-Based Approach to Falls Prevention for

Older Adults Living in the Community,” Journal of Pri-

mary Health Care, Vol. 31, No. 6, 2004.

[8] R. Y. Wang, “Virt ual Reali ty-Enhanced Optimal Exercise

Training Model for Falls Prevention and Its Remote Ap-

plications in Elderly with Different Fall Risks,” NSC

100-2314-B010-022-MY2, 2011.

[9] Y. R. Yang, “Optimized Exercise Model and Remote

Training for Fall Prevention in Elderly (I),” NSC100-

2314-B010-021-MY2, 2011.