Multi-Sensor Ensemble Classifier for Activity Recognition

doi:10.4236/jsea.2012.512B022

Paper Menu >>

Journal Menu >>

A Journal of Software Engineering and Applications, 2012, 5, 113-116

doi:10.4236/jsea.2012.512b022 Published Online December 2012 (http://www.scirp.org/journal/jsea)

Multi-Sensor Ensemble Classifier for Activit y Recognitio n

Lingfei Mo1, Shaopeng Liu2, Robert X. Gao2*, Patty S. Freedson3

1School of Instrument Science and Engineering, Southeast University, Nanjing, China; 2Department of Mechanical Engineering,

University of Connecticut, Storrs, CT, USA; 3Department of Kinesio l ogy, University of Massachusetts, Amherst, MA, USA.

Email: [email protected]

Received 2012.

ABSTRACT

This pap er presents a multi-sensor ensemb le classifier (MSEC) for phys ical activity (P A) pattern recognition of human

subjects. The MSEC, developed for a wearable multi-sensor integrate measurement system (IMS),combines multiple

classifiers based on different sensor feature sets to improve the accuracy of P A type id ent ificatio n.Expe rimenta l e va lua-

tion of 56 subjects has sho wn that the MSECi s mo re effectivein a ssessing activities o f varying intensitiestha n the tradi-

tional homogeneous classifiers. It is able to correctly recognize 6 PA types with an accuracy of 93.50%, which is 7%

higher tha n the non-ensemble support vector machine method . Furt hermo re, the MSECi s effe ctive in red ucing the s ub-

ject-to-subject variabilityin activity recognition.

Keywords:Physical Activit y Assessment;Multi-Sensor Ensemble;Support Vector Machine.

1. Introduction

Physical activity ( PA), d efined as bodi ly mo vement gen-

erated by skeletal muscles [1]such as walki ng, jog gingor

sport activities, is important for maintaining health and

preventing cardiovasc ular diseases, diabetes, and obesity.

Accurate monitoring and assessment of PA under free-

living conditions provides information on the type and

intensity of activitie s that the pe rson has been engage d in,

thus is of significant interest to the research community

[1] and commercial co mpanies.

The goal of PA assessment is to recognize the type,

dura tio n, and i nte ns i t y of a b roa d ra nge o f physical activ-

ities and quantify the energy expenditureof the test sub-

jectdur ing his/her d aily life, as illustrated in Figure 1. In

recent years, multi-sensor systems have been increasing-

ly inve sti gated for PA assessment. For example, multiple

accelerometers have been placed at different loc ations on

the bodyof test subjects [3]or combined with other types

of sensors, such as respiratory sensor or GPS [4] for PA

measurement. Combining advanced computational tech-

niques such as machine learning and sensor fusi o n [3,4],

differentiation of various activities has shown to be im-

proved. For example, a multi-sensor integrate measure-

ment system (IMS) was developed with two accelero-

meters and one ventilation sensor to measure and assess

the physical act ivity [5].

Rec en tly , ens em bl e lea rn ing has been in cre as ing ly inve st-

tigated for pattern recognition [6]. An ensemble learning

method co mbines multiple individual clas sifier s to ob tain

better predictive performance than that obtained by any

of the constituent classifiers [7]. T his is a technique that

usually combines a number of weak classifiers together

to produce a strong classifiers. Through combination of

decisions from multiple classifiers or multiple sensors,

recognition accuracy has shown to be improved effect-

tively [6]. For example, Ravi et al. combined multiple

different clas sifiers to identify eight common activities of

two subjects with a single tri-axial accelerometer [8].

These classifiers used the same four-feature (Mean,

Standard Deviation, Energy and Correlation) datasets,

and had obtained good recognition accuracies, especially

by means ofMajority Voting. Lester et al. used AdaBoost

[9] to select features and combined multiple weak classi-

fiers, each of which accepting only a single feature as

Figure 1. Physical activit y assessment.

Multi-Sensor Ensemble System for Human Physical Activity Recognition

114

input, and obtained good classification result from a

weighted combination of the weak classifiers[10]. The

eight sensors,inlcuding accelerometer, audio sensor, IR/

visible light, high frequency light, barometric pressure,

humidity, temperature and compass, were integrated in a

unit a t ta c hed on t he shoulde r of the test subject, a nd te sted

by two subjects. For multi-sensor measurement system,

combining different sensors at different body locations

would reveal different characteristics of body movement

and have different statistical distribution. Therefore,

combining different classifiers based on the datasets of

different sensors would have better identification results

than a homogeneous classifier.

In this paper , a multi -senso r en se mble classifier (MSEC)

for PA type identification is presentedfor a multi-sensor

integrated measurement system (IMS) [5]. It combines

multiple classifiers of SVM, based on different sensor

datasets of the IMS. Due to different PA type identifica-

tion accuracies of the different classifiers, an instance

specific weight majority voting is proposed for the clas-

sifier combination. The performance of the MSECis ex-

perimentally evaluated by 56 human subjects performing

free-living acti vities.

2. Ensemble Learning

2.1. System Design

The architecture of the multi-sensor ensemble classifieris

sho wn in Figure 2. The sensor sets for ense mble are gen-

erated by choosing different sensors of the multi-sensor

measurement system. Features corresponding to these

sensor datasets are then extracted and selected. Multiple

classifiers with different feature selection can be derived

from each sensor set, and the diversity of the classifiers

can be achieved by choosing the sensor set and feature

combinations. For each classifier, a learning model is

first selected and trained, with a part of the sensor data as

testing dataset to evaluate the classifier. Each classifier

has a decision result, and the final decision is thus ob-

tained b y combining the classification results from all the

classifiers by the weight majority voting.

2.2. Individual Classifiers

Seven clas sifiers with di f f e r en t sens o r d a tasets were devised

for the e nsemble system based on the three sensors in the

IMS. Each sensor data set consists of a cluster of classi fiers,

including 1) C1 (classifier cluster from the wrist

accelerometer dataset), 2) C2 (classifier cluster fro m the

hip accele ro meter da ta set), 3) C 3 (clas sifier cluster fro m the

abdominal ventilation sensor dataset), 4) C4 (classifier

cluster from the hip accelerometer and the wrist accelero-

meter datasets), 5) C5 (classifier cluster from the hip

accelerometer and the abdominal ventilation sensor

datasets), 6) C6 (classifier cluster fro m the wrist accelero -

meter and the abdominal ventilation sensor datasets), and

7) C7 (classifier cluster from the hip and wrist accelero-

meter and the ab d ominal ve ntilation sensor datasets). Each

classifier cluster consists of multiple (n) classifiers by

different feature selection (random selection). As a result,

a total of 7 × n classifiers and 7 × n testing res ult s can b e

obtained. The final ensemble decision is obtained based

on these 7 × n classifiers by instance specific weighted

majo r it y voti ng.

Figure 2. Multi -sensor ensemble system block diagram.

Multi-Sensor Ensemble System for Human Physical Activity Recognition

115

2.3. Activity Recognition

For each single sensor, 7 time-domain features, namely

the 10th, 25th, 50th (median), 75th, and 90th per centiles,

the mean value, and the standard deviation, were ex-

tracted. In additio n, a correlation feature between the hip

accelerometer and the wrist accelerometer was also ex-

tracted, providing a measure for the coordination or vari-

ation between the upper limb and the body during an

activity. For each accelerometer, two frequency domain

features, energy and entrop y were extracted. For the ven-

tilation senso r, the dominant f requency of the resp iratory

signal obtained from a spectral analysis was extracted as

the breathing frequency. These features were computed

for every 30-second data segment, and linear scaling was

then applied to the extracted features in the range of [0,

1], to avoid that features of greater numeric values would

overwhelm those in the smaller numeric ranges.As a re-

sult, a total of 63 features (50 time-domain and 13 fre-

quency-domain features) were extracted. To achieve the

diversity of the input of each classifier, 70% of the fea-

tures were selected randomly from the overall feature

sets for training classifiers.

The SVM algorithm was chosen as the base classifier

of the ensemble system, and the selected features were

used as inputs to the SVM classifier. A two-step proce-

dure was taken for pr edicting the types of physical activ-

ity. First, a training data set that consists of the selected

features from all the 56 subjects but one was constructed

for building the SVM model and selecting the penalty

parameter and Gaussian kernel parameter. The model

parameters were selected through a “grid-search” with

5-fold cross validation. The parameters that yielded the

highest recognition rate were chosen during the process.

Second, upon completion of the training, the SVM model

was applied to the feature set of the subject that was left

out in the training process, to predict the activity type

reflected in the 30-second data segments. Such a two-

step procedure constitutes a “leave-one -subject-out” cross

validation, and was executed on each subject data.

3. Experimental Evaluation

3.1. Design of Experiments

A total of 56 subjects (26 male and 30 female) were re-

cruited forphysical activity assessment, with the follo w -

ing characteristics, expressed in terms of the mean ±

standard de viation:

1) age = 38.7±11.6 years,

2) mass = 71.1±14.5 kg,

3) height = 169.3±9.1 cm and

4) body mass index = 24.7±4.2 kg/m2.

Each subject performed 6 types of activities of varying

intensities, which are commonly seen in daily lives as

illustratedin Table 1. For each subject, the actual PA

types and times performed by the subjects were recorded,

and sensor data whendifferent PA types were performed

were collected by the IMS (as shown in Figure 2) cor-

respondingly [7]. Each PA type was performed for 7

minutes, followed b y a 2-minute rest period.

3.2. Individual Classifier Results

In order to ensemble different classifiers by the instance

specific weight majority voting method,it is necessary to

first investigatethe accuracies of the different classifier

clusters. Furthermore, since these classifiers use features

from different sensor datasets, they yield different accu-

racies and confidences on identifying the PA types. The

average accuracies of the classifiers on the different PA

types are first calculated, as s hown in Table 2. It i s seen

that these classifiers have yielded different accuracies on

PA type identifications, which is due to the fact that dif-

ferent sensor combinations are sensitive to different PA

t yp e s.

3.3. Ensemble Results

The performance of the ensemble system is based on the

number and accuracies of the classifiers integrated for

the ensemble learning. Various ensemble classifiers had

been evaluated. T he definitions of the different ensemble

classifiers are: E1–E7 integrate the classifiers within

each sensor cluster C1-C7, respectively, E8 integrates

classifiers of C1, C2 and C3, E9 integrates classifiers of

C4, C5 , and C6, and E10 integrates classifiers C1-C7.

Table 1. Physical Activities types for testing.

Activities Category Abbr.

Computer work Sedentary activity CW

Moving boxes Household and other MB

Cycling with 1-kp resistance C1

Treadmill a t 3.0 mph Moderate locomotion T3

Treadmill at 4.0 mph 5% grade Vigorous activity T4-5

Tennis

Table 2. Classification accuracies of different sensor

classifiers on differ ent PA types

PA T ypes

CW MB C1 T3 T4-5 TE

Classifier Cluster

89.5%

94.7%

81.6%

80.1%

61.7%

72.1%

C2 88.2% 71.7% 86.2% 91.7% 86.0% 71.5%

46.8%

54.5%

37.7%

47.2%

37.9%

25.6%

C4 91.2% 87.1% 89.5% 91.4% 86.2% 73.4%

83.8%

73.0%

87.4%

91.7%

85.4%

70.9%

C6 91.2% 76.3% 81.3% 82.7% 66.5% 96.7%

C7 92.9% 72.9% 91.0% 94.0% 92.4% 85.4%

Multi-Sensor Ensemble System for Human Physical Activity Recognition

116

Figure 3. E nsemble classifier result comparison.

Figure 3 illustrates t he P A classi ficatio n results of the

various ensemble classifiers E1-E10. It is seen that in

general, the more classifiers and sensor datasets are in-

cluded in the ense mble classifier, the better the result has

been. For example, classifier E10 integrates all the clas-

sifiers (total 21), and hasyielded the best classification

mean accuracy of 93.5% with the smallest standard devi-

ation of 11.6%.

4. Conclusions

In this study, a multi-sensor ensemble classi fier is designed

for physical activity recognition, as a critical component

of a multi-sensor integrated measurement system. Spe-

cifically, MSEC integrates data measured by three sen-

sors, and fuses the multiple classifiers by performing

instance specific weight majority voting. Compared with

non-ensemble classifier, the MSECmethod has shown

higher mean accuracies and lower standard deviations,

thus d emonstrateing better ge ner a liz a tion capabilit y.

Altho ugh promising, the MSEC is generally computa-

tional ly int ensi ve, r eq uir ing more computational resourc es

than non-ensemble single classifier to achieve good per-

formance. Furthermore, there are still questions that re-

maine d unans wered , e.g. , 1) ho w many se nsor s and wha t

type of sensors (including the locations where the sen-

sors are attached) are required for PA assessment, and 2)

what type of features and classifiers are suitable and op-

timal for the e nsemble system. Research is being contin-

ued to systematically investigate the developed MSEC

algorithm in terms of effectiveness and computational

efficiency for improved PA classification.

5. Acknowledgement

The aut hor s gr at ef ull y ackno wl ed ge fu nd in g p r ovi de d for

this research by the National Institutes of Health under

Grant UO1 A130783.

REFERENCES

[1] C. Caspersen, K. Powell, and G. Christenson, “Physical

activity, exercise, and physical fitness: definitions and

distinctions for health-related research”, Public health

reports, Vol. 100, No. 2, pp. 126-131, 1985.

[2] D. Hendelman, K. Miller, C. Bagget, E. Debold, and

P. Freedson, “Validity of accelerometry for the assess-

ment of moderate intensity physical activity in the field”,

Medicine and Science in Sports and Exercise, Vol. 32, No.

9, pp. 442-449, 2000 .

[3] L. Bao, and S. Intille, “Activity recognition from us-

er-annotated acceleration data”, Pervasive Computing,

Vol. 3001, pp. 1-17, 2004.

[4] M. Ermes, J. Parkka, J. Mantyjarvi, and I. Korhonen,

“Detection of daily activities and sports with wearable

sensors in controlled and uncontrolled conditions”, IEEE

Transa ctions on Information Technology in Biomedicine,

Vol. 12, No. 1, pp. 20-26, 2008.

[5] S. Liu, R. Gao, and P. Freedson, “Design of a wearable

multi-sensor system for physical activity assessment”,

Proc. IEEE/ASME Int. Conf. on Advanced Intelligent

Mechatronics, pp. 254-259, 20 10.

[6] J. Lester, T. Choudhury, and G. Borriello, “A practical

approach to recognizing physical activities”, Pervasive

Computing, pp. 1-16, 2006.

[7] L. Breiman, “Bagging predictors”, Machine learning,

Vol.2 4, N o. 2, pp. 123-140, 19 96.

[8] N. Ravi, N. Dandekar, P. Mysore, and M. Littman, “Ac-

tivity recognition from accelerometer data”, Proceedings

of the Seventeenth Conference on Innovative Applications

of Artificial Intelligence(IAAI 2005), pp. 1541-1546,

2005.

[9] M. Gashler, C. Giraud-Carrier, and T. Martinez, “Deci-

sion tree ensemble: small heterogeneous is better than

large homogeneous”, IEEE Seventh International Confe-

rence on Machine Learning and Applications, San Diego ,

CA, USA, pp. 900-905, 2008.

[10] D. Opitz, and R. Macli n, “Popular ensemble metho ds: An

empirical study”, Journal of Artificial Intelligence Re-

search, Vol. 11, No. 1, pp. 169-198, 19 99.