Assessment Center Dimensions Predict Performance-Based Bonus in Business Management Settings

doi:10.4236/psych.2012.36071

Paper Menu >>

Journal Menu >>

Psychology

2012. Vol.3, No.6, 500-503

Published Online June 2012 in SciRes (http://www.SciRP.org/journal/psych) http://dx.doi.org/10.4236/psych.2012.36071

500

Assessment Center Dimensions Predict Performance-Based

Bonus in Business Management Settings

Leehu Zysberg1,2

1Gordon College of Educ a tion, Haifa, Israel

2Department of Psycholo g y, Tel Hai College, Tel Hai, Israel

Email: leehuzysberg@yahoo.com, [email protected]

Received April 8th, 2012; revised May 4th, 20 12; accepted June 1st, 2012

This study sought to add to the literature on the validity of Assessment centers (ACs) by first examining

the factorial structure emerging from observers’ dimension ratings and then examining their predictive

validity using a performance criterion often unavailable to researchers—performance-based bonus pay-

ment. A series of ACs specially designed for the selection of candidates for entry-mid tier management

positions in a large financial corporate (n = 180) was used as the sampling frame. For candidates who

were promoted to managerial position we gathered bonus information within 6 - 12 months of their pro-

motion (n = 75). The dimension ratings and factorial structure of the AC were examined to reveal a

2-factor structure pertaining to cognitive and interpersonal aspects of performance. Both the original di-

mensions and the two factorial grades showed moderate predictive validity using performance-based bo-

nus as the criterion: The ‘organizational commitment’ dimension best predicted bonus payment (r = .38; p

< .01) and the interpersonal factorial grade best predicted bonus (standardized b = .22 p < .01), followed

by the cognitive factor, after controlling for gender and tenure. The theoretical and practical implications

of the findings are briefly discussed.

Keywords: Assessment Centers; Construct Validity; Predictive Validity; Performance-Based Bonus

Introduction

Assessment centers (AC) have earned a mixed bag of re-

views and opinions as instruments of employee screening and

selection. Prominent authors view them as either highly effec-

tive, valid measures of predicting future job-related perform-

ance (e.g.: Cascio, 2010; Schmidt & Hunter, 1998), or a prob-

lematic, unstable method yielding inconsistent content and

construct validity indices (Bowler & Woehr, 2006; Melchers,

Kleinmann & Prinz, 2010). Only few, however, will dispute the

impressive criterion-related validity of ACs. Persistent evidence

gathered across time and various target populations point to the

ability of AC-derived ratings to predict and correlate with a

broad range of job-performance indices (Thornton & Gibbons,

2009). Despite abundant evidence some questions linger re-

garding the validity and relevance of the AC method in HR

selection contexts. This paper depicts a field study aiming at

two of them.

The first issue relates to the question of construct validity in

AC indices: the literature proposes that dimension ratings pro-

duced by observers in ACs do not represent assessment of

separate performance-related entities but rather tend to con-

verge into either a single factor (reflecting perhaps a halo ef-

fect), or a set of factors representing the various tasks in the AC

(For a summary of this issue see Thornton & Gibbons, 2009).

Can observers’ impressions and judgments of participant be-

haviors reflect separate job-related entities?

The second issue pertains to the criteria used in AC validity

studies: While the literature on this subject is quite consistent,

most popular criteria measures are subjective such as job-satis-

faction, supervisors’ evaluations, performance appraisals by the

participants and peers, etc. (Arthur, Day, McNelly, & Edens,

2006; Gaugler et al., 1987). Some studies provide additional

criteria to reflect work-related performance indication such as

salary and organizational hierarchy (e.g.: Bray & Grant, 1966;

Jansen & Stoop, 2001). These however, remain sporadic and

less available in the literature, compared with “soft measures”

as mentioned above.

Using bonus payment as a criterion for job-performance

rarely appears in the literature on AC’s predictive validity since

this information may be sensitive and unavailable to researchers.

It does, however, present a potential as an easily quantifiable

indicator of general job performance in for-profit organizations.

The literature on pay for performance models (e.g.: Werner &

Dudley, 2009) suggests a closely supervised practice, in which

a clear set of performance criteria is used to systematically

determine bonus payments. As such it may be an interesting

criterion in predicting job related performance (Ittner, Larcker,

& Rajan, 1997).

The current study examined a series of ACs designed for the

selection of entry-to-mid level managers from the ranks of pro-

fessional employees in a large financial organization offering

investment, insurance and other related services, headquartered

in Israel. The study investigated both the structure of the mea-

sures produced by the observers in the AC, seeking potential

factors and dimensions underlying their judgments and then

looked at the predictive validity of these factors against a sel-

dom available criterion—performance based bonus payments.

We hypothesized that 1) AC dimensions will show a consistent

factor structure that is not global nor an artifact of the AC tools

and, 2) AC dimensions and factors will predict manageri al per-

formance, represented by performance-based bonus.

L. ZYSBERG

Method

Settings an d S tu dy Design

The study was conducted in a large financial organization

providing investment services to a broad range of clients, in

Israel. The organization employs about 7000 individuals, about

66% of whom are based in 180 branches nationwide. Data col-

lection took place in a series of specially designed assessment

centers used for the selection of Investments department man-

agers. The position is considered mid-tier management and is

open to inside employees only, typically investment profes-

sionals without official managerial background (though many

have some training in management).

Sample

One hundred and eighty candidates took part in ACs for the

above job, after applying for an internal call for candidates, and

passing a basic screening process based on their resumes. Of

the above sample, 75 were placed in managerial positions with-

in 6 month s following the AC. For this sample, ages ranged 29 -

50 (mean age = 41.00; sd = 7.65), 60% were men and 40%

were women. Tenure with the company at the time of applica-

tion ranged 5 - 25 years. They all had at least a Bachelor’s de-

gree and 20% had a graduate level degree or equivalent.

Instruments

Assessment centers were specially designed for this position

by a team of expert Industrial-Organizational psychologists

working for the organization, to reflect behaviors according to

the job description provided by the organization HR division.

The assessment centers included: 1) A self- presentation task in

which participants planned and performed a time-limited self-

presentation; 2) A discussion group, simulating professional

credit-approva l dilemmas in whic h consensus has to be reached

via discussion; 3) A group “in-basket” assignment simulating

daily assignments of an investment department manager and 4)

A competitive ‘branch promotion event’ assignment in which 2

or more sub-groups competed on designing and presenting the

best promotion event for a new investment product. These tasks,

though tailored for the specific job and organization are based

on widely used paradigms (see for example: Bray & Grant,

1966; Thornton & Gibbons, 2009).

Data from 10 groups (total n = 75) was included in this

analysis. The data was summarized across 6 dimensions rated

by 2 observers (one is an HR specialist and the other an I/O

psychologist from outside the organization). The two observers

discussed each candidate after the AC and reached consensus as

for the grade on each dimension, as well as a general recom-

mendation regarding the candidate’s fit for the position. Grades

were given on a Stanine scale. The dimensions were as follows:

1) General cognitive ability: problem solving, effective infor-

mation processing; 2) Work style: planned, orderly perform-

ance, paying attention to technical details while keeping awa-

reness of the group goals; 3) Interpersonal relations: open, ef-

fective and assertive communication, collaboration and sensi-

tivity to others; 4) Service & Sales orientation: service aware-

ness, perceiving client-oriented service as a priority, identifying

and taking opportunities to broaden the business and customer

base; 5) Organizational commitment: Embracing organiza-

tional values and priorities, identification with the organization

and its interests; 6) Leadership potential: initiating action, as-

suming responsibilities and motivating others. These dimen-

sions fit a model of generalized dimension structure presented

in a meta-analysis of a large number of studies of assessment

centers (Arthur, Day, McNelly, & Edens, 2006).

Demographic data was collected via a short questionnaire.

Data analysis included gender, age, and tenure.

Criterion data—bonus for managerial performance: as a cri-

terion for managerial performance we collected data regarding

the managers’ performance-based bonus, at the end of the year

following the AC. Bonus is calculated and paid by the HR divi-

sion, ranges 0 to 30 units, each unit representing a given per-

centage of the employee’s monthly salary. Bonus is paid once a

year only in branches reaching or surpassing their business

objectives and goals. Differential bonus is awarded based on

profits, and performance appraisal provided by the employee’s

supervisor. Though there is room for personal judgment, bonus

sums are monitored carefully by the corporate office and super-

visors are held liable for the fairn ess a nd perf ormance cong ru-

ency of the process. Thus it is assumed that this data is a valid

indication of managerial performance through the corporate

lens.

Procedure

After obtaining the organization’s approval for data collec-

tion, we retrieved data from the corporate files making sure no

identifying markers are left in the data to allow tracing indi-

viduals, thus assuring anonymity.

The data was then analyzed using SPSS 19.0 (IBM, 2012).

Results

Before testing the hypotheses we examined the distribution

of our main variables and the general associations among them.

Table 1 summarizes descriptive statistics and Pearson’s corre-

lations between the study variables.

The results reveal a distribution of grades and the bonus units

allowing for parametric statistical analysis (Coolican, 2010).

The analysis shows that 5 of the 6 dimension ratings in the

AC correlated positively with the bonus criterion. “Organiza-

tional commitment” showed the strongest association while

“cognitive skills” did not associate with the criterion at all. The

correlations reveal association patterns within the AC dimen-

sions suggesting underlying factors. The correlations also indi-

cate moderate associations between various AC dimension

grades, the AC final recommendation grade and the bonus

value.

We then proceeded to conduct an exploratory factor analysis

(EFA) to reveal potential factors within the AC dimensions.

The Varimax rotated model accounted for 68% of the total

variance suggesting 2 factors. Table 2 depicts the EFA results.

The 2 factor structure matches a conceptual differentiation

between cognitive/intellectual aspects and interpersonal aspects

of performance in the AC. We then calculated a mean score for

dimensions loading on the Cognitive factor and the dimensions

loading on the Interpersonal factor. We used these factorial

scores in a regression analysis, as well as tenure and gender to

predict the bonus value. Table 3 summarizes the results of the

analysis.

The results show that a multiple R of .36 (p < .01) was ac-

counted for by the interpersonal and cognitive factor grades.

L. ZYSBERG

Table 1.

Descriptive statistics and pearson’s correlations among the study vari-

ables (n = 75).

Mean

sd 1 2 3 4 5 6 7 8

Cognitive 6.09

.84 -

Work style 5.82

.70 .58** -

Interpersonal 5.84

.71 .09 .30** -

Service & sales 5.50

.77 .23* .31** .58** -

Commitment 6.02

.72 .16* .29** .53** .66** -

Managerial pot 5.99

.75 .52** .44** .26** .44** .27** -

Total AC 5.73

.74 .44** .51** .47** .72** .61** .52** -

Bonus 13.94

6.63 .07 .22* .21* .21* .38** .28** .27** -

*p < .05; **p < .01.

Table 2.

EFA analysis on the AC dimension grades.

Dimension Factor I Factor II

Cognitive .69 –.10

Work style .79 .25

Interpersonal –.15 .85

Service & sales .40 .79

Commitment .03 .76

Managerial pot. .46 .65

Table 3.

Regression coefficients and correlations between the predictors and the

bonus value (in units) (n = 75).

Predictor Simple Pea rso n’s r Standardized

coefficient b

Cognitive Factorial .28** .16*

Interpersonal Factoria l .31** .22*

Gender –.16* –.10

Tenure .07 .05

Multiple R for the en tire model = .36; p < .01; *p < .05; **p < .01.

Gender and tenure did not show any added value in the model,

though simple Pearson correlations were significant for 3 out of

the 4 predictors.

Discussion

The current study tested two hypotheses: The first posited

that AC dimension grades will reflect job criteria rather than a

general impression (a single factor) or factors representing the

various tasks in the AC. This hypothesis was supported by our

analysis, suggesting 2 underlying factors accounting for 68% of

the variance in the AC dimensions. The factors pertained to

skill type (cognitive vs. interpersonal) rather than task type.

These results suggest that AC observers are capable of differen-

tiating between psychological and behavioral qualities via-a-vis

the job description and not necessarily fall victims to the halo

effect or judge each task performance as a stand-alone entity (as

suggested in the literature, e.g.: Bowler & Woehr, 2009; Meriac,

Hoffman, Woehr, & Fleisher, 2008). The results suggest that

the differentiation between the dimensions may not be specific

but skill or job content related, as demonstrated by the factors

found in our analysis.

The second hypothesis posited that AC dimensions will show

satisfactory predictive validity using a rarely utilized criterion:

performance-based bonus. This hypothesis too was supported

by the analysis: Five out of the 6 dimensions showed direct

associations with the bonus sum. Interestingly, the single most

predictive dimension was “organizational commitment”, revea-

ling perhaps an important aspect of the organization’s culture

and values, but also quite congruent with the importance of

commitment, motivation, and other related concepts and mea-

sures, such as conscientiousness in predicting organizational

performance (Jansen, Lievens, & Kleinmann, 2011; Witt,

Burke, Barrick, & Mount, 2002). Factorial grade analysis re-

vealed that interpersonal aspects of performance in the AC as a

whole were better predictors of performance-based bonus sum

than cognitive aspects. Gender and tenure did not associate with

the bonus, when analyzed as a part of the whole model.

The effect sizes of the association between the AC dimen-

sions or factorial grades with the bonus criterion were moderate.

This can be partly accounted for by the fact that our sample

included only candidates who successfully passed the AC,

which may cause a range restriction. Based on this assumption,

actual associations may be significantly higher (Sackett & Yang,

2000).

In interpreting the results one should be aware of the study

limitations: A relatively small sample taken from a specific

industry (finance) and a specific culture. The AC tasks and

criteria (dimensions) were specially developed for the organiza-

tion and the position participants applied for. Despite these

limitations, our sample provided adequate statistical power and

suggested trends congruent with previous evidence mentioned

in the literature (Arthur, Day, McNelly, & Edens, 2006; Bray &

Grant, 1966; Hofman, Melchers, Blair, Klienmann, & Ladd,

2011; Thorenton & Gibbons, 2009). As such it adds to our un-

derstanding of the potential constructs represented and assessed

in ACs, as well as contribute to the body of research supporting

the predictive validity of ACs as applied tools in a real world.

REFERENCES

Arthur, W., Day, E. A., McNelly, T. L., & Edens, P. S. (2006). A

meta-analysis of the criterion related validity of assessment center

dimensions. Personnel Psychology, 56, 125-155.

doi:10.1111/j.1744-6570.2003.tb00146.x

Bowler, M. C., & Woehr, D. J. (2006). A meta-analytic evaluation of

the impact of dimension and exercise factors on assessment center

ratings. Journal of Applied Psychology, 91, 1114- 1124.

doi:10.1037/0021-9010.91.5.1114

Bowler, M. C., & Woehr, D. J. (2009). Assessment center construct

related validity: Stepping beyond the MTMM matrix. Journal of

Vocational Behavior, 754, 173-182. doi:10.1016/j.jvb.2009.03.008

Bray, D. W., & Grant, D. L. (1966). The assessment center in the meas-

urement of potential for business management. Psychological Mono-

graphs: General and Applied, 80, 1-27.

doi:10.1037/h0093895

Cascio, W. F., & Aguinis, H. (2010). Applied psychology in human

resource management. New York: Prentice-Hall.

502

L. ZYSBERG

Coolican, H. (2010). Research methods and statistics in psychology.

London, UK: Hodder.

Gaugler, B. B., Rosenthal, D. B., Thornton III, G. C., & Bentson, C.

(1987). Meta-analysis of assessment center validity. Journal of Ap-

plied Psychology, 72, 493- 511. doi:10.1037/0021-9010.72.3.493

Hofman, B. J., Me lchers, K. G., Blair, C. A., Klien mann, M., & Ladd,

R. T. (2011). Exercises and dimensions are the currency of assess-

ment centers. Personnel Psychology, 64, 351-395.

doi:10.1111/j.1744-6570.2011.01213.x

Ittner, C. D., Larcker, D. F., & Rajan, M. V. (1997). The choice of

performance measures in annual bonus contracts. The Accounting

Review, 72, 231-255.

Jansen, A., Lievens, F., & Kleinmann, M. (2011). Do individual differ-

ences in perceiving situational demands moderate the relationship

between personality and assessment center dimension ratings? Hu-

man Performance, 24, 231-250. doi:10.1080/08959285.2011.580805

Jansen, P. G. W., & Stoop, B. A. M. (2001). The dynamics of assess-

ment center validity: Results of a 7-year study. Journal of Applied

Psychology, 86, 741-753. doi:10.1037/0021-9010.86.4.741

Melchers, K. G., Kleinmann, M., & Prinz, M. (2010). Do assessors

have too much on their plates? International Journal of Selection &

Assessment, 18, 329- 341. doi:10.1111/j.1468-2389.2010.00516.x

Meriac, J. P., Hoffman, B. J., Woehr, D. J., & Fleisher, M. S. (2008).

Further evidence for the validity of assessment center dimensions: A

meta-analysis of the incremental criterion related validity of dimen-

sion ratings. Journal of App lied Psychology, 93, 1042- 1052.

doi:10.1037/0021-9010.93.5.1042

Sackett, P. R., & Yang, H. (2000). Correction for range restriction: An

expanded typology. Journ al of Applied Psychology, 85, 112-118.

doi:10.1037/0021-9010.85.1.112

Schmidt, F. L., & Hunter, J. E. (1998). the validity and utility of selec-

tion methods in personnel psychology. Psychological Bulletin, 124,

262-274. doi:10.1037/0033-2909.124.2.262

Thornton, G. C., & Gibbons, A. M. (2009). Validity of assessment

centers for personnel selection. Human Resources Management Re-

view, 19, 169-187. doi:10.1016/j.hrmr.2009.02.002

Viswesvaran, C., & Ones, D. S. (2000). Perspectives on models of job

performance. International Journal of Selection and Assessment, 8,

216-226. doi:10.1111/1468-2389.00151

Werner, R., & Dudley, R. A. (2009). Making the pay matter in pay for

performance. Health Affairs, 28, 1498-1508.

doi:10.1377/hlthaff.28.5.1498

Witt, L. A., Burke, L. A., Ba rrick, M. A., & Mount, M. K. (2002). The

interactive effects of conscientiousness and agreeableness on job

performance. Journal of Applied Psychology, 87, 164- 169.

doi:10.1037/0021-9010.87.1.164