Assessing Clinical and Academic Performance in a Master’s Level Speech Language Pathology Program: A Path Analysis

doi:10.4236/ce.2012.31023

Paper Menu >>

Journal Menu >>

Creative Education

2012. Vol.3, No.1, 145-148

Published Online February 2012 in SciRes (http://www.SciRP.org/journal/ce) http://dx.doi.org/10.4236/ce.2012.31023

Assessing Clinical and Academic Performance in a Master’s

Level Speech Language Pathology Program: A Path Analysis

Margaret M. Kjelgaard1, Anthony J. Guarino2

1Department of Communication Sciences and Disorders, MGH Institute of Health Professions, Boston, USA

2Center for Interprofessional Studies and Innovation, MGH Institute of Health Professions, Boston, USA

Email: [email protected]

Received November 2nd, 2011; revised December 7th, 2011; accepted December 18th, 2011

This study assessed a path model to determine the direct and indirect effects on clinical and academic

success from students’ undergraduate and graduate performance. Astin’s I-E-O model served as the theo-

retical foundation for the hypothesized model. Input (I) data included students’ undergraduate information

(GPA, major, and GRE scores); Environment (E) data included grades in required graduate courses and

formative clinical evaluation; and Output (O) data included PRAXIS score and summative clinical

evaluation. The sample was 122 students who completed a SLP Program at a New England graduate

school. Results of the path analysis supported the efficacy of the proposed model in determining the direct

and indirect effects on professional competence. Graduate students with an undergraduate degree in SLP

were at a disadvantage upon entry to the program relative to students from other undergraduate majors.

Implications of the study are discussed.

Keywords: Graduate Program; Speech Language Pathology; Path Analysis; Learning Outcomes; Astin’s

I-E-O Model

Introduction

The field of Speech and Language Pathology (SLP) has be-

come increasingly diverse in recent years. The current range of

settings, clinical populations and specialized areas of diagnosis

and treatment are widely varied. In 2011, the American Speech,

Language and Hearing Association (ASHA) website describes

18 independent divisions in the field (www.asha.org). With the

increasing number of special interests, graduate programs are

required to teach theory and practice on a wide variety of clini-

cal issues. Because of the breadth of competence necessary to

practice, many states require a minimum of a master’s degree to

practice. Therefore, a graduate program in SLP must admit and

then graduate highly competent practitioners. The graduate

program evaluated in the study, for example, requires six very

demanding semesters of course-work and at least three clinical

placements for degree completion.

Selecting students for admission is encumbered by the lim-

ited information available the applicant’s admissions file. In

addition to this limited information, it has been argued that, for

example, undergraduate GPA’s suffer from grade inflation since

the 1960’s and are, as a result, compressed at the upper end of

the scale (Dostoevsky & Hartley, 2002). Letters of recommen-

dation likewise, tend to be inflated (Rosovsky & Hartley, 2002).

It has also been shown that undergraduate GPA is less accurate

at predicting success in graduate school than standardized GRE

scores (Kuncel & Hezlett, 2007). GRE scores have themselves,

demonstrated mixed validity in predicting graduate student

success (Kuncel & Hezlet, 2007; Sacks, 1999; Sternberg &

Williams, 1997; Zwick, 2002).

Anecdotally, the authors have experienced surprise by the

students who excel as well as those who have difficulty, based

on information from their admissions file. For example, some

students with very high GRE scores and superior undergraduate

GPAs have failed to achieve to expectations, especially with

regard to clinical competence. Conversely, other students with

marginal scores have exceeded faculty expectations by demon-

strating superior clinical competence. The hypothesized model

in this study aimed to identify the direct and indirect effects on

clinical and academic success from students’ undergraduate and

graduate performance.

Prior research on the topic of academic success is rather lim-

ited with regard to SLP programs. One study by Forrest and

Naremore (1998) analyzed admissions data using a discrimi-

nant analysis to categorize students as either “top of the class

students” or “bottom of the class students”. These categories

were defined by scores on board exams and graduate school

GPA. Their results indicated that undergraduate GPA had a

93% overall accuracy rate, yet, GRE failed to achieve any sta-

tistically significant contribution. The limitations of this study

include a small number of students (N = 30) and few predictor

variables.

Other researchers (Astin, 1991, 1993; Bean, 1983; Tinto, 1993)

have proposed that understanding success in any academic

program requires assessing the interaction between the student

and the institution’s environment. The present study utilized

Astin’s (1991, 1993) I-E-O model. The premise of the I-E-O

model is that educational assessments must include information

on student inputs (I), the educational environment (E), and

student outcomes (O). Inputs include the students’ demograph-

ics, academic background, and previous experiences; Environ-

ment refers to the range of experiences students encountered

during their college tenure; and Outcomes are the knowledge,

beliefs, and values the student achieved at their institution. The

Inputs are presumed to shape outcomes directly and indirectly

with the institutional environment. Thus, academic success (the

M. M. KJELGAARD ET AL.

Outcome), is a function of the interaction between the student

characteristics (Inputs) and the characteristics of the academic

setting (Environment).

Purpose

The primary purpose of this study was to construct and as-

sess a path model to determine the direct and indirect effects on

two Outcome (O) variables (i.e., PRAXIS scores and Summa-

tive Clinical Evaluation). Input (I) data included the student’s

undergraduate information: Undergraduate major (Speech Ma-

jor), Undergraduate GPA (UG GPA), and GRE (GRE Q and

GRE V) while the Environment (E) data were grades in required

graduate courses (Course Average) and Formative Clinical

Evaluation. The hypothesized model is presented in Figure 1.

Method

Participants

The sample for this study was 122 students who completed

the SLP Program in the 2007-2009 academic cycles at a New

England graduate school. The sample was comprised of mostly

women ranging in age from 23 to 65, (M = 27.67, SD = 5.37).

The GRE-Q scores ranged from 250 to 790 (M = 587.64, SD =

97.44) while the GRE-V scores ranged from 290 to 800 (M =

516.75, SD = 103.47). Just under half (40.3%) of the partici-

pants were SLP Majors as undergraduates with the remaining

58.9% representing a wide range of majors.

Procedure

This study analyzed the following students’ Input variables,

1) major (SLP or other); 2) undergraduate GPA; and 3) GRE

scores (verbal and quantitative). The Environment variables

were the Average GPA from required graduate courses and

Formative Clinical Evaluation. The Formative Clinical Evalua-

tion was calculated from scores obtained from the 42 items on

the Clinical Practicum Competency Checklist. This checklist is

composed of eight subscales, 1) Clinical Excellence Across

Contexts; 2) Assessment Data Analysis; 3) Assessment Proce-

dures; 4) Intervention Planning; 5) Intervention Execution; 6)

Clinical Excellence in Writing; 7) Clinical Excellence as a Pro-

fessional; and 8) Self Evaluation. All items are scored on the

following scale: 1 = fails to demonstrate behavior consistently

regardless of amount of supervision or needs excessive and

repetitive instructions, 2 = demonstrates behavior with specific

instruction from supervisor, 3 = demonstrates behavior with

general guidance from supervisor, 4 = demonstrates behavior

with minimal guidance from supervisor, and 5 = demonstrates

behavior independently. The Output variables were PRAXIS

score and Summative Clinical Evaluation. The PRAXIS is de-

signed by experts from The American Speech Language Hear-

ing Association (ASHA), with the goal of testing the skills

necessary to enter professional practice as a new graduate. The

scores on the exam range from 250 - 990 with a passing score of

600 (Educational Testing Service, 2009). The Summative Clinical

Evaluation was comprised of performance scores ranging 1

(poor) to 5 (superior) by the department’s clinical practicum

coordinator. The practicum coordinator oversees all students in

at least three settings outside of the academic setting. This

process involves multiple site visits, counseling, and consulting

between the site supervisors and the students and therefore the

coordinator has a comprehensive understanding of how stu-

dents perform in professional clinical settings. The Summative

Clinical Evaluation was based on the written and verbal as-

sessments of the supervisors in outplacement settings, students’

academic performance, the level of independence exhibited by

the student, as well as commonly-identified characteristics of

good clinicians (e.g., flexibility, proactivity, and critical think-

ing).

Figure 1.

Hypothesized model.

146

M. M. KJELGAARD ET AL.

The proposed model was comprised of seven structural equa-

tions. In the first two equations, PRAXIS and Summative Clini-

cal Evaluation was regressed on Course Average and Formative

Clinical Evaluation. The next two structural equations had

GRE-Quantitative, GRE-Verbal, and Undergraduate GPA pre-

dict Course Average and Clinical Evaluation. The last three

equations had GRE-Quantitative, GRE-Verbal, and Under-

graduate GPA regressing on Undergraduate SLP Major (coded:

“1” as “Yes”, “0” as “No”). The proposed model illustrates the

linear causal relationships among variables from the three time

points. This allows for understanding the variables from the

admissions file most related to course and clinic performance

during the program and which variables from graduate school

performance are most related to scores at graduation.

Data Analysis

A Hotelling’s MANOVA was conducted to detected differ-

ences between SLP majors and non-SLP majors on 1) under-

graduate GPA; 2) GRE verbal and quantitative scores; 3) gradu-

ate GPA; and 4) formative clinical evaluation. Independent sam-

ples t-tests were conducted as univariate follow-ups. A path

analysis, an application of multiple regression in conjunction

with causal theory, was used to analyze the causal models in

this inquiry.

Results

Prior to analysis, all variables were examined through vari-

ous IBM SPSS programs for accuracy of entry, missing values

and assumptions of normality and linearity resulting that data

met assumptions. Results of the Hotelling’s MANOVA re-

ported statistically significant differences between SLP (Speech

Pathology) majors and non-SLP majors, F(5, 117) = 8.97, p <

.001, η2 = .28. Although SLP majors entered the program with

statistically significantly greater GPAs (M = 3.64, SD = .26)

than non-SLP majors (M = 3.50, SD = .31), F(1, 121) = 6.75, p

= .011, η2 = .05; SLP majors had poorer GRE quantitative

scores (M = 550.80, SD = 103.07) than non-SLP majors (M =

612.88, SD = 86.03), F(1, 121) = 13.13, p < .001, η2 = .10 as

well as GRE verbal scores (M = 467.60, SD = 84.65) relative to

their peers from other undergraduate majors (M = 550.41, SD =

102.89), F(1, 121) = 22.11, p < .001, η2 = .16. These students

who entered with SLP majors continued to be the weaker aca-

demically in the program, having statistically significantly

lower GPA’s at the graduate level, (M = 3.69, SD = .26) than

those entering as non-SLP majors (M = 3.76, SD = .17), F(1,

121) = 4.25, p = .041, η2 = .03.

Figure 2 presents the path coefficients as well as the coeffi-

cient of determination for each of the endogenous variables.

Undergraduate Speech Major was negatively associated with

both GRE Q and GRE V, but was positively related to UG GPA.

All three predictors for Course Average (GRE Q, GRE V, and

UG GPA) achieved significant coefficients, while GRE Q and

UG GPA were moderately correlated to Formative Clinical

Evaluation; GRE V failed to predict Formative Clinical Evalua-

tion. Course Average achieved a statistically significant path

coefficient for PRAXIS; however, Formative Clinical Evalua-

tion did not. Both Course Average and Formative Clinical

Evaluation achieved significant path coefficients for Summa-

tive Clinical Evaluation.

Discussion

Findings from the present study have implications for future

theoretical work in the area of clinical and academic perform-

ance in graduate SLP programs. The results indicated a better

understanding of success can be derived by assessing the inter-

Figure 1.

Hypothesized model.

M. M. KJELGAARD ET AL.

action between the student and the institution’s environment.

The path analysis identified the direct and indirect effects on

Summative Clinical Evaluation and PRAXIS score. Students

with the highest successful PRAXIS scores were those who

were able to perform in the highly demanding courses in terms

of content and time-management. For success in those courses,

GRE Q and Undergraduate GPA were significant predictors.

The results also seem to suggest that students entering the pro-

gram with an undergraduate degree in SLP were at a disadvan-

tage upon entry to the program. Although SLP majors entered

the program with greater GPAs than non-SLP majors; they had

poorer GRE-Q and GRE-V than non-SLP majors. Furthermore,

these students who entered with SLP majors continued to be

weaker academically in the program. This finding has implica-

tions for not only admissions decisions, but also for under-

graduate programs in SLP. Undergraduate SLP programs need

to better prepare their students for the rigors of graduate school

by offering more research inquiry courses.

The Input (I) data used to select students for a Masters pro-

gram in SLP is related indirectly to academic success in the

program. In consideration of academic aptitude, the GRE-Q,

GRE-V, and UG-GPA demonstrated predictive validity, while

GRE-Q and UG-GPA were predictive of clinical aptitude. The

institution’s courses (Course Average, i.e., E data) prepared

students well for both academic (PRAXIS) and clinical (Sum-

mative Clinical Evaluation, i.e., O data), whereas the clinical

training (Formative Clinical Evaluation) is related to the clini-

cal outcome measure, but not to the PRAXIS scores achieved

by students.

The generalizability of the findings to other institutions is to

be approached cautiously. As indicated by Astin (1993), the

patterns underlying clinical and academic success may vary by

type of institution, the setting, and the composition of the stu-

dent body. Institutions may use this model as a starting point in

investigating success at their respective campuses.

REFERENCES

Astin, A. (1991). Assessment for excellence: The philosophy and prac-

tice of assessment and evaluation in higher education. New York:

Macmillan.

Astin, A. (1993). What matters in college? Four critical years revisited.

San Francisco: Jossey-Bass.

Bean, J. (1983). The application of a model of turnover in work or-

ganizations to the student attrition process. The Review of Higher

Education, 6, 129-148.

Educational Testing Service (2009). The Praxis Series. Princeton, NJ:

McGraw-Hill.

Forrest, K., & Naremore, R. C. (1998). Analysis of criteria for graduate

admissions in speech-language pathology: Predictive utility of appli-

cation materials. American Journal of Speech-Language Pathology,

7, 57-61.

Kuncel, N. R., & Hezlett, S. A. (2007). Standardized tests predict gra-

duate students’ success. Science, 315, 1080-1081.

doi:10.1126/science.1136618

Rosovsky, H., & Hartley, M. (2002). Evaluation and the Academy: Are

we doing the right thing? Cambridge, MA: American Academy of

Arts and Sciences.

Sacks, P. (1999). Stan dardized minds. Cambridge, MA: Perseus Books.

Sternberg, R. J., & Williams, W. M. (1997). Does the graduate record

examination predict meaningful success in the graduate training of

psychologists? A case study. American Psychologist, 52, 630-641.

doi:10.1037/0003-066X.52.6.630

Tinto, V. (1993). Leaving college: Rethinking the causes and cures of

student attrition. Chicago: The University of Chicago Press.

Zwick, R. (2002). Fair game? The use of standardized admissions in

higher education. New York: RoutledgeFalmer.

148