Impact of a Practical Skills Assessment on the Individual Engagement of Undergraduate Pharmacy Students within Laboratory Coursework Sessions

doi:10.4236/ce.2012.326137

Paper Menu >>

Journal Menu >>

Creative Education

2012. Vol.3, Special Issue, 908-911

Published Online October 2012 in SciRes (http://www.SciRP.org/journal/ce) http://dx.doi.org/10.4236/ce.2012.326137

908

Impact of a Practical Skills Assessment on the Individual

Engagement of Undergraduate Pharmacy Students within

Laboratory Coursework Sessions

Susanne P. Boyle

School of Pharmacy and Life Sciences, Robert Gordon University, Aberdeen, Scotlan d

Email: [email protected]

Received September 4th, 2012; revised October 2nd, 2012; accepted October 18th, 2012

This study reports on the learner impact, practicability and cost effectiveness of an individual practical

skills test designed to assess the interpretative and manipulative skills of undergraduate pharmacy in a

laboratory setting. The reliability of the assessment tool across a 5 year period was examined and refine-

ments introduced in response to constructive feedback from colleagues and learner feedback recorded via

end of year Student Evaluation Questionnaires. A blended learning strategy supported the needs of multi-

ple learning styles and inclusion of a formative assessment increased student confidence and improved

cohort performance in the summative assessment. Future directions include the introduction of a peer

learning activity as a means of reducing group sizes and providing an opportunity for the learners to de-

velop skills in constructive critique and reflective learning.

Keywords: Practical Skills Assessment; MPharm Degree; Formative Assessment; Peer Learning

The Scottish MPharm Degree

In Scotland the undergraduate pharmacy degree is a 4 year

undergraduate Master’s degree (QAA, 2002) accredited by the

General Pharmaceutical Council (GPhC). The curriculum de-

veloped across the four years addresses core modules in phar-

maceutical sciences including medicinal chemistry, pharma-

cognosy, pharmaceutical analysis, biochemistry, pharmaceuti-

cal microbiology, sterile products and medicine design and

manufacture together with professional modules such as pre-

scribing sciences, pharmacy practice, social pharmacy, public

health and pharmaceutical care.

The GPhC standards for the initial education and training of

pharmacists (GPhC, 2011) advises that the “curricula must be

progressive, dealing with issues in an increasingly more com-

plex way until the right level of understanding is reached”

(Harden & Stamper, 2009), is underpinned by appropriate as-

sessment strategies and provides opportunities reflection and

peer learning. The philosophy of the GPhC standards (GPhC,

2011) is driven by a testing of 4 levels of competency defined

in Miller’s triangle (1990) and which describes four levels of

assessment: knows; knows how; shows how (competence); and

does (performance) (Figure 1).

A popular degree choice for undergraduate students, each

Stage of the MPharm degree has typically between 130 - 140

full time students enrolled and such large classes are the main

driver for a group approach (typically 4 - 5 students) being

adopted in practical coursework sessions.

Modes of Assessment in Pharmaceutical

Science Modules

It is widely recognized that the mode of assessment may in-

fluence the behavior of student learners (Biggs, 2003) but

equally issues of class sizes and staff resources are sometimes

constraints to the implementation of the ideal scenario. In the

early years of the MPharm degree most modules have a mini-

mum of two components of assessment aligned to the lecture

material (e.g. written exam) and the laboratory coursework

sessions for which there are a variety of assessment strategies in

use.

Biochemistry is a Stage 2 (Scottish Credit Qualification

Framework level 8) degree module and forms the foundations

for Stage 3 modules such as Biotherapeutics and Pharmaceuti-

cal Analysis both of which require students to be numerate,

competent in the handling of micropipettes and construction of

calibration lines and confident in their ability to interpret and

evaluate experimental data. Historically the assessment of the

Biochemistry coursework sessions has been by an individual

written test and an individual laboratory report. The advantages

of this approach were that it enabled the assessment of core

knowledge and evaluative skills required for later Stages of the

degree including e.g. the preparation of a final year thesis. The

disadvantages were that it gave no measure of a student’s com-

petency within a laboratory setting and there was little incentive

for students to contribute individually to the group coursework

activity since by sharing the data generated as a group, indi-

viduals could produce satisfactory submissions without neces-

sarily having contributed effectively to the laboratory practi-

cals.

Student feedback via the Student Evaluation Questionnaire

(SEQ), consideration of staff resources required to mark and

feedback on the two written assessments together with personal

concerns regarding the inability of some students to compe-

tently handle micropipettes and manipulate calibration curves

were the drivers for change. This led to the design, implemen-

tation and review of an individual practical skills test as a

S. P. BOYLE

means of assessing the manipulative and interpretative skills of

Stage 2 MPharm students undertaking Biochemistry coursework

element.

Design and Operation of the Practical Skills Test

The five core skills assessed via the practical skills test were

the:

1) accurate and precise use of micropipettes,

2) calculation of dilutions and preparation of calibration stan-

dards,

3) accurate representation of data using a graph,

4) use of a UV spectrophotometer,

5) evaluation of the accuracy and precision of experimental

data.

The core skills were assessed via 3 independent workstation

activities each of 12 minutes duration. Workstation 2 assesses

core skill 3 and requires students to construct calibration lines

and interpolate unknowns using exemplar data. Workstation 1

assessed core skills 1, 2 and 4 whilst workstation 3 assessed

core skills 1, 2, 4 and 5. A round robin rotation was employed

with 4 students simultaneously undertaking each of the 3 skills

assessments. Thus 12 students complete the summative as-

sessment in each 45 minute cycle and 5 staff was involved in

the observation, marking and timing of the assessment. Post

assessment the markers complete a prescriptive marking pro

forma which details how effectively the student has performed

key elements of the task. This approach enables students to

receive individual written feedback on their summative assess-

ment and critically has significantly reduced the time (approx.

30% decreases since 2007) assessors commit to this marking

this component.

Appreciative of the stress that some learners experience in in-

dividual skills based assessments and mindful of the potential

benefits that formative assessment and feedback affords (Black

& William, 1998; Dunn & Mulvenon, 2009), the students have

a formative assessment opportunity one week before the sum-

mative event. Since it is recognized that students may readily

practice their skills in graphical analysis outwith the laboratory

environment the formative exercise focuses on other ex-

perimental activities designed to evaluate core skills 1, 2, 4 and

5. Immediately after completion of the formative assessment

staff mark the individual student worksheets, provide individual

feedback on the pass/fail outcome and provided detailed group

feedback. Students are then invited to seek staff guidance if

they are still unsure about the handling of micropipettes or op-

eration of UV spectrophotometers.

Evaluation of Effectiveness and Impact upon

Learners

Student feedback indicated the formative assessment was

perceived as a valuable exercise and the blended approach to

supporting different learner styles was appreciated (see Table

1). It is clear however that some of the challenges of group

work prevail with some students expressing a desire for smal ler

groups (see Table 1).

Cohort performance in the formative assessment has been

quite consistent with 30% - 40% of the population failing to

achieve a pass Grade (i.e. Grade D - Grade A, data not shown).

However, following group feedback on the formative assess-

ment, further individual instruction on the use of micropipettes;

operation of the UV spectrophotometer and the one week for

students to reflect on their performance there is a significant

improvement in the Grade profile of each cohort (see Figure

2).

During the first two years of implementation >85% candi-

dates achieved a Grade A in the summative assessment and

consequently the difficulty level of both workstations 1 and 3

concerned with data generation was increased. These changes

led to a decrease in the proportion of the population achieving

Grade A (see Figure 3) suggesting the tool was more discrimi-

natory and a plateau in the Grade A profile appears to be

emerging.

Discussion

This study summarizes an assessment activity designed to

support a progressive approach to the development of labora-

tory skills within the 4 year MPharm degree. The practical

Table 1.

Post module comments captured via SEQ (May 2012 cohort).

Student Identifier Comments in S t udent Ev aluation Questionnaire (SEQ)

1 The practical skills test mad e me work hard to im p r ove my technique .

2 Formative practical assessment was a good taster for what to expect in the summative.

3 The formative coursework assessment w as a good indication of what to expect in the s ummative coursework assessment. Thought this

was good as it helped reduce nerves b efore the assessment.

4 The fact that I knew I had a practical coursework assessment made me work hard and take an interest in coursework sessions prior to

the test.

5 The coursework test was well handled; the formative ga ve a lot of help for the summative.

6 The maj o ri ty of the experiments were for 4 or 5 people but you could actually do i t o n your own or in pairs so thought thi s was a little

pointless and also m eant if there were people i n the grou p who liked to take the lead, others would simply not get a t urn and be left

behind.

7 The quizzes provided on Moodle which gave good examples of test and exam style questions were particularly useful both in assessing

understan ding of particular topics and in pre paration for the examination—would have been useful to have these for other modules.

8 Very helpful when staff came ro u nd to each student during the l ab coursew ork. I benefited greatly from this as when class is given

answers as a whole its hard to keep up and fully understand.

S. P. BOYLE

DOES

Shows how

Shows

Knows how

Figure 1.

Miller’s triangle (1990).

Figure 2.

Distribution of assessment grades from 2008-2012.

Figure 3.

Grade “A” distribution across the 5 years of implementation.

skills assessment was developed to encourage individual stu-

dent engagement, promote acquisition of core laboratory skills

and is a valid instrument (Bloxham & Boyd, 2007) for assess-

ing the module learning outcome: “To generate, manipulate and

interpret experimental data relating to biochemistry.”

A blended learning strategy was implemented to address mul-

tiple learning styles and included significant periods of face to

face laboratory instruction, online videos to enable students to

view instrument demonstrations and online Articulate quizzes

to enable students to self evaluate their skills in pharmaceutical

science calculations. This approach enhanced the student ex-

perience, received positive student feedback (see Table 1) and

supported a “learning by seeing” and “learning by doing” ex-

perience (Bruner & Olson, 1973) which was accessible to all

students via the University’s virtual lear ning environment.

The development of the practical skills assessment and asso-

ciated marking pro forma required significant staff resources to

ensure: 1) the tasks were appropriate for assessment of the

knowledge and skills of stage 2 students, and 2) the assessment

pro forma enabled an accurate record of individual student

performance to be recorded in a consistent and transparent

manner. However the impact of the assessment on student skills

was almost immediately apparent with anecdotal feedback from

academic colleagues concerned with Stage 3 modules indicat-

ing that the laboratory skills and confidence of progressing

Stage 2 students had improved within the first year of imple-

mentation. Moreover there has been sharing of this good prac-

tice with academic colleagues leading to a modified form of the

assessment recently being introduced within the Stage 1 Physi-

cal Pharmacy module. This vertical integration of the assess-

ment strategy reinforces the message to students that the indi-

vidual acquisition of lab skills is a pre-requisite for progression

and also improves the potential for assessment of competency

(Miller’s Triangle, 1990) in the core skills associated with

pharmaceutical sciences.

External examiner feedback of the assessment has been posi-

tive and constructive whilst student feedback reflected the

findings of Taras (2002) and suggested the formative assess-

ment was both important to supporting individual learning and

an effective way for some students to manage the nerves asso-

ciated with the summative exercise.

It became clear however, in the first two years of application,

that the summative assessment was not as discriminatory as we

would have hoped for (see Figure 2) and whilst from an aca-

demic perspective this was problematic it was also important

that the student learners perceived the process as rigorous and

continued to be motivated and challenged by the tool (Juwah,

2000, 2003). Consequently in 2010 the assessment was refined

to increase the level of difficulty and a reduction in Grade A

awards occurred with a broader distribution of pass Grades

(A-D) being achieved and no significant alteration in the pro-

portion of Grade E or Fs being observed. Current data suggests

the proportion of population achieving Grade As has stabilized

(55% - 63%) but further monitoring of the effectiveness and

reliability of this assessment tool is desirable particularly in

light of the recent introduction of a modified form of the as-

sessment in early years of the degree course.

Future activities will be directed towards finding a solution

to addressing the concerns some students express regarding

individual contribution to and individual learning from a group

setting (see Table 1). One possible solution which will be tri-

aled next academic session is to include an element of peer

learning within the group work, with student pairs construc-

tively critiquing the organizational skills, team working skills

and quality of experimental data generated by their peers (Ju-

wah, 2003).

Conclusion

In conclusion this study evaluated the impact of a practical

skills assessment which reflects the ethos of GPhC standards

for the training and education of undergraduate pharmacy stu-

dents and which has been demonstrated to be a practicable and

cost effective tool which is valued by academics and the student

learners. The continued challenges that group working some-

times presents are recognized but the plan is to use this as an

opportunity to include peer learning and thereby further support

the development of self oriented and reflective practitioners

910

S. P. BOYLE

(GPhC, 2011).

Acknowledgements

Dr Susanne Boyle gratefully acknowledges the contribution

of Dr Raymond Reid and Mrs. Maureen Byres during the de-

velopment of this assessment tool and the constructive feedback

of Dr Simon Officer during the refinement of the marker’s pro-

forma.

REFERENCES

Biggs, J. B. (2003). Teaching for quality learning at university (2nd ed.)

Buckingham: Society for Research into Higher Education and Open

University Press.

Black, P., & William, D. (1998) Assessment of classroom learning.

Assessment in Education, 5, 7- 74. doi:10.1080/0969595980050102

Bloxham, S., & Boyd, P. (2007). Developing effective assessment in

higher education: A practical guide. New York: McGraw-Hill Inter-

national.

Bruner, J. & Olson, D. (1973) Learning through experience and learn-

ing through media. Prospects, 3, 20- 38. doi:10.1007/BF02196942

Dunn, K. E., & Mulvenon, S. W. (2009). A critical review of research

on formative assessment: The limited scientific evidence of the im-

pact of formative assessment in education. Practical Assessment, Re-

search and Evaluation, 14, 1-11.

General Pharmaceutical Council (2011). Future pharmacists: Standards

for the initial education and training of pharmacists. URL (last che-

cked 14 August 2012).

http://www.pharmacyregulation.org/sites/default/files/GPhC_Future_

Pharmacists.pdf

Harden, R. M., & Stamper, N. (1999). What is a spiral curriculum?

Medical Teacher, 21, 141-143. doi:10.1080/01421599979752

Juwah, C. (2000). The quality spiral for assessment. Aberdeen: The Ro-

bert Gordon University Year of Assessment.

Juwah, C. (2003). Using peer assessment to develop skills and capabili-

ties. USDLA Journal, 1 7 , 1-11.

Miller, J. (1990). The assessment of clinical skills, co mpetence, perform-

ance. Academic Medicine, 65, s63-s67 .

doi:10.1097/00001888-199009000-00045

Taras, M. (2002). Using assessment for learning and learning from as-

sessment. Assessment & Evaluation in Higher Education, 27, 501-510.

doi:10.1080/0260293022000020273

The Quality Assurance Agency for Higher Education (2002). Pharmacy .

http://www.qaa.ac.uk/Publications/InformationAndGuidance/Docum

ents/pharmacy.pdf