TITLE:
Item-Level Analysis of the Revised Occupational Therapy Fieldwork Performance Evaluation Applied in Practice
AUTHORS:
Cynthia L. Sears, Brad E. Egan, Patricia F. Tomsic, Craig A. Velozo
KEYWORDS:
Rehabilitation, Competency-Based Assessments, Occupational Therapy, Measurement, Rasch Analysis
JOURNAL NAME:
Open Journal of Therapy and Rehabilitation,
Vol.12 No.2,
May
24,
2024
ABSTRACT: Competency-based assessments for healthcare professionals are critical for safe and effective client outcomes. Rehabilitation clinical skill competency assessments must be validated and revised to produce safe and skilled practitioners. The revised American Occupational Therapy Association (AOTA) Fieldwork Performance Evaluation (FWPE) instrument measures occupational therapy student performance to determine readiness for practice. The assessment includes thirty-seven competencies that address both profession specific clinical skills and general professional behavior skills. The objective of this study was to use Rasch methods to explore the use of the revised FWPE in actual fieldwork practice and to determine the instrument’s psychometric properties when separating the item components into two distinct subdomains: General Health Professions Competences and Occupational Therapy-Specific Competencies. Internal construct validity and test reliability were analyzed using data from 149 occupational therapy students after completing their initial Level II A fieldwork clinical internship. This study examined the item difficulty hierarchy, item fit, person-fit to model, person separation index, person separation reliability coefficient, strata, ceiling and floor effect, and unidimensionality of the FWPE instrument as a whole and as two separate domains. With the exception of not meeting the criteria for unidimensionality, the full FWPE instrument and the Occupational Therapy-Specific Competencies subdomain showed acceptable item-level psychometrics for reliability and precision. While the General Health Professions Competencies subdomain showed good item-level psychometrics, it was below the criterion for reliability and only separated the sample into two strata. Results support the validity, reliability, and clinical use of the revised FWPE full instrument and the Occupational Therapy-Specific Competencies subdomain to measure entry-level clinical skill competencies in practice.