Assessments play a pivotal role in the teaching and learning process by offering valuable feedback to instructors on their teaching practices and on students’ developmental progress ( Voltz, Sims, & Nelson, 2010 ). The credibility and usefulness of any assessment are largely determined by its validity, which refers to the degree to which an assessment accurately measures what it is intended to measure. Research on assessment validity often focuses on several key dimensions, including construct validity, content validity, and criterion-related validity—the latter encompassing predictive and concurrent validity ( Crooks et al., 1996 ).

While consistency, or reliability, in assessment refers to the reproducibility of results across different administrations or contexts, validity pertains specifically to the appropriate interpretation and application of assessment outcomes in evaluating learner performance, skills, or knowledge ( Chang & Seow, 2018 ). In essence, a reliable assessment that lacks validity may consistently measure something irrelevant, whereas a valid assessment ensures meaningful and actionable insights into student learning.

Among scholars such as Sireci (2009) , Downing (2003) , Kane (2001) , Moss (1995) , and Shepard (1993) , there is a consensus that construct validity represents the most comprehensive and enduring conceptualization of validity. Construct validity emphasizes that assessments should measure the theoretical constructs or traits they are intended to evaluate, rather than merely focusing on surface-level content or specific test items. Historically, however, the notion of validity was more narrowly framed. Expert panels were often convened to review tests or programs, providing judgments on whether the items were representative of the intended content. When these panels deemed that the items accurately assessed the relevant aspects of the content, this was referred to as content validity ( Wiliam, 2008 ).

Despite its historical prominence, content validity has faced challenges, particularly when the scope of the content is poorly defined or when the panel’s judgment is subjective. This limitation has led to debates regarding the conceptual clarity of content validity and its appropriateness as a standalone measure of assessment quality. Some validity theorists, including Messick (1989) , argue that the term “content validity” is technically inaccurate, advocating instead for an integrated view that considers multiple sources of evidence, including content representation, response processes, internal structure, and consequential validity. These debates underscore that validity is not a static property of an assessment tool but a dynamic construct that depends on context, purpose, and interpretation.

Furthermore, the practical implications of validity in higher education are significant. Valid assessments can guide curriculum improvements, inform pedagogical interventions, and support fair and equitable evaluation of student performance. Conversely, invalid assessments may misrepresent student abilities, undermine instructional effectiveness, and contribute to inequities in educational outcomes. As such, understanding and applying the principles of validity is critical for educators, administrators, and policymakers committed to evidence-based practice and student-centered learning.

In conclusion, validity remains a central concern in assessment research and practice. Its evolution from content-focused judgments to construct-oriented frameworks reflects broader shifts in educational philosophy, emphasizing both accuracy and fairness. By approaching assessment as a complex, multi-dimensional system, educators can ensure that evaluations not only measure learning outcomes effectively but also foster meaningful growth and development among students.

Borsboom et al. (2004) and Brown et al. (1997) propose that validity can be understood from two complementary perspectives: as an inherent attribute of the test itself or as interpretations derived from test outcomes ( Borsboom et al., 2004 ; MacMillan, 2004 ; Messick, 1996 ). The former perspective is commonly applied in psychological and natural sciences testing, where the focus is on the properties of the test items and measurement tools. In contrast, the latter perspective is more prevalent in disciplines such as design education, where assessments of specific assignments are evaluated based on students’ performance and the interpretation of those results.

Messick (1996) provides an extensive examination of threats to validity identified in the assessment literature and proposes a comprehensive framework that expands traditional approaches, which typically focus on content and construct validity. According to Messick, construct validity can be conceptualized in six interrelated components: consequential, substantive, content, external, structural, and generalizability. Each component addresses a distinct aspect of validity, and while different methods may be employed to evaluate them, none should be disregarded. The substantive component, for instance, ensures that the knowledge and skills being assessed accurately reflect the target population and are relevant to the learning material studied, thereby verifying the meaningfulness of the assessment tasks.

Messick (1996) also differentiates between two key attributes of validity. Generalizability, in particular, refers to the degree to which interpretations of assessment outcomes can be extended across tasks, contexts, or groups. Achieving this attribute requires well-designed marking rubrics and clear criteria, which are the responsibility of the first marker or evaluator. As Gearhart et al. (1995) note, comparing students’ marks across grade levels can serve as an illustration of the generalizability attribute of construct validity, emphasizing that careful design and consistent application of assessment criteria are essential for ensuring fair and meaningful evaluation. The template is used to format your paper and style the text. All margins, column widths, line spaces, and text fonts are prescribed; please do not alter them. You may note peculiarities. For example, the head margin in this template measures proportionately more than is customary. This measurement and others are deliberate, using specifications that anticipate your paper as one part of the entire journals, and not as an independent document. Please do not revise any of the current designations.

Despite the critical importance of assessing validity, its evaluation is often overlooked in practice. To address this, experts in the field have identified various criteria and frameworks to guide the validation process ( Messick, 1995 ; Linn, 1991 ; Frederiksen & Collins, 1989 ; Cole & Moss, 1989 ; Haertel, 1984 ). One notable approach is Crooks’ systematic framework, which provides a methodical structure to support validators in evaluating educational assessments ( Crooks et al., 1996 ). This model consists of eight interrelated stages—administration, scoring, aggregation, generalization, extrapolation, evaluation, decision, and impact—each of which contributes to the overall validity of the assessment. Importantly, the assessment’s validity is ultimately constrained by its weakest component, meaning that excessive investment in strengthening other components may be ineffective or even counterproductive ( Crooks et al., 1996 ) ( Figure 1 ).

To ensure that validity is evaluated effectively, it is imperative to carefully analyze the interconnected threats and challenges that may compromise it. These threats, which are discussed in the following sections, can emerge at any stage of the assessment process and must be addressed systematically to maintain the integrity and usefulness of assessment outcomes.

Black et al. (2010) , drawing on Crooks et al.’ (1996) chain model and influenced by Messick’s and Kane’s work on validity, outlined an approach to assessment validation that emphasizes the interconnected nature of assessment processes. In Crooks’ model, assessment is represented as a chain of eight linked stages, where the strength of the whole process depends on each individual connection. A weakness or breakdown at any stage introduces a threat to validity, as it can undermine the overall fairness, accuracy, or usefulness of the assessment. This model encourages teachers and lecturers to examine each stage critically, identifying where issues may arise and ensuring that assessment practices remain robust, transparent, and defensible.

The revised Code of Practice for student evaluation by the Quality Assurance Agency (QAA) defines pedagogy as the facilitation of student learning through tasks that assess their abilities, provide feedback to enhance performance, and encourage reflective learning approaches. In the field of design, learning can be significantly enhanced by critically guiding students in the development of optimal design and engineering skills ( Quality Assurance Agency for Higher Education, 2006: p. 4, as cited in Heather et al., 2009: p. 134 ).

Gibbs & Simpson (2004) developed a set of pedagogical principles that provide a foundation for assessment design aimed at enhancing learning. These principles highlight that assessments must be communicated clearly and uphold high standards. In the CPD module, students reported confusion regarding the assignment because the lecturer who designed it did not adequately brief them on the content, requirements, scoring methods, or grade expectations. This lack of communication led to misalignment between the intended learning outcomes (ILOs) and the marking criteria, creating a significant threat to assessment validity.

In the CPD assignment, the scoring scheme evaluated certain elements while neglecting or failing to clarify others. This created misunderstandings among students, as the marking process did not accurately capture the constructs that the lecturer intended to assess. As a result, assessment validity was compromised, making this a clear threat identified within the CPD module.

Holistic scoring evaluates a student’s work as a whole, considering overall quality, coherence, creativity, and other subjective factors, rather than focusing on individual components. While this approach can be advantageous for assessing complex, multidimensional tasks where overall impression is important, it has notable limitations. Specifically, holistic scoring can lack transparency and consistency, as different evaluators may interpret quality differently. In the CPD module, the use of holistic scoring was identified as a threat to validity during the module review.

In contrast, analytic scoring evaluates each component of the task separately, providing detailed feedback on specific criteria. This approach enhances clarity, transparency, and fairness, allowing students to understand precisely how to improve their work and achieve higher marks. Therefore, adopting an analytic scoring method is recommended to strengthen assessment validity and guide student learning effectively.

A lack of clear communication can pose significant threats to validity in design education. When tasks are not communicated effectively, students may focus on irrelevant aspects or fail to achieve key learning objectives. Ensuring clarity and transparency in task design is therefore essential for maintaining the validity of assessments. In our department, there was limited emphasis on aligning tasks with intended learning outcomes (ILOs) and marking criteria. Consequently, I did not provide students with detailed guidance on how their work would be evaluated, nor did I supply a breakdown of marking weights, leaving students to interpret an unclear description of the overall marking criteria. This ambiguity led to misunderstanding, misinterpretation, inequity, and poor alignment with the intended learning outcomes.

Furthermore, many tasks in the CPD module were insufficiently defined, which represented a direct threat to validity and hindered students’ achievement of the intended learning goals. Crooks et al. (1996) highlight that well-defined tasks are critical for accurately evaluating students’ mastery of subject matter, whereas ill-defined tasks compromise the validity of assessments.

To address these threats, it is essential to communicate effectively, clearly explain tasks and expectations, ensure consistency in assessment processes, and routinely review assignment descriptions to confirm alignment with course objectives. This can be achieved by providing comprehensive instructions, outlining detailed grading criteria, and clarifying the purpose and objectives of each task. The revised CPD assignment, reflecting these improvements, is presented in Appendix 2.

Crooks et al.’ (1996) chain model emphasizes that scoring systems often overlook critical aspects of task performance, posing a significant challenge for lecturers in addressing threats to assessment validity. In the CPD module, I identified the scoring system as a major threat to validity, particularly when it fails to accurately capture task performance, leading to student misunderstanding regarding how assignments will be evaluated. Lecturers are responsible for ensuring that assessments are equitable, consistent, and clearly communicated.

A well-designed rubric is an essential tool in this process. It provides explicit marking criteria for evaluating students’ work and supports learning by offering feedback and feedforward comments, highlighting areas for improvement. To accurately capture task performance, a clear rubric is necessary to help students understand how their marks are determined. Providing a rubric in advance allows students to plan effectively and take control of their learning.

As the unit leader, the primary challenge I faced in designing the rubric was ensuring that detailed marking criteria would enable students to fully interpret and understand how their work would be assessed (see Appendices 3 and 4). The revised assessment procedure incorporates a rubric that addresses all four criteria, with six rating levels:

1) Poor/Fail (0% - 39%) – Insufficient work

2) Fail (40% - 49%) – Basic work

3) Pass (50% - 59%) – Satisfactory work

4) Merit (60% - 69%) – Very good work

5) Distinction (70% - 79%) – Excellent work

6) High Distinction (80%+) – Outstanding work

Each rating includes clear descriptors for the four marking criteria, each weighted at 25%. For example, work categorized as “very good” to “excellent” content corresponds to a numerical scale of 69% - 79% out of 100% (see Appendix 3). This structured rubric ensures transparency, consistency, and fairness in assessment, while also providing students with actionable guidance for improving their performance.

Two critical components of effective task assessment are the selection of an appropriate grading scale and the establishment of marking criteria that align with the assessment’s objectives. The initial scoring approach in the CPD module was holistic, which contributed to student confusion and a lack of transparency in understanding how their work would be evaluated. To address this, I propose the adoption of analytic scoring.

Within the CPD module, analytic scoring provides educators with detailed insights into student performance while offering students a transparent breakdown of assessment criteria to guide their work prior to submission. By implementing Crooks et al.’ (1996) analytical scoring framework alongside clearly defined marking criteria, this approach directly addresses the previously identified threats to validity and promotes a fairer, more comprehensible assessment process.

Analytic scoring is particularly effective because it clearly defines each component of the assessment and explicitly demonstrates how marks are allocated, making it easier for students to understand expectations and plan their work accordingly. The revised marking criteria and my approach to addressing the identified threat to validity are presented in Appendix 2.