Analysis of Threats to Validity in the Competitive Product Development Module: A Case Study ()
1. Introduction
Assessment in higher education plays a pivotal role in shaping learning and teaching practices, serving as both a measure of achievement and a tool for improvement. Globally, the effectiveness of educational systems—from individual learners to entire institutions—is often judged through assessment outcomes (Newton, 2010; Mansell et al., 2009; Wiliam, 2008). Within design and engineering education, ensuring the validity of assessments is particularly critical, as valid assessments not only evaluate student competence but also inform instructional design, curriculum enhancement, and pedagogical development (Olds et al., 2005).
This paper explores threats to validity within the context of the Competitive Product Development (CPD) module, a postgraduate design unit, and presents strategies to mitigate these challenges. Drawing from theoretical perspectives, including Messick’s (1989, 1996) unified theory of construct validity and Crooks et al.’ (1996) chain model of assessment, this study identifies how factors such as inadequate task communication, misaligned marking criteria, and overly holistic scoring practices can compromise the accuracy and fairness of student evaluation.
The analysis builds upon insights gained from the author’s participation in the Postgraduate Advanced Certificate in Educational Studies at the University of Cambridge, combining theoretical grounding with practical reflection. By systematically identifying and addressing these threats, the paper advocates for a more transparent, analytical, and evidence-informed assessment approach that aligns with institutional learning outcomes (ILOs) and enhances the credibility of design and engineering education.
Ultimately, this study aims to demonstrate how strengthening the validity of assessment practices can enhance not only the reliability of academic judgments but also the overall quality of student learning and engagement in higher education.
2. Literature Review
Assessment holds a central role in higher education and education more broadly, serving as a fundamental mechanism to determine the extent of students’ learning within their programs of study (Wiliam, 2008). Beyond merely measuring student performance, assessments serve multiple functions, including guiding instructional practices, informing curriculum design, and supporting accountability within educational institutions. To ensure effective assessment and to fully understand its diverse uses and impacts, it is essential to view the assessment system as a framework that provides critical information and shapes individuals’ learning and behaviors (Newton, 2010). This perspective emphasizes that assessment is not an isolated process but is deeply embedded in the educational ecosystem, influencing both teaching strategies and learner engagement.
2.1. Validity in Assessments
Assessments play a pivotal role in the teaching and learning process by offering valuable feedback to instructors on their teaching practices and on students’ developmental progress (Voltz, Sims, & Nelson, 2010). The credibility and usefulness of any assessment are largely determined by its validity, which refers to the degree to which an assessment accurately measures what it is intended to measure. Research on assessment validity often focuses on several key dimensions, including construct validity, content validity, and criterion-related validity—the latter encompassing predictive and concurrent validity (Crooks et al., 1996).
While consistency, or reliability, in assessment refers to the reproducibility of results across different administrations or contexts, validity pertains specifically to the appropriate interpretation and application of assessment outcomes in evaluating learner performance, skills, or knowledge (Chang & Seow, 2018). In essence, a reliable assessment that lacks validity may consistently measure something irrelevant, whereas a valid assessment ensures meaningful and actionable insights into student learning.
Among scholars such as Sireci (2009), Downing (2003), Kane (2001), Moss (1995), and Shepard (1993), there is a consensus that construct validity represents the most comprehensive and enduring conceptualization of validity. Construct validity emphasizes that assessments should measure the theoretical constructs or traits they are intended to evaluate, rather than merely focusing on surface-level content or specific test items. Historically, however, the notion of validity was more narrowly framed. Expert panels were often convened to review tests or programs, providing judgments on whether the items were representative of the intended content. When these panels deemed that the items accurately assessed the relevant aspects of the content, this was referred to as content validity (Wiliam, 2008).
Despite its historical prominence, content validity has faced challenges, particularly when the scope of the content is poorly defined or when the panel’s judgment is subjective. This limitation has led to debates regarding the conceptual clarity of content validity and its appropriateness as a standalone measure of assessment quality. Some validity theorists, including Messick (1989), argue that the term “content validity” is technically inaccurate, advocating instead for an integrated view that considers multiple sources of evidence, including content representation, response processes, internal structure, and consequential validity. These debates underscore that validity is not a static property of an assessment tool but a dynamic construct that depends on context, purpose, and interpretation.
Furthermore, the practical implications of validity in higher education are significant. Valid assessments can guide curriculum improvements, inform pedagogical interventions, and support fair and equitable evaluation of student performance. Conversely, invalid assessments may misrepresent student abilities, undermine instructional effectiveness, and contribute to inequities in educational outcomes. As such, understanding and applying the principles of validity is critical for educators, administrators, and policymakers committed to evidence-based practice and student-centered learning.
In conclusion, validity remains a central concern in assessment research and practice. Its evolution from content-focused judgments to construct-oriented frameworks reflects broader shifts in educational philosophy, emphasizing both accuracy and fairness. By approaching assessment as a complex, multi-dimensional system, educators can ensure that evaluations not only measure learning outcomes effectively but also foster meaningful growth and development among students.
2.2. Messick’s Theory
Borsboom et al. (2004) and Brown et al. (1997) propose that validity can be understood from two complementary perspectives: as an inherent attribute of the test itself or as interpretations derived from test outcomes (Borsboom et al., 2004; MacMillan, 2004; Messick, 1996). The former perspective is commonly applied in psychological and natural sciences testing, where the focus is on the properties of the test items and measurement tools. In contrast, the latter perspective is more prevalent in disciplines such as design education, where assessments of specific assignments are evaluated based on students’ performance and the interpretation of those results.
Messick (1996) provides an extensive examination of threats to validity identified in the assessment literature and proposes a comprehensive framework that expands traditional approaches, which typically focus on content and construct validity. According to Messick, construct validity can be conceptualized in six interrelated components: consequential, substantive, content, external, structural, and generalizability. Each component addresses a distinct aspect of validity, and while different methods may be employed to evaluate them, none should be disregarded. The substantive component, for instance, ensures that the knowledge and skills being assessed accurately reflect the target population and are relevant to the learning material studied, thereby verifying the meaningfulness of the assessment tasks.
Messick (1996) also differentiates between two key attributes of validity. Generalizability, in particular, refers to the degree to which interpretations of assessment outcomes can be extended across tasks, contexts, or groups. Achieving this attribute requires well-designed marking rubrics and clear criteria, which are the responsibility of the first marker or evaluator. As Gearhart et al. (1995) note, comparing students’ marks across grade levels can serve as an illustration of the generalizability attribute of construct validity, emphasizing that careful design and consistent application of assessment criteria are essential for ensuring fair and meaningful evaluation. The template is used to format your paper and style the text. All margins, column widths, line spaces, and text fonts are prescribed; please do not alter them. You may note peculiarities. For example, the head margin in this template measures proportionately more than is customary. This measurement and others are deliberate, using specifications that anticipate your paper as one part of the entire journals, and not as an independent document. Please do not revise any of the current designations.
2.3. Crooks’ Theory
Despite the critical importance of assessing validity, its evaluation is often overlooked in practice. To address this, experts in the field have identified various criteria and frameworks to guide the validation process (Messick, 1995; Linn, 1991; Frederiksen & Collins, 1989; Cole & Moss, 1989; Haertel, 1984). One notable approach is Crooks’ systematic framework, which provides a methodical structure to support validators in evaluating educational assessments (Crooks et al., 1996). This model consists of eight interrelated stages—administration, scoring, aggregation, generalization, extrapolation, evaluation, decision, and impact—each of which contributes to the overall validity of the assessment. Importantly, the assessment’s validity is ultimately constrained by its weakest component, meaning that excessive investment in strengthening other components may be ineffective or even counterproductive (Crooks et al., 1996) (Figure 1).
![]()
Figure 1. Educational assessment paradigm for validation and design (adapted from Crooks et al., 1996).
To ensure that validity is evaluated effectively, it is imperative to carefully analyze the interconnected threats and challenges that may compromise it. These threats, which are discussed in the following sections, can emerge at any stage of the assessment process and must be addressed systematically to maintain the integrity and usefulness of assessment outcomes.
3. Threats to Validity in CPD Case Study
The curriculum enhancement described in this report addresses threats to validity in the CPD module, a 20-credit unit taught to Master’s in Design students during the second semester of 2024/2025. In this module, students are required to evaluate an existing business or brand whose products are targeted at the retail consumer market. They are tasked with identifying new product-market opportunities, which may involve exploring new or existing sectors or entirely untapped marketplaces.
The module aims to:
1) Enable students to develop an analytical and critical understanding of product design and development, considering a broad range of business factors, including external pressures such as political, social, environmental, and other contextual changes.
2) Equip students to evaluate customer requirements alongside relevant market constraints and opportunities within the applicable timeframe and competitive landscape, while assessing financial and legal implications.
3) Facilitate the identification and comprehensive analysis of the phases involved in product development and its life cycle, with careful consideration of time and resource allocation.
4) Develop students’ insightful understanding of contemporary computer-aided tools used in product design, assessment, and manufacturing processes, and their analytical comprehension of these tools’ significance and function throughout different stages of product development.
Additionally, students are expected to critically evaluate and differentiate between various design and evaluation tools and processes, in order to formulate coherent project plans and strategies and to select the most appropriate tools for the task.
In 2025, the module was delivered through a combination of lectures, seminars, three-dimensional (3D) design activities, videos, and interactive classroom sessions. Students’ chosen projects were assessed via a 4000-word written assignment, submitted at the conclusion of the module.
3.1. Summary of Threats to Validity in Assessment
Black et al. (2010), drawing on Crooks et al.’ (1996) chain model and influenced by Messick’s and Kane’s work on validity, outlined an approach to assessment validation that emphasizes the interconnected nature of assessment processes. In Crooks’ model, assessment is represented as a chain of eight linked stages, where the strength of the whole process depends on each individual connection. A weakness or breakdown at any stage introduces a threat to validity, as it can undermine the overall fairness, accuracy, or usefulness of the assessment. This model encourages teachers and lecturers to examine each stage critically, identifying where issues may arise and ensuring that assessment practices remain robust, transparent, and defensible.
3.2. The Purpose of Assessment as Defined by the QAA
The revised Code of Practice for student evaluation by the Quality Assurance Agency (QAA) defines pedagogy as the facilitation of student learning through tasks that assess their abilities, provide feedback to enhance performance, and encourage reflective learning approaches. In the field of design, learning can be significantly enhanced by critically guiding students in the development of optimal design and engineering skills (Quality Assurance Agency for Higher Education, 2006: p. 4, as cited in Heather et al., 2009: p. 134).
4. Analysis of Threats to Validity in CPD
Evaluating student performance is one of the most critical responsibilities of lecturers, and essay writing remains one of the most widely used forms of assessment. However, selecting an appropriate topic and ensuring alignment with learning objectives can be challenging (Heather et al., 2009; Race, 2001). According to Heather et al. (2009), the most effective approach is to ensure that the chosen assessment method is appropriate, valid, and reliable for the intended learning outcomes.
Based on my recent experience (2024/2025) with the CPD module, combined with my efforts to enhance its quality and informed by student feedback and critical reflections, several threats to validity have been identified and are presented in this report:
1) Administration: The task was not adequately communicated, corresponding to the administration stage in Crooks’ framework (Crooks et al., 1996). Crooks et al. (1996) argue that poor performance may not reflect a student’s inability but rather a lack of understanding of the assignment or difficulty articulating acquired knowledge. Students expressed concerns about how to meet the extensive research requirements specified in the assignment brief. This issue stems from unclear communication regarding the task, insufficient guidance on the expected level of critical analysis, and a scoring system that does not capture all essential aspects of the task. Consequently, students were unsure how to ensure the quality of their submissions due to confusion over mark allocation, grading procedures, and misalignment between marking criteria and intended learning outcomes (ILOs), as highlighted in Crooks et al.’ (1996) theory.
2) Scoring: The second link in Crooks’ validation chain (Figure 1) relates to scoring. In this module, scoring was overly holistic, failing to capture key qualities of the students’ work. Additionally, the marking process and rubric were not clearly presented, which further undermined the validity of the assessment.
The remaining elements of Crooks et al.’ (1996) framework that were relevant include:
3) The level of evaluation in relation to the subject matter.
4) The depth of task analysis and guidance on how students should approach the task.
5) The method by which marks are awarded.
Overall, the identified threats to validity primarily relate to the misalignment or neglect of marking criteria, intended learning outcomes, and the absence of clearly communicated mark weighting to students.
A comparison between the five identified threats to validity in this assessment and Crooks et al.’ (1996) eight-link model is presented in Table 1.
Table 1. Analysis of threats to validity, weaknesses, and Crooks’ theory.
Threats to Validity |
My Weakness in Assessment |
Crook’s (1996) theory |
Disconnect betweenmarking criteria andlearning objectives |
Lack of clarity in taskevaluation methods and alignment with learning objectives |
Task communication and alignment with intended learning outcomes |
Uncertainty about how marks will be assigned |
Lack of understanding regarding mark allocation criteria |
Lack of clarity in grading procedures |
Misalignment between marking criteria andintended learning outcomes |
Lack of understanding regarding mark allocation criteria |
Misalignment of assessment with curriculum objectives |
Absence of marking weights provided to students |
Failure to present marking weights to students, leading to ambiguity |
Lack of transparency in grading criteria |
Overly holistic scoring methods |
Scoring methods overlyfocused on holistic assessment |
Lack of detailed, analytic assessment criteria |
5. Effect of Threats Identified in Crooks’ Theory on CPD
Assessment
Black et al., (2010) building on Crooks et al.’ (1996) model and drawing on the validity research of Messick and Kane, described an effective approach to assessment validation. In this approach, the assessment process in Crooks’ chain model is divided into eight interconnected stages, as summarized in Table 2. The validity of the assessment depends on the integrity of all eight links, with particular attention given to any weak connections where lecturers may need to investigate potential issues. In the context of continuous classroom assessment for formative purposes, instructors typically need to focus on specific stages of this chain—for example, administration and scoring in the CPD module—which are highlighted in red in Table 2.
Table 2. Threats to validity in CPD associated with each of the eight links outlined in Crooks et al. (1996).
Link |
Threats in CPD |
Administration |
Low motivation Assessment anxiety Inappropriate assessment conditions |
Additional Threats identified |
Task or response not communicated- (Misalignment of the marking criteria, the intended learning objectives ILOs, and rubric) |
Scoring |
Scoring fails to capture important qualities of task performance Undue emphasis on some criteria, forms or styles of response Lack of intra-rater or inter-rater consistency Scoring too analytic Scoring too holistic |
Aggregation |
Aggregated tasks too diverse Inappropriate weights given to different aspects of performance |
Generalization |
Conditions of assessment too variable Inconsistency in scoring criteria for different tasks Too few tasks |
Extrapolation |
Conditions of assessment too constrained Parts of the target domain not assessed or given little weight |
Evaluation |
Poor grasp of assessment information and its limitations Inadequately supported construct interpretation Biased interpretation or explanation |
Decision |
Inappropriate standards Poor pedagogical decision |
Impact |
Positive consequences not achieved Serious negative impact occurs |
5.1. Administration-Task Not Communicated
Gibbs & Simpson (2004) developed a set of pedagogical principles that provide a foundation for assessment design aimed at enhancing learning. These principles highlight that assessments must be communicated clearly and uphold high standards. In the CPD module, students reported confusion regarding the assignment because the lecturer who designed it did not adequately brief them on the content, requirements, scoring methods, or grade expectations. This lack of communication led to misalignment between the intended learning outcomes (ILOs) and the marking criteria, creating a significant threat to assessment validity.
5.2. Scoring
5.2.1. Scoring Fails to Capture Key Qualities of Task Performance
In the CPD assignment, the scoring scheme evaluated certain elements while neglecting or failing to clarify others. This created misunderstandings among students, as the marking process did not accurately capture the constructs that the lecturer intended to assess. As a result, assessment validity was compromised, making this a clear threat identified within the CPD module.
5.2.2. Holistic vs. Analytic Scoring
Holistic scoring evaluates a student’s work as a whole, considering overall quality, coherence, creativity, and other subjective factors, rather than focusing on individual components. While this approach can be advantageous for assessing complex, multidimensional tasks where overall impression is important, it has notable limitations. Specifically, holistic scoring can lack transparency and consistency, as different evaluators may interpret quality differently. In the CPD module, the use of holistic scoring was identified as a threat to validity during the module review.
In contrast, analytic scoring evaluates each component of the task separately, providing detailed feedback on specific criteria. This approach enhances clarity, transparency, and fairness, allowing students to understand precisely how to improve their work and achieve higher marks. Therefore, adopting an analytic scoring method is recommended to strengthen assessment validity and guide student learning effectively.
6. Resolving Threats in CPD
Moving forward, it is essential to address and rectify the identified threats to validity in future assessments. This requires a comprehensive review of the assessment framework, refining task instructions to ensure clarity, aligning marking criteria with intended learning outcomes, providing students with clear mark allocations, adopting more specific assessment criteria, and incorporating detailed rubrics to support transparent and consistent grading.
The strategies outlined below are specifically designed to enhance the validity of assessment in the CPD unit. For reference, the revised assessment framework is presented in Appendix 2, in comparison to the previous assessment framework shown in Appendix 1.
6.1. Better Communication of the Task
A lack of clear communication can pose significant threats to validity in design education. When tasks are not communicated effectively, students may focus on irrelevant aspects or fail to achieve key learning objectives. Ensuring clarity and transparency in task design is therefore essential for maintaining the validity of assessments. In our department, there was limited emphasis on aligning tasks with intended learning outcomes (ILOs) and marking criteria. Consequently, I did not provide students with detailed guidance on how their work would be evaluated, nor did I supply a breakdown of marking weights, leaving students to interpret an unclear description of the overall marking criteria. This ambiguity led to misunderstanding, misinterpretation, inequity, and poor alignment with the intended learning outcomes.
Furthermore, many tasks in the CPD module were insufficiently defined, which represented a direct threat to validity and hindered students’ achievement of the intended learning goals. Crooks et al. (1996) highlight that well-defined tasks are critical for accurately evaluating students’ mastery of subject matter, whereas ill-defined tasks compromise the validity of assessments.
To address these threats, it is essential to communicate effectively, clearly explain tasks and expectations, ensure consistency in assessment processes, and routinely review assignment descriptions to confirm alignment with course objectives. This can be achieved by providing comprehensive instructions, outlining detailed grading criteria, and clarifying the purpose and objectives of each task. The revised CPD assignment, reflecting these improvements, is presented in Appendix 2.
6.2. Design Grading Scoring System and Rubric to Capture
Important Qualities of Task Performance
Crooks et al.’ (1996) chain model emphasizes that scoring systems often overlook critical aspects of task performance, posing a significant challenge for lecturers in addressing threats to assessment validity. In the CPD module, I identified the scoring system as a major threat to validity, particularly when it fails to accurately capture task performance, leading to student misunderstanding regarding how assignments will be evaluated. Lecturers are responsible for ensuring that assessments are equitable, consistent, and clearly communicated.
A well-designed rubric is an essential tool in this process. It provides explicit marking criteria for evaluating students’ work and supports learning by offering feedback and feedforward comments, highlighting areas for improvement. To accurately capture task performance, a clear rubric is necessary to help students understand how their marks are determined. Providing a rubric in advance allows students to plan effectively and take control of their learning.
As the unit leader, the primary challenge I faced in designing the rubric was ensuring that detailed marking criteria would enable students to fully interpret and understand how their work would be assessed (see Appendices 3 and 4). The revised assessment procedure incorporates a rubric that addresses all four criteria, with six rating levels:
1) Poor/Fail (0% - 39%) – Insufficient work
2) Fail (40% - 49%) – Basic work
3) Pass (50% - 59%) – Satisfactory work
4) Merit (60% - 69%) – Very good work
5) Distinction (70% - 79%) – Excellent work
6) High Distinction (80%+) – Outstanding work
Each rating includes clear descriptors for the four marking criteria, each weighted at 25%. For example, work categorized as “very good” to “excellent” content corresponds to a numerical scale of 69% - 79% out of 100% (see Appendix 3). This structured rubric ensures transparency, consistency, and fairness in assessment, while also providing students with actionable guidance for improving their performance.
6.3. Making the Scoring More Analytical
Two critical components of effective task assessment are the selection of an appropriate grading scale and the establishment of marking criteria that align with the assessment’s objectives. The initial scoring approach in the CPD module was holistic, which contributed to student confusion and a lack of transparency in understanding how their work would be evaluated. To address this, I propose the adoption of analytic scoring.
Within the CPD module, analytic scoring provides educators with detailed insights into student performance while offering students a transparent breakdown of assessment criteria to guide their work prior to submission. By implementing Crooks et al.’ (1996) analytical scoring framework alongside clearly defined marking criteria, this approach directly addresses the previously identified threats to validity and promotes a fairer, more comprehensible assessment process.
Analytic scoring is particularly effective because it clearly defines each component of the assessment and explicitly demonstrates how marks are allocated, making it easier for students to understand expectations and plan their work accordingly. The revised marking criteria and my approach to addressing the identified threat to validity are presented in Appendix 2.
7. Implications for Policy and Practice
By analyzing threats to validity, comparing my own weaknesses to Crooks’ theory as outlined in Table 1, and applying Crooks et al.’ (1996) chain model to evaluate assessment validity in design and engineering, I have gained valuable transferable knowledge that can inform policy development. Recognizing the importance of Crooks’ framework for ongoing professional development—particularly for new staff—I have proposed to my manager that I deliver a lecture and conduct training sessions on implementing Crooks’ chain model. This initiative aims to improve assessment practices within the design department and enhance the overall student learning experience.
8. Conclusion
Validity is a critical factor in ensuring the quality and effectiveness of assessments in design and engineering. Through an in-depth analysis of Messick’s (1996) theory and Crooks et al.’ (1996) chain model, which address various aspects of validity—including construct validity and criterion-related validity—this report underscores the importance of employing analytic scoring methods. Such methods enhance transparency, alignment with learning outcomes, and overall assessment validity, ultimately supporting both fair evaluation and improved student learning.
Appendix 1: Old Assignment Brief: Before Identifying
Threats to Validity in a CPD Study
Faculty of Science and Technology, Department of Design & Engineering
Unit Title: Competitive Product Development |
Assessment Title: New product development analysis |
Unit Level: 7 |
Assessment Number: 1 of 1 |
Credit Value of Unit: 20 |
Date Issued: 29/01/2024 |
Marker(s): Dr Moamer Gashoot |
Submission Due Date: 19/05/2024 Time: 12.30 pm |
Quality Assessor: |
Submission Location: Turnitin |
|
Feedback method: Turnitin |
This is an individual assignment which carries 100% of the final unit mark
Assessment Task
Businesses are competing with one another to create novel, distinctive products and advance their position in the marketplace, however, most of those products are unable to gain a competitive edge.
Your task is to evaluate a recently produced product, as well as the launch procedure, for a Company, which provides consumers with cutting-edge designed product.
Using modern evaluation methods, it is your responsibility to find and create a competitive framework for this company’s product development that will make the whole process of developing and launching the product successful and enhance the chance that the product will be accepted by users and launched successfully.
To continue competing in the market, the new competitive product evaluation, and launch model is based on a review of the literature an analysis of the stages of product development and life cycle in terms of resources and time to avoid threat to a product failure.
You must demonstrate a critical awareness of the methods of evaluation, the evaluation tool, and the manufacturing, as well as a critical comprehension of their place, and role in the various stages of product development.
As results of your evaluation, provide professional recommendations for improving the product based on what you have learned.
Submission Format
The findings are to be submitted as follows: -
A Microsoft Word report format submitted electronically for assessment purposes via Turnitin with a word content of 3000 +/− 300 words (excluding title page, table of contents, tables or spreadsheets, and reference list) with double spacing, 12-point Times New Roman font, and one-inch margins.
MARKING CRITERIA
Criteria 1. Assess consumer trends and business ethics for determining their potential impact on branding image and product/business success in a global environment.
Criteria 2. Assess the strengths and weaknesses of organizations in strategically planning new product /business activities.
Criteria 3. Have a critical awareness for product design, evaluation and manufacture, and critical understanding of their place and role in the various stages of product development.
Criteria 4. Provide professional recommendation for continuous improvement processes for regularly reviewing branding, business activities and implementing needed changes.
Learning Outcomes
This assignment tests your ability to:
1) To assess customer needs and the related market constraints and opportunities within the appropriate time and competition framework and be able to evaluate the financial and legal implications;
2) Identify and fully analyse the stages in the product development and life cycle, in terms of time and resources;
3) Have a critical awareness of modern computer tools for product design, evaluation and manufacture, and critical understanding of their place and role in the various stages of product development;
4) Critically appraise and identify types of design and evaluation tools and processes, to draw up strategies and the project plan, and to select the optimum tools;
5) Critically understanding the global context and in particular low-cost manufacturing issues and import/export opportunities.
QUESTIONS ABOUT THE BRIEF
Questions regarding your assignment will be discussed in class seminars or on a one to one basis via email and skyp.
Signature Marker _______________________
Appendix 2: New Assignment Brief: After Resolving Threats
to Validity in a CPD Study
Faculty of Science and Technology, Department of Design & Engineering
Unit Title: Competitive Product Development |
Assessment Title: New product development analysis |
Unit Level: 7 |
Assessment Number: 1 of 1 |
Credit Value of Unit: 20 |
Date Issued: 29/01/2025 |
Marker(s): Dr Moamer Gashoot |
Submission Due Date: 19/05/2025 Time: 12.30 pm |
Quality Assessor: |
Submission Location: Turnitin |
|
Feedback method: Turnitin |
This is an individual assignment which carries 100% of the final unit mark
New Assignment |
Addressing CPD Threat
to Validity |
Crooks’ Threats to Validity |
Businesses are competing with one another to create novel, distinctive products and advance their position in the marketplace; however, most of those products are unable to gain a competitive edge. Your task is to evaluate a recently produced product, as well as the launch procedure, for a Company dealing with design and engineering, which provides consumers with
cutting-edge designed product. You will need to address the following items in full and complete detail: Provide a detailed description of the product they offer, and what is the product classification? Part of the task is to identify product development stages, life cycle costing, time to launch, and the methods of reducing development time. Discuses strengths, weaknesses of the new product, business design concept and how the company meet customer needs. Assess the opportunities that emerging consumer trends and company ethics that may led to the success of the new idea. Assess the company’s tangible and intangible strengths in providing the new product Evaluate all business concepts such as, the design various stages, Global marketing, marketing intelligence, Global branding strategy, advertising, and promotion that used to reach their client. Using modern evaluation methods, assess the level of risk, product acceptance
(look at customer needs and wants and how they achieve customer satisfaction and loyalty), product development strategy, and directly link this to the impact on the organisation’s competitive position. You must demonstrate a critical awareness of the methods of evaluation, the evaluation tool, and the manufacturing as well as a critical comprehension of their place, and role in the various stages of product development. Assess the company strategy and how it design and manufacture low-cost product for local and global markets (Global context). Building on this you will then be able to identify what you believe the key product development opportunities are for the businesses, providing a detailed analysis and evaluation of your findings as well as highlighting the main activities required to operationalize and implement them. It is important that you draw on key theories, quality peer reviewed literature, business models and key tools and techniques relating to new product design and development as well strategies needed in the new product introduction process as this will help guide your analysis and support your ideas for the future. As results of your evaluation, provide professional recommendations, or, competitive framework, or draw up strategy for this company’s product development based on what you have learned. |
a) Communicating the task better |
Administration Task not communicated |
New Marking Criteria |
Addressing CPD threat to validity |
Crooks’ Threats to validity |
Criteria 1. Subject knowledge and understanding – including theories and concepts. (25%) Criteria 2. Assess the strengths and weaknesses of organizations in strategically planning new product /business activities. (25%) Criteria 3. Cognitive skills - including analysis, evaluation, and critical judgement. (25%) Criteria 4. Provide professional recommendations, design competitive framework, and decision-making. (25%) |
b) Making the
scoring rubric
more analytical |
Scouring -Scoring too holistic vs Scoring too analytic. |
Learning Outcomes |
CPD Threats to Validity |
Crooks’ Threats to Validity |
1) To assess customer needs and the related market constraints and opportunities within the appropriate time and competition framework and be able to evaluate the financial and legal implications; 2) Identify and fully analyse the stages in the product development and life cycle, in terms of time and resources; 3) Have a critical awareness of modern computer tools for product design, evaluation and manufacture, and critical understanding of their place and role in the various stages of product development; 4) Critically appraise and identify types of design and evaluation tools and processes, to draw up strategies and the project plan, and to select the optimum tools; |
c) Alignment of the marking criteria, the intended learning objectives ILOs,
and rubric. |
Inappropriate weights given to different aspects of performance |
Appendix 3: New Grading Rubric Aligned with the Marking Criteria and the Intended Learning Objectives
Level 7 Grade Range |
Subject Knowledge
and Understanding, Including Theories and Concepts (25%) |
Cognitive
Skills, Including Analysis, Evaluation,
and Critical Judgement. (25%) |
Practical
Skills, Including the Use of Techniques of Analysis and Enquiry and of Integrating Theory and Practice (25%) |
Transferable
Skills, Including Communication, Exercise of Initiative, Personal Responsibility and Decision-Making. (25%) |
High Distinction 80%+ Exceptional work overall |
Feedback |
The work demonstrates exceptional knowledge and understanding in this field of study. The work demonstrates an exceptional in-depth mastery of theories, paradigms, concepts and principles associated with it, beyond what has been taught. |
The work demonstrates an exceptional ability to deal with complexity, contradictions and incomplete information. Independent critical analysis of current research/knowledge and cogently argued alternative approaches are present. |
The work demonstrates an outstandingly accomplished and innovative application of discipline-specific practical and
problem-solving skills. |
Ideas, problems and solutions are communicated to an exceptional level in verbal, written and electronic formats. |
Feedforward |
Future work could contain even further broader and deeper understanding of theories and concepts. |
Future work could offer an even deeper and
more extensive critical approach with extended analysis, synthesis and evaluation. Further tightly structured,
highly stimulating and very rigorous
arguments and reasoning could be developed. |
Future work could demonstrate an even higher level of mastery of practical skills, research skills and the integration of theory and practice. |
Future work could demonstrate even further- enhanced skills. |
Distinction 70% - 79% Excellent work overall |
Feedback |
The work demonstrates excellent knowledge and understanding in this field of study. The work demonstrates an excellent in-depth mastery of theories, paradigms, concepts and principles associated with it, beyond what has been taught. |
The work demonstrates an excellent ability to deal with complexity, contradictions and incomplete information. Independent critical analysis of current research/knowledge and cogently argued alternative approaches are present. Excellent creative flair and originality demonstrated. The work demonstrates clear ability to undertake further specialist research and make significant contributions to the subject. |
The work demonstrates an excellent application of discipline-specific practical and problem-solving skills. Autonomous completion of practical tasks and/or processes with an excellent degree of accuracy,
co-ordination and proficiency has been demonstrated. |
Ideas, problems and solutions are communicated to an excellent level in verbal, written and electronic formats. The style is accurate, fluent and sophisticated. Excellent numeracy and digital literacy skills are demonstrated. Clear, authoritative and valuable contributions to group work and/or project work are demonstrated along with excellent teamwork and leadership skills. |
Feedforward |
A broader range of sources could help develop subject knowledge and conceptual understanding further. |
In future work further synthesis and critical analysis could be demonstrated, along with further evidence of the ability to handle complexity. Future work could demonstrate additional very tightly structured, highly stimulating and very rigorous arguments and reasoning. |
Future work could demonstrate deeper insights and more extensive innovative and original responses, along with higher levels of problem solving and research skills, in the context of integrating theory and practice. |
Future work could demonstrate an even higher level of development in a range of transferable skills. |
Merit 60% - 69% Very good work overall |
Feedback |
The work demonstrates very good knowledge and understanding in this field of study. The work demonstrates very good in-depth mastery of theories, paradigms, concepts and principles associated with it, beyond what has been taught. |
The work demonstrates a very good ability to deal with complexity, contradictions and incomplete information. Independent critical analysis of current research/knowledge and cogently argued alternative approaches are present. |
The work demonstrates a very good application of discipline-specific practical and
problem-solving skills. Autonomous completion of practical tasks and/or processes with a very good degree of accuracy, co-ordination and proficiency has been demonstrated. A full range of very good technical, creative and/or artistic skills has been demonstrated. |
Ideas, problems and solutions are communicated to a very good level in verbal, written and electronic formats. The style is accurate, fluent and sophisticated. Very good numeracy and digital literacy skills are demonstrated. A high degree of autonomy demonstrated. |
|
Feedforward |
In future work extensive use of sources could support the development of deeper understanding of the current knowledge in this field of study. |
Future work could demonstrate confidence in handling complex ideas and contradictions. More extensive skills of analysis and synthesis could be developed. In future, work could be communicated in a stimulating and rigorous style to demonstrate a higher level of academic skill. |
Future work could demonstrate further insights and more innovative and original responses, along with higher levels of problem solving and research skills, in the context of integrating theory and practice. |
Future work could demonstrate a higher level of development in a range of transferable skills. |
Pass 50% - 59% Good work overall |
Feedback |
The work demonstrates good knowledge and understanding in this field of study. The work demonstrates good
in-depth mastery of theories, paradigms, concepts and principles associated with it, beyond what has been taught. |
The work demonstrates a good ability to deal with complexity, contradictions and incomplete information. Sound critical analysis of current research/knowledge and cogently argued alternative approaches is present. |
The work demonstrates a good application of discipline-specific practical and problem-solving skills. |
Ideas, problems and solutions are communicated to a good level in verbal, written and electronic formats. The style is accurate, fluent and sophisticated. Good numeracy and digital literacy skills are demonstrated. |
Feedforward |
In future work more detailed knowledge and understanding needs to be demonstrated with greater consistency, underpinned by a wide range of relevant sources. |
Future work could demonstrate confidence in handling complex ideas and contradictions. Further skills of analysis and synthesis could be developed. In future, work could be communicated in a stimulating and rigorous style to demonstrates a higher level of academic skill. |
Future work could demonstrate further insights and innovative and original responses, along with higher levels of problem solving and research skills, in the context of integrating theory and practice. |
Future work could demonstrate further development of a range of transferable skills. |
Fail 40% - 49% Basic work overall |
Feedback |
The work demonstrates basic knowledge and understanding in this field of study. The work demonstrates basic mastery of theories, paradigms, concepts and principles associated with it. Limited exploration of possible sources. |
The work demonstrates a basic ability to deal with some complexity, contradictions and incomplete information. A basic analysis of current research/knowledge with some basic arguments present. |
The work demonstrates a basic level of application of discipline-specific practical and problem-solving skills. Practical tasks and/or processes are completed to a basic level of accuracy, co-ordination and proficiency. A limited range of technical, creative and/or artistic skills has been demonstrated. |
Ideas, problems and solutions are communicated to a basic level in verbal, written and electronic formats. The style is limited in accuracy, and fluency. Basic numeracy and digital literacy skills are demonstrated. |
|
Feedforward |
In future work more detailed knowledge and understanding needs to be demonstrated with greater consistency, underpinned by a wide range of relevant sources. |
Future work needs to demonstrate a greater level of critical analysis, backed up with the appropriate use of literature. |
Future work needs to make much clearer links between theory and practice, and greater use of research skills. |
Future work must demonstrate the development of a range of transferable skills. |
Poor fail 0% - 39% Insufficient work overall |
Feedback |
The work demonstrates insufficient knowledge and understanding in this field of study. The work demonstrates insufficient mastery of theories, paradigms, concepts and principles associated with it. |
The work demonstrates a limited and insufficient ability to deal with complexity, contradictions and incomplete information. Very limited and insufficient analysis of current research/knowledge with arguments lacking or unclear. |
The work does not demonstrate a basic level of application of discipline-specific practical and problem-solving skills. |
Ideas, problems and solutions are inadequately communicated in verbal, written and electronic formats. The style lacks accuracy and fluency. |
Feedforward |
In future work significant further reading, research and independent study and thinking are needed to develop and demonstrate detailed understanding in the field of study. |
Future work needs to demonstrate a much greater level of critical analysis, backed up with the appropriate use of literature. |
Clear links between theory and practice are needed in future work, along with evidence of appropriate research skills. |
Future work must demonstrate the development of a range of transferable skills. |