AI-Enabled Comprehensive Quality Assessment in Higher Vocational Education: A Review and Framework Proposal ()
1. Introduction
As a distinct type of education, vocational education relies heavily on its student assessment system to guide the cultivation of high-quality technical and skilled talents. However, traditional assessment models have long been plagued by the drawback of emphasizing “three priorities and three neglects”: prioritizing outcomes over processes, knowledge over competency, and uniformity over individuality [1]. Although mechanisms like point-based systems have been introduced to enhance objectivity [2], most systems remain reliant on manual record-keeping and simplistic statistics, suffering from inherent flaws such as data fragmentation, delayed feedback, and low levels of intelligence [3]. These limitations hinder a fundamental transformation of the assessment paradigm. In recent years, the rapid development of big data and artificial intelligence (AI) technologies has opened new possibilities for addressing these challenges. The emergence of fields like Learning Analytics and Educational Data Mining has made it feasible to conduct deep mining and intelligent interpretation of whole-cycle, multi-dimensional student behavioral data [4]. The core value of AI-enabled student assessment lies in its potential to shift the focus from “static judgment” to “dynamic empowerment”, and to elevate assessment from a mere management tool to an integrated educational support system. In the context of this paper, the term “comprehensive quality” refers to the multi-faceted development of vocational students, encompassing key dimensions such as academic knowledge, practical and operational skills, professional ethics and work attitude, as well as teamwork and communication abilities. Within this context, this paper aims to systematically review research progress in the field of AI-enabled assessment for higher vocational students. It seeks to clarify the evolutionary trajectory, analyze critical issues, and attempt to construct a forward-looking theoretical framework. The objective is to provide a reference for deepening research and fostering innovation in practice within this domain.
2. From Traditional Point Systems to Intelligent
Assessment: Evolutionary Trajectory and Research Status
The evolution of student comprehensive quality assessment systems profoundly reflects the synergistic development of educational philosophies and technological capabilities. This study categorizes this evolution into three distinct stages with generational characteristics to elucidate the underlying logic of its progression from instrumental rationality to value rationality.
2.1. Traditional Point Systems: Initial Attempts at Quantitative
Management
Early research and practice primarily treated point systems as quantitative management tools aimed at enhancing the objectivity and operability of assessment. Their application was concentrated in specific areas such as scholarship awards and extracurricular credentialing. By converting behaviors into points, these systems partially mitigated the drawbacks of purely subjective evaluation and provided basic incentives for student participation. However, systems at this stage were essentially Management Information Systems (MIS), with their core functionality limited to data recording, storage, and linear querying. Lacking deep analytical capabilities, the resulting points were often static, isolated numerical values, unable to reveal the correlations and motivations behind behaviors. Consequently, they failed to instigate a fundamental change in the assessment model, with their “management” attribute far outweighing their “educational” attribute.
2.2. Informatization Platforms: Data Silos and Process
Optimization
With the proliferation of information technology, student comprehensive quality management entered a platform-based stage. A series of customized systems were developed and implemented [5], achieving electronic data entry and online management processes, which significantly improved administrative efficiency. However, this phase exposed a new core problem: different platforms (e.g., for classroom, dormitory, and club management) often operated independently with disparate data standards, creating robust “data silos” [6]. This kept the assessment perspective fragmented, making it difficult to construct an integrated portrait of student behavior. More critically, the underlying assessment logic remained fundamentally summative, focusing on presenting and archiving results. It was unable to dynamically track student growth processes or provide proactive interventions, severely lacking diagnostic, formative, and developmental functions.
2.3. The AI-Enabled Stage: Intelligence, Personalization, and
Predictability
Currently, the research frontier is advancing towards a new paradigm driven by artificial intelligence. The transformation at this stage is fundamental, characterized by three key dimensions:
1) From Data Aggregation to Holistic Profiling: Utilizing technologies like the Internet of Things (IoT) and API interfaces, systems can automatically collect and integrate multi-source, heterogeneous data to construct a dynamic, multi-dimensional digital profile of the student, providing a comprehensive, real-time data foundation for assessment.
2) From Statistical Description to Intelligent Modeling: The introduction of algorithms such as Natural Language Processing (NLP) [7], machine learning clustering, and predictive analytics enables systems to identify complex behavioral patterns, uncover latent correlations, and predict developmental trends. This represents a leap from describing the present state to predicting risks and guiding behaviors.
3) From Management Tool to Educational Vehicle: The fundamental purpose of assessment has shifted. It no longer primarily serves screening and reward/ punishment mechanisms but transforms into a means of education itself. Through visualized feedback, personalized pathway recommendations, and precise early-warning interventions, it aims to stimulate students’ internal motivation and support their personalized growth, ultimately realizing the educational essence of “using assessment to promote development”.
This evolutionary trajectory clearly demonstrates that student assessment is undergoing a profound paradigm shift: from “experience-driven” to “data-driven”, from a “uniform standard” to “personalized measurement”, and from “managing outcomes” to “governing processes”.
3. Analysis of Core Technical Framework and Key
Components
A mature AI-enabled student assessment system features a technical architecture that can be deconstructed into three progressively advanced, interconnected core layers. Together, they form a complete closed loop from data perception to educational intervention.
3.1. Data Fusion Layer: The Foundation for Constructing a
Student Digital Profile
The efficacy of the system is fundamentally rooted in the breadth, quality, and integration of data. The core task of this layer is to break down administrative and managerial barriers by aggregating multi-source, heterogeneous information—such as academic performance, classroom behavior, dormitory conduct, and consumption data—through methods including IoT sensors, API interfaces, and manual input [8]. Critically, in the vocational education context, this data ecosystem must be expanded to include industry- and skill-specific sources, such as workshop logbooks, internship performance reports from enterprise supervisors, and skill certification outcomes, to fully capture the practical dimension of student development. Building upon this, data cleaning, standardization, and fusion techniques are employed to construct a dynamic, multi-dimensional “student digital profile”. This provides a comprehensive, accurate, and consistent data substrate for upper-layer intelligent analysis. This process is the essential prerequisite for transitioning from fragmented awareness to a holistic understanding.
3.2. Intelligent Model Layer: The Core Engine for Achieving a
Cognitive Leap
This layer embodies the system’s intelligence and is responsible for the critical function of extracting knowledge from data. Current research primarily focuses on three types of core models:
Dynamic Weighting Model: By incorporating a configurable rules engine, the weights of assessment indicators can adaptively adjust based on the academic calendar stage and key educational priorities (e.g., study style construction, labor education), thereby enhancing the guidance capability and timeliness of the assessment system [9].
(b) Clustering Analysis Model: Using unsupervised learning algorithms (e.g., K-means), this model automatically identifies and categorizes student behavior patterns. K-means is particularly well-suited for this educational purpose as it efficiently groups students based on multi-dimensional behavioral features (e.g., study habits, workshop engagement, social activity), enabling the identification of distinct student archetypes (e.g., “theory-strong but practice-weak,” “hands-on innovators”) without pre-defined labels, which facilitates targeted group interventions. It reveals latent group characteristics—such as “academic”, “practical”, or “social” types—providing a basis for implementing targeted guidance and precise resource allocation [10].
(c) Predictive Early-Warning Model: Based on time-series analysis and machine learning algorithms, this model establishes behavioral baselines for individual students. Time-series analysis is ideally suited for predicting academic or behavioral risk, as it can model a student’s behavior as a sequence of data points over time, identifying significant deviations from their personal historical pattern or from group norms, which serves as a robust indicator for early warning. It enables the early identification and warning of potential issues, such as academic risks or psychological crises, thereby promoting a fundamental shift in management mode from passive response to proactive intervention [11].
3.3. Feedback and Intervention Layer: The Pathway to Realizing
Educational Value
The outcomes of intelligent analysis must be translated into tangible educational value through effective feedback and intervention mechanisms. This layer interacts directly with end-users and manifests itself in three key aspects:
(a) Visualized Growth Portfolios: Utilizing visualization tools such as radar charts and trend curves, complex data are transformed into intuitive growth reports accessible to students, teachers, and parents. This facilitates student self-reflection (metacognition) and goal management [12].
(b) Personalized Development Recommendations: Based on clustering and predictive analytics results, the system generates tailored activity suggestions, learning resources, and improvement strategies for individual students, truly implementing the concept of “one plan per student” [13].
(c) Collaborative Intervention Mechanisms: Data-driven insights are translated into concrete educational actions, such as targeted teacher guidance and structured school-family communication. This creates a multi-stakeholder educational synergy, ultimately completing the value chain from “data” to “insight” to “action” [14].
4. Key Challenges and Theoretical Reflections
Advancing AI-enabled student assessment necessitates a clear-eyed recognition of its potential risks and critical reflection. Three major challenges currently prevail:
4.1. The Dilemma of Data Privacy and Ethical Governance
While large-scale, multi-dimensional data collection enhances assessment precision, it simultaneously raises serious privacy concerns [15]. Defining the boundaries of data collection, ensuring the effectiveness of anonymization techniques, and balancing data utility with individual privacy rights all demand robust ethical guidelines and technical standards. There is an urgent need to establish a comprehensive privacy protection framework incorporating mechanisms such as tiered data authorization and usage audit trails.
4.2. Challenges of Algorithmic Fairness and Explainability
Machine learning algorithms risk inheriting or even amplifying inherent biases present in training data, potentially leading to unfair assessments of specific student groups (e.g., economically disadvantaged students) [16]. Furthermore, the “black box” nature of complex models like deep learning obscures their decision-making logic, undermining educator trust and adoption. Future efforts must strengthen the application of Explainable AI (XAI) techniques in educational settings and institute algorithmic fairness evaluation and auditing mechanisms.
4.3. Exploring Pathways for Deep Integration of Technology and
Education
The most fundamental challenge lies in avoiding the pitfall of “technological solutionism”. AI systems should be positioned to empower educators and support student development, not to replace the essence of education itself [17]. The proposed framework is explicitly designed to ensure the educator remains central to the assessment process. AI-driven insights are intended to inform and augment pedagogical judgment by providing data-driven evidence, such as identifying at-risk students or suggesting potential intervention strategies, but the final decision-making regarding educational interventions must rest with the educator, who integrates this information with their professional experience and contextual understanding of the student. It is crucial to deeply integrate educational psychology theories—such as Self-Determination Theory and the Response to Intervention model—into system design. This ensures technological applications align with educational principles, preventing the assessment system from devolving into an alienating “dataism”. Furthermore, beyond these ethical and technical issues, practical implementation barriers pose significant challenges, including the need for substantial institutional investment in technical infrastructure (e.g., data servers, integrated software platforms) and comprehensive, ongoing training for faculty and staff to build data literacy and ensure effective system utilization.
5. Future Prospects: Building a New Paradigm of
“Data-Driven Governance”
Looking ahead, AI-enabled student assessment is poised to evolve towards a new paradigm of “data-driven governance”. This paradigm extends beyond simple data-driven decision-making by establishing a holistic ecosystem for continuous improvement. It involves the formulation of clear institutional policies (e.g., data ethics charters, standardized assessment protocols), defines the roles and responsibilities of all stakeholders (e.g., administrators setting strategic goals, IT staff maintaining system integrity, teachers interpreting and acting on insights, students engaging with their own data), and institutes formal feedback loops (e.g., regular review cycles where the outcomes of interventions are fed back into the system to refine algorithms and strategies). This paradigm emphasizes the synergistic co-evolution of technology, institutions, and culture. Future research should concentrate on the following priorities:
Technological Iteration: Future work should explore the analysis and application of multimodal data (e.g., voice, video), develop more accurate and robust predictive models, and enhance the interactivity and human-centric design of these systems.
Theoretical Construction: It is crucial to strengthen interdisciplinary collaboration to establish a theoretical framework that bridges computer science, education, psychology, and management. This framework would guide the scientific design and evaluation of AI-enabled assessment systems.
Policy Safeguards: There is a pressing need to promote the development of data security standards, algorithmic ethics guidelines, and assessment quality certification systems at national and industry levels. These measures are essential to provide the institutional safeguards necessary for large-scale implementation.
6. Conclusion
The application of AI in comprehensive quality assessment for vocational education represents a profound paradigm shift, whose ultimate objective transcends the generation of “digital scores” to focus on cultivating “well-rounded individuals”. This research has significant implications for both policymakers and educational practitioners. For policymakers, it underscores the need to fund infrastructure development, create supportive policy environments that encourage innovation while safeguarding ethics, and establish national standards for data and algorithms in education. For practitioners, it provides a conceptual framework and technical roadmap for designing and implementing intelligent assessment systems, emphasizing the importance of building data competency among staff and fostering a culture that values evidence-informed educational practice. Future research must adhere to the core principle of “people-centeredness”, seeking an optimal balance between technological empowerment and educational humanism. By developing scientific, equitable, and ethically-guided assessment systems, we can genuinely steer vocational education into a new era of high-quality development.
Funding
This work was supported by the Taizhou Technician College 2025 Annual Entrusted Research Project (grant number: 2025WTYJ23) to Qingqing Lu.