Stufflebeam’s CIPP model divides educational evaluation into four interlocking dimensions—Context, Input, Process and Product—and emphasises whole-process, developmental and improvement-oriented evaluation rather than a single terminal judgement [5]. By breaking the limitation of summative evaluation, the model matches the diversified evaluation demands of College English ideological and political education, where the value growth process matters as much as its outcome, and supplies the structural skeleton for the multi-dimensional indicator system developed in Section 3.

Learning analytics relies on digital platforms to capture multi-dimensional learning behaviour and, through semantic analysis and emotion recognition, to mine the attitudes and value tendencies embedded in behavioural traces. Big-data portrait techniques have already been applied to precise curriculum ideological and political education, building label systems from behavioural and online-learning data so that teachers can track dynamic changes in students’ thinking [6]. In this study, the same logic is used to move from subjective qualitative judgement to a data-informed, mixed quantitative and qualitative evaluation of otherwise invisible ideological and political literacy.

Digital-intelligent integration reshapes not only resources but also the teaching design of College English ideological and political courses, requiring value guidance to be embedded naturally in language tasks rather than added on mechanically [7]. Within the smart classroom, technology mediates interaction between language input, cultural experience and value guidance, making continuous process evaluation feasible at scale [8]. These two strands justify choosing a platform-and-data-driven design that treats evaluation as part of teaching rather than as a separate end-of-course act.

The study adopted an instrumental case-study design. The case was the College English course taught to first-year non-English majors at Qilu Normal University in the first semester of 2025-2026. Participants were 216 students in four intact classes taught by four College English instructors, taught over sixteen weeks. Data sources included the university intelligent learning platform logs, students’ online discussion posts, cross-cultural reflection journals, recorded oral presentations and group debates, and final cross-cultural reports and English presentations on Chinese topics. A five-point Likert scale of cultural identity and cross-cultural confidence was administered before and after the semester. Algorithmic scores generated by the platform were compared with independent teacher ratings to estimate validity, and the pre-post scale scores were compared with paired-samples t-tests to examine value-added effect.

Taking the university’s existing intelligent learning platform as the carrier, a dedicated evaluation module for College English ideological and political education was developed around three data-collection functions. First, text semantic analysis automatically identifies the value tendency in students’ cross-cultural essays, reading-reflection logs and online discussion posts, extracts keywords related to cultural confidence and national identity, and returns a quantitative ideological-literacy score. Second, audio and video behaviour tracking collects oral presentations, group debates and classroom speech, and analyses emotional polarity and cross-cultural cognitive attitude through speech-recognition algorithms. Third, a whole-cycle learning portrait integrates online study duration, resource-browsing preferences, ideological task completion and classroom interaction frequency into a personalised growth profile.

Combined with the characteristics of College English ideological and political teaching, a four-level indicator system covering Context, Input, Process and Product was built. The weights—Context 15%, Input 20%, Process 40% and Product 25%—were not assigned intuitively. Following the Delphi procedure and the Analytic Hierarchy Process (AHP) validated in CIPP-based curriculum ideological and political evaluation, thirteen experts (foreign-language teaching researchers, curriculum-ideology specialists and educational measurement specialists) completed two rounds of consultation; pairwise comparison matrices were constructed using Saaty’s nine-point scale and all consistency ratios were below 0.10 [9]. The heavy weight on Process (40%) reflects the developmental nature of value formation, which unfolds over time rather than being captured by a single terminal task, and is consistent with expert consensus that process implementation is the core lever of curriculum ideological and political quality.

The observable variables behind each dimension are specified as follows. Context evaluation (15%) draws on teachers’ ideological-political teaching designs and the matching degree of bilingual ideological resources with teaching objectives. Input evaluation (20%) measures students’ pre-course ideological-cognition baseline, willingness to engage in digital learning, and cross-cultural preparation, captured by the entrance survey and pre-course tasks. Process evaluation (40%) is the core link and combines classroom interaction, ideological-reflection task completion, online cross-cultural exchange behaviour and collaborative group performance, scored jointly by the platform, peers and teachers. Product evaluation (25%) covers the final cross-cultural report, the English presentation on a Chinese topic, and an integrated score of language ability and ideological literacy.

To balance objectivity and subjectivity, the single-teacher model was replaced by a four-subject mechanism, in line with evidence that digitally mediated combinations of automated, teacher, peer and self-assessment jointly promote learning [10]. The intelligent algorithm provides a preliminary evaluation (40%) by automatically scoring objective data on learning behaviour, text semantics and audio emotion. The teacher provides a professional evaluation (30%), reviewing classroom performance, individual ideological growth and practical performance, and correcting machine bias. The student completes a digital self-evaluation (15%), summarising changes in cross-cultural cognition and ideological gains. Peer mutual evaluation (15%) scores group members’ collaborative tasks, cross-cultural communication attitude and cultural expression under a unified rubric. The four scores are aggregated into a composite ideological-literacy grade.

Because algorithmic scoring is novel and carries bias risk, its validity was checked before full rollout. Two trained instructors independently rated a stratified sample of 120 cross-cultural essays and reflection journals on the same rubric used by the platform. Pearson correlation between algorithmic and mean teacher scores reached r = 0.83, and Cronbach’s alpha across the two human raters was 0.87, indicating acceptable agreement. A two-stage correction mechanism then operated throughout the semester: the platform flagged any script whose algorithmic score deviated by more than one standard deviation from the peer/self-evaluation mean for that task for mandatory teacher review, and teachers annotated every score they overrode so that the scoring model could be recalibrated. To control bias, scoring was blind to student identity, and length, genre and dialect distributions were monitored each task; the teacher’s 30% weight acts as the final human-in-the-loop safeguard.

The system ran as a closed, repeatable sequence across the sixteen-week semester. At Week 1, the entrance survey established the Input baseline and introduced the evaluation rubric to students. From Week 2 to Week 14, continuous data collection fed the platform, which produced weekly preliminary algorithm scores. Each fortnight, teachers reviewed flagged scripts, released individual and class feedback through the platform, and adjusted subsequent ideological-political tasks. At Week 15, Product evaluation was completed through final reports and presentations. At Week 16, the exit survey measured value-added growth, and the semester evaluation report was generated. The sequence—data collection, platform scoring, teacher review, feedback release and teaching adjustment—thus formed an “evaluation-analysis-feedback-teaching reform” loop rather than a one-off terminal test.

For transparency, each CIPP dimension was tied to concrete observable variables. Context was measured by the completeness of ideological-political elements in the syllabus and the expert-rated matching of bilingual resources to objectives. Input was measured by the entrance cultural-identity scale, pre-course video completion and readiness quiz scores. Process was the richest dimension: it used platform log variables (study duration, resource access, discussion-post frequency and quality keywords), the length and value-tendency score of weekly reflection journals, audio emotion polarity from oral tasks, and rubric scores for group collaboration. Product was measured by the final cross-cultural report, the English presentation on a Chinese topic, and an integrated language-and-value score. This specification prevents the evaluation from resting on vague judgement and makes every score traceable to a recorded behaviour or text.

Three kinds of evidence emerged. First, algorithm-teacher agreement was stable at r = 0.83 across tasks, giving confidence in the automated component. Second, the paired pre-post comparison on the cultural-identity and cross-cultural confidence scale showed a significant improvement: the mean rose from 3.42 (SD = 0.61) to 4.01 (SD = 0.58), t (215) = 11.74, p < 0.01, with a medium-to-large effect. This value-added pattern is consistent with empirical work showing that process-oriented value-added evaluation of College English ideological and political education enhances students’ learning self-efficacy, professional identity and patriotic attitude [11]. Third, teacher feedback indicated that continuous portraits reduced the invisible workload of tracking 216 students, while the four-subject mechanism made students more aware of their own value growth. These preliminary findings support the feasibility of the system at a local normal university of comparable scale.

Digital intelligence remains an auxiliary evaluation tool and cannot replace teachers’ humanistic care and value guidance. Semantic and emotion analysis reliably captures surface lexical and sentiment signals, but it may not fully capture reflective depth, tacit ideological growth, or context-specific meaning that depends on cultural nuance and teacher-student dialogue [12]. Scores are therefore treated as evidence to inform, not replace, teacher judgement; the evaluation process is designed around communication and guidance rather than simple data scoring, so that excessive quantification does not erode the humanistic nature of ideological and political education. A single case at one local normal university also limits generalisability; multi-site replication and longer longitudinal tracking are needed before the weights and agreement levels can be treated as stable norms.

Because the system stores sensitive ideological-learning data, special management measures were formulated to clarify access authority, prohibit unauthorised disclosure of students’ ideological evaluation records, and adopt encrypted storage and desensitisation. The Foreign Languages Department, Academic Affairs Office and Information Technology Center operate jointly: the Academic Affairs Office standardises the evaluation system, the Information Technology Center maintains technology and data security, and the Foreign Languages Department designs and iterates the indicators. Teachers receive regular training in curriculum-ideology theory, platform operation and educational big-data analysis, and the system is reviewed on an academic-year cycle, with indicator weights and platform modules adjusted on the basis of teacher and student feedback.