Archaic Methods in a Data Rich World: Why Educational Research Must Embrace AI Research Methods ()
1. Introduction
Educational research has a critical issue to consider: can yesterday’s methods still yield trustworthy evidence in a world flooded with complex, real-time data or is it time to embrace new ways of knowing that reflect the scale, speed, and messiness of contemporary learning? Traditional approaches, which are grounded in small samples, clear models, and controlled settings, have long been valued for their rigor and clarity (Byrne, 2016; Agresti, 2018). But do these methods still hold when educational life plays out across learning platforms, social media, and institutional systems? This brings us to a paradigm choice: continue investigating education through legacy methods or explore it through tools that can make sense of high-dimensional, digital traces. Emerging work in learning analytics and AI suggests the latter path is not only possible, but necessary (Siemens & Long, 2011; Zawacki-Richter et al., 2019). The aim of this commentary is to clarify how integrating causal and interpretable machine learning methods can revitalize educational research by bridging predictive and explanatory traditions and by reclaiming methodological authority for scholars within education rather than corporate technologists.
2. Literature Review
The review of the literature is organized to trace the methodological evolution in educational research, beginning with the foundational roles of statistical, qualitative, and mixed methods traditions. Then the review examines the growing integration of machine learning (ML) and natural language processing (NLP), highlighting their technical utility and epistemic significance. Finally, the review explores the broader implications of this shift, focusing on the emergence of hybrid expertise, the need for interpretability, and the ethical and political stakes of AI adoption in education.
An Epistemic Shift in Educational Research
Foundational statistical approaches such as regression and structural equation modeling have shaped educational research for decades. They anchor causal inference, ensure clarity in research design, and allow scholars to communicate evidence transparently (Agresti, 2018; Byrne, 2016). Qualitative traditions likewise provide indispensable contextual depth, cultural interpretation, and narrative insight (Creswell & Poth, 2018). Mixed methods research intentionally integrate the strengths of both traditions to produce findings that are simultaneously rigorous and contextually meaningful (Creswell & Clark, 2017). These approaches remain central to educational inquiry. However, the truth is their limitation lies in scope: they were not designed for the vast, multimodal, and dynamic datasets that define twenty-first century education. For this reason, empirical reviews have started documenting a sharp increase in studies applying Artificial Intelligence (AI) research methods, specifically ML, in higher education.
Zawacki-Richter et al. (2019) synthesized 146 studies and highlighted trends in classification, clustering, and NLP while noting limited theoretical integration. Similarly, systematic reviews of dropout prediction show that random forests, support vector machines, and boosting methods often outperform logistic regression when analyzing engagement and demographic data (e.g., Andrade-Girón et al., 2023; Hellas et al., 2018; Lottering et al., 2020). Furthermore, when the supervised models (models mapping inputs to known outputs) are applied to institutional data, they offer practical utility. For example, Xu et al. (2019) showed that decision tree ensembles can classify students’ likelihood of dropout in online environments with high accuracy, while Joksimović et al. (2018) used learning analytics to reveal writing processes in student essays at a scale impossible with manual coding. These findings suggest that ML methods are not only technically useful but also epistemically necessary when the research problem involves large numbers of variables and nonlinear interactions.
More critically, these studies illustrate that ML is not limited to prediction; it can provide new entry points for interpretive work by uncovering the underlying themes in the ways people communicate and behave. In fact, NLP expands the frontier of qualitative scaling. Topic modeling, originally formalized by Blei et al. (2003), has been widely adopted in education to analyze large discussion corpora. More recent applications demonstrate how topic modeling and transformer-based language models can enrich understanding of collaborative learning by uncovering latent discourse structures (Chiu & Fujita, 2014; Dowell & Kovanović, 2022). These tools do not replace qualitative/interpretive inquiry but rather broaden its scope. But who are the scholars driving this methodological shift? Are these studies primarily the work of computer scientists applying algorithms to education, or of educational researchers deeply grounded in theory? The answer is both, though the number of such scholars is limited, their convergence is quietly reshaping the field.
Many of the most influential contributions to learning analytics and educational data mining now emerge from interdisciplinary teams where ML is integrated with robust understanding of learning theory, pedagogy, and equity. Scholars like Ryan Baker, Dragan Gašević, and Alyssa Wise exemplify this fusion: their research not only advances technical methods but also interrogates what constitutes meaningful learning and fair assessment. However, scholars with such hybrid expertise are so rare that even educational research journals struggle to find qualified reviewers who can engage with both the technical and pedagogical dimensions of submitted work. That said, the hybridity reflects a broader epistemic shift where knowledge production demands more than computational skill; it requires theoretical grounding in how people learn, teach, and interact with complex systems. As a result, methodological innovation in education is not merely a technical enhancement but a redefinition of what counts as valid insight. Supporting this shift, Perrotta and Selwyn (2020) advocate for a relational perspective on AI in education, emphasizing that computational tools must be embedded in institutional contexts and educational values. Without this anchoring, research risks becoming either narrowly technical or detached from the pedagogical realities it seeks to transform.
Causality, Transparency, and the Urgency of Data Politics in Education
A persistent critique of ML is that it excels in prediction but falters in explanation (Breiman, 2003). Yet recent advances in causal ML provide tools that directly address this gap. Double or debiased ML enables estimation of treatment effects in high-dimensional settings (Chernozhukov et al., 2018). Double (debiased) machine learning is a statistical framework that isolates causal treatment effects while controlling for many confounders by combining regularized machine-learning estimators with econometric orthogonalization. Causal forests also extend ensemble methods to identify heterogeneous treatment effects, allowing researchers to ask not only whether an intervention works but also for whom and under what conditions (Wager & Athey, 2018). For instance, a university might deploy a machine-learning system to predict which students are likely to drop out based on clickstream and demographic data. Such a predictive model flags risk but does not reveal why students disengage. A causal analysis, by contrast, could estimate how much targeted tutoring or mentoring changes the probability of persistence, thereby informing policy decisions rather than merely ranking students by risk.
A causal forest is an adaptation of the random-forest algorithm that estimates heterogeneous treatment effects for individual observations, revealing how an intervention’s impact varies across subgroups. These methods align with pressing educational priorities such as personalized learning and equity-driven interventions (Knaus, 2022). Most importantly, education is a high-stakes context where transparency is essential; black box models raise risks of misinterpretation and a loss of accountability (Lipton, 2018). Explainable AI (XAI) methods such as SHAP (Lundberg & Lee, 2017) and LIME (Ribeiro et al., 2016) attempt to mitigate these risks by providing case-specific insights into how models arrive at predictions. SHAP (SHapley Additive exPlanations) is an interpretability technique that attributes each model prediction to specific input variables, allowing researchers to quantify how strongly each feature influences an outcome. For example, SHAP values can reveal whether prior GPA or peer engagement most strongly influences predicted success, and such findings can then be situated in established theories of academic integration (Gašević et al., 2015). Interpretability in this sense is both an epistemic safeguard and an ethical imperative.
However, the stakes extend far beyond methodology. With the rise of platform-driven education, student data is increasingly commodified for commercial ends, a process Couldry and Mejias (2019) term data colonialism. Williamson (2017) similarly warns of the corporatization of educational research as learning analytics infrastructures migrate into vendor-controlled ecosystems. These critiques are no longer speculative. Recent analyses show that educational technologies are already shaping how evidence is defined and used in practice, often privileging corporate priorities over pedagogy (Bond et al., 2024; Miao & Holmes, 2021). The acceleration of large language models further amplifies these risks, making the urgency of interpretability and accountability in education greater than ever (Gan et al., 2023). For this reason, critical scholarship emphasizes the need to reclaim interpretive sovereignty (e.g., Knox, 2020). Thus, educators must ask themselves: who benefits when classroom data is extracted, packaged, and sold? If scholars remain silent, they risk becoming passive agents in the commodification of learning they claim to critique. The question is not whether AI methods have a place in education, but whether researchers will have the courage to shape their development and application in ways that protect student dignity and advance meaningful learning over corporate interests.
3. Discussion
The integration of ML into educational research demands enthusiasm tempered with caution. Methodologically, ML expands the field’s capacity to analyze scale, capture heterogeneity, and produce interpretable causal estimates. Philosophically, it demands reflection on what counts as valid explanation and who holds authority over knowledge. The most promising path forward is methodological bilingualism, where researchers are trained in both conventional design logics and computational methods (Perrotta & Selwyn, 2020). That is, doctoral programs must evolve accordingly: they must abandon the comfort of outdated toolkits and confront the reality that without programming, algorithmic literacy, causal ML, and data ethics at their core, they risk graduating scholars fluent in yesterday’s methods but illiterate in tomorrow’s evidence.
Additionally, faculty hiring and tenure processes should recognize scholarship that bridges traditions rather than siloing computational work. Journals should encourage submissions that combine interpretive theory with ML analyses and reward methodological transparency. Institutions should also practice transparency by investing in open data infrastructures and resist the outsourcing of analytic capacity to corporations. Without such integration, the field risks bifurcation. Computational researchers may drift toward technical optimization detached from pedagogy, while traditional researchers may struggle to remain relevant in a world defined by data traces. The worst outcome is that educational research cedes authority entirely to platform vendors, allowing data colonialism to shape policy. The best outcome is that scholars reclaim interpretive sovereignty by embedding computational methods in educational values, ensuring that the digital transformation strengthens rather than weakens the field’s commitments to justice and equity.
Beyond methodological reform, the integration of AI in education raises critical concerns about data privacy, algorithmic bias, and the responsible governance of student information. Current international frameworks such as UNESCO’s (2021) Recommendation on the Ethics of Artificial Intelligence and the Organization for Economic Co-operation and Development’s (OECD) AI Principles stress transparency, human oversight, and fairness as non-negotiable standards. Educational researchers should therefore align analytic pipelines with these ethical benchmarks, ensuring informed consent, auditability of models, and safeguards against demographic bias in training data. While the present commentary advocates methodological reform in education, it should be noted that it offers a conceptual synthesis of theoretical and methodological perspectives rather than presenting new empirical evidence. This limitation suggests future research to empirically test the propositions advanced here, for example, by comparing conventional regression, causal-ML, and interpretability techniques on shared educational datasets. Such validation would clarify not only performance differences but also practical implications for policy and pedagogy.
4. Conclusion
Educational research stands at the cusp of a methodological revolution. Traditional quantitative, qualitative, and mixed methods remain indispensable, but they are increasingly insufficient on their own. ML and causal inference provide tools that can address complexity, scale, and heterogeneity in ways that extend, and not replace, established traditions. Interpretability and fairness must remain non-negotiable, and critical attention to the political economy of educational data is essential. The way forward requires deliberate synthesis, cultivating bilingual scholars who can move fluently between regression tables and neural networks, between ethnographic coding and topic modeling, between explanation and prediction. Only then can educational research remain both scientifically rigorous and socially just in a data rich century. The cost of inaction is irrelevance: if educational research clings to archaic methods, it risks being bypassed by corporate analytics and policy regimes that redefine knowledge without educators at the table.