TITLE:
Core Narrative Information Retention and Readability in AI-Generated Reading Materials: A Comparative Evaluation of Generative Systems
AUTHORS:
Zhihan Fu, Yaowei Xu
KEYWORDS:
Generative AI, Educational Technology, Reading Materials, Readability, Multi-Agent Systems, Core Narrative Information Retention
JOURNAL NAME:
Open Access Library Journal,
Vol.13 No.10,
October
29,
2026
ABSTRACT: Generative artificial intelligence (AI) systems are increasingly used to create and adapt educational reading materials, but the quality of these materials requires careful evaluation before they can be used in learning contexts. This study examines two questions: whether different generative AI systems can preserve the narrative information of source materials, and whether these systems produce different levels of reading difficulty in their generated texts. Three generative AI systems, GPT-5.6 Sol, Gemini 3.7 Flash, and Lingualite, were compared using three selected chapters from Journey to the West. Before generation, the source texts underwent limited preprocessing. Extended poetic passages and highly literary descriptions were excluded because the present study focused on AI-generated reading materials for general educational use rather than advanced literary analysis or translation studies. Core narrative events, character relationships, and causal connections were retained. Each system generated five versions of each chapter. Core Narrative Information Retention (CNIR) was defined as a chapter-specific, eight-item binary measure of whether each generated passage preserved eight predefined central events, character relationships, or narrative outcomes. Each content point received a score of 1 when its central content was clearly preserved and 0 when it was absent, materially altered, or assigned to the wrong character. The retention rate was calculated as the number of preserved points divided by eight. CNIR was used as a baseline measure of narrative content preservation; it was not intended to measure semantic equivalence, factual completeness, cultural fidelity, literary quality, or translation accuracy, while reading difficulty was examined using Lexile and Flesch-Kincaid Grade Level (FKGL), supported by linguistic indicators including sentence length and lexical diversity. The results showed that all generated texts retained the predefined narrative information. However, the three systems produced significantly different readability profiles. GPT-5.6 Sol generated texts with the lowest average Lexile and FKGL scores, while Gemini 3.7 Flash and Lingualite showed different relative patterns across the two measures. Repeated generation analysis further indicated no systematic shift across the five runs within the current dataset. These findings suggest that evaluating AI-generated educational materials requires separating Core Narrative Information Retention from readability rather than relying on a single quality indicator. The study provides a text-level evaluation approach for examining the suitability of AI-generated reading materials in educational contexts.