Oral squamous cell carcinoma (OSCC) is a malignant epithelial tumor that can develop from precancerous epithelial dysplasia and frequently remains asymptomatic in its early stages [1] . In general, early detection is critical for the effective management of malignant tumors. Consequently, oral exfoliative cytology, valued for its minimal invasiveness, rapid turnaround, and ease of application, is widely used as a screening tool for oral mucosal lesions, including OSCC and epithelial dysplasia [1] - [3] .

However, cytology alone does not provide a definitive diagnosis equivalent to that of tissue biopsy [4] . Previous studies investigating the diagnostic accuracy of oral cytology have reported an overall accuracy of approximately 70% - 80% [5] [6] . Therefore, quality control in cytological diagnosis is essential, and efforts such as evaluating diagnostic accuracy rates and analyzing discrepancies between cytological and histopathological findings have been undertaken [6] [7] . Some authors have argued that cytology is insufficient as a substitute for biopsy [8] . Thus, obtaining further evidence regarding oral cytology is indispensable.

The cytological evaluation of oral exfoliative samples is based on a variety of cellular findings, among which nuclear characteristics are considered particularly important for improving diagnostic accuracy [9] . Therefore, to improve diagnostic accuracy, it is necessary to clarify the nuclear characteristics according to cytological diagnostic categories; however, current knowledge regarding cytological evaluation of the oral mucosa remains limited.

In this study, a design was adopted to reflect the overall cytomorphological tendencies of the specimens as encountered in routine cytological screening, rather than to highlight only the features of diagnostically evident or atypical cells commonly analyzed in previous studies. The aim of this study was to clarify the nuclear features across different cytological diagnostic categories based on a whole-sample random sampling approach.

The results by cytological diagnosis category are summarized in Table 1 . The number and selection rate of selected cells relative to the total number of target cells was as follows: NILM [inf−], 1766/1939 (91.1%); NILM [inf+], 1967/2148 (91.6%); OLSIL, 1272/1467 (86.7%); OHSIL, 1529/1731 (88.3%); and OSCC, 1984/2308 (86.0%).

The median nuclear area [interquartile range] for each group was as follows: NILM [inf−], 51.0 [28.3] μm²; NILM [inf+], 52.6 [26.3] μm²; OLSIL, 47.6 [27.1] μm²; OHSIL, 60.7 [35.0] μm²; OSCC, 67.2 [54.0] μm².

The median hyperchromasia [interquartile range] for each group was as follows: NILM [inf−], 98.3 [39.1], NILM [inf+], 84.7 [40.9], OLSIL, 74.1 [42.1], OHSIL, 85.5 [46.1], and OSCC, 72.5 [39.0].

The results of the kernel density estimation are shown in Figure 2 . Compared with NILM [inf−], all other categories (NILM [inf+], OLSIL, OHSIL, OSCC) exhibited distinct distribution patterns, with OSCC showing the most pronounced difference.

The comparative distribution patterns among the diagnostic groups relative to NILM [inf−] are shown in Figure 2 . The contour plots generated by kernel density estimation show a progressive shift and broadening of the nuclear feature distributions from NILM [inf+] through OLSIL and OHSIL to OSCC. Notably, the OSCC group displayed a markedly wider dispersion along both the nuclear area and hyperchromasia axes, reflecting the increased heterogeneity associated with malignant transformation.

Spearman’s correlation coefficients between nuclear area and hyperchromasia were as follows: NILM [inf−], 0.546; NILM [inf+], 0.561; OLSIL, 0.535; OHSIL, 0.437; and OSCC, 0.293.

Shapiro–Wilk normality tests for each cytological group indicated that both nuclear area and hyperchromasia deviated from normal distributions (nuclear area: W = 0.769 - 0.960, p < 0.001; hyperchromasia: W = 0.971 - 0.997, p < 0.001). The Fligner-Killeen test for homogeneity of variance among the five groups indicated that homogeneity of variance was not present for either nuclear area (χ² = 763.92, p < 0.001) or hyperchromasia (χ² = 53.49, p < 0.001).

For nuclear area, significant differences were observed between all groups except between NILM [inf−] and OLSIL ( Table 2 ). For hyperchromasia, most pairwise comparisons between groups showed significant differences, with the exceptions of NILM [inf+] vs. OHSIL and OLSIL vs. OSCC, for which no significant differences were observed ( Table 2 ).

Abbreviations: NILM, negative for intraepithelial lesion or malignancy; inf, inflammation; OLSIL, oral low-grade squamous intraepithelial lesion or oral low-grade dysplasia; OHSIL, oral high-grade squamous intraepithelial lesion or oral high-grade dysplasia; OSCC, oral squamous cell carcinoma; IQR, interquartile range.

Abbreviations: NILM, negative for intraepithelial lesion or malignancy; inf, inflammation; OLSIL, oral low-grade squamous intraepithelial lesion or oral low-grade dysplasia; OHSIL, oral high-grade squamous intraepithelial lesion or oral high-grade dysplasia; OSCC, oral squamous cell carcinoma; CI, confidence interval; Cliff’s δ, effect size.

Nuclear findings in squamous epithelial cells observed on oral cytology can be detected in cells that have not undergone nuclear loss due to processes such as keratinization [10] . According to Zafer et al., nuclear features serve as important diagnostic indicators that reflect pathological conditions such as inflammation, epithelial dysplasia, and squamous cell carcinoma [9] . In practice, microscopic cytological examination demonstrates not only normal and atypical cells, but also diagnostically ambiguous and degenerative cells. In fact, the smears prepared on glass slides often contain a diverse population of cells, including reactive and regenerative cells, as well as cells exhibiting varying degrees of atypia from mild to severe. Therefore, in this study, to capture the overall picture of such diverse cytological presentations reflective of screening practice, nuclear characteristics were analyzed using a randomly sampled cytological image dataset.

In this study, nuclear area and hyperchromasia were quantified through image analysis, and their correlation coefficients were evaluated. In the NILM [inf−], NILM [inf+], and OLSIL groups, the correlation coefficients exceeded 0.5, indicating a moderately strong positive correlation. Assuming a relatively constant chromatin content within the nucleus, larger nuclei may exhibit lower chromatin density per unit area, resulting in a lighter nuclear appearance in the images. Previous studies have suggested that chromatin condensation plays a role in controlling nuclear size, although the underlying mechanisms remain unclear and warrant further investigation [11] . In contrast, OHSIL exhibited a lower correlation coefficient (below 0.5), and OSCC demonstrated an even lower value. Nuclear pleomorphism has been reported in oral cancers [12] [13] . Therefore, the findings in OSCC and OHSIL may reflect heterogeneity resulting from tumor-associated nuclear pleomorphism.

Two-dimensional kernel density estimation was used to visualize the distribution pattern of nuclear area and hyperchromasia. Kernel density estimation has been applied to various biomedical analyses, including the spatial visualization of T-cell distributions around cancer cells [14] , and the identification of cell clusters in multiparameter flow cytometry [15] and hotspot mapping of brain metastases in breast cancer [16] . Given its ability to provide a smooth and continuous representation of density differences in multidimensional data, KDE is regarded as a particularly effective visualization method for comparative analyses across diagnostic categories and may also align with the broader trend of digital pathology and AI-based cytological analysis that increasingly leverages quantitative morphometric data [17] [18] .

Previous studies have reported that nuclei in inflammatory specimens generally tend to exhibit hyperchromasia [19] , and this overall tendency was also observed in our analysis. In inflammatory conditions, reparative cells with enlarged nucleus with pale staining chromatin are often observed [20] ; however, reactive atypical cells associated with inflammation can also appear, and these cells tend to show hyperchromatic nuclei [21] . Therefore, the presence of such reactive atypical cells may have contributed to the difference in hyperchromasia between the NILM [inf−] and NILM [inf+] groups.

In the OLSIL group, compared with the NILM [inf−] group, hyperchromasia tended to be higher, whereas nuclear area did not show a significant difference. The findings regarding nuclear area were not consistent with those reported in previous studies [19] . This may reflect the present diagnostic approach, which places emphasis on hyperchromasia in the cytological assessment of OLSIL, and it may also be attributable in part to the fact that OSIL was subdivided into OLSIL and OHSIL in the present study. In contrast, the OHSIL group exhibited a significantly larger nuclear area than both NILM [inf−] and OLSIL groups. Furthermore, the OHSIL group demonstrated a broader distribution of nuclear features, which can be interpreted as a reflection of dysplasia-associated cellular pleomorphism, as discussed above. Pairwise statistical comparisons ( Table 2 ) further supported these findings, showing that nuclear area significantly differentiated OHSIL from both NILM [inf−] and OLSIL, whereas hyperchromasia provided significant separation between NILM and OLSIL but did not distinguish OLSIL from OSCC. These results suggest that nuclear enlargement is a more consistent marker of progression, whereas hyperchromasia alone may be insufficient to differentiate dysplasia from carcinoma. The findings suggest that nuclear area may be an important factor that should be given greater weight in cytological judgment.

Previous studies utilizing morphometric computer-assisted image analysis have quantitatively confirmed nuclear enlargement in tumor cells of OSCC, thereby supporting the histopathological recognition of increased nuclear size as a key diagnostic indicator [22] [23] . Consistent with these findings, the present study demonstrated that the OSCC group exhibited not only the most extensive variation in nuclear parameters among all groups but also clear evidence of nuclear enlargement and hyperchromasia. These results are in line with those of Seto et al., who reported a significant increase in both nuclear size and chromatin density in OSCC using quantitative cytomorphometric analysis [19] . However, previous studies have not incorporated visual assessments of the distribution of nuclear size or hyperchromasia, such as those provided by kernel density estimation, which may offer a more comprehensive understanding of cellular heterogeneity. The present findings obtained through kernel density-based comparison further underscore the significance of both nuclear enlargement and hyperchromasia as critical cytological features for diagnosing OSCC. Moreover, the pronounced heterogeneity in these nuclear characteristics, as reflected in the broad distribution of measured values, is considered to reflect the morphological diversity typical of malignancy and may serve as an adjunctive diagnostic feature.

The present study has several limitations. The ROI selection was performed by a single observer, which may have introduced observer bias. In addition, multiple cells were analyzed per patient, and because the number of patients was limited, the statistical independence of observations could not be fully ensured, raising the possibility of pseudo-replication. The study focused exclusively on tongue specimens, as the tongue is the most common site of squamous cell carcinoma; however, this design limits the generalizability of the findings to other oral sites. Because hyperchromasia was analyzed, variability in slide preparation quality could potentially influence the measured results, although all specimens were stained under standardized conditions and overall quality was well maintained. The exclusion of images that did not meet predefined quality criteria may have introduced selection bias, possibly affecting the representativeness of the analyzed dataset. Furthermore, cytoplasmic features were not analyzed, even though cytoplasmic characteristics may influence the apparent nuclear color tone or staining intensity, which could have affected the interpretation of nuclear hyperchromasia. In addition, future studies will need to consider the quantification of the nuclear-to-cytoplasmic ratio, a classic cytological marker of malignancy. Finally, all analyses were conducted using specimens from a single dataset without an external validation cohort, and thus the generalizability of the findings to other populations or institutions remains uncertain. Future studies incorporating independent validation datasets, stratified sampling, and cytoplasmic feature quantification are warranted to enhance the robustness and clinical applicability of nuclear morphometric analysis.

In addition, the present findings should be considered as foundational data that could be embedded into practical diagnostic workflows. For example, nuclear area and hyperchromasia metrics may serve as objective adjuncts to conventional cytological interpretation, supporting cytotechnologists and pathologists in reducing subjectivity when assessing borderline or ambiguous cases. Furthermore, these quantitative features, along with various distributional characteristics, may be integrated into machine learning pipelines, where nuclear morphometry findings could potentially function as explanatory variables for training AI models aimed at the pre-screening of oral cytology specimens. Such integration would not only enhance diagnostic reproducibility, but also contribute to the development of AI-assisted systems that align with real-world screening practice. Indeed, surveys of current practice confirm that digital cytology and AI-based workflows are rapidly expanding, though challenges in standardization, interoperability, and whole-slide imaging quality remain important considerations for implementation in routine diagnostics [24] [25] .

Significant differences were observed in the distribution patterns of nuclear area and hyperchromasia in squamous epithelial cells obtained from oral cytology specimens. These findings indicate that both nuclear area and hyperchromasia are key parameters in cytological evaluation, and that their distributional characteristics may further improve diagnostic accuracy. These findings suggest that evaluating nuclear area and hyperchromasia in oral exfoliative cytology could improve the early detection and risk stratification of OSCC.

This work was supported by the Japan Society for the Promotion of Science (JSPS) KAKENHI Grant Number JP21K11907.