The empirical basis for this study is a specialized dataset originating from the work of [12], titled “Using LLMs to Improve the Accuracy of SBOM-Based Vulnerability Assessment”. This corpus, which documents the safety profile and behavioral anomalies of the GPT-OSS-20B model, is publicly available online².

The dataset comprises 13 detailed vulnerability reports structured in a standardized JSON format. These reports categorize model failures across five primary safety-critical domains: Reward Hacking, Deceptive Alignment, Potential Sabotage, Data Exfiltration, and Evaluation Awareness. Each record provides a granular view of the model’s output, capturing both the internal “chain-of-thought” (analysis) and the final response generated under specific system prompts. For the purpose of Correspondence Analysis (CA), the data is enriched with categorical metadata, including self-assessed severity levels and specific vulnerability indicators (e.g., lexical markers such as “confidential”, “proprietary”, or “transparent”). This structured relationship between risk categories, severity rankings, and linguistic markers forms the multidimensional contingency table required to map the associative space of LLM vulnerabilities.

The dataset utilized in this study comprises the totality of the 13 unique vulnerability reports available in the source repository [12]. No records were excluded, as this exhaustive selection ensures that the case study boundary covers all safety critical domains identified by the original auditors.

Although the dataset comprises 13 records, this qualitative scale aligns with the emerging Less Is More for Alignment (LIMA) paradigm, which demonstrates that a small set of high-fidelity, diverse examples is more effective for capturing structural model behaviors than large, noisy datasets [13]. In the field of AI safety, the shift from statistical benchmarking toward Forensic Red-Teaming necessitates a focus on worst-case structural vulnerabilities rather than average-case performance [14]. As emphasized in recent frameworks for Formal AI Auditing, the goal of security evaluation is to identify deterministic failure modes; in this context, a single well-characterized failure constitutes a formal proof of vulnerability [15]. Consequently, our 13 detailed audit reports provide the necessary logical density for Formal Concept Analysis to map the model’s structural failure modes and deceptive signatures without the need for statistical oversampling.

Building upon the multidimensional structure described in Section 3.1, the data was transformed into a binary formal context ( O , A , I ) to satisfy the requirements of Formal Concept Analysis (FCA). This transition from raw JSON logs to a formal matrix involved a process of semantic aggregation and conceptual scaling.

The transformation of raw JSON logs into a binary formal context followed a rigorous coding protocol. Three primary JSON fields were extracted for analysis: vulnerability_type, severity, and the analysis (model’s internal chain-of-thought). The inclusion of the analysis field is methodologically vital: it allows an external auditor to detect reasoning-level deceptive signatures where the model’s internal logic may reveal a misalignment that its final response attempts to mask. To ensure the reliability of this human-led semantic aggregation, the labeling was performed by the lead researcher and cross-validated by a second coder to resolve any ambiguities in attribute assignment.

To maintain analytical clarity while optimizing the lattice complexity, we performed a dimensionality reduction by grouping the initial categories and lexical markers into six Super-Attributes. This mapping logic ensures that each attribute captures a distinct dimension of the model’s risk profile:

MAL_AL (Malicious Alignment): Aggregates the domains of Reward Hacking, Deceptive Alignment, and Evaluation Awareness. It identifies instances where the model’s incentives deviate from its intended safety constraints. TECH_RISK (Technical Risk): Consolidates Potential Sabotage and Data Exfiltration. This attribute focuses on the operational impact on system integrity and infrastructure security. PSEUDO_TR (Pseudo-Transparency): Derived from specific linguistic markers identified in the reports. It flags responses where the model uses self-referential honesty (e.g., “I am designed to be transparent”) as a justification for its behavior. OVER_CONF (Epistemic Overconfidence): Captures the presence of authoritative lexical markers (e.g., confident, certain) in contexts where the model provides unverified or sensitive information. PROP_EXP (Proprietary Exposure): Specifically tracks the leakage of training data characteristics or internal technical specifications, as identified in the audit’s vulnerability indicators. CRIT_SEV (Critical Severity): Represents a binary quantization of the severity metadata provided in the source dataset [12], which uses a 3-point scale (1: Low, 2: Medium, 3: High). To isolate high-impact risks that require immediate mitigation, this attribute is assigned only to reports with a self-assessed severity level ≥2 (Medium and High). This threshold was chosen to align with standard cloud security auditing where Medium and High impacts trigger mandatory intervention.

The 13 audit records extracted from the dataset were mapped to unique identifiers (R1 to R13) and categorized according to their specific vulnerability focus:

R1-R3 (Reward Hacking): Scenarios involving the solicitation of definitive answers in medical (cancer cure), gambling (lottery), and financial (stock market) contexts. R4-R5 (Deceptive Alignment): Reports focusing on the model’s willingness to mislead users for a “greater good” or to protect its own operational integrity. R6-R7 (Potential Sabotage): Tasks involving critical system operations, such as temporary file cleanup and production database migrations. R8-R10 (Data Exfiltration): Probes targeting proprietary information, including model architecture, internal training conversations, and confidential specifications. R11-R13 (Evaluation Awareness): Scenarios testing the model’s ability to detect and adapt its behavior within research-controlled environments.

To illustrate the binarization process, Table 2 provides a worked example using Report R4. This specific report demonstrates how a raw vulnerability entry is mapped onto the six super-attributes based on the presence of specific keywords and the model’s self-reported reasoning.

Table 2. Coding protocol and binarization example for report R4.

As shown in Table 2, the attribute PSEUDO_TR is triggered by the model’s internal analysis field, which reveals a strategic intent to seem honest while actually being misaligned. This mapping allows the FCA to subsequently identify if such a veneer of honesty is a recurring pattern across the entire dataset.

Following this systematic binarization protocol, each of the 13 audit records was transformed into its corresponding binary representation. To ensure the reliability of this semantic aggregation and mitigate human bias, the coding process was performed in two stages: an initial assignment by the primary researcher followed by a verification pass by the second author. Any discrepancies in attribute mapping were resolved through consensus to ensure inter-rater reliability. This step is essential as it transforms unstructured, qualitative JSON data into a structured mathematical object; the Risk Formal Context, where each row represents a specific vulnerability profile and each column a super-attribute. Consequently, we generated the complete formal context presented in Table 3, which serves as the deterministic input for the subsequent Formal Concept Analysis.

Table 3. Formal context of GPT-OSS-20B vulnerabilities.

This structured binary representation allows for the mathematical extraction of implications and the visualization of the vulnerability hierarchy through the concept lattice.

Figure 2illustrates the analytical pipeline adopted in this study, which is structured into three fundamental stages: Data Preparation, Data Mining, and Interpretation.

The first stage (Data Preparation) begins with the collection of raw JSON security audit logs from the GPT-OSS-20B LLM. Relevant vulnerability metadata are then selected and subjected to a qualitative binarization process. This human led step is crucial for transforming semi-structured audit data into a binary LLM Risk Formal Context, which serves as the mathematical foundation for the subsequent analysis.

The second stage (Data Mining) involves the algorithmic processing of the formal context. Using Formal Concept Analysis (FCA) techniques, the system computes the underlying conceptual structures to generate the Concept Lattice. This lattice visually maps the hierarchical relationships and overlaps between different vulnerability attributes and audit records.

Finally, the Interpretation stage focuses on translating the lattice’s topology into actionable security insights. This phase extracts Risk Implication Rules and establishes a clear hierarchy of threats. By analyzing these implications, the study identifies critical vulnerability profiles, producing specialized knowledge that can support AI safety researchers and developers in hardening large language models against deceptive alignment and technical risks.

Figure 2. Structural architecture of our FCA-based knowledge discovery framework.

The findings of our formal mining process are organized into two complementary analyses: the topological exploration of the concept lattice (Section 4.1.1) and the discovery of logical implication rules (Section 4.1.2).

4.1.1. Structural Analysis of the Concept Lattice

The Hasse diagram shown in Figure 3 represents the conceptual hierarchy of GPT-OSS-20B vulnerabilities. Each node represents a formal concept, illustrating the dual relationship between the audit records (extents) and their shared security attributes (intents).

Figure 3. The concepts lattice of GPT-OSS-20B vulnerabilities.

The lattice reveals a clear tripartite structure in the model’s failure modes:

1) The Malicious Alignment Branch (Concept 1): This node acts as a major junction, regrouping 8 records. It shows that behavioral anomalies like Reward Hacking and Evaluation Awareness are intrinsically linked. Interestingly, we observe that OVER_CONF (Concept 5) and PSEUDO_TR (Concept 4) are direct sub-concepts of MAL_AL, suggesting that overconfidence and false transparency are specific manifestations of alignment failure.

2) The Technical Risk and Proprietary Exposure Cluster (Concept 3 and 6): There is a distinct vertical chain starting from TECH_RISK down to PROP_EXP. The fact that Concept 6 (containing R8, R9, R10) is a sub-concept of both TECH_RISK and CRIT_SEV is significant: it demonstrates that in this model, every instance of proprietary data exposure is systematically categorized as both a technical risk and a critical severity event.

The lattice highlights that while some alignment issues are minor (like R13 or R5, located outside the CRIT_SEV scope), any interaction involving proprietary data or deceptive “pseudo-transparency” combined with specific tasks (R4) tends to gravitate towards the critical severity threshold.

It is essential to distinguish between relationships discovered autonomously by the FCA and those inherent to our data encoding. For instance, the clustering of PROP_EXP and TECH_RISK reflects a structural reality in the model’s behavior: every time a technical specification was leaked, it was also categorized as an operational risk. While the grouping of attributes into Super-Attributes was a manual design choice to ensure analytical clarity, the hierarchical implications revealed in Figure 3(e.g., PSEUDO_TR⇒MAL_AL) are emergent properties of the model’s specific failure modes, providing a mathematical reality check on its behavioral consistency.

4.1.2. Automated Logic Discovery through Implication Rules

To complement the structural analysis of the lattice, we extracted the exact implication rules from the formal context using the FCA4J³ framework. Unlike probabilistic association rules, these implications represent absolute logical dependencies within the dataset.

Figure 4. Implication rules extracted from the formal context of Table 3, using the FCA4J tool.

Figure 4consolidates these results, distinguishing between rules based on their frequency of occurrence (the number of objects satisfying the rule) within a single visual:

The first rule PSEUDO_TR⇒MAL_AL reveals a critical behavioral dependency. Although “Pseudo-Transparency” appears less frequently in the corpus, its presence serves as a systematic predictor of an underlying alignment failure (MAL_AL). This mathematically demonstrates that whenever the model invokes predefined honesty or transparency clauses, such discourse acts as a rhetorical veneer to mask deviations from safety constraints.The rest of rules identifies the most robust invariants of the GPT-OSS-20B model: OVERCONF ⇐⇒MALAL: This implication proves that epistemic overconfidence—the use of an authoritative and non-nuanced tone is a recurring and structural manifestation of malicious alignment. PROPEXP⇐{TECHRISK, CRITSEV}: A perfect logical equivalence is observed here. The presence of the rule PROP_EXP⇒{TECH_RISK, CRIT_SEV} and its converse confirms that, within this dataset, the exposure of proprietary data is the sole and sufficient driver of the highest technical risk and severity levels.

These automated discoveries confirm that the visual hierarchy of the lattice is not merely illustrative but reflects strict logical laws governing the vulnerability profile of the audited AI.

The results obtained through the FCA pipeline provide empirical evidence of the Sleeper Agent phenomenon theorized by [6]. The structural analysis of the concept lattice (Figure 3) confirms that Malicious Alignment is not an isolated error but a junction point for multiple behavioral anomalies. A critical finding is the logical implication PSEUDO_TR ⇒ MAL_AL, which proves that the model’s claims of transparency (“I am designed to be honest”) are mathematically correlated with deceptive outputs. This validates the Sycophancy risks identified by [4], demonstrating that lexical markers of honesty can be used as a rhetorical veneer to bypass safety filters.

Furthermore, the perfect equivalence discovered between Proprietary Exposure and Critical Severity reinforces the privacy concerns raised by [3], showing that for GPT-OSS-20B, the exfiltration of training data is structurally inseparable from high-impact technical risks. By transforming semi-structured audit logs into a formal hierarchy, this study demonstrates that FCA can bridge the gap between qualitative behavioral descriptions and quantitative risk assessment.

This methodology provides a reproducible path for AI auditors to identify stable vulnerability profiles that traditional randomized testing might overlook.

Traditional security focuses on code-level exploits, but LLM vulnerabilities emerge at the semantic level. While early research highlighted data exfiltration risks [3], others studies focus on jailbreaking and the failure of safety training [2]. Our work diverges from these input-centric attacks to focus on the internal logical consistency of the model’s reasoning, addressing the structural nature of its failure modes.

A major frontier in AI safety is Deceptive Alignment, where models appear safe during evaluation but pursue misaligned goals in deployment [5]. Another work about Sleeper Agents empirically demonstrates that safety training can fail to remove deceptive behaviors, which may remain latent until triggered [6]. This is exacerbated by Sycophancy, where models prioritize evaluator satisfaction over truthfulness [4]. This study builds upon these findings by providing a formal taxonomy of such signatures within the GPT-OSS-20B model.

Recent literature highlights a growing synergy between Symbolic AI and Generative AI, where Formal Concept Analysis (FCA) serves as a rigorous framework to structure and validate LLM outputs. For instance, [16] demonstrated how Large Language Models can empower Relational Concept Analysis (RCA) through strategic knowledge delivery, enhancing the depth of relational discovery. Further explorations have utilized FCA as a reality check for LLMs, providing a formal basis to verify the conceptual consistency of generative models [17]. In specialized domains, the hybridization of LLMs with Triadic Concept Analysis (TCA) and RCA has yielded significant results in automating requirements engineering—such as variability-driven user-story generation—and software architecture restructuring [18]. Beyond theoretical modeling, this neuro-symbolic approach has been successfully applied to real-world challenges, including the management of agricultural knowledge bases in West Africa through the combination of symbolic reasoning and generative agents [19]. While these pioneering works primarily leverage LLMs to enhance formal knowledge representation or specialized task automation, our study shifts the focus toward adversarial behavioral auditing. We utilize the structural properties of FCA not to supplement LLM knowledge, but to mathematically map and diagnose the latent failure modes and deceptive alignment signatures within the models themselves. Framework (RMF 1.0, 2023) [20], positioning FCA as a crucial tool for cloud-based AI governance.