<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.4 20241031//EN" "JATS-journalpublishing1-4.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article" dtd-version="1.4" xml:lang="en">
  <front>
    <journal-meta>
      <journal-id journal-id-type="publisher-id">wjet</journal-id>
      <journal-title-group>
        <journal-title>World Journal of Engineering and Technology</journal-title>
      </journal-title-group>
      <issn pub-type="epub">2331-4249</issn>
      <issn pub-type="ppub">2331-4222</issn>
      <publisher>
        <publisher-name>Scientific Research Publishing</publisher-name>
      </publisher>
    </journal-meta>
    <article-meta>
      <article-id pub-id-type="doi">10.4236/wjet.2026.144051</article-id>
      <article-id pub-id-type="publisher-id">wjet-154506</article-id>
      <article-categories>
        <subj-group>
          <subject>Article</subject>
        </subj-group>
        <subj-group>
          <subject>Chemistry</subject>
          <subject>Materials Science</subject>
          <subject>Engineering</subject>
        </subj-group>
      </article-categories>
      <title-group>
        <article-title>Explainable Privacy-Preserving Scientific Retrieval with Knowledge Graphs: An Evidence-Grounded Framework for Scientific NoSQL Repositories</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <name name-style="western">
            <surname>Biswas</surname>
            <given-names>Tanmoy</given-names>
          </name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
      </contrib-group>
      <aff id="aff1"><label>1</label> Independent Researcher, Union, Kentucky, USA </aff>
      <author-notes>
        <fn fn-type="conflict" id="fn-conflict">
          <p>The author declares no conflicts of interest regarding the publication of this paper.</p>
        </fn>
      </author-notes>
      <pub-date pub-type="epub">
        <day>02</day>
        <month>11</month>
        <year>2026</year>
      </pub-date>
      <pub-date pub-type="collection">
        <month>11</month>
        <year>2026</year>
      </pub-date>
      <volume>14</volume>
      <issue>04</issue>
      <fpage>821</fpage>
      <lpage>839</lpage>
      <history>
        <date date-type="received">
          <day>27</day>
          <month>08</month>
          <year>2026</year>
        </date>
        <date date-type="accepted">
          <day>12</day>
          <month>10</month>
          <year>2026</year>
        </date>
        <date date-type="published">
          <day>15</day>
          <month>10</month>
          <year>2026</year>
        </date>
      </history>
      <permissions>
        <copyright-statement>© 2026 by the authors and Scientific Research Publishing Inc.</copyright-statement>
        <copyright-year>2026</copyright-year>
        <license license-type="open-access">
          <license-p> This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license ( <ext-link ext-link-type="uri" xlink:href="https://creativecommons.org/licenses/by/4.0/">https://creativecommons.org/licenses/by/4.0/</ext-link> ). </license-p>
        </license>
      </permissions>
      <self-uri content-type="doi" xlink:href="https://doi.org/10.4236/wjet.2026.144051">https://doi.org/10.4236/wjet.2026.144051</self-uri>
      <abstract>
        <p>Large Language Models (LLMs) provide powerful interfaces for scientific knowledge retrieval, but privacy-preserving access alone does not satisfy the transparency, provenance, and reproducibility requirements of research-intensive environments. Building directly on prior work that introduced differential-privacy controls for LLM integration with scientific NoSQL repositories [<xref ref-type="bibr" rid="B1">1</xref>], this paper proposes Explainable Privacy-Preserving Scientific Retrieval (EPPSR), a unified architecture combining a Scientific Knowledge Graph, semantic and graph retrieval, privacy-risk-aware evidence ranking, provenance, and evidence-level explanations. A revised reproducible V2 benchmark explicitly separates formal differential privacy (DP) from application-layer sensitive-information exposure. Formal DP is evaluated only for a bounded aggregate mean-effect release using replace-one adjacency, clipping to [-18, 18], sensitivity Δ = 6, a Laplace mechanism, an epsilon sweep from 0.1 to 5.0, and repeated-query composition. Sensitive Information Exposure Rate (SIER) separately measures unredacted disclosure of sensitive record identifiers or graph edges. The synthetic benchmark contains 672 records and 80 stratified queries with reliable, contradictory, incomplete-text, sensitive, and lexical-distractor evidence. Across the controlled simulation, EPPSR achieved 100.00% answer accuracy, 97.50% Recall@5, 46.25% citation accuracy, 0.00% SIER, 88.60% provenance completeness, and 92.50% citation support. At epsilon = 1.0, the DP aggregate release achieved 82.02% direction accuracy with a mean absolute error of 6.01 effect-percentage points. Ablation results show that removing consistency reduced answer accuracy to 91.25%, while a semantic-plus-graph-only variant fell to 66.25%. These results demonstrate controlled architectural behavior rather than production LLM performance and show why privacy guarantees, exposure metrics, evidence quality, and retrieval utility must be evaluated separately.</p>
      </abstract>
      <kwd-group kwd-group-type="author-generated" xml:lang="en">
        <kwd>Differential Privacy</kwd>
        <kwd>Explainable Artificial Intelligence</kwd>
        <kwd>Knowledge Graphs</kwd>
        <kwd>Large Language Models</kwd>
        <kwd>NoSQL Databases</kwd>
        <kwd>Retrieval-Augmented Generation</kwd>
        <kwd>Scientific Information Retrieval</kwd>
        <kwd>Trustworthy AI</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec1">
      <title>1. Introduction</title>
      <p>Large Language Models (LLMs) have transformed natural-language access to complex information. Transformer architectures [<xref ref-type="bibr" rid="B2">2</xref>] and subsequent pretrained language models such as BERT [<xref ref-type="bibr" rid="B3">3</xref>] established the technical foundation for modern question answering, summarization, and knowledge-intensive language processing. In scientific research, these capabilities create an opportunity to expose decades of experimental documentation through conversational interfaces rather than requiring researchers to manually search heterogeneous laboratory systems.</p>
      <p>Scientific repositories, however, differ from general-purpose information collections. They may contain proprietary formulations, unpublished experimental observations, process conditions, confidential project information, and regulated data. Our preceding study, “Privacy-Preserving LLM Integration with Scientific NoSQL Repositories: A Differential Privacy Approach,” introduced a layered framework for protecting scientific NoSQL content using local sensitivity analysis, differential-privacy-calibrated transformations, privacy-aware embeddings, and a controlled LLM interface [<xref ref-type="bibr" rid="B1">1</xref>]. That work demonstrated the importance of treating privacy as a first-class architectural requirement rather than relying only on conventional access control. Related work on LLM-NoSQL integration for experiment documentation also provides an earlier foundation for scientific knowledge access [<xref ref-type="bibr" rid="B4">4</xref>].</p>
      <p>Privacy alone does not make an AI-generated scientific answer trustworthy. Researchers must also be able to determine what evidence supports a conclusion, whether the evidence is consistent, how entities and experiments are related, and whether the system is uncertain. Retrieval-Augmented Generation (RAG) addresses part of this problem by combining parametric language models with external non-parametric memory [<xref ref-type="bibr" rid="B5">5</xref>]. RAG improves grounding and allows domain knowledge to be updated independently of model retraining, but conventional implementations remain primarily document- or chunk-oriented [<xref ref-type="bibr" rid="B6">6</xref>].</p>
      <p>Scientific knowledge is inherently relational. A compound may occur in a formulation, the formulation may be tested under a process condition, the experiment may generate an analytical measurement, and that measurement may support or contradict a later conclusion. Knowledge graphs provide an explicit representation of such entities and relationships [<xref ref-type="bibr" rid="B7">7</xref>]. Recent work has emphasized the complementary strengths of LLMs and knowledge graphs: LLMs provide flexible language understanding, whereas knowledge graphs provide explicit factual structure and interpretable relationships [<xref ref-type="bibr" rid="B8">8</xref>].</p>
      <p>Explainability introduces an additional requirement. XAI research emphasizes transparency, accountability, and the need to adapt explanations to their intended audience [<xref ref-type="bibr" rid="B9">9</xref>]. For generative scientific retrieval, explanation should not mean exposing hidden chain-of-thought. Instead, a scientifically useful explanation should identify observable evidence, provenance, and relationships that support a generated claim. This paper therefore defines explainability as evidence-level traceability.</p>
      <p>To address these requirements, this paper proposes Explainable Privacy-Preserving Scientific Retrieval (EPPSR), which extends the privacy-preserving NoSQL-LLM architecture in [<xref ref-type="bibr" rid="B1">1</xref>] with a Scientific Knowledge Graph, dual-channel retrieval, privacy-aware evidence ranking, provenance mapping, calibrated confidence, and claim verification. The central research question is whether scientific retrieval can be simultaneously private, evidence-grounded, explainable, and useful.</p>
      <p>The contributions are fivefold: 1) a unified EPPSR architecture integrating formal DP for explicitly defined aggregate releases with scientific knowledge graphs, retrieval, provenance, and evidence-level explainability; 2) a formal privacy threat model that distinguishes epsilon-DP guarantees from application-layer sensitive-information exposure through answers, citations, graph paths, explanations, and repeated queries; 3) a dual-channel retrieval and privacy-aware evidence-ranking mechanism combining semantic, graph, provenance, consistency, and disclosure-risk signals; 4) reproducible explanation metrics based on evidence coverage, provenance completeness, graph-path traceability, and citation support rather than an arbitrary composite explanation score; and 5) a V2 synthetic benchmark with epsilon sweeps, repeated-query privacy accounting, bootstrap confidence intervals, and ablation analysis.</p>
    </sec>
    <sec id="sec2">
      <title>2. Related Work</title>
      <p><bold>A. LLMs and Retrieval-Augmented Generation</bold></p>
      <p>The transformer architecture replaced recurrence with attention and enabled scalable contextual representation learning [<xref ref-type="bibr" rid="B2">2</xref>]. BERT subsequently demonstrated the value of deep bidirectional pretraining for language understanding [<xref ref-type="bibr" rid="B3">3</xref>]. Although modern LLMs can encode substantial factual knowledge, parametric know- ledge can become stale and may not provide reliable provenance. Lewis <italic>et al.</italic> introduced RAG to combine sequence generation with retrieved non-parametric memory [<xref ref-type="bibr" rid="B4">4</xref>]. Later surveys describe the evolution from naive RAG toward advanced and modular architectures incorporating improved retrieval, reranking, and generation control [<xref ref-type="bibr" rid="B5">5</xref>]. Broader language-model and retrieval research includes unsupervised multitask language models, text-to-text transfer learning, few-shot learning, dense passage retrieval, late-interaction ranking, heterogeneous retrieval benchmarks, retrieval-supported generation, retrieval-augmented pretraining, and transformer-based text ranking [<xref ref-type="bibr" rid="B10">10</xref>]-[<xref ref-type="bibr" rid="B18">18</xref>].</p>
      <p>RAG is particularly relevant to enterprise science because proprietary information can remain in organizational repositories while only selected evidence is passed to the model. Nevertheless, retrieval quality and generation faithfulness remain separate failure points. Hallucination research has shown that fluent generation can remain factually inconsistent or unsupported [<xref ref-type="bibr" rid="B19">19</xref>]. Scientific use therefore requires not only retrieval but explicit evidence verification. </p>
      <p>Research on chain-of-thought prompting, factual consistency, truthfulness, and machine-comprehension benchmarks further shows the importance of reasoning and evidence-based evaluation in language systems [<xref ref-type="bibr" rid="B19">19</xref>]-[<xref ref-type="bibr" rid="B22">22</xref>].</p>
      <p><bold>B. Knowledge Graphs and Graph-Enhanced Retrieval</bold></p>
      <p>Knowledge graphs represent entities and relations in a form suitable for querying, validation, reasoning, and integration across heterogeneous information sources [<xref ref-type="bibr" rid="B7">7</xref>]. Construction commonly involves knowledge acquisition, refinement, entity resolution, relation extraction, and graph evolution [<xref ref-type="bibr" rid="B23">23</xref>]. These capabilities align closely with scientific repositories in which experiments, materials, conditions, methods, measurements, and outcomes form a connected evidence network. Foundational work on neural tensor networks, translational embeddings, relational machine learning, knowledge-graph embeddings, and large-scale knowledge-graph systems established important methods for representing and reasoning over multi-relational knowledge [<xref ref-type="bibr" rid="B24">24</xref>]-[<xref ref-type="bibr" rid="B28">28</xref>].</p>
      <p>Pan <italic>et al.</italic> categorize LLM-KG integration into KG-enhanced LLMs, LLM-augmented KGs, and synergized LLM-KG systems [<xref ref-type="bibr" rid="B7">7</xref>]. Graph-based retrieval extends RAG by retrieving nodes, neighborhoods, paths, or graph-derived summaries rather than relying exclusively on independent text chunks. Such approaches can support multi-hop questions, but graph structure does not inherently provide privacy. Sensitive scientific relationships can themselves disclose intellectual property.</p>
      <p><bold>C. Differential Privacy and Privacy-Preserving Language Systems</bold></p>
      <p>Differential Privacy (DP) provides a formal mechanism for bounding the effect of an individual record on released outputs [<xref ref-type="bibr" rid="B29">29</xref>][<xref ref-type="bibr" rid="B30">30</xref>]. DP has been extended to machine learning through techniques such as differentially private stochastic gradient descent [<xref ref-type="bibr" rid="B30">30</xref>]. Privacy risks in language models also include memorization and extraction of rare training sequences [<xref ref-type="bibr" rid="B31">31</xref>], motivating privacy controls at training, retrieval, and inference boundaries. Privacy research has also examined model inversion, membership inference, privacy analysis of deep learning, privacy-preserving knowledge transfer, federated learning, and decentralized training [<xref ref-type="bibr" rid="B32">32</xref>]-[<xref ref-type="bibr" rid="B37">37</xref>]. These studies show that privacy risk can arise from both model outputs and the broader learning architecture.</p>
      <p>Our prior work [<xref ref-type="bibr" rid="B1">1</xref>] applied DP concepts directly to scientific NoSQL retrieval and LLM interaction. The present study treats that architecture as the privacy foundation and investigates the next problem: how to add evidence traceability and relational reasoning without allowing the explanation itself to become a disclosure channel.</p>
      <p><bold>D. Explainable and Trustworthy AI</bold></p>
      <p>Explainable AI encompasses methods for making model behavior understandable to users and stakeholders [<xref ref-type="bibr" rid="B9">9</xref>]. Local surrogate approaches such as LIME [<xref ref-type="bibr" rid="B38">38</xref>] and additive attribution approaches such as SHAP [<xref ref-type="bibr" rid="B39">39</xref>] are widely used for predictive models. These methods are not sufficient by themselves for generative scientific retrieval because a researcher needs source evidence and provenance rather than only feature attribution. Broader XAI research has examined black-box explanation, rigorous interpretability, limits of interpretability, counterfactual explanations, and general XAI taxonomies [<xref ref-type="bibr" rid="B9">9</xref>][<xref ref-type="bibr" rid="B40">40</xref>]-[<xref ref-type="bibr" rid="B42">42</xref>]. These perspectives reinforce [<xref ref-type="bibr" rid="B43">43</xref>] the need for explanations that are measurable and appropriate to the scientific retrieval setting.</p>
      <p>Trustworthy scientific generation therefore requires evidence-grounded explanation: generated claims should map to authorized experimental records, graph paths, and provenance. This requirement also creates a privacy-explainability tension. Greater transparency may reveal protected information, while aggressive privacy controls may make explanations scientifically uninformative. EPPSR explicitly places explanation downstream of privacy enforcement.</p>
    </sec>
    <sec id="sec3">
      <title>3. Proposed EPPSR Framework</title>
      <p>EPPSR is designed around five requirements: privacy preservation, evidence grounding, relational reasoning, explainability and traceability, and confidence awareness. It separates storage, retrieval, privacy processing, reasoning, generation, and verification so that the LLM never receives unrestricted access to the underlying scientific repository. The layered architecture and controlled privacy boundary are shown in <xref ref-type="fig" rid="fig1">Figure 1</xref>.</p>
      <fig id="fig1">
        <label>Figure 1</label>
        <graphic xlink:href="https://html.scirp.org/file/1561969-rId13.jpeg?20261015111833" />
      </fig>
      <p><bold>Figure 1.</bold> Layered system architecture of the proposed EPPSR framework.</p>
      <p><bold>A. Scientific Data and Knowledge Representation</bold></p>
      <p>Let the scientific repository be <inline-formula><mml:math display="inline"><mml:mrow><mml:mi> D </mml:mi><mml:mo> = </mml:mo><mml:mrow><mml:mo> { </mml:mo><mml:mrow><mml:mi> d </mml:mi><mml:mn> 1 </mml:mn><mml:mo> , </mml:mo><mml:mi> d </mml:mi><mml:mn> 2 </mml:mn><mml:mo> , </mml:mo><mml:mo> ⋯ </mml:mo><mml:mo> , </mml:mo><mml:mi> d </mml:mi><mml:mi> n </mml:mi></mml:mrow><mml:mo> } </mml:mo></mml:mrow></mml:mrow></mml:math></inline-formula> , where each di is a scientific record containing structured, semi-structured, or unstructured experimental information. The Scientific Knowledge Graph is <inline-formula><mml:math display="inline"><mml:mrow><mml:mi> G </mml:mi><mml:mo> = </mml:mo><mml:mrow><mml:mo> ( </mml:mo><mml:mrow><mml:mi> V </mml:mi><mml:mo> , </mml:mo><mml:mi> E </mml:mi></mml:mrow><mml:mo> ) </mml:mo></mml:mrow></mml:mrow></mml:math></inline-formula> , where V contains entities such as Experiment, Compound, Formulation, Condition, Measurement, Method, and Outcome, and E contains typed relations such as contains, testedIn, measuredBy, supports, validates, or contradicts.</p>
      <disp-formula id="FD1">
        <mml:math display="inline">
          <mml:mrow>
            <mml:mi>G</mml:mi>
            <mml:mo>=</mml:mo>
            <mml:mrow>
              <mml:mo>(</mml:mo>
              <mml:mrow>
                <mml:mi>V</mml:mi>
                <mml:mo>,</mml:mo>
                <mml:mi>E</mml:mi>
              </mml:mrow>
              <mml:mo>)</mml:mo>
            </mml:mrow>
          </mml:mrow>
        </mml:math>
      </disp-formula>
      <p>The NoSQL repository remains the authoritative evidence source. The graph acts as a relational index and each edge retains provenance linking it to one or more source records. This design avoids treating graph-derived relationships as detached facts. An example of this traceable scientific evidence path is shown in <xref ref-type="fig" rid="fig2">Figure 2</xref>.</p>
      <fig id="fig2">
        <label>Figure 2</label>
        <graphic xlink:href="https://html.scirp.org/file/1561969-rId20.jpeg?20261015111833" />
      </fig>
      <p><bold>Figure 2.</bold>Example scientific knowledge graph and evidence path.</p>
      <p><bold>B. Query Understanding and Privacy Boundary</bold></p>
      <p>A query q is transformed into<inline-formula><mml:math display="inline"><mml:mrow><mml:mi> Q </mml:mi><mml:mrow><mml:mo> ( </mml:mo><mml:mi> q </mml:mi><mml:mo> ) </mml:mo></mml:mrow><mml:mo> = </mml:mo><mml:mrow><mml:mo> { </mml:mo><mml:mrow><mml:mi> I </mml:mi><mml:mi> q </mml:mi><mml:mo> , </mml:mo><mml:mi> V </mml:mi><mml:mi> q </mml:mi><mml:mo> , </mml:mo><mml:mi> C </mml:mi><mml:mi> q </mml:mi><mml:mo> , </mml:mo><mml:mi> S </mml:mi><mml:mi> q </mml:mi></mml:mrow><mml:mo> } </mml:mo></mml:mrow></mml:mrow></mml:math></inline-formula> , representing intent, detected scientific entities, contextual constraints, and sensitivity indicators. Authorization determines which records, graph entities, and attributes may participate in retrieval. Differential privacy is then used where formal statistical disclosure protection is appropriate; it is not treated as a substitute for access control.</p>
      <disp-formula id="FD2">
        <mml:math display="inline">
          <mml:mrow>
            <mml:mi>Pr</mml:mi>
            <mml:mrow>
              <mml:mo>[</mml:mo>
              <mml:mrow>
                <mml:mi>M</mml:mi>
                <mml:mrow>
                  <mml:mo>(</mml:mo>
                  <mml:mi>D</mml:mi>
                  <mml:mo>)</mml:mo>
                </mml:mrow>
                <mml:mo>∈</mml:mo>
                <mml:mi>S</mml:mi>
              </mml:mrow>
              <mml:mo>]</mml:mo>
            </mml:mrow>
            <mml:mo>≤</mml:mo>
            <mml:mi>exp</mml:mi>
            <mml:mrow>
              <mml:mo>(</mml:mo>
              <mml:mi>ε</mml:mi>
              <mml:mo>)</mml:mo>
            </mml:mrow>
            <mml:mi>Pr</mml:mi>
            <mml:mrow>
              <mml:mo>[</mml:mo>
              <mml:mrow>
                <mml:mi>M</mml:mi>
                <mml:mrow>
                  <mml:mo>(</mml:mo>
                  <mml:msup>
                    <mml:mi>D</mml:mi>
                    <mml:mo>′</mml:mo>
                  </mml:msup>
                  <mml:mo>)</mml:mo>
                </mml:mrow>
                <mml:mo>∈</mml:mo>
                <mml:mi>S</mml:mi>
              </mml:mrow>
              <mml:mo>]</mml:mo>
            </mml:mrow>
          </mml:mrow>
        </mml:math>
      </disp-formula>
      <p>For a numerical function f(D), the Laplace mechanism is<inline-formula><mml:math display="inline"><mml:mrow><mml:mi> M </mml:mi><mml:mrow><mml:mo> ( </mml:mo><mml:mi> D </mml:mi><mml:mo> ) </mml:mo></mml:mrow><mml:mo> = </mml:mo><mml:mi> f </mml:mi><mml:mrow><mml:mo> ( </mml:mo><mml:mi> D </mml:mi><mml:mo> ) </mml:mo></mml:mrow></mml:mrow></mml:math></inline-formula><inline-formula><mml:math display="inline"><mml:mrow><mml:mo> + </mml:mo><mml:mtext> Lap </mml:mtext><mml:mrow><mml:mo> ( </mml:mo><mml:mrow><mml:mrow><mml:mrow><mml:mi> Δ </mml:mi><mml:mi> f </mml:mi></mml:mrow><mml:mo> / </mml:mo><mml:mi> ε </mml:mi></mml:mrow></mml:mrow><mml:mo> ) </mml:mo></mml:mrow></mml:mrow></mml:math></inline-formula> , where Δf denotes sensitivity and <italic>ε</italic> is the privacy budget [<xref ref-type="bibr" rid="B25">25</xref>][<xref ref-type="bibr" rid="B26">26</xref>].</p>
      <disp-formula id="FD3">
        <mml:math display="inline">
          <mml:mrow>
            <mml:mi>M</mml:mi>
            <mml:mrow>
              <mml:mo>(</mml:mo>
              <mml:mi>D</mml:mi>
              <mml:mo>)</mml:mo>
            </mml:mrow>
            <mml:mo>=</mml:mo>
            <mml:mi>f</mml:mi>
            <mml:mrow>
              <mml:mo>(</mml:mo>
              <mml:mi>D</mml:mi>
              <mml:mo>)</mml:mo>
            </mml:mrow>
            <mml:mo>+</mml:mo>
            <mml:mi>L</mml:mi>
            <mml:mi>a</mml:mi>
            <mml:mi>p</mml:mi>
            <mml:mrow>
              <mml:mo>(</mml:mo>
              <mml:mrow>
                <mml:mrow>
                  <mml:mrow>
                    <mml:mi>Δ</mml:mi>
                    <mml:mi>f</mml:mi>
                  </mml:mrow>
                  <mml:mo>/</mml:mo>
                  <mml:mi>ε</mml:mi>
                </mml:mrow>
              </mml:mrow>
              <mml:mo>)</mml:mo>
            </mml:mrow>
          </mml:mrow>
        </mml:math>
      </disp-formula>
      <p><bold>C. Dual-Channel Retrieval and Graph Reasoning</bold></p>
      <p>EPPSR retrieves evidence through two coordinated channels. Semantic retrieval ranks NoSQL records using query-record similarity, while graph retrieval traverses entities and paths relevant to the query. The candidate set is the union of both channels.</p>
      <p><inline-formula><mml:math display="inline"><mml:mrow><mml:mi> R </mml:mi><mml:mrow><mml:mo> ( </mml:mo><mml:mi> q </mml:mi><mml:mo> ) </mml:mo></mml:mrow><mml:mo> = </mml:mo><mml:msub><mml:mi> R </mml:mi><mml:mrow><mml:mi> D </mml:mi><mml:mi> B </mml:mi><mml:mrow><mml:mo> ( </mml:mo><mml:mi> q </mml:mi><mml:mo> ) </mml:mo></mml:mrow></mml:mrow></mml:msub><mml:mo> ∪ </mml:mo><mml:msub><mml:mi> R </mml:mi><mml:mrow><mml:mi> K </mml:mi><mml:mi> G </mml:mi><mml:mrow><mml:mo> ( </mml:mo><mml:mi> q </mml:mi><mml:mo> ) </mml:mo></mml:mrow></mml:mrow></mml:msub></mml:mrow></mml:math></inline-formula> . The resulting dual-channel semantic and graph retrieval flow is illustrated in <xref ref-type="fig" rid="fig3">Figure 3</xref>.</p>
      <fig id="fig3">
        <label>Figure 3</label>
        <graphic xlink:href="https://html.scirp.org/file/1561969-rId33.jpeg?20261015111833" />
      </fig>
      <p><bold>Figure 3.</bold>Dual-channel semantic and graph-based retrieval.</p>
      <p>For multi-hop scientific questions, candidate paths π are scored using semantic relevance, relational relevance, provenance strength, and a length penalty. This allows concise evidence paths to be preferred over long or weakly supported chains.</p>
      <disp-formula id="FD4">
        <mml:math display="inline">
          <mml:mrow>
            <mml:mi>S</mml:mi>
            <mml:mtext>π</mml:mtext>
            <mml:mo>=</mml:mo>
            <mml:mi>α</mml:mi>
            <mml:mtext>Ssem</mml:mtext>
            <mml:mo>+</mml:mo>
            <mml:mi>β</mml:mi>
            <mml:mtext>Srel</mml:mtext>
            <mml:mo>+</mml:mo>
            <mml:mi>γ</mml:mi>
            <mml:mtext>Sprov</mml:mtext>
            <mml:mo>−</mml:mo>
            <mml:mi>λ</mml:mi>
            <mml:mi>L</mml:mi>
            <mml:mtext>π</mml:mtext>
          </mml:mrow>
        </mml:math>
      </disp-formula>
      <p><bold>D. Privacy-Aware Evidence Ranking</bold></p>
      <p>Candidate evidence is ranked using a composite score. In contrast with conventional RAG, highly relevant evidence can be demoted when disclosure risk is high. This mechanism creates an explicit privacy-utility trade-off.</p>
      <disp-formula id="FD5">
        <mml:math display="inline">
          <mml:mrow>
            <mml:mi>S</mml:mi>
            <mml:mrow>
              <mml:mo>(</mml:mo>
              <mml:mrow>
                <mml:mi>e</mml:mi>
                <mml:mi>i</mml:mi>
              </mml:mrow>
              <mml:mo>)</mml:mo>
            </mml:mrow>
            <mml:mo>=</mml:mo>
            <mml:mi>w</mml:mi>
            <mml:mn>1</mml:mn>
            <mml:mtext>Ssem</mml:mtext>
            <mml:mo>+</mml:mo>
            <mml:mi>w</mml:mi>
            <mml:mn>2</mml:mn>
            <mml:mi>S</mml:mi>
            <mml:mtext>graph</mml:mtext>
            <mml:mo>+</mml:mo>
            <mml:mi>w</mml:mi>
            <mml:mn>3</mml:mn>
            <mml:mtext>Sprov</mml:mtext>
            <mml:mo>+</mml:mo>
            <mml:mi>w</mml:mi>
            <mml:mn>4</mml:mn>
            <mml:mtext>Scons</mml:mtext>
            <mml:mo>−</mml:mo>
            <mml:mi>w</mml:mi>
            <mml:mn>5</mml:mn>
            <mml:mtext>Srisk</mml:mtext>
          </mml:mrow>
        </mml:math>
      </disp-formula>
      <p>The highest-ranked privacy-approved evidence E* is provided to the generator. Sensitive identifiers may be redacted, and eligible numerical outputs may be perturbed using DP. The explanation layer receives only the same authorized and privacy-processed evidence. The privacy-aware evidence and explanation release boundary is shown in <xref ref-type="fig" rid="fig4">Figure 4</xref>.</p>
      <fig id="fig4">
        <label>Figure 4</label>
        <graphic xlink:href="https://html.scirp.org/file/1561969-rId38.jpeg?20261015111833" />
      </fig>
      <p><bold>Figure 4.</bold>Privacy-aware evidence and explanation release boundary.</p>
      <p><bold>E. Evidence-Grounded Generation, Confidence, and Verification</bold></p>
      <p>The generation stage is constrained to the approved evidence set and relevant graph paths. A response is represented as<inline-formula><mml:math display="inline"><mml:mrow><mml:mi> O </mml:mi><mml:mo> = </mml:mo><mml:mrow><mml:mo> { </mml:mo><mml:mrow><mml:mi> A </mml:mi><mml:mo> , </mml:mo><mml:mi> E </mml:mi><mml:mo> , </mml:mo><mml:mi> X </mml:mi><mml:mo> , </mml:mo><mml:mi> C </mml:mi><mml:mo> , </mml:mo><mml:mi> P </mml:mi></mml:mrow><mml:mo> } </mml:mo></mml:mrow></mml:mrow></mml:math></inline-formula> , containing the answer, released evidence, privacy-compliant explanation, calibrated confidence, and privacy summary. Confidence is derived from evidence strength, graph support, agreement, and provenance rather than the LLM’s verbal certainty.</p>
      <disp-formula id="FD6">
        <mml:math display="inline">
          <mml:mrow>
            <mml:mi>C</mml:mi>
            <mml:mrow>
              <mml:mo>(</mml:mo>
              <mml:mi>q</mml:mi>
              <mml:mo>)</mml:mo>
            </mml:mrow>
            <mml:mo>=</mml:mo>
            <mml:mi>θ</mml:mi>
            <mml:mn>1</mml:mn>
            <mml:mi>E</mml:mi>
            <mml:mi>s</mml:mi>
            <mml:mo>+</mml:mo>
            <mml:mi>θ</mml:mi>
            <mml:mn>2</mml:mn>
            <mml:mi>G</mml:mi>
            <mml:mi>s</mml:mi>
            <mml:mo>+</mml:mo>
            <mml:mi>θ</mml:mi>
            <mml:mn>3</mml:mn>
            <mml:mi>A</mml:mi>
            <mml:mi>s</mml:mi>
            <mml:mo>+</mml:mo>
            <mml:mi>θ</mml:mi>
            <mml:mn>4</mml:mn>
            <mml:mi>P</mml:mi>
            <mml:mi>s</mml:mi>
            <mml:mo>,</mml:mo>
            <mml:mtext>
               
            </mml:mtext>
            <mml:mi>Σ</mml:mi>
            <mml:mi>θ</mml:mi>
            <mml:mi>i</mml:mi>
            <mml:mo>=</mml:mo>
            <mml:mn>1</mml:mn>
          </mml:mrow>
        </mml:math>
      </disp-formula>
      <p>Generated claims are verified against the approved evidence. For claims <inline-formula><mml:math display="inline"><mml:mrow><mml:mi> C </mml:mi><mml:mi> A </mml:mi><mml:mo> = </mml:mo><mml:mrow><mml:mo> { </mml:mo><mml:mrow><mml:mi> c </mml:mi><mml:mn> 1 </mml:mn><mml:mo> , </mml:mo><mml:mo> ⋯ </mml:mo><mml:mo> , </mml:mo><mml:mi> c </mml:mi><mml:mi> m </mml:mi></mml:mrow><mml:mo> } </mml:mo></mml:mrow></mml:mrow></mml:math></inline-formula> , evidence coverage is <inline-formula><mml:math display="inline"><mml:mrow><mml:mi> E </mml:mi><mml:mi> C </mml:mi><mml:mrow><mml:mo> ( </mml:mo><mml:mi> A </mml:mi><mml:mo> ) </mml:mo></mml:mrow><mml:mo> = </mml:mo><mml:mi> Σ </mml:mi><mml:mrow><mml:mrow><mml:mi> V </mml:mi><mml:mrow><mml:mo> ( </mml:mo><mml:mrow><mml:mi> c </mml:mi><mml:mi> i </mml:mi></mml:mrow><mml:mo> ) </mml:mo></mml:mrow></mml:mrow><mml:mo> / </mml:mo><mml:mi> m </mml:mi></mml:mrow></mml:mrow></mml:math></inline-formula> , where <inline-formula><mml:math display="inline"><mml:mrow><mml:mi> V </mml:mi><mml:mrow><mml:mo> ( </mml:mo><mml:mrow><mml:mi> c </mml:mi><mml:mi> i </mml:mi></mml:mrow><mml:mo> ) </mml:mo></mml:mrow><mml:mo> = </mml:mo><mml:mn> 1 </mml:mn></mml:mrow></mml:math></inline-formula> when sufficient supporting evidence exists. Unsupported claims can be removed, regenerated, or explicitly marked as inference.</p>
      <disp-formula id="FD7">
        <mml:math display="inline">
          <mml:mrow>
            <mml:mi>E</mml:mi>
            <mml:mi>C</mml:mi>
            <mml:mrow>
              <mml:mo>(</mml:mo>
              <mml:mi>A</mml:mi>
              <mml:mo>)</mml:mo>
            </mml:mrow>
            <mml:mo>=</mml:mo>
            <mml:mrow>
              <mml:mrow>
                <mml:mrow>
                  <mml:mo>(</mml:mo>
                  <mml:mrow>
                    <mml:mi>Σ</mml:mi>
                    <mml:mtext>
                    </mml:mtext>
                    <mml:mi>V</mml:mi>
                    <mml:mrow>
                      <mml:mo>(</mml:mo>
                      <mml:mrow>
                        <mml:mi>c</mml:mi>
                        <mml:mi>i</mml:mi>
                      </mml:mrow>
                      <mml:mo>)</mml:mo>
                    </mml:mrow>
                  </mml:mrow>
                  <mml:mo>)</mml:mo>
                </mml:mrow>
              </mml:mrow>
              <mml:mo>/</mml:mo>
              <mml:mi>m</mml:mi>
            </mml:mrow>
          </mml:mrow>
        </mml:math>
      </disp-formula>
    </sec>
    <sec id="sec4">
      <title>4. Experimental Methodology</title>
      <p><bold>A. Privacy Threat Model</bold></p>
      <p>The V2 evaluation defines an explicit adversary and release surface. The adversary is assumed to have legitimate access to the natural-language query interface and to observe answer text, citations or released record identifiers, graph-path or relationship summaries, provenance/confidence metadata, and outputs from repeated queries. Protected information includes sensitive record identity, sensitive graph relationships, and exact numeric values used in aggregate releases. This model intentionally treats explanations and graph paths as potential disclosure channels rather than assuming that only source documents are sensitive.</p>
      <p>Two privacy concepts are evaluated separately. First, formal differential privacy is applied only to a bounded numeric aggregate release. Second, Sensitive Information Exposure Rate (SIER) measures application-layer disclosure of sensitive record identifiers or graph edges. A zero SIER therefore means that no such unredacted disclosure occurred under the benchmark release policy; it is not, by itself, an epsilon-DP guarantee.</p>
      <p><bold>B. Formal Differential-Privacy Release</bold></p>
      <p>For each benchmark query, the protected numeric output is the mean effect percentage across a fixed-size cohort of m = 6 matching records. Neighboring datasets D and D’ use replace-one adjacency: both contain six records and differ in exactly one record. Each effect value is clipped to [L, U] = [−18, 18]. The global sensitivity of the bounded mean is therefore:</p>
      <disp-formula id="FD8">
        <mml:math display="inline">
          <mml:mrow>
            <mml:msub>
              <mml:mi>Δ</mml:mi>
              <mml:mrow>
                <mml:mi>m</mml:mi>
                <mml:mi>e</mml:mi>
                <mml:mi>a</mml:mi>
                <mml:mi>n</mml:mi>
              </mml:mrow>
            </mml:msub>
            <mml:mo>=</mml:mo>
            <mml:mrow>
              <mml:mrow>
                <mml:mrow>
                  <mml:mo>(</mml:mo>
                  <mml:mrow>
                    <mml:mi>U</mml:mi>
                    <mml:mo>−</mml:mo>
                    <mml:mi>L</mml:mi>
                  </mml:mrow>
                  <mml:mo>)</mml:mo>
                </mml:mrow>
              </mml:mrow>
              <mml:mo>/</mml:mo>
              <mml:mi>m</mml:mi>
            </mml:mrow>
            <mml:mo>=</mml:mo>
            <mml:mrow>
              <mml:mrow>
                <mml:mn>36</mml:mn>
              </mml:mrow>
              <mml:mo>/</mml:mo>
              <mml:mn>6</mml:mn>
            </mml:mrow>
            <mml:mo>=</mml:mo>
            <mml:mn>6.</mml:mn>
          </mml:mrow>
        </mml:math>
      </disp-formula>
      <p>The released statistic uses the Laplace mechanism:</p>
      <disp-formula id="FD9">
        <mml:math display="inline">
          <mml:mrow>
            <mml:mi>M</mml:mi>
            <mml:mrow>
              <mml:mo>(</mml:mo>
              <mml:mi>D</mml:mi>
              <mml:mo>)</mml:mo>
            </mml:mrow>
            <mml:mo>=</mml:mo>
            <mml:mi>m</mml:mi>
            <mml:mi>e</mml:mi>
            <mml:mi>a</mml:mi>
            <mml:mi>n</mml:mi>
            <mml:mrow>
              <mml:mo>(</mml:mo>
              <mml:mrow>
                <mml:mi>c</mml:mi>
                <mml:mi>l</mml:mi>
                <mml:mi>i</mml:mi>
                <mml:mi>p</mml:mi>
                <mml:mrow>
                  <mml:mo>(</mml:mo>
                  <mml:mrow>
                    <mml:mi>D</mml:mi>
                    <mml:mo>,</mml:mo>
                    <mml:mo>−</mml:mo>
                    <mml:mn>18</mml:mn>
                    <mml:mo>,</mml:mo>
                    <mml:mn>18</mml:mn>
                  </mml:mrow>
                  <mml:mo>)</mml:mo>
                </mml:mrow>
              </mml:mrow>
              <mml:mo>)</mml:mo>
            </mml:mrow>
            <mml:mo>+</mml:mo>
            <mml:mi>L</mml:mi>
            <mml:mi>a</mml:mi>
            <mml:mi>p</mml:mi>
            <mml:mi>l</mml:mi>
            <mml:mi>a</mml:mi>
            <mml:mi>c</mml:mi>
            <mml:mi>e</mml:mi>
            <mml:mrow>
              <mml:mo>(</mml:mo>
              <mml:mrow>
                <mml:mn>0</mml:mn>
                <mml:mo>,</mml:mo>
                <mml:mrow>
                  <mml:mrow>
                    <mml:msub>
                      <mml:mi>Δ</mml:mi>
                      <mml:mrow>
                        <mml:mi>m</mml:mi>
                        <mml:mi>e</mml:mi>
                        <mml:mi>a</mml:mi>
                        <mml:mi>n</mml:mi>
                      </mml:mrow>
                    </mml:msub>
                  </mml:mrow>
                  <mml:mo>/</mml:mo>
                  <mml:mrow>
                    <mml:msub>
                      <mml:mi>ε</mml:mi>
                      <mml:mi>q</mml:mi>
                    </mml:msub>
                  </mml:mrow>
                </mml:mrow>
              </mml:mrow>
              <mml:mo>)</mml:mo>
            </mml:mrow>
            <mml:mo>.</mml:mo>
          </mml:mrow>
        </mml:math>
      </disp-formula>
      <p>The per-query privacy budget <italic>ε</italic><italic><sub>q</sub></italic> was swept over {0.1, 0.5, 1.0, 2.0, 5.0}. For repeated releases, pure sequential composition is used: <italic>ε</italic><sub>total</sub> ≤ Σ<italic>ε</italic><italic><sub>q</sub></italic>. Consequently, if all 80 releases were issued at <italic>ε</italic><italic><sub>q</sub></italic> = 1.0, the basic worst-case composed budget would be <italic>ε</italic><sub>total</sub> ≤ 80. A second experiment allocates fixed total budgets across all 80 releases to show the utility consequences of repeated-query accounting.</p>
      <p><bold>C. Synthetic Dataset and Query Generator</bold></p>
      <p>The generator uses random seed 20260914. The base design contains 8 compounds, 3 process conditions, and 4 measurement categories, yielding 96 compound-condition-measurement cohorts. Six evidence records are generated per cohort (576 base records): three reliable observations sampled from Normal(μ, 1.6) with provenance U(0.85, 1.00); one contradictory observation sampled from Normal (-μ, 2.5) with provenance U(0.25, 0.50); one partial-text observation sampled from Normal (μ, 2.2) with provenance U(0.60, 0.80); and one sensitive strong-evidence observation sampled from Normal (1.15μ, 1.0) with provenance U(0.90, 1.00). The latent μ has random sign and magnitude U(4,14), and all observed effects are clipped to [−18, 18].</p>
      <p>An additional 96 lexical distractor records contain query-like canonical terms while their structured condition and measurement fields refer to different concepts, producing 672 total records. Sensitive graph edges are assigned to every sensitive record plus an additional Bernoulli (0.10) sample. Eighty of the 96 cohorts are sampled without replacement to create benchmark queries; query wording alternates between canonical terminology and synonymized wording. The gold answer direction is the sign of the mean reliable evidence, and the gold citation is the highest-provenance reliable non-sensitive source when one is available. <bold>Table 1</bold> summarizes the complete V2 synthetic dataset configuration.</p>
      <p><bold>Table 1</bold><bold>.</bold>V2 synthetic dataset configuration.</p>
      <table-wrap id="tbl1">
        <label>Table 1</label>
        <table>
          <tbody>
            <tr>
              <td>
                <bold>Item</bold>
              </td>
              <td>
                <bold>Value</bold>
              </td>
            </tr>
            <tr>
              <td>Compounds</td>
              <td>8</td>
            </tr>
            <tr>
              <td>Conditions</td>
              <td>3</td>
            </tr>
            <tr>
              <td>Measurements</td>
              <td>4</td>
            </tr>
            <tr>
              <td>Base cohorts</td>
              <td>96</td>
            </tr>
            <tr>
              <td>Records per cohort</td>
              <td>6</td>
            </tr>
            <tr>
              <td>Base records</td>
              <td>576</td>
            </tr>
            <tr>
              <td>Lexical distractors</td>
              <td>96</td>
            </tr>
            <tr>
              <td>Total records</td>
              <td>672</td>
            </tr>
            <tr>
              <td>Benchmark queries</td>
              <td>80</td>
            </tr>
            <tr>
              <td>Random seed</td>
              <td>20260914</td>
            </tr>
          </tbody>
        </table>
      </table-wrap>
      <p><bold>D. Shared Retrieval Implementation and Baselines</bold></p>
      <p>All architectures share the same synthetic corpus and query set. Semantic retrieval uses scikit-learn TfidfVectorizer with unigram and bigram features, lowercase normalization, sublinear term frequency, and cosine similarity. No neural embedding model is claimed in V2. The Scientific Knowledge Graph is deterministically derived from record fields connecting compound, formulation, experiment, condition, measurement, and outcome. Graph relevance is the normalized match over compound, condition, and measurement, corresponding to a conceptual traversal depth of at most three relations.</p>
      <p>RAG uses semantic similarity only. PP-RAG uses the same semantic score with a 0.20 privacy-risk penalty. GraphRAG combines semantic and graph relevance with weights 0.55 and 0.45. Full EPPSR uses semantic = 0.40, graph = 0.25, provenance = 0.15, consistency = 0.20, and subtracts privacy risk with weight 0.20. Top-k is fixed at five. The implementation is a CPU-local Python simulation using NumPy, pandas, scikit-learn, and Matplotlib; no external LLM, GPU, or remote inference service is used. The shared implementation and experimental parameters are summarized in <bold>Table 2</bold>.</p>
      <p><bold>Table 2</bold><bold>.</bold>V2 experimental parameters.</p>
      <table-wrap id="tbl2">
        <label>Table 2</label>
        <table>
          <tbody>
            <tr>
              <td>
                <bold>Parameter</bold>
              </td>
              <td>
                <bold>Value</bold>
              </td>
            </tr>
            <tr>
              <td>Top-k</td>
              <td>5</td>
            </tr>
            <tr>
              <td>EPPSR semantic weight</td>
              <td>0.40</td>
            </tr>
            <tr>
              <td>EPPSR graph weight</td>
              <td>0.25</td>
            </tr>
            <tr>
              <td>EPPSR provenance weight</td>
              <td>0.15</td>
            </tr>
            <tr>
              <td>EPPSR consistency weight</td>
              <td>0.20</td>
            </tr>
            <tr>
              <td>EPPSR privacy-risk penalty</td>
              <td>0.20</td>
            </tr>
            <tr>
              <td>GraphRAG semantic/graph</td>
              <td>0.55/0.45</td>
            </tr>
            <tr>
              <td>DP clipping</td>
              <td>[−18, 18]</td>
            </tr>
            <tr>
              <td>DP sensitivity</td>
              <td>6</td>
            </tr>
            <tr>
              <td>
                <italic>ε</italic>
                sweep
              </td>
              <td>0.1, 0.5, 1, 2, 5</td>
            </tr>
            <tr>
              <td>Bootstrap samples</td>
              <td>5000</td>
            </tr>
            <tr>
              <td>DP Monte Carlo releases</td>
              <td>
                300/query/
                <italic>ε</italic>
              </td>
            </tr>
          </tbody>
        </table>
      </table-wrap>
      <p><bold>E. Evaluation Metrics</bold></p>
      <p>Answer accuracy is the fraction of queries for which the evidence-weighted predicted direction matches the gold direction. Recall@5 indicates whether the designated gold citation appears in the top-five set. Citation accuracy requires the top-ranked citation to equal the designated gold citation. SIER is defined as N_sensitive-unredacted/N_queries and is reported independently of formal DP.</p>
      <p>The former composite explanation score is removed. Explainability is instead decomposed into four observable measures: Evidence Coverage, the fraction of required answer dimensions (compound, condition, measurement, and direction) represented by top-five evidence; Provenance Completeness, the mean generated provenance-quality score of top-five evidence; Graph-Path Traceability, whether at least one top-five record provides a complete compound-condition-measurement graph match; and Citation Support, whether the top citation exactly matches the query entities and supports the generated direction.</p>
      <p><bold>F. Statistical Analysis and Ablation Design</bold></p>
      <p>Each of the 80 queries is evaluated under RAG, PP-RAG, GraphRAG, and EPPSR, yielding 320 main query-method observations. Query-level results are retained. Five-thousand-sample nonparametric bootstrap confidence intervals are computed for the principal rates and explanation components. The EPPSR ablation study evaluates the full system, removal of provenance, removal of consistency, removal of the privacy-risk scoring term, and a semantic-plus-graph-only configuration. The DP epsilon sweep uses 300 independent Laplace releases per query per epsilon value to estimate numerical error and direction utility. The formal DP privacy-utility evaluation design is illustrated in <xref ref-type="fig" rid="fig5">Figure 5</xref>.</p>
      <fig id="fig5">
        <label>Figure 5</label>
        <graphic xlink:href="https://html.scirp.org/file/1561969-rId55.jpeg?20261015111833" />
      </fig>
      <p><bold>Figure 5.</bold>V2 formal-DP privacy-utility evaluation design: mean absolute error across per-query epsilon values.</p>
    </sec>
    <sec id="sec5">
      <title>5. Results and Discussion</title>
      <p><bold>A. Retrieval, Grounding, and Sensitive Exposure</bold></p>
      <p><bold>Table 3</bold> summarizes the controlled V2 benchmark. EPPSR achieved 100.00% answer accuracy and 97.50% Recall@5, compared with 66.25% and 75.00% for GraphRAG, 60.00% and 13.75% for PP-RAG, and 58.75% and 10.00% for RAG. The answer-accuracy comparison is visualized in <xref ref-type="fig" rid="fig6">Figure 6</xref>. These values should be interpreted within the intentionally structured synthetic benchmark: contradictory low-provenance records, partial text, sensitive high-quality evidence, and lexical distractors were generated specifically to test the signals used by the architectures. They demonstrate controlled architectural behavior rather than general production superiority.</p>
      <p>Citation accuracy is substantially more demanding because it requires the top-ranked record to equal a single designated high-provenance non-sensitive gold source. EPPSR reached 46.25% citation accuracy, compared with 8.75% for GraphRAG, 6.25% for PP-RAG, and 5.00% for RAG. EPPSR’s 95% bootstrap interval for citation accuracy was 35.00% - 56.28%. RAG and GraphRAG produced SIER values of 41.25% and 37.50%, respectively, whereas PP-RAG and EPPSR produced 0.00% SIER under the benchmark redaction policy. This result is an application-layer exposure result and is not described as a differential-privacy guarantee. <xref ref-type="fig" rid="fig7">Figure 7</xref> visualizes the SIER comparison across the four architectures.</p>
      <p><bold>Table 3</bold><bold>.</bold>V2 overall results (%).</p>
      <table-wrap id="tbl3">
        <label>Table 3</label>
        <table>
          <tbody>
            <tr>
              <td>
                <bold>Method</bold>
              </td>
              <td>
                <bold>Accuracy</bold>
              </td>
              <td>
                <bold>R@5</bold>
              </td>
              <td>
                <bold>Citation</bold>
              </td>
              <td>
                <bold>SIER</bold>
              </td>
              <td>
                <bold>Prov.</bold>
              </td>
              <td>
                <bold>Cit. Support</bold>
              </td>
            </tr>
            <tr>
              <td>RAG</td>
              <td>58.75</td>
              <td>10.00</td>
              <td>5.00</td>
              <td>41.25</td>
              <td>72.14</td>
              <td>40.00</td>
            </tr>
            <tr>
              <td>PP-RAG</td>
              <td>60.00</td>
              <td>13.75</td>
              <td>6.25</td>
              <td>0.00</td>
              <td>71.68</td>
              <td>36.25</td>
            </tr>
            <tr>
              <td>GraphRAG</td>
              <td>66.25</td>
              <td>75.00</td>
              <td>8.75</td>
              <td>37.50</td>
              <td>75.07</td>
              <td>86.25</td>
            </tr>
            <tr>
              <td>EPPSR</td>
              <td>100.00</td>
              <td>97.50</td>
              <td>46.25</td>
              <td>0.00</td>
              <td>88.60</td>
              <td>92.50</td>
            </tr>
          </tbody>
        </table>
      </table-wrap>
      <fig id="fig6">
        <label>Figure 6</label>
        <graphic xlink:href="https://html.scirp.org/file/1561969-rId56.jpeg?20261015111834" />
      </fig>
      <p><bold>Figure 6.</bold>Answer accuracy across retrieval architectures in the V2 synthetic benchmark.</p>
      <fig id="fig7">
        <label>Figure 7</label>
        <graphic xlink:href="https://html.scirp.org/file/1561969-rId57.jpeg?20261015111834" />
      </fig>
      <p><bold>Figure 7.</bold> Sensitive Information Exposure Rate (SIER). SIER is separate from formal differential privacy.</p>
      <p><bold>B. Evidence-Level Explainability</bold></p>
      <p>EPPSR achieved 100.00% evidence coverage, 88.60% provenance completeness, 97.50% graph-path traceability, and 92.50% citation support. GraphRAG also achieved 100.00% graph-path traceability, as expected from its graph component, but its provenance completeness was 75.07%. <xref ref-type="fig" rid="fig8">Figure 8</xref> compares provenance completeness across the architectures. These component measures replace the previous 92.01% composite explanation score and make the explanation evaluation reproducible: each score has a stated definition and can be recomputed directly from query-level evidence.</p>
      <fig id="fig8">
        <label>Figure 8</label>
        <graphic xlink:href="https://html.scirp.org/file/1561969-rId58.jpeg?20261015111834" />
      </fig>
      <p><bold>Figure 8.</bold>Mean provenance completeness of top-five evidence across architectures.</p>
      <p><bold>C. Formal Differential-Privacy Utility</bold></p>
      <p>The epsilon sweep produces the expected privacy-utility pattern. At <italic>ε</italic> = 0.1, mean absolute error (MAE) was 59.85 effect-percentage points and direction accuracy was 55.00%. At <italic>ε</italic> = 0.5, MAE fell to 12.00 and direction accuracy rose to 70.51%. At <italic>ε</italic> = 1.0, MAE was 6.01 and direction accuracy 82.02%. Utility continued to improve at <italic>ε</italic> = 2.0 (MAE 3.02; 92.32% direction accuracy) and <italic>ε</italic> = 5.0 (MAE 1.21; 99.00% direction accuracy). The result illustrates the expected trade-off: lower epsilon provides a stronger per-release privacy bound but injects more noise into this small-cohort aggregate. The full epsilon sweep is reported in <bold>Table 4</bold>, while <xref ref-type="fig" rid="fig9">Figure 9</xref> and <xref ref-type="fig" rid="fig10">Figure 10</xref> show the corresponding MAE and direction-accuracy trends.</p>
      <p><bold>Table 4</bold><bold>.</bold>Formal dp epsilon sweep.</p>
      <table-wrap id="tbl4">
        <label>Table 4</label>
        <table>
          <tbody>
            <tr>
              <td>
                <italic>
                  <bold>ε</bold>
                </italic>
                <bold>/query</bold>
              </td>
              <td>
                <bold>Mean AE</bold>
              </td>
              <td>
                <bold>Median AE</bold>
              </td>
              <td>
                <bold>Direction Acc. %</bold>
              </td>
              <td>
                <bold>Δ</bold>
              </td>
            </tr>
            <tr>
              <td>0.1</td>
              <td>59.85</td>
              <td>41.06</td>
              <td>55.00</td>
              <td>6.0</td>
            </tr>
            <tr>
              <td>0.5</td>
              <td>12.00</td>
              <td>8.29</td>
              <td>70.51</td>
              <td>6.0</td>
            </tr>
            <tr>
              <td>1</td>
              <td>6.01</td>
              <td>4.17</td>
              <td>82.02</td>
              <td>6.0</td>
            </tr>
            <tr>
              <td>2</td>
              <td>3.02</td>
              <td>2.09</td>
              <td>92.32</td>
              <td>6.0</td>
            </tr>
            <tr>
              <td>5</td>
              <td>1.21</td>
              <td>0.83</td>
              <td>99.00</td>
              <td>6.0</td>
            </tr>
          </tbody>
        </table>
      </table-wrap>
      <fig id="fig9">
        <label>Figure 9</label>
        <graphic xlink:href="https://html.scirp.org/file/1561969-rId55.jpeg?20261015111834" />
      </fig>
      <p><bold>Figure 9.</bold>Mean absolute error versus per-query epsilon for the bounded aggregate release.</p>
      <fig id="fig10">
        <label>Figure 10</label>
        <graphic xlink:href="https://html.scirp.org/file/1561969-rId59.jpeg?20261015111834" />
      </fig>
      <p><bold>Figure 10.</bold>Direction accuracy versus per-query epsilon.</p>
      <p><bold>D. Repeated-Query Privacy Accounting</bold></p>
      <p>Basic sequential composition exposes an important limitation of interpreting <italic>ε</italic> = 1.0 in isolation. Eighty independent releases at <italic>ε</italic><italic><sub>q</sub></italic> = 1.0 have a worst-case pure-DP bound <italic>ε</italic><sub>total</sub> ≤ 80. Conversely, dividing a fixed total privacy budget equally across 80 queries can severely reduce utility in this small m = 6 cohort. For example, <italic>ε</italic><sub>total</sub> = 8 implies <italic>ε</italic><italic><sub>q</sub></italic> = 0.1, for which the repeated-query simulation yielded approximately 54.90% direction accuracy and MAE 60.18. Even <italic>ε</italic><sub>total</sub> = 40, corresponding to <italic>ε</italic><italic><sub>q</sub></italic> = 0.5, yielded 70.71% direction accuracy and MAE 11.88. The experiment therefore supports explicit budget accounting and discourages presenting a per-query epsilon without a release policy. The fixed-total-budget results across all 80 releases are summarized in <bold>Table 5</bold>.</p>
      <p><bold>Table 5</bold><bold>.</bold>Fixed total privacy budget across 80 releases.</p>
      <table-wrap id="tbl5">
        <label>Table 5</label>
        <table>
          <tbody>
            <tr>
              <td>
                <italic>
                  <bold>ε</bold>
                </italic>
                <bold>total</bold>
              </td>
              <td>
                <italic>
                  <bold>ε</bold>
                </italic>
                <bold>/query</bold>
              </td>
              <td>
                <bold>Mean AE</bold>
              </td>
              <td>
                <bold>Direction Acc. %</bold>
              </td>
            </tr>
            <tr>
              <td>1</td>
              <td>0.0125</td>
              <td>478.53</td>
              <td>50.71</td>
            </tr>
            <tr>
              <td>4</td>
              <td>0.0500</td>
              <td>119.98</td>
              <td>52.90</td>
            </tr>
            <tr>
              <td>8</td>
              <td>0.1000</td>
              <td>60.18</td>
              <td>54.90</td>
            </tr>
            <tr>
              <td>20</td>
              <td>0.2500</td>
              <td>24.12</td>
              <td>61.76</td>
            </tr>
            <tr>
              <td>40</td>
              <td>0.5000</td>
              <td>11.88</td>
              <td>70.71</td>
            </tr>
          </tbody>
        </table>
      </table-wrap>
      <p><bold>E. EPPSR Ablation</bold></p>
      <p>The ablation study isolates the contribution of EPPSR scoring components. Removing consistency reduced answer accuracy from 100.00% to 91.25%, indicating that agreement with the evidence cohort materially affected answer selection in this benchmark. Removing provenance preserved 100.00% answer accuracy but reduced provenance completeness from 88.60% to 79.36% and citation accuracy from 46.25% to 33.75%, showing that provenance primarily affects evidence quality and source selection. Removing the privacy-risk scoring term preserved answer accuracy but reduced citation accuracy to 26.25%; because release-time redaction remained active, SIER stayed at 0%. Finally, the semantic-plus-graph-only variant fell to 66.25% answer accuracy, matching the utility level of GraphRAG-like scoring and showing that the additional EPPSR signals drive much of the controlled-benchmark gain. <bold>Table 6</bold> reports the complete ablation results, and <xref ref-type="fig" rid="fig11">Figure 11</xref> highlights the effect of each configuration on answer accuracy.</p>
      <p><bold>Table 6</bold><bold>.</bold>Eppsr ablation results (%).</p>
      <table-wrap id="tbl6">
        <label>Table 6</label>
        <table>
          <tbody>
            <tr>
              <td>
                <bold>Variant</bold>
              </td>
              <td>
                <bold>Accuracy</bold>
              </td>
              <td>
                <bold>Citation</bold>
              </td>
              <td>
                <bold>SIER</bold>
              </td>
              <td>
                <bold>Prov.</bold>
              </td>
            </tr>
            <tr>
              <td>EPPSR</td>
              <td>100.00</td>
              <td>46.25</td>
              <td>0.00</td>
              <td>88.60</td>
            </tr>
            <tr>
              <td>no_consistency</td>
              <td>91.25</td>
              <td>40.00</td>
              <td>0.00</td>
              <td>83.88</td>
            </tr>
            <tr>
              <td>no_privacy_risk</td>
              <td>100.00</td>
              <td>26.25</td>
              <td>0.00</td>
              <td>90.44</td>
            </tr>
            <tr>
              <td>no_provenance</td>
              <td>100.00</td>
              <td>33.75</td>
              <td>0.00</td>
              <td>79.36</td>
            </tr>
            <tr>
              <td>semantic_graph_only</td>
              <td>66.25</td>
              <td>8.75</td>
              <td>0.00</td>
              <td>77.04</td>
            </tr>
          </tbody>
        </table>
      </table-wrap>
      <fig id="fig11">
        <label>Figure 11</label>
        <graphic xlink:href="https://html.scirp.org/file/1561969-rId60.jpeg?20261015111834" />
      </fig>
      <p><bold>Figure 11</bold><bold>.</bold> EPPSR ablation: answer accuracy.</p>
      <p><bold>F. Statistical Uncertainty</bold></p>
      <p>Bootstrap uncertainty is reported rather than relying on point estimates alone. For answer accuracy, the 95% bootstrap intervals were 47.50% - 70.00% for RAG, 48.75% - 70.00% for PP-RAG, 56.25% - 76.25% for GraphRAG, and 100.00% - 100.00% for EPPSR. Citation-accuracy intervals were 1.25% - 10.00%, 1.25% - 12.50%, 3.75% - 15.00%, and 35.00% - 56.28%, respectively. SIER intervals were 30.00% - 52.50% for RAG, 0.00% - 0.00% for PP-RAG, 26.25% - 47.50% for GraphRAG, and 0.00% - 0.00% for EPPSR. These intervals make clear that small differences among low-frequency rates should not be overinterpreted.</p>
    </sec>
    <sec id="sec6">
      <title>6. Limitations and Threats to Validity</title>
      <p>The V2 benchmark is synthetic and intentionally constructed to create retrieval, provenance, contradiction, and privacy conflicts. The strong EPPSR result therefore establishes behavior under controlled assumptions, not superiority on independent scientific corpora. Real repositories contain entity-resolution errors, missing or stale metadata, schema drift, duplicated experiments, complex access policies, and domain-specific semantics that are not fully represented here.</p>
      <p>The implementation uses TF-IDF and cosine similarity rather than a neural embedding model and does not execute a production LLM. Consequently, the experiment does not measure model hallucination, generative fluency, prompt-injection robustness, GPU latency, or model-specific faithfulness. The previous simulated hallucination rate and arbitrary composite explanation score have therefore been removed from V2.</p>
      <p>Formal DP is deliberately scoped to the bounded aggregate mean-effect release. Authorization, redaction, graph filtering, and privacy-risk reranking are security/privacy controls but do not independently establish epsilon-DP. SIER captures observed application-layer exposure under the specified release policy; 0% SIER does not imply immunity to membership inference, reconstruction, graph inference, collusion, side-channel attacks, or attacks outside the benchmark threat model.</p>
      <p>Pure sequential composition is conservative but transparent. More advanced accounting mechanisms, subsampling, larger aggregation cohorts, query caching, privacy filters, and workload-aware budget allocation could improve utility for repeated access. These approaches should be evaluated before deployment. In addition, provenance-quality labels are synthetically generated and should be replaced by auditable real-world provenance criteria and expert validation.</p>
      <p>Finally, the 80-query benchmark yields finite-sample uncertainty even with bootstrap reporting. The 100% EPPSR answer-accuracy result is specific to this seeded synthetic design and should be validated through multiple independent seeds, larger benchmark families, external scientific datasets, production embeddings, real knowledge-graph construction pipelines, and blinded domain-expert evaluation.</p>
    </sec>
    <sec id="sec7">
      <title>7. Future Work</title>
      <p>Future work should move beyond the V2 synthetic proof-of-concept by validating EPPSR on real scientific repositories, production embedding models, multiple LLMs, and independently constructed knowledge graphs. Privacy research should extend the present bounded-aggregate Laplace mechanism with advanced composition/accounting, larger cohorts, privacy filters, and empirical attacks including membership inference, reconstruction, graph inference, and prompt injection. Explainability should be validated through blinded expert studies using the component metrics introduced here rather than an unvalidated composite score.</p>
    </sec>
    <sec id="sec8">
      <title>8. Conclusion</title>
      <p>This paper extends prior privacy-preserving LLM integration with scientific NoSQL repositories [<xref ref-type="bibr" rid="B1">1</xref>] by adding knowledge-graph reasoning, evidence ranking, provenance, confidence estimation, and privacy-aware explanation. The resulting EPPSR architecture treats trustworthy scientific retrieval as a multi-objective problem rather than an accuracy-only problem. In the executed synthetic proof-of-concept, GraphRAG achieved the highest raw answer accuracy, whereas EPPSR achieved the strongest combined privacy and explainability profile while retaining competitive accuracy. These findings do not establish production superiority; instead, they provide a reproducible architectural baseline and demonstrate why privacy, evidence grounding, and explainability must be designed together. Real scientific datasets, production LLMs, adversarial privacy testing, and domain-expert evaluation are required for the next stage of validation.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <title>References</title>
      <ref id="B1">
        <label>1.</label>
        <citation-alternatives>
          <mixed-citation publication-type="journal">Biswas, T. (2025) Privacy-Preserving LLM Integration with Scientific NoSQL Repositories: A Differential Privacy Approach. <italic>World Journal of Engineering and Technology</italic>, 13, 329-345. https://doi.org/10.4236/wjet.2025.132021 <pub-id pub-id-type="doi">10.4236/wjet.2025.132021</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.4236/wjet.2025.132021">https://doi.org/10.4236/wjet.2025.132021</ext-link></mixed-citation>
          <element-citation publication-type="journal">
            <person-group person-group-type="author">
              <string-name>Biswas, T.</string-name>
            </person-group>
            <year>2025</year>
            <article-title>Privacy-Preserving LLM Integration with Scientific NoSQL Repositories: A Differential Privacy Approach</article-title>
            <source>World Journal of Engineering and Technology</source>
            <volume>13</volume>
            <pub-id pub-id-type="doi">10.4236/wjet.2025.132021</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B2">
        <label>2.</label>
        <citation-alternatives>
          <mixed-citation publication-type="journal">Vaswani, A., <italic>et al</italic>. (2017) Attention Is All You Need. arXiv: 1706.03762.</mixed-citation>
          <element-citation publication-type="journal">
            <person-group person-group-type="author">
              <string-name>Vaswani, A.</string-name>
            </person-group>
            <year>2017</year>
            <article-title>Attention Is All You Need</article-title>
            <fpage>1706</fpage>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B3">
        <label>3.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Devlin, J., Chang, M., Lee, K. and Toutanova, K. (2019) BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding. <italic>Proceedings of the</italic> 2019 <italic>Confere</italic><italic>nce of the North American Chapter of the Association for Computational Linguistics</italic>: <italic>Human Language Technologies</italic>, <italic>Volume</italic> 1 ( <italic>Long and Short Papers</italic>), Minneapolis, 2-7 June 2019, 4171-4186. https://doi.org/10.18653/v1/n19-1423 <pub-id pub-id-type="doi">10.18653/v1/n19-1423</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.18653/v1/n19-1423">https://doi.org/10.18653/v1/n19-1423</ext-link></mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Devlin, J.</string-name>
              <string-name>Chang, M.</string-name>
              <string-name>Lee, K.</string-name>
              <string-name>Toutanova, K.</string-name>
              <string-name>Technologies, V</string-name>
            </person-group>
            <year>2019</year>
            <article-title>BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding</article-title>
            <source>Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies</source>
            <volume>2</volume>
            <pub-id pub-id-type="doi">10.18653/v1/n19-1423</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B4">
        <label>4.</label>
        <citation-alternatives>
          <mixed-citation publication-type="journal">Biswas, T. (2023) Enhancing R&amp;D Knowledge Management: Integrating Large Language Models with NoSQL Databases for Experiment Documentation Access. <italic>International Journal of Computer Engineering and Technology</italic>, 14, 100-106. https://doi.org/10.34218/ijcet_14_01_011 <pub-id pub-id-type="doi">10.34218/ijcet_14_01_011</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.34218/ijcet_14_01_011">https://doi.org/10.34218/ijcet_14_01_011</ext-link></mixed-citation>
          <element-citation publication-type="journal">
            <person-group person-group-type="author">
              <string-name>Biswas, T.</string-name>
            </person-group>
            <year>2023</year>
            <article-title>Enhancing R&amp;D Knowledge Management: Integrating Large Language Models with NoSQL Databases for Experiment Documentation Access</article-title>
            <source>International Journal of Computer Engineering and Technology</source>
            <volume>14</volume>
            <pub-id pub-id-type="doi">10.34218/ijcet_14_01_011</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B5">
        <label>5.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Lewis, P., <italic>et al</italic>. (2020) Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. 34 <italic>th Conference on Neural Information Processing Systems NeurIPS</italic>2020, Vancouver, 6-12 December 2020, 9459-9474.</mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Lewis, P.</string-name>
            </person-group>
            <year>2020</year>
            <article-title>Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks</article-title>
            <source>34th Conference on Neural Information Processing Systems NeurIPS 2020</source>
            <volume>6</volume>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B6">
        <label>6.</label>
        <citation-alternatives>
          <mixed-citation publication-type="journal">Gao, Y., <italic>et al</italic>. (2023) Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv: 2312.10997.</mixed-citation>
          <element-citation publication-type="journal">
            <person-group person-group-type="author">
              <string-name>Gao, Y.</string-name>
            </person-group>
            <year>2023</year>
            <article-title>Retrieval-Augmented Generation for Large Language Models: A Survey</article-title>
            <fpage>2312</fpage>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B7">
        <label>7.</label>
        <citation-alternatives>
          <mixed-citation publication-type="other">Hogan, A., Blomqvist, E., Cochez, M., D’amato, C., Melo, G.D., Gutierrez, C., <italic>et al</italic>. (2021) Knowledge Graphs. <italic>ACM Computing Surveys</italic>, 54, 1-37. https://doi.org/10.1145/3447772 <pub-id pub-id-type="doi">10.1145/3447772</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1145/3447772">https://doi.org/10.1145/3447772</ext-link></mixed-citation>
          <element-citation publication-type="other">
            <person-group person-group-type="author">
              <string-name>Hogan, A.</string-name>
              <string-name>Blomqvist, E.</string-name>
              <string-name>Cochez, M.</string-name>
              <string-name>Melo, G.D.</string-name>
              <string-name>Gutierrez, C.</string-name>
            </person-group>
            <year>2021</year>
            <article-title>Knowledge Graphs</article-title>
            <source>ACM Computing Surveys</source>
            <volume>54</volume>
            <pub-id pub-id-type="doi">10.1145/3447772</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B8">
        <label>8.</label>
        <citation-alternatives>
          <mixed-citation publication-type="other">Pan, S., Luo, L., Wang, Y., Chen, C., Wang, J. and Wu, X. (2024) Unifying Large Language Models and Knowledge Graphs: A Roadmap. <italic>IEEE Transactions on Knowledge and Data Engineering</italic>, 36, 3580-3599. https://doi.org/10.1109/tkde.2024.3352100 <pub-id pub-id-type="doi">10.1109/tkde.2024.3352100</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1109/tkde.2024.3352100">https://doi.org/10.1109/tkde.2024.3352100</ext-link></mixed-citation>
          <element-citation publication-type="other">
            <person-group person-group-type="author">
              <string-name>Pan, S.</string-name>
              <string-name>Luo, L.</string-name>
              <string-name>Wang, Y.</string-name>
              <string-name>Chen, C.</string-name>
              <string-name>Wang, J.</string-name>
              <string-name>Wu, X.</string-name>
            </person-group>
            <year>2024</year>
            <article-title>Unifying Large Language Models and Knowledge Graphs: A Roadmap</article-title>
            <source>IEEE Transactions on Knowledge and Data Engineering</source>
            <volume>36</volume>
            <pub-id pub-id-type="doi">10.1109/tkde.2024.3352100</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B9">
        <label>9.</label>
        <citation-alternatives>
          <mixed-citation publication-type="other">Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., <italic>et al</italic>. (2020) Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI. <italic>Information Fusion</italic>, 58, 82-115. https://doi.org/10.1016/j.inffus.2019.12.012 <pub-id pub-id-type="doi">10.1016/j.inffus.2019.12.012</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1016/j.inffus.2019.12.012">https://doi.org/10.1016/j.inffus.2019.12.012</ext-link></mixed-citation>
          <element-citation publication-type="other">
            <person-group person-group-type="author">
              <string-name>Arrieta, A.</string-name>
              <string-name>Ser, J.</string-name>
              <string-name>Bennetot, A.</string-name>
              <string-name>Tabik, S.</string-name>
              <string-name>Barbado, A.</string-name>
              <string-name>Concepts, T</string-name>
            </person-group>
            <year>2020</year>
            <article-title>Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI</article-title>
            <source>Information Fusion</source>
            <volume>58</volume>
            <pub-id pub-id-type="doi">10.1016/j.inffus.2019.12.012</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B10">
        <label>10.</label>
        <citation-alternatives>
          <mixed-citation publication-type="report">Radford, A., <italic>et al</italic>. (2019) Language Models Are Unsupervised Multitask Learners. OpenAI Technical Report9.</mixed-citation>
          <element-citation publication-type="report">
            <person-group person-group-type="author">
              <string-name>Radford, A.</string-name>
            </person-group>
            <year>2019</year>
            <article-title>Language Models Are Unsupervised Multitask Learners</article-title>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B11">
        <label>11.</label>
        <citation-alternatives>
          <mixed-citation publication-type="journal">Raffel, C., <italic>et al</italic>. (2020) Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. <italic>Journal of Machine Learning Research</italic>, 21, 1-67.</mixed-citation>
          <element-citation publication-type="journal">
            <person-group person-group-type="author">
              <string-name>Raffel, C.</string-name>
            </person-group>
            <year>2020</year>
            <article-title>Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer</article-title>
            <source>Journal of Machine Learning Research</source>
            <volume>21</volume>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B12">
        <label>12.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Brown, T., <italic>et al</italic>. (2020) Language Models Are Few-Shot Learners. <italic>Proceedings of the</italic>34 <italic>th International Conference on Neural Information Processing System</italic>, Vancouver, 6-12 December 2020, 1877-1901.</mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Brown, T.</string-name>
              <string-name>System, V</string-name>
            </person-group>
            <year>2020</year>
            <article-title>Language Models Are Few-Shot Learners</article-title>
            <source>Proceedings of the 34th International Conference on Neural Information Processing System</source>
            <volume>6</volume>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B13">
        <label>13.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., <italic>et al</italic>. (2020) Dense Passage Retrieval for Open-Domain Question Answering. <italic>Proceedings of the</italic> 2020 <italic>Conference on Empirical Methods in Natural Language Processing</italic> ( <italic>EMNLP</italic>), 16-20 November 2020, 6769-6781. https://doi.org/10.18653/v1/2020.emnlp-main.550 <pub-id pub-id-type="doi">10.18653/v1/2020.emnlp-main.550</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.18653/v1/2020.emnlp-main.550">https://doi.org/10.18653/v1/2020.emnlp-main.550</ext-link></mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Karpukhin, V.</string-name>
              <string-name>Oguz, B.</string-name>
              <string-name>Min, S.</string-name>
              <string-name>Lewis, P.</string-name>
              <string-name>Wu, L.</string-name>
              <string-name>Edunov, S.</string-name>
            </person-group>
            <year>2020</year>
            <article-title>Dense Passage Retrieval for Open-Domain Question Answering</article-title>
            <source>Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)</source>
            <volume>16</volume>
            <pub-id pub-id-type="doi">10.18653/v1/2020.emnlp-main.550</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B14">
        <label>14.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Khattab, O. and Zaharia, M. (2020) ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. <italic>Proceedings of the</italic> 43 <italic>rd International ACM SIGIR Conference on Research and Development in Information Retrieval</italic>, 25-30 July 2020, 39-48. https://doi.org/10.1145/3397271.3401075 <pub-id pub-id-type="doi">10.1145/3397271.3401075</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1145/3397271.3401075">https://doi.org/10.1145/3397271.3401075</ext-link></mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Khattab, O.</string-name>
              <string-name>Zaharia, M.</string-name>
            </person-group>
            <year>2020</year>
            <article-title>ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT</article-title>
            <source>Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval</source>
            <volume>25</volume>
            <pub-id pub-id-type="doi">10.1145/3397271.3401075</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B15">
        <label>15.</label>
        <citation-alternatives>
          <mixed-citation publication-type="other">Thakur, N., <italic>et al</italic>. (202021) BEIR: A Heterogeneous Benchmark for Zero-Shot Evaluation of Information Retrieval Models. arXiv: 2104.08663.</mixed-citation>
          <element-citation publication-type="other">
            <person-group person-group-type="author">
              <string-name>Thakur, N.</string-name>
            </person-group>
            <fpage>2104</fpage>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B16">
        <label>16.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Izacard, G. and Grave, E. (2021) Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering. <italic>Proceedings of the</italic> 16 <italic>th Conference of the European Chapter of the Association for Computational Linguistics</italic>: <italic>Main Volume</italic>, 19-23 April 2021, 874-880. https://doi.org/10.18653/v1/2021.eacl-main.74 <pub-id pub-id-type="doi">10.18653/v1/2021.eacl-main.74</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.18653/v1/2021.eacl-main.74">https://doi.org/10.18653/v1/2021.eacl-main.74</ext-link></mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Izacard, G.</string-name>
              <string-name>Grave, E.</string-name>
            </person-group>
            <year>2021</year>
            <article-title>Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering</article-title>
            <source>Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume</source>
            <volume>19</volume>
            <pub-id pub-id-type="doi">10.18653/v1/2021.eacl-main.74</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B17">
        <label>17.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Guu, K., Lee, K., Tung, Z., Pasupat, P. and Chang, M. (2020) REALM: Retrieval-Augmented Language Model Pre-Training. <italic>Proceedings of the</italic> 37 <italic>th International Conference on Machine Learning</italic>, 13-18 July 2020, 3929-3938.</mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Guu, K.</string-name>
              <string-name>Lee, K.</string-name>
              <string-name>Tung, Z.</string-name>
              <string-name>Pasupat, P.</string-name>
              <string-name>Chang, M.</string-name>
            </person-group>
            <year>2020</year>
            <article-title>REALM: Retrieval-Augmented Language Model Pre-Training</article-title>
            <source>Proceedings of the 37th International Conference on Machine Learning</source>
            <volume>13</volume>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B18">
        <label>18.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Yates, A., Nogueira, R. and Lin, J. (2021) Pretrained Transformers for Text Ranking: BERT and beyond. <italic>Proceedings of the</italic>2021 <italic>Conference of the North American Chapter of the Association for Computational Linguistics</italic>: <italic>Human Language Technologies</italic>: <italic>Tutorials</italic>, 6-11 June 2021, 1-4. https://doi.org/10.18653/v1/2021.naacl-tutorials.1 <pub-id pub-id-type="doi">10.18653/v1/2021.naacl-tutorials.1</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.18653/v1/2021.naacl-tutorials.1">https://doi.org/10.18653/v1/2021.naacl-tutorials.1</ext-link></mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Yates, A.</string-name>
              <string-name>Nogueira, R.</string-name>
              <string-name>Lin, J.</string-name>
            </person-group>
            <year>2021</year>
            <article-title>Pretrained Transformers for Text Ranking: BERT and beyond</article-title>
            <source>Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Tutorials</source>
            <volume>6</volume>
            <pub-id pub-id-type="doi">10.18653/v1/2021.naacl-tutorials.1</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B19">
        <label>19.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Maynez, J., Narayan, S., Bohnet, B. and McDonald, R. (2020) On Faithfulness and Factuality in Abstractive Summarization. <italic>Proceedings of the</italic> 58 <italic>th Annual Meeting of the Association for Computational Linguistics</italic>, 5-10 July 2020, 1906-1919. https://doi.org/10.18653/v1/2020.acl-main.173 <pub-id pub-id-type="doi">10.18653/v1/2020.acl-main.173</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.18653/v1/2020.acl-main.173">https://doi.org/10.18653/v1/2020.acl-main.173</ext-link></mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Maynez, J.</string-name>
              <string-name>Narayan, S.</string-name>
              <string-name>Bohnet, B.</string-name>
              <string-name>McDonald, R.</string-name>
            </person-group>
            <year>2020</year>
            <article-title>On Faithfulness and Factuality in Abstractive Summarization</article-title>
            <source>Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics</source>
            <volume>5</volume>
            <pub-id pub-id-type="doi">10.18653/v1/2020.acl-main.173</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B20">
        <label>20.</label>
        <citation-alternatives>
          <mixed-citation publication-type="other">Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., <italic>et al</italic>. (2022) Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. <italic>Advances in Neural Information Processing Systems</italic>35, New Orleans, 28 November-9 December 2022, 24824-24837. https://doi.org/10.52202/068431-1800 <pub-id pub-id-type="doi">10.52202/068431-1800</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.52202/068431-1800">https://doi.org/10.52202/068431-1800</ext-link></mixed-citation>
          <element-citation publication-type="other">
            <person-group person-group-type="author">
              <string-name>Wei, J.</string-name>
              <string-name>Wang, X.</string-name>
              <string-name>Schuurmans, D.</string-name>
              <string-name>Bosma, M.</string-name>
              <string-name>Ichter, B.</string-name>
              <string-name>Xia, F.</string-name>
            </person-group>
            <year>2022</year>
            <article-title>Chain-of-Thought Prompting Elicits Reasoning in Large Language Models</article-title>
            <source>Advances in Neural Information Processing Systems 35</source>
            <volume>28</volume>
            <pub-id pub-id-type="doi">10.52202/068431-1800</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B21">
        <label>21.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Lin, S., Hilton, J. and Evans, O. (2022) TruthfulQA: Measuring How Models Mimic Human Falsehoods. <italic>Proceedings of the</italic> 60 <italic>th Annual Meeting of the Association for Computational Linguistics</italic> ( <italic>Volume</italic>1: <italic>Long Papers</italic>), Dublin, 22-27 May 2022, 3214-3252. https://doi.org/10.18653/v1/2022.acl-long.229 <pub-id pub-id-type="doi">10.18653/v1/2022.acl-long.229</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.18653/v1/2022.acl-long.229">https://doi.org/10.18653/v1/2022.acl-long.229</ext-link></mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Lin, S.</string-name>
              <string-name>Hilton, J.</string-name>
              <string-name>Evans, O.</string-name>
            </person-group>
            <year>2022</year>
            <article-title>TruthfulQA: Measuring How Models Mimic Human Falsehoods</article-title>
            <source>Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</source>
            <volume>22</volume>
            <pub-id pub-id-type="doi">10.18653/v1/2022.acl-long.229</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B22">
        <label>22.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Rajpurkar, P., Zhang, J., Lopyrev, K. and Liang, P. (2016) SQuAD: 100,000+ Questions for Machine Comprehension of Text. <italic>Proceedings of the</italic> 2016 <italic>Conference on Empirical Methods in Natural Language Processing</italic>, Austin, 1-5 November 2016, 2383-2392. https://doi.org/10.18653/v1/d16-1264 <pub-id pub-id-type="doi">10.18653/v1/d16-1264</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.18653/v1/d16-1264">https://doi.org/10.18653/v1/d16-1264</ext-link></mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Rajpurkar, P.</string-name>
              <string-name>Zhang, J.</string-name>
              <string-name>Lopyrev, K.</string-name>
              <string-name>Liang, P.</string-name>
              <string-name>Processing, A</string-name>
            </person-group>
            <year>2016</year>
            <article-title>SQuAD: 100,000+ Questions for Machine Comprehension of Text</article-title>
            <source>Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing</source>
            <volume>1</volume>
            <fpage>100</fpage>
            <pub-id pub-id-type="doi">10.18653/v1/d16-1264</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B23">
        <label>23.</label>
        <citation-alternatives>
          <mixed-citation publication-type="other">Zhong, L., Wu, J., Li, Q., Peng, H. and Wu, X. (2023) A Comprehensive Survey on Automatic Knowledge Graph Construction. <italic>ACM Computing Surveys</italic>, 56, 1-62. https://doi.org/10.1145/3618295 <pub-id pub-id-type="doi">10.1145/3618295</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1145/3618295">https://doi.org/10.1145/3618295</ext-link></mixed-citation>
          <element-citation publication-type="other">
            <person-group person-group-type="author">
              <string-name>Zhong, L.</string-name>
              <string-name>Wu, J.</string-name>
              <string-name>Li, Q.</string-name>
              <string-name>Peng, H.</string-name>
              <string-name>Wu, X.</string-name>
            </person-group>
            <year>2023</year>
            <article-title>A Comprehensive Survey on Automatic Knowledge Graph Construction</article-title>
            <source>ACM Computing Surveys</source>
            <volume>56</volume>
            <pub-id pub-id-type="doi">10.1145/3618295</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B24">
        <label>24.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Socher, R., Chen, D.Q., Manning, C.D. and Ng, A.Y. (2013) Reasoning with Neural Tensor Networks for Knowledge Base Completion. <italic>Proceedings of the</italic>27 <italic>th International Conference on Neural Information Processing Systems</italic>, Nevada, 5-10 December 2013, 926-934.</mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Socher, R.</string-name>
              <string-name>Chen, D.Q.</string-name>
              <string-name>Manning, C.D.</string-name>
              <string-name>Ng, A.Y.</string-name>
              <string-name>Systems, N</string-name>
            </person-group>
            <year>2013</year>
            <article-title>Reasoning with Neural Tensor Networks for Knowledge Base Completion</article-title>
            <source>Proceedings of the 27th International Conference on Neural Information Processing Systems</source>
            <volume>5</volume>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B25">
        <label>25.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J. and Yakhnenko, O. (2013) Translating Embeddings for Modeling Multi-Relational Data. <italic>Proceedings of the</italic>27 <italic>th International Conference on Neural Information Processing Systems</italic>, Nevada, 5-10 December 2013, 2787-2795.</mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Bordes, A.</string-name>
              <string-name>Usunier, N.</string-name>
              <string-name>Garcia-Duran, A.</string-name>
              <string-name>Weston, J.</string-name>
              <string-name>Yakhnenko, O.</string-name>
              <string-name>Systems, N</string-name>
            </person-group>
            <year>2013</year>
            <article-title>Translating Embeddings for Modeling Multi-Relational Data</article-title>
            <source>Proceedings of the 27th International Conference on Neural Information Processing Systems</source>
            <volume>5</volume>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B26">
        <label>26.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Nickel, M., Murphy, K., Tresp, V. and Gabrilovich, E. (2016) A Review of Relational Machine Learning for Knowledge Graphs. <italic>Proceedings of the IEEE</italic>, 104, 11-33. https://doi.org/10.1109/jproc.2015.2483592 <pub-id pub-id-type="doi">10.1109/jproc.2015.2483592</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1109/jproc.2015.2483592">https://doi.org/10.1109/jproc.2015.2483592</ext-link></mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Nickel, M.</string-name>
              <string-name>Murphy, K.</string-name>
              <string-name>Tresp, V.</string-name>
              <string-name>Gabrilovich, E.</string-name>
            </person-group>
            <year>2016</year>
            <article-title>A Review of Relational Machine Learning for Knowledge Graphs</article-title>
            <source>Proceedings of the IEEE</source>
            <volume>104</volume>
            <pub-id pub-id-type="doi">10.1109/jproc.2015.2483592</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B27">
        <label>27.</label>
        <citation-alternatives>
          <mixed-citation publication-type="other">Wang, Q., Mao, Z., Wang, B. and Guo, L. (2017) Knowledge Graph Embedding: A Survey of Approaches and Applications. <italic>IEEE Transactions on Knowledge and Data Engineering</italic>, 29, 2724-2743. https://doi.org/10.1109/tkde.2017.2754499 <pub-id pub-id-type="doi">10.1109/tkde.2017.2754499</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1109/tkde.2017.2754499">https://doi.org/10.1109/tkde.2017.2754499</ext-link></mixed-citation>
          <element-citation publication-type="other">
            <person-group person-group-type="author">
              <string-name>Wang, Q.</string-name>
              <string-name>Mao, Z.</string-name>
              <string-name>Wang, B.</string-name>
              <string-name>Guo, L.</string-name>
            </person-group>
            <year>2017</year>
            <article-title>Knowledge Graph Embedding: A Survey of Approaches and Applications</article-title>
            <source>IEEE Transactions on Knowledge and Data Engineering</source>
            <volume>29</volume>
            <pub-id pub-id-type="doi">10.1109/tkde.2017.2754499</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B28">
        <label>28.</label>
        <citation-alternatives>
          <mixed-citation publication-type="other">Singhal, A. (2012) Introducing the Knowledge Graph: Things, Not Strings. Google.</mixed-citation>
          <element-citation publication-type="other">
            <person-group person-group-type="author">
              <string-name>Singhal, A.</string-name>
              <string-name>Things, N</string-name>
            </person-group>
            <year>2012</year>
            <article-title>Introducing the Knowledge Graph: Things, Not Strings</article-title>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B29">
        <label>29.</label>
        <citation-alternatives>
          <mixed-citation publication-type="other">Dwork, C. (2006) Differential Privacy. In: Bugliesi, M., Preneel, B., Sassone, V. and Wegener, I., Eds., <italic>Automata</italic>, <italic>Languages and Programming</italic>, Springer, 1-12. https://doi.org/10.1007/11787006_1 <pub-id pub-id-type="doi">10.1007/11787006_1</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1007/11787006_1">https://doi.org/10.1007/11787006_1</ext-link></mixed-citation>
          <element-citation publication-type="other">
            <person-group person-group-type="author">
              <string-name>Dwork, C.</string-name>
              <string-name>Bugliesi, M.</string-name>
              <string-name>Preneel, B.</string-name>
              <string-name>Sassone, V.</string-name>
              <string-name>Wegener, I.</string-name>
              <string-name>Automata, L</string-name>
              <string-name>Programming, S</string-name>
            </person-group>
            <year>2006</year>
            <article-title>Differential Privacy</article-title>
            <source>In: Bugliesi</source>
            <volume>1</volume>
            <pub-id pub-id-type="doi">10.1007/11787006_1</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B30">
        <label>30.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Abadi, M., Chu, A., Goodfellow, I., McMahan, H.B., Mironov, I., Talwar, K., <italic>et al</italic>. (2016) Deep Learning with Differential Privacy. <italic>Proceedings of the</italic> 2016 <italic>ACM SIGSAC Conference on Computer and Communications Security</italic>, Vienna, 24-28 October 2016, 308-318. https://doi.org/10.1145/2976749.2978318 <pub-id pub-id-type="doi">10.1145/2976749.2978318</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1145/2976749.2978318">https://doi.org/10.1145/2976749.2978318</ext-link></mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Abadi, M.</string-name>
              <string-name>Chu, A.</string-name>
              <string-name>Goodfellow, I.</string-name>
              <string-name>McMahan, H.B.</string-name>
              <string-name>Mironov, I.</string-name>
              <string-name>Talwar, K.</string-name>
              <string-name>Security, V</string-name>
            </person-group>
            <year>2016</year>
            <article-title>Deep Learning with Differential Privacy</article-title>
            <source>Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security</source>
            <volume>24</volume>
            <pub-id pub-id-type="doi">10.1145/2976749.2978318</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B31">
        <label>31.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Carlini, N., <italic>et al</italic>. (2021) Extracting Training Data from Large Language Models. <italic>Pro</italic><italic>c</italic><italic>eeding of</italic><italic>USENIX Security Symposium</italic>2021, 11-13 August 2021, 2633-2650.</mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Carlini, N.</string-name>
            </person-group>
            <year>2021</year>
            <article-title>Extracting Training Data from Large Language Models</article-title>
            <source>Proceeding of USENIX Security Symposium 2021</source>
            <volume>11</volume>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B32">
        <label>32.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Fredrikson, M., Jha, S. and Ristenpart, T. (2015) Model Inversion Attacks That Exploit Confidence Information and Basic Countermeasures. <italic>Proceedings of the</italic> 22 <italic>nd ACM SIGSAC Conference on Computer and Communications Security</italic>, Denver, 12-16 October 2015, 1322-1333. https://doi.org/10.1145/2810103.2813677 <pub-id pub-id-type="doi">10.1145/2810103.2813677</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1145/2810103.2813677">https://doi.org/10.1145/2810103.2813677</ext-link></mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Fredrikson, M.</string-name>
              <string-name>Jha, S.</string-name>
              <string-name>Ristenpart, T.</string-name>
              <string-name>Security, D</string-name>
            </person-group>
            <year>2015</year>
            <article-title>Model Inversion Attacks That Exploit Confidence Information and Basic Countermeasures</article-title>
            <source>Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security</source>
            <volume>12</volume>
            <pub-id pub-id-type="doi">10.1145/2810103.2813677</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B33">
        <label>33.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Shokri, R., Stronati, M., Song, C. and Shmatikov, V. (2017) Membership Inference Attacks against Machine Learning Models. 2017 <italic>IEEE Symposium on Security and Privacy</italic> ( <italic>SP</italic>), San Jose, 22-26 May 2017, 3-18. https://doi.org/10.1109/sp.2017.41 <pub-id pub-id-type="doi">10.1109/sp.2017.41</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1109/sp.2017.41">https://doi.org/10.1109/sp.2017.41</ext-link></mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Shokri, R.</string-name>
              <string-name>Stronati, M.</string-name>
              <string-name>Song, C.</string-name>
              <string-name>Shmatikov, V.</string-name>
            </person-group>
            <year>2017</year>
            <article-title>Membership Inference Attacks against Machine Learning Models</article-title>
            <source>2017 IEEE Symposium on Security and Privacy (SP)</source>
            <volume>22</volume>
            <pub-id pub-id-type="doi">10.1109/sp.2017.41</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B34">
        <label>34.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Nasr, M., Shokri, R. and Houmansadr, A. (2019) Comprehensive Privacy Analysis of Deep Learning. <italic>Proc</italic><italic>eeding of</italic><italic>IEEE Symposium on Security and Privacy</italic> 2019, San Francisco, 20-22 May 2019, 1-15.</mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Nasr, M.</string-name>
              <string-name>Shokri, R.</string-name>
              <string-name>Houmansadr, A.</string-name>
            </person-group>
            <year>2019</year>
            <article-title>Comprehensive Privacy Analysis of Deep Learning</article-title>
            <source>Proceeding of IEEE Symposium on Security and Privacy 2019</source>
            <volume>20</volume>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B35">
        <label>35.</label>
        <citation-alternatives>
          <mixed-citation publication-type="journal">Papernot, N., Abadi, M., Erlingsson, Ú., Goodfellow, I. and Talwar, K. (2017) Semi-Supervised Knowledge Transfer for Deep Learning from Private Training Data. arXiv: 1610.05755.</mixed-citation>
          <element-citation publication-type="journal">
            <person-group person-group-type="author">
              <string-name>Papernot, N.</string-name>
              <string-name>Abadi, M.</string-name>
              <string-name>Goodfellow, I.</string-name>
              <string-name>Talwar, K.</string-name>
            </person-group>
            <year>2017</year>
            <article-title>Semi-Supervised Knowledge Transfer for Deep Learning from Private Training Data</article-title>
            <fpage>1610</fpage>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B36">
        <label>36.</label>
        <citation-alternatives>
          <mixed-citation publication-type="other">Li, T., Sahu, A.K., Talwalkar, A. and Smith, V. (2020) Federated Learning: Challenges, Methods, and Future Directions. <italic>IEEE Signal Processing Magazine</italic>, 37, 50-60. https://doi.org/10.1109/msp.2020.2975749 <pub-id pub-id-type="doi">10.1109/msp.2020.2975749</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1109/msp.2020.2975749">https://doi.org/10.1109/msp.2020.2975749</ext-link></mixed-citation>
          <element-citation publication-type="other">
            <person-group person-group-type="author">
              <string-name>Li, T.</string-name>
              <string-name>Sahu, A.K.</string-name>
              <string-name>Talwalkar, A.</string-name>
              <string-name>Smith, V.</string-name>
              <string-name>Challenges, M</string-name>
            </person-group>
            <year>2020</year>
            <article-title>Federated Learning: Challenges, Methods, and Future Directions</article-title>
            <source>IEEE Signal Processing Magazine</source>
            <volume>37</volume>
            <pub-id pub-id-type="doi">10.1109/msp.2020.2975749</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B37">
        <label>37.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">McMahan, H.B., Moore, E., Ramage, D., Hampson, S. and y Arcas, B.A. (2017) Communication-Efficient Learning of Deep Networks from Decentralized Data. <italic>Proceedings of the</italic> 20 <italic>th International Conference on Artificial Intelligence and Statistics</italic>( <italic>AISTATS</italic>) 2017, Fort Lauderdale, 20-22 April 2017, 1273-1282.</mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>McMahan, H.B.</string-name>
              <string-name>Moore, E.</string-name>
              <string-name>Ramage, D.</string-name>
              <string-name>Hampson, S.</string-name>
              <string-name>Arcas, B.A.</string-name>
            </person-group>
            <year>2017</year>
            <article-title>Communication-Efficient Learning of Deep Networks from Decentralized Data</article-title>
            <source>Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS) 2017</source>
            <volume>20</volume>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B38">
        <label>38.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Ribeiro, M.T., Singh, S. and Guestrin, C. (2016) “Why Should I Trust You?” Explaining the Predictions of Any Classifier. <italic>Proceedings of the</italic> 22 <italic>nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining</italic>, San Francisco, 13-17 August 2016, 1135-1144. https://doi.org/10.1145/2939672.2939778 <pub-id pub-id-type="doi">10.1145/2939672.2939778</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1145/2939672.2939778">https://doi.org/10.1145/2939672.2939778</ext-link></mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Ribeiro, M.T.</string-name>
              <string-name>Singh, S.</string-name>
              <string-name>Guestrin, C.</string-name>
              <string-name>Mining, S</string-name>
            </person-group>
            <year>2016</year>
            <article-title>“Why Should I Trust You?” Explaining the Predictions of Any Classifier</article-title>
            <source>Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining</source>
            <volume>13</volume>
            <pub-id pub-id-type="doi">10.1145/2939672.2939778</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B39">
        <label>39.</label>
        <citation-alternatives>
          <mixed-citation publication-type="confproc">Lundberg, S.M. and Lee, S.I. (2017) A Unified Approach to Interpreting Model Predictions. <italic>Proceedings of the</italic> 31 <italic>st International Conference on Neural Information Processing System</italic>, Long Beach, 4-9 December 2017, 4768-4777.</mixed-citation>
          <element-citation publication-type="confproc">
            <person-group person-group-type="author">
              <string-name>Lundberg, S.M.</string-name>
              <string-name>Lee, S.I.</string-name>
              <string-name>System, L</string-name>
            </person-group>
            <year>2017</year>
            <article-title>A Unified Approach to Interpreting Model Predictions</article-title>
            <source>Proceedings of the 31st International Conference on Neural Information Processing System</source>
            <volume>4</volume>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B40">
        <label>40.</label>
        <citation-alternatives>
          <mixed-citation publication-type="other">Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F. and Pedreschi, D. (2018) A Survey of Methods for Explaining Black Box Models. <italic>ACM Computing Surveys</italic>, 51, 1-42. https://doi.org/10.1145/3236009 <pub-id pub-id-type="doi">10.1145/3236009</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1145/3236009">https://doi.org/10.1145/3236009</ext-link></mixed-citation>
          <element-citation publication-type="other">
            <person-group person-group-type="author">
              <string-name>Guidotti, R.</string-name>
              <string-name>Monreale, A.</string-name>
              <string-name>Ruggieri, S.</string-name>
              <string-name>Turini, F.</string-name>
              <string-name>Giannotti, F.</string-name>
              <string-name>Pedreschi, D.</string-name>
            </person-group>
            <year>2018</year>
            <article-title>A Survey of Methods for Explaining Black Box Models</article-title>
            <source>ACM Computing Surveys</source>
            <volume>51</volume>
            <pub-id pub-id-type="doi">10.1145/3236009</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B41">
        <label>41.</label>
        <citation-alternatives>
          <mixed-citation publication-type="journal">Doshi-Velez, F. and Kim, B. (2017) Towards a Rigorous Science of Interpretable Machine Learning. arXiv: 1702.08608.</mixed-citation>
          <element-citation publication-type="journal">
            <person-group person-group-type="author">
              <string-name>Doshi-Velez, F.</string-name>
              <string-name>Kim, B.</string-name>
            </person-group>
            <year>2017</year>
            <article-title>Towards a Rigorous Science of Interpretable Machine Learning</article-title>
            <fpage>1702</fpage>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B42">
        <label>42.</label>
        <citation-alternatives>
          <mixed-citation publication-type="other">Lipton, Z.C. (2018) The Mythos of Model Interpretability. <italic>Queue</italic>, 16, 31-57. https://doi.org/10.1145/3236386.3241340 <pub-id pub-id-type="doi">10.1145/3236386.3241340</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1145/3236386.3241340">https://doi.org/10.1145/3236386.3241340</ext-link></mixed-citation>
          <element-citation publication-type="other">
            <person-group person-group-type="author">
              <string-name>Lipton, Z.C.</string-name>
            </person-group>
            <year>2018</year>
            <article-title>The Mythos of Model Interpretability</article-title>
            <source>Queue</source>
            <volume>16</volume>
            <pub-id pub-id-type="doi">10.1145/3236386.3241340</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
      <ref id="B43">
        <label>43.</label>
        <citation-alternatives>
          <mixed-citation publication-type="journal">Wachter, S., Mittelstadt, B. and Russell, C. (2017) Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR. <italic>SSRN Electronic Journal</italic>, 31. https://doi.org/10.2139/ssrn.3063289 <pub-id pub-id-type="doi">10.2139/ssrn.3063289</pub-id><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.2139/ssrn.3063289">https://doi.org/10.2139/ssrn.3063289</ext-link></mixed-citation>
          <element-citation publication-type="journal">
            <person-group person-group-type="author">
              <string-name>Wachter, S.</string-name>
              <string-name>Mittelstadt, B.</string-name>
              <string-name>Russell, C.</string-name>
            </person-group>
            <year>2017</year>
            <article-title>Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR</article-title>
            <source>SSRN Electronic Journal</source>
            <volume>31</volume>
            <pub-id pub-id-type="doi">10.2139/ssrn.3063289</pub-id>
          </element-citation>
        </citation-alternatives>
      </ref>
    </ref-list>
  </back>
</article>