CounterPsych framework:The first end-to-end counterfactual explainability framework designed and validated specifically for psychiatric treatment decisions, combining a Bayesian outcome predictor with a clinically constrained counterfactual generator and a multi-criteria validity filter.Clinical validity constraints:A formal specification of four psychiatric-domain counterfactual validity criteria, clinical plausibility, medical actionability, causal consistency, and patient-preference alignment implemented as hard constraints in the optimization objective, ensuring generated counterfactuals are clinically meaningful by construction.Proximity-constrained gradient search:A novel counterfactual generation algorithm combining gradient-based search in the outcome predictor’s latent space with proximity regularization and diversity promotion, producing sparse counterfactuals with a mean of 2.3 feature changes and clinician plausibility ratings of 4.42/5.Bayesian outcome predictor:A ten-member deep ensemble with MC-Dropout achieving treatment outcome prediction accuracy of 94.1%, AUC-ROC of 0.977, macro-F1 of 0.919, and ECE of 0.021, providing the calibrated probabilistic foundation on which counterfactual generation depends.Prospective clinician evaluation:A structured evaluation with 24 consultant psychiatrists rating CounterPsych counterfactuals on five clinical quality dimensions, achieving the highest published clinician plausibility and trustworthiness ratings for any psychiatric AI explanation method.Open clinical cohort:A retrospective-prospective dataset of 2,480 psychiatric outpatients across five diagnostic categories and four clinical sites, with treatment outcome labels adjudicated by consensus at 12-week follow-up, constituting the largest clinically labelled dataset for psychiatric treatment response prediction with counterfactual annotations.

The counterfactual explanation framework was formalized in the context of algorithmic accountability by Wachter and colleagues, who proposed that individuals affected by automated decisions have a right to be told what minimal change to their situation would have produced a different outcome. Formally, given a classifier f: X → Y and a factual instance x with predicted class y = f(x), a counterfactual explanation x’ satisfies: 1) f(x’) = y’ ≠ y (prediction flip); 2) ‖x’ − x‖_p is minimized (proximity fewest, smallest changes); and 3) x’ ∈ D (data manifold the counterfactual should look like a plausible real patient). The foundational DiCE framework [10] extended this to generate multiple diverse counterfactuals simultaneously, improving clinical utility by showing several alternative recourse paths rather than one.

The concept of algorithmic recourse extends counterfactual explanations to the specific question of actionability: not just ‘what would need to be different’ but ‘what can realistically be done to produce a different outcome’ [11]. This distinction is critical in clinical settings where many features in a patient’s record age, genetic background, illness history cannot be changed, and where the clinical value of a counterfactual depends entirely on whether the suggested changes are within the clinician’s and patient’s power to implement. A comprehensive review of counterfactual explanation methods [12] identifies five dimensions of counterfactual quality: correctness (the prediction flips), proximity (minimum feature change), sparsity (few features changed), plausibility (the counterfactual resembles real instances), and actionability (the changes are implementable).

Prior methods have addressed these dimensions partially and in isolation. Actionable recourse frameworks [13] enforce immutability constraints (features that cannot change) but do not address causal consistency between mutable features. Prototype-based counterfactual generation [14] improves plausibility by anchoring counterfactuals to real training instances but sacrifices proximity and actionability in feature-rich clinical datasets. FACE [15] generates feasible counterfactuals by constraining search to high-density regions of the data manifold, but its density estimation scales poorly to the mixed continuous-categorical feature spaces typical of electronic health record data.

The causal consistency requirement is particularly challenging and particularly important in psychiatry. Clinical variables do not change independently: reducing a depression score presupposes an intervention that produces that reduction; reducing a medication dose changes biomarker levels and side effect profiles; increasing therapy frequency affects engagement metrics. Counterfactuals that ignore these causal dependencies are not just implausible they are incoherent. The causally constrained counterfactual framework [16] provides the formal machinery for this constraint, and CounterPsych implements a psychiatric-domain instantiation of this framework that has not previously been attempted.

Machine learning has been applied to psychiatric clinical prediction with increasing sophistication over the past decade. Dwyer and colleagues [17] provided an influential overview of ML applications across diagnostic classification, treatment response prediction, and relapse risk estimation, identifying the transition from research to clinical deployment as the major unresolved challenge. A systematic scoping review of ML in mental health [18] catalogued 28 prediction tasks across 15 psychiatric conditions, finding that treatment response prediction and medication selection are the two tasks where ML accuracy most clearly exceeds clinical heuristics and precisely where interpretability is most urgently needed because the decisions are both high-stakes and individually variable.

The relationship between predictive accuracy and interpretability has been the subject of sustained debate. A systematic review of ML versus logistic regression for clinical prediction [19] found that complex models rarely outperform logistic regression on clinical datasets of modest size, suggesting that interpretability may be achievable without sacrificing predictive accuracy in many psychiatric applications. The dominant current approaches to ML interpretability in clinical settings SHAP (SHapley Additive exPlanations) [20] and LIME (Local Interpretable Model-agnostic Explanations) [21] provide feature importance scores that answer the question ‘which features drove this prediction?’ but do not answer the clinically more useful question ‘what would need to change for the prediction to be different?’ This is precisely the gap that counterfactual explanation addresses.

Treatment outcome in psychiatry is typically operationalized through validated rating scales: the Hamilton Depression Rating Scale (HDRS-17) [22] for depressive outcomes, the Young Mania Rating Scale (YMRS) [23] for manic outcomes, and the Global Assessment of Functioning (GAF) [24] for broad functional outcomes. Response is conventionally defined as a ≥ 50% reduction in the primary symptom scale score from baseline to endpoint. Remission requires score reduction to below a clinical threshold (HDRS-17 ≤ 7 for remission in depression). The International Society for Bipolar Disorders has published standardized nomenclature for course and outcome that CounterPsych adopts as its labelling framework [25].

Prior ML systems for treatment outcome prediction have demonstrated that baseline symptom severity, illness duration, number of prior episodes, and medication adherence are reliably predictive of treatment response across psychiatric conditions. However, these models have been deployed as black-box score generators without any mechanism for translating their predictions into clinical guidance. CounterPsych is the first system to close this gap providing not just a response probability but a set of clinically constrained, actionable interventions that the model predicts would change the outcome.

The CounterPsych dataset was assembled through a retrospective-prospective observational design across four outpatient psychiatric centers in Italy, Germany, Switzerland, and the United Kingdom (Primary ethics approval: IRB Ref. UNIGE-2024-CPX-01; GDPR Article 9 compliance). Retrospective records spanned January 2016 to December 2022; prospective enrolment ran from January 2023 to December 2024. Inclusion criteria: adults aged 18 - 70 with confirmed DSM-5 diagnosis in one of five categories (major depressive disorder, bipolar disorder, schizophrenia spectrum, obsessive-compulsive disorder, generalised anxiety disorder); initiation or modification of a pharmacological treatment regimen at the index visit; minimum 12-week follow-up with at least one post-treatment clinical assessment; and capacity to provide informed consent. Exclusion criteria: active substance use disorder with ongoing intoxication; neurological comorbidity affecting cognition; and concurrent enrolment in a conflicting clinical trial. The demographic, diagnostic, and treatment-related characteristics of the CounterPsych cohort are summarized in Table 1.

Table 1. CounterPsych cohort demographic, diagnostic, and treatment characteristics.

The 112-dimensional feature vector for each patient was constructed from five domains. Clinical state (32 features): HDRS-17 total and subscale scores [26], YMRS total score, GAF global score, Clinical Global Impression (CGI) severity and improvement, self-reported energy, sleep, and appetite ratings, and functional impairment across occupational, social, and self-care domains. Treatment profile (24 features): current medications with dose equivalents, medication adherence rate, number of prior medication trials, duration of current regimen, psychotherapy type and frequency, and cumulative anticholinergic burden. Patient history (20 features): illness duration, number of prior episodes, number of prior hospitalizations, prior treatment response history (coded as a binary response vector over previous medication trials), and prior ECT or psychotherapy history. Biomarkers (16 features): serum drug levels where available, thyroid function, inflammatory markers (CRP, IL-6), EEG alpha asymmetry, and genetic CYP2D6 and CYP3A4 metabolizer status. Demographics (20 features): age, gender, education level, employment status, social support index, living situation, and comorbid physical health conditions.

Data was extracted from clinical records using a standardized OMOP Common Data Model harmonization pipeline applied to each site’s EHR system [27]. Missing biomarker data (present in 31.4% of patients) were imputed using multivariate imputation by chained equations (MICE), with missingness indicators included as auxiliary features. Medication doses were standardized to chlorpromazine equivalents for antipsychotics and diazepam equivalents for benzodiazepines using published conversion tables from the Maudsley Prescribing Guidelines [28]. All the features were confirmed to precede the index treatment initiation visit. CGI-Improvement was derived from the most recent prior clinical assessment, not the 12-week follow-up, reflecting the patient’s trajectory entering the index visit. Medication adherence was computed over the 4-week window prior to the index visit using prescription refill records and clinician-documented adherence ratings. Serum drug levels and inflammatory markers (CRP, IL-6, thyroid) were extracted from the most recent laboratory record predating the index visit by no more than 8 weeks. CYP2D6/CYP3A4 metabolizer status was extracted from any prior genotyping record. No post-treatment information entered the predictor at any stage; outcome labels at 12-week follow-up were held strictly separate from the feature construction pipeline.

Treatment outcomes were assessed at 12-week follow-up by a trained clinical rater using all available clinical information: repeat ratings on the HDRS-17 and GAF, clinical notes from interim visits, patient self-report, and collateral from treating clinicians. Response was defined as ≥50% reduction in HDRS-17 from baseline; remission required HDRS-17 ≤ 7 and GAF ≥ 61. Partial response was defined as 25% - 49% HDRS-17 reduction. Non-response was defined as <25% reduction or clinical worsening. For non-depressive presentations, equivalent operationalizations were applied using YMRS (mania) and CGI (schizophrenia, OCD, GAD). All borderline classifications were reviewed by a second independent rater, achieving inter-rater reliability of κ = 0.86 (95% CI: 0.83 - 0.89). Cross-Diagnostic Outcome Harmonization. Outcome operationalization was adapted by primary diagnosis while maintaining the three-class label structure (Response/Partial Response/Non-Response) across all five diagnostic categories. For MDD (n = 942): HDRS-17 ≥ 50% reduction = Response; 25% - 49% = Partial Response; <25% = non-response. For BD (n = 645): YMRS ≥50% reduction from mania baseline, or HDRS-17 ≥ 50% reduction from depressive baseline, depending on index episode polarity = Response; 25% - 49% = Partial Response; <25% = non-response. For Schizophrenia spectrum (n = 446): CGI-Improvement score ≤ 2 (much/very much improved) = Response; CGI-I = 3 (minimally improved) = Partial Response; CGI-I ≥ 4 = non-response. For OCD (n = 273): Y-BOCS ≥ 35% reduction = Response; 25% - 34% = Partial Response; <25% = non-response. For GAD (n = 174): GAD-7 ≥ 50% reduction = Response; 25% - 49% = Partial Response; <25% = non-response. Scale availability by diagnosis is reported in Supplementary Table S3. The three-class label structure was applied uniformly across disorders; all threshold definitions were pre-specified in the study protocol prior to data analysis.

4.1.1. Encoder Architecture

Each of the five feature domains is encoded by a domain-specific module. Clinical state and treatment profile features are encoded by a 4-layer Clinical Transformer [29] with 8 attention heads and d_model = 256, operating on the feature vector with learned positional encodings that encode feature-type rather than sequence position. Patient history features are encoded by a 3-layer Temporal Convolutional Network [30] with dilations d_l ∈ {1, 2, 4} operating on the patient’s longitudinal treatment history vector each prior treatment trial is a timestep, and the TCN learns to extract trajectory patterns (e.g., a sequence of partial responses that predicts eventual non-response). Biomarker and demographic features are each encoded by 3-layer MLPs with residual connections. All encoders produce 128-dimensional embeddings that are concatenated and projected to a unified 512-dimensional patient state representation h patient ∈ ℝ 512 .

4.1.2. Bayesian Ensemble Output

The outcome predictor deploys a Bayesian deep ensemble of M = 10 independently initialized network instances, following the deep ensemble methodology [31]. At inference time, Monte Carlo Dropout [32] with p = 0.3 is maintained active across all ensemble members, generating T = 50 stochastic forward passes per member. The treatment outcome probability vector and its uncertainty are computed as:

When predictive entropy H = − ∑ c p ¯ c log ( p ¯ c ) exceeds threshold τ = 0.44 nats calibrated on the validation set via temperature scaling [33] the model flags the prediction as high-uncertainty and routes it for mandatory clinician review rather than automated counterfactual generation. The abstention rate on the test set is 8.7%; within non-abstained predictions, accuracy rises to 96.2%.

4.2.1. Formal Problem Statement

Given a factual instance x with predicted class y = f(x) = ‘Non-Response’, we seek a counterfactual x’ satisfying:

where L_proximity is the weighted L1 distance between x’ and x (weighted by feature mutability scores assigned by clinical domain experts); L_validity is a differentiable penalty for violating clinical plausibility and causal consistency constraints; L_diversity promotes dissimilarity between x’ and previously generated counterfactuals X ′ prev ; C(x) is the space of clinically actionable modifications to x; and k_max = 5 is the maximum allowed number of feature changes (in practice, the mean observed is 2.3).

Y_target(x) is the class-conditional recourse target, defined as follows. For patients predicted as Non-Response, Y_target = {‘Response’, ‘Partial Response’} the generator first seeks ‘Response’ and returns ‘Partial Response’ counterfactuals if no valid ‘Response’ counterfactual is found within 500 gradient iterations. For patients predicted as Partial Response, Y_target = {‘Response’} the recourse target is always full response. For patients predicted as Response, no counterfactual is generated. This formulation ensures clinically meaningful recourse is defined for all three prediction classes, not only non-response cases.

4.2.2. Feature Mutability and Causal Constraint Graph

A key innovation of CounterPsych is the explicit encoding of two domain-specific constraint types. First, each of the 112 features is assigned a mutability score m_j ∈ {0, 0.5, 1} by a panel of five consultant psychiatrists: immutable features (m_j = 0) include age, genetic markers, age at first episode, and number of prior episodes; partially mutable features (m_j = 0.5) include illness duration and number of prior hospitalizations; fully mutable features (m_j = 1) include medication dose, therapy frequency, sleep duration, medication adherence, and social support measures. Second, a causal constraint graph G_C encodes the directed causal dependencies between clinical features: reducing the HDRS-17 score requires an intervention that produces that reduction (it cannot be changed independently); increasing therapy frequency is causally downstream of an actionable clinical decision; medication dose changes propagate to serum level estimates and side effect profiles.

The causal constraint graph G_C was constructed from published psychiatric treatment literature and validated by the same clinician panel. During counterfactual search, any proposed feature change that violates a causal constraint in G_C is projected back onto the constraint-satisfying manifold through a differentiable constraint projection layer, ensuring that all generated counterfactuals respect the causal structure of the clinical domain.

Constraint Construction, Agreement, and Usage Statistics: The mutability map was constructed by a panel of five consultant psychiatrists who independently scored all 112 features as immutable (m_j = 0), partially mutable (m_j = 0.5), or fully mutable (m_j = 1). Inter-rater agreement for mutability classification was κ = 0.83 (95% CI: 0.79 - 0.87), indicating strong expert consensus. Disagreements were resolved by majority vote; two features (illness duration, number of prior hospitalizations) required structured panel discussion before consensus. The causal constraint graph G_C contains 47 directed edges encoding causal dependencies validated against CANMAT 2023, BAP 2019, and NICE Clinical Guidelines NG222. Patient-preference data (e.g., stated aversion to specific drug classes, appointment frequency constraints) was extractable from clinical records for 61.4% of patients (n = 1523/2480); for the remaining 38.6%, the patient-preference constraint was inactive. Constraint impact statistics across the full test set: the mutability filter excluded at least one proposed feature change in 34.7% of first-pass counterfactuals; the causal constraint projection modified at least one feature change in 28.3% of counterfactuals; the patient-preference filter rejected at least one candidate counterfactual entirely in 19.1% of cases where preference data was available, triggering a replacement search. These rejection rates confirm that all three constraint layers are operationally active and materially shape the counterfactual output.

4.2.3. Gradient Search Algorithm

The counterfactual search is implemented as a constrained gradient descent in the continuous feature space, starting from the factual instance x and following the gradient of the outcome predictor’s log-probability surface toward the target class. Unlike prototype-based methods [34], which anchor counterfactuals to training instances, our gradient search explores the full feature space subject to the mutability and causal constraints. The search terminates when the predictor’s probability for the target class exceeds 0.65 and all constraint violations are below a tolerance threshold ε = 0.01. A set of K = 5 diverse counterfactuals is generated per patient by running the search K times with diversity-promoting initializations that enforce a minimum cosine distance of 0.3 between any two returned counterfactuals.

The diversity regularization term L_diversity draws on the counterfactual fairness framework [35] to ensure that the returned counterfactual set covers qualitatively distinct recourse paths for example, one counterfactual emphasizing medication adjustment, another emphasizing psychotherapy increase, and a third emphasizing lifestyle and adherence changes rather than K near-identical variations of the same intervention.

Generated counterfactuals pass through a rule-based clinical validity filter before being presented to the clinician. The filter enforces four validity criteria:

Clinical plausibility:The counterfactual’s feature values must fall within clinically realistic ranges for each feature type (e.g., medication doses within approved therapeutic ranges; symptom scores within scale bounds; therapy frequency within available service configurations).Medical actionability:All suggested feature changes must correspond to interventions within the prescribing and referral authority of a consultant psychiatrist (e.g., medication switch or dose adjustment; referral to structured psychotherapy; inpatient admission for monitoring). Features flagged as immutable by the clinician panel are excluded from all counterfactuals.Causal consistency:The counterfactual must satisfy all directed constraints in the causal graph G_C no feature change is permitted that implies a downstream consequence that is not also reflected in the counterfactual. For example, a reduction in depression score must be accompanied by a plausible intervention that would produce it.Patient-preference alignment:Where patient-stated preferences are available from the clinical record (e.g., preference against certain medication classes, stated reluctance to increase appointment frequency), the filter excludes counterfactuals that violate these preferences. This criterion is applied as a soft constraint with a clinician-override option.

Counterfactuals that fail any hard constraint are discarded and replaced through additional gradient search iterations. The validity filter achieves a pass rate of 91.4% on the first-generated counterfactual per patient and 97.8% across the full K = 5 set.

The outcome predictor and all modality encoders were jointly trained using AdamW (lr = 2 × 10⁻⁴, weight decay = 1 × 10⁻², β₁ = 0.9, β₂ = 0.999) with cosine annealing over 150 epochs. Weighted cross-entropy addressed class imbalance. Multi-task loss combined outcome classification (primary), recourse sparsity regularization (auxiliary), and causal constraint consistency (auxiliary). Data splits: 70% training, 15% validation, 15% test, stratified jointly on diagnosis, treatment class, and response rate tertile. Implementation: PyTorch [36] with HuggingFace Transformers. Hardware: NVIDIA A100 80GB. Figure 2Figure 2 shows training convergence CounterPsych reaches plateau at epoch 108, training accuracy 0.963, validation accuracy 0.941.

Figure 3Figure 3 presents the confusion matrix on the held-out test set. CounterPsych achieves per-class accuracy of 94.8% for Treatment Response, 93.1% for Partial

Figure 2. Training and Validation Convergence. Accuracy (left) and cross-entropy loss (right) over 150 epochs for CounterPsych (navy), WACHUNG CF baseline (teal), and DiCE (coral). CounterPsych achieves the highest validation accuracy with a narrow train-validation gap (0.022). Early stopping fires at epoch 108 (gold dotted line).

Figure 3. Treatment Outcome Confusion Matrix. Raw counts (left) and row-normalized accuracy (right) on the held-out test set. CounterPsych achieves >93% per-class accuracy across all three outcome categories. The primary misclassification occurs at the Response-Partial Response boundary, reflecting the inherent clinical uncertainty at the 50% symptom reduction threshold.

Response, and 94.6% for non-response. The most frequent misclassification is between Response and Partial Response (5.2% of Response cases classified as Partial Response), which is clinically expected given that the distinction between full and partial response at the 12-week endpoint involves threshold judgments that are genuinely uncertain even for experienced clinicians.

Figure 4Figure 4 presents the per-class ROC curves and the comparative AUC-ROC ranking. CounterPsych achieves macro-AUC of 0.977, with per-class AUC of 0.982 (Response), 0.971 (Partial Response), and 0.974 (Non-Response).

Figure 4. ROC Analysis and AUC-ROC Model Comparison. Left: Per-class ROC curves for CounterPsych all three classes achieve AUC > 0.97. Right: Macro-AUC comparison across all eight models. CounterPsych (0.977) significantly outperforms all baselines including WACHUNG CF (0.903) and the ablation without Bayesian ensemble (0.941) (DeLong test, p < 0.01 for all comparisons).

Table 2 presents the full performance comparison across eight models. CounterPsych achieves statistically significant improvements over all baselines on all four primary metrics.

Table 2.Comparative outcome prediction performance held-out test set (n = 496 patients).

Statistically significant improvement over all baselines (DeLong test, p < 0.01; McNemar test, p < 0.001). Abstaining predictions (8.7%) excluded. AUC-ROC reported as macro one-vs-rest average.

Site-Held-Out and Chronological Validation. To assess generalization beyond the standard patient-stratified split, two additional evaluations were performed. First, a site-held-out validation was conducted by training on three sites and testing on the fourth, repeated for each of the four sites (leave-one-site-out cross-validation). CounterPsych achieved mean accuracy of 89.7% (SD 1.4%), mean AUC-ROC of 0.961 (SD 0.012) across the four held-out site folds, a 4.4 pp accuracy reduction relative to the same-distribution test set, indicating moderate but expected performance degradation under domain shift. Second, a chronological validation was performed by training on admissions prior to January 2023 and testing on the prospective 2023-2024 cohort (n = 214 patients). Accuracy on the chronological holdout was 91.3%, AUC-ROC 0.969. Abstentions (predictions with H> τ = 0.44) were included in all reported denominators; when abstentions are excluded (8.7% of predictions), accuracy on the chronological holdout rises to 93.1%.

Figure 5Figure 5 presents the core counterfactual analysis: a worked single-patient example, the distribution of counterfactual proximity distances across the cohort, and the clinical validity rates across five constraint categories.

Figure 5. Counterfactual Generation Analysis. Panel A: Single-patient worked example comparing the factual (non-responding) treatment configuration with the CounterPsych counterfactual, annotated with the predicted feature changes and their direction. For this patient, the model identifies that increasing therapy sessions from 0 to 4/month, reducing quetiapine by 150 mg, increasing medication adherence by 22%, and improving sleep duration by 1.8 hours/night would flip the prediction from Non-Response to Response. Panel B: Distribution of L2 distances between factual and counterfactual in feature space CounterPsych CFs (mean 1.62, teal) are substantially more proximate to the factual than DiCE CFs (mean 4.21, coral), confirming greater actionability. Panel C: Clinical validity rates across five constraint dimensions CounterPsych achieves > 83% validity across all criteria, dramatically outperforming DiCE on actionability (91% vs 62%) and causal consistency (88% vs 54%).

The proximity analysis confirms that CounterPsych counterfactuals are substantially more actionable than prior methods. The mean L2 distance of 1.62 in standardized feature space compared to 4.21 for DiCE indicates that CounterPsych recourse plans involve smaller, more clinically realistic changes. Prototype-based methods [37] achieve comparable proximity but substantially lower clinical plausibility because their anchoring to training instances does not distinguish between plausible and implausible prototype configurations. Case-based counterfactual methods [38] achieve high plausibility but sacrifice sparsity, generating counterfactuals that require changes to many more features than the clinical setting permits.

Figure 6Figure 6 presents the feature group importance decomposition and the top-15 individual feature importances.

Figure 6. Feature Group and Individual Feature Importance. Left: Feature group importance from Random Forest decomposition across the five clinical domains. Clinical state features dominate (38.7%), consistent with the established predictive value of baseline symptom severity. Treatment profile contributes 28.4% reflecting the substantial predictive information in medication history, prior response patterns, and adherence. Right: Top 15 individual features. HDRS-17 total score, GAF, and medication dose equivalent are the three strongest predictors; prior treatment response history ranks fourth, confirming that the trajectory of prior responses is highly informative about future response probability.

Clinical state features contribute the largest share (38.7%) of predictive information, driven by HDRS-17 total score and GAF. Treatment profile accounts for 28.4%, with prior treatment response history as the single most informative treatment feature a patient who has responded to a prior trial of the same medication class is substantially more likely to respond again, a clinical heuristic that the model learns and quantifies precisely. Patient history contributes 19.3%, biomarkers 8.9%, and demographics 4.7%. The relatively modest contribution of demographics, particularly age and gender, to the predictive model is encouraging from a fairness perspective and consistent with the subgroup analysis showing maximum accuracy disparity of only 1.7% across demographic groups. (See Figure 7Figure 7)

Figure 8Figure 8 presents the calibration reliability diagram and ablation study.

Figure 7. All-Metrics Comparison and Bayesian Uncertainty Distribution. Left: Grouped bar comparison of Accuracy, Macro-F1, Precision, and Recall across all eight models. CounterPsych leads consistently across all metrics. Right: Epistemic uncertainty distribution for correct (teal) and incorrect (coral) predictions. The review threshold τ = 0.17 cleanly separates the distributions the abstention protocol correctly identifies predictions in the uncertain regime.

Figure 8. Calibration Reliability Diagram and Module Ablation. Left: CounterPsych (navy, ECE = 0.021) tracks the perfect calibration diagonal closely across all confidence bins. WACHUNG CF (teal, ECE = 0.054) and DiCE (coral, ECE = 0.119) show systematic overconfidence particularly dangerous in clinical applications where overconfident non-response predictions might lead to premature treatment termination. Right: Ablation study removing the Bayesian ensemble produces the largest calibration degradation (ECE rises from 0.021 to 0.061); removing the counterfactual generator has no effect on accuracy but disables the entire explainability functionality; removing the History TCN produces the largest accuracy drop (−4.3 pp).

The ablation confirms every module contributes independently. The Bayesian ensemble contributes primarily to calibration rather than accuracy removing it reduces accuracy by 1.5 pp but nearly triples ECE (0.021 → 0.061). The History TCN contributes the largest accuracy gain (−4.3 pp when removed), confirming that longitudinal treatment trajectory is the most informative single architectural component. The Clinical Transformer contributes −3.4 pp when removed. The predictor-only ablation achieves 85.2% accuracy competitive with XGBoost but loses all explainability functionality.

Figure 9Figure 9 presents the three-panel clinician evaluation: Likert rating scores across five quality dimensions, the recourse sparsity distribution, and subgroup fairness analysis.

Figure 9. Clinician Evaluation, Recourse Sparsity, and Subgroup Fairness. Left: CounterPsych counterfactuals receive mean ratings of 4.28-4.51/5 across five clinical quality dimensions from 24 consultant psychiatrists, substantially outperforming DiCE (3.08-3.21/5). The largest gap is on actionability (CounterPsych 4.28 vs DiCE 2.98) and trustworthiness (4.51 vs 3.14). Centre: Recourse sparsity distribution 75.3% of CounterPsych counterfactuals change 3 or fewer features, confirming that the generated recourse plans are parsimonious and clinically manageable. Right: Subgroup accuracy analysis maximum accuracy disparity of 1.7% across demographic and diagnostic subgroups confirms equitable performance.

The clinician evaluation was conducted as a prospective, blinded rating study with 24 consultant psychiatrists drawn from three European academic psychiatric centers (8 per site). Raters held a mean of 11.4 years of post-certification clinical experience (range 5–28 years). Each rater evaluated a randomly sampled set of 20 de-identified patient cases, 10 CounterPsych counterfactuals and 10 DiCE counterfactuals presented in random interleaved order with method identity withheld (blinded presentation). Cases were sampled from the held-out test set, stratified by diagnosis and outcome class to ensure representation across all five diagnostic categories and all three outcome classes. The same 20 cases were rated by all 24 psychiatrists, yielding 480 ratings per method. Presentation order was randomized per rater using a Latin square design to control for sequence effects. Raters scored each counterfactual on five 5-point Likert dimensions: clinical plausibility, medical actionability, causal coherence, patient-appropriateness, and trustworthiness. Inter-rater agreement was assessed using intraclass correlation coefficient (ICC): ICC (2, 1) = 0.81 (95% CI: 0.77 - 0.84) for plausibility and ICC (2, 1) = 0.79 (95% CI: 0.75 - 0.83) for trustworthiness indicating good to excellent agreement. Paired Wilcoxon signed-rank tests comparing CounterPsych versus DiCE ratings were significant on all five dimensions (p < 0.001 for all). No rater evaluated cases from their own institution [39][40].

The recourse sparsity analysis shows that 75.3% of CounterPsych counterfactuals change 3 or fewer clinical features. This is clinically important: a treatment recommendation that requires simultaneous changes to 7 different aspects of a patient’s care plan is not actionable in any realistic outpatient setting, regardless of its mathematical validity. The mean of 2.3 feature changes per counterfactual corresponds to the scale of a typical treatment adjustment in psychiatric outpatient practice for example, increasing the dose of a current medication, adding structured psychotherapy, and setting a specific sleep hygiene target.

The central finding is that counterfactual explainability in psychiatric AI is achievable at clinical quality not as a theoretical property of the optimization objective, but as a practically validated characteristic of the system’s outputs as judged by the clinicians who would use them. The mean plausibility rating of 4.42/5 and trustworthiness of 4.51/5 from 24 experienced psychiatrists are not cosmetic improvements over DiCE’s 3.21/5 and 3.14/5. They represent the difference between an explanation system that clinicians will engage with and one they will dismiss. Machine learning tools for psychiatric readmission prediction [41] have consistently failed to achieve clinical adoption despite reasonable predictive accuracy precisely because the outputs could not be trusted or acted on. CounterPsych’s counterfactual generator is designed to solve this trust deficit at its root by making the explanations causally consistent with the clinical domain rather than simply numerically proximate to the factual.

The causal constraint graph is the architectural innovation that makes this possible. The key insight is that clinical plausibility is not just about whether feature values fall within realistic ranges it is about whether the pattern of feature changes tells a causally coherent clinical story. A counterfactual that suggests a patient’s HDRS-17 score would be 8 points lower without specifying an intervention that would produce that reduction is not a clinical explanation it is a mathematical artifact. By enforcing causal consistency through the constraint graph G_C, CounterPsych ensures that every generated counterfactual corresponds to a coherent sequence of clinical actions, each of which is within the treating psychiatrist’s authority and the patient’s realistic circumstances.

The proximity advantage CounterPsych CFs at mean L2 = 1.62 versus DiCE at 4.21 is a direct consequence of the clinical constraint structure. When the optimization is constrained to actionable, causally consistent feature changes, the resulting counterfactuals are automatically closer to the factual because the mutable, actionable features are a small and carefully selected subset of the full feature space. The constraint that initially appears to restrict the search space is simultaneously reducing the distance from the factual to the nearest valid counterfactual because valid counterfactuals lie in a region of the feature space that is clinically adjacent to the patient’s current situation.

CounterPsych’s outputs have three concrete clinical use cases. First, at treatment initiation, the model’s prediction and its accompanying counterfactuals can inform the comparative choice between treatment options showing the psychiatrist which configuration of medication, dose, and psychotherapy the model predicts will produce response, with the specific feature changes quantified and ranked by their estimated contribution. Second, during treatment monitoring, serial CounterPsych evaluations can track whether the patient’s trajectory is moving toward or away from the counterfactual target providing early warning of emerging non-response before it becomes clinically manifest. Third, at treatment review or switching decisions, the counterfactual set provides a structured set of evidence-based recourse options that the clinician can evaluate against the patient’s preferences and circumstances rather than relying on heuristic rule-of-thumb switching algorithms [42].

The ECE of 0.021 is a clinically meaningful calibration result. It means that when CounterPsych reports 80% confidence in a response prediction, approximately 80% of such predictions are correct enabling the psychiatrist to use the model’s confidence score as genuine probabilistic information in their decision-making, rather than treating the score as an uninterpretable black-box output. Combined with the counterfactual explanations, this creates a complete clinical decision support workflow: the model says how confident it is in its prediction, what that prediction is, and what would need to change for it to be different.

12-week outcome horizon.Treatment response was assessed at 12 weeks. For conditions with longer treatment timescales particularly bipolar disorder, where mood stabiliser response may require 6 - 12 months of continuous treatment the 12-week horizon may miss genuine responders classified as non-responders. Extended follow-up analyses at 6 and 12 months are planned.Causal graph construction.The causal constraint graph G_C was built by expert elicitation from five consultant psychiatrists and validated against published treatment guidelines. It necessarily reflects the current state of clinical knowledge and expert consensus, which may not capture all relevant causal dependencies, particularly for novel drug combinations or less-studied patient subgroups.Single clinician evaluation site.The clinician evaluation with 24 psychiatrists was conducted at a single European academic center, where psychiatrists may have higher baseline familiarity with AI-assisted tools than average. Replication of the evaluation at community mental health settings and in non-European clinical contexts is required before generalizing the usability findings.Retrospective-prospective heterogeneity.The retrospective component introduced protocol variability across sites and years that was partially but not fully mitigated by harmonization. The prospective component, while collected under a standardized protocol, covers only two years and may not capture the full range of treatment responses across longer illness trajectories.Missing biomarker data.Biomarker data was available for only 68.6% of patients, and the MICE imputation, while principled, introduces additional uncertainty in biomarker-dependent counterfactuals. Future prospective data collection should prioritize standardized biomarker acquisition across all sites.