The evaluation of acute psychiatric emergencies in emergency departments presents unique diagnostic challenges distinct from outpatient psychiatric assessment. Time pressures, patient agitation, limited historical information, and high stakes for misclassification create conditions where clinical judgment alone may prove insufficient [33]-[35]. Observational studies suggest diagnostic disagreement rates of 15% - 30% between emergency physicians and psychiatrists for acute presentations, with difficulty distinguishing agitation from emerging psychosis and identifying delirium masked by behavioural disturbance [36]-[38].

The consequences of diagnostic error in emergency settings are severe. Administration of antipsychotics to patients with delirium can worsen confusion and prolong hospitalization; benzodiazepines given to agitated psychotic patients may paradoxically increase behavioural decontrols; and failure to recognize delirium may delay treatment of underlying medical emergencies [39]-[41]. Current structured assessment instruments such as the Brief Psychiatric Rating Scale and Richmond Agitation-Sedation Scale require trained raters and substantial time, limiting utility in busy emergency departments [42]-[44].

Psycholinguistic research has identified numerous speech and language markers associated with psychiatric conditions. In schizophrenia and acute psychosis, studies consistently report increased semantic incoherence, reduced syntactic complexity, abnormal prosody, and distinctive patterns of word association [45]-[47]. Thought disorder measured through linguistic analysis correlates with functional impairment and predicts long-term outcome, suggesting clinical utility beyond diagnosis [48]-[50].

Agitation and emotional arousal manifest in speech through increased rate, volume, and pitch variability, with content analysis revealing elevated hostility markers and reduced complexity [51]-[53]. Acoustic analysis has demonstrated elevated fundamental frequency and formant dispersion in aggressive states, potentially providing pre-linguistic indicators of behavioural escalation [54]-[56].

Cognitive disorganization in delirium and severe affective states produces characteristic linguistic patterns including semantic paraphasias, perseveration, and marked reduction in lexical diversity [57]-[59]. Temporal analysis reveals progressive coherence degradation over the course of conversation, distinguishing delirium from psychiatric conditions with more stable presentation [60]-[62].

Natural language processing has increasingly been applied to psychiatric assessment, with machine learning demonstrating potential for diagnostic classification and symptom severity estimation [63]-[65]. In schizophrenia specifically, NLP analysis of clinical interviews and spontaneous speech has achieved classification accuracies of 70% - 85% against healthy controls [66]-[68].

For emergency psychiatry, NLP applications remain limited. Most studies examine written clinical notes rather than direct speech analysis, and few address the acute presentations typical of emergency settings [69]-[71]. Recent work applying transformer models to emergency department psychiatric notes has shown promise for predicting admission and identifying high-risk presentations, but direct speech analysis in emergency contexts remains largely unexplored [72]-[74].

Deep learning approaches to speech analysis have demonstrated success in neurological and psychiatric applications. Convolutional neural networks operating on spectrograms have achieved robust classification of Parkinson’s disease, Alzheimer’s disease, and depression with accuracies exceeding 80% [75]-[77]. Recurrent architectures including LSTM and GRU networks have proven effective for modelling temporal dynamics in speech, capturing progressive changes associated with neurodegenerative conditions [78]-[80].

Transformer architectures, originally developed for natural language processing, have shown promise for speech emotion recognition and paralinguistic analysis [81]-[83]. The self-attention mechanism enables modelling of long-range dependencies in speech production, potentially capturing patterns of coherence degradation relevant to acute psychiatric assessment [84]-[86].

Despite these advances, no prior study has integrated multi-scale CNN analysis with temporal deep learning specifically for emergency psychiatric speech analysis. Our work addresses this gap by developing a multimodal framework that leverages complementary strengths of spatial feature extraction (CNN) and temporal modelling (LSTM/transformer) for rapid diagnostic stratification.

To enable controlled evaluation and reproducible experimentation, we constructed clinically grounded synthetic datasets representing speech features during emergency psychiatric consultations. Synthetic data generation allows systematic manipulation of psycholinguistic parameters while preserving realistic feature distributions, providing a stable testbed for model development and validation [87]-[91].

Label Definitions. Four mutually exclusive diagnostic categories were used as classification targets. Control/Normal (label = 0): speech produced during psychiatric emergency assessment with no evidence of formal thought disorder, psychomotor agitation, or cognitive impairment; this class represents patients who present in behavioural crisis but are assessed as not meeting criteria for the three pathological categories below. Psychosis (label = 1): acute psychotic presentation characterized by formal thought disorder including tangentiality, loose associations, neologisms, and poverty of content arising from dysregulated dopaminergic signalling; operationally defined by BPRS conceptual disorganization subscale ≥ 4 and clinician documentation of active psychotic symptoms. Agitation (label = 2): acute behavioural agitation with pressured, high-intensity speech and elevated hostility markers, without primary thought disorder; operationally anchored to RASS score ≥ +2 or BPRS hostility subscale ≥ 4. Cognitive Disorganization (label = 3): incoherent, fragmented speech with severely impaired semantic coherence, perseveration, and marked lexical poverty, consistent with delirium or severe acute confusional state; operationally defined by CAM-ICU positive screen or equivalent clinician delirium rating. Although these four presentations overlap substantially in real emergency settings agitation frequently co-occurs with psychosis, and delirium may present with behavioural agitation the synthetic generator treats them as mutually exclusive by design, which represents a key limitation of the current proof-of-concept.

3.1.1. Speech Feature Domains

We generated synthetic speech feature vectors of dimension 48, organized across five clinical domains reflecting established psycholinguistic research:

Prosodic Features (8 dimensions): Fundamental frequency (mean and variability), speech rate, pause duration, intensity (mean and range), voice tremor, and prosodic flatness. These features capture emotional and arousal-related aspects of speech production mediated by autonomic and limbic circuits [92]-[94].

Lexical Features (12 dimensions): Word frequency rarity, abstract-concreteness ratio, word association strength, semantic coherence, lexical diversity, repetition rate, neologism count, word salad index, clang associations, poverty of content, circumstantiality, and tangentiality. These markers reflect thought organization and semantic processing [95]-[97].

Syntactic Features (8 dimensions): Syntactic complexity, sentence length variation, grammatical errors, incomplete sentences, word order disruptions, embedding depth, syntactic coherence, and phrase fragmentation. These features capture grammatical organization and structural planning [98]-[100].

Discourse Features (12 dimensions):Topic maintenance, topic switches, global coherence, local coherence, reference clarity, inference demand, discourse markers, repair attempts, interruptions, response latency, turn-taking violations, and hostility markers. These reflect conversational pragmatics and interpersonal regulation [101]-[103].

Acoustic-Phonetic Features (8 dimensions): Jitter, shimmer, harmonics-to-noise ratio, formant dispersion, voice breaks, breathiness, tension index, and resonance changes. These capture voice quality and phonatory control [104]-[106].

3.1.2. Signal Generation Process

Each feature vector was constructed using a clinically informed generative process modelling distinct speech patterns across diagnostic categories. For control/normal speech (label = 0): balanced prosodic variation (intensity: 0.5 - 0.8 normalized), high semantic coherence (0.7 - 0.9), normal lexical diversity (0.6 - 0.8), complex syntactic structure (0.6 - 0.8), coherent discourse maintenance (0.7 - 0.9), and stable acoustic parameters (0.4 - 0.7).

For psychosis (label = 1): reduced prosodic variation (0.2 - 0.5) with inappropriate flatness, markedly reduced semantic coherence (0.2 - 0.4), elevated lexical disturbance including neologisms (0.6 - 1.0) and word salad (0.4 - 0.8), syntactic disorganization (0.2 - 0.4), poor discourse coherence (0.2 - 0.4) with elevated tangentiality (0.6 - 0.9), and variable acoustic instability (0.3 - 0.8).

For agitation (label = 2): elevated prosodic arousal (0.7 - 1.0) with increased rate and intensity, preserved semantic coherence (0.5 - 0.7) with reduced complexity, reduced lexical diversity (0.3 - 0.5) with elevated repetition (0.6 - 0.9), simplified syntactic structure (0.3 - 0.5), disrupted discourse with elevated interruptions (0.7 - 1.0) and hostility (0.5 - 0.8), and tense voice quality (0.6 - 0.9).

For cognitive disorganization (label = 3): inconsistent prosodic patterns (0.3 - 0.7), severely impaired semantic coherence (0.1 - 0.3), reduced lexical diversity (0.3 - 0.5) with confused word selection, marked syntactic fragmentation (0.1 - 0.3), severely incoherent discourse (0.1 - 0.3), and unstable acoustic parameters (0.2 - 0.6).

Gaussian noise (σ = 0.15) was added to simulate natural variation in speech production. Final features were z-score normalized to ensure consistent distributions across samples.

3.1.3. Dataset Composition

The primary dataset comprised 2000 consultation samples (700 control, 500 psychosis, 400 agitation, 400 cognitive disorganization), reflecting emergency department prevalence with higher representation of normal assessments. The dataset was split into training (70%), validation (10%), and test (20%) sets using stratified random sampling.

3.2.1. Multi-Scale Domain-Aware CNN (PsychiatricSpeechNet)

We designed a multi-scale convolutional neural network integrating domain-specific processing with cross-attention mechanisms for comprehensive speech feature analysis.

Domain-Specific Encoders:

- Prosodic encoder: Linear (8 → 32) → LayerNorm → ReLU → Linear (32 → 32)

- Lexical encoder: Linear (12 → 48) → LayerNorm → ReLU → Linear (48 → 48)

- Syntactic encoder: Linear (8 → 32) → LayerNorm → ReLU → Linear (32 → 32)

- Discourse encoder: Linear (12 → 48) → LayerNorm → ReLU → Linear (48 → 48)

- Acoustic encoder: Linear (8 → 32) → LayerNorm → ReLU → Linear (32 → 32)

Cross-Domain Attention: Multi-head attention (8 heads, dropout 0.4) operating on concatenated domain features (192 dimensions) to model interactions between speech domains.

Convolutional Pathway:

- Conv1D (1 → 64, kernel = 3) → BatchNorm → ReLU → MaxPool

- Conv1D (64 → 128, kernel = 3) → BatchNorm → ReLU → MaxPool

Multi-Scale Inception Module:

- Parallel convolutions: 1 × 1 (32 channels), 3 × 3 (32 channels), 5 × 5 (32 channels), and pooled features (32 channels)

- Concatenation to 128 channels with global average pooling

Classification Head:

- Concatenation of convolutional (128) and attention (192) features

- Linear (320 → 256) → BatchNorm → ReLU → Dropout (0.4)

- Linear (256 → 128) → BatchNorm → ReLU → Residual connection → Dropout (0.4)

- Linear (128 → 4) → Softmax

Total parameters: 2,847,236

3.2.2. LSTM with Attention (LSTMAttentionNet)

The LSTM model captures temporal dependencies in speech feature sequences:

Feature Processing

- Input: 48-dimensional feature vectors treated as sequence (batch, 1, 48)

- Bidirectional LSTM: 2 layers, hidden size 128, dropout 0.3

Attention Mechanism

- Multi-head self-attention on LSTM outputs (8 heads)

- Global average pooling across sequence

Classification Head

- Linear (256 → 128) → LayerNorm → ReLU → Dropout (0.3)

- Linear (128 → 4) → Softmax

Clarification of Prediction Tasks and Temporal Input. The four-class macro metrics in Table 1 are reported for the CNN (PsychiatricSpeechNet) only; the LSTM and transformer models were trained and evaluated on a binary agitation-versus-cognitive-disorganization subtask, reflecting the most clinically consequential diagnostic boundary in emergency triage. Their reported metrics in Table 1 therefore reflect binary rather than four-class performance and should not be compared directly to the CNN’s four-class metrics. This distinction is clarified in the revised Table 1 header below. Regarding temporal dynamics: both the LSTM and transformer receive input of shape (batch, 1, 48) a single-timestep sequence in the current proof-of-concept implementation. This means neither architecture exploits genuine temporal dynamics; the sequence dimension of length 1 means the LSTM reduces to a single-step gated update and the transformer’s positional encoding carries no temporal information. The reported performance advantage of these models over traditional baselines therefore reflects their capacity to learn non-linear feature interactions, not temporal modelling. In the intended clinical deployment, the input would be a sequence of 48-dimensional feature vectors extracted at consecutive 5-second windows across a 2 - 5-minute consultation segment, yielding sequences of length 24 - 60. The temporal architectures are designed for this multi-timestep setting; single-timestep evaluation in this proof-of-concept is a limitation requiring prospective data to address.

Table 1. Model performance comparison. PsychiatricSpeechNet metrics: four-class macro-average on held-out synthetic test set. LSTM and transformer metrics: binary agitation-versus-cognitive-disorganization task. Metrics are not directly comparable across rows due to task difference.

Note: Metrics reported as macro-averages across four diagnostic classes. MCC = Matthews correlation coefficient.

3.2.3. Transformer Architecture (TransformerPsychNet)

The transformer model employs self-attention for global dependency modeling:

Input Projection: Linear (48 → 128)

Positional Encoding: Sinusoidal encoding added to projected features

Transformer Encoder

- 3 layers with 8 attention heads, feedforward dimension 512, dropout 0.3

- Pre-norm residual connections

Classification Head

- Global average pooling across sequence

- Linear (128 → 64) → ReLU → Dropout (0.3)

- Linear (64 → 4) → Softmax

Optimization

- Optimizer: AdamW with weight decay 1 × 10⁻⁴

- Learning rate: 1 × 10⁻³ with ReduceLROnPlateau scheduling

- Batch size: 32

- Epochs: 100 with early stopping (patience 15)

Loss Function: Cross-entropy loss with class weighting to address distribution imbalance:

where,

N = total number of training samples y i = true class label of sample i p y i = predicted probability for the true class y i w y i = weight assigned to class y i L = weighted cross-entropy loss

Regularization

- Dropout: 0.3 - 0.4 across layers

- Batch normalization after linear layers

- Gradient clipping (max norm 1.0)

- Early stopping based on validation loss

For comparative evaluation, we implemented traditional machine learning approaches:

Random Forest: 200 estimators, max depth 15, balanced class weighting.

Support Vector Machine: RBF kernel, C = 10, gamma = “scale”, probability calibration.

Gradient Boosting: 150 estimators, learning rate 0.1, max depth 5.

Performance was quantifiedusing

- Accuracy: Correct classifications/total samples

- Precision: TP/(TP + FP) per class

- Recall (Sensitivity): TP/(TP + FN) per class

- F1-Score: Harmonic mean of precision and recall

- AUC-ROC: Area under receiver operating characteristic curve (one-vs-rest macro-average)

- Matthews Correlation Coefficient: Balanced measure for multiclass evaluation

Given the clinical importance of identifying agitation and psychosis (avoiding false negatives for safety-critical conditions), we particularly emphasize recall for these classes.

Model interpretability was assessed through:

Attention Visualization: Attention weight matrices from cross-domain and self-attention mechanisms visualized as heatmaps.

SHAP Values: Shapley additive explanations for feature importance quantification.

Domain Contribution Analysis: Relative contribution of prosodic, lexical, syntactic, discourse, and acoustic domains to classification decisions.

Model comparisons utilized McNemar’s test for paired classification outcomes. Confidence intervals were calculated using 1000 bootstrap replications. All analyses were performed using Python 3.9 with PyTorch 2.0, scikit-learn, and Captum for interpretability.

Before evaluating predictive performance, we verified that the synthetic dataset exhibits clinically meaningful structure and class-dependent variability.

Visual inspection of feature distributions in Figure 1Figure 1, reveals distinct patterns across diagnostic categories. Control samples show tight clustering around normal values with balanced domain contributions. Psychosis samples exhibit marked divergence in lexical and discourse features with elevated tangentiality and reduced coherence. Agitation demonstrates distinctive prosodic and acoustic profiles with increased intensity and tension markers. Cognitive disorganization shows widespread impairment across syntactic and discourse domains with high variability.

Figure 1. Dataset overview and feature distributions. (A) Class distribution reflecting emergency department prevalence; (B) Feature domain composition across 48 speech markers; (C) Correlation matrix of key psycholinguistic features; (D) Feature distributions by diagnostic category for selected discriminative markers.

All deep learning models achieved excellent classification performance, with the multi-scale CNN demonstrating marginally superior results. Table 1 presents comprehensive performance metrics across all models.

The multi-scale CNN achieved perfect classification on held-out test data, with all diagnostic categories correctly identified without confusion. Traditional machine learning approaches performed well but failed to achieve perfect separation, suggesting that the deep learning architecture was able to capture complex, non-linear interactions among speech domains that simpler models could not detect. These findings are illustrated in Figure 2Figure 2 below.

Figure 2.Model performance comparison and diagnostic accuracy. (A) Accuracy comparison across all models with clinical threshold indicated; (B) Confusion matrix for PsychiatricSpeechNet showing perfect classification; (C) ROC curves (macro-averaged) for deep learning models; (D) Top 15 feature importances from Random Forest baseline.

In Figure 3Figure 3, training loss decreased smoothly over epochs with rapid convergence to near-zero values. Validation accuracy reached 100% by epoch 20 and remained stable throughout training. The absence of overfitting evidenced by maintained validation performance suggests the model learned robust, generalizable features rather than memorizing training examples.

Per-class metrics demonstrate balanced performance across all diagnostic categories, with no systematic bias toward majority classes. This balance reflects the effectiveness of class weighting and the architectural design emphasizing domain-aware feature extraction.

Figure 3. Training dynamics and clinical interpretability. (A) Learning curves showing training and validation loss convergence; (B) Per-class precision, recall, and F1-scores; (C) Feature domain contributions from attention mechanism analysis; (D) Clinical risk stratification visualization for emergency decision support.

Attention visualization reveals in Figure 4Figure 4, clinically meaningful patterns. For psychosis classification, the model attends heavily to lexical features (tangentiality, word salad index) and discourse coherence. Agitation detection emphasizes prosodic features (intensity, speech rate) and hostility markers. Cognitive disorganization classification distributes attention across syntactic complexity and discourse maintenance features.

Domain contribution analysis identifies lexical features (28% attention weight) and discourse features (25%) as primary drivers of classification, followed by prosodic (18%), syntactic (15%), and acoustic (14%) domains. This distribution aligns with clinical emphasis on thought disorder assessment in emergency psychiatric evaluation.

Feature importance analysis confirms that semantic coherence, syntactic complexity, speech rate, and hostility markers constitute the most discriminative individual features, consistent with established clinical assessment priorities.

Figure 4.Advanced clinical analysis and model interpretability. (A) Class-specific attention patterns across feature domains; (B) Temporal speech dynamics showing semantic coherence over time; (C) Model calibration curves comparing predicted and observed probabilities; (D) Proposed clinical integration workflow for emergency department implementation.

Given perfect classification performance, error analysis focused on examination of decision boundaries and near-miss cases in validation data. The model showed highest confidence (probability > 0.95) for clear-cut cases and appropriately calibrated uncertainty for samples near category boundaries. No systematic patterns of misclassification tendency were observed in validation monitoring.

The multi-scale domain-aware CNN achieved perfect classification performance (AUC = 1.000) across all diagnostic categories, demonstrating the feasibility of computational speech analysis for emergency psychiatric diagnosis. This performance substantially exceeds previously reported accuracies for automated psychiatric speech analysis, likely reflecting both the architectural innovations specifically domain-aware encoding and cross-domain attention and the controlled nature of synthetic data generation.

The superiority of deep learning over traditional machine learning (accuracy 1.000 vs. 0.971 for best baseline) indicates that complex, non-linear interactions between speech domains carry diagnostic information not captured by simpler feature combinations. The attention mechanism’s ability to weight domain contributions dynamically based on input characteristics represents a particular advance over static feature engineering approaches.

If validated on real clinical data, this framework could support several emergency department applications:

1) Rapid Diagnostic Screening: The model could analyse speech patterns during initial patient contact, providing objective probability estimates for diagnostic categories to guide clinical assessment. For patients with unclear presentations, this information could prompt targeted questioning or observation protocols.

2) Triage Prioritization: Automated risk stratification could identify patients requiring immediate psychiatric evaluation versus those appropriate for extended observation or medical workup. The substantial accuracy advantage supports clinically meaningful decision thresholds.

3) Safety Monitoring: Continuous speech analysis during emergency department stay could detect deterioration or emergence of agitation, enabling pre-emptive intervention before behavioural escalation.

4) Quality Assurance: Retrospective analysis of misclassified cases could identify systematic assessment failures and inform continuing education priorities.

Previous speech analysis studies in psychiatry have focused primarily on chronic schizophrenia or stable outpatient conditions, achieving accuracies of 70% - 85% against healthy controls [107]-[109]. Our focus on acute emergency presentations and multiclass discrimination represents a distinct application domain. The perfect performance observed here exceeds prior reports, though this reflects synthetic data characteristics rather than expected real-world performance.

The domain-aware architecture with explicit modelling of prosodic, lexical, syntactic, discourse, and acoustic features extends prior work typically focusing on single feature categories [110]-[112]. The integration of cross-domain attention to model interactions between speech dimensions represents a novel contribution with potential relevance for other clinical speech analysis applications.

Synthetic-Only Validation: All experiments employed synthetically generated data. While designed to reflect realistic psycholinguistic patterns, synthetic data cannot capture the full complexity of clinical speech production including idiosyncratic expressions, cultural and linguistic variation, and comorbid conditions affecting communication. External validation on real emergency department recordings is essential before clinical consideration.

Deterministic Class Separation: The synthetic generation process created nearly deterministic class boundaries based on programmed parameter differences. Real clinical speech exhibits substantial overlap between diagnostic categories, and perfect separation is neither expected nor observed in clinical practice. We anticipate real-world performance in the range of 75% - 85% accuracy based on published studies of clinical speech analysis.

Simplified Feature Representation: Our feature vectors reduce complex speech production to 48 dimensions. Real speech contains rich information in temporal dynamics, pragmatic context, and non-verbal vocalizations not fully captured by this representation. Future work should incorporate raw audio analysis and complete linguistic transcriptions.

Absence of Confounding Variables: Real clinical data includes numerous factors affecting speech production medication effects, medical comorbidities, fatigue, substance intoxication that were not modelled. These factors substantially complicate pattern learning in real applications.

Real-World Validation: Priority should be given to validating these models on recorded emergency department consultations. Partnerships with emergency departments and psychiatric crisis services could provide annotated datasets for external validation.

Raw Audio Integration: Future architectures should incorporate end-to-end learning from raw audio waveforms, potentially capturing markers missed by engineered features. Convolutional neural networks operating on spectrograms and wav2vec-style self-supervised pretraining represent promising directions.

Multimodal Fusion: Integration of speech analysis with video-based behavioural observation, vital signs, and electronic health record data could improve robustness and provide comprehensive assessment. Federated learning approaches may enable multi-site validation while preserving patient privacy [113]-[115].

Temporal Modelling: Extension to continuous monitoring during emergency department stay could track clinical evolution and predict deterioration. Online learning approaches would enable model adaptation to individual patient trajectories.

Interpretability Enhancement: While attention visualization provides insight, natural language generation of explanatory narratives could enhance clinical utility by describing specific speech patterns contributing to classification decisions.