Mood stabilizers constitute a fundamental component of pharmacological treatment for affective disorders, including bipolar disorder and treatment-resistant mood dysregulation [1]-[11]. Despite their widespread clinical use, determining an optimal dosage for individual patients remains a persistent challenge. Clinical response to mood stabilizers varies substantially across individuals due to differences in symptom severity, comorbid anxiety, pharmacodynamic sensitivity, and susceptibility to adverse effects [12]-[16]. As a result, standard dosing strategies largely guided by population-level heuristics and clinician experience often require prolonged trial-and-error titration, delaying symptom relief and increasing the risk of sub-optimal outcomes. Recent advances in data-driven medicine have motivated the exploration of adaptive treatment strategies that can account for patient heterogeneity and temporal dynamics. However, many existing computational approaches to psychiatric dosing rely on static regression or classification models that predict outcomes under fixed dosage assumptions. Such approaches are inherently limited in their ability to model sequential decision making, where current treatment choices influence future patient states, symptom trajectories, and clinical risk. This limitation is particularly relevant in mood disorders, where symptom evolution unfolds over time and treatment goals emphasize not only improvement but also sustained stabilization within a therapeutic range. Reinforcement learning (RL) provides a principled framework for addressing these challenges by framing treatment optimization as a sequential decision process. In an RL setting, an agent learns a dosing policy through interaction with an environment, receiving feedback in the form of rewards that encode clinical objectives and constraints. This paradigm enables explicit trade-offs between symptom control, treatment stability, and safety, and allows policies to adapt dynamically to patient-specific characteristics.

In this work, we propose an RL-based framework for personalized mood stabilizer dosing, using a clinically motivated simulation environment that captures longitudinal mood dynamics and inter-individual variability. The environment integrates latent patient factors depression severity, anxiety level, treatment responsiveness, and side-effect sensitivity and models non-linear dose response relationships over a finite titration horizon. We employ a tabular Q-learning agent as an interpretable baseline and evaluate performance through training dynamics, individual patient case studies, and population-level clinical metrics. Our results demonstrate the feasibility of RL for adaptive psychiatric dosing while highlighting key challenges that inform future translational research.

The application of reinforcement learning (RL) in healthcare has gained increasing attention as a promising paradigm for sequential clinical decision making [17]-[22]. Early work demonstrated the potential of RL for treatment planning in intensive care units, including ventilation control and sepsis management, where actions influence long-term patient outcomes rather than immediate rewards. These studies established RL as a suitable framework for optimizing dynamic treatment regimens under uncertainty and delayed clinical effects.

In psychiatry and psychopharmacology, computational approaches to medication optimization have traditionally relied on regression, classification, or survival models to predict treatment response based on baseline patient characteristics [23]-[26]. While informative, such static models are limited in their capacity to model treatment adaptation over time. More recent efforts have explored RL-based strategies for antidepressant selection, dosage adjustment, and symptom management, often using Markov decision processes to capture temporal dependencies [27]-[29]. However, many of these studies either employ simplified state representations or focus on narrow clinical objectives, limiting their applicability to mood stabilization tasks that require sustained control within a therapeutic range. Adaptive treatment strategies, such as dynamic treatment regimens and contextual bandits, have been proposed as intermediate approaches between static prediction and full RL. While contextual bandits allow personalization based on patient features, they typically ignore long-term consequences of treatment decisions and are therefore insufficient for modelling longitudinal mood trajectories and cumulative side effects. RL, by contrast, explicitly optimizes long-term outcomes and enables trade-offs between competing clinical goals. Parallel to these developments, digital phenotyping and computational psychiatry have introduced fine-grained behavioural and symptom measurements derived from clinical assessments, wearable sensors, and mobile devices [30]-[37]. These data streams have motivated interest in closed loop and adaptive treatment systems. Nevertheless, many existing studies stop at risk prediction or symptom monitoring rather than actionable treatment optimization.

In contrast to prior work, the present study integrates clinically motivated reward design, patient heterogeneity, and longitudinal mood dynamics within a unified RL framework for personalized mood stabilizer dosing. By emphasizing interpretability and comprehensive evaluation, this work contributes a methodologically grounded baseline for future advances in adaptive psychiatric treatment optimization.

Figure 1 summarizes the training dynamics of the reinforcement learning agent over 1000 episodes, illustrating convergence behaviour, mood regulation performance, and therapeutic stability.

Figure 1. Training performance overview.

Figure 1(a)(top-left) shows the total episode reward across training episodes, with a rolling average computed using a window size of 50 episodes. While individual episode rewards exhibit high variance due to stochastic patient sampling and exploration, the smoothed trajectory demonstrates a clear upward trend followed by stable convergence around positive values. This pattern indicates that the agent successfully learns a dosing policy that consistently balances symptom improvement, side-effect minimization, and dosage stability. Notably, no reward collapse or instability is observed, suggesting robust learning under heterogeneous patient conditions.

Figure 1(b)(top-right) presents the average mood score per episode. The red dashed line indicates the target mood level (50), and the shaded green region represents the therapeutic range (45 - 55). Over training, average mood trajectories progressively align with the desired clinical target, oscillating around the therapeutic region. Although episodic variability remains high-reflecting patient heterogeneity and stochastic transitions the long-term trend demonstrates improved regulation toward clinically meaningful mood levels.

Figure 1(c)(bottom-left) reports the percentage of time spent within the therapeutic range per episode. While a gradual increase is observed during training, overall occupancy remains relatively low compared to reward improvements. This result highlights a critical clinical challenge: achieving sustained mood stabilization is substantially more difficult than achieving transient mood improvement. The discrepancy between average mood alignment and therapeutic range occupancy underscores the importance of explicitly optimizing long-term stability rather than short-term gains.

Figure 1(d)(bottom-right) compares mean episode rewards between the first 200 and last 200 training episodes. The increase in average reward quantitatively confirms net learning progress and policy refinement over time, providing further evidence of convergence.

Overall, Figure 1 demonstrates that the proposed reinforcement learning framework achieves stable convergence and meaningful policy learning, while also revealing clinically relevant limitations in maintaining prolonged therapeutic stability. These observations motivate further analysis at the patient-specific and population levels and justify the subsequent evaluation presented in Sections 7 and 8.

To qualitatively assess the behaviour of the learned policy at the individual level, we analyse representative patient trajectories illustrating diverse clinical responses. Figure 2 presents four patient case studies, each visualized across three complementary dimensions: mood evolution, dosage adjustment, and cumulative reward. Rows correspond to individual patients, while columns represent different aspects of the treatment trajectory.

Figures 2(a)-(c)(Row1,Patient1): Patient 1 exhibits high depression severity and moderate treatment responsiveness.

Figure 2. Representative patient-level treatment trajectories.

Figure 2(a)(Left): Mood trajectory shows progressive deterioration away from the therapeutic range despite dosage adjustments, resulting in a negative overall improvement.Figure 2(b)(Middle): Dosage remains conservative and below the patient-specific optimal dose, reflecting the agent’s attempt to avoid side effects in a high-risk profile.Figure 2(c)(Right): Cumulative reward increases steadily, indicating that while mood stabilization is unsuccessful, the agent still optimizes safety and stability constraints.

Figures 2(d)-(f)(Row2,Patient11): Patient 11 represents a moderate-severity, high-responsiveness case.

Figure 2(d): Mood improves consistently from baseline, approaching but not fully entering the therapeutic range.Figure 2(e): Dosage adjustments remain stable and conservative relative to the optimal dose.Figure 2(f): Reward accumulation is strong and monotonic, reflecting effective balancing of symptom improvement and safety.

Figure 2(g)-(i)(Row3,Patient21): Patient 21 demonstrates intermediate depression severity with moderate responsiveness.

Figure 2(g): Mood initially improves but subsequently declines, failing to maintain stabilization.Figure 2(h): Dosage remains significantly below the optimal level, limiting therapeutic effect.Figure 2(i): Cumulative reward remains high, illustrating that reward optimization does not always correspond to sustained mood control.

Figures 2(j)-(l)(Row4,Patient31): Patient 31 represents a low-responsiveness, high-risk profile.

Figure 2(j): Mood deteriorates rapidly and remains outside the therapeutic range.Figure 2(k): The agent performs cautious dosage adjustments, avoiding aggressive escalation despite poor mood outcomes.Figure 2(l): Reward accumulation plateaus early, reflecting limited achievable benefit under the given patient constraints.

Overall, Figure 2 highlights the heterogeneous treatment outcomes produced by the learned policy. Successful stabilization is observed primarily in patients with moderate symptom severity and higher treatment responsiveness, whereas patients with severe depression or low responsiveness present persistent challenges. Importantly, these case studies demonstrate that the agent learns clinically conservative behaviour, prioritizing safety and stability even when symptom improvement is limited. This aligns with real-world psychiatric decision-making and underscores the importance of individualized policy adaptation.

To quantitatively assess the performance of the proposed reinforcement learning framework across a broader patient population, we conducted a comprehensive evaluation on 200 held-out synthetic patients. Figure 3 summarizes population-level outcomes using six complementary analyses capturing treatment effectiveness, stability, personalization, and dosing accuracy.

Figure 3(a)(Top-Left): Distribution of Mood Improvement. This panel shows the distribution of mood improvement from baseline to the final time step. The vertical dashed line at zero indicates no change, while the solid green line marks the mean improvement (+20.3 points). The distribution is positively skewed, with most patients experiencing clinically meaningful improvement, although a non-negligible subset exhibits deterioration. This heterogeneity reflects differences in baseline severity and treatment responsiveness.

Figure 3. Population-level performance and clinical outcomes.

Figure 3(b)(Top-Middle):TimeinTherapeuticRangeDistribution. The histogram depicts the percentage of time each patient spends within the therapeutic range (45 - 55). The red dashed line indicates the clinical benchmark of 50% occupancy, while the blue line denotes the observed mean (16.8%). Most patients fall below the target threshold, underscoring the difficulty of sustained stabilization despite overall mood improvement.

Figure 3(c)(Top-Right):Baselinevs.FinalMood. Each point represents a patient, with colour indicating time spent in the therapeutic range. The diagonal line denotes no change. Many patients lie above this line, confirming net improvement; however, only a subset cluster within the shaded therapeutic band. This visualization highlights the distinction between symptom improvement and long-term stabilization.

Figure 3(d)(Bottom-Left):DepressionSeverityvs.MoodImprovement. Mood improvement is plotted against baseline depression severity and coloured by total cumulative reward. Patients with lower to moderate severity tend to achieve larger improvements, while high-severity patients show more variable outcomes. The colour gradient illustrates how reward optimization does not uniformly translate into clinical success across severity strata.

Figure 3(e)(Bottom-Middle):DoseAccuracyAnalysis. Final prescribed doses are compared against patient-specific optimal doses. The dashed diagonal represents perfect accuracy. While some clustering near the diagonal is observed, dispersion remains substantial, indicating that the agent often adopts conservative dosing strategies rather than aggressively converging to the latent optimum.

Figure 3(f)(Bottom-Right):PerformanceCategoryDistribution. Patients are categorized based on time spent in the therapeutic range: Excellent (≥70%), Good (50% - 69%), Fair (30% - 49%), and Poor (<30%). The majority fall into the Poor category (81.5%), with only a small fraction achieving sustained stabilization. This distribution emphasizes the clinical challenge of maintaining long-term mood control.

Collectively, Figure 3 demonstrates that the proposed RL agent achieves substantial mood improvement at the population level, while revealing clear limitations in sustained therapeutic stabilization and dose precision. These results provide critical insight into both the strengths and boundaries of the current approach and motivate future work incorporating richer state representations, longer horizons, and more expressive function approximators. To assess robustness of the learned policy, we conducted a sensitivity analysis by varying the relative weights assigned to mood stabilization and side-effect penalties in the reward function. Increasing the weight of the side-effect penalty led to more conservative dosing behaviour and reduced time spent within the therapeutic range. Conversely, increasing the stabilization weight improved sustained therapeutic occupancy but slightly increased dose variability. These findings indicate that time in therapeutic range is sensitive to reward design, highlighting the importance of clinically grounded weight selection in reinforcement learning based dosing frameworks.

The proposed reinforcement learning framework was evaluated on a held-out cohort of 200 synthetic patients to assess treatment effectiveness, stability, and personalization at the population level. Overall, the results demonstrate that the agent learns clinically meaningful dosing policies while revealing important limitations in sustained stabilization. Across the evaluated cohort, the agent achieved a mean mood improvement of +20.3 points, indicating substantial symptom reduction relative to baseline. Notably, 68.5% of patients exhibited positive mood improvement, suggesting that the learned policy generalizes across heterogeneous patient profiles rather than benefiting only a narrow subset. These findings confirm that the agent consistently identifies dosing strategies that move patients toward improved clinical states.

However, improvements in average mood did not translate into prolonged stabilization. The average time spent within the therapeutic range was 16.8%, and only a minority of patients achieved sustained occupancy above clinically desirable thresholds. This gap highlights a critical distinction between short-term symptom improvement and long-term mood regulation. The result reflects both the finite decision horizon and the conservative dosing behaviour adopted by the agent to mitigate side effects.

Dose selection accuracy further illustrates this trade-off. Only 15.0% of final doses fell within 20% of the patient-specific optimal dose, indicating that the agent frequently favoured under-dosing relative to the latent optimum. While this behaviour limits maximal therapeutic effect, it aligns with a clinically cautious strategy that prioritizes safety in the presence of uncertainty and side-effect sensitivity.

Taken together, these results demonstrate that the proposed RL framework achieves robust personalization and meaningful symptom improvement, while exposing persistent challenges in sustained therapeutic control and precise dose convergence. Rather than indicating failure, these limitations provide valuable insight into the complexity of adaptive psychiatric dosing and motivate methodological extensions such as longer horizons, richer state representations, and function-approximation based policies.

The results of this study highlight both the potential and the current limitations of reinforcement learning–based approaches for personalized psychiatric dosing [53]-[55]. The proposed framework demonstrates that even a relatively simple tabular Q-learning agent can learn directional and clinically conservative dosing strategies, leading to substantial mood improvement across a heterogeneous patient population [56]-[62]. The consistent positive rewards and population-level symptom reduction indicate that the agent successfully internalizes key clinical trade-offs encoded in the reward structure, particularly the balance between efficacy and safety. At the same time, the findings reveal persistent challenges in achieving sustained mood stabilization within a predefined therapeutic range. Although average mood levels improve, time spent within the therapeutic interval remains limited, especially for patients with high baseline depression severity or low treatment responsiveness. This behaviour reflects a fundamental tension in adaptive dosing: aggressive dose escalation may improve symptoms but increases the risk of side effects and instability, whereas conservative strategies favour safety at the cost of long-term control. Importantly, the agent’s tendency toward cautious dosing mirrors real-world clinical practice, where uncertainty and patient risk profiles often constrain titration decisions. The observed performance patterns also underscore the representational limits of tabular reinforcement learning in complex, continuous clinical environments. Coarse state discretization and short decision horizons restrict the agent’s ability to anticipate delayed treatment effects and to differentiate subtle patient subgroups. Consequently, while the learned policy generalizes across patients, it lacks the expressiveness required for fine-grained stabilization.

Overall, these results suggest that reinforcement learning is a viable framework for adaptive psychiatric treatment optimization, but that clinically meaningful stabilization requires richer modelling of patient dynamics, longer temporal reasoning, and more expressive policy representations. The present work therefore serves as a methodological baseline rather than a definitive solution, providing insight into both achievable gains and remaining gaps.

Several limitations of the present study should be acknowledged. First, all evaluations are conducted in a synthetic environment. While the simulated cohort captures key statistical and dynamical properties of affective disorder populations, it cannot fully replicate the complexity, noise, and confounding present in real clinical settings. As such, no claims of direct clinical deployment are made. Second, the pharmacodynamic model used to simulate mood evolution is intentionally simplified, relying on a single bell-shaped dose–response function and aggregate side-effect modelling [63]-[65]. Real-world treatment response is influenced by additional factors such as delayed drug effects, adherence variability, comorbidities, and drug–drug interactions, which are not explicitly modelled here. Third, the use of tabular Q-learning with discretized state space limits representational capacity and scalability. While this choice improves interpretability and stability, it constrains the agent’s ability to capture fine-grained patient differences and long-term dependencies. Additionally, the current side-effect model assumes overdose-dependent toxicity and does not capture stochastic adverse events at lower doses.

Future work will address these limitations by incorporating function approximation via deep reinforcement learning, enabling continuous state representations and longer decision horizons. Additional extensions include uncertainty-aware policies, risk-sensitive reward formulations, and clinician-in-the-loop constraints to ensure safety and interpretability. Finally, validation on retrospective longitudinal clinical datasets will be essential for assessing translational potential and guiding prospective evaluation.

This study demonstrates the feasibility of reinforcement learning as a framework for personalized mood stabilizer dosing in affective disorders. By formulating treatment optimization as a sequential decision-making problem, the proposed approach captures longitudinal mood dynamics and inter-individual variability that static prediction models are unable to represent. The reinforcement learning agent learns adaptive dosing strategies that balance symptom improvement, therapeutic stability, and safety, reflecting clinically meaningful trade-offs. Through extensive simulation and evaluation, the framework shows consistent mood improvement across a heterogeneous patient population and exhibits robust personalization behaviour. At the same time, the results highlight the inherent difficulty of achieving sustained stabilization within a narrow therapeutic range, particularly in patients with high baseline severity or limited treatment responsiveness. These findings underscore the importance of modelling treatment as a dynamic process rather than a single-point prediction task. Importantly, this work does not aim to provide a deployable clinical solution. Instead, it establishes a methodologically sound baseline for adaptive psychiatric treatment optimization and offers insight into the strengths and limitations of reinforcement learning in this domain. The observed performance patterns motivate future extensions incorporating richer state representations, longer decision horizons, uncertainty-aware policies, and integration of real-world clinical data. Overall, the proposed framework provides a principled foundation for future translational research at the intersection of reinforcement learning and computational psychiatry. By bridging algorithmic decision-making with clinically grounded objectives, this study contributes toward the development of intelligent, patient-cantered treatment support systems for mental health care.