In today’s wireless communications landscape, securing the physical layer has become an essential defense against the emergence of increasingly sophisticated and adaptive jamming threats [1]. While traditional frequency-hopping methods often rely on predefined switching sequences that are vulnerable to interception or prediction, the integration of artificial intelligence opens new avenues for enhanced resilience. This article explores an intelligent frequency-hopping strategy designed to counter malicious adversaries in dynamic and uncertain environments. By leveraging the power of Q-learning, a reinforcement learning algorithm, the system becomes capable of autonomously learning jamming patterns and selecting optimal transmission channels in real time, without prior knowledge of the attacker’s strategy [2].

To overcome the limitations of purely random exploration and avoid getting stuck in local optima, we introduce a stochastic dispersion mechanism. This approach allows us to diversify frequency hops probabilistically, ensuring maximum unpredictability against the adversary’s tracking attempts while minimizing packet collisions. Through this synergy between reinforcement learning and controlled randomness, our work aims to optimize transmission success rates and reduce energy consumption, thereby offering a robust and self-adaptive solution to secure the critical communication networks of tomorrow.

Securing the physical layer has traditionally relied on the intrinsic properties of the transmission channel to ensure confidentiality, moving away from purely cryptographic approaches used in higher layers. In this field, Frequency Hopping is a pioneering spread-spectrum technique, initially designed to combat interference and intentional jamming. However, foundational literature highlights a major limitation: the use of static pseudorandom sequences which, once intercepted by an adversary with advanced computational capabilities, render the system vulnerable [3].

The emergence of Smart Jammers, capable of learning and predicting hopping patterns, necessitated a shift toward proactive defense mechanisms. The introduction of reinforcement learning, and more specifically Q-Learning, marked a decisive turning point. Research by Wang et al. has demonstrated how an agent can optimize its channel selection policy by interacting with a hostile environment, modeled as a Markov Decision Process [4]. Unlike reactive methods, Q-Learning allows the transmitter to anticipate the jammer’s actions by maximizing a reward function linked to the signal-to-interference-plus-noise ratio. Nevertheless, a recurring issue in the literature concerns the trade-off between exploration and exploitation. Insufficient exploration causes the transmitter to remain on suboptimal frequencies, while excessive exploration degrades quality of service. To address this rigidity, recent work explores stochastic dispersion as a complement to artificial intelligence, suggesting that introducing controlled randomness into the agent's action space enhances the system’s unpredictability [5].

This approach relies on wave propagation properties and channel stochasticity to conceal the transmitter’s intentions. By incorporating a dispersion component, the system no longer simply follows the “best” learned frequency, but navigates within a subspace of secure frequencies, making any attempt at tracking mathematically complex [6]. The current literature thus converges toward hybrid architectures where reinforcement learning ensures adaptation to traffic conditions, while the stochasticity of the physical layer guarantees robustness against interception and traffic analysis [7].

The system is modeled as a Q-learning agent interacting with a dynamic and hostile radio environment. At each time step t , the transmitter observes the state of the spectrum s t ∈ S (interference levels on the channels) and selects an action a t ∈ A corresponding to a hopping frequency. The value function is updated according to the Bellman equation:

where the reward r is correlated with the success of the transmission (no collision with the jammer). To break the predictability of deterministic optimal policies, we incorporate a stochastic dispersion layer. Rather than systematically choosing the action a * = arg max Q ( s , a ) , the agent selects a frequency according to a Boltzmann probability distribution weighted by the channel entropy [8]. This approach allows the hops to be randomly dispersed within a subspace of high-reward frequencies, making the transmission pattern mathematically unpredictable to an external observer. The model’s convergence is evaluated by the bit error rate and the probability of interception under various reactive jamming configurations [9].

The system consists of a transmitter (Alice), a legitimate receiver (Bob), and an intelligent jammer (Eve) operating in a two-dimensional space. The signal received by Bob at time t on frequency f i is modeled by:

where P A is the transmit power, d A B is the distance, η is the path loss exponent, and h A B is the Rayleigh channel coefficient. The jamming interference J ( t ) depends on Eve’s collision strategy. Stochastic scattering exploits the random nature of the electromagnetic field. Based on Maxwell’s equations in a complex medium, the electric field E → resulting from the superposition of multiple paths follows a statistical distribution. The spectral power density can be described by a spatio-temporal correlation function:

This intrinsic stochasticity is used to parameterize the variance of channel selection. The channel is discretized into N orthogonal subbands. The system state s t is defined by the vector of received interference powers I = [ i 1 , i 2 , ⋯ , i N ] . The instantaneous Shannon channel capacity, which will serve as the basis for the Q-learning reward, is the

where P J is the jammer power and N 0 is the thermal noise. This model ensures that the agent does not simply learn a fixed sequence, but adapts to the physical dynamics of the signal and the channel uncertainty, thereby maximizing the probability of secure transmission against a reactive adversary.

In our intelligent frequency-hopping model, the objective function must balance three criteria: successful transmission, energy efficiency, and resistance to interference. We define a composite objective function given by:

(5)

where ω i are the normalized weighting coefficients. The first term, S t , represents transmission success, defined as the normalized bit rate achieved over the selected channel; it is maximized when the signal-to-interference-plus-noise ratio (SINR) exceeds a decoding threshold γ t h . The second term,

, penalizes the energy cost associated with frequency switching, since each fast hopping operation consumes computational and synchronization resources. Finally, C t is the collision penalty, activated when the selected frequency coincides with the jammer’s frequency band detected by the receiver [10]. To account for stochastic dispersion, we modify the reward structure by adding an entropy term:

where H is the entropy of the selection policy and β is a temperature parameter. This formulation encourages the agent to maintain a certain degree of variability in its frequency choices. A high reward is therefore not only assigned to the absence of jamming, but to a strategy that remains unpredictable (entropic) while remaining effective [11]. This design forces the Q-learning algorithm to converge toward a robust policy that discourages pattern prediction attacks, thereby ensuring optimal physical-layer security in a highly unstable environment.

Analysis of the results in Figure 1 obtained by the standard Q-learning algorithm reveals a clear advantage in terms of adaptability compared to conventional frequency hopping. In the early stages of the simulation, the error rate is unstable, corresponding to the exploration phase during which the agent randomly tests channels, including the jammer’s channel (150 Hz). However, after a small number of iterations, the cumulative reward curve shows logarithmic growth before stabilizing. This convergence indicates that the agent has correctly identified the attacker’s frequency signature and updated its Q-table to minimize the probability of selecting the compromised channel. Unlike the stationary system, which suffers from interference in a fatal and repetitive manner, the intelligent agent achieves a transmission success rate close to 100% in steady state.

Figure 1.A robust implementation of a standard Q-learning agent.

The final distribution of the Q values shows a marked contrast: the healthy frequencies (50, 100, 200, 250 Hz) exhibit positive and balanced values, while the 150 Hz frequency shows a strongly negative value, acting as a mathematical barrier. The effectiveness of the ϵ -greedy strategy is crucial here, as it allows for continuous monitoring of the environment; if the jammer were to change targets, the agent would be able to rediscover a new secure path. These results confirm that reinforcement learning transforms physical-layer security from a reactive approach into a proactive strategy. By dynamically optimizing spectrum usage, this method ensures not only confidentiality but also the operational resilience of communication networks in the face of persistent and localized threats.

Figure 2. Performance graph.

Analysis of performance metrics in Figure 2 confirms the superiority of the intelligent approach in terms of reliability and spectral efficiency. The first graph, illustrating the convergence rate, shows that the Q-learning agent stabilizes its strategy in fewer than 150 episodes. This rapid learning phase is crucial for real-time communications, as it limits the duration during which the system is vulnerable to collisions. Once convergence is reached, the cumulative average reward plateaus at its maximum value, proving that the algorithm has mathematically excluded the noisy channel from its usual decision space.

The second indicator, the success rate as a function of SNR, validates the system’s physical robustness. We observe that the success rate reaches a plateau of 100% as soon as the signal-to-noise ratio exceeds the 8 dB threshold, demonstrating excellent resilience not only against intentional jamming but also against ambient thermal noise. Unlike conventional frequency-hopping methods, which would suffer a constant 20% loss (with 1 out of 5 channels compromised), our model maintains optimal service continuity.

This synergy between artificial intelligence and physical layer modeling enables a constant Quality of Service, even under degraded channel conditions. In conclusion, these quantitative results support the idea that self-adaptation is the key to securing wireless networks against dynamic and unpredictable threats.

The algorithmic complexity of standard Q-learning lies primarily in updating the value table and selecting the action. For a state space S and a set of actions A , the spatial complexity is O ( | S | × | A | ) , which, in our case of frequency hopping with a single state (the current spectrum), reduces to O ( | A | ) . In terms of time, each iteration requires a search for the maximum max a ′ Q ( s ′ , a ′ ) , resulting in a complexity of O ( | A | ) .

The introduction of stochastic dispersion via the Softmax function adds an exponential calculation for each channel:

The total computation time per slot, T c a l c , must satisfy the real-time condition:

where T s l o t is the dwell time.

In practice, for | A | = 64 channels, the calculation takes only a few microseconds (μs), whereas standard FHSS slots are on the order of milliseconds (ms). The additional latency introduced by artificial intelligence is therefore negligible, ensuring that the selection of the next frequency is completed before the end of the current transmission. This efficiency allows inTable 2for maximum spectral agility to be maintained without degrading the bit rate, confirming the viability of the approach for securing the physical layer in highly dynamic environments. The critical analysis presented in the table highlights that, although standard Q-learning provides a robust proof of concept for anti-jamming, scaling it up to an industrial level requires structural adjustments. The combinatorial explosion of the state space in dense IoT networks justifies the transition to Deep Q-Learning, which is capable of handling continuous variables via neural networks. Furthermore, real-time viability will depend on hardware integration on FPGA chips, enabling the reduction of computational latency below the dwell time threshold.

Table 2. Critical analysis and the evolution of the intelligent frequency-hopping model.

Finally, security cannot be viewed as static; in the face of a “Smart Jammer” that also uses artificial intelligence, the adoption of Multi-Agent Game Theory (MARL) models becomes essential to maintaining a strategic advantage. Incorporating energy efficiency criteria into the reward function will enable this technology to be adapted to the strict constraints of autonomous sensors, thereby ensuring long-term and sustainable protection for next-generation networks.