We design a communication network of $J$ agents distributed over a geographic area. Mathematically, we represent the communication network by a graph $\mathcal{G}=\left(\mathcal{V},ℰ,A\right)$ of order $J$ with $\mathcal{V}=\left\{1,\cdots ,J\right\}$ depicting the set of vertices, $ℰ\subseteq \mathcal{V}\times \mathcal{V}$ representing the edge set, and $A=\left[a_{ij}\right]\in {ℝ}^{J\times J}$ illustrates the weighted adjacency matrix, where $a_{ij}>0$ if $\left(j,i\right)\in ℰ$ and $a_{ij}=0$, otherwise. The graph has no loops or multiple edges. If there is a connection between node $i$ and node $j$, $\left(j,i\right)\in ℰ$. The set of neighbors of node $i$ is represented by ${\mathcal{N}}_i=\left\{j:\left(j,i\right)\in ℰ\right\}$. Define the Laplacian matrix $ℒ=\left[l_{ij}\right]\in {ℝ}^{J\times J}$, where $l_{ij}=-a_{ij}$ for $i\ne j$ and $l_{ii}={\displaystyle {\sum }_{j=1}^J{a}_{ij}}$. The matrix $ℒ$ is symmetric and positive semi-definite, with all eigenvalues being non-negative.

Lemma 1 (graphs) For an undirected connected graph $\mathcal{G}$ with $J$ vertices, the eigenvalues of $ℒ$ contains only one zero entry. $0={\lambda }_1\left(ℒ\right)$ possessing a right eigenvector, $1$, i.e., $ℒ1=0$, and other have positive real parts. Suppose $0={\lambda }_1\left(ℒ\right)<{\lambda }_2\left(ℒ\right)\le \cdots \le {\lambda }_J\left(ℒ\right)={\lambda }_{\max }\left(ℒ\right)$ in an ascending order. The minimum nonzero eigenvalue ${\lambda }_2\left(ℒ\right)={\min }_{x\ne 0,1^{\top }x=0}\frac{x^{\top }ℒx}{x^{\top }x}$, where $x={\left[x_1,\cdots ,x_J\right]}^{\top }$. We have

$x^{\top }ℒx=\frac{1}{2}\underset{i=1}{\overset{J}{{\displaystyle \sum }}}\underset{j=1}{\overset{J}{{\displaystyle \sum }}}{a}_{ij}{\left(x_j-x_i\right)}^2.$(1)

Furthermore, if $1^{\top }x=0$, then $x^{\top }ℒx\ge {\lambda }_2\left(ℒ\right)x^{\top }x$.

Lemma 2 (nonsingular) Let ${\kappa }_1,{\kappa }_2,\cdots ,{\kappa }_J\ge 0$. Then

${\left(\underset{i=1}{\overset{J}{{\displaystyle \sum }}}{\kappa }_i\right)}^p\le \underset{i=1}{\overset{J}{{\displaystyle \sum }}}{\kappa }_i^p\le J^{1-p}{\left(\underset{i=1}{\overset{J}{{\displaystyle \sum }}}{\kappa }_i\right)}^p$(2)

if $0<p\le 1$, and

$J^{1-p}{\left(\underset{i=1}{\overset{J}{{\displaystyle \sum }}}{\kappa }_i\right)}^p\le \underset{i=1}{\overset{J}{{\displaystyle \sum }}}{\kappa }_i^p\le {\left(\underset{i=1}{\overset{J}{{\displaystyle \sum }}}{\kappa }_i\right)}^p$(3)

if $1<p<\infty$.

ESN consists of three parts: an input layer, a hidden layer (reservoir), and an output layer. It is a variant of feedforward neural networks, with the training process involving feeding back output results to the hidden layer to adjust connection weights. The input vector $x_n\in {ℝ}^{D_{\text{in}}}$, with a dimensionality of $D_{\text{in}}$, is fed into a reservoir of dimensionality $D_{\text{res}}$. The internal state of the reservoir, $g_n\in {ℝ}^{D_{\text{res}}}$, is computed as follows:

$g_n=f_{\text{res}}\left({\beta }_{\text{in}}^{\text{res}}{x}_n+{\beta }_{\text{res}}^{\text{res}}{g}_{n-1}+{\beta }_{\text{out}}^{\text{res}}{y}_{n-1}\right),$(4)

where ${\beta }_{\text{in}}^{\text{res}}\in {ℝ}^{D_{\text{res}}\times D_{\text{in}}},{\beta }_{\text{res}}^{\text{res}}\in {ℝ}^{D_{\text{res}}\times D_{\text{res}}}$ and ${\beta }_{\text{out}}^{\text{res}}\in {ℝ}^{D_{\text{res}}\times D_{\text{out}}}$ are matrices generated randomly. The nonlinear function $f_{\text{res}}(\cdot )$ is applied, where $y_{n-1}\in {ℝ}^{D_{\text{out}}}$ denotes the output from the previous time step. The output at the current step is then calculated as

$y_n=f_{\text{out}}\left({\beta }_{\text{in}}^{\text{out}}{x}_n+{\beta }_{\text{res}}^{\text{out}}{g}_n\right).$(5)

Depending on the specific dataset, the parameters ${\beta }_{\text{in}}^{\text{out}}\in {ℝ}^{D_{\text{out}}\times D_{\text{in}}}$ and ${\beta }_{\text{res}}^{\text{out}}\in {ℝ}^{D_{\text{out}}\times D_{\text{res}}}$ are adapted, so as an invertible nonlinear function $f_{\text{out}}(\cdot )$. To train the readout, assume that there is a sequence of $N$ input-output pairs $\left(x_1,y_1\right),\cdots ,\left(x_N,y_N\right)$. The inputs are mapped to the reservoir, generating a series of internal states $g_1,\cdots ,g_N$. At this stage, due to the unavailability of the ESN output for feedback, the designated output is substituted in (4). We obtain the hidden matrix $H$ and output matrix $S$ as

$H=\left[\begin{array}{c}{x}_1^{\top },g_1^{\top }\\ ⋮\\ x_N^{\top },g_N^{\top }\end{array}\right],S=\left[\begin{array}{c}{f}_{\text{out}}^{-1}\left(y_1^{\top }\right)\\ ⋮\\ f_{\text{out}}^{-1}\left(y_N^{\top }\right)\end{array}\right].$(6)

The optimal weight vector for the output is obtained by solving the subsequent regularized least-squares problem:

${\beta }^{\text{*}}=\underset{\beta }{\arg \min }\left\{\frac{1}{2}{\Vert H\beta -S\Vert }^2+\frac{C}{2}{\Vert \beta \Vert }^2\right\},$(7)

where $\beta ={\left[{\beta }_{\text{in}}^{\text{out}},{\beta }_{\text{res}}^{\text{out}}\right]}^{\top }\in {ℝ}^{\left(D_{\text{in}}+D_{\text{res}}\right)\times D_{\text{out}}}$ and $C>0$ is a regularization parameter. The solution to the problem in (7) can be written as

${\beta }^{\text{*}}={\left(H^{\top }H+C{I}_L\right)}^{-1}{H}^{\top }S,$(8)

where $L=D_{\text{in}}+D_{\text{res}}$.

To transform the centralized ESN into a distributed ESN, at the outset, we break down the centralized ESN task into several subtasks, each corresponding to a specific agent. Obtaining the optimal solution for the regularized problem leads to the determination of the output weights ${\beta }^{\text{*}}$.

$\begin{array}{ll}\underset{\left\{{\beta }_i\right\},\left\{{\xi }_{n,i}\right\}}{\text{min}}\hfill & \frac{1}{2}\underset{i=1}{\overset{J}{{\displaystyle \sum }}}\underset{n=1}{\overset{N_i}{{\displaystyle \sum }}}{\Vert {\xi }_{n,i}\Vert }^2+\frac{C}{2J}\underset{i=1}{\overset{J}{{\displaystyle \sum }}}{\Vert {\beta }_i\Vert }^2\hfill \\ \text{s}.\text{t}.\hfill & {\beta }_i^{\top }{h}_{n,i}=s_{n,i}^{\top }-{\xi }_{n,i}^{\top },\forall i\in \mathcal{V},n=1,\cdots ,N_i,\hfill \\ \hfill & {\beta }_i={\beta }_j,\forall i\in \mathcal{V},j\in {\mathcal{N}}_i.\hfill \end{array}$(9)

The training error vector for the output neurons, corresponding to the training example $x_{n,i}$ at the $i$-th learning agent, is denoted by ${\xi }_{n,i}\in {ℝ}^{D_{\text{out}}}$. To convert a centralized ESN into a fully distributed ESN, the overall output weights $\beta$ are first duplicated and assigned to each agent. Equality constraints are then enforced along the edges of the communication network. By incorporating the constraints into the objective function, problem (9) can be reformulated as

$\begin{array}{ll}\underset{\left\{{\beta }_i\right\}}{\text{min}}\hfill & \underset{i=1}{\overset{J}{{\displaystyle \sum }}}\left(\frac{1}{2}{\Vert H_i{\beta }_i-S_i\Vert }^2+\frac{C}{2J}{\Vert {\beta }_i\Vert }^2\right)\hfill \\ \text{s}.\text{t}.\hfill & {\beta }_i={\beta }_j,\forall i\in \mathcal{V},j\in {\mathcal{N}}_i,\hfill \end{array}$(10)

where the hidden matrix $H_i$ and the training target matrix $S_i$ are autonomously acquired from the respective dataset ${\mathcal{D}}_i$ observed by agent $i$ itself. For simplicity in presentation, we begin by simplifying problem (10) as

${\beta }^{\text{*}}=\underset{\beta }{\arg \min }\left\{\underset{i=1}{\overset{J}{{\displaystyle \sum }}}{u}_i\left(\beta \right)\triangleq u\left(\beta \right)\right\},$(11)

where

$u_i\left(\beta \right)=\frac{1}{2}{\Vert H_i\beta -S_i\Vert }^2+\frac{C}{2J}{\Vert \beta \Vert }^2$(12)

is the individual function performed by agent $i$ locally. Agent $i$ is capable of independently optimizing its local objective and establishing local consensus constraints through communication with neighboring agent $j$.

The ZGS algorithm aims to achieve optimal consensus on a manifold where the sum of the gradients of local functions is consistently zero. Let ${\beta }_i\left(t\right)$ denote the state of agent $i$ estimating the unknown optimum ${\beta }^{\text{*}}$ at iterative step $k$, with ${\beta }_i\left(0\right)$ indicating the initial value. The objective is to drive ${\beta }_i\left(t\right)$ of each agent asymptotically to ${\beta }^{\text{*}}$, i.e., assuming convexity in the objective function, the global objective can be optimally achieved uniquely. According to the form of the objective function, it is known that it is a convex function. The global objective can be uniquely achieved optimally.

Assumption 1 For each $i\in \mathcal{V}$, the function $u_i$ demonstrates strong convexity and maintains twice continuous differentiability.

Under Assumption 1, there is a unique minimizer ${\beta }_i^{\text{*}}$, i.e.,

${\beta }_i^{\text{*}}={\left(H_i^{\top }{H}_i+\frac{C}{J}{I}_L\right)}^{-1}{H}_i^{\top }{S}_i$(13)

such that the gradient $\nabla u_i\left({\beta }_i^{\text{*}}\right)=0$. The ZGS algorithm is formulated as an iterative procedure under the assumption of a zero gradient for $u_i$. The following implicit distributed algorithm is considered to achieve the goal.

$\begin{array}{l}\nabla u_i\left({\beta }_i\left(t+1\right)\right)-\nabla u_i\left({\beta }_i\left(t\right)\right)\\ ={\gamma }_1\underset{j\in {\mathcal{N}}_i}{{\displaystyle \sum }}{a}_{ij}{\left({\beta }_j\left(t\right)-{\beta }_i\left(t\right)\right)}^{\left[{\alpha }_1\right]}+{\gamma }_2\underset{j\in {\mathcal{N}}_i}{{\displaystyle \sum }}{a}_{ij}{\left({\beta }_j\left(t\right)-{\beta }_i\left(t\right)\right)}^{\left[{\alpha }_2\right]}\end{array}$(14)

with ${\beta }_i\left(0\right)={\beta }_i^{\text{*}}$, where ${\gamma }_1,{\gamma }_2>0,0<{\alpha }_1<1,{\alpha }_2>1$ and $a_{ij}$ represents entries in the weighted adjacency matrix $A$ of graph $\mathcal{G}$. Under the assumption that the graph $\mathcal{G}$ is undirected and connected, we have

$\underset{i=1}{\overset{J}{{\displaystyle \sum }}}\left(\nabla u_i\left({\beta }_i\left(t+1\right)\right)-\nabla u_i\left({\beta }_i\left(t\right)\right)\right)=0.$(15)

Assuming ${\beta }_i\left(0\right)={\beta }_i^{\text{*}}$, we have

$\underset{i=1}{\overset{J}{{\displaystyle \sum }}}\nabla u_i\left({\beta }_i\left(0\right)\right)=\underset{i=1}{\overset{J}{{\displaystyle \sum }}}\nabla u_i\left({\beta }_i^{\text{*}}\right)=0.$(16)

Consequently, the consensus value ${\beta }^o$ must be the optimizer ${\beta }^{\text{*}}$. In such sense, a shared ${\beta }^{\text{*}}$ is determined for each agent, ensuring $\nabla u\left({\beta }^{\text{*}}\right)={\displaystyle {\sum }_{i=1}^J\nabla u_i}\left({\beta }^{\text{*}}\right)=0$. So that (10) holds. Furthermore, the gradient of the function $u_i\left(\beta \right)$ with respect to $\beta$ can be easily computed as $\nabla u_i\left(\beta \right)=\left(H_i^{\top }{H}_i+\frac{C}{J}{I}_L\right)\beta -H_i^{\top }{S}_i$. The Hessian of $u_i\left(\beta \right)$ is

${\nabla }^2{u}_i\left(\beta \right)=H_i^{\top }{H}_i+\frac{C}{J}{I}_L\triangleq {\tilde{\Omega }}_i.$(17)

The matrix ${\tilde{\Omega }}_i$ is static, nonsingular, symmetric and positive definite. Letting ${\theta }_i={\lambda }_{\text{min}}\left({\tilde{\Omega }}_i\right)>0$ and ${\text{Θ}}_i={\lambda }_{\text{max}}\left({\tilde{\Omega }}_i\right)>0$ denote the minimum and maximum eigenvalues of ${\tilde{\Omega }}_i$, respectively, then

${\theta }_i{I}_L\underset{\_}{\prec }{\tilde{\Omega }}_i\underset{\_}{\prec }{\text{Θ}}_i{I}_L.$(18)

From the gradient of $u_i\left(\beta \right)$, we have $\nabla u_i\left({\beta }_i\left(t+1\right)\right)-\nabla u_i\left({\beta }_i\left(t\right)\right)$ $={\tilde{\Omega }}_i\left({\beta }_i\left(t+1\right)-{\beta }_i\left(t\right)\right)$. Then, ${\beta }_i\left(t+1\right)-{\beta }_i\left(t\right)={\tilde{\Omega }}_i^{-1}\left(\nabla u_i\left({\beta }_i\left(t+1\right)\right)-\nabla u_i\left({\beta }_i\left(t\right)\right)\right)$. Substituting this expression into the left-hand side of (14), an algorithm that iteratively solves the problem in a distributed manner is designed. For each agent $i\in \mathcal{V}$, we design an algorithm for distributed ESN (ZGS-ESN). The ZGS-ESN iterates as follows:

$\begin{array}{c}{\beta }_i\left(t+1\right)={\beta }_i\left(t\right)+{\gamma }_1{\tilde{\Omega }}_i^{-1}\underset{j\in {\mathcal{N}}_i}{{\displaystyle \sum }}{a}_{ij}{\left({\beta }_j\left(t\right)-{\beta }_i\left(t\right)\right)}^{\left[{\alpha }_1\right]}\\ +{\gamma }_2{\tilde{\Omega }}_i^{-1}\underset{j\in {\mathcal{N}}_i}{{\displaystyle \sum }}{a}_{ij}{\left({\beta }_j\left(t\right)-{\beta }_i\left(t\right)\right)}^{\left[{\alpha }_2\right]}\end{array}$(19)

where ${\gamma }_1,{\gamma }_2>0,0<{\alpha }_1<1,{\alpha }_2>1$, with initial values

${\beta }_i\left(0\right)={\tilde{\Omega }}_i^{-1}{H}_i^{\top }{S}_i,$(20)

where the matrix ${\tilde{\Omega }}_i$ is defined in (17). However, the continuous updating strategy may result in elevated communication consumption and frequent updates. The dynamic event-triggered strategy is an effective approach to overcome these disadvantages, in which the information interactions only happen at the trigger instants.

The ZGS-FxTdt-ESN algorithm was validated on synthetic dataset designed for nonlinear system identification and the prediction of chaotic time series. For a large-scale simulation analysis, we utilize datasets that are approximately one to two orders of magnitude larger than those used in earlier studies. To be specific, for each dataset, we create 50 sequences, each comprising 2,000 elements. These sequences start from varying initial conditions, resulting in a total of 100,000 samples for each experiment.

The Mackey-Glass chaotic time-series (referred to as MKG) dataset is used for prediction tasks. Characterized in continuous time, this series follows the subsequent differential equation:

${\stackrel{\dot{}}{x}}_n=-0.1x_n+\frac{0.2x_{n-\tau }}{1+x_{n-30}^{10}}.$(26)

For $\tau >16.8$, the time-series given by (26) undergoes integration using a fourth-order Runge-Kutta method with a time step of 0.1. Subsequently, it is sampled every 10 time-instants. The task then becomes a 10-step-ahead prediction problem

$y_n=x_{n+10}.$(27)

The functional form of Geometric Brownian Motion (GBM) is: $S_t=S_{t-1}\cdot {\text{e}}^{\left(\mu -\frac{1}{2}{\sigma }^2\right)dt+\sigma dW}$. In this model: $\mu =0.0003$ represents the drift rate, governing the long-term trend of the asset price; $\sigma =0.005$ denotes the volatility, determining the magnitude of price fluctuations; $S\left(t\right)$ is the asset price at time $t$; $dW\left(t\right)$ is the increment of Brownian motion over an infinitesimal time interval $dt$. Based on this model, we will generate 50 time series, each containing 2000 consecutive time steps of simulated asset price data.

We create an agent network employing a model with a randomly generated topology with a 55% connectivity rate. Our experimentation involves varying the number of agents, starting from 4 and incrementing by 4 up to 20. To assess testing errors, we employ a 3-fold cross-validation on the initial set of 50 sequences. Each sequence is 2000 units in length. In every fold of the cross-validation process, the training sequences are distributed across the agents, after which we assess the performance of the four algorithms listed below:

Each of the algorithms employs an identical ESN architecture, which is detailed in the following subsection. The threefold cross-validation process is iterated 10 times with variations in ESN initialization and data partitioning . Errors from each iteration and fold are accumulated. The trained ESN is applied to the test sequences to compute the error , and the resulting predicted outputs ${\tilde{y}}_1,\cdots ,{\tilde{y}}_{N_{\text{test}}}$ are compiled. In this context, $N_{\text{test}}$ denotes the count of testing samples excluding the initial washout elements from the test sequences. The normalized root mean-square error (N-RMSE) is defined as follows:

$\text{NRMSE}=\sqrt{\frac{{{\displaystyle \sum }}_{n=1}^{N_{\text{test}}}{\left({\tilde{y}}_n-y_n\right)}^2}{N_{\text{test}}{\hat{\sigma }}_t}}.$(28)

In this context, ${\hat{\sigma }}_t$ signifies an empirical estimate of the variance in the actual output samples $y_1,\cdots ,y_{N_{\text{test}}}$, where $N_{\text{test}}$ is the count of testing samples, and ${\tilde{y}}_1,\cdots ,{\tilde{y}}_{N_{\text{test}}}$, denote the predicted outputs.

A regularization factor of $C=10$ and a reservoir size of $D_{\text{res}}=300$ were selected, and these choices proved to be effective in all situations. In the reservoir, we employ $\text{tanh}(\cdot )$ nonlinearities, and for the output function, a scaled identity $f\left(s\right)={\alpha }_{\text{t}}s$ is utilized. In all instances, the parameter ${\alpha }_{\text{t}}$ is set by searching within the set $\left\{0.1,0.3,\cdots ,0.9\right\}$. The matrix ${\beta }_{\text{in}}^{\text{res}}$ is populated with entries drawn from a uniform distribution within the range $\left[-{\alpha }_{\text{i}},{\alpha }_{\text{i}}\right]$. The parameter ${\alpha }_{\text{i}}$ is explored within the same range as ${\alpha }_{\text{t}}$. The matrix ${\beta }_{\text{out}}^{\text{res}}$, which links the output to the reservoir, is initialized as a complete matrix with entries drawn from a uniform distribution within the range $\left[-{\alpha }_{\text{f}},{\alpha }_{\text{f}}\right]$. The parameter ${\alpha }_{\text{f}}$ is searched within the same range as ${\alpha }_{\text{t}}$. ${\alpha }_{\text{f}}=0$ is allowed for the case in which no output feedback is required.

The reservoir matrix ${\beta }_{\text{res}}^{\text{res}}$ is initialized using a uniform distribution in the range of $\left[-1,1\right]$. Subsequently, around 55% of its connections are set to zero to foster sparsity. Lastly, the matrix is adjusted in scale to achieve a target spectral radius $\rho$, with $\rho$ being explored within the same range as ${\alpha }_{\text{t}}$. Furthermore, uniform noise is introduced during the state update of the reservoir, where the noise is sampled from a uniform distribution within the interval $\left[0,{10}^{-3}\right]$. Additionally, the initial $D=100$ elements are excluded from each sequence. Table 1 displays the ultimate configurations obtained through the grid-search process.

The test outputs of the four algorithms are presented in Figure 1 , which demonstrate the prediction capability of the ESN. ZGS-FxTdt-ESN consistently follows the performance of the centralized solution across all scenarios. In Figure 1 , the output value of ZGS-FxTdt-ESN basically coincides with the actual output value, indicating that ZGS-FxTdt-ESN maintains a relatively accurate output value. With the increasing number of agents, five algorithms show their test errors in Figure 2 . We can see the test error of ZGS-FxTdt-ESN closely matches that of ZGS-FxTet-ESN. This shows that ZGS-FxTdt-ESN has the same test error as ZGS-FxTet-ESN, but ZGS-FxTdt-ESN has more advantages. Not only can its accuracy be comparable to that of ZGS-FxTet-ESN, but it also has a faster convergence speed. Figure 1 and Figure 2 indicate that the output values of C-ESN, ZGS-ESN and ZGS-FxTdt-ESN coincide and their test errors are all identical to that of ZGS-FxTet-ESN , which all suggests that their accuracies are similar. It has a faster convergence speed and only updates the state at the moment when the trigger condition is reached, which can save communication costs more. As shown in Figure 3 , ZGS-FxTet-ESN achieves a significantly shorter training time than ZGS-FxTdt-ESN, primarily due to the latter’s need for continuous internal-state monitoring. In Figure 4 , the event-triggered instants of agents are depicted, and it can be inferred that Zeno behavior is absent.