The solution of the three-dimensional (3D) Poisson equation plays an important role in many fields. In particle accelerator research, the nonlinear space-charge effects due to Coulomb interactions among charged particles play an important role in high-intensity and high-brightness beam physics since such effects can cause beam quality degradation, halo formation, and even particle losses. In order to study the space-charge effects in the charged particle beam self-consistently, one needs to solve the 3D Poisson equation at each time step with the evolving particle density distribution. When the beam (e.g. electron beam) energy increases, in the beam frame computational domain, the longitudinal to transverse aspect ratio becomes larger and larger due to the relativistic effect. In such a case, an integrated Green’s function method was developed and showed to be more effective than the standard Green’s function method to solve the 3D Poisson equation subject to open boundary conditions since it does not require resolving the variation of the Green’s function through the computational domain [1] . In recent years, this method has been adopted in a number of particle accelerator beam dynamics codes for the self-consistent simulation of the space-charge effects [2] - [11] . However, there is no publication on how this method is implemented. Direct implementation of the mathematical expression of the integrated Green’s function method can result in substantially more computational cost. In this paper, we report on an efficient implementation method that reduces the computational cost by more than a factor 50 and alternative expressions that avoid cancellation errors under certain extreme conditions.

The electric potential $\varphi$ of the space-charge fields satisfies the following Poisson’s Equation:

${\nabla }^2\varphi =-\frac{\rho }{{ϵ}_0}$(1)

where $\rho$ is the charge density distribution function, and ${ϵ}_0$ is the permittivity in vacuum. The solution of the above Poisson’s Equation subject to the 3D free space open boundary condition can be written as:

$\varphi \left(x,y,z\right)=\frac{1}{4\pi {ϵ}_0}{\displaystyle \iiint G\left(x-x^{\prime },y-y^{\prime },z-z^{\prime }\right)\rho \left(x^{\prime },y^{\prime },z^{\prime }\right)\text{d}{x}^{\prime }\text{d}{y}^{\prime }\text{d}{z}^{\prime }}$(2)

where the Green’s function G is given by:

$G\left(x-x^{\prime },y-y^{\prime },z-z^{\prime }\right)=\frac{1}{\sqrt{{\left(x-x^{\prime }\right)}^2+{\left(y-y^{\prime }\right)}^2+{\left(z-z^{\prime }\right)}^2}}$(3)

To compute the above integral numerically, we define a computational domain containing the beam with a range of $\left(0,L_x\right)$, $\left(0,L_y\right)$ and $\left(0,L_z\right)$, and discretize each dimension using $N_x$, $N_y$ and $N_z$ grid points. Then, we decompose this integral in the entire computational domain as a summation of $N_x\times N_y\times N_z$ small cell integrals with each grid point located at the center of the cell.

$\begin{array}{c}\varphi \left(x_i,y_j,z_k\right)=\frac{1}{4\pi {ϵ}_0}{\displaystyle \underset{i^{\prime }=1}{\overset{N_x}{\sum }}{\displaystyle \underset{j^{\prime }=1}{\overset{N_y}{\sum }}{\displaystyle \underset{k^{\prime }=1}{\overset{N_z}{\sum }}{\displaystyle {\int }_{x_{i^{\prime }}-h_x/2}^{x_{i^{\prime }}+h_x/2}\text{d}{x}^{\prime }}{\displaystyle {\int }_{x_{j^{\prime }}-h_y/2}^{x_{j^{\prime }}+h_y/2}\text{d}{y}^{\prime }}{\displaystyle {\int }_{x_{k^{\prime }}-h_z/2}^{x_{k^{\prime }}+h_z/2}\text{d}{z}^{\prime }}}}}\\ \times G\left(x_i-x^{\prime },y_j-y^{\prime },z_k-z^{\prime }\right)\rho \left(x^{\prime },y^{\prime },z^{\prime }\right)\end{array}$(4)

where $h_x=L_x/\left(N_x-1\right)$, $h_y=L_y/\left(N_y-1\right)$, and $h_z=L_x/\left(N_z-1\right)$. If we assume that the charge density is constant within each cell centered at the grid point $\left(x_i,y_j,z_k\right)$, i.e. $\rho \left(x^{\prime },y^{\prime },z^{\prime }\right)=\rho \left(x_i,y_j,z_k\right)$, from the above equation, the electric potential on this grid point can be approximated as:

$\varphi \left(x_i,y_j,z_k\right)=\frac{1}{4\pi {ϵ}_0}{\displaystyle \underset{i^{\prime }=1}{\overset{N_x}{\sum }}{\displaystyle \underset{j^{\prime }=1}{\overset{N_y}{\sum }}{\displaystyle \underset{k^{\prime }=1}{\overset{N_z}{\sum }}\bar{G}\left(x_i-x_{i^{\prime }},y_j-y_{j^{\prime }},z_k-z_{k^{\prime }}\right)\rho \left(x_{i^{\prime }},y_{j^{\prime }},z_{k^{\prime }}\right)}}}$(5)

where $x_i=\left(i-1\right)h_x$, $y_j=\left(j-1\right)h_y$, and $z_k=\left(k-1\right)h_z$, and the effective Green function $\bar{G}$ is given as:

$\begin{array}{l}\bar{G}\left(x_i-x_{i^{\prime }},y_j-y_{j^{\prime }},z_k-z_{k^{\prime }}\right)\\ ={\displaystyle {\int }_{x_{i^{\prime }}-h_x/2}^{x_{i^{\prime }}+h_x/2}\text{d}{x}^{\prime }}{\displaystyle {\int }_{x_{j^{\prime }}-h_y/2}^{x_{j^{\prime }}+h_y/2}\text{d}{y}^{\prime }}{\displaystyle {\int }_{x_{k^{\prime }}-h_z/2}^{x_{k^{\prime }}+h_z/2}\text{d}{z}^{\prime }}\times G\left(x_i-x^{\prime },y_j-y^{\prime },z_k-z^{\prime }\right)\end{array}$(6)

where $h_x$, $h_y$, and $h_z$ are cell size in each dimension, respectively. The above integral can be calculated analytically in a closed form for the Green’s function given in Equation (3) as [12] :

$\begin{array}{c}\bar{G}\left(x,y,z\right)=f\left(x+\frac{h_x}{2},y+\frac{h_y}{2},z+\frac{h_z}{2}\right)-f\left(x+\frac{h_x}{2},y+\frac{h_y}{2},z-\frac{h_z}{2}\right)\\ +f\left(x-\frac{h_x}{2},y-\frac{h_y}{2},z+\frac{h_z}{2}\right)-f\left(x-\frac{h_x}{2},y-\frac{h_y}{2},z-\frac{h_z}{2}\right)\\ +f\left(x+\frac{h_x}{2},y-\frac{h_y}{2},z-\frac{h_z}{2}\right)-f\left(x+\frac{h_x}{2},y-\frac{h_y}{2},z+\frac{h_z}{2}\right)\\ +f\left(x-\frac{h_x}{2},y+\frac{h_y}{2},z-\frac{h_z}{2}\right)-f\left(x-\frac{h_x}{2},y+\frac{h_y}{2},z+\frac{h_z}{2}\right)\end{array}$(7)

where

$\begin{array}{c}f\left(x,y,z\right)=yz\ln \left(x+r\right)+xz\ln \left(y+r\right)+xy\ln \left(z+r\right)\\ -\frac{z^2}{2}\arctan \left(\frac{xy}{zr}\right)-\frac{y^2}{2}\arctan \left(\frac{xz}{yr}\right)-\frac{x^2}{2}\arctan \left(\frac{yz}{xr}\right)\end{array}$(8)

where $r=\sqrt{x^2+y^2+z^2}$. With the effective Green’s function $\bar{G}$, the summation of Equation (5) can be computed effectively using the FFT method [1] [13] .

The effective Green’s function in the above equation involves eight f-function evaluations. The range of the variables in the function f covers the range from $-L_x$ to $L_x$, $-L_y$ to $L_y$, and $-L_z$ to $L_z$, which suggests two times grid points are needed in each dimension for this function. However, a careful check of Equation (6) suggests that the effective Green’s function should have the same symmetry property as the original Green’s function, i.e., changing the sign of an individual variable in Equation (7) will not affect the value of the function. This can be seen from Equation (6), $\bar{G}\left(x_i-x_{i^{\prime }},y_j-y_{j^{\prime }},z_k-z_{k^{\prime }}\right)=G\left(x_i-x_{i^{\prime }},y_j-y_{j^{\prime }},z_k-z_{k^{\prime }}\right)h_x{h}_y{h}_z+O\left(h_x^3{h}_y^3{h}_z^3\right)$ or by using Equation (7) and $f\left(-x,y,z\right)=-f\left(x,y,z\right)+yz\ln \left(y^2+z^2\right)$ results in $\bar{G}\left(-x,y,z\right)=\bar{G}\left(x,y,z\right)$ for x (same applies to y and z). Hence, only the first quadrant of the effective Green’s function is needed. This saves the computational cost by about a factor of eight. Furthermore, by computing the f function on one corner of the integrated cell for an extended grid (i.e., $N+1$ instead of N) $G^{tmp}\left(i,j,k\right)=f\left(x_i-h_x/2,y_j-h_y/2,z_k-h_z/2\right)$ for $i=1,\cdots ,N_x+1$, $j=1,\cdots ,N_y+1$, and $k=1,\cdots ,N_z+1$, the f function values at the other seven corners of the cell in the Equation (7) can be obtained from the shift of this function on the grid. For example, the f function value at the upper right corner of the cell will be $f\left(x_i+h_x/2,y_j+h_y/2,z_k+h_z/2\right)=G^{tmp}\left(i+1,j+1,k+1\right)$ for $i=1,\cdots ,N_x$, $j=1,\cdots ,N_y$, and $k=1,\cdots ,N_z$. Only one f function evaluation is needed instead of eight function evaluations in Equation (7). This saves the computational cost by another factor of eight. In total, the efficient implementation can save the computing time of the effective Green’s function by more than a factor of 60 compared with the direct brute force implementation. An illustration of this implementation in Fortran90 is given in Figure 1 .

As a test of the practical performance of the above implementation, we measured the computing time of the above implementation and the computing time of the brute force implementation with a variety of problem sizes $N\times N\times N$. Figure 2 shows the speedup of the above implementation as a function of one-dimensional grid points in the real application. More than 50 speedup is achieved for problem sizes greater than 64 × 64 × 64.

From the above implementation, it is seen that the variable x, y, z in Equation (8) will be less than zero only when $i0=1$, or $j0=1$, or $k0=1$. The negative value of the variable could cause cancellation error in the evaluation of $x+r$, or $y+r$, or $z+r$. In some extreme applications, e.g., with very high electron beam energy, the longitudinal bunch length in the beam frame can be much larger than the transverse beam size. This results in the $\left|z\right|\simeq r$ and a large cancellation error in $z+r$ and even overflow of $\ln \left(z+r\right)$ for $z<0$.

The aforementioned cancellation error is due to the subtraction of two close real numbers on finite precision digital computers. This numerical cancellation error can be mitigated by either increasing the precision of each number on a computer or by using the following alternative expressions to replace the original summation in the Equation (8) or the Equation (8) itself.

Firstly, we can declare the variables x, y, z, and r in the above implementation as quadruple precision. This substantially increases the accuracy of each variable on a digital computer and reduces the cancellation error due to the double-precision representation of a variable.

Secondly, by making use of the function relationship $\text{arcsinh}\left(x\right)=\ln \left(x+\sqrt{1+x^2}\right)$, neglecting the terms that do not contribute to the field calculation (i.e. $\frac{1}{2}yz\ln \left(y^2+z^2\right)+\frac{1}{2}xz\ln \left(x^2+z^2\right)+\frac{1}{2}xy\ln \left(x^2+y^2\right)$), the Equation (8) can be rewritten as:

$\begin{array}{c}f\left(x,y,z\right)=yz\text{arcsinh}\left(\frac{x}{\sqrt{y^2+z^2}}\right)+xz\text{arcsinh}\left(\frac{y}{\sqrt{x^2+z^2}}\right)\\ +xy\text{arcsinh}\left(\frac{z}{\sqrt{x^2+y^2}}\right)-\frac{z^2}{2}\arctan \left(\frac{xy}{zr}\right)\\ -\frac{y^2}{2}\arctan \left(\frac{xz}{yr}\right)-\frac{x^2}{2}\arctan \left(\frac{yz}{xr}\right)\end{array}$(9)

This equation avoids the sum of two close but opposite sign variables in the original equation and the resultant cancellation error.

Thirdly, one can define a small tolerance number $ϵ$ (e.g. 10⁻¹⁰) and use the Taylor expansion to obtain the following approximation:

$\begin{array}{l}x+r=x+\left|x\right|+\frac{1}{2}\frac{y^2+z^2}{\left|x\right|},\text{for}\left|x+r\right|<ϵ\\ y+r=y+\left|y\right|+\frac{1}{2}\frac{x^2+z^2}{\left|y\right|},\text{for}\left|y+r\right|<ϵ\\ z+r=z+\left|z\right|+\frac{1}{2}\frac{x^2+y^2}{\left|z\right|},\text{for}\left|x+r\right|<ϵ\end{array}$(10)

The above approximation is used in the natural logarithm function to mitigate the cancellation error.

Fourthly, one can rewrite the expression in the natural logarithm function of Equation (8) as:

$\begin{array}{l}x+r=\frac{y^2+z^2}{r-x},\text{for}x<0\\ y+r=\frac{x^2+z^2}{r-y},\text{for}y<0\\ z+r=\frac{x^2+y^2}{r-z},\text{for}z<0\end{array}$(11)

The above expression turns the original summation of two opposite sign variables into an expression that includes subtraction of these variables and avoids the cancellation error.

As a test, we used a 100 pC electron beam with a 3D Gaussian density distribution and computed the electric fields in the beam frame from all of the above schemes and from a semi-analytical solution. The semi-analytical electric potential in the rest beam frame for a Gaussian density distribution is given by [14] :

$\varphi \left(x,y,z\right)=\frac{Q}{4\pi {ϵ}_0}\sqrt{\frac{2}{\pi }}{\displaystyle {\int }_0^{\infty }\frac{{\text{e}}^{\frac{-{\lambda }^2{x}^2}{2\left({\lambda }^2{\sigma }_x^2+1\right)}}{\text{e}}^{\frac{-{\lambda }^2{y}^2}{2\left({\lambda }^2{\sigma }_y^2+1\right)}}{\text{e}}^{\frac{-{\lambda }^2{z}^2}{2\left({\lambda }^2{\sigma }_z^2+1\right)}}}{\sqrt{\left({\lambda }^2{\sigma }_x^2+1\right)\left({\lambda }^2{\sigma }_y^2+1\right)\left({\lambda }^2{\sigma }_z^2+1\right)}}\text{d}\lambda }$(12)

where Q is the total charge of the beam, and ${\sigma }_x$, ${\sigma }_y$, ${\sigma }_z$ are the RMS sizes of the beam in each dimension. In this test, we assumed that ${\sigma }_x={\sigma }_y=0.5$ mm and ${\sigma }_z=1$ mm. We varied electron beam kinetic energy so that the electron beam longitudinal bunch length $\gamma {\sigma }_z$ in the beam frame increased with the increase of beam energy. This increases the longitudinal-to-transverse aspect ratio with the increase of beam energy in the beam frame. We used 129 × 129 × 257 grid points to solve the Poisson equation using the above integrated Green’s function method in the rest beam frame. Electric fields are numerically computed from $E=-\nabla \varphi$ in this frame using a second-order finite difference approximation.

Figure 3 shows horizontal and longitudinal electric fields as a function of longitudinal coordinate z in the beam frame from the semi-analytical solution and from the original integrated Green’s function method at 100 GeV electron beam energy. It is seen that the electric fields from the numerical integrated Green’s function solution agree with those from the semi-analytical solution very well.

In order to check the valid regime of the integrated Green’s function method, we varied the electron beam kinetic energy from 100 GeV to 100 TeV. This results in about 2 × 10⁵ to 2 × 10⁸ longitudinal-to-transverse aspect ratio for the computational cells in the rest beam frame. Figure 4 shows the maximum horizontal relative electric field error and the maximum longitudinal relative electric field error as a function of the electron beam kinetic energy using the original double

precision logarithm integrated Green’s function (IGF), the quadruple precision logarithm IGF, the arcsinh IGF, the approximated logarithm IGF, and the rewritten logarithm IGF. Here, the differences between the numerical solutions from the integrated Green’s function method and the semi-analytical solutions were calculated on the three-dimensional 33 × 33 × 65 grid points for both the horizontal electric field and the longitudinal electric field. These differences are normalized by the maximum values of the horizontal electric field and the longitudinal field from the semi-analytical model, respectively, to attain the 3D relative errors. The maximum relative errors are attained from the relative errors on the 3D grid. It is seen that all five integrated Green’s function implementations yield nearly the same less than 0.1% relative errors up to 50 TeV beam energy. This is probably due to the fact that the cancellation error occurs only $z=-\frac{h_z}{2}$ and

has a small contribution to the total 3D summation. At the 100 TeV electron beam energy, the original logarithm integrated Green’s function fails due to the cancellation error in the evaluation of $z+r$ and the resultant overflow of the logarithm function. The other four mitigation schemes all work well and yield less than 0.1% relative errors. The computational cost of the quadruple precision implementation of the IGF is the highest (more than 10 times the original IGF) due to the lack of direct hardware support for such operations. The computational cost of the arcsinh implementation of the IGF is about a factor of two of that of the original IGF. The computational costs of the approximated IGF and the rewritten IGF implementations are close to that of the original IGF, while the rewritten IGF implementation does not need to specify any tolerance number.

In this paper, we present an efficient implementation of the integrated Green’s function method to solve the 3D Poisson’s Equation in a large aspect ratio computational domain subject to the open boundary conditions. Our implementation suggests more than a factor of 50 reduction of computational cost compared with the direct brute force implementation. Furthermore, several alternative expressions are proposed to avoid cancellation errors and work well under extreme conditions. This implementation can have applications in many fields, such as high-brightness beam physics in particle accelerators, plasma physics, and micromagnetics, where the solution of 3D Poisson equation in the large aspect ratio computational domain is needed.

This work was supported by the U.S. Department of Energy under Contract No. DE-AC02-05CH11231 and used computer resources at the National Energy Research Scientific Computing Center.