The Selberg trace formula, introduced by A. Selberg in 1956, describes the spectrum of the hyperbolic Laplacian in terms of geometric data involving the lengths of geodesics on a Riemann surface. Motivated by analogy between this trace formula and the explicit formulas of number theory relating the zeroes of the Riemann zeta function to prime numbers, Selberg [1] introduced a zeta function whose analytic properties are encoded in the Selberg trace formula. By focusing on the Selberg zeta function, H. Huber ( [2] , p. 386; [3] , p. 464), proved an analogue of the prime number theorem for compact Rie-

mann surfaces with the error term that agrees with Selberg’s one.

Using basically the same method as in [4] , D. Hejhal ( [5] , p. 475), established also the prime geodesic theorem for non-compact Riemann surfaces with the remainder

. However, in the compact case there exist several different proofs (see,

B. Randol [6] , p. 245; P. Buser [7] , p. 257, Th. 9.6.1; M. Avdispahić and L. Smajlović

[8] , Th. 3.1) that give the remainder. Thanks to new integral repre-

sentations of the logarithmic derivative of the Selberg zeta function (cf. [9] , p. 185; [10] , p. 128), M. Avdispahić and L. Smajlović ( [11] , p. 13) were in position to improve

error term in a non-compact, finite volume case up to.

Whereas the authors in [8] and [11] approached the prime number theorem in various settings via explicit formulas for the Jorgenson-Lang fundamental class of functions, our main goal is to obtain this improvement for non-compact Riemann surfaces with cusps following a more direct method of B. Randol [6] .

Let X be a non-compact Riemann surface regarded as a quotient of the upper half-plane by a finitely-generated Fuchsian group of the first kind, containing cusps. Let denote the fundamental region of. We shall assume that the fundamental region of has a finite non-Euclidean area. We put

and denote by v the multiplier system of the weight for. Let be an irreducible unitary representation on and,. For an r dimensional vector space V over we consider an essentially self-adjoint operator

on the space of all twice continuously differentiable functions, such that f and are square integrable on, and satisfy the equality

The operator has the unique self-adjoint extension to the space, a dense subspace of. Let, be the set of parabolic transformations corresponding to cusps of. does not depend on the choice of a representative of the parabolic class and can be considered as a matrix from. By we will denote the multiplicity of 1 as an eigen-value of the matrix

, and will be the degree of singularity of W. We mention that oper-

ator has both the discrete and continuous spectrum in the case, and only the discrete spectrum in the case. The discrete spectrum will be denoted as (). The continuous spectrum is expressed through zeros (or equivalently poles) of the hyperbolic scattering determinant (see, [12] ).

Let denotes the set of -conjugacy classes of a primitive hyperbolic element in, and denotes the set of -conjugacy classes of a hyperbolic element P in that satisfy property. Assume that. We define the Selberg zeta function associated to the pair () by

is absolutely convergent for. Analytic considerations given in ( [5] , pp. 499-501) yield that the Selberg zeta function in this setting satisfies the functional equation

with the fudge factor

Here, denotes the hyperbolic scattering determinant. It can be represented in the form

where the coefficients and depend on the group (see, [5] , p. 437). Here, denotes the degree of singularity of W (see Section 2). An explicit expression for the fudge factor in the Equation (1) is given in ( [5] , p. 501, Equation (5.10)).

The logarithmic derivative of the Selberg zeta function is given by

where denotes the norm of the class P and for a primi-

tive element such that for some. We will omit the indices in in the sequel.

Lemma 1. For,

where for a primitive element such that for some.

Proof.

We shall spend the rest of this section to derive a representation of in the form (11) bellow. We choose not to write it in a separate statement because of the length of expressions involved. However, it will serve as a base for the proof of the prime geodesic theorem in Section 5.

Let us recall the following theorem given in ( [13] , p. 51, Th. 40).

Theorem 1. If the Dirichlet’s series is summable for and, , then

By Lemma 1,

We have,

Therefore, substituting ω = 1, , and hence, in (2), we get

Then,

Now, put

and

for. Using ( [14] , p. 12, Th. 1.3.5), it is easy to get that

For, let, , be the zeros of in. Let, denote all zeros of the hyperbolic scattering determinant in.

Assume, , , where and for

,. Following ( [5] , p. 468), we may also assume, , where, , are the zeros of the Selberg zeta function for each zero, , , of the hyperbolic scattering determinant f. Let be a large constant such that, ,. We put.

Without loss of generality we may assume that, ,. Let. By the Cauchy residue theorem one has

and

Arguing as in [5] (p. 474) and [4] (pp. 105-108), we easily find that the sum of the

first eight integrals on the right hand side of (5) is. Similarly, taking into account that is bounded for, we obtain that the sum of the first eight integrals on the right hand side of (6) is. Following [5] (p. 474) and [4] (p. 85,

Prop. 5.7), we obtain that the ninth resp. the third integral on the right hand side of (5) resp. (6) are. Now, if we take, (5) and (6) will give us

and

Bearing in mind location of the poles of given in ( [5] , p. 439, Th. 2.16; or [5] , p. 498, Th. 5.3) and the fact that, we may assume without loss of generality that

Calculating residues and passing to the limit in (7) and (8) we get

and

The implied constants on the right sides of (9) and (10) depend solely on, m and W. With in (3), in (4), Equations (4), (3), (9) and (10) yield

where the first sum ranges over the finite set of poles s of

with, , the second sum ranges over the set of poles s of the same functions with, and the third sum ranges over the finite set of their poles s with.

In our setting, the prime geodesic counting function is defined by

where the sum on the right is taken over all primitive hyperbolic classes with respect to (see, [5] , p. 473, [11] , p. 13).

Theorem 2. For, the formula

holds true, where, for, and the implied constant depends solely on, m and W.

Proof. Following [6] (p. 245) and [15] (p. 11), for a positive number, we define the second difference operator by

Here, d is a constant which will be fixed later. By the mean value theorem, we have

for some. It is easy to verify that

Reasoning as in [5] (p. 475), we may assume without loss of generality that is non-decreasing. Hence, (12) implies

Since (14) holds true, one can easily deduce that, , , , , ,. Thus, (13) and finiteness of the sums contained in on the right hand side (11) yield

Similarly,

In order to estimate, we will first consider

By (14) it is evident that

On the other hand, the mean value theorem (13) gives us

Let be the number of roots of on the critical line in the interval. It is known ( [5] , p. 477, Th. 3.8) that. Taking and following ( [3] , pp. 463-464; [6] , p. 246), we use (19) resp. (18) in the sums over, , resp. sum over, (below) to get

Thus,

Similarly,

Observe that (see, [5] , p. 437, Prop. 2.13). Thus, application of to the third and the fourth sum in gives us

.

Let us write

where denotes the sum of the first four sums in and denotes the sum of the last four sums in. Now, Equations (11), (16), (17), (20), (21) and (22) give us

Putting, , the Equation (23) becomes

Since the left sides of Equations (20), (21) are for such choice of M and d, we get. Now, it is obvious that. Finally, Equation (24) gives us

Returning to (15), we conclude that inequality

holds true. Following ( [15] , p. 11), we analogously obtain that

Hence,

Arguing as in [5] (p. 475) and [4] (p. 113), one immediately sees that equality (25) proves the theorem.

We thank the Editor and the referee for their comments.

Avdispahić, M. and Gušić, Dž. (2016) On the Prime Geodesic Theorem for Non-Compact Riemann Surfaces. Advances in Pure Mathematics, 6, 903- 914. http://dx.doi.org/10.4236/apm.2016.612068