This paper is concerned with the oscillations of solutions of a generalized Liénard system of the type

$\begin{array}{l}\frac{\text{d}x}{\text{d}t}=\frac{1}{a\left(x\right)}\left[h\left(y\right)-F\left(x\right)\right]\mathrm{,}\\ \frac{\text{d}y}{\text{d}t}=-a\left(x\right)g\left(x\right).\end{array}$ (1)

The system (1) has in recent years been the object of intensive studies with particular emphasis on the asymptotic behavior of solutions (see [1] [2] ), because it can be considered as a natural generalization of the Liénard system

$\begin{array}{l}\frac{\text{d}x}{\text{d}t}=y-F\left(x\right),\\ \frac{\text{d}y}{\text{d}t}=-g\left(x\right).\end{array}$ (2)

As the system (2) appears in many mathematical models in physics, engineering, chemistry, biology, economics, etc., it naturally has been studied by a number of authors; many results can be found e.g. in the books [3] - [9] , and the references cited therein.

It is well known that system (1) is of great importance in various applications, many other systems can be transformed into this form. Hence, qualitative and asymptotic behavior of this system and some of its extensions have been widely studied by many authors. To study the oscillation of solutions of (1), as discussed in some recent papers (see [10] - [17] ) with $a\left(x\right)=1$, for the right half plane, a significant point is to find conditions ensuring that all positive orbits ${\gamma }^{+}\left(P\right)$ (where $P=\left(\mathrm{0,}p\right)$ with $p>0$) intersect the characteristic curve $h\left(y\right)=F\left(x\right)$ and then cross the negative y-axis; this property of ${\gamma }^{+}\left(P\right)$ plays an important role in the analysis of oscillation, asymptotic stability and boundedness conditions of (1). There have been many works in this direction, in which sufficient conditions to obtain the above mentioned property of ${\gamma }^{+}\left(P\right)$ were given. For example, see [18] - [26] , no solution of (1) with $h\left(y\right)=y$ and $a\left(x\right)=1$ approaches the origin directly in the right half plane (i.e., in a nonoscillatory way) if one of the following conditions is satisfied (in the following, $f\left(x\right)\mathrm{<mo>\; :\; </mo>}=F^{\prime }\left(x\right)$ if $F\left(x\right)$ is continuously differentiable, $G\left(x\right)\mathrm{<mo>\; :\; </mo>}={\displaystyle {\int }_0^x}g\left(s\right)\text{d}s$):

1) (McHarg [22] ) $f\left(x\right)>0$ for $x>0$ and there exist $k>0$ and $a>0$ such that

$f\left(x\right)<kg\left(x\right)\text{for}0<x<a.$

2) (Wendel [25] ) There exist $k>0$ and $a>0$ such that

$0<f\left(x\right)<kg\left(x\right)\text{for}0<x<a.$

3) (Nemyckii and Stepanov [23] ) There exist $\alpha >\frac{1}{4}$ and $a>0$ such that

$f\left(x\right)>0,\alpha f\left(x\right)F\left(x\right)\le g\left(x\right)\text{for}0<x<a.$

4) (Filippov [18] ) There exist $0<\beta <8$ and $a>0$ such that

$F^2\left(x\right)\le \beta G\left(x\right)\text{for}0<x<a.$

5) (Opial [24] ) There exist $\alpha >\frac{1}{4}$ and $a>0$ such that

$\alpha \left|F\left(x\right)\right|\le {\displaystyle {\int }_0^x}\frac{g\left(u\right)}{\left|F\left(u\right)\right|}\text{d}u\text{for}0<x<a.$

6) (Hara and Yoneyama [20] , Hara, Yoneyama and Sugie [21] , Sugie [27] ) If one of the following conditions holds:

i) there exists a positive sequence $\left\{x_n\right\}$ such that $x_n\to 0$ as $n\to \infty$ and $F\left(x_n\right)\le 0$ for $n\ge 1$;

ii) There exist $\alpha >\frac{1}{4}$ and $a>0$ such that

$F\left(x\right)>0,\frac{1}{F\left(x\right)}{\displaystyle {\int }_0^x}\frac{g\left(u\right)}{F\left(u\right)}\text{d}u\ge \alpha \text{for}0<x<a.$

7) (Yu and Zhang [9] ) There exist $a>0$, $k_1>0$ and $k_2<0$ such that

$k_2\le \frac{f\left(x\right)}{g\left(x\right)}\le k_1\text{for}0<x<a.$

Our investigation in this paper shows that condition (6) is really weaker than condition (4) (see Remark 3.2 in this paper). The problem concerning the oscillation of solutions of (1) with $a\left(x\right)=1$ has been studied by some authors (see, for example [12] [16] and the references cited therein). Li and Tang [12] discussed the oscillation of solutions of (1) with $a\left(x\right)=1$ requiring the existence of $h^{″}\left(y\right)$ and $h^{\prime }\left(0\right)>0$, Yan and Jiang [16] proved that the solutions of (1) with $a\left(x\right)=1$ are oscillatory under the condition $h^{\prime }\left(0\right)>0$, but the problem of what happens when $h^{\prime }\left(0\right)=0$ or $h^{\prime }\left(0\right)=\infty$ is left. In the present paper, no restrictions on the differentiability of $h\left(y\right)$ are required, we give necessary and sufficient conditions that all nontrivial solutions of (1) are oscillatory, our theorem can be applied to system (1.1) even for $h^{\prime }\left(0\right)=0$, $h^{\prime }\left(0\right)=\infty$ and $\underset{\left|x\right|\to \infty }{\bar{\lim }}F\left(x\right)\mathrm{sgn}x=-\infty$. Our results substantially extend and improve some results known in the literature.

The technical tool of this paper is based on a new nonlinear integral inequality and a phase plane analysis. Also the methods for Liénard-type systems, especially those developed by Villari and Zanolin [14] , Hara and Sugie [11] , and Sugie and Hara [13] will be applied in our paper

The organization of this paper is as follows. In Section 2 we agree on some notation, present assumptions and some lemmas which will be essential to our proofs. In Section 3 we give sufficient and necessary conditions for the oscillation of all solutions of (1). Some examples illustrating the results are also given in this paper.

We consider the generalized Liénard system

$\begin{array}{l}\frac{\text{d}x}{\text{d}t}=\frac{1}{a\left(x\right)}\left[h\left(y\right)-F\left(x\right)\right]\mathrm{,}\\ \frac{\text{d}y}{\text{d}t}=-a\left(x\right)g\left(x\right).\end{array}$ (3)

where $F\left(x\right)$, $g\left(x\right)$, $a\left(x\right)$ and $h\left(y\right)$ are continuous real functions defined on $R$ satisfying:

(A₀) $F\left(0\right)=0$, $a\left(x\right)>0$ for $x\in R$, $xg\left(x\right)>0$ for $x\ne 0$;

(A₁) $yh\left(y\right)>0$ for $y\ne 0$, $h\left(y\right)$ is strictly increasing and $h\left(\pm \infty \right)=\pm \infty$.

These assumptions guarantee that the origin is the only critical point of (3). We also assume that the initial value problem always has a unique solution.

We call the curve $h\left(y\right)=F\left(x\right)$ the characteristic curve of system (3). We write ${\gamma }^{+}\left(P\right)$ (resp., ${\gamma }^{-}\left(P\right)$) the positive (resp., negative) semiorbit of (3) starting at a point $P\in R^2$. For the sake of convenience, we denote

$\begin{array}{ll}{D}_1=\left\{\left(x,y\right):x\ge 0,h\left(y\right)>F\left(x\right)\right\},\hfill & D_2=\left\{\left(x,y\right):x>0,h\left(y\right)\le F\left(x\right)\right\},\hfill \\ D_3=\left\{\left(x,y\right):x\le 0,h\left(y\right)<F\left(x\right)\right\},\hfill & D_4=\left\{\left(x,y\right):x<0,h\left(y\right)\ge F\left(x\right)\right\}.\hfill \end{array}$

$\begin{array}{ll}{F}_{+}\left(x\right)=\max \left\{0,F\left(x\right)\right\},\hfill & F_{-}\left(x\right)=\max \left\{0,-F\left(x\right)\right\},\hfill \\ {\Gamma }_{+}\left(x\right)={\displaystyle {\int }_0^x}{a}^2\left(s\right)g\left(s\right){\left(1+F_{+}\left(s\right)\right)}^{-1}\text{d}s,\hfill & {\Gamma }_{-}\left(x\right)={\displaystyle {\int }_0^x}{a}^2\left(s\right)g\left(s\right){\left(1+F_{-}\left(s\right)\right)}^{-1}\text{d}s.\hfill \end{array}$

$\begin{array}{ll}{Y}^{+}=\left\{\left(0,y\right):y>0\right\},\hfill & Y^{-}=\left\{\left(0,y\right):y<0\right\},\hfill \\ C^{+}=\left\{\left(x,y\right):x>0,h\left(y\right)=F\left(x\right)\right\},\hfill & C^{-}=\left\{\left(x,y\right):x<0,h\left(y\right)=F\left(x\right)\right\}\hfill \\ \hfill & G\left(x\right)={\displaystyle {\int }_0^x}{a}^2\left(s\right)\left|g\left(s\right)\right|\text{d}s.\hfill \end{array}$

Then by (A₀), $G\left(x\right)$ is strictly increasing, and therefore, the inverse function $G^{-1}\left(w\right)$ of $w=G\left(x\right)$ exists.

Lemma 2.1 Let $Y\left(x\right)\mathrm{,}\psi \left(x\right)$ be positive continuous functions in $0<a\le x\le b$ and let $\omega \left(u\right)$ be a positive increasing continuous function for $u>0$, and let $\Omega \left(u\right)={\displaystyle {\int }_{0^{+}}^u}\frac{\text{d}t}{\omega \left(t\right)}$ exists for $u>0$ with $\Omega \left(0\right)=0$. Then for $\lambda >0$ the inequality

$Y\left(x\right)\ge \lambda {\displaystyle {\int }_a^x}\psi \left(t\right)\omega \left(Y\left(t\right)\right)\text{d}t\text{for}a\le x\le b$ (4)

implies the inequality

$\Omega \left(Y\left(x\right)\right)\ge \lambda {\displaystyle {\int }_a^x}\psi \left(t\right)\text{d}t\text{for}a\le x\le b.$ (5)

Proof. Define

$V\left(x\right)=\lambda {\displaystyle {\int }_a^x}\psi \left(t\right)\omega \left(Y\left(t\right)\right)\text{d}t\text{for}a\le x\le b.$ (6)

Then (4) can be restated as $Y\left(x\right)\ge V\left(x\right)$. Because $\omega \left(u\right)$ is increasing, this may be rewritten as follows

$\begin{array}{l}\omega \left(Y\left(x\right)\right)\ge \omega \left(V\left(x\right)\right)\mathrm{,}\\ \frac{V^{\prime }\left(x\right)}{\omega \left(V\left(x\right)\right)}\ge \lambda \psi \left(x\right)\end{array}$

for $a<x\le b$. By making use of the notation $\Omega \left(u\right)$, we have

$\frac{\text{d}\Omega \left(V\left(x\right)\right)}{\text{d}x}\ge \lambda \psi \left(x\right)\text{for}a<x\le b.$ (7)

Now, integrating from a to x, we get by (7),

$\Omega \left(V\left(x\right)\right)-\Omega \left(V\left(a\right)\right)\ge \lambda {\displaystyle {\int }_a^x}\psi \left(t\right)\text{d}t\mathrm{.}$

Since $V\left(a\right)=0$, it follows that

$\Omega \left(V\left(x\right)\right)\ge \lambda {\displaystyle {\int }_a^x}\psi \left(t\right)\text{d}t\text{for}a\le x\le b\mathrm{.}$ (8)

Because $Y\left(x\right)\ge V\left(x\right)$ for $a\le x\le b$, and $\Omega \left(u\right)$ is increasing, we obtain by (8),

$\Omega \left(Y\left(x\right)\right)\ge \lambda {\displaystyle {\int }_a^x}\psi \left(t\right)\text{d}t\text{for}a\le x\le b\mathrm{.}$

This completes the proof.

In this section, we give our main result about necessary and sufficient conditions for the oscillation of solutions of (1). We assume that all solutions of (1) can be continued in the forward direction up to $t=\infty$. A solution $\left(x\left(t\right)\mathrm{,}y\left(t\right)\right)$ of (1) is oscillatory if there are two sequences $\left\{t_n\right\}$ and $\left\{{\tau }_n\right\}$ tending monotonically to $\infty$ such that $x\left(t_n\right)=0$ and $y\left({\tau }_n\right)=0$ for every $n\ge 1$.

We say that (1) satisfies the assumption (A₂) if both ( ${\text{A}}_2^{+}$) and ( ${\text{A}}_2^{-}$) hold.

The system (1) is said to satisfy ( ${\text{A}}_2^{+}$) if one of the following conditions holds:

( ${\text{A}}_2^{+}$)₁ There exists a positive decreasing sequence $\left\{x_n\right\}$ such that $x_n\to 0$ as $n\to \infty$, and $F\left(x_n\right)\le 0$ for $n\ge 1$;

( ${\text{A}}_2^{+}$)₂ There exist constants $\alpha >\frac{1}{4}$ and ${\delta }_1>0$ such that

$F\left(x\right)>0\text{for}0<x\le {\delta }_1,$

and for any fixed real number $k\ge 1$,

${\displaystyle {\int }_{0^{+}}^x}\frac{g\left(s\right)}{F\left(s\right)}\text{d}s\ge \frac{1}{k}{h}^{-1}\left(k\alpha F\left(x\right)\right)\text{for}0<x\ll 1,$

where $h^{-1}\left(u\right)$ is the inverse function of $u=h\left(y\right)$, and the notation $0<x\ll 1$ denotes x sufficiently small.

The system (1) is said to satisfy ( ${\text{A}}_2^{-}$) if one of the following conditions holds:

( ${\text{A}}_2^{-}$)₁ There exists a negative decreasing sequence $\left\{x_n\right\}$ such that $x_n\to 0$ as $n\to \infty$, and $F\left(x_n\right)\ge 0$ for $n\ge 1$;

( ${\text{A}}_2^{-}$)₂ There exist constants $\alpha >\frac{1}{4}$ and ${\delta }_2>0$ such that

$F\left(x\right)<0\text{for}0<-x\le {\delta }_2,$

and for any fixed real number $k\ge 1$,

${\displaystyle {\int }_{0^{-}}^x}\frac{g\left(s\right)}{F\left(s\right)}\text{d}s\le \frac{1}{k}{h}^{-1}\left(k\alpha F\left(x\right)\right)\text{for}0<-x\ll 1.$

Lemma 3.1 Suppose that the conditions (A₀), (A₁), and ( ${\text{A}}_2^{+}$) hold. Then for any $P=\left(x_0\mathrm{,}{y}_0\right)\in C^{+}$, the positive semiorbit ${\gamma }^{+}\left(P\right)$ intersects the negative y-axis.

Proof. Let $P=\left(x_0\mathrm{,}{y}_0\right)\in C^{+}$ and $\left(x\left(t\right)\mathrm{,}y\left(t\right)\right)$ be the solution of (3) with $x\left(0\right)=x_0$, $y\left(0\right)=y_0$. By the uniqueness of the solutions of (3), we only have to show that every orbit ${\gamma }^{+}\left(P\right)$ of (3) passing through $P=\left(x_0,y_0\right)$ $\left(0<x_0\ll 1\right)$ intersect $Y^{-}$ at $B\left(\mathrm{0,}{y}_B\right)$ with $y_B<0$. Since $\underset{y\to -\infty }{\lim }h\left(y\right)=-\infty$, the system (3) has no vertical asymptote in the fourth quadrant. Therefore, ${\gamma }^{+}\left(P\right)$ must intersect the y-axis at $B\left(\mathrm{0,}{y}_B\right)$ with $y_B\le 0$. We still have to show that $y_B\ne 0$. We do this separately for the different cases of (A₂).

Case (A₂)₁: It is obvious in this case.

Case (A₂)₂: It follows from (A₀) that the orbit ${\gamma }^{+}\left(P\right)$ of (3) does not touch the characteristic curve at any point $\left(x\mathrm{,}{h}^{-1}\left(F\left(x\right)\right)\right)$ with $0\le x<x_0$. Thus, we consider only the region $\left\{\left(x,y\right):x>0,h\left(y\right)<F\left(x\right)\right\}$.

If $F\left(x\right)<0$ for $0<x\le x_0$, it is clear that $y_B<0$. Suppose that $F\left(x\right)>0$ for $0<x\le x_0$ and that the conclusion does not hold. Then there exists a point $P\in C^{+}$ such that ${\gamma }^{+}\left(P\right)$ does not intersect $Y^{-}$. Let $\left(x\left(t\right)\mathrm{,}y\left(t\right)\right)$ $\left(0\le t<\infty \right)$ denote the solution of (3) which passes through such a point P. Then ${\gamma }^{+}\left(P\right)$ must be contained in the first quadrant, and $x\left(t\right)$ decreases and $y\left(t\right)$ decreases as t is increasing. Since the origin is the unique equilibrium of (3), $\underset{t\to \infty }{\lim }x\left(t\right)=\underset{t\to \infty }{\lim }y\left(t\right)=0$. The solution $\left(x\left(t\right)\mathrm{,}y\left(t\right)\right)$ defines a function $y=y\left(x\right)$ on $0\le x\le x_0$, which is a solution on $0<x<x_0$ of the following equation

$\frac{\text{d}y}{\text{d}x}=-\frac{a^2\left(x\right)g\left(x\right)}{h\left(y\right)-F\left(x\right)}\mathrm{.}$ (9)

It follows from ${\lim }_{x\to 0^{+}}y\left(x\right)=0$ that $y\left(x\right)>0$ for $0<x\le x_0$. By assumption (A₂)₂, there exist $\alpha >\frac{1}{4}$ and $x_1\in \left(\mathrm{0,}{x}_0\right)$ such that $F\left(x\right)>0$ for $0<x\le x_1$, and

${\displaystyle {\int }_{0^{+}}^x}\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\ge h^{-1}\left(\alpha F\left(x\right)\right)\text{for}0<x\le x_1.$ (10)

Now, we restrict our attention to the interval $\left(\mathrm{0,}{x}_1\right]$. Putting $H_1\left(u\right)={\displaystyle {\int }_0^u}h\left(y\right)\text{d}y$, we have by (9), for any $0<\epsilon \ll 1$,

$\begin{array}{c}{H}_1\left(y\left(x\right)\right)-H_1\left(y\left(\epsilon \right)\right)={\displaystyle {\int }_{\epsilon }^x}{H^{\prime }}_1\left(y\left(s\right)\right)\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)-h\left(y\left(s\right)\right)}\text{d}s\\ \ge {\displaystyle {\int }_{\epsilon }^x}h\left(y\left(s\right)\right)\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\\ ={\displaystyle {\int }_{\epsilon }^x}\left(h\circ H_1^{-1}\right)H_1\left(y\left(s\right)\right)\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\end{array}$

for $\epsilon \le x\le x_1$. Hence

$H_1\left(y\left(x\right)\right)\ge {\displaystyle {\int }_{\epsilon }^x}\left(h\circ H_1^{-1}\right)H_1\left(y\left(s\right)\right)\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s$

for $\epsilon \le x\le x_1$. It follows from Lemma 2.1 that

$H_2\left(H_1\left(y\left(x\right)\right)\right)\ge {\displaystyle {\int }_{\epsilon }^x}\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\text{for}\epsilon \le x\le x_1\mathrm{,}$ (11)

where $H_2\left(u\right)={\displaystyle {\int }_{0^{+}}^u}\frac{\text{d}t}{\left(h\circ H_1^{-1}\right)\left(t\right)}$. Changing variables $H_1^{-1}\left(t\right)=\tau$, it is easy to see that $H_2\left(u\right)=H_1^{-1}\left(u\right)$. By (11), we have

$y\left(x\right)\ge {\displaystyle {\int }_{\epsilon }^x}\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\text{for}\epsilon \le x\le x_1\mathrm{.}$ (12)

(i) If ${\displaystyle {\int }_{0^{+}}^{x_1}}\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s=\infty$, we reach a contradiction by (12).

(ii) If ${\displaystyle {\int }_{0^{+}}^{x_1}}\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s<\infty$, we see from (12) that

$y\left(x\right)\ge {\displaystyle {\int }_{0^{+}}^x}\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\text{for}0<x\le x_1.$ (13)

By virtue of (10) and (13), we have $y\left(x\right)\ge h^{-1}\left(\alpha F\left(x\right)\right)$ for $0<x\le x_1$. Because $h\left(y\right)$ is strictly increasing, we obtain $h\left(y\left(x\right)\right)\ge \alpha F\left(x\right)$ for $0<x\le x_1$. Since $y=y\left(x\right)$ is under the characteristic curve $h\left(y\right)=F\left(x\right)$, we have $\frac{1}{4}<\alpha <1$. Let ${\alpha }_1=1-\alpha$, then we get that $F\left(x\right)-h\left(y\left(x\right)\right)\le {\alpha }_{1}F\left(x\right)$ for $0<x\le x_1$. In a similar way, for any $0<\epsilon \ll 1$, we have

$\begin{array}{c}{H}_1\left(y\left(x\right)\right)-H_1\left(y\left(\epsilon \right)\right)={\displaystyle {\int }_{\epsilon }^x}h\left(y\left(s\right)\right)\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)-h\left(y\left(s\right)\right)}\text{d}s\\ \ge \frac{1}{{\alpha }_1}{\displaystyle {\int }_{\epsilon }^x}h\left(y\left(s\right)\right)\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\end{array}$

for $\epsilon \le x\le x_1$. Therefore

$\begin{array}{c}{H}_1\left(y\left(x\right)\right)\ge \frac{1}{{\alpha }_1}{\displaystyle {\int }_{\epsilon }^x}h\left(y\left(s\right)\right)\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\\ =\frac{1}{{\alpha }_1}{\displaystyle {\int }_{\epsilon }^x}\left(h\circ H_1^{-1}\right)H_1\left(y\left(s\right)\right)\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\end{array}$

for $\epsilon \le x\le x_1$. By Lemma 2.1, we have

$H_2\left(H_1\left(y\left(x\right)\right)\right)\ge \frac{1}{{\alpha }_1}{\displaystyle {\int }_{\epsilon }^x}\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\mathrm{,}$

$y\left(x\right)\ge \frac{1}{{\alpha }_1}{\displaystyle {\int }_{\epsilon }^x}\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s$

for $\epsilon \le x\le x_1$. Hence

$y\left(x\right)\ge \frac{1}{{\alpha }_1}{\displaystyle {\int }_{0^{+}}^x}\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s$ (14)

for $\epsilon \le x\le x_1$. By assumption (A₂)₂, there exists $x_2\in \left(\mathrm{0,}{x}_1\right)$ such that

${\displaystyle {\int }_{0^{+}}^x}\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\ge {\alpha }_1{h}^{-1}\left(\frac{\alpha }{{\alpha }_1}F\left(x\right)\right)$ (15)

for $0<x\le x_2$. By virtue of (14) and (15), we have $y\left(x\right)\ge h^{-1}\left(\frac{\alpha }{{\alpha }_1}F\left(x\right)\right)$ for $0<x\le x_2$. Because $h\left(y\right)$ is strictly increasing, we get $h\left(y\left(x\right)\right)\ge \frac{\alpha }{{\alpha }_1}F\left(x\right)$ for $0<x\le x_2$. Thus, $F\left(x\right)-h\left(y\left(x\right)\right)\le {\alpha }_{2}F\left(x\right)$ with ${\alpha }_2=1-\frac{\alpha }{{\alpha }_1}$. Repeating this procedure, we obtain two sequences $\left\{x_n\right\}$ and $\left\{{\alpha }_n\right\}$ such that ${\alpha }_n=1-\frac{\alpha }{{\alpha }_{n-1}}$ and $F\left(x\right)-h\left(y\left(x\right)\right)\le {\alpha }_{n}F\left(x\right)$ for $0<x\le x_n$. If ${\alpha }_n\le 0$, we have a contradiction. Suppose ${\alpha }_n>0$ $\left(n=1,2,\cdots \right)$, then $\left({\alpha }_n-{\alpha }_{n-1}\right)\left(1-{\alpha }_n\right)=-{\alpha }_n+{\alpha }_n-\alpha <-{\left({\alpha }_n-\frac{1}{2}\right)}^2\le 0$, $\left\{{\alpha }_n\right\}$ is decreasing, and hence $\left\{{\alpha }_n\right\}$ converges to some real number $\lambda$. On the other hand, $\lambda =1-\frac{\alpha }{\lambda }$ and $\alpha >\frac{1}{4}$ show that $\lambda$ is a complex number, which is a contradiction. This completes the proof.

By a similar argument, we have the following lemma in the left half plane.

Lemma 3.2 Suppose that the conditions (A₀), (A₁), and ( ${\text{A}}_2^{-}$) hold. Then for any $P=\left(x_0\mathrm{,}{y}_0\right)\in C^{-}$, the positive semiorbit ${\gamma }^{+}\left(P\right)$ intersects the positive y-axis.

Remark 3.1. If $h\left(y\right)\equiv y$, then condition ( ${\text{A}}_2^{+}$)₂ is condition (ii) of (6) in Section 1 (cf. [20] [21] [27] ).

Remark 3.2. By the above discussion, condition (A₂) is a generalization of condition (A₃) in [1] , condition (A₁₀) in [15] , condition (A₂) in [16] , and condition (C) in [12] .

The final assumptions presented here are to guarantee that all positive orbits ${\gamma }^{+}\left(P\right)$ for $\in D_1$ (resp., $P\in D_3$) intersect $C^{+}$ (resp., $C^{-}$).

We say (1) satisfies the assumption (A₃) if both ( ${\text{A}}_3^{+}$) and ( ${\text{A}}_3^{-}$) hold.

The system (1) is said to satisfy ( ${\text{A}}_3^{+}$) if one of the following conditions holds:

( ${\text{A}}_3^{+}$)₁ $\underset{x\to \infty }{\bar{\lim }}F\left(x\right)\ne -\infty$;

( ${\text{A}}_3^{+}$)₂ $\underset{x\to \infty }{\bar{\lim }}F\left(x\right)=-\infty$, and there exist $\beta >\frac{1}{4}$ and $N_1>0$ such that $F\left(x\right)<0$ for $x\ge N_1$, and for any fixed $k\ge 1$ and $b\ge N_1$, there exists $\bar{b}>b$ satisfying

${\displaystyle {\int }_b^x}\frac{g\left(s\right)}{F\left(s\right)}\text{d}s\le \frac{1}{k}{h}^{-1}\left(k\beta F\left(x\right)\right)\text{for}x\ge \bar{b}.$

The system (1) is said to satisfy ( ${\text{A}}_3^{-}$) if one of the following conditions hold:

( ${\text{A}}_3^{-}$)₁ $\underset{x\to -\infty }{\underset{\_}{\lim }}F\left(x\right)\ne \infty$;

( ${\text{A}}_3^{-}$)₂ $\underset{x\to -\infty }{\underset{\_}{\lim }}F\left(x\right)=\infty$, and there exist $\beta >\frac{1}{4}$ and $N_1>0$ such that $F\left(x\right)>0$ for $x\ge -N_1$, and for any fixed $k\ge 1$ and $b\ge N_1$, there exist $\bar{b}>b$ satisfying

${\displaystyle {\int }_{-b}^x}\frac{g\left(s\right)}{F\left(s\right)}\text{d}s\ge \frac{1}{k}{h}^{-1}\left(k\beta F\left(x\right)\right)\text{for}x\le -\bar{b}\mathrm{.}$

Lemma 3.3 Suppose that the conditions (A₀), (A₁), and ( ${\text{A}}_3^{+}$) hold. Then every positive semiorbit of (1) departing from $D_1$ intersects the characteristic curve $C^{+}$ if and only if

$\underset{x\to \infty }{\bar{\lim }}{\displaystyle {\int }_0^x}\frac{g\left(s\right)}{1+F_{-}\left(s\right)}\text{d}s=\infty$ or $\underset{x\to \infty }{\bar{\lim }}F\left(x\right)=\infty$, (16)

where $F_{-}\left(x\right)\mathrm{<mo>\; =\; </mo>}\max \left\{\mathrm{0,}-F\left(x\right)\right\}$.

Proof. Sufficiency. Suppose the conclusion is false. Then there is a point $P=\left(x_0\mathrm{,}{y}_0\right)\in D_1$ such that ${\gamma }^{+}\left(P\right)$ does not intersect $C^{+}$. Let $\left(x\left(t\right)\mathrm{,}y\left(t\right)\right)$ $\left(t\ge 0\right)$ be the solution of (3) passing through such a point P whose maximal existence interval is $\left[\mathrm{0,}{\omega }_{+}\right)$. Note that $x^{\prime }\left(t\right)>0$ and $y^{\prime }\left(t\right)<0$ in the region $D_1$, hence $x\left(t\right)$ is increasing and $y\left(t\right)$ is decreasing as t is increasing. Suppose that $x\left(t\right)$ is bounded, then $\left(x\left(t\right)\mathrm{,}y\left(t\right)\right)$ stays in the region $\{\left(x,y\right):0<x<K_1$, and $h\left(y\right)>F\left(x\right)\}$ for some $K_1>0$. Hence it must intersect the characteristic curve, which is a contradiction. Therefore $x\left(t\right)\to \infty$ as $t\to {\omega }_{+}$.

Case 1: Suppose $\underset{x\to \infty }{\bar{\lim }}F\left(x\right)=\infty$, that is, there exists a sequence $\left\{x_n\right\}$ such that $x_n\to \infty$ $\left(n\to \infty \right)$ and $\underset{n\to \infty }{lim}F\left(x_n\right)=\infty$, then $\left(x\left(t\right)\mathrm{,}y\left(t\right)\right)$ must intersect the characteristic curve, which is a contradiction.

Case 2: Suppose ${\displaystyle {\int }_0^{\infty }}\frac{a^2\left(x\right)g\left(x\right)}{1+F_{-}\left(x\right)}\text{d}x=\infty$, then

$\begin{array}{c}y\left(t\right)-y_0=-{\displaystyle {\int }_0^t}a\left(x\left(s\right)\right)g\left(x\left(s\right)\right)\text{d}s\\ =-{\displaystyle {\int }_0^t}\frac{a^2\left(x\left(s\right)\right)g\left(x\left(s\right)\right)}{h\left(y\left(s\right)\right)-F\left(x\left(s\right)\right)}\stackrel{\dot{}}{x}\left(s\right)\text{d}s\\ =-{\displaystyle {\int }_{x_0}^{x\left(t\right)}}\frac{a^2\left(\xi \right)g\left(\xi \right)}{h\left(y\left(s\right)\right)-F\left(\xi \right)}\text{d}\xi \\ \le -{\displaystyle {\int }_{x_0}^{x\left(t\right)}}\frac{a^2\left(\xi \right)g\left(\xi \right)}{h\left(y_0\right)+F_{-}\left(\xi \right)}\text{d}\xi \to -\infty \end{array}$

as $t\to {\omega }_{+}$. Then the orbit of the above solution can be considered as a function $y\left(x\right)$ which is a solution of the equation (4), and $y\left(x\right)\to -\infty$ as $x\to \infty$.

Case ( ${\text{A}}_3^{+}$)₁: There exist $c>0$ and a sequence $\left\{x_n\right\}$ such that $x_n\to \infty$ $\left(n\to \infty \right)$, and $F\left(x_n\right)\ge -c$, hence $\left(x\left(t\right)\mathrm{,}y\left(t\right)\right)$ must intersect the characteristic curve, which is a contradiction.

Case ( ${\text{A}}_3^{+}$)₂: There exists $b>N_1$ such that $F\left(x\right)<0$ and $y\left(x\right)<0$ for $x\ge b$. Since $y\left(x\right)$ is a solution of (4), putting $H_3\left(u\right)={\displaystyle {\int }_0^u}h\left(y\right)\text{d}y$ for $u\le 0$, we have

$\begin{array}{c}{H}_3\left(y\left(x\right)\right)-H_3\left(y\left(b\right)\right)={\displaystyle {\int }_b^x}{H^{\prime }}_3\left(y\left(s\right)\right)\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)-h\left(y\left(s\right)\right)}\text{d}s\\ \ge {\displaystyle {\int }_b^x}h\left(y\left(s\right)\right)\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\\ ={\displaystyle {\int }_b^x}\left(h\circ H_3^{-1}\right)H_3\left(y\left(s\right)\right)\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\end{array}$

for $x\ge b$. Hence

$H_3\left(y\left(x\right)\right)\ge {\displaystyle {\int }_b^x}\left(-h\circ H_3^{-1}\right)H_3\left(y\left(s\right)\right)\frac{a^2\left(s\right)g\left(s\right)}{-F\left(s\right)}\text{d}s$

for $x\ge b$. It follows from Lemma 2.1 that

$H_4\left(H_3\left(y\left(x\right)\right)\right)\ge {\displaystyle {\int }_b^x}\frac{a^2\left(s\right)g\left(s\right)}{-F\left(s\right)}\text{d}s\text{for}x\ge b\mathrm{,}$ (17)

where $H_4\left(u\right)={\displaystyle {\int }_{0^{+}}^u}\frac{\text{d}t}{\left(-h\circ H_3^{-1}\right)\left(t\right)}$. Changing variable $H_3^{-1}\left(t\right)=\tau$, then $H_4\left(u\right)=-H_3^{-1}\left(u\right)$, by (17), it is easy to see that

$y\left(x\right)\le {\displaystyle {\int }_b^x}\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\text{for}x\ge b\mathrm{.}$ (18)

From the assumption ( ${\text{A}}_3^{+}$)₂, there exist $\beta >\frac{1}{4}$ and $b_1>b$ such that

${\displaystyle {\int }_b^x}\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\le h^{-1}\left(\beta F\left(x\right)\right)\text{for}x\ge b_1\mathrm{.}$ (19)

By virtue of (18) and (19), we have $y\left(x\right)\le h^{-1}\left(\beta F\left(x\right)\right)$ for $x\ge b_1$. Because $h\left(y\right)$ is strictly increasing, we obtain $h\left(y\left(x\right)\right)\le \beta F\left(x\right)$ for $x\ge b_1$. Hence $F\left(x\right)-h\left(y\left(x\right)\right)\ge {\beta }_{1}F\left(x\right)$ for $x\ge b_1$, where ${\beta }_1=1-\beta$. By a similar argument, we have

$\begin{array}{c}{H}_3\left(y\left(x\right)\right)-H_3\left(y\left(b_1\right)\right)={\displaystyle {\int }_{b_1}^x}h\left(y\left(s\right)\right)\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)-h\left(y\left(s\right)\right)}\text{d}s\\ \ge \frac{1}{{\beta }_1}{\displaystyle {\int }_{b_1}^x}h\left(y\left(s\right)\right)\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\\ =\frac{1}{{\beta }_1}{\displaystyle {\int }_{b_1}^x}\left(h\circ H_3^{-1}\right)H_3\left(y\left(s\right)\right)\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\end{array}$

for $x\ge b_1$. Hence

$H_3\left(y\left(x\right)\right)\ge \frac{1}{{\beta }_1}{\displaystyle {\int }_{b_1}^x}\left(-h\circ H_3^{-1}\right)H_3\left(y\left(s\right)\right)\frac{a^2\left(s\right)g\left(s\right)}{-F\left(s\right)}\text{d}s$

for $x\ge b_1$. By Lemma 2.1, it can be shown that

$H_4\left(H_3\left(y\left(x\right)\right)\right)\ge \frac{1}{{\beta }_1}{\displaystyle {\int }_{b_1}^x}\frac{a^2\left(s\right)g\left(s\right)}{-F\left(s\right)}\text{d}s,$

$y\left(x\right)\le \frac{1}{{\beta }_1}{\displaystyle {\int }_{b_1}^x}\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s$ (20)

for $x\ge b_1$. From the assumption ( ${\text{A}}_5^{+}$)₂, there exists $b_2>b_1$ such that

${\displaystyle {\int }_{b_1}^x}\frac{a^2\left(s\right)g\left(s\right)}{F\left(s\right)}\text{d}s\le {\beta }_1{h}^{-1}\left(\frac{\beta }{{\beta }_1}F\left(x\right)\right)\text{for}x\ge b_2.$ (21)

By virtue of (20) and (21), we have $y\left(x\right)\le h^{-1}\left(\frac{\beta }{{\beta }_1}F\left(x\right)\right)$ for $x\ge b_2$. Thus $F\left(x\right)-h\left(y\left(x\right)\right)\ge {\beta }_{2}F\left(x\right)$ for $x\ge b_2$, where ${\beta }_2=1-\frac{\beta }{{\beta }_1}$. Repeating this procedure, we obtain two sequences $\left\{b_n\right\}$ and $\left\{{\beta }_n\right\}$ such that ${\beta }_n=1-\frac{\beta }{{\beta }_{n-1}}$ and $F\left(x\right)-h\left(y\left(x\right)\right)\ge {\beta }_{n}F\left(x\right)$ for $x\ge b_n$. If ${\beta }_n>0$ $\left(n=1,2,\cdots \right)$, then $\left\{{\beta }_n\right\}$ is decreasing, and $\left\{{\beta }_n\right\}$ converges to some real number $\lambda$, on the other hand $\lambda =1-\frac{\beta }{\lambda }$ and $\beta >\frac{1}{4}$ show that $\lambda$ is a complex number, which is a contradiction. Hence, ${\beta }_n\le 0$ for some n, that is $F\left(x\right)\ge h\left(y\left(x\right)\right)$ for all $x\ge b_n$, a contradiction. This completes the proof of sufficiency.

Necessity. Suppose (16) does not hold. Then there exist $M_1>0$ and $L>0$ such that $F\left(x\right)<M_1$ for $x\ge 0$ and ${\displaystyle {\int }_L^{\infty }}\frac{a^2\left(x\right)g\left(x\right)}{1+F_{-}\left(x\right)}\text{d}x<1$. Suppose $\left(x\left(t\right)\mathrm{,}y\left(t\right)\right)$ is a solution of (3), and $\left(x\left(0\right),y\left(0\right)\right)=\left(L\mathrm{,}{M}_1+M_0+1\right)=P$ where $M_0>0$ satisfying $h\left(M_1+M_0\right)\ge M_1+1$.

We will show that $y\left(t\right)>M_1+M_0$ for $t>0$. Suppose not. There exists $t_1>0$ such that $y\left(t_1\right)=M_1+M_0$ and $M_1+M_0<y\left(t\right)\le M_1+M_0+1$ for all $t\in \left[\mathrm{0,}{t}_1\right)$, and we have

$\begin{array}{c}y\left(t_1\right)=M_1+M_0+1-{\displaystyle {\int }_0^{t_1}}\frac{a^2\left(x\left(s\right)\right)g\left(x\left(s\right)\right)}{h\left(y\left(s\right)\right)-F\left(x\left(s\right)\right)}\stackrel{\dot{}}{x}\left(s\right)\text{d}s\\ \ge M_1+M_0+1-{\displaystyle {\int }_L^{x\left(t_1\right)}}\frac{a^2\left(\xi \right)g\left(\xi \right)}{1+F_{-}\left(\xi \right)}\text{d}\xi >M_1+M_0.\end{array}$

This is a contradiction. Hence, $\stackrel{\dot{}}{x}\left(t\right)=h\left(y\left(t\right)\right)-F\left(x\left(t\right)\right)>M+1-F\left(x\left(t\right)\right)>1$ for all $t\ge 0$. Thus the solution $\left(x\left(t\right)\mathrm{,}y\left(t\right)\right)$ is unbounded and ${\gamma }^{+}\left(P\right)$ is above the characteristic curve $h\left(y\right)=F\left(x\right)$. Thus the necessity is proved. This completes the proof.

In a similar way, we can prove the following lemma in the left half plane.

Lemma 3.4 Suppose that the conditions (A₀), (A₁), and ( ${\text{A}}_3^{-}$) hold. Then every positive semiorbit of (1) departing from $D_3$ intersects the characteristic curve $C^{-}$ if and only if

$\underset{x\to -\infty }{\bar{\lim }}{\displaystyle {\int }_0^x}\frac{g\left(s\right)}{1+F_{+}\left(s\right)}\text{d}s=\infty$ or $\underset{x\to -\infty }{\bar{\lim }}\left(-F\left(x\right)\right)=\infty$, (22)