Liouville property of fractional Lane-Emden equation in general unbounded domain

Our purpose of this paper is to consider Liouville property for the fractional Lane-Emden equation (−∆)αu = up in Ω, u = 0 in RN \ Ω, where α ∈ (0, 1), N ≥ 1, p > 0 and Ω ⊂ RN−1 × [0, +∞) is an unbounded domain satisfying that Ωt := {x′ ∈ RN−1 : (x′, t) ∈ Ω} with t ≥ 0 has increasing monotonicity, that is, Ωt ⊂ Ωt′ for t′ ≥ t. The shape of Ω∞ := limt→∞ Ωt in RN−1 plays an important role to obtain the nonexistence of positive solutions for the fractional Lane-Emden equation.


Introduction
In this paper, we consider Liouville property for the fractional Lane-Emden equation where α ∈ ( , ), p > , Ω is an unbounded domain in R N with N ≥ , and (−∆) α with α ∈ ( , ) is the fractional Laplacian de ned in the principle value sense, here Bϵ( ) is the ball with radius ϵ centered at the origin and c N,α > is the normalized constant. We say that u is a bounded solution of (1.1) if u ∈ C(R N ) ∩ L ∞ (R N ) and u satis es (1.1) pointwisely.
As an important property, the Liouville theorem for Lane-Emden equation has attracted a lot of attentions by many mathematician by the application in the derivation of uniform bound via blowing-up analysis. Note that the nonexistence of stable solution is studied in [1] by nite morse index with restrictions on the boundary and at in nity. Without the zero Dirichlet boundary condition, Liouville results could be obtained by Hadamard property in [2,3], by iterating the decaying rate at in nity in [4] and by Hardy estimates in [5].
It is known that the Leray-Schauder degree theory is a very useful method for deriving solutions of elliptic equations on bounded domains. The essential step is to obtain a uniform bound by considering a sequence of solutions {un}n such that un(xn) = un L ∞ = Mn → +∞ as n → +∞ for some {xn}n ⊂ Ω. Then let Ωn = {x ∈ R N : M κ n x + xn ∈ Ω} and vn(x) = Mn un( M κ n x + xn) for some κ > , then {vn}n is uniformly bounded, *Corresponding Author: Ying Wang, Department of Mathematics, Jiangxi Normal University, Nanchang, Jiangxi 330022, PR China, E-mail: yingwang00@126.com Yuanhong Wei, School of Mathematics, Jilin University, Changchun, Jilin 130012, PR China, E-mail: weiyuanhong@jlu.edu.cn and the limit v∞ of {vn}n is a solution of related limit semilinear equation in the limit domain Ω ∞ . Note that for C domain Ω, the limit domain of Ωn is either Ω ∞ = R N or Ω ∞ = R N− × ( , ∞). While if the original domain contains a cone point on the boundary, then the limit domain has the third possibility that Ω ∞ is a cone. As a consequence, the nonexistence of positive solutions to the limit equation in a cone has to be involved additionally. As far as we know, the nonexistence of elliptic equation depends on the shape of limit domain at in nity in some direction. Our concern in this article is to consider the non-existence of elliptic equations in one type of unbounded domain.
Recently, qualitative properties of solutions for nonlocal elliptic equations have been studied extensively, such as the existence of weak solutions or very weak solutions in [6,7] by variational methods, a survey in [8] on variational methods, large solution [9] by Perron's method, the regularities in [10], Pohozaev's identity [11] and Liouville results [12][13][14]. In [15] it develops the method of moving plane in fractional setting to obtain the classi cation of critical elliptic equations in an integral form and then it is used to obtain nonexistence of bounded solutions for semilinear elliptic equations in half space [12,13,16] subject to various boundary type conditions. In particular, Fall and Weth in [13] obtained the nonexistence of positive bounded solution of (1.1) when Ω = R N + and p < N− + α N− − α , by applying the method of moving plane, via some interesting estimates of Green's kernel in R N + . It is worth noting that star-shaped domain with respect to in nity is involved in obtaining the nonexistence of fractional Lane-Emden equation in [14] and it is an important notation in our derivation of nonexistence to (1.1).
Before stating our main result, we introduce the following notations.
For t ≥ , O t is nonempty, bounded and the mapping t → O t is increasing in the following sense Now we introduce two types domains: The main results state as follows.
and Ω∞ is given as (1.2) with O = Ω. Then problem (1.1) has no nonnegative, nontrivial and bounded solutions if one of the following holds: Our basic tool is the traditional method of moving plane, involved by [17] in the fractional setting, we develop this traditional method of moving planes to obtain the increasing monotonicity in the direction x N and reduce problem (1.1) into subject to zero Dirichlet boundary condition when Ω∞ ≠ R N− , where (−∆) α R N− is the fractional Laplacian in R N− . Then the nonexistence results could be obtained for (1.3) as in [13,14]. Remark 1.1. In the particular case that Ω is a cone such as {x = (x , x N ) ∈ R N : x N > θ|x |} for some θ > , problem (1.1)

has no nonnegative, nontrivial and bounded solutions.
If Ω∞ also veri es the similar assumptions of Theorem 1.1, we can repeat our above procedure and derive the following corollary directly:

The proof of nonexistence results
For the domain Ω verifying (D), we shall prove that the solution of (1.1) has the x N -increasing property by using the method of moving planes. To this end, we introduce the following notations. For λ > , denote For any subset A of R N , we write A λ = {x λ : x ∈ A} the re ection of A with regard to T λ . Since Ω t is bounded for t ≥ , then the domain Σ λ is always bounded for any λ > .
Proof. We divide the proof into two steps.
Step 1: By the assumption of Ω ⊂ R N + , we may assume The purpose of this step is to show that if λ > λ is close to λ , then w λ > in Σ λ . To this end, let By contradiction, we assume (2.5) is not true, that is Σ − λ ≠ ∅. We denote It is obvious that w + λ (x) = w λ (x) − w − λ (x) for all x ∈ R N . By direct computation, for x ∈ Σ − λ , we have We look at each of these integrals separately. Since u = in Ω λ \ Ω and u λ = in Ω \ Ω λ , then by the fact that u λ ≥ and |x − z λ | > |x − z| for all x ∈ Σ − λ and z ∈ Ω λ \ Ω. In order to x the sign of I , note that w λ (z λ ) = −w λ (z) for any z ∈ R N and then Hence, we obtain that for all λ > λ , and then for x ∈ Σ − λ , Combining (1.1) with (2.9) and (2.6), we have that Hence, we have (2.10) and observe that w + λ = in (Σ − λ ) c , then we have that where c is a positive constant independent of Σ − λ and Proof of Theorem 1.1. We prove this argument by contradiction. Assume that u ≥ is nontrivial solution of (1.1). When Ω veri es (D), then by Proposition 2.1, we have that u satis es (2.4). Let then {um}m is an increasing and bounded sequence of functions and satis es Note that Ω t ⊂ Ω t for ≤ t ≤ t < +∞, By the increasing property of {um}m, we have that which implies that u∞ is x N -independent. Letting v∞(x ) = u∞(x , x N ), by the standard argument, we have that , ∀ x ∈ Ω∞ × R, then v∞ is a positive, bounded and classical solution for