Gross--Prasad periods for reducible representations

We study GL_2(F)-invariant periods on representations of GL_2(A), where F is a nonarchimedean local field and A/F a product of field extensions of total degree 3. For irreducible representations, a theorem of Prasad shows that the space of such periods has dimension at most 1, and is non-zero when a certain epsilon-factor condition holds. We give an extension of this result to a certain class of reducible representations (of Whittaker type), extending results of Harris--Scholl when A is the split algebra F x F x F.


Introduction
One of the central problems in the theory of smooth representations of reductive groups over nonarchimdean local fields is to determine when a representation of a group G admits a linear functional invariant under a closed subgroup H (an H-invariant period).
The Gross-Prasad conjectures [GP92] give a very precise and elegant description of when such periods exist, for many natural pairs (G, H), in terms of ε-factors. However, the original formulation of these conjectures applies to members of generic L-packets for G; and the analogous picture for representations in non-generic L-packets is rather more complex. Although the ε-factor is still well-defined for all such Lpackets, the conjecture formulated in [GGP20] only applies when the L-parameters satisfy an additional "relevance" condition, raising the natural question of whether the ε-factors for non-relevant L-packets have any signficance in terms of invariant periods.
In this short note, we describe some computations of branching laws in the following simple case: G is GL 2 (A), where A/F is a cubicétale algebra, and H is the subgroup GL 2 (F ). Our computations suggest an alternative approach to the theory: rather than studying branching laws for non-generic irreducible representations, we focus on representations which are possibly reducible, but satisfy a certain "Whittaker type" condition. We show that H-invariant periods on these representations are unique if they exist, and that their existence is governed by ε-factors, extending the results of Prasad [Pra90,Pra92] for irreducible generic representations, and Harris-Scholl [HS01] for A the split algebra (in which case the ε-factor is always +1). In this optic, the "relevance" condition appears as a criterion for the H-invariant period to factor through the unique irreducible quotient.
This result, combined with other recent works such as that of Chan [Ch21] in the case (G, H) = (GL n (F ) × GL n+1 (F ), GL n (F )), would seem to suggest that many "Gross-Prasad-style" branching results should extend to Whittaker-type representations, and we hope to explore this further in future works.
We conclude with an application to global arithmetic. For π a Hilbert modular form over a real quadratic field, the constructions of [Kin98,LLZ18,Gro20] give rise to a family of cohomology classes taking values in the 4-dimensional Asai Galois representation associated to π. We show that if π is not of CM type and not a base-change from Q, then these elements all lie in a 1-dimensional subspace. This is the analogue for quadratic Hilbert modular forms of the result proved in [HS01] for Beilinson's elements attached to the Rankin convolution of two modular forms.

Statements
Throughout this paper, F denotes a non-archimedean local field of characteristic 0. If G is a reductive group over F , then a "representation" of G(F ) shall mean a smooth linear representation on a complex vector space.
2.1. Epsilon-factors. We choose a nontrivial additive character ψ of F . For Weil-Deligne representations ρ of F , we define epsilon-factors ε(ρ) = ε(ρ, ψ) following Langlands (the "ε L " convention in [Tat79, §3.6]), so that ε(ρ) is independent of ψ if det(ρ) = 1. We note that where det(ρ) is identified with a character of F × via class field theory. We write sp(n) for the n-dimensional Weil-Deligne representation given by the (n − 1)-st symmetric power of the Langlands parameter of the Steinberg representation, so that the eigenvalues of the Frobenius element on sp(n) are q where q is the size of the residue field.
2.2. The generic Langlands correspondence for GL 2 . The classical local Langlands correspondence for GL 2 is a bijection between irreducible smooth representations of GL 2 (F ), and 2-dimensional Frobenius-semisimple representations of the Weil-Deligne group of F . In this paper, we will use the following modification of the correspondence. A representation of GL 2 (F ) is said to be of Whittaker type if it is either irreducible and generic, or a reducible principal series representation with 1-dimensional quotient. (These are precisely the representations of GL 2 (F ) which have well-defined Whittaker models.) The generic Langlands correspondence is a bijection between Whittaker-type representations of GL 2 (F ) and 2-dimensional Frobenius-semisimple Weil-Deligne representations; it agrees with the classical Langlands correspondence on irreducible generic representations, and maps a reducible Whittaker-type principal series to the classical Langlands parameter of its 1-dimensional quotient. 1 In particular, the unramified Weil-Deligne representation with Frobenius acting as q 1/2 q −1/2 corresponds to the reducible principal series Σ F containing the Steinberg representation St F as subrepresentation and trivial 1-dimensional quotient. (We omit the subscript F if it is clear from context.) 2.3. Statement of the theorem. We now state our main theorem. Let A/F be a separable cubic algebra, so A is a product of field extensions of F of total degree 3. Let ω A be the quadratic character of F × determined by the class of disc(A) in F × /F ×2 . We let G = GL 2 (A), and H = GL 2 (F ), embedded in G in the obvious way.
The Langlands dual group of GL 2 /A has a natural 8-dimensional Asai, or multiplicative induction, representation; in the case A = F 3 this is simply the tensor product of the defining representations of the factors. We use this representation, and the generic Langlands correspondence for GL 2 above, to define Asai epsilon-factors ε(As(Π)) for Whittaker-type representations of GL 2 (A).
Finally, we consider Jacquet-Langlands transfers. Let H = D × where D/F is the unique non-split quaternion algebra. Let G = (D ⊗ F A) × , and let Π be the Jacquet-Langlands transfer of Π to G if this exists, and 0 otherwise.
Main Theorem. Let Π be a representation of GL 2 (A) of Whittaker type, whose central character is trivial on F × (embedded diagonally in A × ). Then we have and dim Hom H (Π, 1) + dim Hom H (Π , 1) = 1.
If Π is an irreducible generic representation, then this is the main result of [Pra90] for A the split algebra, and [Pra92] for non-split A (modulo the case of supercuspidal representations of cubic fields, completed in [PSP08]). The new content of the above theorem is that this also holds for reducible Whittaker-type Π.
Remark 2.2. Any such Π can be written as the specialisation at s = 0 of an analytic family of Whittakertype representations Π(s) indexed by a complex parameter s, which are irreducible for generic s and all have central character trivial on F × . For such families, the ε-factors ε(As Π(s)) are locally constant as a function of s; hence, given the results of [Pra92, PSP08] in the irreducible case, our theorem is equivalent to the assertion that dim Hom H (Π(s), 1) and dim Hom H (Π(s) , 1) are locally constant in s.

2.4.
Relation to results of Moeglin-Waldspurger. The Proposition in section 1.3 of [MW12] gives a formula for branching multiplicities for certain parabolically-induced representations of special orthogonal groups SO(d)×SO(d ) (with d−d odd), expressing these in terms of multiplicities for irreducible tempered representations of smaller special orthogonal groups. These results are applied in op.cit. to prove the Gross-Prasad conjecture for irreducible representations in non-tempered generic L-packets (by reduction to the tempered case); but the results are also valid for reducible representations.
Since the split form of SO(3) is PGL(2), and SO(4) is closely related to PGL(2) × PGL(2), one can derive many cases of our Main Theorem from their result applied to various forms of SO(3) × SO(4). In fact, if A = F 3 or A = E × F for E quadratic, we can obtain in this way all cases of the Main Theorem not already covered by Prasad's results.
However, the case when A is a cubic field extension does not appear to fit into the framework of op.cit.; and the proof given in op.cit. is rather indirect, particularly in the case when the SO(3) representation is reducible, in which case their argument requires a delicate switch back and forth between representations of SO(3) × SO(4) and SO(4) × SO(5). So we hope that the alternative, more direct approach given here will be of interest.
If the π i are all irreducible, then the above is the main result of [Pra90]. If one or more of the π i is isomorphic to a twist of Σ F , then the ε-factor is automatically +1, and π 1 ⊗ π 2 ⊗ π 3 is the zero representation. So all that remains to be shown is that in this case we have dim Hom GL2(F ) (π 1 ⊗ π 2 ⊗ π 3 , 1) = 1. This is estabished in Propositions 1.5, 1.6 and 1.7 of [HS01], except for one specific case, which is when all three of the π i are twists of Σ by characters.
In this case, by twisting we may assume π 2 = π 3 = Σ and π 1 = Σ ⊗ η where η is a character of F × with η 2 = 1. The case η = 1 is covered by Proposition 1.7 of op.cit., so we assume η is a nontrivial quadratic character. In this case Hom , 1), which has dimension 1 by [HS01, Proposition 1.6]. Thus Hom H (π 1 ⊗ π 2 ⊗ π 3 , 1) has dimension ≤ 1. Since one can easily write down a nonzero element of this space using the Rankin-Selberg zeta integral, we conclude that its dimension is 1 as required.
Remark 4.2. Note that the case when E/F is unramified, and π is unramified and tempered, is part of [Gro20, Theorem 4.1.1]. However, the proof of this statement loc.cit. has a minor error which means the argument does not work when π is the normalised induction of the trivial character of B E . So the argument below fixes this small gap.
Proof. We first observe that Hom H (π Σ F , 1) is non-zero. Since π is generic, it has a Whittaker model W(π) with respect to any non-trivial additive character of E. We may suppose that this additive character is trivial on F , so that we may define the Asai zeta-integral for W ∈ W(π) and Φ ∈ S(F 2 ) (the space of Schwartz functions on F ). Here N H is the upper-triangular unipotent subgroup of H.
It is well known that this integral converges for (s) 0 and has meromorphic continuation to the whole complex plane; and the values of Z(−, −, s) span a nonzero fractional ideal of C[q s , q −s ], generated by an L-factor independent of Φ and W , which is the Asai L-factor L(As(π), s). Thus the map defines a non-zero, H-invariant bilinear form W (π) ⊗ S(F 2 ) → C. Since the maximal quotient of S(F 2 ) on which F × acts trivially is isomorphic to Σ F (see e.g. [Loe20, Proposition 3.3(b)]), this shows that Hom H (π Σ F , 1) = 0 as claimed. So, to prove Theorem 4.1(a), it suffices to show that dim Hom H (π Σ F , 1) ≤ 1. As π has unitary central character, it is either a discrete-series representation, in which case it is automatically tempered, or an irreducible principal series, which may or may not be tempered. We shall consider these cases separately.
Remark 4.3. It follows, in particular, that for a generic irreducible representation π of GL 2 (E), we have Hom H (π, 1) = 0 (i.e. π is "F -distinguished") if and only if the zeta-integral ( †) factors through the 1dimensional quotient of Σ F , and thus vanishes on all Φ with Φ(0, 0) = 0; that is, s = 0 is an exceptional pole of the Asai L-factor. This is the n = 2 case of a theorem due to Matringe [Mat10, Theorem 3.1] applying to GL n (E)-representations. See [Loe20] for analogous results and conjectures regarding poles of zeta-integrals for GSp 4 and GSp 4 × GL 2 .
For case (b) of the main theorem, we need the following lemma: Lemma 4.4. Let σ be an irreducible generic representation of GL 2 (F ) with ω σ = 1. Then ε(As(Σ E ) × σ) = ε(σ)ε(σ × ω E/F ). Moreover, if σ = St F , then we have Proof. If σ is not a twist of Steinberg, then its Weil-Deligne representation has trivial monodromy action, so we compute that Since σ has trivial central character, ε(σ) = ±1. If σ is supercuspidal we are done, since in this case ρ I F σ = 0. If σ is principal series, then ρ I F σ must be either 0, or all of ρ σ , since ρ σ has determinant 1. Thus det(− Frob : ρ I F σ ) = 1, so ε(As(St E ) × σ) = ε(σ)ε(σ × ω E/F ), proving the claim in this case. The case when σ is a twist of the Steinberg by a nontrivial (necessarily quadratic) character can be computed similarly.
Proof. We first consider the situation for H . This case is relatively simple, since H is compact modulo centre, and hence the functor of H -invariants is exact on the category of H -representations trivial on Using Prasad's results for Hom H (St E ⊗ σ , 1) and the preceding lemma, we see that dim Hom H (Σ E ⊗ σ , 1) has dimension 1 if ε(σ)ε(σ × ω E/F ) = −ω E/F (−1) and is zero otherwise, as required.
For the group H, the situation is a little more complicated: since σ is generic, we have Hom H (σ, 1) is zero, and hence there is an exact sequence σ, 1). Claim: The group Ext 1 PGL2(F ) (σ, 1) is 1-dimensional if σ = St F , and zero otherwise. Proof of Claim: If σ is supercuspidal the result is immediate, since σ is projective in the category of PGL 2 (F )-representations. The remaining cases can be handled directly using Frobenius reciprocity, or alternatively, one can appeal to Schneider-Stuhler duality (as reformulated in [NP20, Theorem 2]) to show that the Ext group is dual to Hom H (D(1), σ) where D is the Aubert-Zelevinsky involution, which sends 1 to St F . This gives the desired formula for dim Hom H (Σ E ⊗ σ, 1) in all cases except when σ = St F , in which case we must show that the non-trivial H-invariant period of St E ⊗ St F does not lift to Σ E ⊗ St F . This can be done directly: we can compute Σ E | GL2(F ) via Mackey theory, using the two orbits of H on P 1 (E) to obtain the exact sequence The latter representation is irreducible and has no homomorphisms to St F ; and we saw in the proof of Theorem 4.1(a) that Remark 4.2. We are grateful to the anonymous referee for pointing out the significance of the vanishing of Ext 1 PGL2(F ) (σ, 1); the original version of this paper used a different and rather more complicated argument. Proof. The computation of the epsilon-factor is immediate; and by a zeta-integral argument as before, we can show that Hom H (Σ E Σ F , η) = 0 (since the representation Σ E , despite being reducible, has a well-defined Whittaker model). So it suffices to show that the hom-space has dimension ≤ 1.
If η is trivial, then we have seen above that Hom H (Σ E St F , 1) is zero. So Hom H (Σ E Σ F , 1) = Hom H (Σ E , 1). From the Mackey decomposition of Σ E | GL2(F ) above, one sees easily that this space is 1-dimensional.

Cubic fields
We briefly discuss the case where A is a cubic extension of F .
Proof. The case of irreducible generic π is proved in [Pra92] assuming π non-supercuspidal, and the supercuspidal case is filled in by [PSP08]. In this case, the only example of a reducible Whittaker-type representation of G is Σ E ⊗ η, where η is a character of E × ; and the central-character condition implies that λ = η| F × must be trivial or quadratic.
To complete the proof, we must show that when λ = 1, the H-invariant homomorphism Hom H (St E , λ) extends to Σ E . However, this is clear since the obstruction lies in Ext 1 H (1, λ), which is zero. This completes the proof of the Main Theorem.

An application to Euler systems
We now give a global application, a strengthening of some results from [LLZ18] and [Gro20] on Euler systems for quadratic Hilbert modular forms. Let K/Q be a real quadratic field and write G = Res K/Q (GL 2 ) and H = GL 2/Q ⊂ G; set G f = G(A f ) = GL 2 (A K,f ) and H f similarly.
6.1. Adelic representations. Let χ be a finite-order character of A × f and define a representation of H f by I(χ) = I (χ ), where I (χ ) denotes the representation of H given by normalised induction of the character χ | · | 1/2 | · | −1/2 of the Borel subgroup. For χ = 1, we let I 0 (1) denote the codimension 1 subrepresentation of I(1). Exactly as in [HS01,§2], the local results above imply the following branching law for G f -representations: Proposition 6.1. Let π be an irreducible admissible representation of G f , all of whose local factors are generic, with ω π | A × f = χ −1 .
6.2. Hilbert modular forms. Suppose now that π is (the finite part of) a cuspidal automorphic representation, arising from a Hilbert modular cusp form of parallel weight k + 2 ≥ 2, normalised so that ω π has finite order.
Proposition 6.2. Suppose π is not a twist of a base-change from GL 2/Q . Then, for any Dirichlet character τ , there exist infinitely many primes such that Hom H (π ⊗ τ , 1) = 0.
There is a natural H f -representation O × (Y ) C of modular units, where Y is the infinite-level modular curve (the Shimura variety for GL 2 ). Note that this representation is smooth, but not admissible. It fits into a long exact sequence with H f acting on (Q ab ) × via the Artin reciprocity map of class field theory, and the sum is over all even Dirichlet characters η.
There is a canonical homorphism, the Asai-Flach map, constructed in [LLZ18] (building on several earlier works such as [Kin98]): AF [π,k] : π ⊗ O × (Y ) C H f → H 1 (Q, V As (π) * (−k)), where V As (π) is the Asai Galois representation attached to π, and we have fixed an isomorphism Q p ∼ = C. The subscript H f indicates H f -coinvariants. Theorem 6.3. Suppose π is not a twist of a base-change from Q. Then the Asai-Flach map factors through π ⊗ I(χ), and its image is contained in a 1-dimensional subspace of H 1 (Q, V As (π) * (−k)).
As in the GSp 4 case described in [LZ20, §6.6], one can remove the dependency on the test data entirely: using zeta-integrals, we can construct a canonical basis vector Z can ∈ Hom(π f ⊗ I(χ), 1), and define AF [π,k] can ∈ H 1 Q, V As (π) * (−k) as the unique class such that AF [π,k] = Z can · AF [π,k] can . We hope that this perspective may be useful in formulating and proving explicit reciprocity laws in the Asai setting.
Remark 6.4. The constructions of [LLZ18] also apply to other twists of V As (π), and to Hilbert modular forms of non-parallel weight; but in these other cases the input data for the Asai-Flach map lies in an irreducible principal series representation of H f , so the necessary multiplicity-one results are standard. (The delicate cases are those which correspond to near-central values of L-series.)