July 16, 2008

UNITARY SPHERICAL SPECTRUM FOR SPLIT CLASSICAL GROUPS

DAN BARBASCH

1. Introduction

                                                                       {sec:1}

This paper gives a complete classification of the spherical unitary dual of the split groups Sp(n) and So(n) over the real and p−adic field. Most of the results were known earlier from [B1], [B2], [B3] and [BM3]. As is explained in these references, in the p−adic case the classification of the spherical unitary dual is equivalent to the classification of the unitary generic (in the sense of admitting Whittaker models) Iwahori-spherical modules. The new result is the proof of necessary conditions for unitarity in the real case. Following a suggestion of D. Vogan, I find a set of K−types which I call relevant which detect the nonunitarity. They have the property that they are in 1-1 correspondence with certain irreducible Weyl group representations (called relevant) so that the intertwining operators are the same in the real and p−adic case. The fact that these relevant W −types detect unitarity in the p−adic case is also new. Thus the same proof applies in both cases. Since the answer is independent of the field, this establishes a form of the Lefschetz principle.

Let G be a split symplectic or orthogonal group over a local field 𝔽 which is either ℝ or a p−adic field. Fix a maximal compact subgroup K. In the real case, there is only one conjugacy class. In the p−adic case, let K = G(R) where 𝔽 ⊃ R ⊃ P, with R the ring of integers and P the maximal prime ideal. Fix also a rational Borel subgroup B = AN. Then G = KB. An admissible representation (π, V) is called spherical if V^{K} ≠ (0).

The classification of irreducible admissible spherical modules is well known. For every irreducible spherical π, there is a character χ ∈ Â such that χ|_{A∩K} = triv, and π is the unique spherical subquotient of Ind_{B}^{G}[χ ⊗ 11]. We will call a character χ whose restriction to A∩K is trivial, unramified. Write X(χ) for the induced module (principal series) and L(χ) for the irreducible spherical subquotient. Two such modules L(χ) and L(χ′) are equivalent if and only if there is an element in the Weyl group W such that wχ = χ′. An L(χ) admits a nondegenerate hermitian form if and only if there is w ∈ W such that wχ = −\overline{χ}.

The character χ is called real if it takes only positive real values. For real groups, χ is real if and only if L(χ) has real infinitesimal character ([K], chapter 16). As is proved there, any unitary representation of a real reductive group with nonreal infinitesimal character is unitarily induced from a unitary representation with real infinitesimal character on a proper Levi component. So for real groups it makes sense to consider only real infinitesimal character. In the p−adic case, χ is called real if the infinitesimal character is real in the sense of [BM2]. The results in [BM1] show that the problem of determining the unitary irreducible representations with Iwahori fixed vectors is equivalent to the same problem for the Iwahori-Hecke algebra. In [BM2], it is shown that the problem of classifying the unitary dual for the Hecke algebra reduces to determining the unitary dual with real infinitesimal character of some smaller Hecke algebra (not necessarily one for a proper Levi subgroup). So for p−adic groups as well it is sufficient to consider only real χ.

So we start by parametrizing real unramified characters of A. Since G is split, A ≅ (𝔽^{×})^{n} where n is the rank. Define

𝔞^{∗} = X^{∗}(A) ⊗_{ℤ} ℝ, {(1.0.1) \atop \{eq:1.1\}}

where X^{∗}(A) is the lattice of characters of the algebraic torus A. Each element ν ∈ 𝔞^{∗} defines an unramified character χ_{ν} of A, characterized by the formula

χ_{ν}(τ(f)) = |f|^{⟨τ,ν⟩}, f ∈ 𝔽^{×}, {(1.0.2) \atop \{eq:1.2\}}

where τ is an element of the lattice of one parameter subgroups X_{∗}(A). Since the torus is split, each element of X_{∗}(A) can be regarded as a homomorphism of 𝔽^{×} into A. The pairing in the exponent in (1.0.2) corresponds to the natural identification of 𝔞^{∗} with Hom[X_{∗}(A), ℝ]. The map ν ⟶ χ_{ν} from 𝔞^{∗} to real unramified characters of A is an isomorphism. We will often identify the two sets writing simply χ ∈ 𝔞^{∗}.

Let Ǧ be the (complex) dual group, and let Ǎ be the torus dual to A. Then 𝔞^{∗} ⊗_{ℝ}ℂ is canonically isomorphic to 𝔞̌, the Lie algebra of Ǎ. So we can regard χ as an element of 𝔞̌. We attach to each χ a nilpotent orbit Ǒ(χ) as follows. By the Jacobson-Morozov theorem, there is a 1-1 correspondence between nilpotent orbits Ǒ and Ǧ-conjugacy classes of Lie triples {ě, ȟ, f̌}; the correspondence satisfies ě ∈ Ǒ. Choose the Lie triple such that ȟ ∈ 𝔞̌. Then there are many Ǒ such that χ can be written as wχ = ȟ/2 + ν with ν ∈ 𝔷(ě, ȟ, f̌), the centralizer in 𝔤̌ of the triple. For example this is always possible with Ǒ = (0). The results in [BM1] guarantee that for any χ there is a unique Ǒ(χ) satisfying

(1) there exists w ∈ W such that wχ = \frac{1}{2} ȟ + ν with ν ∈ 𝔷(ě, ȟ, f̌),

(2) if χ satisfies property (1) for any other Ǒ′, then Ǒ′ ⊂ \overline{Ǒ}(χ).

Here is another characterization of the orbit Ǒ(χ). Let

𝔤̌_{1} := { x ∈ 𝔤̌ : [χ, x] = x },  𝔤̌_{0} := {x ∈ 𝔤̌ : [χ, x] = 0 }.

Then Ǧ_{0}, the Lie group corresponding to the Lie algebra 𝔤̌_{0} has an open dense orbit in 𝔤̌_{1}. Its Ǧ saturation in 𝔤̌ is Ǒ(χ).

The pair (Ǒ(χ), ν) has further nice properties. For example assume that ν = 0 in (1) above. Then the representation L(χ) is one of the parameters that the Arthur conjectures predict to play a role in the residual spectrum. In particular, L(χ) should be unitary. In the p−adic case one can verify the unitarity directly as follows. In [BM1] it is shown how to calculate the Iwahori-Matsumoto dual of L(χ) in the Kazhdan-Lusztig classification of representations with Iwahori-fixed vector. It turns out that in the case ν = 0, it is a tempered module, and therefore unitary. Since the results in [BM1] show that the Iwahori-Matsumoto involution preserves unitarity, L(χ) is unitary as well. In the real case, a direct proof of the unitarity of L(χ) (still with ν = 0 as in (1) above) is given in [B3], and in section 9 of this paper.

In the classical Lie algebras, the centralizer 𝔷(ě, ȟ, f̌) is a product of symplectic and orthogonal Lie algebras. We will often abbreviate it as 𝔷(Ǒ). The orbit Ǒ is called distinguished if 𝔷(Ǒ) does not contain a nontrivial torus; equivalently, the orbit does not meet any proper Levi component. Let 𝔪̌_{BC} be the centralizer of a Cartan subalgebra in 𝔷(Ǒ). This is the Levi component of a parabolic subalgebra. The subalgebra 𝔪̌_{BC} is the Levi subalgebra attached to Ǒ by the Bala-Carter classification of nilpotent orbits. The intersection of Ǒ with 𝔪̌_{BC} is the other datum attached to Ǒ, a distinguished orbit in 𝔪̌_{BC}. We will usually denote it 𝔪̌_{BC}(Ǒ) if we need to emphasize the dependence on the nilpotent orbit. Let M_{BC} ⊂ G be the Levi subgroup whose Lie algebra 𝔪_{BC} has 𝔪̌_{BC} as its dual.

The parameter χ gives rise to a spherical irreducible representation L_{M_{BC}}(χ) on M_{BC} as well as a L(χ). Then L(χ) is the unique spherical irreducible subquotient of

I_{M_{BC}}(χ) := Ind_{M_{BC}}^{G}[L_{M_{BC}}(χ)].  (1.0.3) {eq:1.3}

To motivate why we consider M_{BC}(Ǒ), we need to recall some facts about the Kazhdan-Lusztig classification of representations with Iwahori fixed vectors in the p-adic case. Denote by τ the Iwahori-Matsumoto involution. Then the space of Iwahori fixed vectors of τ(L(χ)) is a W −representation (see 5.2), and contains the W −representation sgn. Irreducible representations with Iwahori-fixed vectors are parametrized by Kazhdan-Lusztig data; these are Ǧ conjugacy classes of (ě, χ, ψ) where ě ∈ 𝔤̌ is such that [χ, ě] = ě, and ψ is an irreducible representation of the component group A(χ, ě). To each such parameter there is associated a standard module X(ě, χ, ψ) which contains a unique irreducible submodule L(ě, χ, ψ). All other factors have parameters (ě′, χ′, ψ′) such that

Ǒ(ě) ⊂ \overline{Ǒ(ě′)},  Ǒ(ě) ≠ Ǒ(ě′).

As explained in section 4 and 8 in [BM1], X(ě′, χ′, ψ′) contains sgn if and only if ψ′ = triv. Thus if we assume Ǒ satisfies (1) and (2) with respect to χ, it follows that X(ě, χ, triv) = L(ě, χ, triv). We would like it to equal I_{M_{BC}} but this is not true. In general (for an M which contains M_{BC}), L(ě, χ, triv) = Ind_{M}^{G}[X_{M}(ě, χ, triv)] if and only if the component A_{M}(ě, χ) equals the component group A(ě, χ). We will enlarge M_{BC}(Ǒ) to an M_{KL} so that A_{M_{KL}}(ě, χ) = A(ě, χ). Note that if 𝔪̌ ⊂ 𝔪̌′, then A_{M}(ě, χ) ⊂ A_{M′}(ě, χ). Then

Ind_{M_{KL}}^{G}[X_{M_{KL}}(ě, χ, triv)] = X(ě, χ, triv) = L(ě, χ, triv) {(1.0.4) \atop \{eq:1.4\}}

and

L(χ) = I_{M_{KL}}(χ) := Ind_{M_{KL}}^{G}[L_{M_{KL}}(χ)] {(1.0.5) \atop \{eq:1.5\}}

follows by applying τ. We remark that M_{KL} depends on χ as well as ě. It will be described explicitly in section 2. A more general discussion about how canonical 𝔪̌_{KL} is, appears in [BC2].

In the real case, we use the same Levi components as in the p−adic case. Then equality (1.0.5) does not hold for any proper Levi component. A result essential for the paper is that equality does hold at the level of multiplicities of the relevant K−types (section 4.2).

We will use the data (Ǒ, ν) to parametrize the unitary dual. Fix an Ǒ. A representation L(χ) will be called a complementary series attached to Ǒ, if it is unitary, and Ǒ(χ) = Ǒ. To describe it, we need to give the set of ν such that L(χ) with χ = ȟ/2 + ν is unitary. Viewed as an element of 𝔷(Ǒ), the element ν gives rise to a spherical parameter ((0), ν) where (0) denotes the trivial nilpotent orbit. The main result in section 3.2 says that the ν giving rise to the complementary series for Ǒ coincide with the ones giving rise to the complementary series for (0) on 𝔷(Ǒ). This is suggestive of Langlands functoriality.

It is natural to conjecture that such a result will hold for all split groups. Recent work of D. Ciubotaru for F_{4}, and by D. Ciubotaru and myself for the other exceptional cases, show that this is generally true, but there are exceptions.

I give a more detailed outline of the paper. Section 2 reviews notation from earlier papers. Section 3 gives a statement of the main results. A representation is called spherical unipotent if its parameter is of the form ȟ/2 for the neutral element of a Lie triple associated to a nilpotent orbit Ǒ. The unitarity of the spherical unipotent representations is dealt with in section 8. For the p−adic case I simply cite [BM3]. The real case (sketched in [B2]) is redone in section 9.5. The proofs are simpler than the original ones because I take advantage of the fact that wave front sets, asymptotic supports and associated varieties “coincide” due to [SV]. Section 10.1 proves an irreducibility result in the real case which is clear in the p−adic case from the work of Kazhdan-Lusztig. This is needed for determining the complementary series (definition 3.1 in section 3.1).

Sections 4 and 5 deal with the nonunitarity. The decomposition χ = ȟ/2 + ν is introduced in section 3. It is more common to parametrize the χ by representatives in 𝔞̌ which are dominant with respect to some positive root system. We use Bourbaki’s standard realization of the positive system. It is quite messy to determine the data (Ǒ, ν) from a dominant parameter, because of the nature of the nilpotent orbits and the Weyl group. Sections 2.3 and 2.8 give a combinatorial description of (Ǒ, ν) starting from a dominant χ.

In the classical cases, the orbit Ǒ is given in terms of partitions. To such a partition we associate the Levi component

𝔪̌_{BC} := gl(a_{1}) × ⋯ × gl(a_{k}) × 𝔤̌_{0}(n_{0})

given by the Bala-Carter classification. (The 𝔤̌_{0} in this formula is not related to the one just after conditions (1) and (2)). The intersection of Ǒ with 𝔪̌_{BC} is an orbit of the form

(a_{1}) × ⋯ × (a_{r}) × Ǒ_{0}

where Ǒ_{0} is a distinguished nilpotent orbit, and (a_{i}) is the principal nilpotent orbit on gl(a_{i}). This is the distinguished orbit associated to Ǒ by Bala-Carter. Then χ gives rise to irreducible spherical modules L_{M}(χ), L(χ) and I_{M}(χ) as in (1.0.3) and (1.0.5). The module L(χ) is the irreducible spherical subquotient of I_{M}(χ). As already mentioned, I_{M_{KL}}(χ) = L(χ) in the p−adic case, but not the real case. In all cases, the multiplicities of the relevant K−types in L(χ), I_{M}(χ) coincide. These are representations of the Weyl group in the p−adic case, representations of the maximal compact subgroup in the real case. Their definition is in section 4.2; they are a small finite set of representations which provide necessary conditions for unitarity which are also sufficient. The relationship between the real and p−adic case is investigated in chapter 4. In particular the issue is addressed of how the relevant K−types allow us to deal with the p−adic case only. A more general class of K−types for split real groups (named petite K-types), on which the intertwining operator is equal to the p-adic operator, is defined in [B6], and the proofs are more conceptual. Sections 4.4, and 4.5 are included for completeness. The interested reader can consult [B6] and [BC1] for results where these kinds of K−types and W −types are useful.

The determination of the nonunitary parameters proceeds by induction on the rank of 𝔤̌ and by the inclusion relations of the closure of the orbit Ǒ. Section 5 completes the induction step; it shows that conditions (B) in section 3.1 is necessary. The last part of the induction step is actually done in section 3.1.

I would like to thank David Vogan for generously sharing his ideas about the relation between K−types, Weyl group representations and signatures. They were the catalyst for this paper.

This research was supported by NSF grants DMS-9706758,DMS-0070561 and DMS-03001712.

2. Description of the spherical parameters

                                                                       {sec:2}

                                                                       {1.1}

2.1. Explicit Langlands parameters. We consider spherical irreducible representations of the split connected classical groups of rank n of type B, C, D, precisely, G = So(2n + 1), G = Sp(2n) and G = So(2n). These groups will be denoted by G(n) when there is no danger of confusion (n is the rank). Levi components will be written as

M = GL(k_{1}) × ⋯ × GL(k_{r}) × G_{0}(n_{0}), {(2.1.1) \atop \{eq:1.1levi\}}

where G_{0}(n_{0}) is the factor of the same type as G. The corresponding complex Lie algebras are denoted 𝔤(n) and 𝔪 = gl(k_{1}) × ⋯ × gl(k_{r}) × 𝔤_{0}(n_{0}).

As already explained in the introduction, we deal with real unramified characters only. In the case of classical groups, such a character can be represented by a vector of size the rank of the group. Two such vectors parametrize the same irreducible spherical module if they are conjugate via the Weyl group which acts by permutations and sign changes for type B, C and by permutations and an even number of sign changes in type D. For a given χ, let L(χ) be the corresponding irreducible spherical module. We will occasionally refer to χ as the infinitesimal character.

For any nilpotent orbit Ǒ ⊂ 𝔤̌ we attach a parameter χǑ ∈ 𝔞^{∗} as follows. Recall from the introduction that 𝔞^{∗} ⊗_{ℝ} ℂ is canonically isomorphic to 𝔞̌. Let {ě, ȟ, f̌} be representatives for the Lie triple associated to a nilpotent orbit Ǒ. Then χǑ := ȟ/2.

Conversely, to each χ we will attach a nilpotent orbit Ǒ ⊂ 𝔤̌ and Levi components M_{BC}, M_{KL} := GL(k_{1}) × ⋯ × GL(k_{r}) × G_{0}(n_{0}). In addition we will specify an even nilpotent orbit Ǒ_{0} ⊂ 𝔤̌_{0}(n_{0}) with unramified character χ_{0} := χǑ_{0} on 𝔤_{0}(n_{0}), and unramified characters χ_{i} on the GL(k_{i}). These data have the property that L(χ) is the spherical subquotient of

Ind_{M_{KL}}^{G}[⨂_{i} L(χ_{i}) ⊗ L(χ_{0})]. {(2.1.2) \atop \{1.1.0\}}

{1.2a}

2.2. We introduce the following notation (a variant of the one used by Zelevinski [ZE]).

{d:1.1}

Definition. A string is a sequence

(a, a + 1, …, b − 1, b)

of numbers increasing by 1 from a to b. A set of strings is called nested if for any two strings either the coordinates do not differ by integers, or if they do, then their coordinates, say (a_{1}, …, b_{1}) and (a_{2}, …, b_{2}), satisfy

a_{1} ≤ a_{2} ≤ b_{2} ≤ b_{1}  or  a_{2} ≤ a_{1} ≤ b_{1} ≤ b_{2}, {(2.2.1) \atop \{eq:d1\}}

or

b_{1} + 1 < a_{2}  or  b_{2} + 1 < a_{1}. □ {(2.2.2) \atop \{eq:d2\}}

A set of strings is called strongly nested if the coordinates of any two strings either do not differ by integers or else satisfy (2.2.1).

Each string represents a 1-dimensional spherical representation of a GL(n_{i}) with n_{i} = b_{i} − a_{i} + 1. The matchup is

(a, …, b) ⟷ | det |^{\frac{a+b}{2}} , of GL(b − a + 1).

(2.2.3) {eq:1.1.1a}

In the case of G = GL(n), we record the following result. For the p−adic case, it originates in the work of Zelevinski, and Bernstein-Zelevinski ([ZE] and references therein). To each set of strings (a_{1}, …, b_{1}; …; a_{k}, …, b_{k}) we can attach a Levi component M_{BC} := ∏_{1≤i≤k} GL(n_{i}), and an induced module

I(χ) := Ind_{M_{BC}}^{GL(n)}[⨂ L(χ_{i})]  (2.2.4) {eq:1.1.1b}

where χ_{i} are obtained from the strings as in (2.2.3).

In general, if the set of strings is not nested, then the corresponding induced module is not irreducible. The coordinates of χ in 𝔞^{∗} ≃ ℝ^{n} determine a set of nested strings as follows. Extract the longest sequence starting with the smallest element in χ that can form a string. Continue to extract sequences from the remainder until there are no elements left. This set of strings is, up to the order of the strings, the unique set of nested strings one can form out of the entries of χ.

                                                                       {t:2.1}

Theorem. Suppose 𝔽 is p-adic. Let (a_{1}, …, b_{1}; … a_{r}, …, b_{r}) be a set of nested strings, and M := GL(b_{1} − a_{1} + 1) × ⋯ × GL(b − r − a_{r} + 1). Then

L(χ) = Ind_{M}^{GL(n)}[ | det |^{\frac{a_{1}+b_{1}}{2}} · · · · · | det |^{\frac{ar+br}{2}} ].

In the language of section 2.1, M_{BC} = M_{KL} = M, where M is the one defined in the theorem. The nilpotent orbit Ǒ corresponds to the partition of n with entries (b_{i} − a_{i} + 1); it is the orbit Ǒ(χ) satisfying (1) and (2) in the introduction, with respect to χ = (a_{1}, …, b_{1}; …; a_{r}, …, b_{r}).

For the real case (still GL(n)), the induced module in theorem 2.2 fails to be irreducible. However equality holds on the level of multiplicities of relevant K-types. These K-types will be defined in section 4.7.

We will generalize this procedure to the other classical groups. As before, the induced modules that we construct fail to be irreducible in the real case, but equality of multiplicity of relevant K−types in the two sides of (1.0.5) holds.

                                                                       {sec:2.3}

2.3. Nilpotent orbits. In this section we attach a set of parameters to each nilpotent orbit Ǒ ⊂ 𝔤̌. Let {ě, ȟ, f̌} be a Lie triple so that ě ∈ Ǒ, and let 𝔷(Ǒ) be its centralizer. In order for χ to be a parameter attached to Ǒ we require that

χ = ȟ/2 + ν,  ν ∈ 𝔷(Ǒ), semisimple,  (2.3.1) {eq:2.3.1}

but also that if

χ = ȟ′/2 + ν′,  ν′ ∈ 𝔷(Ǒ′), semisimple  (2.3.2) {eq:2.3.2}

for another nilpotent orbit Ǒ′ ⊂ 𝔤̌, then Ǒ′ ⊂ \overline{Ǒ}. In [BM1], it is shown that the orbit of χ, uniquely determines Ǒ and the conjugacy class of ν ∈ 𝔷(Ǒ). We describe the pairs (Ǒ, ν) explicitly in the classical cases.

Nilpotent orbits are parametrized by partitions

(\underbrace{1, …, 1}_{r_{1}}, \underbrace{2, …, 2}_{r_{2}}, …, \underbrace{j, …, j}_{r_{j}}, …), {(2.3.3) \atop \{eq:2.3.3\}}

satisfying the following constraints.

A_{n−1}: gl(n), partitions of n.

B_{n}: so(2n + 1), partitions of 2n + 1 such that every even part occurs

an even number of times. C_{n}: sp(2n), partitions of 2n such that every odd part occurs an even

number of times. D_{n}: so(2n), partitions of 2n such that every even part occurs an even number of times. In the case when every part of the partition is even, there are two conjugacy classes of nilpotent orbits with the same Jordan blocks, labelled (I) and (II). The two orbits are conjugate under the action of O(2n).

The Bala-Carter classification is particularly well suited for describing the parameter spaces attached to the Ǒ ⊂ 𝔤̌. An orbit is called distinguished if it does not meet any proper Levi component. In type A, the only distinguished orbit is the principal nilpotent orbit, where the partition has only one part. In the other cases, the distinguished orbits are the ones where each part of the partition occurs at most once. In particular, these are even nilpotent orbits, i.e. ad ȟ has even eigenvalues only. Let Ǒ ⊂ 𝔤̌ be an arbitrary nilpotent orbit. We need to put it into as small as possible Levi component 𝔪̌. In type A, if the partition is (a_{1}, …, a_{k}), the Levi component is 𝔪̌_{BC} = gl(a_{1})×⋯×gl(a_{k}). In the other classical types, the orbit Ǒ meets a proper Levi component if and only if one of the r_{j} > 1. So separate as many pairs (a, a) from the partition as possible, and rewrite it as

((a_{1}, a_{1}), …, (a_{k}, a_{k}); d_{1}, …, d_{l}), {(2.3.4) \atop \{eq:2.3.4\}}

with d_{i} < d_{i+1}. The Levi component 𝔪̌_{BC} attached to this nilpotent by Bala-Carter is

𝔪̌_{BC} = gl(a_{1}) × ⋯ × gl(a_{k}) × 𝔤̌_{0}(n_{0}) n_{0} := n − ∑ a_{i}, {(2.3.5) \atop \{eq:2.3.5\}}

The distinguished nilpotent orbit is the one with partition (d_{i}) on 𝔤̌(n_{0}), principal nilpotent on each gl(a_{j}). The χ of the form ȟ/2 + ν are the ones with ν an element of the center of 𝔪̌_{BC}. The explicit form is

(…; −\frac{a_{i} − 1}{2} + ν_{i}, …, \frac{a_{i} − 1}{2} + ν_{i}, …; ȟ_{0}/2), {(2.3.6) \atop \{eq:2.3.6\}}

where ȟ_{0} is the middle element of a triple corresponding to (d_{i}). We will write out (d_{i}) and ȟ_{0}/2 in sections 2.4-2.7.

We will consider more general cases where we write the partition of Ǒ in the form (2.3.4) so that the d_{i} are not necessarily distinct, but (d_{i}) forms an even nilpotent orbit in 𝔤̌_{0}(n_{0}). This will be the situation for 𝔪̌_{KL}.

The parameter χ determines an irreducible spherical module L(χ) for G as well as an L_{M}(χ) for M = M_{BC} or M_{KL} of the form

L_{1}(χ_{1}) ⊗ ⋯ ⊗ L_{k}(χ_{k}) ⊗ L_{0}(χ_{0}),  (2.3.7) {eq:2.3.7}

where the L_{i}(χ_{i}) i = 1, …, k are one dimensional. We will consider the relation between the induced module

I_{M}(χ) := Ind_{M}^{G}[L_{M}(χ)],  (2.3.8) {eq:2.3.8}

and L(χ). {sec:2.3a}

2.4. G of Type A. We write the ȟ/2 for a nilpotent Ǒ corresponding to (a_{1}, …, a_{k}) with a_{i} ≤ a_{i+1} as

(…; −\frac{a_{i} − 1}{2}, …, \frac{a_{i} − 1}{2}; …).

The parameters of the form χ = ȟ/2 + ν are then

(…; −\frac{a_{i} − 1}{2} + ν_{i}, …, \frac{a_{i} − 1}{2} + ν_{i}; …).  (2.4.1) \{2.3a.1\}

Conversely, given a parameter as a concatenation of strings

χ = (…; A_{i}, …, B_{i}; …),  (2.4.2) {eq:2.3a.2}

it is of the form ȟ/2+ν where ȟ is the neutral element for the nilpotent orbit with partition (A_{i} + B_{i} + 1) (the parts need not be in any particular order) and ν_{i} = \frac{A_{i}−B_{i}}{2} . We recall the following well known result about closures of nilpotent orbits.

                                                                       {l:2.3a}

Lemma. Assume Ǒ and Ǒ′ correspond to the (increasing) partitions (a_{1}, …, a_{k}) and (b_{1}, …, b_{k}) respectively, where some of the a_{i} or b_{j} may be zero in order to have the same number k. The following are equivalent

(1) Ǒ′ ⊂ \overline{Ǒ}.

(2) ∑_{i≥s} a_{i} ≥ ∑_{i≥s} b_{i} for all k ≥ s ≥ 1.

                                                                       {p:2.3a}

Proposition. A parameter χ as in (2.4.1) is attached to Ǒ in the sense of satisfying (2.3.1) and (2.3.2) if and only if it is nested.

Proof. Assume the strings are not nested. There must be two strings

(A, …, B),  (C, …, D)  (2.4.3) {eq:2.3a.3}

such that A − C ∈ ℤ, and A < C ≤ B < D, or C = B + 1. Then by conjugating χ by the Weyl group to a χ′, we can rearrange the coordinates of the two strings in (2.4.3) so that the strings

(A, …, D), (C, … B),  or (A, …, D).  (2.4.4) {eq:2.3a.4}

appear. Then by the lemma, χ′ = ȟ′/2 + ν′ for a strictly larger nilpotent Ǒ′.

Conversely, assume χ = ȟ/2 + ν, so it is written as strings, and they are nested. The nilpotent orbit for which the neutral element is ȟ/2 has partition given by the lengths of the strings, say (a_{1}, … a_{k}) in increasing order. If χ is nested, then a_{k} is the length of the longest string of entries we can extract from the coordinates of χ, a_{k−1} the longest string we can extract from the remaining coordinates and so on. Then (2) of lemma 2.4 precludes the possibility that some conjugate χ′ equals ȟ′/2 + ν′ for a strictly larger nilpotent orbit. □

In type A, 𝔪̌_{KL} = 𝔪̌_{BC}.

{sec:2.3b}

2.5. G of Type B. Rearrange the parts of the partition of Ǒ ⊂ sp(2n, ℂ), in the form (2.3.4),

((a_{1}, a_{1}), …, (a_{k}, a_{k}); 2x_{0}, …, 2x_{2m}) {(2.5.1) \atop \{eq:2.3b.1\}}

The d_{i} have been relabeled as 2x_{i} and a 2x_{0} = 0 is added if necessary, to insure that there is an odd number. The x_{i} are integers, because all the odd parts of the partition of Ǒ occur an even number of times, and were therefore extracted as (a_{i}, a_{i}). The χ of the form ȟ/2 + ν are

\begin{matrix}(…; −\frac{a_{i} − 1}{2} + ν_{i}, & …, \frac{a_{i} − 1}{2} + ν_{i}; …; \\ \underbrace{1/2, …, 1/2}_{n_{1/2}}, …, \underbrace{x_{2m} − 1/2, …, x_{2m} − 1/2}_{n_{x_{2m}−1/2}}). \end{matrix} {(2.5.2) \atop \{eq:2.3b.2\}}

where

n_{l−1/2} = #\{x_{i} ≥ l\}. {(2.5.3) \atop \{eq:2.3b.3\}}

Lemma 2.4 holds for this type verbatim. So the following proposition holds.

{p:2.3b}

Proposition. A parameter χ = ȟ/2 + ν cannot be conjugated to one of the form ȟ′/2 + ν′ for any larger nilpotent Ǒ′ if and only if

(1) the set of strings satisfying \frac{a_{i}−1}{2} + ν_{i} − \frac{a_{j}−1}{2} − ν_{j} ∈ ℤ are nested.

(2) the strings satisfying \frac{a_{i}−1}{2} + ν_{i} ∈ 1/2 + ℤ satisfy the additional condition that either x_{2m} + 1/2 < −\frac{a_{i}−1}{2} + ν_{i} or there is j such that

x_{j} + 1/2 < −\frac{a_{i} − 1}{2} + ν_{i} ≤ \frac{a_{i} − 1}{2} + ν_{i} < x_{j+1} + 1/2. {(2.5.4) \atop \{eq:2.3b.4\}}

The Levi component 𝔪̌_{KL} is obtained from 𝔪̌_{BC} as follows. Consider the strings for which a_{i} is even, and ν_{i} = 0. If a_{i} is not equal to any 2x_{j}, then remove one pair (a_{i}, a_{i}), and add two 2x_{j} = a_{i} to the last part of (2.5.1). For example, if the nilpotent orbit Ǒ is

(2, 2, 2, 3, 3, 4, 4), {(2.5.5) \atop \{eq:2.3b.5\}}

then the parameters of the form ȟ/2 + ν are

\begin{aligned}( & − 1/2 + ν_{1}, 1/2 + ν_{1}; −1 + ν_{2}, ν_{2}, 1 + ν_{2}; \\ & − 3/2 + ν_{3}, −1/2 + ν_{3}, 1/2 + ν_{3}, 3/2 + ν_{3}; 1/2) \end{aligned} {(2.5.6) \atop \{eq:2.3b.6\}}

The Levi component is 𝔪̌_{BC} = gl(2) × gl(3) × gl(4) × 𝔤̌(1). If ν_{3} ≠ 0, then 𝔪̌_{BC} = 𝔪̌_{KL}. But if ν_{3} = 0, then 𝔪̌_{KL} = gl(2) × gl(3) × 𝔤̌(5). The parameter is rewritten

Ǒ ⟷ ((2, 2)(3, 3); 2, 4, 4)  (2.5.7) {eq:2.3b.7}

χ ⟷ (−1/2 + ν_{1}, 1/2 + ν_{1}; −1 + ν_{2}, ν_{2}, 1 + ν_{2}; 1/2, 1/2, 1/2, 3/2, 3/2).

The explanation is as follows. For a partition (2.3.3),

𝔷(Ǒ) = sp(r_{1}) × so(r_{2}) × sp(r_{3}) × …  (2.5.8) {eq:2.3b.8}

and the centralizer in Ǧ is a product of Sp(r_{2j+1}) and O(r_{2j}), i.e. Sp for the odd parts, O for the even parts. Thus the component group A(ȟ, ě), which by [BV2] also equals A(ě), is a product of ℤ_{2}, one for each r_{2j} ≠ 0. Then A(χ, ě) = A(ν, ȟ, ě). In general A_{M_{BC}}(χ, ě) = A_{M_{BC}}(ě) embeds canonically into A(χ, ě), but the two are not necesarily equal. In this case they are unless one of the ν_{i} = 0 for an even a_{i} with the additional property that there is no 2x_{j} = a_{i}.

We can rewrite each of the remaining strings

(−\frac{a_{i} − 1}{2} + ν_{i}, …, \frac{a_{i} − 1}{2} + ν_{i})  (2.5.9) \{eq:2.3b.9\}

as

χ_{i} :=(f_{i} + τ_{i}, f_{i} + 1 + τ_{i}, …, F_{i} + τ_{i}),  (2.5.10) {eq:2.3b.10}

satisfying

f_{i} ∈ ℤ + 1/2, 0 ≤ τ_{i} ≤ 1/2, F_{i} = f_{i} + a_{i}, .  (2.5.11) {eq:2.3b.11}

|f_{i} + τ_{i}| ≥ |F_{i} + τ_{i}| if τ_{i} = 1/2

This is done as follows. We can immediately get an expression like (2.5.10) with 0 ≤ τ_{i} < 1, by defining f_{i} to be the largest element in ℤ+1/2 less than or equal to −\frac{a_{i}−1}{2} +ν_{i}. If τ_{i} ≤ 1/2 we are done. Otherwise, use the Weyl group to change the signs of all entries of the string, and put them in increasing order. This replaces f_{i} by −F_{i} − 1, and τ_{i} by 1 − τ_{i}. The presentation of the strings subject to (2.5.11) is unique except when τ_{i} = 1/2. In this case the argument just given provides the presentation (f_{i} + 1/2, …, F_{i} + 1/2), but also provides the presentation

(−F_{i} − 1 + 1/2, …, −f_{i} − 1 + 1/2).  (2.5.12) {eq:2.3b.12}

We choose between (2.5.10) and (2.5.12) the one whose leftmost term is larger in absolute value. That is, we require |f_{i} + τ_{i}| ≥ |F_{i} + τ_{i}| whenever τ_{i} = 1/2.

                                                                       {sec:2.3c}

2.6. G of Type C. Rearrange the parts of the partition of Ǒ ⊂ so(2n + 1, ℂ), in the form (2.3.4),

((a_{1}, a_{1}), …, (a_{k}, a_{k}); 2x_{0} + 1, …, 2x_{2m} + 1);  (2.6.1) {eq:2.3c.1}

The d_{i} have been relabeled as 2x_{i} + 1. In this case it is automatic that there is an odd number of nonzero x_{i}. The x_{i} are integers, because all the even parts of the partition of Ǒ occur an even number of times, and were threrefore extracted as (a_{i}, a_{i}). The χ of the form ȟ/2 + ν are

(…; −\frac{a_{i} − 1}{2} + ν_{i}, …, \frac{a_{i} − 1}{2} + ν_{i}; …; \underbrace{0, …, 0}_{n_{0}}, …, \underbrace{x_{2m}, …, x_{2m}}_{n_{x_{2m}}}). {(2.6.2) \atop \{eq:2.3c.2\}}

where

n_{l} = \begin{cases}m & if l = 0, \\ #\{x_{i} ≥ l\} & if l ≠ 0. \end{cases} {(2.6.3) \atop \{eq:2.3c.3\}}

Lemma 2.4 holds for this type verbatim. So the following proposition holds.

{p:2.3c}

Proposition. A parameter χ = ȟ/2 + ν cannot be conjugated to one of the form ȟ′/2 + ν′ for any larger nilpotent Ǒ′ if and only if

(1) the set of strings satisfying \frac{a_{i}−1}{2} + ν_{i} − \frac{a_{j}−1}{2} − ν_{j} ∈ ℤ are nested.

(2) the strings satisfying \frac{a_{i}−1}{2} + ν_{i} ∈ ℤ satisfy the additional condition that either x_{2m} + 1 < −\frac{a_{i}−1}{2} + ν_{i} or there is j such that

x_{j} + 1 < −\frac{a_{i} − 1}{2} + ν_{i} ≤ \frac{a_{i} − 1}{2} + ν_{i} < x_{j+1} + 1. {(2.6.4) \atop \{eq:2.3c.4\}}

The Levi component 𝔪̌_{KL} is obtained from 𝔪̌_{BC} as follows. Consider the strings for which a_{i} is odd and ν_{i} = 0. If a_{i} is not equal to any 2x_{j} + 1, then remove one pair (a_{i}, a_{i}), and add two 2x_{j} + 1 = a_{i} to the last part of (2.6.1). For example, if the nilpotent orbit is

(1, 1, 1, 3, 3, 4, 4) = ((1, 1), (3, 3), (4, 4); 1), {(2.6.5) \atop \{eq:2.3c.5\}}

then the parameters of the form ȟ/2 + ν are

\begin{matrix}( & ν_{1}; −1 + ν_{2}, ν_{2}, 1 + ν_{2}; \\ − 3/2 + ν_{3}, −1/2 + ν_{3}, 1/2 + ν_{3}, 3/2 + ν_{3}) \end{matrix} {(2.6.6) \atop \{eq:2.3c.6\}}

The Levi component is 𝔪̌_{BC} = gl(1) × gl(3) × gl(4). If ν_{2} ≠ 0, then 𝔪̌_{BC} = 𝔪̌_{KL}. But if ν_{2} = 0, then 𝔪̌_{KL} = gl(1) × gl(4) × 𝔤̌(3). The parameter is rewritten

Ǒ ⟷ ((1, 1), (4, 4); 1, 3, 3) {(2.6.7) \atop \{eq:2.3c.7\}}

χ ⟷ (ν_{1}; −3/2 + ν_{3}, −1/2 + ν_{3}, 1/2 + ν_{3}, 3/2 + ν_{3}; 0, 1, 1).

The Levi component 𝔪̌_{KL} is unchanged if ν_{1} = 0.

The explanation is as follows. For a partition (2.3.3),

𝔷(Ǒ) = so(r_{1}) × sp(r_{2}) × so(r_{3}) × … {(2.6.8) \atop \{eq:2.3c.8\}}

and the centralizer in Ǧ is a product of O(r_{2j+1}) and Sp(r_{2j}), i.e. O for the odd parts, Sp for the even parts. Thus the component group is a product of ℤ_{2}, one for each r_{2j+1} ≠ 0. Then A(χ, ě) = A(ν, ȟ, ě), and so A_{M_{BC}}(χ, ě) = A(χ, ě) unless one of the ν_{i} = 0 for an odd a_{i} with the additional property that there is no 2x_{j} + 1 = a_{i}.

We can rewrite each of the remaining strings

(−\frac{a_{i} − 1}{2} + ν_{i}, …, \frac{a_{i} − 1}{2} + ν_{i})  (2.6.9) \{eq:2.3c.9\}

as

χ_{i} :=(f_{i} + τ_{i}, f_{i} + 1 + τ_{i}, …, F_{i} + τ_{i}),  (2.6.10) {eq:2.3c.10}

satisfying

f_{i} ∈ ℤ, 0 ≤ τ_{i} ≤ 1/2, F_{i} = f_{i} + a_{i}  (2.6.11) {eq:2.3c.11}

|f_{i} + τ_{i}| ≥ |F_{i} + τ_{i}| if τ_{i} = 1/2.

This is done as follows. We can immediately get an expression like (2.6.10) with 0 ≤ τ_{i} < 1, by defining f_{i} to be the largest element in ℤ less than or equal to −\frac{a_{i}−1}{2} +ν_{i}. If τ_{i} ≤ 1/2 we are done. Otherwise, use the Weyl group to change the signs of all entries of the string, and put them in increasing order. This replaces f_{i} by −F_{i} − 1, and τ_{i} by 1 − τ_{i}. The presentation of the strings subject to (2.6.11) is unique except when τ_{i} = 1/2. In this case the argument just given also provides the presentation

(−F_{i} − 1 + 1/2, …, −f_{i} − 1 + 1/2).  (2.6.12) {eq:2.3c.12}

We choose between (2.6.10) and (2.6.12) the one whose leftmost term is larger in absolute value. That is, we require |f_{i} + τ_{i}| ≥ |F_{i} + τ_{i}| whenever τ_{i} = 1/2.

                                                                       {sec:2.3d}

2.7. G of Type D. Rearrange the parts of the partition of Ǒ ⊂ so(2n, ℂ), in the form (2.3.4),

((a_{1}, a_{1}), …, (a_{k}, a_{k}); 2x_{0} + 1, …, 2x_{2m−1} + 1)  (2.7.1) {eq:2.3d.1}

The d_{i} have been relabeled as 2x_{i} + 1. In this case it is automatic that there is an even number of nonzero 2x_{i} + 1. The x_{i} are integers, because all the even parts of the partition of Ǒ occur an even number of times, and were therefore extracted as (a_{i}, a_{i}). The χ of the form ȟ/2 + ν are

(…; −\frac{a_{i} − 1}{2} + ν_{i}, …, \frac{a_{i} − 1}{2} + ν_{i}; …; \underbrace{0, …, 0}_{n_{0}}, …, \underbrace{x_{2m−1}, …, x_{2m−1}}_{n_{x_{2m−1}}}).

                                                         (2.7.2) {eq:2.3d.2}

where

n_{l} = \begin{cases}m & if l = 0, \\ #\{x_{i} ≥ l\} & if l ≠ 0. \end{cases}  (2.7.3) \{eq:2.3d.3\}

Lemma 2.4 holds for this type verbatim. So the following proposition holds.

                                                                       {p:2.3d}

Proposition. A parameter χ = ȟ/2 + ν cannot be conjugated to one of the form ȟ′/2 + ν′ for any larger nilpotent Ǒ′ if and only if

(1) the set of strings satisfying \frac{a_{i}−1}{2} + ν_{i} − \frac{a_{j}−1}{2} − ν_{j} ∈ ℤ are nested.

(2) the strings satisfying \frac{a_{i}−1}{2} + ν_{i} ∈ ℤ satisfy the additional condition that either x_{2m−1} + 1 < −\frac{a_{i}−1}{2} + ν_{i} or there is j such that

x_{j} + 1 < −\frac{a_{i} − 1}{2} + ν_{i} ≤ \frac{a_{i} − 1}{2} + ν_{i} < x_{j+1} + 1. {(2.7.4) \atop \{eq:2.3d.4\}}

The Levi component 𝔪̌_{KL} is obtained from 𝔪̌_{BC} as follows. Consider the strings for which a_{i} is odd and ν_{i} = 0. If a_{i} is not equal to any 2x_{j} + 1, then remove one pair (a_{i}, a_{i}), and add two 2x_{j} + 1 = a_{i} to the last part of (2.7.1). For example, if the nilpotent orbit is

(1, 1, 3, 3, 4, 4), {(2.7.5) \atop \{eq:2.3d.5\}}

then the parameters of the form ȟ/2 + ν are

\begin{matrix}( & ν_{1}; −1 + ν_{2}, ν_{2}, 1 + ν_{2}; \\ − 3/2 + ν_{3}, −1/2 + ν_{3}, 1/2 + ν_{3}, 3/2 + ν_{3}) \end{matrix} {(2.7.6) \atop \{eq:2.3d.6\}}

The Levi component is 𝔪̌_{BC} = gl(1) × gl(3) × gl(4). If ν_{2} ≠ 0 and ν_{1} ≠ 0, then 𝔪̌_{BC} = 𝔪̌_{KL}. If ν_{2} = 0 and ν_{1} ≠ 0, then 𝔪̌_{KL} = 𝔤̌(3) × gl(1) × gl(4). If ν_{2} ≠ 0 and ν_{1} = 0, then 𝔪̌_{KL} = gl(3) × gl(4) × 𝔤̌(1). If ν_{1} = ν_{2} = 0, then 𝔪̌_{KL} = gl(4) × 𝔤̌(4). The parameter is rewritten

Ǒ ⟷ ((1, 1), (4, 4); 3, 3) {(2.7.7) \atop \{eq:2.3d.7\}}

χ ⟷ (ν_{1}; −3/2 + ν_{3}, −1/2 + ν_{3}; 1/2 + ν_{3}, 3/2 + ν_{3}; 0, 1, 1).

The explanation is as follows. For a partition (2.3.3),

𝔷(Ǒ) = so(r_{1}) × sp(r_{2}) × so(r_{3}) × … {(2.7.8) \atop \{eq:2.3d.8\}}

and the centralizer in Ǧ is a product of O(r_{2j+1}) and Sp(r_{2j}), i.e. O for the odd parts, Sp for the even parts. Thus the component group is a product of ℤ_{2}, one for each r_{2j+1} ≠ 0. Then A(χ, ě) = A(ν, ȟ, ě), and so A_{M_{BC}}(χ, ě) = A(χ, ě) unless one of the ν_{i} = 0 for an odd a_{i} with the additional property that there is no 2x_{j} + 1 = a_{i}.

We can rewrite each of the remaining strings

(−\frac{a_{i} − 1}{2} + ν_{i}, …, \frac{a_{i} − 1}{2} + ν_{i}) {(2.7.9) \atop \{eq:2.3d.9\}}

as

χ_{i} :=(f_{i} + τ_{i}, f_{i} + 1 + τ_{i}, …, F_{i} + τ_{i}), {(2.7.10) \atop \{eq:2.3d.10\}}

satisfying f_{i} ∈ ℤ, 0 ≤ τ_{i} ≤ 1/2, F_{i} = f_{i} + a_{i} {(2.7.11) \atop \{eq:2.3d.11\}}

|f_{i} + τ_{i}| ≥ |F_{i} + τ_{i}| if τ_{i} = 1/2.

This is done as in types B and C, but see the remarks which have to do with the fact that −Id is not in the Weyl group. We can immediately get an expression like (2.7.10) with 0 ≤ τ_{i} < 1, by defining f_{i} to be the largest element in ℤ less than or equal to −\frac{a_{i}−1}{2} + ν_{i}. If τ_{i} ≤ 1/2 we are done. Otherwise, use the Weyl group to change the signs of all entries of the string, and put them in increasing order. This replaces f_{i} by −F_{i} − 1, and τ_{i} by 1 − τ_{i}. The presentation of the strings subject to (2.7.11) is unique except when τ_{i} = 1/2. In this case the argument just given also provides the presentation

(−F_{i} − 1 + 1/2, …, −f_{i} − 1 + 1/2).  (2.7.12) {eq:2.3d.12}

We choose between (2.7.10) and (2.7.12) the one whose leftmost term is larger in absolute value. That is, we require |f_{i} + τ_{i}| ≥ |F_{i} + τ_{i}| whenever τ_{i} = 1/2. Remarks

(1) A (real) spherical parameter χ is hermitian if and only if there is w ∈ W (D_{n}) such that wχ = −χ. This is the case if the parameter has a coordinate equal to zero, or if none of the coordinates are 0, but then n must be even.

(2) Assume the nilpotent orbit Ǒ is very even, i.e. all the parts of the partition are even (and therefore occur an even number of times). The nilpotent orbits labelled (I) and (II) are characterized by the fact that 𝔪̌_{BC} is of the form

\begin{aligned}(I) & ⟷ gl(a_{1}) × ⋯ × gl(a_{k−1}) × gl(a_{k}), \\ (II) & ⟷ gl(a_{1}) × ⋯ × gl(a_{k−1}) × gl(a_{k})′. \end{aligned}

The last gl factors differ by which extremal root of the fork at the end of the diagram for D_{n} is in the Levi component. The string for k is

\begin{aligned}(I) & ⟷ (−\frac{a_{k} − 1}{2} + ν_{k}, …, \frac{a_{k} − 1}{2} + ν_{k}), \\ (II) & ⟷ (−\frac{a_{k} − 1}{2} + ν_{k}, … \frac{a_{k} − 3}{2} + ν_{k}, −\frac{a_{k} − 1}{2} − ν_{k}). \end{aligned}

We can put the parameter in the form (2.7.10) and (2.7.11), because all strings are even length. In any case (I) and (II) are conjugate by the outer automorphism, and for unitarity it is enough to consider the case of (I).

The assignment of a nilpotent orbit (I) or (II) to a parameter is unambiguous. If a χ has a coordinate equal to 0, it might be written as h_{I}/2+ν_{I} or h_{II}/2+ν_{II}. But then it can also be written as h′/2+ν′ for a larger nilpotent orbit. For example, in type D_{2}, the two cases are (2, 2)_{I} and (2, 2)_{II}, and we can write

\begin{matrix}(I) ⟷ (1/2, −1/2) + (ν, ν), \\ (II) ⟷ (1/2, 1/2) + (ν, −ν). \end{matrix}

The two forms are not conjugate unless the parameter contains a 0. But then it has to be (1, 0) and this corresponds to (1, 3), the larger principal nilpotent orbit.

(3) Because we can only change an even number of signs using the Weyl group, we might not be able to change all the signs of a string. We can always do this if the parameter contains a coordinate equal to 0, or if the length of the string is even. If there is an odd length string, and none of the coordinates of χ are 0, changing all of the signs of the string cannot be achieved unless some other coordinate changes sign as well. However if χ = ȟ/2 + ν cannot be made to satisfy (2.7.10) and (2.7.11), then χ′, the parameter obtained from χ by applying the outer automorphism, can. Since L(χ) and L(χ′) are either both unitary of both nonunitary, it is enough to consider just the cases that can be made to satisfy (2.7.10) and (2.7.11). For example, the parameters

\begin{matrix}(−1/3, 2/3, 5/3; −7/4, −3/4, 1/4), \\ (−5/3, −2/3, 1/3; −7/4, −3/4, 1/4) \end{matrix}

in type D_{6} are of this kind. Both parameters are in a form satisfying (2.7.10) but only the second one satisfies (2.7.11). The first one cannot be conjugated by W (D_{6}) to one satisfying (2.7.11).

{sec:2.4}

2.8. Relation between infinitesimal characters and strings. In the previous sections we described for each nilpotent orbit Ǒ the parameters of the form ȟ/2 + ν with ν ∈ 𝔷(Ǒ) semisimple, along with condition (2.3.2). In this section we give algorithms to find the data (Ǒ, ν) satisfying (2.3.1) and (2.3.2), and the various Levi components from a χ ∈ 𝔞̌. The formulation was suggested by S. Sahi. Given a χ ∈ 𝔞̌, we need to specify,

(a): strings, same as sequences of coordinates with increment 1, (b): a partition, same as a nilpotent orbit Ǒ ⊂ 𝔤̌, (c): the centralizer of a Lie triple corresponding to 𝔷(Ǒ), (d): coordinates of the parameter ν, coming from the decompositon

χ = ȟ/2 + ν, Furthermore, we give algorithms for

(e): two Levi components 𝔪̌_{BC} and 𝔪̌_{KL}, (f): another two Levi components 𝔪̌_{e} and 𝔪̌_{o}, (g): one dimensional characters χ_{e} and χ_{o} of the Levi components 𝔪̌_{e}

and 𝔪̌_{o}. Parts (f) and (g) are described in detail in section 5.3. These Levi components are used to compute multiplicities of relevant K−types in L(χ).

Algorithms for (a) and (b).

Step 0. G of type C. Double the number of 0’s and add one more. G of type D. Double the number of 0’s. If there are no coordinates equal to 0 and the rank is odd, the parameter is not hermitian. If the rank is even, only an even number of sign changes are allowed in the subsequent steps.

Step 1. G of type C,D. Extract maximal strings of the form (0, 1, …). Each contributes a part in the partition of size 2(length of string)-1 to Ǒ. G of type B. Extract maximal strings of type (1/2, 3/2, …). Each contributes a part of size 2(length) to Ǒ.

Step 2.

For all types, extract maximal strings from the remaining entries after Step 1, changing signs if necessary. Each string contributes two parts of size (length of string) to Ǒ. In type D, if the rank is odd and no coordinate of the original χ is 0, the parameter is not hermitian. If there are no 0’s and the rank of type D is even, only an even number of sign changes is allowed. In this case, the last string might be (…, b, −b − 1). If so, and all strings are of even size, Ǒ is very even, and is labelled II. If all strings are of the form (…, b, b + 1), then the very even orbit is labelled I.

Algorithms for (c).

G of type C,D. 𝔷(Ǒ) = so(m_{1}) × sp(m_{2}) × so(m_{3}) × …, where m_{i} are the number of parts of Ǒ equal to i. G of type B. 𝔷(Ǒ) = sp(m_{1}) × so(m_{2}) … sp(m_{3}) × … where again m_{i} is the number of parts of Ǒ equal to i.

Algorithms for (d).

The parameter ν is a vector of size equal to rk G. For each 𝔷_{i}, of 𝔷(O), add rk 𝔷_{i} coordinates each equal to the average of the string cooresponding to the size i part of Ǒ. For each factor 𝔷_{i}, of 𝔷(Ǒ), the coordinates are the averages of the corresponding strings. The remaining coordinates of ν are all zero.

Algorithm for (e).

The Levi subgroups are determined by specifying the GL factors. There is at most one other factor G_{0}(n_{0}) of the same type as the group.

For 𝔪̌_{BC}, each pair of parts (k, k) yields a GL(k). If the corresponding string comes from Step 2, then the character on GL(k) is given by | det(∗)|^{average of string}. Otherwise it is the trivial character. The parts of the remaining partition have multiplicity 1 corresponding to a distinguished orbit in 𝔤̌_{0}(n_{0}).

For 𝔪̌_{KL}, apply the same procedure as for 𝔪̌_{BC}, except for pairs coming from Step 1. If originally there was an odd number of parts, then there is no change. If there was an even number, leave behind one pair. The parts in the remaining partition have multiplicity 1 or 2 corresponding to an even orbit in 𝔤̌_{0}(n_{0}).

Algorithms for (f) and (g).

Both 𝔪̌_{e} and 𝔪̌_{o} acquire a GL(k) factor for each pair of parts (k, k) in Step 2, with character given by the average of the corresponding string as before.

For the parts coming from Step 1, write them in decreasing order a_{r} ≥ ⋯ ≥ a_{1} > 0.

For 𝔪̌_{e}, there are additional GL factors

G of type B: (a_{1} + a_{2})/2, (a_{3} + a_{4})/2, …, (a_{r−2} + a_{r−1})/2 if r is odd, (a_{1})/2, (a_{2} + a_{3})/2, …, (a_{r−2} + a_{r−1})/2 when r is even. The characters are given by the averages of the strings (−(a_{r−1}−1)/2, …, (a_{r−2}− 1)/2) and so on, and (−(a_{1} − 1)/2, …, −1/2), … when r is even. Recall that the a_{i} are all even because Ǒ is a nilpotent orbit in type C.

G of type C: (a_{1} + a_{2})/2, (a_{3} + a_{4})/2, …, with characters given by the averages of the strings (−(a_{1} − 1)/2, …, (a_{2} − 1)/2), and so on. In this case Ǒ is type B, so there are an odd number of odd parts.

G of type D: (a_{1} + a_{2})/2, (a_{3} + a_{4})/2, …, with characters obtained by the same procedure as in type C. In this case Ǒ has an even number of odd parts.

For 𝔪̌_{o}, there are additional GL factors

G of type B: (a_{2} +a_{3})/2, (a_{4} +a_{5})/2, …, (a_{r−1} +a_{r})/2 leaving a_{1} out if r is odd, (a_{1} + a_{2})/2, (a_{3} + a_{4})/2, …, (a_{r−1} + a_{r})/2 if r is even. The characters are given by the averages of the strings (−(a_{r} − 1)/2, …, (a_{r−1} − 1)/2) … and so on.

G of type C: (a_{2} + a_{3})/2, (a_{4} + a_{5})/2, …, and characters given by the averages of the strings (−(a_{3} − 1)/2, …, (a_{2} − 1)/2), . . .. In this case Ǒ has an odd number of odd sized parts.

G of type D: (a_{1}+a_{2})/2, …, (a_{r−3} + a_{r−2})/2, ((a_{r−1}−1)/2) with characters obtained by the averages of the strings (−(a_{2} −1)/2, …, (a_{1} − 1)/2), …, ((−a_{r−1} − 1)/2, …, −1). In this case Ǒ has an even number of odd sized parts.

{sec:2.6}

2.9. Let χ = ȟ/2 + ν be associated to the orbit Ǒ. Recall from 2.3

I_{M}(χ) := Ind_{M}^{G}[L_{M}(χ)], {(2.9.1) \atop \{eq:2.6.1\}}

where L_{M}(χ) is the irreducible spherical module of M with parameter χ. Write the nilpotent orbit in (2.3.4) with the (d_{1}, …, d_{l}) as in sections 2.5-2.7 depending on the Lie algebra type. Then 𝔪̌_{BC} = gl(a_{1})×⋯×gl(a_{k})×𝔤̌_{0}(n_{0}) is as in (2.3.5). Thus χ determines a spherical irreducible module

L_{M_{BC}}(χ) = L_{1}(χ_{1}) ⊗ ⋯ ⊗ L_{k}(χ_{k}) ⊗ L_{0}(χ_{0}), {(2.9.2) \atop \{eq:2.6.2\}}

with χ_{i} = (−\frac{a_{i}−1}{2} +ν_{i}, …, \frac{a_{i}−1}{2} +ν_{i}), while χ_{0} = ȟ_{0}/2 for the nilpotent (d_{i}).

Let 𝔪̌_{KL} be the Levi component attached to χ = ȟ/2 + ν in sections 2.5-2.7. As for 𝔪̌_{BC} we have a parameter L_{M_{KL}}(χ). In this case Ǒ = ((a′_{1}, a′_{1}), …, (a′_{r}, a′_{r}); d′_{1}, … d′_{l}) as described in 2.5-2.7. Then (a W -conjugate of) χ can be written as in (2.5.2)-(2.7.2)), and

\begin{matrix}𝔪̌_{KL} = gl(a′_{1}) × ⋯ × gl(a′_{r}) × 𝔤̌_{0}(n′_{0}), \\ L_{M_{KL}}(χ) = L_{1}(χ′_{1}) ⊗ ⋯ ⊗ L_{r}(χ′_{r}) ⊗ L_{0}(χ′_{0}). \end{matrix}  (2.9.3) \{eq:2.6.3\}

                                                                       {t:2.6}

Theorem. In the p-adic case

I_{M_{KL}}(χ) = L(χ).

Proof. This is in [BM1], 𝔪̌_{KL} was defined in such a way that this result

holds. □ {c:2.6}

Corollary. The module I_{M_{BC}}(χ) equals L(χ) in the p−adic case if all the ν_{i} ≠ 0.

3. The Main Result

                                                                       {sec:3}

                                                                       {2.1}

3.1. Recall that Ǧ is the (complex) dual group, and Ǎ ⊂ Ǧ the maximal torus dual to A. Assuming as we may that the parameter is real, a spherical irreducible representation corresponds to an orbit of a hyperbolic element χ ∈ 𝔞̌, the Lie algebra of Ǎ. In section 2 we attached a nilpotent orbit Ǒ in 𝔤̌ with partition (\underbrace{a_{1}, … a_{1}}_{r_{1}}, …, \underbrace{a_{k}, …, a_{k}}_{r_{k}}) to such a parameter. Let \{ě, ȟ, f̌\}

be a Lie triple attached to Ǒ. Let χ := ȟ/2 + ν satisfy (2.3.1)-(2.3.2).

                                                                       {d:2.1}

Definition. A representation L(χ) is said to be in the complementary series for Ǒ, if the parameter χ is attached to Ǒ in the sense of satisfying (2.3.1) and (2.3.2), and is unitary.

We will describe the complementary series explicitly in coordinates.

The centralizer ZǦ(ě, ȟ, f̌) has Lie algebra 𝔷(Ǒ) which is a product of sp(r_{l}, ℂ) or so(r_{l}, ℂ), 1 ≤ l ≤ k, according to the rule

Ǧ of type B, D: sp(r_{l}) for a_{l} even, so(r_{l}) for a_{l} odd,

Ǧ of type C: sp(r_{l}) for a_{l} odd, so(r_{l}) for a_{l} even.

The parameter ν determines a spherical irreducible module LǑ(ν) for the split group whose dual is ZǦ(ě, ȟ, f̌)^{0}. It is attached to the trivial orbit in 𝔷(Ǒ).

                                                                       {thm:3.1}

Theorem. The complementary series attached to Ǒ coincides with the one attached to the trivial orbit in 𝔷(Ǒ). For the trivial orbit (0) in each of the classical cases, the complementary series are

G of type B:

0 ≤ ν_{1} ≤ ⋯ ≤ ν_{k} < 1/2.

G of type C, D:

0 ≤ ν_{1} ≤ ⋯ ≤ ν_{k} ≤ 1/2 < ν_{k+1} < ⋯ < ν_{k+l} < 1

so that ν_{i} + ν_{j} ≤ 1. There are

(1) an even number of ν_{i} such that 1 − ν_{k+1} < ν_{i} ≤ 1/2,

(2) for every 1 ≤ j ≤ l, there is an odd number of ν_{i} such that 1 − ν_{k+j+1} < ν_{i} < 1 − ν_{k+j}.

(3) In type D of odd rank, ν_{1} = 0 or else the parameter is not hermitian.

Remarks.

(1) The complementary series for Ǒ = (0) consists of representations which are both spherical and generic in the sense that they have Whittaker models.

(2) The condition that ν_{i} + ν_{j} ≠ 1 implies that in types C,D there is at most one ν_{k} = 1/2.

(3) In the case of Ǒ ≠ (0), χ = ȟ/2 + ν, and each of the coordinates ν_{i} for the parameter on 𝔷(Ǒ) comes from a string, i.e. each ν_{i} comes from (−\frac{a_{i}−1}{2} + ν_{i}, …, \frac{a_{i}−1}{2} + ν_{i}). The parameter does not satisfy (2.7.11). For (2.7.11) to hold, it suffices to change ν_{k+j} for types C, D to 1 − ν_{k+j}. More precisely, for 1/2 < ν_{k+j} < 1 the connection with the strings in the form (2.7.10) and (2.7.11) is as follows. Write (−\frac{a_{k+j}−1}{2} + ν_{k+j}, …, \frac{a_{k+j}−1}{2} + ν_{k+j}) as (−\frac{a_{k+j}−3}{2} + (ν_{k+j} − 1), …, \frac{a_{k+j}+1}{2} + (ν_{k+j} − 1)) and then conjugate each entry to its negative to form (−\frac{a_{k+j}−3}{2} + ν′_{k+j}, …, \frac{a_{k+j}+1}{2} + ν′_{k+j}), with 0 < ν′_{k+j} = 1 − ν_{k+j} < 1/2.

An Algorithm. For types C and D we give an algorithm, due to S. Sahi, to decide whether a parameter in types C and D is unitary. This algorithm is for the complememntary series for Ǒ = (0). For arbitrary Ǒ it applies to the parameter for 𝔷(Ǒ) obtained as in remark (3) above.

Order the parameter in dominant form,

0 ≤ ν_{1} ≤ ⋯ ≤ ν_{n}, for type C,
0 ≤ |ν_{1}| ≤ ⋯ ≤ ν_{n}, for type D.

(3.1.1) {eq:3.1.1}

The first condition is that ν_{n} < 1, and in addition that if the type is D_{n} with n odd, then ν_{1} = 0. Next replace each coordinate 1/2 < ν_{i} by 1−ν_{i}. Reorder the new coordinates in increasing order as in (3.1.1). Let F (ν) be the set of new positions of the 1 − ν_{i}. If any position is ambiguous, the parameter is not unitary, or is attached to a different nilpotent orbit. This corresponds to either a ν_{i} + ν_{j} = 1, or a 1/2 < ν_{i} = ν_{j}. Finally, L(ν) is unitary if and only if F (ν) consists of odd numbers only.

{sec:3.2}

3.2. We prove the unitarity of the parameters in the theorem for Ǒ = (0) for types B,C, and D. First we record some facts.

Let G := GL(2a) and

χ := (−\frac{a − 1}{2} − ν, …, \frac{a − 1}{2} − ν; −\frac{a − 1}{2} + ν, …, \frac{a − 1}{2} + ν). {(3.2.1) \atop \{eq:3.2.1\}}

Let M := GL(a) × GL(a) ⊂ GL(2a). Then the two strings of χ determine an irreducible spherical (1-dimensional) representation L_{M}(χ) on M. Recall I_{M}(χ) := Ind_{M}^{G}[L_{M}(χ)].

                                                                       {l1:3.2}

Lemma (1). The representation I_{M}(χ) is unitary irreducible for 0 ≤ ν < 1/2. The irreducible spherical module L(χ) is not unitary for ν > \frac{1}{2}, 2ν ∉ ℤ.

Proof. This is well known and goes back to [Stein] (see also [T] and [V1]). □

We also recall the following well known result due to Kostant in the real case, Casselman in the p−adic case.

                                                                       {l2:3.2}

Lemma (2). If none of the ⟨χ, α⟩ for α ∈ Δ(𝔤̌, 𝔞̌) is a nonzero integer, then X(χ) is irreducible. In particular, if χ = 0, then

L(χ) = X(χ) = Ind_{A}^{G}[χ],

and it is unitary.

Let 𝔪̌ ⊂ 𝔤̌ be a Levi component, and ξ_{t} ∈ 𝔷(𝔪̌), where 𝔷(𝔪̌) is the center of 𝔪̌, depending continuously on t ∈ [a, b].

                                                                       {l3:3.2}

Lemma (3). Assume that

I_{M}(χ_{t}) := Ind_{M}^{G}[L_{M}(χ_{0}) ⊗ ξ_{t}]

is irreducible for a ≤ t ≤ b, and L_{M}(χ_{0}) ⊗ ξ_{t} is hermitian. Then I_{M}(χ_{t}) (equal to L(χ_{t})) is unitary if and only if L_{M}(χ_{0}) is unitary.

This is well known, and amounts to the fact that (normalized) induction preserves unitarity. I don’t know the original reference.

When the conditions of lemma 3 are satisfied, we say that I_{M}(χ_{t}) is a continuous deformation of L(χ_{0}).

We now start the proof of the unitarity.

Type B. In this case there are no roots α ∈ Δ(𝔤̌, 𝔞̌) such that ⟨χ, α⟩ is a nonzero integer. Thus

L(χ) = Ind_{A}^{G}[χ]

as well. When deforming χ to 0 continuously, the induced module stays irreducible. Since Ind_{A}^{G}[0] is unitary, so is L(χ).

Type C,D. There is no root such that ⟨χ, α⟩ is a nonzero integer, so L(χ) = Ind_{A}^{G}[χ]. If there are no ν_{k+i} > 1/2 the argument for type B carries over word for word. When there are ν_{k+i} > 1/2 we have to be more careful with the deformation. We will do an induction on the rank. Suppose that ν_{j−1} = ν_{j} for some j. Necessarily, ν_{j} < 1/2. Conjugate χ by the Weyl group so that

χ = (ν_{1}, …, ν_{i}, … ν̂_{j−1}, ν̂_{j}, …,; ν_{j−1}; ν_{j}) := (χ_{0}; ν_{j−1}; ν_{j}).  (3.2.2) {eq:3.2.2}

Let 𝔪̌ := 𝔤̌(n − 2) × gl(2), and denote by M the corresponding Levi component. Then by induction in stages,

L(χ) = Ind_{M}^{G}[L_{M}(χ)], {(3.2.3) \atop \{eq:3.2.3\}}

where L_{M}(χ) = L_{0}(χ_{0}) ⊗ L_{1}(ν_{j−1}, ν_{j}). By lemma (1) of 3.2, L_{1}(ν_{j−1}, ν_{j}) is unitary. Thus L(χ) is unitary if and only if L_{0}(χ_{0}) is unitary. If χ satisfies the assumptions of the theorem, then so does χ_{0}. By the induction hypothesis, L_{0}(χ_{0}) is unitary, and therefore so is L(χ). Thus we may assume that

0 ≤ ν_{1} < ⋯ < ν_{k} ≤ 1/2 < ν_{k+1} < ⋯ < ν_{k+l}. {(3.2.4) \atop \{eq:3.2.4\}}

If ν_{k} < 1 − ν_{k+1}, then the assumptions imply 1 − ν_{k+2} < ν_{k}. Consider the parameter

χ_{t} := (…, ν_{k}, ν_{k+1} − t, …). {(3.2.5) \atop \{eq:3.2.5\}}

Then

L(χ_{t}) = Ind_{A}^{G}[χ_{t}], for 0 ≤ t ≤ ν_{k+1} − ν_{k}, {(3.2.6) \atop \{eq:3.2.6\}}

because no ⟨χ_{t}, α⟩ is a nonzero integer. At t = ν_{k+1} − ν_{k}, the parameter is in the case just considered earlier. By induction we are done.

If on the other hand 1 − ν_{k+1} < ν_{k}, the assumptions on the parameter are such that necessarily 1 − ν_{k+1} < ν_{k−1} < ν_{k}. Then repeat the argument with

χ_{t} := (…, ν_{k−1}, ν_{k} − t, …), 0 ≤ t ≤ ν_{k} − ν_{k−1}. {(3.2.7) \atop \{eq:3.2.7\}}

This completes the proof of the unitarity of the parameters in theorem 3.1 when Ǒ = (0).

{sec:3.3}

3.3. We prove the unitarity of the parameters in theorem 3.1 in the general case when Ǒ ≠ (0).

The proof is essentially the same as for Ǒ = (0), but special care is needed to justify the irreducibility of the modules. Recall the notation of the partition of Ǒ (2.3.3).

The factors of 𝔷(Ǒ) isomorphic to sp(r_{j}), contribute r_{j}/2 factors of the form gl(a_{i}) to 𝔪̌_{KL}. The factors of type so(r_{j}) with r_{j} odd, contribute a d_{i} (notation (2.3.4)) to the expression (2.3.4) of the partition of Ǒ, and \frac{r_{j}−1}{2} gl(a_{i}). The factors so(r_{j}) of type D (r_{j} even) are more complicated. Write the strings coming from this factor as in (2.3.6),

(−\frac{a_{i} − 1}{2} + ν_{i}, …, \frac{a_{i} − 1}{2} + ν_{i})

with the ν_{i} satisfying the assumptions of theorem 3.1. If r_{j} is not divisible by 4, then there must be a ν_{1} = 0, (otherwise the corresponding spherical parameter is not hermitian), and 𝔪̌_{BC} ≠ 𝔪̌_{KL}. Similarly when r_{j} is divisible by 4 and ν_{1} = 0, 𝔪̌_{BC} ≠ 𝔪̌_{KL}. In all situations, we consider

I_{M_{KL}}(χ)  (3.3.1) {eq:3.3.1}

as for Ǒ = (0). We aim to show that this module stays irreducible under the deformations used for Ǒ = (0), separately for the ν_{i} for the same partition size or equivalently simple factor of 𝔷(Ǒ). It is enough to prove that under these deformations the strings stay strongly nested at all the values of the parameters. Then proposition 3.3 applies.

Using the conventions of section 2, the strings are of the form

(−A − 1 + ν_{1}, …, A − 1 + ν_{1})  (3.3.2) {eq:3.3.1a}
(−B + ν_{2}, …, B − 1 + ν_{2})  (3.3.3) {eq:3.3.1b}
(−C + ν_{3}, …, C + ν_{3}).  (3.3.4) {eq:3.3.1c}

The ν_{i} satisfy 0 ≤ ν_{i} ≤ 1/2. When ν_{1} = 1/2, the string is (−A−1/2, …, A− 1/2) so it conforms to (2.5.11), (2.6.11), (2.7.11). Similarly when ν_{2} = 1/2, the string is (−B + 1/2, …, B − 1/2). But when ν_{3} = 1/2, the string is (−C +1/2, …, C +1/2) and it must be replaced by (−C −1/2, …, C −1/2) to conform to (2.5.11), (2.6.11), (2.7.11). The string in (3.3.2) gives a ν_{1,𝔷(Ǒ)} = 1 − ν_{1} (as explained in remark (3) of section 3.1). The other ones ν_{2} and ν_{3} give ν_{j,𝔷(Ǒ)} = ν_{2} or ν_{3} respectively. Suppose first that there is only one size of string. This means that the corresponding entries ν_{j,𝔷(Ǒ)} belong to the same simple factor of 𝔷(Ǒ). Then the strings are either all of the form (3.3.2) and (3.3.4) with A = C or all of the form (3.3.3). Consider the first case. If there is a string (3.3.2) then ν_{1,𝔷(Ǒ)} = 1 − ν_{1} ≥ 1/2, and ν_{1,𝔷(Ǒ)} is deformed downward. By the assumptions ν_{1,𝔷(Ǒ)} does not equal any ν_{j,𝔷(Ǒ)} nor does ν_{1,𝔷(Ǒ)} + ν_{j,𝔷(Ǒ)} = 1 for any j. The module stays irreducible. When ν_{1,𝔷(Ǒ)} crosses 1/2, the string becomes (−A+ν′_{1}, …, A+ν′_{1}), and ν′_{1} is deformed downward from 1/2 to either some ν_{3} or to 0. Again ν′_{1}+ν_{j,𝔷(Ǒ)} ≠ 1 for any j, so no reducibility occurs. The irreduciblity in the case when ν′_{1} reaches 0 is dealt with by section 10.

Remains to check that in these deformations no reducibility occurs because the string interacts with one of a different length, in other words, when a ν_{i,𝔷(Ǒ)} for one size string or equivalently factor of 𝔷(Ǒ) becomes equal to a ν_{j,𝔷(Ǒ)} from a distinct factors of 𝔷(Ǒ). We explain the case when the deformation involves a string of type (3.3.2), the others are similar and easier. Consider the deformation of ν_{1} from 0 to 1/2. If the module is to become reducible ν_{1} must reach a ν_{3} so that there is a string of the form (3.3.4) satisfying

−A − 1 < −C,  A − 1 < C.  (3.3.5) {eq:3.3.1d}

This is the condition that the strings are not nested for some value of the parameter. This implies that A − 1 < C < A + 1 must hold. Thus A = C, and the two strings correspond to the same size partition or simple factor in 𝔷(Ǒ).

In the p-adic case, the irreducibility of (3.3.1) in the case of strongly nested strings follows from the results of Kazhdan-Lusztig. In the case of real groups, the same irreducibility results hold, but are harder to prove. Given χ, consider the root system

Δ̌_{χ} := \{α̌ ∈ Δ̌ : ⟨χ, α̌⟩ ∈ ℤ\}. {(3.3.6) \atop \{eq:3.3.5\}}

Let G_{χ} be the connected split real group whose dual root system is Δ̌_{χ}. Then χ determines an irreducible spherical representation L_{G_{χ}}(χ). The Kazhdan-Lusztig conjectures for nonintegral infinitesimal character provide a way to prove any statement about the character of L(χ) by proving it for L_{G_{χ}}(χ). This is beyond the scope of this paper (or my competence), I refer to [ABV], chapters 16 and 17 for an explanation.

Since G_{χ} is not simple, it is sufficient to prove the needed irreducibility result for each simple factor. This root system is a product of classical systems as follows. For each 0 ≤ τ ≤ 1/2 let A_{τ} be the set of coordinates of χ congruent to τ modulo ℤ. Each A_{τ} contributes to Δ̌_{χ} as follows.

G of type B: Every 0 < τ < 1/2 contributes a type A of size equal to the number of coordinates in A_{τ}. Every τ = 0, 1/2 contributes a type C of rank equal to the number of coordinates in A_{τ} .

G of type C: Every 0 < τ < 1/2 contributes a type A as for type B. Every τ = 0 contributes a type B, while τ = 1/2 contributes type D.

G of type D: Every 0 < τ < 1/2 contributes a type A. Every τ =

0, 1/2 contributes a type D. The irreducibility results for I_{M_{KL}}(χ_{t}) needed to carry out the proof are

contained in the following proposition.

{p:3.3}

Proposition. Let

χ := (…; −\frac{a_{i} − 1}{2} + ν_{i}, …, \frac{a_{i} − 1}{2} + ν_{i}; …)

be given in terms of strings, and let 𝔪̌ = gl(a_{1}) × ⋯ × gl(a_{k}) be the corresponding Levi component. Assume that χ is integral (i.e. ⟨χ, α̌⟩ ∈ ℤ). In addition assume that the coordinates of χ are

• in ℤ, in type B,

• in 1/2 + ℤ for type D. If the strings are strongly nested, then

I_{M}(χ) = Ind_{M}^{G}[L_{M}(χ)].

The proof of the proposition will be given in section 10.

Remark. For the ν_{j} attached to factors of type D in 𝔷(Ǒ), it is important in the argument that we do not deform to (0). The next example illustrates why.

Assume Ǒ = (2, 2, 2, 2) ⊂ sp(4). The parameters of the form ȟ/2 + ν are

(−1/2 + ν_{1}, 1/2 + ν_{1}; −1/2 + ν_{2}, 1/2 + ν_{2}),  (3.3.7) {eq:3.3.2}

and, because parameters are up to W −conjugacy, we may restrict attention to the region 0 ≤ ν_{1} ≤ ν_{2}. In this case 𝔷(Ǒ) = so(4), and the unitarity region is 0 ≤ ±ν_{1} +ν_{2} < 1. Furthermore 𝔪̌_{BC} = gl(2)×gl(2), but 𝔪̌_{KL} = 𝔪̌_{BC} only if 0 < ν_{1}. When ν_{1} = 0, 𝔪̌_{KL} = sp(2)×gl(2), the nilpotent orbit is rewritten (2, 2; (2, 2)), and ȟ_{0}/2 = (1/2, 1/2). For ν_{1} = 0, the induced representations

I_{M_{KL}}(χ_{ν_{2}}) := Ind_{Sp(2)×GL(2)}^{Sp(4)}[L_{0}(1/2, 1/2)⊗L_{1}(−1/2+ν_{2}, 1/2+ν_{2})] (3.3.8) {eq:3.3.3}

are induced irreducible in the range 0 ≤ ν_{2} < 1. For 0 < ν_{1} the representation

Ind_{GL(4)}^{Sp(4)}[L((−1/2+ν_{1} +t, 1/2+ν_{1} +t); (−1/2−ν_{1} −t, 1/2−ν_{1} −t))] (3.3.9) {eq:3.3.4}

is induced irreducible for 0 ≤ t ≤ 1/2 − ν_{1}.

The main point of the example is that Ind_{GL(2)}^{Sp(2)}[L(−1/2 + t, 1/2 + t)] is reducible at t = 0. So we cannot conclude that L(χ) is unitary for a (ν_{1}, ν_{2}) with 0 < ν_{1} from the unitarity of L(χ) for a parameter with ν_{1} = 0. Instead we conclude that the representation is unitary in the region 0 ≤ ±ν_{1} + ν_{2} < 1 because it is a deformation of the irreducible module for ν_{1} = ν_{2} which is unitarily induced irreducible from a Stein complementary series on GL(4). □

4. Relevant K−types

                                                                       {sec:4}

4.1. In the real case we will call a K−type (μ, V) quasi-spherical if it occurs in the spherical principal series. In section 4, we will use the notation M = K ∩ B. By Frobenius reciprocity (μ, V) is quasi-spherical if and only if V^{K∩B} ≠ 0. Because the Weyl group W (G, A) may be realized as N_{K}(A)/Z_{K}(A), this Weyl group acts naturally on this space.

The representations of W (A_{n−1}) = S_{n} are parametrized by partitions

(a) := (a_{1}, …, a_{k}), a_{i} ≤ a_{i+1}, of n, and we write σ((a)) for the corresponding representation. The representations of W (B_{n}) ≅ W (C_{n}) are parametrized as in [L1] by pairs of partitions, and we write as

σ((a_{1}, …, a_{r}), (b_{1}, …, b_{s})),
a_{i} ≤ a_{i+1}, b_{j} ≤ b_{j+1}, ∑ a_{i} + ∑ b_{j} = n.  (4.1.1) {eq:4.0.4}

Precisely the representation parametrized by (4.1.1) is as follows. Let k = ∑ a_{i}, l = ∑ b_{j}. Recall that W ≅ S_{n}⋉ℤ_{2}^{n}. Let χ be the character of ℤ_{2}^{n} which is trivial on the first k ℤ_{2}’s, sign on the last l. Its centralizer in S_{n} is S_{k} ×S_{l}. Let σ((a)) and σ((b)) be the representations of S_{k}, S_{l} corresponding to the partitions (a) and (b). Then let σ((a), (b), χ) be the unique representation of (S_{k} ×S_{l})⋉Z_{2}^{n} which is a multiple of χ when restricted to Z_{2}^{n}, and σ(a)⊗σ(b) when restricted to S_{k} × S_{l}. The representation in (4.1.1), is

σ((a), (b)) = Ind_{(S_{k}×S_{l})⋉ℤ_{2}^{n}}^{W} [σ(a, b, χ)]. {(4.1.2) \atop \{eq:4.0.5\}}

If (a) ≠ (b), the representations σ((a), b) and σ((b), (a)) restrict to the same irreducible representation of W (D_{n}), which we denote again by the same symbol. When a = b, the restriction is a sum of two inequivalent representations which we denote σ((a), (a))_{I, II}. Let W_{(a),I} := S_{a_{1}} × ⋯ × S_{a_{r}} and W_{(a),II} := S_{a_{1}} × ⋯ × S′_{a_{r}}, be the Weyl groups corresponding to the Levi components considered in Remark (2) in section 2.7. Then σ((a), (a))_{I} is characterized by the fact that its restriction to W_{(a),I} contains the trivial representation. Similarly σ((a), (a))_{II} is the one that contains the trivial representation of W_{(a),II}.

{sec:4.2}

4.2. Symplectic Groups. The group is Sp(n) and the maximal compact subgroup is U(n). The highest weight of a K-type will be written as μ(a_{1}, …, a_{n}) with a_{i} ≥ a_{i+1} and a_{i} ∈ ℤ, or

μ(a_{1}^{r_{1}}, …, a_{k}^{r_{k}}) := (\underbrace{a_{1}, …, a_{1}}_{r_{1}}, …, \underbrace{a_{k}, …, a_{k}}_{r_{k}}). {(4.2.1) \atop \{eq:4.2.1\}}

when we want to emphasize the repetitions. We will repeatedly use the following restriction formula

{l:4.2}

Lemma. The restriction of μ(a_{1}, …, a_{n}) to U(n − 1) × U(1) is

∑ μ(b_{1}, …, b_{n−1}) ⊗ μ(b_{n}),

where the sum ranges over all possible a_{1} ≥ b_{1} ≥ a_{2} ≥ ⋯ ≥ b_{n−1} ≥ a_{n}, and b_{n} = ∑_{1≤i≤n} a_{i} − ∑_{1≤j≤n−1} b_{j}.

{def:4.2}

Definition. The representations μ_{e}(n − r, r) := μ(2^{r}, 0^{n−r}) and μ_{o}(k, n − k) := μ(1^{k}, 0^{n−2k}, −1^{k}) are called relevant.

{p:4.2}

Proposition. The relevant K−types are quasispherical. The representation of W (C_{n}) on V^{M} is

\begin{matrix}μ_{e}(n − r, r) ⟷ σ[(n − r), (r)], \\ μ_{o}(k, n − k) ⟷ σ[(k, n − k), (0)], \end{matrix}

The K-types μ(0^{n−r}, (−2)^{r}), dual to μ_{e}(n − r, r) are also quasispherical, and could be used in the same way as μ_{e}(n − r, r).

Proof. We do an induction on n. When n = 1, the only relevant representations of U(1) are μ_{e}(1, 0) = μ(0) and μ_{e}(0, 1) = (2), which correspond to the trivial and the sign representations of W (C_{1}) = ℤ/2ℤ, respectively. Consider the case n = 2. There are four relevant representations of U(2) with highest weights (2, 0), (1, −1), (2, 2) and (0, 0). The first representation is the symmetric square of the standard representation, the second one is the adjoint representation and the fourth one is the trivial representation. The normalizer of A in K can be identified with the diagonal subgroup (±1, ±1) inside U(1) × U(1) ⊂ U(2). The Weyl group is generated by the elements

[\begin{matrix}i & 0 \\ 0 & 1 \end{matrix}] , [\begin{matrix}1 & 0 \\ 0 & i \end{matrix}] , [\begin{matrix}0 & 1 \\ −1 & 0 \end{matrix}] .  (4.2.2) \{eq:4.2.3\}

The restriction to U(1) × U(1) of the four representations of U(2) is

(2, 0) ⟶ (2) ⊗ (0) + (1) ⊗ (1) + (0) ⊗ (2),
(1, −1) ⟶ (1) ⊗ (−1) + (0) ⊗ (0) + (−1) ⊗ (1),
(2, 2) ⟶ (2) ⊗ (2),  (4.2.3) {eq:4.2.4}
(0, 0) ⟶ (0) ⊗ (0).

The space V^{M} is the sum of all the weight spaces (p) ⊗ (q) with both p and q even. For the last one, the representation of W on V^{M} is σ[(2), (0)]. The third one is 1-dimensional so V^{M} is 1-dimensional; the Weyl group representation is σ((0), (2)). The second one has V^{M} 1-dimensional and the Weyl group representation is σ((11), (0)). For the first one, V^{M} is 2dimensional and the Weyl group representation is σ((1), (1)). These facts can be read off from explicit realizations of the representations.

Assume that the claim is proved for n − 1. Choose a parabolic subgroup so that its Levi component is M′ = Sp(n − 1) × GL(1) and M is contained in it. Let H = U(n − 1) × U(1) be such that M ⊂ M′ ∩ K ⊂ H.

Suppose that μ is relevant. The cases when k = 0 or r = 0 are 1dimensional and are straightforward. So we only consider k, r > 0. The K-type μ(2^{r}, 0^{n−r}) restricts to the sum of

μ(2^{r}, 0^{n−r−1}) ⊗ μ(0)  (4.2.4) {eq:4.2.5}
μ(2^{r−1}, 1, 0^{n−r−1}) ⊗ μ(1)  (4.2.5) {eq:4.2.6}
μ(2^{r−1}, 0^{n−r}) ⊗ μ(2).  (4.2.6) {eq:4.2.7}

Of the representations appearing, only μ(2^{r}, 0^{n−r−1}) ⊗ μ(0) and μ(2^{r−1}, 1, 0^{n−r−1}) ⊗ μ(2) are quasispherical. So the restriction of V^{M} to W (C_{n−1}) × W (C_{1}) is the sum of

σ[(n − r − 1), (r)] ⊗ σ[(1), (0)]  (4.2.7) {eq:4.2.8}
σ[(n − r), (r − 1)] ⊗ σ[(0), (1)]  (4.2.8) {eq:4.2.9}

The only irreducible representations of W (C_{n}) containing (4.2.7) in their restrictions to W (C_{n−1}) are

σ[(1, n − r − 1), (r)]  (4.2.9) {eq:4.2.10}
σ[(n − r), (r)].  (4.2.10) {eq:4.2.11}

But the restriction of σ[(1, n − r − 1), (r)] to W (C_{n−1}) × W (C_{1}) contains σ[(1, n−r−1), (r−1)]⊗σ[(0), (1)], and this does not appear in (4.2.7)-(4.2.8). Thus the representation of W (C_{n}) on V^{M} for (4.2.9) must be (4.2.5), and the claim is proved in this case.

Consider the case μ(1^{k}, 0^{l}, −1^{k}) for k > 0, 2k + l = n. The restriction of this K-type to U(n − 1) × U(1) is the sum of

μ(1^{k}, 0^{l}, −1^{k−1}) ⊗ μ(−1)  (4.2.11) {eq:4.2.12}
μ(1^{k−1}, 0^{l}, −1^{k}) ⊗ μ(1)  (4.2.12) {eq:4.2.13}
μ(1^{k−1}, 0^{l+1}, −1^{k−1}) ⊗ μ(0)  (4.2.13) {eq:4.2.14}
μ(1^{k}, 0^{l−1}, −1^{k}) ⊗ μ(0)  (4.2.14) {eq:4.2.15}

Of the representations appearing, only (4.2.13) and (4.2.14) are quasispherical. So the restriction of V^{M} to W (C_{n−1}) × W (C_{1}) is the sum of

σ[(k − 1, k + l), (0)] ⊗ σ[(1), (0)], {(4.2.15) \atop \{eq:4.2.16\}}

σ[(k, k + l − 1), (0)] ⊗ σ[(1), (0)]. {(4.2.16) \atop \{eq:4.2.17\}}

The representation (4.2.16) can only occur in the restriction to W (C_{n−1}) × W (C_{1}) of σ[(1, k, k + l − 1), (0)] or σ[(k, k + l), (0)]. If k > 1, the first one contains σ[(1, k − 1, k + l − 1), (0)] in its restriction, which is not in the sum of (4.2.15) and (4.2.16). If k = 1 then (4.2.15) can only occur in the restriction of σ[(0, l + 2), (0)], or σ[(1, l + 1), (0)]. But V^{M} cannot consist of σ[(0, l + 2), (0)] alone, because (4.2.15) does not occur in its restriction. If it consists of both σ[(0, l + 2), (0)] and σ[(1, l), (0)], then the restriction is too large. The claim is proved in this case. □

{sec:4.3}

4.3. Orthogonal groups. Because we are dealing with the spherical case, we can use the groups O(a, b), SO(a, b), or the connected component of the identity, SO_{e}(a, b). The corresponding K’s are O(a)×O(b), S(O(a)×O(b)), and SO(a) × SO(b), respectively. We will use O(a, b) for the calculation of relevant K-types. For SO(a), an irreducible representation will be identified by its highest weight in coordinates, μ(x_{1}, …, x_{[a/2]}), or if there are repetitions, μ(x_{1}^{n_{1}}, …, x_{k}^{n_{k}}). For O(a) we use the parametrization of Weyl, [Weyl]. Embed O(a) ⊂ U(a) in the standard way. Then we denote by μ(x_{1}, …, x_{k}, 0^{[a/2]−k}; ϵ) the irreducible O(a)−component generated by the highest weight of the representation

μ(x_{1}, …, x_{k}, 1^{(1−ϵ)(a/2−k)}, 0^{a−k−(1−ϵ)(a/2−k)})

of U(a). In these formulas, ϵ = ±, is often written as + for 1, and − for −1.

{sec:4.4}

4.4. We describe the relevant K-types for the orthogonal groups O(a, a).

{def:4.4}

Definition (even orthogonal groups). The relevant K−types for O(a, a)

μ_{e}([a/2] − r, r) := μ(0^{[a/2]}; +) ⊗ μ(2^{r}, 0^{l}; +) {(4.4.1) \atop \{eq:4.4.1\}}

μ_{o}(r, [a/2] − r) := μ(1^{r}, 0^{l}; +) ⊗ μ(1^{r}, 0^{l}; +). {(4.4.2) \atop \{eq:4.4.2\}}

where r + l = [a/2].

{p:4.4}

Proposition. The relevant K−types are quasispherical. The representation of W (D_{a}) of O(a, a) on V^{M} is

σ[(r, a − r), (0)]  ⟷  μ(0^{[a/2]}; +) ⊗ μ(2^{r}, 0^{l}; +), {(4.4.3) \atop \{eq:4.4.3\}}
σ[(a − k), (k)],  ⟷  μ(1^{k}, 0^{l}; +) ⊗ μ(1^{k}, 0^{l}; +), {(4.4.4) \atop \{eq:4.4.4\}}

When l = 0, and a is even,

σ[(a/2, a/2), (0)]  ⟷ μ(0^{a/2}) ⊗ μ(2^{a/2−1}, ±2),  (4.4.5) {eq:4.4.5}
σ[(a/2), (a/2)]_{I,II}  ⟷ μ(1^{a/2−1}, ±1) ⊗ μ(1^{a/2−1}, ±1). (4.4.6)

We will prove this together with the corresponding proposition for O(a+1, a) in section 4.6.

                                                                       {sec:4.5}

4.5. We describe the relevant K-types for O(a + 1, a)

                                                                       {def:4.5}

Definition (odd orthogonal groups). The relevant K-types for O(a + 1, a) are

μ_{e}(a − r, r) := μ(0^{[(a+1)/2]}; +) ⊗ μ(2^{r}, 0^{l}; +)  (4.5.1) {eq:4.5.1}
μ_{o}(a − k, k) := μ(1^{k}, 0^{l}; +) ⊗ μ(1^{k}, 0^{s}; +)  (4.5.2) {eq:4.5.2}
μ_{o}(k, a − k) := μ(1^{k+1}, 0^{l}; +) ⊗ μ(1^{k}, 0^{s}; +)  (4.5.3) {eq:4.5.3}

where r + l = [a/2] in (4.5.1), k + l = [(a + 1)/2], k + s = [a/2] in (4.5.2), and k + 1 + l = [(a + 1)/2], k + s = [a/2] in (4.5.3).

                                                                       {p:4.5}

Proposition. The representations of W (B_{a}) on V^{M} for the relevant Ktypes are

σ[(r, a − r), (0)] ⟷ μ(0^{[(a+1)/2]}; +) ⊗ μ(2^{r}, 0^{l}; +)  (4.5.4) {eq:4.5.4}
σ[(a − k), (k)] ⟷ μ(1^{k}, 0^{[(a+1)/2]−k}; +) ⊗ μ(1^{k}, 0^{[a/2]−l}; +), (4.5.5) {eq:4.5.5}
σ[(k), (a − k)] ⟷ μ(1^{k+1}, 0^{[(a+1)/2]−k−1}; +) ⊗ μ(1^{k}, 0^{[a/2]−k}; +)._{(4.5.6) {eq:4.5.6}}

When a is even,

σ[(a/2), (a/2)]  ⟷  μ(1^{a/2}) ⊗ μ(1^{a/2−1}, ±1).  (4.5.7) {eq:4.5.7}

When a is odd,

σ[(\frac{a − 1}{2}), (\frac{a − 1}{2})]  ⟷ μ(1^{(a−1)/2}, ±1) ⊗ μ(1^{(a−1)/2}). (4.5.8) \{eq:4.5.8\}

The proof will be in section 4.6.

                                                                       {sec:4.6}

4.6. Proof of propositions 4.4 and 4.5. We use the standard realization of the orthogonal groups O(a + 1, a) and O(a, a). Let

M̃:= {(η_{0}, η_{1}, …, η_{a}, ϵ_{1}, …, ϵ_{a}) : η_{i}, ϵ_{j} = ±1, ∏ η_{i} = ∏ ϵ_{j} = 1}, (4.6.1) {eq:4.6.1}

viewed as the subgroup of O(a + 1) × O(a) with the η_{i}, ϵ_{j} on the diagonal. With the appropriate choice of 𝔞 ≅ ℝ^{a}, M̃ ⊂ N_{K}(𝔞), and the action is

(η_{i}, ϵ_{j}) · (…, x_{k}, …) = (…, η_{k}ϵ_{k}x_{k}, …).  (4.6.2) {eq:4.6.2}

Then M := K ∩ B is the subgroup of M̃ determined by the relations η_{j} = ϵ_{j}, j = 1, …, a. Similarly for O(a) × O(a) but there is no η_{0}.

We do the case O(a + 1, a), O(a, a) is similar. The representations μ_{o}(a − k, k) and μ_{o}(k, a − k) can be realized as ⋀^{k} ℂ^{a+1} ⊗ ⋀^{k} ℂ^{a}, respectively ⋀^{k+1} ℂ^{a+1} ⊗ ⋀^{k}(ℂ^{a}). Let e_{i} be a basis of ℂ^{a+1} and f_{j} a basis of ℂ^{a}. The space V^{M} is the span of the vectors e_{i_{1}} ∧ ⋯ ∧ e_{i_{k}} ⊗ f_{i_{1}} ∧ ⋯ ∧ f_{i_{k}}, and e_{0} ∧ e_{i_{1}} ∧ ⋯ ∧ e_{i_{k}} ⊗ f_{i_{1}} ∧ ⋯ ∧ f_{i_{k}}. The elements of W corresponding to short root reflections all have representatives of the form η_{0} = −1, η_{j} = −1, the rest zero. The action of S_{a} ⊂ W on the space V^{M} is by permuting the e_{i}, f_{j} diagonally. Claims (4.4.4-4.4.5) and (4.5.5-4.5.6) follow from these considerations, we omit further details.

For cases (4.4.3) and (4.5.4) we do an induction on r. We do the case O(a, a) only. The claim is clear for r = 0. Since the first factor of μ_{e}([a/2] − r, r) is the trivial representation, we only concern ourselves with the second factor. Consider ⋀^{r} ℂ^{a}⊗⋀^{r} ℂ^{a}. The space of M−fixed vectors has dimension \binom{a}{r}, and a basis is

e_{i_{1}} ∧ ⋯ ∧ e_{i_{r}} ⊗ e_{i_{1}} ∧ ⋯ ∧ e_{i_{r}} {(4.6.3) \atop \{eq:4.6.3\}}

As a module of S_{a}, this is

Ind_{S_{r}×S_{a−r}}^{S_{a}}[triv ⊗ triv] = ∑_{1≤j≤r}(j, a − j) {(4.6.4) \atop \{eq:4.6.4\}}

On the other hand, the tensor product ⋀^{r} ℂ^{r} ⊗ ⋀^{r} ℂ^{a} consists of representations with highest weight μ(2^{α}, 1^{β}, 0^{γ}). From the explicit description of ⋀^{k} ℂ^{a}, and the action of M, we can infer that V^{M} for β ≠ 0 is (0). This is because the representation occurs in ⋀^{α+β} ℂ^{a} ⊗⋀^{α} ℂ^{a}, which has no M−fixed vectors. But μ(2^{j}, 0^{l}) for j ≤ r occurs (for example by the P-R-V conjecture). By the induction hypothesis, (j, a − j) occurs in μ(2^{j}, 0^{l}), for j < r, and so only (r, a − r) is unaccounted for. Thus V^{M} for μ_{e}([a/2] − r, r) be (r, a−r). The claim now follows from the fact that the action of the short root reflections is trivial, and the description of the irreducible representations of W (B_{a}).

{sec:4.7}

4.7. General linear groups. The maximal compact subgroup of GL(a, ℝ)

is O(a), the Weyl group is W (A_{a−1}) = S_{a} and M ≅ \underbrace{O(1) × ⋯ × O(1)}_{a}. We

list the case of the connected component GL(a, ℝ)^{+} (matrices with positive determinant) instead, because its maximal compact group is K = SO(a) which is connected, and irreducible representations are parametrized by their highest weights.

                                                                       {def:4.7}

Definition. The relevant K−types are the ones with highest weights

μ(2^{k}, 0^{l}).

The corresponding Weyl group representations on V^{M} are σ[(k, a − k)].

We omit the details, the proof is essentially the discussion about the representation of S_{a} on ⋀ ℂ^{a} ⊗ ⋀ ℂ^{a} for the orthogonal groups.

                                                                       {sec:5.8}

4.8. Relevant W −types.

Definition. Let W be the Weyl group of type B,C,D. The following W −types

                                                                       {d:5.8}

will be called relevant.

σ_{e}(n − r, r) := σ[(n − r), (r)],  σ_{o}(k, n − k) := σ[(k, n − k), (0)] (4.8.1) {eq:4.8.1}

In type D for n even, and r = n/2 there are two W − types, σ_{e}[(n/2), (n/2)]_{I,II} := σ[(n/2), (n/2)]_{I,II}. If the root system is not simple, the relevant W −types are tensor products of relevant W −types on each factor.

5. Intertwining Operators

                                                                       {sec:5}
                                                                       {sec:5.1}

5.1. Recall that X(ν) denotes the spherical principal series. Let w ∈ W. Then there is an intertwining operator

I(w, ν) : X(ν) ⟶ X(wν).  (5.1.1) {eq:5.1.1}

If (μ, V) is a K−type, then I(w, ν) induces a map

I_{V} (w, ν) : Hom_{K}[V, X(ν)] ⟶ Hom_{K}[V, X(wν)].  (5.1.2) {eq:5.1.2}

By Frobenius reciprocity, we get a map

R_{V} (w, ν) : (V^{∗})^{K∩B} ⟶ (V^{∗})^{K∩B}.  (5.1.3) {eq:5.1.3}

In case (μ, V) is trivial the spaces are 1-dimensional and R_{V} (w, ν) is a scalar. We normalize I(w, ν) so that this scalar is 1. The R_{V} (w, ν) are meromorphic functions in ν, and the I(w, ν) have the following additional properties.

(1) If w = w_{1} ·w_{2} with ℓ(w) = ℓ(w_{1})+ℓ(w_{2}), then I(w, ν) = I(w_{1}, w_{2}ν)∘ I(w_{2}, ν). In particular if w = s_{α_{1}} ⋯ s_{α_{k}} is a reduced decomposition, then I(w, ν) factors into a product of intertwining operators I_{j}, one for each s_{α_{j}}. These operators are

I_{j} : X(s_{α_{j+1}} … s_{α_{k}} · ν) ⟶ X(s_{α_{j}} … s_{α_{k}} · ν) {(5.1.4) \atop \{eq:5.1.4\}}

(2) Let P = MN be a standard parabolic subgroup (so A ⊂ M) and w ∈ W (M, A). The intertwining operator

I(w, ν) : X(ν) = Ind_{P}^{G}[X_{M}(ν)] ⟶ X(wν) = Ind_{P}^{G}[X_{M}(wν)]

is of the form I(w, ν) = Ind_{M}^{G}[I_{M}(w, ν)].

(3) If Re⟨ν, α⟩ ≥ 0 for all positive roots α, then R_{V} (w_{0}, ν) has no poles, and the image of I(w_{0}, ν) (w_{0} ∈ W is the long element) is L(ν).

(4) If −\overline{ν} is in the same Weyl group orbit as ν, let w be the shortest element so that wν = −\overline{ν}. Then L(ν) is hermitian with inner product

⟨v_{1}, v_{2}⟩ := ⟨v_{1}, I(w, ν)v_{2}⟩.

Let α be a simple root and P_{α} = M_{α}N be the standard parabolic subgroup so that the Lie algebra of M_{α} is isomorphic to the sl(2, ℝ) generated by the root vectors E_{±α}. We assume that θE_{α} = −E_{−α}, where θ is the Cartan involution corresponding to K. Let D_{α} = \sqrt{−1}(E_{α} − E_{−α}) and s_{α} = e^{\sqrt{−1}πD_{α}/2}. Here by s_{α}, we actually mean the representative in N_{K}(A) of the Weyl group reflection. Then s_{α}^{2} = m_{α} is in K ∩ B ∩ M_{α}. Since the square of any element in K ∩ B is in the center, and K ∩ B normalizes the the root vectors, Ad m(D_{α}) = ±D_{α}. Grade V^{∗} = ⊕V_{i}^{∗} according to the absolute values of the eigenvalues of D_{α} (which are integers). Then K ∩ B preserves this grading and

(V^{∗})^{K∩B} = ⨁_{i even}(V_{i}^{∗})^{K∩B}.

The map ψ_{α} : sl(2, ℝ) ⟶ 𝔤 determined by

ψ_{α} [\begin{matrix}0 & 1 \\ 0 & 0 \end{matrix}] = E_{α}, ψ_{α} [\begin{matrix}0 & 1 \\ 0 & 0 \end{matrix}] = E_{−α}

determines a map

Ψ_{α} : SL(2, ℝ) ⟶ G {(5.1.5) \atop \{eq:5.1.5\}}

with image G_{α}, a connected group with Lie algebra isomorphic to sl(2, ℝ).

{p:5.1}

Proposition. On (V_{2m}^{∗})^{K∩B},

R_{V} (s_{α}, ν) = \begin{cases}Id & if m = 0, \\ ∏_{0≤j<m} \frac{2j+1−<ν,α̌>}{2j+1+<ν,α̌>} Id & if m ≠ 0. \end{cases}

In particular, I(w, ν) is an isomorphism unless ⟨ν, α̌⟩ ∈ 2ℤ + 1.

Proof. The formula is well known for SL(2, ℝ). The second assertion follows from this and the listed properties of intertwining operators. □

{c:5.1}

Corollary. For relevant K−types the formula is

R_{V} (s_{α}, ν) = \begin{cases}Id & on the +1 eigenspace of s_{α}, \\ \frac{1−<ν,α̌>}{1+<ν,α̌>} Id & on the -1 eigenspace of s_{α}. \end{cases}

When restricted to (V^{∗})^{K∩B}, the long intertwining operator is the product of the R_{V} (s_{α}, ∗) corresponding to the reduced decomposition of w_{0} and depends only on the Weyl group structure of (V^{∗})^{K∩B}.

Proof. Relevant K−types are distinguished by the property that the even eigenvalues of D_{α} are 0, ±2 only. The element s_{α} acts by 1 on the zero eigenspace of D_{α} and by −1 on the ±2 eigenspace. The claim follows from this. □

                                                                       {sec:5.2}

5.2. We now show that the formulas in the previous section coincide with corresponding ones in the p−adic case. In the split p-adic case, spherical representations are a subset of representations with I-fixed vectors, where I is an Iwahori subgroup. As explained in [B], the category of representations with I fixed vectors is equivalent to the category of finite dimensional representations of the Iwahori-Hecke algebra H := H(I\G/I). The equivalence is

V ⟶ V^{I}.  (5.2.1) {eq:5.2.1}

The papers [BM1] and [BM2] show that the problem of the determination of the unitary dual of representations with I fixed vectors, is equivalent to the problem of the determination of the unitary irreducible representations of H with real infinitesimal character. In fact it is the affine graded Hecke algebras we will need to consider, and they are as follows.

Let 𝔸 := S(𝔞), and define the affine graded Hecke algebra to be ℍ := ℂ[W] ⊗ 𝔸 as a vector space, and usual algebra structure for ℂ[W] and 𝔸.

The generators of ℂ[W] are denoted by t_{α} corresponding to the simple reflections s_{α}, while the generators of 𝔸 are ω ∈ 𝔞. Impose the additional relation

ωt_{α} = s_{α}(ω)t_{α}+ < ω, α >,  ω ∈ 𝔞,  (5.2.2) {eq:5.2.2}

where t_{α} is the element in ℂ[W] corresponding to the simple root α. If X(χ) is the standard (principal series) module determined by χ, then

X(χ)^{I} = ℍ ⊗_{𝔸} ℂ_{χ}.  (5.2.3) {eq:5.2.3}

The intertwining operator I(w, χ) is a product of operators I_{α_{i}} according to a reduced decomposition of w = s_{α_{1}} · · · · · s_{α_{k}}. If α is a simple root,

r_{α} := (t_{α}α − 1)\frac{1}{α − 1},  I_{α} : x ⊗ 11_{χ} ↦ xr_{α} ⊗ 11_{s_{α}χ}.  (5.2.4) \{eq:5.2.4\}

The I(w, ν) have the same properties as in the real case. The r_{α} are multiplied on the right, so we can replace α with −⟨ν, α⟩ in the formulas. Furthermore,

ℂ[W] = ∑_{σ∈Ŵ} V_{σ} ⊗ V_{σ}^{∗}.

Since r_{α} acts as multiplication on the right, it gives rise to an operator

r_{σ}(s_{α}, ν) : V_{σ}^{∗} ⟶ V_{σ}^{∗}.

{t:5.2}

Theorem. The R_{V} (s_{α}, ν) for the real case on relevant K−types coincide with the r_{σ}(s_{α}, ν) on the V_{σ}^{∗} ≅ (V^{∗})^{K∩B}

Proof. These operators act the same way:

r_{σ}(s_{α}, ν) = \begin{cases}Id & on the + 1 eigenspace of s_{α}, \\ \frac{1−⟨ν,α⟩}{1+⟨ν,α⟩}Id & on the − 1 eigenspace of s_{α}. \end{cases}

(5.2.5) {eq:5.2.5}

The assertion is now clear from corollary (5.1) and formula (5.2.2). We emphasize that the Hecke algebra for a p-adic group G is defined using the dual root system of the complex group Ǧ so that there is no discrepancy between α and α̌ in the formulas. □

{sec:5.3}

5.3. The main point of section 5.2 is that for the real case, and a relevant K-type (V, μ), the intertwining operator calculations coincide with the intertwining operator calculations for the affine graded Hecke algebra on the space V^{K∩B}. Thus we will deal with the Hecke algebra calculations exclusively, but the conclusions hold for both the real and p-adic case. Recall from section 2.3 that to each χ we have associated a nilpotent orbit Ǒ, and Levi components 𝔪̌_{BC} and 𝔪̌_{KL}. These are special instances of the following situation. Assume that Ǒ is written as in (2.3.4) (i.e. ((a_{1}, a_{1}), …, (a_{k}, a_{k}); (d_{i})) with

𝔤̌ of type B: (d_{i}) all odd; they are relabelled (2x_{0} + 1, …, 2x_{2m} + 1),

𝔤̌ of type C: (d_{i}) all even; they are relabelled (2x_{0}, …, 2x_{2m}),

𝔤̌ of type D: (d_{i}) all odd; they are relabelled (2x_{0}+1, …, 2x_{2m−1}+1).

Similar to (2.3.5), let

𝔪̌ := gl(a_{1}) × ⋯ × gl(a_{k}) × 𝔤̌(n_{0}), n_{0} = n − ∑ a_{i}. {(5.3.1) \atop \{eq:5.3.1\}}

We consider parameters of the form χ = ȟ/2 + ν.

Write χ_{0} for the parameter ȟ/2, and χ_{i} := (−\frac{a_{i}−1}{2} +ν_{i}, …, \frac{a_{i}−1}{2} +ν_{i}). We focus on χ_{0} as a parameter on 𝔤̌(n_{0}). We attach two Levi components

\begin{matrix}𝔤̌_{e} : \\ B & gl(x_{2m−1} + x_{2m−2} + 1) × ⋯ × gl(x_{1} + x_{0} + 1) × 𝔤̌(x_{2m}) \\ C & gl(x_{2m−1} + x_{2m−2}) × ⋯ × gl(x_{1} + x_{0}) × 𝔤̌(x_{2m}) \\ D & gl(x_{2m−1} + x_{2m−2} + 1) × ⋯ × gl(x_{1} + x_{0} + 1) \\ 𝔤̌_{o} : \\ B & gl(x_{2m} + x_{2m−1} + 1) × ⋯ × gl(x_{2} + x_{1} + 1) × 𝔤̌(x_{0}) \\ C & gl(x_{2m} + x_{2m−1}) × ⋯ × gl(x_{2} + x_{1}) × 𝔤̌(x_{0}) \\ D & gl(x_{2m−3} + x_{2m−4} + 1) × ⋯ × gl(x_{2m−2}) × 𝔤̌(x_{2m−1} + 1). \end{matrix} {(5.3.2) \atop \{eq:5.3.2\}}

There are 1-dimensional representations L(χ_{e}) and L(χ_{o}) such that the spherical irreducible representation L(χ_{0}) = \overline{X}(χ_{0}) with infinitesimal character χ_{0} is the spherical irreducible subquotient of X_{e} := Ind_{P_{e}}^{G}(L(χ_{e})) and X_{o} := Ind_{P_{o}}^{G}(L(χ_{o})) respectively

The parameters χ_{e} and χ_{o} are written in terms of strings as follows: X_{e}:

B : … (−x_{2i−1}, …, x_{2i−2}) … (−x_{2m}, …, −1)

C : … (−x_{2i−1} + 1/2, …, x_{2i−2} − 1/2) … (−x_{2m} + 1/2, …, −1/2) D : … (−x_{2i−1}, …, x_{2i−2}) …

(5.3.3) {eq:5.3.3}

X_{o}:

\begin{matrix}B : … (−x_{2i}, …, x_{2i−1}) … (−x_{0}, …, −1) \\ C : … (−x_{2i} + 1/2, …, x_{2i−1} − 1/2) … (−x_{0} + 1/2, …, −1/2) \\ D : … (−x_{2i}, …, x_{2i−1}) … (−x_{2m−2}, …, −1)(−x_{2m−1} + 1, …, 0) \end{matrix} (5.3.4)

                                                                       {th:5.3}

Theorem. For the Hecke algebra, p-adic groups,

\begin{matrix}[σ[(n − r), (r)] : X_{e}] = [σ[(n − r), (r)] : L(χ_{0})], \\ [σ[(k, n − k), (0)] : X_{o}] = [σ[(k, n − k), (0)] : L(χ_{0})] \end{matrix}

hold.

The proof is in section 6.8.

For a general parameter χ = χ_{0} + ν, the strings defined in section 2 and the above construction define parabolic subgroups with Levi components gl(a_{1}) × ⋯ × gl(a_{k}) × 𝔤̌_{e} and gl(a_{1}) × ⋯ × gl(a_{r}) × 𝔤̌_{o}, and corresponding L_{e}(χ) and L_{o}(χ). We denote these induced modules by X_{e} and X_{o} as well.

Corollary. The relations

\begin{matrix}[σ[(n − r), (r)] : X_{e}] = [σ[(n − r), (r)] : L(χ)], \\ [σ[(k, n − k), (0)] : X_{o}] = [σ[(k, n − k), (0)] : L(χ)] \end{matrix}

hold in general. For real groups, in the notation of sections 4.2-4.4,

\begin{matrix}[μ_{e}(r, n − r) : X_{e}] = [μ_{e}(r, n − r) : L(χ)], \\ [μ_{o}(k, n − k) : X_{o}] = [μ_{o}(k, n − k) : L(χ)]. \end{matrix}

Proof. The results in section 5.2 show that the intertwining operators on σ_{e}(k, n−k) for the p−adic group equal the intertwining operators for μ_{e}(k, n− k) for the real group, and similarly for σ_{o} and μ_{o}. Thus the multiplicities of the σ_{e}/σ_{o} in L(χ) for the p−adic case equal the multiplicities of the corresponding μ_{e}/μ_{o} in L(χ) in the real case. We do the p−adic case first. Recall theorem 2.9 which states that I_{M_{KL}}(χ) = L(χ). The Levi subgroup M_{KL} is a product of GL factors, which we will denote M_{A}, with a factor G(n_{0}). So for W −type multiplicities we can replace I_{M_{KL}}(χ) by Ind_{M_{A}×G(n_{0})}[⊗triv ⊗ L(χ_{0})]. We explain the case of σ_{e} = σ_{e}(k, n − k), that of σ_{o} being identical. By Frobenius reciprocity,

Hom_{W} [σ_{e} : L(χ)] = Hom_{W} [σ_{e} : I_{M_{KL}}(χ)] {(5.3.5) \atop \{eq:5.3.4\}}
= Hom_{W (M_{A})×W (G(n_{0}))}[σ_{e} : triv ⊗ L(χ_{0})].  (5.3.6)

Using the formulas for restrictions of representations for Weyl groups of classical types, it follows that

dim Hom_{W} [σ_{e} : L(χ)] = ∑_{k′} dim Hom_{W (G(n_{0}))}[σ_{e}(k′, n_{0} − k′) : L(χ_{0})]

{(5.3.7) \atop \{eq:5.3.5\}}

= ∑_{k′} dim Hom_{W (G(n_{0}))}[σ_{e}(k′, n_{0} − k′) : X_{e,G(n_{0})}],

where the last step is theorem 5.3. Since X_{e} = Ind_{M_{A}×G(n_{0})}^{G}[⊗L(χ_{i}) ⊗ X_{e,G(n_{0})}(χ_{0})], again by Frobenius reciprocity, one can show that this is also equal to dim Hom_{W} [σ_{e} : X_{e}]. This proves the claim for the p-adic case.

In the real case the proof is complete once we observe that in all the steps for the p−adic case, the multiplicity of triv ⊗ μ_{e}(k′, n_{0} − k′) in the restriction of μ_{e}(k, n − k) matches the multiplicity of triv ⊗ σ_{e}(k′, n_{0} − k′) in the restriction of σ_{e}(k, n − k). Similarly for μ_{o} and σ_{o}.

□

6. Hecke algebra calculations

{sec:6}
{sec:6.1}

6.1. The proof of the results in 5.3 is by a computation of intertwining operators on the relevant K−types. It only depends on the W −type of V^{K∩B}, so we work in the setting of the Hecke algebra. The fact that we can deal exclusively with W −types, is a big advantage. In particular we do not have to worry about disconnectedness of Levi components. We will write GL(k) for the Hecke algebra of type A and G(n) for the types B, C or D as the case may be. This is so as to emphasize that the results are about groups, real or p−adic.

The intertwining operators will be decomposed into products of simpler operators induced from operators coming from maximal Levi subgroups. We introduce these first.

Suppose M is a Levi component of the form

GL(a_{1}) × ⋯ × GL(a_{l}) × G(n_{0}).  (6.1.1) {eq:6.1.1}

Let χ_{i} be characters for GL(a_{i}). We simplify the notation somewhat by writing

χ_{i} ⟷ (ν_{i}) := (−\frac{a_{i} − 1}{2} + ν_{i}, …, \frac{a_{i} − 1}{2} + ν_{i}).  (6.1.2) \{eq:6.1.2\}

The parameter is antidominant, and so L(χ_{i}) occurs as a submodule of the principal series X((ν_{i})). The module is spherical 1-dimensional, and the action of 𝔞 is

χ_{i}(ω) = ⟨ω, (\frac{a_{i} − 1}{2} + ν_{i}, …, −\frac{a_{i} − 1}{2} + ν_{i})⟩,  ω ∈ 𝔞,  (6.1.3) \{eq:6.1.3\}

while W acts trivially. The trivial representation of G(n_{0}) corresponds to the string (−n_{0} + ϵ, …, −1 + ϵ) where

ϵ := \begin{cases}0 & ℍ of type B, \\ 1/2, & ℍ of type C, \\ 1, & ℍ of type D. \end{cases}  (6.1.4) \{eq:6.1.4\}

We abbreviate this as (ν_{0}). Again L(χ_{0}) is the trivial representation, and because χ_{0} is antidominant, it appears as a submodule of the principal series X(χ_{0}). We abbreviate

X_{M}(… (ν_{i}) …) := Ind_{∏ GL(a_{i})×G(n_{0})}^{G}[⊗χ_{i} ⊗ triv].  (6.1.5) {eq:6.1.5}

The module X_{M}(… (ν_{i}) …) is a submodule of the standard module X(χ) with parameter corresponding to the strings

χ := (…, −\frac{a_{i} − 1}{2} + ν_{i}, …, \frac{a_{i} − 1}{2} + ν_{i}, …, −n_{0} + ϵ, …, −1 + ϵ). (6.1.6) \{eq:6.1.6\}

In the setting of the Hecke algebra, the induced modules (6.1.5) is really X_{M}(… (ν_{i}) …) = ℍ ⊗_{ℍ_{M}} [⨂ χ_{i} ⊗ triv].

Let w_{i,i+1} ∈ W be the shortest Weyl group element which interchanges the strings (ν_{i}) and (ν_{i+1}) in ν, and fixes all other coordinates. The intertwining operator I_{w_{i,i+1}} : X(ν) ⟶ X(w_{i,i+1}ν) restricts to an intertwining operator

\begin{aligned}I_{M,i,i+1} & (… (ν_{i})(ν_{i+1}) …) : \\ & X_{M}(… (ν_{i})(ν_{i+1}) …) ⟶ X_{w_{i,i+1}M}(… (ν_{i+1})(ν_{i}) …). \end{aligned} (6.1.7) \{eq:6.1.7\}

This operator is induced from the same kind for GL(a_{i}+a_{i+1}) where M_{i,i+1} = GL(a_{i}) × GL(a_{i+1}) ⊂ GL(a_{i} + a_{i+1}) is the Levi component of a maximal parabolic subgroup.

Let w_{l} ∈ W be the shortest element which changes (ν_{l}) to (−ν_{l}), and fixes all other coordinates. It induces an intertwining operator

I_{M,l}(… (ν_{l})(ν_{0})) : X_{M}(… (ν_{l}), (ν_{0})) ⟶ X_{w_{l}M}(… (−ν_{l}), (ν_{0})). (6.1.8) {eq:6.1.8}

In this case, w_{l}M = M, so we will not always include it in the notation. In type D, if n_{0} = 0, the last entry of the resulting string might have to stay −\frac{a_{l}−1}{2} + ν_{l} instead of \frac{a_{l}−1}{2} − ν_{l}. This operator is induced from the same kind on G(a_{l} + n_{0}) with M_{l} = GL(a_{l}) × G(n_{0}) ⊂ G(a_{l} + n_{0}) the Levi component of a maximal parabolic subgroup.

Lemma. The operators I_{M,i,i+1} and I_{M,l} are meromorphic in ν_{i} in both the real and p-adic case.

(1) I_{M,i,i+1} has poles only if \frac{a_{i}−1}{2} + ν_{i} − \frac{a_{i+1}−1}{2} − ν_{i+1} ∈ ℤ. If so, a pole only occurs if

−\frac{a_{i} − 1}{2} + ν_{i} < −\frac{a_{i+1} − 1}{2} − ν_{i+1}, \frac{a_{i} − 1}{2} + ν_{i} < \frac{a_{i+1} − 1}{2} + ν_{i+1}.

(2) I_{M,l} has a pole only if \frac{a_{l}−1}{2} + ν_{l} ≡ ϵ(mod ℤ). In that case, a pole only occurs if

−\frac{a_{l} − 1}{2} + ν_{l} < 0.

Proof. We prove the assertion for I_{M,i,i+1}, the other one is similar. The fact that the integrality condition is necessary is clear. For the second condition, it is sufficient to consider the case M = GL(a_{1}) × GL(a_{2}) ⊂ GL(a_{1} + a_{2}). If the strings are strongly nested, then the operator cannot have any pole because X_{M} is irreducible. The remaining case is to show there is no pole in the case when −\frac{a_{2}−1}{2} + ν_{2} ≤ −\frac{a_{1}−1}{2} + ν_{1}, and \frac{a_{1}−1}{2} + ν_{1} > \frac{a_{2}−1}{2} + ν_{2}. Let

\begin{matrix}M′ := GL(\frac{a_{1} + a_{2}}{2} + ν_{2} + ν_{1}) × GL(\frac{a_{1} − a_{2}}{2} + ν_{1} − ν_{2}) × GL(a_{2}), \\ (ν′_{1}) = (−\frac{a_{1} − 1}{2} + ν_{1}, …, \frac{a_{2} − 1}{2} + ν_{2}) \\ (ν′_{2}) = (\frac{a_{2} − 1}{2} + 1 + ν_{2}, …, \frac{a_{1} − 1}{2} + ν_{1}) \\ (ν′_{3}) = (ν_{2}) = (−\frac{a_{2} − 1}{2} + ν_{2}, …, \frac{a_{2} − 1}{2} + ν_{2}). \end{matrix} {(6.1.9) \atop \{eq:6.1.9\}}

Then X_{M}((ν_{1})(ν_{2})) ⊂ X_{M′}((ν′_{1})(ν′_{2})(ν′_{3})), and I_{M,1,2} is the restriction of I_{w_{2,3}M′,1,2}, ((ν′_{1})(ν′_{3})(ν′_{2}) ∘ I_{M′,2,3}((ν′_{1})(ν′_{2})(ν′_{3})) to X_{M}. Because the strings (ν′_{1})(ν′_{3}) are strongly nested, I_{w_{2,3}M′,1,2} has no pole, and I_{M′,2,3} has no pole because it is a restriction of operators coming from SL(2)’s which do not have poles. The claim follows. □

Let σ be a W −type. We are interested in computing r_{σ}(w, … (ν_{i}) …), where w changes all the ν_{i} for 1 ≤ i to −ν_{i}. The operator can be factored into a product of r_{σ}(w_{i,i+1}, ∗) of the type (6.1.7) and r_{σ}(w_{l}, ∗) of the type (6.1.8).

These operators are more tractable. Here’s a more precise explanation. Let M be the Levi component

GL(a_{1}) × ⋯ × GL(a_{i} + a_{i+1}) × … in case (6.1.7) {(6.1.10) \atop \{eq:6.1.11\}}

GL(a_{1}) × ⋯ × G(a_{l} + n_{0}) in case (6.1.8) {(6.1.11) \atop \{eq:6.1.12\}}

Since X_{M} is induced from the trivial W (M) module,

Hom_{W} [σ, X_{M}((ν_{i}))] = Hom_{W (M)}[σ|_{W (M)} : triv ⊗ X_{M_{i,i+1}}((ν_{i}), (ν_{i+1})) ⊗ triv]
in case (6.1.7)  (6.1.12) {eq:6.1.13}
Hom_{W} [σ, X((ν_{i}))] = Hom_{W (M)}[σ|_{W (M)} : triv ⊗ X_{M_{l}}((ν_{l}), (ν_{0}))]
in case (6.1.8)  (6.1.13) {eq:6.1.14}

where M_{i,i+1} = GL(a_{i})×GL(a_{i+1}) is a maximal Levi component of GL(a_{i} + a_{i+1}) and M_{l} = GL(a_{l}) × G(n_{0}) is a maximal Levi component of G(a_{l} + n_{0}). To compute the r_{σ}(w_{i,i+1}, ∗) and r_{σ}(w_{l}, ∗), it is enough to compute the corresponding r_{σ_{j}} for the σ_{j} ocuring in the restriction σ |_{W (M)} in the cases GL(a_{i}) × GL(a_{i+1}) ⊂ GL(a_{i} + a_{i+1}) and GL(a_{l}) × G(n_{0}) ⊂ G(a_{l} + n_{0}). The restrictions of relevant W −types to Levi components consist of relevant W −types of the same kind, i.e. σ[(n − r), (r)] restricts to a sum of representations of the kind σ_{e}, and σ[(k, n − k), (0)] restricts to a sum of σ_{o}. Typically the multiplicities of the factors are 1.

We also note that

X_{M} |_{W} = ∑_{σ∈Ŵ} V_{σ} ⊗ (V_{σ}^{∗})^{W (M)}.  (6.1.14) {eq:6.1.15}

So the r_{σ}(w, ∗) map (V_{σ}^{∗})^{W (M)} to (V_{σ}^{∗})^{W (wM)}.

In the next sections we will compute the cases of Levi components of maximal parabolic subgroups.

                                                                       {6.2}

6.2. GL(a) × GL(b) ⊂ GL(a + b). This is the case of I_{i,i+1} with i < l. Let n = a + b and G = GL(n) and M = GL(a) × GL(b). The module X_{M}((ν_{1}), (ν_{2})) induced from the characters corresponding to

(−\frac{a − 1}{2} + ν_{1}, …, \frac{a − 1}{2} + ν_{1}), (−\frac{b − 1}{2} + ν_{2}, …, \frac{b − 1}{2} + ν_{2}) (6.2.1) \{eq:6.2.1\}

has the following S_{a+b} structure. Let m := min(a, b) and write σ(k, a+b−k) for the module corresponding to the partition (k, a + b − k), 0 ≤ k ≤ m. Then

X_{M}((ν_{1}), (ν_{2})) |_{W} = ⨁_{0≤k≤m} σ(k, a + b − k).  (6.2.2) {eq:6.2.2}

                                                                       {l:6.2}

Lemma. For 1 ≤ k ≤ m, the intertwining operator I_{M,1,2}((ν_{1})(ν_{2})) restricted to σ gives

r_{σ(k,a+b−k)}(a, b, ν_{1}, ν_{2}) = ∏_{0≤j≤k−1} \frac{(ν_{1} − \frac{a−1}{2}) − (\frac{b−1}{2} + ν_{2} + 1) + j}{(ν_{1} + \frac{a−1}{2}) − (−\frac{b−1}{2} + ν_{2} − 1) − j} .

Proof. The proof is an induction on a, b and k. We omit most details but give the general idea. Assume 0 < k < m, the case k = m is simpler. Embed X_{M}((ν_{1}), (ν_{2})) into X_{M′}((ν′), (ν′′), (ν_{2})), where M′ = GL(a − 1) × GL(1) × GL(b), corresponding to the strings

(−\frac{a − 1}{2} + ν_{1}, …, \frac{a − 3}{2} + ν_{1}), (\frac{a − 1}{2} + ν_{1}), (−\frac{b − 1}{2} + ν_{2}, …, \frac{b − 1}{2} + ν_{2}).

(6.2.3) {eq:6.2.3} The intertwining operator I_{M,1,2}(ν_{1}, ν_{2}) is the restriction of

I_{M′,1,2}(ν′, ν_{2}, ν′′) ∘ I_{M′,2,3}(ν′; ν′′, ν_{2}) {(6.2.4) \atop \{eq:6.2.4\}}

to X_{M}((ν_{1}), (ν_{2})) ⊂ X_{M′}((ν′), (ν′′), (ν_{2})). By an induction on n we can assume that these operators are known. The W −type σ(k, n − k) occurs with multiplicity 1 in X_{M}((ν_{1}), (ν_{2})) and with multiplicity 2 in X_{M′}((ν′), (ν′′), (ν_{2})). The restrictions are

σ(k, n − k) |_{W (M′)} = triv ⊗ σ(k − 1, b + 1 − k) + triv ⊗ σ(k, b − k) (6.2.5)

for I_{M′,1,2} {(6.2.6) \atop \{eq:6.2.5\}}

σ(k, n − k) |_{W (M′)} = σ(1, b) + σ(0, b + 1) for I_{M′,2,3} {(6.2.7) \atop \{eq:6.2.6\}}

The representation σ(k, n − k) has a realization as harmonic polynomials in S(𝔞) spanned by

∏_{1≤ℓ≤k}(ϵ_{i_{ℓ}} − ϵ_{j_{ℓ}}) {(6.2.8) \atop \{eq:6.2.7\}}

where (i_{1}, j_{1}), …, (i_{k}, j_{k}) are k pairs of integers satisfying 1 ≤ i_{ℓ}, j_{ℓ} ≤ n, and i_{ℓ} ≠ j_{ℓ}. We apply the intertwining operator to the S_{a} × S_{b}−fixed vector

e := ∑_{x∈S_{a}×S_{b}} x · [(ϵ_{1} − ϵ_{a+1}) × ⋯ × (ϵ_{k} − ϵ_{a+k})]. {(6.2.9) \atop \{eq:6.2.8\}}

The intertwining operator I_{M′,2,3}, has a simple form on the vectors

e_{1} := ∑_{x∈S_{a−1}×S_{b+1}} x · [(ϵ_{1} − ϵ_{a+1}) × ⋯ × (ϵ_{k} − ϵ_{a+k})], in σ(0, b + 1) {(6.2.10) \atop \{eq:6.2.9\}}

e_{2} := ∑_{x∈S_{a−1}×S_{1}×S_{b}} x · [(ϵ_{1} − ϵ_{a+1}) × ⋯ × (ϵ_{k−1} − ϵ_{a+k−1})(ϵ_{a} − ϵ_{a+k})], in σ(1, b) {(6.2.11) \atop \{eq:6.2.10\}}

which appear in (6.2.7). They are mapped into scalar multiples (given by the lemma) of the vectors e′_{1}, e′_{2} which are invariant under S_{a−1} × S_{b} × S_{1}, and transform according to triv ⊗ σ(0, b + 1) and triv ⊗ σ(1, b). We choose

\begin{matrix}e′_{1} = e_{1}, \\ e′_{2} := ∑_{x∈S_{a−1}×S_{b}×S_{1}} x · [(ϵ_{1} − ϵ_{a}) × ⋯ × (ϵ_{k−1} − ϵ_{a+k−2})(ϵ_{n} − ϵ_{a+k−1})] \end{matrix}

(6.2.12) {eq:6.2.11} The intertwining operator I_{M′,1,2} has a simple form on the vectors invariant under S_{a−1} × S_{b} × S_{1} transforming according to σ(k, n − k − 1) and σ(k − 1, n − k). We can choose multiples of

f_{1} :=  (6.2.13) {eq:6.2.12}
∑_{x∈S_{a−1}×S_{b}×S_{1}}x · [(ϵ_{1} − ϵ_{a}) × ⋯ × (ϵ_{k−1} − ϵ_{a+k−2})(ϵ_{k} − ϵ_{a+k−1})],
in σ(k − 1, n − k)
f_{2} :=  (6.2.14) {eq:6.2.13}
∑_{x∈S_{a−1}×S_{b}×S_{1}}x · [(ϵ_{1} − ϵ_{a}) × ⋯ × (ϵ_{k−1} − ϵ_{a+k−2})·
·(e_{k} + ⋯ + ϵ_{a−1}+ϵ_{a} + ϵ_{a+k} + ⋯ + ϵ_{n−1} − (n − 2k + 1)ϵ_{n})]
in σ(k, n − k − 1)

The fact that f_{1} transforms according to σ(k, n−1) follows from (6.2.8). The fact that f_{2} transforms according to σ(k −1, n) is slightly more complicated. The product ∏(ϵ_{1} − ϵ_{a}) × ⋯ × (ϵ_{k−1} − ϵ_{a+k−2}) transforms according to σ(k − 1, k − 1) under S_{2k−2}. The vector (e_{k} + ⋯ + ϵ_{a−1} + ϵ_{a} + ϵ_{a+k} + ⋯ + ϵ_{n−1}−(n−2k+1)ϵ_{n}) is invariant under the S_{n−2k−1} acting on the coordinates ϵ_{k}, … ϵ_{a}, ϵ_{a+k}, …, ϵ_{n−1}. Since σ(k, n − k − 1) does not have such invariant vectors, the product inside the sum in (6.2.14) must transform according to σ(k − 1, n − k). The average under x in (6.2.14) is nonzero. The operator I_{M′,2,3} maps f_{1} and f_{2} into multiples (using the induction hypothesis) of the vectors f′_{1}, f′_{2} which are the S_{b} × S_{a−1} × S_{1} invariant vectors transforming according to σ(k, n − 1) and σ(k − 1, n − k). The composition I_{M′,1,2} ∘ I_{M′,2,3} maps e into a multiple of

e′ := ∑_{σ∈S_{b}×S_{a}} σ · [(ϵ_{1} − ϵ_{b+1}) × ⋯ × (ϵ_{k} − ϵ_{b+k})].  (6.2.15) {eq:6.2.14}

The multiple is computable by using the induction hypothesis and the expression of

e in terms of e_{1}, e_{2},
e′_{1}, e′_{2} in terms of f_{1}, f_{2}, and
e′ in terms of f′_{1}, f′_{2}.

For example for the case k = 1, we get the following formulas.

\begin{matrix}e = b(ϵ_{1} + ⋯ + ϵ_{a}) − a(ϵ_{a+1} + ⋯ + ϵ_{n}), \\ e_{1} = (b + 1)(ϵ_{1} + ⋯ + ϵ_{a−1}) − (a − 1)(ϵ_{a} + ⋯ + ϵ_{n}), \\ e_{2} = bϵ_{a} − (ϵ_{a+1} + ⋯ + ϵ_{n}), \\ f_{1} = b(ϵ_{1} + ⋯ + ϵ_{a−1}) − (a − 1)(ϵ_{a} + ⋯ + ϵ_{n−1}), \\ f_{2} = (ϵ_{1} + ⋯ + ϵ_{a−1}) + (ϵ_{a} + ⋯ + ϵ_{n−1}) − (n − 1)ϵ_{n}, \\ e′ = −a(ϵ_{1} + ⋯ + ϵ_{b}) − b(ϵ_{b+1} + ⋯ + ϵ_{n}), \\ e′_{1} = (b + 1)(ϵ_{1} + ⋯ + ϵ_{a−1}) − (a − 1)(ϵ_{a} + ⋯ + ϵ_{n}), \\ e′_{2} = −(ϵ_{a} + ⋯ + ϵ_{n−1}) + b(ϵ_{n}), \\ f′_{1} = −(a + 1)(ϵ_{1} + ⋯ + ϵ_{b}) + b(ϵ_{b+1} + ⋯ + ϵ_{n−1}), \\ f′_{2} = (ϵ_{1} + ⋯ + ϵ_{b}) + (ϵ_{b+1} + ⋯ + ϵ_{n−1}) − (n − 1)ϵ_{n}. \end{matrix}  (6.2.16) \{eq:6.2.14a\}

Then

\begin{matrix}e = \frac{a − 1}{b + 1}e_{1} − \frac{n}{b + 1}e_{2}, \\ e′_{1} = \frac{n}{n − 1}f_{1} + \frac{a − 1}{n − 1}f_{2}, \\ e′_{2} = \frac{1}{n − 1}f_{1} − \frac{b}{n − 1}f_{2}, \\ e′ = \frac{n}{n − 1}f′_{1} − \frac{b}{n − 1}f′_{2}. \end{matrix} {(6.2.17) \atop \{eq:6.2.15\}}

□

{sec:6.3}

6.3. GL(k) × G(n) ⊂ G(n + k). In the next sections we prove theorem 5.3 in the case of a parabolic subgroup with Levi component GL(k) × G(n) for the induced module

X_{M}((ν_{1})(ν_{0})) = Ind_{M}^{G}[L(χ_{1}) ⊗ L(χ_{0})]. {(6.3.1) \atop \{eq:6.3.1\}}

The notation is as in section 6.1.

The strings are

(−\frac{k − 1}{2} + ν, …, \frac{k − 1}{2} + ν)(−n + 1 + ϵ, …, −1 + ϵ). {(6.3.2) \atop \{eq:6.3.2\}}

Recall that ϵ = 0 when the Hecke algebra is type B, ϵ = 1/2 for type C, and ϵ = 1 for type D, and

r_{σ}(ν) : (V_{σ}^{∗})^{W (M)} ⟶ (V_{σ}^{∗})^{W (M)}. {(6.3.3) \atop \{eq:6.3.3\}}

We will compute r_{σ}(w_{1}, (ν)(ν_{0})) by induction on k. In this case the relevant W −types have multiplicity ≤ 1 so r_{σ} is a scalar.

{sec:6.4}

6.4. We start with the special case k = 1 when the maximal parabolic subgroup P has Levi component M = GL(1) × G(n) ⊂ G(n + 1). In type D we assume n ≥ 1. Then

X_{M} |_{W} = σ[(n + 1), (0)] + σ[(1, n), (0)] + σ[(n), (1)], {(6.4.1) \atop \{eq:6.4.1\}}

and all the W −types occuring are relevant. In types B,C the operator r_{σ}(ν) is the restriction to (V_{σ}^{∗})^{W (M)} of the product

r_{1,2} ∘ ⋯ ∘ r_{n,n+1} ∘ r_{n+1} ∘ r_{n,n+1} ∘ ⋯ ∘ r_{1,2} {(6.4.2) \atop \{eq:6.4.2\}}

as an operator on V_{σ}. Here r_{i,j} is the r_{σ}(w, ∗) corresponding to the root ϵ_{i} − ϵ_{j} and r_{n+1} is the r_{σ} corresponding to ϵ_{n+1} or 2ϵ_{n+1} in types B and C. In type D, the operator is

r_{1,2} ∘ ⋯ ∘ r_{n,n+1} ∘ { \atop r_{n,n+1}} ∘ ⋯ ∘ r_{1,2}  (6.4.3) \{eq:6.4.2d\}

where r_{i,i+1} are as before, and { \atop r_{n,n+1}} corresponds to ϵ_{n} + ϵ_{n+1}. Since the multiplicities are 1, this is a scalar.

                                                                       {p:6.4}

Proposition. The scalar r_{σ}(w_{1}, ((ν)(ν_{0}))) is

\begin{aligned}& σ_{e}(1, n) = σ[(n), (1)] & σ_{o}(1, n) = σ[(1, n), (0)] \\ B & \frac{n+1−ν}{n+1+ν} & −\frac{n+1−ν}{n+1+ν} \\ C & \frac{1/2+n−ν}{1/2+n+ν} & \frac{1/2+n−ν}{1/2+n+ν} · \frac{1/2−ν}{1/2+ν} \\ D & \frac{n−ν}{n+ν} & \frac{n−ν}{n+ν} \frac{1−ν}{1+ν} \end{aligned}  (6.4.4) \{eq:6.4.3\}

Proof. We do an induction on n.

The reflection representation σ[(n), (1)] has dimension n+1 and the usual basis {ϵ_{i}}. The W (M)−fixed vector is ϵ_{1}. The representation σ[(1, n), (0)] has a basis ϵ_{i}^{2} − ϵ_{j}^{2} with the symmetric square action. The W (M)−fixed vector is ϵ_{1}^{2} − \frac{1}{n}(ϵ_{2}^{2} + ⋯ + ϵ_{n+1}^{2}).

The case n = 0 for type C is clear; the intertwining operator is 1 on μ_{o}(1, 0) = triv and \frac{1/2−ν}{1/2+ν} on μ_{e}(0, 1) = sgn. We omit the details for type B. In type for n = 1, i.e. D_{2}, the middle W −type in (6.4.1) decomposes further

σ[(2), (0)] + σ[(1), (1)]_{I} + σ[(1), (1)]_{II} + σ[(0), (2)].  (6.4.5) {eq:6.4.4}

The representations σ[(1), (1)]_{I,II} are 1-dimensional with bases ϵ_{1} ± ϵ_{2}. The result is straightforward in this case as well.

We now do the induction step. We give details for type B. In the case σ_{e}(1, n), embed X_{M} in the induced module from the characters corresponding to

(ν)(−n)(−n + 1, …, −1).  (6.4.6) {eq:6.4.5}

Write M′ = GL(1)×GL(1)×G(n−1) for the Levi component corresponding to these three strings. Then the intertwining operator I : X_{M}((ν)(ν_{0})) ⟶ X_{M}((−ν)(ν_{0})) is the restriction of

I_{M′,1,2}((−n), (−ν)(ν_{0})) ∘ I_{M′,2}((−n)(ν)(ν_{0})) ∘ I_{M′,1,2}(ν, (−n), (ν_{0})). (6.4.7) {eq:6.4.6}

The r_{σ} have a corresponding decomposition

(r_{σ})_{M′,1,2}((−ν), (−n)(ν_{0})) ∘ (r_{σ})_{M′,2}((−n)(ν)(ν_{0})) ∘ (r_{σ})_{M′,1,2}((ν)(−n)(ν_{0})).

(6.4.8) {eq:6.4.7} We need the restrictions of μ_{e}(1, n) and μ_{o}(1, n) to W (M′). We have

Ind_{W (B_{n−1})}^{W (B_{n+1})}[σ[(n − 1), (0)]] = σ[(n + 1), (0)] + 2σ[(n), (1)] + 2σ[(1, n), (0)]

+ 2σ[(1, n − 1), (1)] + σ[(n − 1), (2)] + σ[(n − 1), (1, 1)]+

+ σ[(2, n − 1), (0)] + σ[(1, 1, n − 1), (0)], {(a) \atop \{eq:6.4.8\}}

Ind_{W (B_{n})}^{W (B_{n+1})}[σ[(n), (0)]] = σ[(n + 1), (0)] + σ[(n), (1)] + σ[(1, n), (0)], (b)

(6.4.9)

Ind_{W (B_{1})W (B_{n})}^{W (B_{n+1})}[σ[(1), (0)] ⊗ σ[(n), (0)]] = σ[(n + 1), (0)] + σ[(1, n), (0)] (c)
Ind_{W (B_{1})W (B_{n})}^{W (B_{n+1})}[σ[(0), (1)] ⊗ σ[(n), (0)]] = σ[(n − 1), (1)]  (d)

Thus μ_{e}(1, n) occurs with multiplicity 2 in X_{M′}. The W (M′) fixed vectors are the linear span of ϵ_{1}, ϵ_{2}. The intertwining operators I_{M′,1,2} and I_{M′,2} are induced from maximal parabolic subgroups whose Levi components we label M_{1} and M_{2}. Then ϵ_{1} +ϵ_{2} transforms like triv ⊗triv under W (M_{1}) and ϵ_{1}−ϵ_{2} transforms like sgn⊗triv. The vector ϵ_{1} is fixed under W (B_{n}) (which corresponds to M_{2}) and the vector ϵ_{2} is fixed under W (B_{n−1}) and transforms like μ_{o}(1, n) under W (B_{n}). The matrix r_{σ} is, according to (6.4.8),

[\begin{matrix}\frac{1}{2+ν−n} & \frac{ν−n+1}{2+ν−n} \\ \frac{ν−n+1}{1+ν−n+1} & \frac{1}{2+ν−n} \end{matrix}] · [\begin{matrix}1 & 0 \\ 0 & \frac{n−ν}{n+ν} \end{matrix} ] · [\begin{matrix}\frac{1}{1+ν+n} & \frac{ν+n}{1+ν+n} \\ \frac{ν+n}{1+ν+n} & \frac{1}{2+ν+n} \end{matrix}] . {(6.4.10) \atop \{eq:6.4.9\}}

So the vector ϵ_{1} is mapped into \frac{n+1−ν}{n+1+ν} ϵ_{1} as claimed. For σ_{o}(1, n) we apply

the same method. In this case the operator I_{M′,2} is the identity because in the representation μ_{o}(1, n) the element t_{n} corresponding to the short simple root acts by 1.

The calculation for type D is analogous, we sketch some details. We decompose the strings into

(ν)(−n + 1, …, −1)(0), {(6.4.11) \atop \{eq:6.4.10\}}

and M′ = GL(1) × GL(n − 1) × GL(1). Then

I_{M,1}((ν)(ν_{0})) = {(6.4.12) \atop \{eq:6.4.11\}}
I_{M′,1,2}((−n + 1, …, −1)(−ν)(0)) ∘ I_{M′,1}((−n + 1, …, −1)(ν)(0))∘
I_{M′,1,2}((ν)(−n + 1, …, −1)(0)).

□ {sec:6.5}

6.5. In this section we consider (6.3.2) for k > 1, n ≥ 1 and the W −types σ_{e}(m, n + k − m) for 0 ≤ m ≤ k (notation as in definition 4.8). These are the W −types which occur in X_{M}, with M = GL(k) × G(n) ⊂ G(k + n).

                                                                       {p:6.5}

Proposition. The r_{σ}(w_{1}, ((ν)(ν_{0})) for σ = σ_{e}(m, n + k − m) are scalars. They equal

Type B:

∏_{0≤j≤m−1} \frac{n + 1 − (−\frac{k−1}{2} + ν) − j}{n + 1 + (\frac{k−1}{2} + ν) − j}  (6.5.1) \{eq:6.5.1\}

Type C:

∏_{0≤j≤m−1} \frac{n + 1/2 − (−\frac{k−1}{2} + ν) − j}{n + 1/2 + (\frac{k−1}{2} + ν) − j}  (6.5.2) \{eq:6.5.2\}

Type D:

∏_{0≤j≤m−1} \frac{n − (−\frac{k−1}{2} + ν) − j}{n + (\frac{k−1}{2} + ν) − j}  (6.5.3) \{eq:6.5.3\}

Proof. The proof is by induction on k. The case k = 1 was done in section 6.4 so we only need to do the induction step. For types B,C factor the intertwining operator as follows. Decompose the string

((ν′)(\frac{k − 1}{2} +ν)(ν_{0})) := ((−\frac{k − 1}{2} +ν, …, \frac{k − 3}{2} +ν)(\frac{k − 1}{2} +ν)(ν_{0})) (6.5.4) \{eq:6.5.4\}

and let M′ := GL(k − 1) × GL(1) × G(n), and M′′ = GL(1) × GL(k − 1) × G(n). Thus

\begin{aligned}I_{M,1} & = I_{M′′,2}((−\frac{k − 1}{2} − ν)(ν′)(ν_{0}))∘ \\ & I_{M′,1,2}((ν′)(−\frac{k − 1}{2} − ν)(ν_{0}))∘ \\ & I_{M′,2}((ν′)(\frac{k − 1}{2} + ν)(ν_{0})) \end{aligned}  (6.5.5) \{eq:6.5.5\}

I_{M′,1,2} and I_{M′,2} were computed earlier, while I_{M′′,2} is known by induction. Then

σ_{e}(m, n + k − m) |_{W (GL(k−1)×W (G(n+1))}=
triv ⊗ [σ_{e}(1, n) + σ_{e}(0, n + 1) + …  (6.5.6) {eq:6.5.6}
σ_{e}(m, n + k − m) |_{W (GL(k)×W (G(n+k−1))}=
[(k) ⊗ triv + (1, k − 1) ⊗ triv] + …  (6.5.7) {eq:6.5.7}
σ_{e}(m, n + k − m) |_{W (GL(1)×W (G(n+k−1))}=
triv ⊗ [σ_{e}(m − 1, n + k − m) + σ_{e}(m, n + k − 1 − m)] + …  (6.5.8) {eq:6.5.8}

where … denote W −types which are not spherical for W (M), so do not matter for the computations.

The W −type σ_{e}(m, n + k − m) ≅ ⋀^{m} σ_{e}(1, n + k − 1). It occurs with multiplicity 2 in X_{M′} for 0 < m < min(k, n) and multiplicity 1 for m = min(k, n). We will write out an explicit basis for the invariant S_{1} × S_{k−1} × W (B_{n}) vectors. Formulas (6.5.2)-(6.5.4) then come down to a computation with 2 × 2 matrices as in the case k = 1. Let

e := \frac{1}{m!(k − m)!} ∑_{x∈S_{k}} x · [ϵ_{1} ∧ ⋯ ∧ ϵ_{m}]. {(6.5.9) \atop \{eq:6.5.9\}}

This is the S_{k} × W (B_{n}) fixed vector of σ_{e}(m, n + k − m). It decomposes as

e = e_{0} + e_{1} = f_{0} + f_{1} {(6.5.10) \atop \{eq:6.5.10\}}

where

\begin{matrix}e_{0} = \frac{1}{m!(k − 1 − m)!} ∑_{x∈S_{k−1}×S_{1}} x · [ϵ_{1} ∧ ⋯ ∧ ϵ_{m}], \\ e_{1} = \frac{1}{(m − 1)!(k − m)!} ∑_{x∈S_{k−1}×S_{1}} x · [ϵ_{1} ∧ ⋯ ∧ ϵ_{m−1}] ∧ ϵ_{k}, \\ f_{0} = \frac{1}{m!(k − 1 − m)!} ∑_{x∈S_{1}×S_{k−1}} x · [ϵ_{2} ∧ ⋯ ∧ ϵ_{m+1}], \\ f_{1} = \frac{1}{(m − 1)!(k − m)!} ∑_{x∈S_{1}×S_{k−1}} ϵ_{1} ∧ x · [ϵ_{2} ∧ ⋯ ∧ ϵ_{m}]. \end{matrix} {(6.5.11) \atop \{eq:6.5.11\}}

Let also

\begin{matrix}e′_{0} = e′′_{0} = \frac{1}{(m − 1)!(k − m)!} ∑_{x∈S_{k}} x · [ϵ_{1} ∧ ⋯ ∧ ϵ_{m}], \\ e′_{1} = ∑_{x∈S_{k−1}×S_{1}} x · [ϵ_{1} ∧ ⋯ ∧ ϵ_{m−1} ∧ (ϵ_{m} − ϵ_{k})], \\ e′′_{1} = ∑_{x∈S_{1}×S_{k−1}} x · [(−ϵ_{1} + ϵ_{m+1}) ∧ ϵ_{2} ∧ ⋯ ∧ ϵ_{m+1}]. \end{matrix} {(6.5.12) \atop \{eq:6.5.12\}}

Then

\begin{matrix}e_{0} = \frac{k − m}{k}e′_{0} + \frac{m}{k}e′_{1}, & e_{1} = \frac{m}{k}e′_{0} − \frac{m}{k}e′_{1}, \\ e′′_{0} = f_{0} + f_{1}, & e′′_{1} = f_{0} − \frac{k − m}{m}f_{1}. \end{matrix} {(6.5.13) \atop \{eq:6.5.13\}}

We now compute the action of the intertwining operators. The following relations hold:

\begin{matrix}I_{M′,2}(e_{0}) = e_{0},  I_{M′,2}(e_{1}) = \frac{n + ϵ − (\frac{k−1}{2} + ν)}{n + ϵ + (\frac{k−1}{2} + ν)}e_{1}, \\ I_{M′,12}(e′_{0}) = e′′_{0},  I_{M′,12}(e′_{1}) = \frac{2ν − 1}{2ν + k − 1}e′′_{1}, \\ I′_{M′′,2}(f_{0}) = ∏_{0≤j≤m−2} \frac{n + ϵ − (−\frac{k−1}{2} + ν) − j}{n + ϵ + (\frac{k−3}{2} + ν) − j} f_{0}, \\ I_{M′′,2}(f_{1}) = ∏_{0≤j≤m−1} \frac{n + ϵ − (−\frac{k−1}{2} + ν) − j}{n + ϵ + (\frac{k−3}{2} + ν) − j} f_{1}, \end{matrix} {(6.5.14) \atop \{eq:6.5.14\}}

where ϵ = 1 in type B, ϵ = 1/2 in type C, and ϵ = 0 in type D. Then

I_{M′,2}(e_{0} + e_{1}) = e_{0} + \frac{n + ϵ − (\frac{k−1}{2} + ν)}{n + ϵ + (\frac{k−1}{2} + ν)}e_{1}.  (6.5.15) \{eq:6.5.15\}

Substituting the expressions of e_{0}, e_{1} in terms of e′_{0}, e′_{1}, we get

[\frac{k − m}{k} + \frac{m}{k} \frac{n + ϵ − (\frac{k−1}{2} + ν)}{n + ϵ + (\frac{k−1}{2} + ν)}]e′_{0} + \frac{m}{k}[1 − \frac{n + ϵ − (\frac{k−1}{2} + ν)}{n + ϵ + (\frac{k−1}{2} + ν)}]e′_{1}. (6.5.16) \{eq:6.5.16\}

Applying I_{M,2} to this has the effect that e′_{0} is sent to e′′_{0} and the term in e′_{1} is multiplied by \frac{2ν−1}{2ν+k−1} and e′_{1} is replaced by e′′_{1}. Substituting the formulas for e′′_{0} and e′′_{1} in terms of f_{0}, f_{1}, and applying I_{M′′,2}, we get the claim of the proposition. □

                                                                       {sec:6.6}

6.6. We now treat the case σ = σ_{o}(m, n + k − m). We assume n > 0 or else these W −types do not occur in the induced module X_{M}.

                                                                       {p:6.6}

Proposition. The r_{σ}(w_{1}, ((ν)(ν_{0})) are scalars. They equal

∏_{0≤j≤m−1} \frac{(ν − \frac{k−1}{2}) − (1 − ϵ) + j}{(ν + \frac{k−1}{2}) − (−n − ϵ) − j} · \frac{(−n − ϵ) − (−ν + \frac{k−1}{2}) + j}{(1 − ϵ) − (−ν − \frac{k−1}{2}) − j} (6.6.1) \{eq:6.6.1\}

Proof. The intertwining operator I_{M}(ν) decomposes in the same way as (6.5.5). Furthermore, σ_{o}(m, n+k −m) = ⋀^{m} σ_{o}(1, n+k −1). The difference from the cases σ_{e} is that while σ_{e}(1, n+k−1) is the reflection representation, and therefore realized as the natural action on ϵ_{1}, … ϵ_{n+k}, σ_{o}(1, n + k − 1) occurs in S^{2}σ_{e}(1, n + k − 1), generated by ϵ_{i}^{2} − ϵ_{j}^{2} with i ≠ j. We can apply the same technique as for σ_{e}(m, n + k − m), and omit the details. □

GL(k) ⊂ G(k) in types B, C. The formulas in proposition 6.5 and 6.6 hold with n = 0. The proof is the same, but because n = 0, (ν_{0}) is not present. The operator I_{M′,2} is an intertwining operator in SL(2) and therefore simpler.

                                                                       {sec:6.7}

6.7. GL(k) ⊂ G(k) in type D. In this section we consider the maximal Levi components M := GL(k) ⊂ G(k) and M′ := GL(k)′ ⊂ G(k) for type D_{n}. The parameter corresponds to the string (ν) := (−\frac{k−1}{2} +ν, …, \frac{k−1}{2} +ν) or (ν′) := (−\frac{k−1}{2} + ν, …, −\frac{k−1}{2} − ν).

k even: The W −structure of X_{M}((ν)) and X_{M′}((ν)′) is σ_{e}[(n−r), (r)] for 0 ≤ r < k/2, and σ_{e}[(k/2), (k/2)]_{I}, or σ_{e}[k/2), (k/2)]_{II} respectively, with multiplicity 1. There are intertwining operators

\begin{matrix}I_{M}((ν)) : X_{M}((ν)) ⟶ X_{M}((−ν)), \\ I_{M′}((ν)′) : X_{M′}((ν)′) ⟶ X_{M′}((−ν)′). \end{matrix} {(6.7.1) \atop \{eq:6.7.1\}}

corresponding to the shortest Weyl group element changing ((ν)) to ((−ν)). They determine scalars r_{σ}((ν)) and r_{σ}((ν)′).

k odd: The W −structure in this case is σ_{e}[(n − r), (r)] with 0 ≤ r ≤ [k/2] for both X_{M} and X_{M′}, again with multiplicity 1. In this case there is a shortest Weyl group element which changes ((ν)) to ((−ν)′), and one which changes ((ν)′) to ((−ν)). These elements give rise to intertwining operators

\begin{matrix}I_{M}((ν)) : X_{M}((ν)) ⟶ X_{M′}((−ν)′), \\ I_{M′}((ν)) : X_{M′}((ν)′) ⟶ X_{M}((−ν)). \end{matrix} {(6.7.2) \atop \{eq:6.7.2\}}

Because the W −structure of X_{M} and X_{M′} is the same, and W −types occur with multiplicity 1, these intertwning operators define scalars r_{σ}(ν) and r_{σ}((ν)′).

{p:6.7}

Proposition. The scalars r_{σ}((ν)) and r_{σ}((ν)′) are

r_{σ_{e}[(n−r),(r)]}((ν)) = ∏_{0≤j<r} \frac{(\frac{k−1}{2} − ν) − j}{(\frac{k−1}{2} + ν) − j} . {(6.7.3) \atop \{eq:6.7.3\}}

These numbers are the same for ((ν)) and ((ν)′), and representations with subscripts I, II they depend only on r.

{sec:6.8}

6.8. Proof of theorem 5.3. We use the results in the previous sections to prove the theorem in general. We give the details in the case of the group of type B and W −types σ_{e}. Thus the Hecke algebra is type C. There are no significant changes in the proof for the other cases. Recall the notation from section 2.3. Conjugate ν the middle element for the nilpotent orbit with partition (2x_{0}, …, 2x_{2m}) so that it is dominant (i.e. the coordinates are in decreasing order),

ν = (x_{2m} − 1/2, …, x_{2m} − 1/2, …, x_{0} − 1/2, …, x_{0} − 1/2, …, 1/2, …, 1/2)

Then ν is dominant, so X(ν) has a unique irreducible quotient L(ν). We factor the long intertwining operator so that

X(ν) ⟶^{I_{1}} X_{e}(ν) ⟶^{I_{2}} X(−ν), {(6.8.1) \atop \{eq:6.8.1\}}

where X_{e} was defined in section 5.3. The claim will follow if the decomposition has the property that the operator I_{1} is onto, and I_{2} is into, when restricted to the σ_{e} isotypic component. Proof that I_{1} is onto. The operator I_{1} is a composition of several operators. First take the long intertwining operator induced from the Levi component GL(n),

X(x_{2m} − 1/2, …, 1/2) ⟶ X(1/2, …, x_{2m} − 1/2),  (6.8.2) {eq:6.8.2}

corresponding to the shortest Weyl group element that permutes the entries of the parameter from decreasing order to increasing order. The image is the induced from the corresponding irreducible spherical module L(1/2, …, x_{2m} − 1/2) on GL(n). In turn this is induced irreducible from 1-dimensional spherical characters on a GL(x_{0}) × ⋯ × GL(x_{2m}) Levi component corresponding to the strings

(1/2, …, x_{0} − 1/2) … (1/2, …, x_{2m} − 1/2)

or any permutation thereof. This is well known by results of Bernstein-Zelevinski in the p−adic case, [V1] for the real case.

Compose with the intertwining operator

X(… (1/2, …, x_{2m} − 1/2)) ⟶ X(… (−x_{2m} + 1/2, …, −1/2)), (6.8.3) {eq:6.8.3}

all other entries unchanged. This intertwining operator is induced from the standard long intertwining operator on G(x_{2m}) which has image equal to the trivial representation. The image is an induced module from characters on GL(x_{0})×⋯×GL(x_{2m−1})×G(x_{2m}). Now compose with the intertwining operator

X(… (1/2, …, x_{2m−1} − 1/2)(−x_{2m} + 1/2, …, −1/2))  (6.8.4) {eq:6.8.4}
⟶ X(… (−x_{2m−1} + 1/2, …, −1/2)(−x_{2m} + 1/2, …, −1/2))

(again all other entries unchanged). This is I_{M,2m−1} defined in (6.1.8), so its restriction of (6.8.4) to the σ_{e} isotypic component is an isomorphism. Now compose this operator with the one corresponding to

X(… (1/2, …, x_{2m−2}−1/2)(−x_{2m−1} + 1/2, …, 1/2) …)  (6.8.5) {eq:6.8.5}
⟶ X(… (−x_{2m−1}+1/2, …, x_{2m−2} − 1/2) …)

with all other entries unchanged. This is induced from

GL(x_{0}) × ⋯ × GL(x_{2m−3}) × GL(x_{2m−2} + x_{2m−1}) × G(x_{2m})

and the image is the representation induced from the character corresponding to the string

(−x_{2m−1} − 1/2, …, −1/2, 1/2, …, x_{2m−2}) on GL(x_{2m−2} + x_{2m−1}).

Now compose further with the intertwining operator

X(… (−x_{2m−1} + 1/2, …, x_{2m−2} − 1/2)(−x_{2m} − 1/2, …, −1/2)) (6.8.6) {eq:6.8.6}
⟶ X((−x_{2m−1} + 1/2, …, x_{2m−2} − 1/2) … (−x_{2m} − 1/2, …, −1/2))

from the representation induced from

GL(x_{0}) × ⋯ × GL(x_{2m−3}) × GL(x_{2m−2} + x_{2m−1}) × G(x_{2m})

to the induced from

GL(x_{2m−2} + x_{2m−1}) × GL(x_{0}) × ⋯ × GL(x_{2m−3}) × G(x_{2m}).

By lemma 6.2, this intertwining operator is an isomorphism on any σ_{e} isotypic component. In fact, because the strings are strongly nested, the irreducibility results for GL(n), 3.3 imply that the induced modules are isomorphic.

We have constructed a composition of intertwining operators from the standard module X(ν) where the coordinates of ν are positive and in decreasing order (i.e. dominant) to a module induced from

GL(x_{2m−2} + x_{2m−1}) × GL(x_{0}) × ⋯ × GL(x_{2m−3}) × G(x_{2m})

corresponding to the strings

((−x_{2m−1} + 1/2, …, x_{2m−2} − 1/2)(1/2, …, x_{0} − 1/2), …

…, (1/2, …, x_{2m−3} − 1/2)(−x_{2m} + 1/2, …, −1/2))

so that the restriction to any σ_{e} isotypic component is onto. We can repeat the procedure with x_{2m−4}, x_{2m−3} and so on to get an intertwining operator from X(ν) to the induced from

GL(x_{2m−1} + x_{2m−2}) × ⋯ × GL(x_{1} + x_{0}) × G(x_{2m})

corresponding to the strings

((−x_{2m−1} + 1/2, …, x_{2m−2} − 1/2) … (−x_{1} + 1/2, …, x_{0} − 1/2),
(−x_{2m} + 1/2, …, −1/2)).

This is the operator I_{1}, and it is onto on the σ_{e}(∗) isotypic components.

Proof that I_{2} is into. We now deal with I_{2}. Consider the group G(x_{1} + x_{0} + x_{2m}) and the Levi component M = GL(x_{1} + x_{0}) × G(x_{2m}). Let M′ be the Levi component

M′ := GL(x_{2m−1} +x_{2m−2})×⋯×GL(x_{3} +x_{2})×GL(x_{1})×GL(x_{0})×G(x_{2m}).

(6.8.7) {eq:6.8.7} Then X_{e} embeds in X_{M′}(… (−x_{1}+1/2, ⋯−1/2)(1/2, …, x_{0}−1/2)(−x_{2m}+ 1/2, …, −1/2)). The intertwining operator I_{M′,m+1} which changes the string (x_{0} − 1/2, …, 1/2) to (−x_{0} + 1/2, …, −1/2) is an isomorphism on the σ_{e} W −types, by the results in sections 6.1-6.5. Since the strings are strongly nested, the operators I_{M,i,i+1} are all isomorphisms, so we can construct an intertwining operator to an induced module X_{M′′}(ν′′) where

M′′ = GL(x_{1}) × GL(x_{0}) × GL(x_{2m−1} + x_{2m−2}) × ⋯ × G(x_{2m}),

ν′′ = (−x_{1} + 1/2, ⋯ − 1/2)(−x_{0} + 1/2, …, −1/2) …) {(6.8.8) \atop \{eq:6.8.8\}}

which is an isomorphism on the σ_{e} isotypic components. Repeating this argument for x_{3}, x_{2} up to x_{2m−1}, x_{2m−2} we get an intertwining operator from X_{e} to an induced module X_{M^{(3)}}(ν^{(3)}) where

M^{(3)} := GL(x_{1}) × GL(x_{0}) × ⋯ × GL(x_{2m−1}) × GL(x_{2m−2}) × G(x_{2m})

ν^{(3)} := (−x_{1} + 1/2, …, 1/2)(−x_{0} + 1/2, …, −1/2) … (−x_{2m−1} + 1/2, …, −1/2)(−x_{2m−2} + 1/2, …, −1/2)(−x_{2m} + 1/2, …, −1/2).

(6.8.9) {eq:6.8.9} which is an isomorphism on the σ_{e} isotypic components. Let

M^{(4)} := GL(x_{1}) × GL(x_{0}) × ⋯ × GL(x_{2m−1}) × G(x_{2m})

and let ν^{(4)} be the same as ν^{(3)} but the last string is viewed as giving a parameter of a 1-dimensional representation on GL(x_{2m}). Then M^{(4)} ⊂ M^{(3)}, and X_{M^{(3)}}(ν^{(3)}) is a submodule of X_{M^{(4)}}(ν^{(4)}). The induced module from M^{(4)} to M^{(5)} := GL(x_{2m} + ⋯ + x_{0}) is irreducible because the strings are strongly nested on the GL factors. Thus the intertwining operator which takes ν^{(4)} to −ν is an isomorphism on X_{M^{(4)}}(ν^{(4)}). So the induced intertwining operator to G is therefore injective and maps to X(−ν). The composition of all these operators is I_{1}, and is therefore injective on the σ_{e}-isotypic components. The proof is complete in this case.

The case of σ_{o} is similar, and we omit the details.

7. Necessary conditions for unitarity

                                                                       {sec:7}

                                                                       {sec:7.1}

7.1. We will need the following notions.

Definition. We will say a spherical irreducible module L(χ) is r-unitary if the form is positive on all the relevant W −types. Similarly, an induced module X(χ) is r-irreducible if all relevant W −types occur with the same multiplicity in X(χ) as in L(χ).

                                                                       {sec:7.2}

7.2. We recall (6.1.4),

ϵ = \begin{cases}1/2 & G of type B, (ℍ of type C) \\ 0 & G of type C, (ℍ of type B) \\ 1 & G of type D. \end{cases}

                                                                       {d:7.2}

Definition. A string of the form (f + ν, …, F + ν) with f, F ∈ ϵ + ℤ is called adapted, if it is

of even length for G of type B, of odd length for G of types C,D.

Otherwise we say the string is not adapted.

We will consider the following case. Let Ǒ ⊂ 𝔤̌ correspond to the partition

Ǒ ⟷ ((a_{1}, a_{1}), … (a_{r}, a_{r}); d_{1}, …, d_{l})  (7.2.1) {eq:7.2.1}

so that Ǒ meets the Levi component 𝔪̌ = gl(a_{1}) × ⋯ × gl(a_{r}) × 𝔤̌(n_{0}), with 2n_{0}+[1 − ϵ] = d_{1}+⋯+d_{l}, where [x] is the integer part of x. The intersection of Ǒ with each gl(a_{i}) is the principal nilpotent, and the intersection with 𝔤̌(n_{0}) is the even nilpotent orbit Ǒ_{0} with partition (d_{1}, …, d_{l}). Let

χ_{i} = (f_{i} + ν_{i}, …, F_{i} + ν_{i}), 1 ≤ i ≤ r,
χ_{0} = ȟ_{0}/2, where ȟ_{0} is a neutral element for (d_{1}, …, d_{l}).

(7.2.2) {eq:7.2.2}

and χ be the parameter obtained by concatenating the χ_{i}. Then L(χ) is the spherical subquotient of

Ind_{M}^{G}[ ⨂_{1≤i≤r} L(χ_{i}) ⊗ L(χ_{0})] {(7.2.3) \atop \{eq:7.2.3\}}

The next theorem gives necessary conditions for the unitarity of L(χ).

{t:7.2}

Theorem. In types B,C, the nilpotent orbit Ǒ_{0} is arbitrary. In type D assume that either Ǒ_{0} ≠ (0), or else that the rank is even. The representation L(χ) is unitary only if

(1) Any string that is not adapted can be written in the form

(−E + τ, …, E − 1 + τ) 0 < τ ≤ 1/2, E ≡ ϵ(mod ℤ). {(7.2.4) \atop \{eq:7.2.4\}}

(2) Any string that is adapted can be written in the form

\begin{matrix}(−E + τ, …, E + τ)  0 < τ ≤ 1/2, E ≡ ϵ(mod ℤ), \\ or \\ (−E − 1 + τ, …, E − 1 + τ)  0 < τ ≤ 1/2, E ≡ ϵ(mod ℤ). \end{matrix} {(7.2.5) \atop \{eq:7.2.5\}}

This is simply the fact that the ν_{j} satisfy 0 < ν_{j} < 1/2 or 1/2 < ν_{j} < 1 in theorem 3.1. The proof will be given in the next sections. It is by induction on the dimension of 𝔤̌, the number of strings with coordinates in an A_{τ} (definition after (3.3.6)) with τ ≠ 0, and by downward induction on the dimension of Ǒ. The unitarity of the representation when there are no coordinates in any A_{τ} with τ ≠ 0 is done in section 9.

{sec:7.3}

7.3. The proposition in this section is a restatement of theorem 7.2 for the case of a parameter of the form (7.3.1). It is the first case in the initial step of the induction proof of theorem 7.2. I have combined the two cases in (2) into a single string (−E + ν, …, E + ν) with 0 < ν < 1 by changing (−E − 1 + τ, …, E − 1 + τ) into (−E + (1 − τ), …, E + (1 − τ)). This notation seemed more convenient for the case when there is a single such string present.

Consider the representation L(χ) corresponding to the strings

(a+ϵ+ν, …, A+ϵ+ν)(−x_{0} +ϵ, …, −1+ϵ), |a| ≤ A, 0 < ν < 1, {(7.3.1) \atop \{eq:7.3.1\}}

where a, A ∈ ℤ, and ϵ is as in 7.2. This is the case when L(χ) is the spherical subquotient of an induced from a character on a maximal parabolic subalgebra of the form gl(A − a + 1) × 𝔤(x_{0}). The second string may not be present; these are the cases x_{0} = −1 for 𝔤 of type B, D, x_{0} = 0 for type C.

{p:7.3}

Proposition. In type D, assume that if there is no string (−x_{0}+ϵ, …, −1+ ϵ), then A − a + 1 is even. Let L(χ) correspond to (7.3.1). Then L(χ) is r-unitary if and only if a + ϵ = −A − ϵ, and the following hold.

(1) Assume that (a + ϵ + ν, …, A + ϵ + ν) is adapted. If x_{0} = A − a + 1, then 0 ≤ ν < 1, otherwise ν = 0.

(2) If (a + ϵ + ν, …, A + ϵ + ν) is not adapted, then 0 ≤ ν < 1/2.

Proof. This is a corollary of the formulas in section 6.2. □

                                                                       {sec:7.4}

7.4. Initial Step. We do the case when there is a single A_{τ} with 0 < τ < 1/2, and the coordinates form a single string. We write the string as in (7.3.1), (a + ϵ + ν, …, A + ϵ + ν) with 0 < ν < 1. We let Ǒ be the nilpotent orbit with partition ((A − a + 1, A − a + 1); d_{1}, …, d_{l}). Let 𝔪 := gl(A−a+1)×𝔤̌(n_{0}), and let Ǒ_{0} be the intersection of Ǒ with 𝔤̌(n_{0}). In type D, either Ǒ_{0} ≠ (0) or else A − a + 1 is even. The statement of theorem 7.2 is equivalent to the following proposition.

                                                                       {p:7.4}

Proposition. Assume Ǒ_{0} is even, and χ is attached to Ǒ. Then L(χ) is r-unitary only if a + ϵ = −A − ϵ, and the following hold.

(1) If (a + ϵ + ν, …, A + ϵ + ν) is adapted, then ν = 0, unless there is d_{j} = A − a + 1, in which case 0 ≤ ν < 1.

(2) If (a + ϵ + ν, …, A + ϵ + ν) is not adapted, then 0 ≤ ν < 1/2.

Proof. We do the case of G of type C only, the others are similar. So ϵ = 0, and adapted means the length of the string is odd, not adapted means the length of the string is even. The nilpotent orbit Ǒ_{0} corresponds to the partition (2x_{0} + 1, …, 2x_{2m} + 1) and the parameter has strings

(1, … x_{0})(0, 1, …, x_{1}) … (1, …, x_{2m}).

The partition of Ǒ is (A − a + 1, A − a + 1, 2x_{0} + 1, …, 2x_{2m} + 1).

We want to show that if A + a > 0, or if A + a = 0 and there is no x_{i} = A, then L(χ) is not r-unitary. We do an upward induction on the rank of 𝔤̌ and a downward induction on the dimension of Ǒ. In the argument below with deformations of strings, we use implicitly the irreducibility results from section 2.6. So the first case is when Ǒ is maximal, i.e. the principal nilpotent (m = 0). The claim follows from proposition 7.3. So we assume that m is strictly greater than 0. Assume x_{2i} < A ≤ x_{2i+1} for some i. This case includes the possibility x_{2m} < A. We will show by induction on rank of 𝔤̌ and dimension of Ǒ that the form is negative on a W -type of the form σ[(n − r), (r)]. So we use the module X_{e} (notation as in 5.3). If there is any pair x_{2j} = x_{2j+1}, the module X_{e} is unitarily induced from GL(2x_{2j} + 1) × G(n − 2x_{2j} − 1) and all W −types σ[(n−r), (r)] have the same multiplicity in L(χ) as in X_{e}. We can remove the string corresponding to (x_{2j}x_{2j+1}) in X_{e} as explained in section 3.2, lemma (3). By induction on rank we are done. Similarly we can remove any pair (x_{2j}, x_{2j+1}) such that either x_{2j+1} ≤ |a| or A ≤ x_{2j} as follows. Let M := GL(x_{2j} + x_{2j+1} + 1) × G(n − x_{2j} − x_{2j+1} − 1). There is χ_{M} such that L(χ) is the spherical subquotient of

Ind_{M}^{G}[L(−x_{2j+1}, …, x_{2j}) ⊗ L(χ_{M})]. {(7.4.1) \atop \{eq:7.4.1\}}

Precisely, χ_{M} is obtained from χ by removing the entries (1, …, x_{2j}), (0, 1, …, x_{2j+1}). Write

χ_{t} := (−x_{2j+1} + t, …, x_{2j} + t; χ_{M}). {(7.4.2) \atop \{eq:7.4.2\}}

The induced module

X_{e}(χ_{t}) := Ind_{M}^{G}[L(−x_{2j+1} + t, …, x_{2j} + t) ⊗ L(χ_{M})]. {(7.4.3) \atop \{eq:7.4.3\}}

has L(χ_{t}) as its irreducible spherical subquotient. For 0 ≤ t ≤ \frac{x_{2j+1}−x_{2j}}{2} , the multiplicities of σ[(n − r, r)] in L(χ_{t}) and X_{e}(χ_{t}) coincide. Thus the signatures on the σ[(n − r), (r)] in L(χ_{t}) are constant for t in the above interval. At t = \frac{x_{2j+1}−x_{2j}}{2} , X_{e}(χ_{t}) is unitarily induced from triv ⊗ X′_{e} on GL(x_{2j} + x_{2j+1} + 1) × G(n − x_{2j} − x_{2j+1} − 1) and we can remove the string corresponding to (x_{2j}x_{2j+1}). The induction hypothesis applies to X′_{e}.

When A + a = 0, by the above argument, we are reduced to the case

Ǒ_{0} ⟷ (2x_{0} + 1, 2x_{1} + 1, 2x_{2} + 1), x_{0} < A < x_{1} ≤ x_{2}. {(7.4.4) \atop \{eq:7.4.4\}}

We reduce to (7.4.4) when A + a > 0 as well. We assume 2m = 2i + 2, since pairs (x_{2j}, x_{2j+1}) with A ≤ x_{2j} can be removed. Suppose there is a pair (x_{2j}, x_{2j+1}) such that |a| < x_{2j+1}, and j ≠ i. The assumption is that x_{2i} < A ≤ x_{2i+1} so x_{2j+1} ≤ x_{2i} < A.

We consider the deformation χ_{t} in (7.4.2) with

0 ≤ t < ν,  a < 0,
−ν < t ≤ 0,  a ≥ 0.

In either case X_{e}(χ_{t}) = X_{e}(χ), so the multiplicities of the σ[(n − r), (r)] do not change until t reaches ν in the first case, −ν in the second case. If the signature on some σ[(n − r), (r)] isotypic component is positive semidefinite on L(χ), the same has to hold when t = ν or −ν respectively. The corresponding nilpotent orbit for this parameter is strictly larger, but it has two strings with coordinates which are not integers (so the induction hypothesis does not apply yet). For example, if a < 0, the strings for X_{e}(χ_{ν}) are (aside from the ones that were unchanged)

(−x_{2j+1} + ν, …, A + ν), (a + ν, …, x_{2j} + ν). {(7.4.5) \atop \{eq:7.4.5\}}

We can deform the parameter further by replacing the second string by (a + ν − t′, …, x_{2j} + ν − t′) with 0 ≤ t′ < ν. The strings of the corresponding X_{e} do not change until t′ reaches ν. At t′ = ν the corresponding nilpotent orbit Ǒ′ has partition

(…, 2|a| + 1, …, { \atop 2x_{2j+1} + 1}, …, A + x_{2j+1} + 1, A + x_{2j+1} + 1, …) {(7.4.6) \atop \{eq:7.4.6\}}

which contains Ǒ in its closure. Since x_{2j+1} < A, the induction hypothesis applies. The form is indefinite on a W −type σ[(n − r), (r)], so this holds for the original χ as well.

We have reduced to case (7.4.4), i.e. the partition of Ǒ_{0} has just three terms (2x_{0} + 1, 2x_{1} + 1, 2x_{2} + 1). We now reduce further to the case

Ǒ_{0} ⟷ (2x_{0} + 1),  x_{0} < A.  (7.4.7) {eq:7.4.7}

which is the initial step.

Let I(t) be the induced module coresponding to the strings

(−x_{2} + t, …, x_{1} + t)(a + ν, …, A + ν)(−x_{0}, …, −1).  (7.4.8) {eq:7.4.8}

i.e. induced from

GL(x_{1} + x_{2}) × GL(−a + A + 1) × G(x_{0}).  (7.4.9) {eq:7.4.9}

Consider the irreducible spherical module for the last two strings in (7.4.8), inside the induced module from the Levi component GL(−a + A + 1) × G(x_{0}) ⊂ G(−a + A + 1 + x_{0}). By section 7.1, the form is negative on σ[(x_{0} − a+A), (1)] if x_{0} < a, negative on σ[(A), (x_{0} +1−a)] if a ≤ x_{0}. In the second case the form is positive on all σ[(A+r), (x_{0}+1−a−r)] for 1 < r < x_{0}+1−a. So let r_{0} := 1 or x_{0} + 1 − a depending on these two cases. The multiplicity formulas from section 6.2 imply that

[σ[(n−r_{0}), (r_{0})] : I(t)] = [σ[(n−r_{0}), (r_{0})] : L(χ)] for 0 ≤ t ≤ \frac{x_{2} − x_{1}}{2} .

Thus signatures do not change when we deform t to \frac{x_{2}−x_{1}}{2} , where I(t) is unitarily induced. We conclude that the form on L(χ) is negative on σ[(n − r_{0}), (r_{0})]. Assume x_{2i−1} < A ≤ x_{2i}. In this case we can do the same arguments using X_{o} and σ[(k, n − k), (0)]. We omit the details. □

                                                                       {7.5}

7.5. Induction step. The case when the parameter has a single string with coordinates in an A_{τ} with 0 < τ < 1/2 was done in section 7.4. So we assume there is more than one string. Again we do the case G of type C, and omit the details for the other ones.

Write the two strings as in (2.6),

(e + τ_{1}, …, E + τ_{1}),  (f + τ_{2}, …, F + τ_{2}).  (7.5.1) {eq:7.5.1}

where 0 < τ_{1} ≤ 1/2 and 0 < τ_{2} ≤ 1/2. Recall that because we are in type C, e, E, f, F ∈ ℤ, and ϵ = 0.

We need to show that if F + f > 0 or F + f < −2 when F + f is even, or F + f < −1 when F + f is odd, then the form is negative on a relevant W −type. Because τ_{1}, τ_{2} > 0, and since r-reducibility and runitarity are not affected by small deformations, we may as well assume that (f + τ_{2}, …, F + τ_{2}) is the only string with coordinates in A_{τ_{2}}, and (e + τ_{1}, …, E + τ_{1}) the only one with coordinates in A_{τ_{1}}.

The strategy is as follows. Assume that L(χ) is r-unitary. We deform (one of the strings of) χ to a χ_{t} in such a way that the coresponding induced module is r-irreducible over a finite interval, but is no longer so at the endpoint, say t_{0}. Because of the continuity in t, the module L(χ_{t_{0}}) is still r-unitary. The deformation is such that L(χ_{t_{0}}) belongs to a larger nilpotent orbit than L(χ), so the induction hypothesis applies, and we get a contradiction. Sometimes we have to repeat the procedure before we arrive at a contradiction.

So replace the first string by

(e + τ_{1} + t, …, E + τ_{1} + t). {(7.5.2) \atop \{eq:7.5.2\}}

If χ = (e + τ_{1}, … E + τ_{1}; χ_{M}), then

χ_{t} = (e + τ_{1} + t, …, E + τ_{1} + t; χ_{M}),
X(χ_{t}) := Ind_{M}^{G}[L(e + τ_{1} + t, …, E + τ_{1} + t) ⊗ L(χ_{M})],

where 𝔪 = gl(E − e + 1) × 𝔤(n − E + e − 1),

If E < |e|, we deform t in the negative direction, otherwise in the positive direction. If t + τ_{1} reaches 0 or 1/2, before the nilpotent orbit changes, we should rewrite the string to conform to the conventions (2.6.10) and (2.6.11). This means that we rewrite the string as (e′ + τ′_{1}, …, E′ + τ′_{1}) with 0 ≤ τ′_{1} ≤ 1/2, and continue the deformation with a t going in the direction t < 0 if E′ < |e′|, and t > 0 if E′ ≥ |e′|. This is not essential for the argument. We may as well assume that the following cases occur.

(1) The nilpotent orbit changes at t_{0} = −τ_{1}.

(2) the nilpotent orbit does not change, and at t_{0} = −τ_{1}, either e, E > x_{2m} + 1 or −e, −E > x_{2m} + 1. This is the easy case when t can be deformed to ∞ without any r-reducibility occuring.

(3) The nilpotent orbit changes at a t_{0} such that 0 < τ_{1} + t_{0} ≤ 1/2.

In the first case, the induction hypothesis applies, and since the string (f + τ_{2}, …, F + τ_{2}) is unaffected, we conclude that the signature is negative on a relevant W −type. In the second case we can deform the string so that either e + τ_{1} + t = x_{2m} + 1 or E + τ_{1} + t = −x_{2m} − 1. The induction hypothesis applies, and the form is negative definite on a W -relevant type. In the third case, the only way the nilpotent orbit can change is if the string (e + τ_{1} + t_{0}, …, E + τ_{1} + t_{0}) can be combined with another string to form a strictly longer string. If τ_{1} + t_{0} ≠ τ_{2}, the induction hypothesis applies, and since the string (f + τ_{2}, …, F + τ_{2}) is unaffected, the form is negative on a relevant W −type. If the nilpotent does not change at t = τ_{2} − τ_{1}, continue the deformation in the same direction. Eventually either (1) or (2) are satisfied, or else we are in case (3), and the strings in (7.5.1) combine to

give a larger nilpotent. There are four cases:

\begin{matrix}(1) e < f ≤ E ≤ F, e ≤ f ≤ E < F \\ (2) f ≤ e ≤ F < E, f < e ≤ F ≤ E \\ (3) e ≤ E = f − 1 < F, \\ (4) f ≤ F = e − 1 < E. \end{matrix} {(7.5.3) \atop \{eq:7.5.3\}}

Assume |e| ≤ E. Then t is deformed in the positive direction so τ_{1} < τ_{2}. If e ≤ 0, we look at the deformation (7.5.2) for −τ_{1} ≤ t ≤ 0. If the nilpotent changes for some −τ_{1} < t < 0, the string (f +τ_{2}, …, F +τ_{2}) is not involved, the induction hypothesis applies, so the parameter is not r-unitary. Otherwise at t = −τ_{1} there is one less string with coordinates in an A_{τ} with τ ≠ 0, and again the induction hypothesis applies so the original parameter is not r-unitary. Thus we are reduced to the case 0 < e < E. Then consider the nilpotent orbit for the parameter with t = −τ_{1} + τ_{2}. In cases (1), (2) and (3) of (7.5.3), the new nilpotent is larger, and one of the strings is

(e + τ_{2}, …, F + τ_{2}),  (7.5.4) {eq:7.5.4}

instead of (7.5.1), and e + F > 0. The induction hypothesis applies, so the parameter is not r-unitary, nor is the original one.

In case (4) of (7.5.3), the new nilpotent corresponds to the strings

(f + τ_{2}, …, E + τ_{2})  (7.5.5) {eq:7.5.7}

The induction hypothesis applies, so f + E = 0, −2 if f + E is even or f + E = −1 if it is odd. If this is the case, consider a new deformation in 7.5.2, this time −1 + τ_{2} < t ≤ 0. We may as well assume that the parameter is r-irreducible in this interval, or else the argument from before gives the desired conclusion. So we arrive at the case when t = −1 + τ_{2}. The new nilpotent corresponds to the strings

(f + τ_{2}, …, E − 1 + τ_{2}), (e − 1 + τ_{1}), (F + τ_{2}).  (7.5.6) {eq:7.5.8}

Write the parameter as (χ′; e − 1 + τ_{2}, F + τ_{2}). Since e − 1 = F, the induced module

I = Ind_{GL(2)×G(n−2)}^{G}[L(e − 1 + τ_{2}, F + τ_{2}) ⊗ L(χ′)]  (7.5.7) {eq:7.5.9}

is unitarily induced from a module which is hermitian and r-irreducible. But the parameter on GL(2) is not unitary unless e − 1 = F = 0. Furthermore f + E − 1 = 0, −2 if f + E is odd or f + E − 1 = −1 if f + E is even. So the original parameter (7.5.1) is

\begin{matrix}(1 + τ_{1}, …, E + τ_{1}), (1 − E + τ_{2}, …, τ_{2}) f + E = 0, \\ (1 + τ_{1}, …, E + τ_{1}), (−E + τ_{2}, …, τ_{2}) f + E = −2, \\ (1 + τ_{1}, …, E + τ_{1}), (−E − 1 + τ_{2}, …, τ_{2}) f + E = −1. \end{matrix}  (7.5.8) \{eq:7.5.10\}

Apply the deformation t + τ_{2} in the second string with −τ_{2} < t ≤ 0. We may as well assume that the parameter stays r-irreducible in this interval. But then the induction hypothesis applies at t = −τ_{2} because there is one less string with coordinates in A_{τ} with τ ≠ 0. However the first string does no satisfy the induction hypothesis. Assume |e| > E. The same argument applies, but this time it is e < E < 0 that requires extra arguments, and in case (3) instead of case (4) of (7.5.3) we have to consider several deformations.

{sec:7.6}

7.6. Proof of necessary condition for unitarity in theorem 3.1. We first reduce to the case of theorem 7.2. The difference is that the coordinates in A_{0} may not form a ȟ/2 for an even nilpotent orbit. However because of theorem 2.9, and properties of petite K−types, r-reducibility and r-unitary are unaffected by small deformations of the χ′_{1}, …, χ′_{r} (notation as in (2.9.3)). So we can deform the strings corresponding to χ′_{1}, …, χ′_{r} with coordinates in A_{0}, so that their coordinates are no longer in A_{0}. Then the assumptions in theorem 7.2 are satisfied.

The argument now proceeds by analyzing each size of strings separately. In the deformations that we will consider, strings of different sizes cannot combine so that the nilpotent orbit attached to the parameter changes.

Fix a size of strings with coordinates not in A_{0}. If the strings are not adapted, they can be written in the form

(−E − 1 + τ_{i}, …, E + τ_{i}) 0 < τ_{i} ≤ 1/2, E ≡ ϵ(mod ℤ). {(7.6.1) \atop \{eq:7.6.1\}}

So there is nothing to prove. Now consider a size of strings that are adapted. Suppose there are two strings of the form

(−E − 1 + τ_{i}, …, E − 1 + τ_{i}), 0 < τ_{i} ≤ 1/2, E ≡ ϵ(mod ℤ). {(7.6.2) \atop \{eq:7.6.2\}}

Let 𝔪 := gl(2E + 1) × 𝔤(n − 2E − 1), (𝔤(a) means a subalgebra/Levi component of the same type as 𝔤 of rank a) and write

χ := ((−E − 1 + τ_{i}, …, E − 1 + τ_{i}; −E − 1 + τ_{i}, …, E − 1 + τ_{i}); χ_{M}). {(7.6.3) \atop \{eq:7.6.3\}}

The module

Ind_{M}^{G}[L(−E − 1 + τ_{i}, …, E − 1 + τ_{i}; −E − 1 + τ_{i}, …, E − 1 + τ_{i}) ⊗ L(χ_{M})]

(7.6.4) {eq:7.6.4} is r-irreducible, and unitarily induced from a hermitian module on M where the module on GL(2E +1) is not unitary. Thus L(χ) is not unitary either. So L(χ) is unitary only if for each τ_{i} there is at most one string of the form (−E − 1 + τ_{i}, …, E − 1 + τ_{i}).

Suppose there are two strings as in (7.6.1) with τ_{1} < τ_{2}. If there is no string (−E + τ_{3}, …, E + τ_{3}) with τ_{1} < τ_{3} < τ_{2}, then when we deform (−E − 1 + τ_{1} + t, …, E − 1 + τ_{1} + t) for 0 ≤ t ≤ τ_{2} − τ_{1}, X(χ_{t}) stays rirreducible. At t = τ_{2} − τ_{1} we are in case (7.6.2), so the parameter is not unitary.

On the other hand suppose that there are two strings of the form

(−E + τ_{i}, …, E + τ_{i}), same τ_{i}. {(7.6.5) \atop \{eq:7.6.5\}}

Let 𝔪 be as before, and write

χ := ((−E + τ_{i}, …, E + τ_{i}; −E + τ_{i}, …, E + τ_{i}); χ_{M}). {(7.6.6) \atop \{eq:7.6.6\}}

The module

Ind_{M}^{G}[L(−E + τ_{i}, …, E + τ_{i}; −E + τ_{i}, …, E + τ_{i}) ⊗ L(χ_{M})] {(7.6.7) \atop \{eq:7.6.7\}}

is irreducible, and unitarily induced from a hermitian module on M where the module on GL(2E + 1) is unitary. Thus L(χ) is unitary if and only if L(χ_{M}) is unitary.

So we may assume that for each τ_{i} there is at most one string of the form (−E + τ_{i}, …, E + τ_{i}).

Similarly if there are two strings of the form (−E + τ_{1}, …, E + τ_{1}) and (−E + τ_{2}, …, E + τ_{2}), such that there is no string of the form (−E − 1 + τ_{3}, …, E − 1 + τ_{3}) with τ_{1} < τ_{3} < τ_{2} we reduce to the case (7.6.5).

Let τ_{k} be the largest such that a string of the form (−E + τ_{k}, …, E + τ_{k}) occurs, and τ_{k+1} the smallest such that a string (−E − 1 + τ_{k+1}, …, E − 1 + τ_{k+1}) occurs. If τ_{k} > τ_{k+1}, we can deform (−E + τ_{k} + t, …, E + τ_{k} + t) with 0 ≤ t ≤ 1 − τ_{k} − τ_{k+1}. No r-reducibility occurs, and we are again in case (7.6.2). The module is not unitary. If on the other hand τ_{k} < τ_{k+1}, the deformation (−E −1+τ_{k+1} +t, …, E −1+τ_{k+1} −t) for 0 ≤ t ≤ 1−τ_{k} −τ_{k+1} brings us to the case (7.6.5).

Together the above arguments show that conditions (1) and (2) of theorem 3.1 in types C,D must be satisfied. Remains to check that for the case of adapted strings, if there is an odd number of a given size 2E + 1, then there is a d_{j} = 2E + 1. This is condition (3) in theorem 3.1.

The arguments above (also the unitarity proof in the case Ǒ = (0)) show that an L(χ) is unitary only if it is of the following form. There is a Levi component 𝔪 = gl(a_{1})×⋯×gl(a_{r})×𝔤(n−∑ a_{i}), and parameters χ_{1}, …, χ_{r}, χ_{0} such that,

L(χ) = Ind_{M}^{G}[⨂ L(χ_{i}) ⊗ L(χ_{0})],  (7.6.8) {eq:7.6.8}

with the following additional properties:

(1) The χ_{i} for i > 0 are as in lemma (1) of section 3.2, with 0 < ν < 1/2, in particular unitary.

(2) χ_{0} is such that there is at most one string for every A_{τ} with τ ≠ 0,

and the strings are of different sizes. In addition, conditions (1) and (2) of theorem 3.1 are satisfied for the strings. To complete the proof we therefore only need to consider the case of L(χ_{0}). We can deform the parameters of the strings in the A_{τ} with τ ≠ 0 to zero without r-reducibility occuring. If L(χ) is unitary, then so is the parameter where we deform all but one τ ≠ 0 to zero. But for a parameter with a single string belonging to an A_{τ} with τ ≠ 0, the necessary conditions for unitarity are given in section 7.4.

8. Real nilpotent orbits

                                                                       {sec:8}

In this section we review some well known results for real nilpotent orbits. Some additional details and references can be found in [CM].

                                                                       {sec:8.1}

8.1. Fix a real form 𝔤 of a complex semisimple Lie algebra 𝔤_{c}. Let θ_{c} be the complexification of the Cartan involution θ of 𝔤, and write for the conjugation. Let G be the adjoint group with Lie algebra 𝔤_{c}, and let

𝔤_{c} = 𝔨_{c} + 𝔰_{c}, 𝔤 = 𝔨 + 𝔰 {(8.1.1) \atop \{eq:8.1.1\}}

be the Cartan decomposition. Write K_{c} ⊂ G_{c} for the subgroup corresponding to 𝔨_{c}, and G and K for the real Lie groups corresponding to 𝔤 and 𝔨.

{t:8.1}

Theorem (Jacobson-Morozov).

(1) There is a one to one correspondence between G_{c}-orbits O_{c} ⊂ 𝔤_{c} of nilpotent elements and G_{c}-orbits of Lie triples {e, h, f} i.e. elements satisfying

[h, e] = 2e,  [h, f] = −2f,  [e, f] = h.

This correspondence is realized by completing a nilpotent element e ∈ O to a Lie triple.

(2) Two Lie triples {e, h, f} and {e′, h′, f′} are conjugate if and only

if the elements h and h′ are conjugate.

{sec:8.2}

8.2. Suppose e ∈ 𝔤 is nilpotent. Then one can still complete it to a Lie triple e, h, f ∈ 𝔤. Such a Lie triple is called real or ρ−stable. A Lie triple is called Cayley if in addition θ(h) = −h, θ(e) = f. Every real Lie triple is conjugate by G to one which is Cayley.

{t:8.2}

Theorem (Kostant-Rao). Two real Lie triples are conjugate if and only if the elements e − f and e′ − f′ are conjugate under G. Equivalently, two Cayley triples are conjugate if and only if e − f and e′ − f′ are conjugate under K.

{sec:8.3}

8.3. Suppose e ∈ 𝔰_{c} is nilpotent. Then e can be completed to a Lie triple satisfying

θ_{c}(e) = −e, θ_{c}(h) = h, θ_{c}(f) = −f. {(8.3.1) \atop \{eq:8.3.1\}}

We call such a triple θ-stable. To any Cayley triple one can associate a θ-stable triple as in (8.3.1), by the formulas

ẽ:= \frac{1}{2}(e + f + ih), h̃ := i(e − f), f̃:= \frac{1}{2}(e + f − ih). {(8.3.2) \atop \{eq:8.3.2\}}

A Lie triple is called normal if in addition to (8.3.1) it satisfies \overline{e} = f, \overline{h} =

−h. {t:8.3}

Theorem (Kostant-Sekiguchi).

(1) Any θ-stable triple is conjugate via K_{c} to a normal one.

(2) Two nilpotent elements ẽ, e′ ∈ 𝔰 are conjugate by K_{c}, if and only if the corresponding Lie triples are conjugate by K_{c}. Two θ-stable triples are conjugate under K if and only if the elements h̃, h̃′ are conjugate under K_{c}.

(3) The correspondence (8.3.2) is a bijection between G orbits of nilpo-

tent elements in 𝔤 and K_{c} orbits of nilpotent elements in 𝔰_{c}.

{t:8.3.1}

Proposition. The correspondence between real and θ stable orbits preserves closure relations.

Proof. This is the main result in [BS]. □

                                                                       {sec:8.4}

8.4. Let 𝔭_{c} = 𝔪_{c} + 𝔫_{c} be a parabolic subalgebra of 𝔤_{c}. Let 𝔠_{c} := Ad M_{c} · e be the orbit of a nilpotent element e ∈ 𝔪_{c}. According to [LS], the induced orbit from 𝔠_{c} is the unique G_{c} orbit ℭ_{c} which has the property that ℭ_{c} ∩ [𝔠_{c} + 𝔫_{c}] is dense (and open) in 𝔠 + 𝔫_{c}.

Proposition (1). Let E = e + n ∈ e + 𝔫_{c}.

(1) dim Z_{M_{c}}(e) = dim Z_{G_{c}}(E).

(2) ℭ_{c} ∩ [𝔠_{c} + 𝔫_{c}] is a single P_{c} orbit.

This is theorem 1.3 in [LS]. In particular, an element E′ = e′ +n′ ∈ 𝔠_{c} +𝔫_{c} is in ℭ_{c} if and only if the map

ad E′ : 𝔭_{c} ⟶ T_{e′}𝔠 + 𝔫_{c},  ad E′(y) = [E′, y]  (8.4.1) {eq:8.4.1}

is onto.

Another characterization of the induced orbit is the following.

Proposition. The orbit ℭ_{c} is the unique open orbit in Ad G_{c}(e + 𝔫_{c}) = Ad G_{c}(𝔠_{c} + 𝔫_{c}), as well as in the closure \overline{Ad G_{c}(e + 𝔫_{c})} = Ad G_{c}(\overline{𝔠_{c}} + 𝔫_{c}).

We omit the proof, but note that the statements about the closures follow from the fact that G_{c}/P_{c} is compact.

Proposition (2). The orbit ℭ_{c} depends on 𝔠_{c} ⊂ 𝔪_{c}, but not on 𝔫_{c}.

Proof. This is proved in section 2 of [LS]. We give a different proof which generalizes to the real case. Let ξ ∈ 𝔥_{c} ⊂ 𝔪_{c} be an element in the center of 𝔪_{c} such that ⟨ξ, α⟩ ≠ 0 for all roots α ∈ Δ(𝔫_{c}, 𝔥_{c}). Then by a standard argument,

Ad P_{c}(ξ + e) = ξ + 𝔠_{c} + 𝔫_{c}.  (8.4.2) {eq:8.4.2}

Again because G_{c}/P_{c} is compact,

\overline{⋃_{t>0} Ad G_{c}(tξ + e)}\ ⋃_{t>0} Ad G_{c}(tξ + e) = Ad G_{c}(\overline{𝔠_{c}} + 𝔫_{c}).  (8.4.3) \{eq:8.4.3\}

Formula 8.4.3 is valid for any parabolic subgroup with Levi component M_{c}. The claim follows because the left hand side of (8.4.3) only depends on M_{c} and the orbit 𝔠_{c}. □

We now consider the case of real induction. Let 𝔭 = 𝔪 + 𝔫 be a real parabolic subalgebra, e ∈ 𝔪 a nilpotent element, and 𝔠 := Ad Me.

                                                                       {d:8.4}

Definition. The ρ−induced set from 𝔠 to 𝔤 is the finite union of orbits ℭ_{i} := Ad GE_{i} such that one of the following equivalent conditions hold.

(1) ℭ_{i} is open in Ad G(𝔠 + 𝔫) and \overline{⋃ ℭ_{i}} = \overline{Ad G(e + 𝔫)}.

(2) The intersection ℭ_{i} ∩ [𝔠 + 𝔫] is open in 𝔠 + 𝔫, and the union of the intersections is dense in 𝔠 + 𝔫.

We write

ind_{𝔭}^{𝔤}(𝔠) = ⋃ ℭ_{i}. {(8.4.4) \atop \{eq:8.4.4\}}

and we say that each E_{i} is real or ρ−induced from e. Some times we will write ind_{𝔭}^{𝔤}(e).

We omit the details of the proof of the equivalence of the two statements.

{p:8.4}

Proposition (3). The ρ−induced set depends on the orbit 𝔠 of e and the Levi component 𝔪, but not on 𝔫.

Proof. The proof is essentially identical to the one in the complex case. We omit the details. □

In terms of the θ-stable versions ẽ of e, and Ẽ_{i} of E_{i}, ρ−induction is computed in [BB]. This is as follows. Let 𝔥_{c} ⊂ 𝔪_{c} be the complexification of a maximally split real Cartan subalgebra 𝔥, and ξ ∈ Z(𝔪_{c})∩𝔰_{c} an element of 𝔥 such that

α ∈ Δ(𝔫, 𝔥) if and only if α(ξ) > 0.

Then

\overline{⋃ Ad K_{c}(Ẽ_{i})} = \overline{⋃_{t>0} Ad K_{c}(tξ + ẽ)}\ ⋃_{t>0} Ad K_{c}(tξ + ẽ). (8.4.5) {eq:8.4.5}

{sec:8.5}

8.5. Let 𝔮_{c} = 𝔩_{c} +𝔲_{c} be a θ-stable parabolic subgroup, and write \overline{𝔮_{c}} = 𝔩_{c} +\overline{𝔲_{c}} for its complex conjugate. Let e ∈ 𝔩_{c} ∩ 𝔰_{c} be a nilpotent element.

{p:8.5}

Proposition. There is a unique K_{c}−orbit orbit O_{K_{c}}(E) so that its intersection with O_{L_{c}∩K_{c}}(e) + (𝔲_{c} ∩ 𝔰_{c}) is open and dense.

Proof. This follows from the fact that e + (𝔲_{c} ∩ 𝔰_{c}) is formed of nilpotent orbits, there are a finite number of nilpotent orbits, and being complex, the K_{c}−orbits have even real dimension. □

{d:8.5}

Definition. The orbit O_{K_{c}}(E) as in the proposition above is called θ−induced from e, and we write

ind_{𝔮_{c}}^{𝔤_{c}}(O_{𝔩_{c}}(e)) = O(E),

and say that E is θ−induced from e.

Remark. The induced orbit is characterized by the property that it is the (unique) largest dimensional one which meets e + 𝔲_{c} ∩ 𝔰_{c}. It depends on e as well as 𝔮_{c}, not just e and 𝔩_{c}.

{sec:8.7}

8.6. u(p,q). Let V be a complex finite dimensional vector space of dimension n. There are two inner classes of real forms of gl(V). One is such that θ is an outer automorphism. It consists of the real form GL(n, ℝ), and when n is even, also U^{∗}(n). The other one is such that θ is inner, and consists of the real forms U(p, q) with p + q = n. In sections 8.6-8.12, we investigate ρ and θ induction for the forms u(p, q), and then derive the corresponding results for so(p, q) and sp(n, ℝ) from them in sections 8.13-8.14. The corresponding results for the other real forms area easier, the case of GL(n, ℝ) is well known, and we will not need u(n)^{∗}. The usual description of u(p, q) is that V is endowed with a hermitian form ( , ) of signature (p, q), and u(p, q) is the Lie algebra of skew hermitian matrices with respect to this form. Fix a positive definite hermitian form ⟨ , ⟩. We will identify the complexification of 𝔤 := u(p, q) with 𝔤_{c} := gl(V), and the complexification of U(p, q) with GL(V). Up to conjugacy by GL(V),

(v, w) = ⟨θv, w⟩,  θ^{2} = 1,  (8.6.1) {eq:8.7.1}

The eigenspaces of θ on V will be denoted V^{±}. The Cartan decomposition is 𝔤_{c} = 𝔨_{c} + 𝔰_{c}, where 𝔨 is the +1 eigenspace, and 𝔰 the −1 eigenspace of Ad θ.

The classification of nilpotent orbits of u(p, q) is by signed tableaus as in theorem 9.3.3 of [CM]. The same parametrization applies to θ stable orbits under K_{c}. We need some results about closure relations between nilpotent orbits. For a θ-stable nilpotent element e, we write a_{±}(e^{k}) for the signature of θ on the kernel of e^{k}, and a(e^{k}) = a_{+}(e^{k}) + a_{−}(e^{k}) for the dimension of the kernel. If it is clear what nilpotent element they refer to, we will abbreviate them as a_{±}(k). The interpretation of these numbers in terms of signed tableaus is as follows. (a_{+}(k), a_{−}(k)) are the numbers of +’s respectively −’s in the longest k columns of the tableau.

                                                                       {t:8.7.2}

Theorem. Two θ−stable nilpotent elements e and e′ are conjugate by K_{c} if and only if e^{k} and e^{′k} have the same signatures. The relation O_{K_{c}}(e′) ⊂ \overline{O_{K_{c}}(e)} holds if and only if for all k,

a_{+}(e^{′k}) ≥ a_{+}(e^{k}), a_{−}(e^{′k}) ≥ a_{−}(e^{k}).

Proof. For real nilpotent orbits, the analogue of this result is in [D]. The theorem follows by combining [D] with proposition 8.3. We sketch a direct proof, ommitting most details except those we will need later.

Decompose

V = ⨁ V_{i}

into sl(2) representations which are also stabilized by θ. Let ϵ_{i} be the eigenvalue of θ on the highest eigenweight of V_{i} (also the kernel of e). We encode the information about e into a tableau with rows equal to the dimensions of V_{i} and alternate signs + and − starting with the sign of ϵ_{i}. The number of +’s and −’s in the first column gives the signature of θ on the kernel of e.

Then the number of ± in the first two columns gives the signature of θ on the kernel of e^{2} and so on. The total number of +’s equals p, the number of −’s equals q. Write V = V_{+} + V_{−}, where V_{±} are the ±1 eigenspaces of θ. The element e is given by a pair (A, B), where A ∈ Hom[V_{+}, V_{−}], and B ∈ Hom[V_{−}, V_{+}]. Then e^{k} is represented by (ABAB …, BABA …), k factors each, and a_{±}(k) is the dimension of the kernel of the corresponding composition of A and B. The fact that the condition in the theorem is necessary, follows from this interpretation. □

{sec:8.8}

8.7. A parabolic subalgebra of gl(V) is the stabilizer of a generalized flag

(0) = W_{0} ⊊ W_{1} ⊊ ⋯ ⊊ W_{k} = V. {(8.7.1) \atop \{eq:8.8.1\}}

Fix complementary spaces V_{i},

W_{i} = W_{i−1} + V_{i}, i > 0. {(8.7.2) \atop \{eq:8.8.2\}}

They determine a Levi component

𝔩 ≅ gl(V_{1}) × ⋯ × gl(V_{k}). {(8.7.3) \atop \{eq:8.8.1.2\}}

{sec:8.9}

8.8. Conjugacy classes under K_{c} of θ-stable parabolic subalgebras are parametrized by ordered pairs (p_{1}, q_{1}), …, (p_{k}, q_{k}) such that the sum of the p_{i} is p, and the sum of the q_{i} is q. A realization in terms of flags is as follows. Choose the W_{i} to be stable under θ, or equivalently that the restriction of the hermitian form to each W_{i} is nondegenerate. In this case we may assume that the V_{i} are θ-stable as well, and let 𝔮_{c} = 𝔩_{c} + 𝔲_{c} be the corresponding parabolic subalgebra of gl(V). The signature of the form restricted to V_{i} is (p_{i}, q_{i}), so that

𝔩_{c} ∩ 𝔤 ≅ u(p_{1}, q_{1}) × ⋯ × u(p_{k}, q_{k}). {(8.8.1) \atop \{eq:8.9.1\}}

{sec:8.10}

8.9. Conjugacy classes under G of real parabolic subalgebras are given by ordered subsequences n_{1}, …, n_{k} and a pair (p_{0}, q_{0}) such that ∑ n_{i} + p_{0} = p and ∑ n_{i} + q_{0} = q. The complexification of the corresponding real parabolic subalgebra is given as follows. Start with a partial flag

(0) = W_{0} ⊊ ⋯ ⊊ W_{k} {(8.9.1) \atop \{eq:8.10.1\}}

such that the hermitian form is trivial when restricted to W_{k}, and complete it to

(0) = W_{0} ⊊ ⋯ ⊊ W_{k} ⊊ W_{k}^{∗} ⊊ ⋯ ⊊ W_{0}^{∗} = V {(8.9.2) \atop \{eq:8.10.2\}}

Choose transverse spaces

W_{i} = W_{i−1} + V_{i},  W_{i}^{∗} = W_{i−1}^{∗} + V_{i}^{∗},  W_{k}^{∗} = W_{k} + V_{0}. {(8.9.3) \atop \{eq:8.10.3\}}

They determine a Levi component

𝔩_{c} = gl(V_{1}) × ⋯ × gl(V_{k}) × gl(V_{0}) × gl(V_{k}^{∗}) × ⋯ × gl(V_{1}^{∗}), {(8.9.4) \atop \{eq:8.8.10.4\}}

so that

𝔩_{c} ∩ 𝔤 = gl(V_{1}, ℂ) × ⋯ × gl(V_{k}, ℂ) × u(p_{0}, q_{0}). {(8.9.5) \atop \{eq:8.10.5\}}

Then n_{i} = dim V_{i}, and (p_{0}, q_{0}) is the signature of V_{0}.

{sec:8.11}

8.10. Let now 𝔮_{c} = 𝔩_{c} + 𝔲_{c} be a maximal θ stable parabolic subalgebra corresponding to the flag W_{1} = V_{1} ⊊ W_{2} = V_{1} + V_{2} = V. Then

𝔩_{c} ≅ Hom[V_{1}, V_{1}] + Hom[V_{2}, V_{2}] = gl(V_{1}) × gl(V_{2}), 𝔲_{c} ≅ Hom[V_{2}, V_{1}].

(8.10.1) {eq:8.11.1} Write n_{i} := dim V_{i}, and θ = θ_{1} + θ_{2} with θ_{i} ∈ End(V_{i}). A nilpotent element e ∈ gl(V_{2}) satisfying θ_{2}e = −eθ_{2}, can be viewed as a θ−stable nilpotent element in 𝔩_{c} by making it act by 0 on V_{1}. Let E = e + X, with X ∈ 𝔲_{c} (so X ∈ gl(V) acts by 0 on V_{1}). Then Xθ_{2} = −θ_{1}X. Decompose

V_{2} = ⨁ W_{i}^{ϵ_{i}}  (8.10.2) {eq:8.11.2}

where W_{i}^{ϵ_{i}} are irreducible sl(2, ℂ) representations stabilized by θ such that the eigenvalue of θ_{2} on the highest weight v_{i} is ϵ_{i}. Order the W_{i}^{ϵ_{i}} so that dim W_{i} ≥ dim W_{i+1}. Write A_{±}(k) for the signatures of E^{k} and a_{±}(k) for the signatures of e^{k}.

                                                                       {p:8.11}

Proposition. The signature (A_{+}(k), A_{−}(k)) of E^{k} satisfies

A_{+}(k) ≥ dim V_{1,+} + a_{+}(k − 1)+

+ max [0 , #{ i | dim W_{i}^{ϵ_{i}} ≥ k, ϵ_{i} = (−1)^{k−1}} − dim V_{1}^{(−1)^{k}}],

A_{−}(k) ≥ dim V_{1,−} + a_{−}(k − 1)+

+ max [0 , #{ i | dim W_{i}^{ϵ_{i}} ≥ k, ϵ_{i} = (−1)^{k}} − dim V_{1}^{(−1)^{k−1}}].

Proof. Since E^{k} = e^{k} + Xe^{k−1}, an element v ∈ V_{2}, is in the kernel of E^{k} if and only if e^{k−1}v is in the kernel of X as well as e. Thus V_{1} ⊂ ker E. This accounts for the terms dim V_{1}^{±}. Since ker e^{k−1} ⊂ ker Xe^{k−1} ∩ ker e^{k}, this accounts for the terms a_{±}(k − 1).

The representation theory of sl(2, ℂ) implies

ker e ∩ Im e^{k−1} = span{v_{i}^{ϵ_{i}} | e · v_{i}^{ϵ_{i}} = 0, dimW_{i}^{ϵ_{i}} ≥ k}  (8.10.3) {eq:8.11.3}

If the sign of v_{i} is ϵ_{i}, and v_{i} = e^{k−1}w_{j}, then the sign of θ on w_{j} is ϵ_{j}(−1)^{k−1}. Then X : V_{2}^{ϵ_{i}} ⟶ V_{1}^{−ϵ_{i}}, and the minimum possible dimension of the kernel of X on the space in (8.10.3) is the last term in the inequalities of the proposition. The claim follows. □

                                                                       {sec:8.12}

8.11. We now construct an E such that the inequalities in proposition 8.10 are equalities.

For any integers a, b, let

𝕂_{a}^{+} := span{ first a v_{i} with ϵ_{i} = 1},
𝕂_{b}^{−} := span{ first b v_{j} with ϵ_{j} = −1}

(8.11.1) {eq:8.12.1}

Note that

X(𝕂_{a}^{+}) ⊂ V_{1}^{−},  X(𝕂_{b}^{−}) ⊂ V_{1}^{+}.  (8.11.2) {eq:8.11.4}

                                                                       {t:8.12}

Theorem. Let E = e + X with notation as in 8.11.2. Choose X such that it is nonsingular on 𝕂_{a,b}^{±} for as large an a and b as possible. Then Ad K_{c}(E) = ind_{𝔮}^{𝔤_{c}}e.

Proof. ¿From the proposition it follows that the A_{±}^{k} of any element in e + (𝔲_{c} ∩ 𝔰_{c}) are minimal when they are equal to the RHS of proposition 8.10. Theorem 8.6 implies that if a nilpotent element achieves this minimum, its orbit contains any other e + X in its closure. This minumum is achieved by the choice of X in the proposition, bering in mind that the W_{i} were ordered in decreasing order of their dimension. Thus Ad K_{c}(E) has maximal dimension among all orbits meeting e + (𝔲 ∩ 𝔰) and so the claim follows from the observation at the end of 8.5. □

This theorem implies the following algorithm for computing the induced orbit in the case 𝔤 ≅ u(p, q) :

Suppose the signature of V_{1} is (a_{+}, a_{−}). Then add a_{+} +’s to the beginning of largest possible rows of e starting with a − and a_{−} −’s to the largest possible rows of e starting with a +. If a_{+} is larger than the number of rows starting with −, add a new row of size 1 starting with +. The similar rule applies to a_{−}.

If e ∈ gl(V_{1}), the analogous procedure applies, but the a_{+} +’s are added at the end of the largest possible rows finishing in − and a_{−} −’s to the end of the largest possible rows finishing in +.

Because induction is transitive, the above algorithm can be generalized to compute the θ-induced of any nilpotent orbit. We omit the details.

{sec:8.13}

8.12. Suppose 𝔭_{c} = 𝔪_{c} + 𝔫_{c} is the complexification of a real parabolic subalgebra corresponding to the flag (0) ⊂ V_{1} ⊂ V_{1} + V_{0} ⊂ V_{1} + V_{0} + V_{1}^{∗}, and let e ⊂ gl(V_{0}) be a real nilpotent element. The rest of the notation is as in section 8.4.

{t:8.13}

Theorem. The tableau of an orbit Ad G(E_{i}) which is in the ρ−induced set ind_{𝔭}^{𝔤}(𝔠), is obtained from the tableau of e as follows.

Add two boxes to the end of each of dim V_{1} of the largest rows such that the result is still a signed tableau.

Proof. We use (8.4.2) and (8.4.3). Let α ∈ Hom[V_{1}, V_{1}^{∗}] ⊕ Hom[V_{1}^{∗}, V_{1}] be nondegenerate such that α^{2} = Id ⊕ Id, and extend it to an endomorphism ξ ∈ gl(V) so that its restriction to V_{0} is zero. This is an element such that the centralizer of ad ξ is 𝔪, in particular, [ξ, e] = 0. Let

P (X) = X^{m} + a_{m−1}X^{m−1} + ⋯ + a_{0} {(8.12.1) \atop \{eq:8.13.1\}}

be any polynomial in X ∈ gl(V). Suppose t_{i} ∈ ℝ are such that t_{i} → 0, and assume there are g_{i} ∈ K such that t_{i}g_{i}(ξ + e)g_{i}^{−1} → E. Then

ker t_{i}^{m}P (g_{i}(ξ + e)g_{i}^{−1}) ≅ ker P (ξ + e).

(8.12.2) {eq:8.13.2}

On the other hand,

\begin{matrix}t_{i}^{m}P (g_{i}(ξ + e)g_{i}^{−1}) = [t_{i}g_{i}(ξ + e)g_{i}^{−1}]^{m}+ \\ + a_{m−1}t_{i}[t_{i}g_{i}(ξ + e)g_{i}^{−1}]^{m−1} + ⋯ + t_{i}^{m}Id → E^{m}, \end{matrix} {(8.12.3) \atop \{eq:8.13.3\}}

as t_{i} → 0. Thus

dim ker E^{m} |_{V_{±}}≥ dim ker P (ξ + e) |_{V_{±}} .  (8.12.4) {eq:8.13.4}

Choosing P (X) = (X^{2} − 1)X^{n}, we conclude that E must be nilpotent. Choosing P (X) = X^{m}, (X ± 1)X^{m−1} or P (X) = (X^{2} − 1)X^{m−2}, we can bound the dimensions of ker E^{m} |_{V_{±}} to conclude that it must be in the closure of one of the nilpotent orbits given by the algorithm of the theorem. The fact that these nilpotent orbits are in (8.4.3) follows by a direct calculation which we omit. □

                                                                       {sec:8.14}

8.13. sp(V). Suppose 𝔤_{c} ≅ sp(V_{0}), where (V_{0}, ⟨ , ⟩) is a real symplectic vector space of dimension n. The complexification (V, ⟨ , ⟩) admits a complex conjugation , and we define a nondegenerate hermitian form

(v, w) := ⟨v, \overline{w}⟩  (8.13.1) \{eq:8.14.1\}

which is of signature (n, n). Denote by u(n, n) the corresponding unitary group. Since sp(V_{0}) stabilizes ( , ), it embeds in u(n, n), and the Cartan involutions are compatible. The results of sections 8.1-8.3 together with section 8.6 imply the following classification of nilpotent orbits of sp(V_{0}) or equivalently θ-stable nilpotent orbits. See chapter 9 of [CM] for a more detailed explanation. Each orbit corresponds to a signed tableau so that every odd part occurs an even number of times. Odd sized rows occur in pairs, one starting with + the other with −.

A real parabolic subalgebra of sp(V) is the stabilizer of a flag of isotropic subspaces

(0) = W_{0} ⊂ ⋯ ⊂ W_{k},  (8.13.2) {eq:8.14.2}

so that the symplectic form restricts to 0 on W_{k}. As before, complete this to a flag

(0) = W_{0} ⊂ ⋯ ⊂ W_{k} ⊂ W_{k}^{∗} ⊂ ⋯ ⊂ W_{0}^{∗} = V.  (8.13.3) {eq:8.14.3}

We choose transverse spaces

W_{i} = W_{i−1} + V_{i}, W_{k}^{∗} = W_{k} + W, W_{i−1}^{∗} = W_{i}^{∗} + V_{i}^{∗}  (8.13.4) {eq:8.14.4}

in order to fix a Levi component. We get

𝔩 ≅ gl(V_{1}) × ⋯ × gl(V_{k}) × sp(W).  (8.13.5) {eq:8.14.5}

If we assume that V_{i}, W are θ-stable, then the corresponding parabolic subalgebra is θ-stable as well, and the real points of the Levi component are

𝔩_{0} ≅ u(p_{1}, q_{1}) × ⋯ × u(p_{k}, q_{k}) × sp(W_{0}).  (8.13.6) {eq:8.14.6}

where (p_{i}, q_{i}) is the signature of V_{i}. The parabolic subalgebra corresponding to (8.13.4) in gl(V) satisfies

𝔩′ ≅ u(p_{1}, q_{1})×⋯×u(p_{k}, q_{k})×u(n_{0}, n_{0})×u(q_{k}, p_{k})×⋯×u(q_{1}, p_{1}). (8.13.7) {eq:8.14.7}

For a maximal θ-stable parabolic subalgebra, the Levi component 𝔩 satisfies 𝔩 ≅ u(p_{1}, q_{1}) × sp(W_{0}). Let e ∈ sp(W) be a θ-stable nilpotent element. The algorithm for induced nilpotent orbits in section 8.8 implies the following algorithm for ind_{𝔩}^{𝔤_{c}}(e).

(1) add p boxes labelled +’s to the beginning of the longest rows starting with −’s, and q −’s to the beginning of the longest rows starting with +’s.

(2) add q +’s to the ending of the longest possible rows starting with −’s, and p −’s to the beginning of the longest possible rows starting with +’s.

Unlike in the complex case, the result is automatically the signed tableau corresponding to a nilpotent element in sp(V).

For a maximal real parabolic subalgebra, we must assume that \overline{V_{1}} = V_{1}, \overline{W} = W. Let V_{1,0} and W_{0} be their real points. The Levi component satisfies

𝔩 ≅ gl(V_{1,0}) × sp(W_{0}). {(8.13.8) \atop \{eq:8.14.8\}}

The results in section 8.12 imply the following algorithm for real induction.

(1) add two boxes to the largest dim V_{1} rows of e so that the result is still a signed tableau for a nilpotent orbit.

(2) Suppose dim V_{1} is odd and the last row that would be increased by 2 is odd size as well. In this case there is a pair of rows of this size, one starting with + the other with −. In this case increase these two rows by one each.

{sec:8.15}

8.14. so(p,q). Suppose 𝔤_{c} ≅ so(V_{0}), where (V_{0}, ⟨ , ⟩) is a real nondegenerate quadratic space of signature (p, q). The complexification admits a hermitian form ⟨ , ⟩ with signature (p, q) as well as a complex nondegenerate quadratic form ( , ), which restrict to ⟨ , ⟩ on V_{0}. The form ⟨ , ⟩ gives an embedding of o(p, q) into u(p, q) with compatible Cartan involutions. The results of sections 8.1-8.3 together with section 8.6 imply the following classification of nilpotent orbits of so(V_{0}) or equivalently θ-stable nilpotent orbits. See chapter 9 of [CM] for more details. Orbits correspond to signed tableaus so that every even part occurs an even number of times. Even sized rows occur in pairs, one starting with + the other with −. When all the rows have even sizes, there are two nilpotent orbits denoted I and II.

A parabolic subalgebra of so(V) is the stabilizer of a flag of isotropic subspaces

(0) = W_{0} ⊂ ⋯ ⊂ W_{k}, {(8.14.1) \atop \{eq:8.15.1\}}

so that the quadratic form restricts to 0 on W_{k}. As before, complete this to a flag

(0) = W_{0} ⊂ ⋯ ⊂ W_{k} ⊂ W_{k}^{∗} ⊂ ⋯ ⊂ W_{0}^{∗} = V. {(8.14.2) \atop \{eq:8.15.2\}}

We choose transverse spaces

W_{i} = W_{i−1} + V_{i}, W_{k}^{∗} = W_{k} + W, W_{i−1}^{∗} = W_{i}^{∗} + V_{i}^{∗}  (8.14.3) {eq:8.15.3}

in order to fix a Levi component,

𝔩 ≅ gl(V_{1}) × ⋯ × gl(V_{k}) × so(W).  (8.14.4) {eq:8.15.4}

To get a θ-stable parabolic subalgebra we must assume V_{i}, W are θ-stable and so \overline{V_{i}} = V_{i}^{∗}, \overline{W} = W. If the signature of V_{i} with respect to ⟨ , ⟩ is (p_{i}, q_{i}), and that of W is (p_{0}, q_{0}), then

𝔩_{0} ≅ u(p_{1}, q_{1}) × ⋯ × u(p_{k}, q_{k}) × so(p_{0}, q_{0}).  (8.14.5) {eq:8.15.5}

The parabolic subalgebra corresponding to (8.14.2) in gl(V) satisfies

𝔩′ ≅ u(p_{1}, q_{1})×⋯×u(p_{k}, q_{k})×u(p_{0}, q_{0})×u(p_{k}, q_{k})×⋯×u(p_{1}, q_{1}). (8.14.6) {eq:8.15.6}

For a maximal θ-stable parabolic subalgebra, the Levi component 𝔩 satisfies 𝔩 ≅ u(p_{1}, q_{1}) × so(W_{0}). Let e ∈ so(W) be a θ-stable nilpotent element. The algorithm for induced nilpotent orbits in section 8.8 implies the following algorithm for ind_{𝔩}^{𝔤_{c}}(e).

(1) add p_{1} +’s to the beginning of the longest possible rows starting with −’s, and q_{1} −’s to the beginning of the longest possible rows starting with +’s.

(2) add p_{1} +’s to the ending of the longest possible rows starting with −’s, and q_{1} −’s to the beginning of the longest possible rows starting with +’s.

Unlike in the complex case, the result is automatically a signed tableau for a nilpotent element in so(V).

For a maximal real parabolic subalgebra, we must assume that \overline{V_{1}} = V_{1}, \overline{W} = W. Let V_{1,0} and W_{0} be their real points. The Levi component satisfies

𝔩 ≅ gl(V_{1,0}) × so(W_{0}).  (8.14.7) {eq:8.15.7}

The results in section 8.12 imply the following algorithm for real induction.

Add two boxes to dim V_{1} of the largest possible rows so that the result is still a signed tableau for a nilpotnet orbit. Suppose dim V_{1} is even and the last row that would be increased by 2 is even size as well. In this case there is a pair of rows of this size, one starting with + the other with −. Increase these two rows by one each so that the result is still a signed tableau. When there are only even sized rows and dim V_{1} is even as well, type I goes to type I and type II goes to type II.

9. Unitarity

                                                                       {sec:9}

In this section we prove the unitarity of the representations of the form L(χ) where χ = ȟ/2. As already mentioned, in the p−adic case this is done in [BM1]. It amounts to the observation that the Iwahori-Matsumoto involution preserves unitarity, and takes such an L(χ) into a tempered representation.

The idea of the proof in the real case is described in [B2]. We will do an induction on rank. We rely heavily on the properties of the wave front set, asymptotic support and associated variety, and their relations to primitive ideal cells and Harish-Chandra cells. We review the needed facts in sections 9.1-9.4. Details are in [BV1], [BV2], and [B3]. In sections 9.5-9.7 we give details of the proof of the unitarity in the case of SO(2n + 1). The proof is simpler than in [B2].

{sec:9.1}

9.1. Let π be an admissible (𝔤_{c}, K) module. We review some facts from [BV1]. The distribution character Θ_{π} lifts to an invariant eigendistribution θ_{π} in a neighborhood of the identity in the Lie algebra. For f ∈ C_{c}^{∞}(U), where U ⊂ 𝔤 is a small enough neighborhood of 0, let f_{t}(X) := t^{− dim 𝔤_{c}}f(t^{−1}X). Then

θ_{π}(f_{t}) = t^{−d} ∑_{j} c_{j}{ \atop μ_{O_{j}(ℝ)}}(f) + ∑_{i>0} t^{d+i}D_{d+i}(f)]. {(9.1.1) \atop \{eq:9.1.1\}}

The D_{i} are homogeneous invariant distributions (each D_{i} is tempered and the support of its Fourier transform is contained in the nilpotent cone). The μ_{O_{j}} are invariant measures supported on real forms O_{j} of a single complex orbit O_{c}, and μ_{O_{j}(ℝ)} is the Liouville measure on the nilpotent orbit associated to the symplectic form induced by the Cartan-Killing form. Furthermore d = dim_{ℂ} O_{c}/2, and the number c_{j} is called the multiplicity of O_{j}(ℝ) in the leading term of the expansion. The closure of the union of the supports of the Fourier transforms of all the terms occuring in (9.1.1) is called the asymptotic support, denoted AS(π). The leading term in (9.1.1) will be called AC(π). We will use the fact that the nilpotent orbits in the leading term are contained in the wave front set of θ_{π} at the origin, denoted W F (π).

Alternatively, [V3] attaches to each π a combination of θ-stable orbits with integer coefficients

AV (π) = ∑ a_{j}O_{j}, {(9.1.2) \atop \{eq:9.1.2\}}

where O_{j} are nilpotent K_{c}−orbits in 𝔰_{c}. The main result of [SV] is that AC(π) in (9.1.1) and AV (π) in (9.1.2) correspond via theorem 8.3. Precisely, the leading term in formula (8.3.2), and (9.1.2) are the same, when we identify real and θ stable nilpotent orbits via the Kostant-Sekiguchi correspondence. The algorithms in section 8 compute the associated variety of an induced representation as a set, which we also denote by AV (π) when there is no possibility of confusion. The multiplicities are computed in the real setting in [B4] theorem 5.0.7. The formula is as follows. Let v_{j} ∈ O_{j} and v_{ij} = v_{j} + X_{ij} be representatives of the induced orbits O_{ij} from O_{j,𝔪}. If AV (π) = ∑ c_{j}O_{j,𝔪}, then

AV (ind_{𝔭}^{𝔤_{c}}(π)) = ∑_{i,j} c_{j} \frac{|C_{G}(v_{ij})|}{|C_{P} (v_{ij})|}O_{ij}.  (9.1.3) \{eq:9.1.3\}

We use [SV] to compare multiplicities of real and θ induced modules. Formula (9.1.3) is straightforward for real induction and AC(π). Its analogue for θ stable induction and AV (π) is also straightforward. It is the passage from AC(π) to AV (π) that is nontrivial.

                                                                       {sec:9.2}

9.2. Fix a regular integral infinitesimal character χ_{reg}. Let G(χ_{reg}) be the set of parameters of irreducible admissible (𝔤_{c}, K) modules with infinitesimal character χ_{reg}, and denote by ℤG(χ_{reg}) the corresponding Grothendieck group of characters. Recall from [V2] (and references therein) that there is an action of the Weyl group on ℤG(χ_{reg}), called the coherent continuation action. As a set, G(χ_{reg}) decomposes into a disjoint union of blocks B, and so ℤG decomposes into a direct sum

ℤG(χ_{reg}) = ⨁ ℤG_{B}(χ_{reg}).  (9.2.1) {eq:9.2.1}

Each ℤG(χ_{reg}) is preserved by the coherent continuation action. We give the explicit description of the ℤG_{B} in all classical cases. We will suppress the ℤ whenever there is no danger of confusion. Type B: In order to conform to the duality between type B and type C in [V2], we only count the real forms with p > q. The representation G(χ_{reg}) equals

ℤG(χ_{reg}) = ∑_{a,b,τ} Ind_{W_{a}×W_{b}×W_{2s}×S_{t}}^{W_{n}}[sgn ⊗ sgn ⊗ σ[τ, τ] ⊗ triv], (9.2.2) {eq:9.2.2}

where τ is a partition of s, and a + b + 2s + t = n. The multiplicity of a σ[τ_{L}, τ_{R}] in one of the induced modules in (9.2.2) is as follows. Choose a τ that fits inside both τ_{L} and τ_{R}, and label it by •’s. Add “a” r and “b” r′ to τ_{R}, at most one to each row, and “t” c, at most one to each column, to τ_{L} or τ_{R}. The multiplicity of σ in the induced module for a given (τ, a, b) is then the number of ways that τ_{l}, τ_{R} can be filled in this way. This procedure uses induction in stages, and the well known formula

Ind_{S_{n}}^{W_{n}}(triv) = ∑_{k+l=n} σ[(k), (l)].  (9.2.3) {eq:9.2.3}

Example. Let 𝔤_{c} = so(5). The real forms are so(3, 2), so(4, 1), so(5). The choices of (τ, a, b, t) are

\begin{matrix}(1, 1, 0, 0), & (1, 0, 1, 0), & (1, 0, 0, 1), \\ (0, 2, 0, 0), & (0, 1, 1, 0)), & (0, 1, 0, 1), & (0, 0, 2, 0), & (0, 0, 1, 1), & (0, 0, 0, 2). \end{matrix}

(9.2.4) {eq:9.2.4}

Let σ = σ[(1), (1)]. Then its multiplicity is given by the number of labelings

\begin{matrix}(•, •) & ∅, & ∅, \\ ∅, & ∅, & (c, r), & ∅, & (c, r′), & (c, c). \end{matrix} {(9.2.5) \atop \{eq:9.2.5\}}

For σ = σ[(0), (2)] we get

\begin{matrix}∅, & ∅, & ∅ \\ ∅, & (0, rr′), & (0, rc), & ∅, & (0, r′c), & (0, cc). \end{matrix} {(9.2.6) \atop \{eq:9.2.6\}}

□

The following formula sorts the representations according to the various real forms of SO(p, q) with p + q = 2n + 1. Each real form gives a single block. A representation occuring in G, labelled as above, occurs in ℤG_{SO(p,q)} with

p = n + 1+ | #r′ − #r | −ϵ, where ϵ = \begin{cases}0 & if #r′ ≥ #r, \\ 1 & otherwise. \end{cases}

(9.2.7) {eq:9.2.7}

In the above example, (•, •), (c, c), (0, rr′), (0, rc) and (0, cc) belong to so(3, 2) while (c, r′) and (0, r′c) belong to so(4, 1).

To each pair of partitions parametrizing a representation of W,

τ_{L} = (r_{0}, …, r_{2m}),  τ_{R} = (r_{1}, …, r_{2m−1}),  r_{i} ≤ r_{i+2}, {(9.2.8) \atop \{eq:9.2.8\}}

Lusztig attaches a symbol

(\begin{matrix}r_{0} & & r_{2} + 1 & & & … & & r_{2m} + m \\ & r_{1} & & r_{3} + 1 & … & & r_{2m−1} + m − 1 \end{matrix}) . {(9.2.9) \atop \{eq:9.2.9\}}

The symbol is called special if

r_{0} ≤ r_{1} ≤ r_{2} + 1 ≤ r_{3} + 1 ≤ ⋯ ≤ r_{2m} + m. {(9.2.10) \atop \{eq:9.2.10\}}

Two representations belong to the same double cell if and only if their symbols have the same entries. Given a special symbol of the form (9.2.9), the corresponding nilpotent orbit O_{c} has partition obtained as follows. Form the set

\{2r_{2i} + 2i + 1, 2r_{2j−1} + 2j − 2\}, {(9.2.11) \atop \{eq:9.2.11\}}

and order the numbers in increasing order, x_{0} ≤ ⋯ ≤ x_{2m}. The partition of O_{c} is

(x_{0}, x_{1} − 1, …, x_{i} − i, …, x_{2m} − 2m). {(9.2.12) \atop \{eq:9.2.12\}}

Type C: The representation G(χ_{reg}) is obtained from the one in type B by tensoring with sign. Thus

ℤG(χ_{reg}) = ∑_{a,b,τ} Ind_{S_{t}×W_{2s}×W_{a}×W_{b}}^{W_{n}}[sgn ⊗ σ[τ, τ] ⊗ triv ⊗ triv], {(9.2.13) \atop \{eq:9.2.13\}}

where τ is a partition of s, and a+b+2s+t = n. This takes into account the duality in [V2] of types B and C. We write r for the sign representation of S_{t}, and c and c′ for the trivial representations of W_{a}, W_{b}. A representation of W is parametrized by a pair of partitions (τ_{L}, τ_{R}), with

τ_{L} = (r_{0}, …, r_{2m}),  τ_{R} = (r_{1}, …, r_{2m−1}),  r_{i} ≤ r_{i+2}. (9.2.14) {eq:9.2.14}

The associated symbol is

(\begin{matrix}r_{0} & & r_{2} + 1 & & & … & & r_{2m} + m \\ & r_{1} & & r_{3} + 1 & … & & r_{2m−1} + m − 1 \end{matrix}) , (9.2.15) \{eq:9.2.15\}

and it is called special if

r_{0} ≤ r_{1} ≤ r_{2} + 1 ≤ r_{3} + 1 ≤ ⋯ ≤ r_{2m} + m.  (9.2.16) {eq:9.2.16}

Two representations belong to the same double cell if their symbols have the same entries. Given a special symbol as in (9.2.15), the nilpotent orbit O_{c} attached to the double cell has partition obtained as follows. Order the set

{2r_{2i} + 2i, 2r_{2j−1} + 2j − 1}  (9.2.17) {eq:9.2.17}

in increasing order, x_{0} ≤ ⋯ ≤ x_{2m}. Then the partition of O_{c} is

(x_{0}, …, x_{j} − j, …, x_{2m} − 2m).  (9.2.18) {eq:9.2.18}

The decomposition into blocks is obtained from the one for type B by tensoring with sgn.

Type D: Since in this case σ[τ_{L}, τ_{R}] and σ[τ_{R}, τ_{L}] parametrize the same representation, (except of course when τ_{L} = τ_{R} which corresponds to two nonisomorphic representations), we assume that the size of τ_{L} is the larger one. The Cartan subgroups are parametrized by integers (t, u, 2s, p, q), p + q + 2s + t + u = n. There are actually two Cartan subgroups for each s > 0, related by the outer automorphism of order 2. Then G(χ_{reg}) equals

\begin{matrix}ℤG(χ_{reg}) = \\ = ∑_{p+q+2s+t+u=n} Ind_{W_{a}×W_{b}×W′_{2s}×W_{t}×W_{u}}^{W′_{n}}[sgn ⊗ sgn ⊗ σ[τ, τ]_{I,II} ⊗ triv ⊗ triv]. \end{matrix}

(9.2.19) {eq:9.2.19}

The sum is also over τ which is a partition of s. We label the σ by •’s, trivial representations by c and c′ and the sgn representations by r and r′. These are added to τ_{L} when inducing. In this case we count all the real forms SO(p, q) with p + q = 2n. Each real form gives riswe to a single block. A represntation labelled as above belongs to the block ℤG_{SO(p,q)} with p = n + #r′ − #r.

If

τ_{L} = (r_{0}, …, r_{2m−2}), τ_{R} = (r_{1}, …, r_{2m−1}), (9.2.20) {eq:9.2.20}

then the associated symbol is

(\begin{matrix}r_{0} & r_{2} + 1 & … & r_{2m−2} + m − 1 \\ r_{1} & r_{3} + 1 & … & r_{2m−1} + m − 1 \end{matrix}) . {(9.2.21) \atop \{eq:9.2.21\}}

A representation is called special if the symbol satisfies

r_{0} ≤ r_{1} ≤ r_{2} + 1 ≤ r_{3} + 1 ≤ ⋯ ≤ r_{2m−1} + m − 1. {(9.2.22) \atop \{eq:9.2.22\}}

Two representations belong to the same double cell if their symbols have the same entries. The nilpotent orbit O_{c} attached to the special symbol is given by the same procedure as for type B.

{sec:9.3}

9.3. We follow section 14 of [V2]. We say that π′ ≤ π if π′ is a factor of π ⊗ F with F a finite dimensional representation with highest weight equal to an integer sum of roots. Two irreducible representations π, π′ are said to be in the same Harish-Chandra cell if π′ ≤ π and π ≤ π. The Harish-

Chandra cell of π is denoted C(π). Recall the relation <^{LR} from definition 14.6 of [V2]. The cone above π is defined to be

\overline{C}^{LR}(π) := \{π′ : π′ <^{LR} π\}. {(9.3.1) \atop \{eq:9.3.1\}}

The subspace in ℤG generated by the elements in \overline{C}^{LR}(π) is a representation

of W denoted \overline{V}^{LR}(π). The equivalence ≈^{LR} is defined by

γ ≈^{LR} ϕ ⟺ γ <^{LR} ϕ <^{LR} γ. {(9.3.2) \atop \{eq:9.3.2\}}

The Harish-Chandra cell C(π) is then

C(π) = \{π′ : π′ ≈^{LR} π\}. {(9.3.3) \atop \{eq:9.3.3\}}

Define

C_{+}^{LR}(π) = \overline{C}^{LR}(π) \ C(π) {(9.3.4) \atop \{eq:9.3.5\}}

and \overline{V}(π) and V^{LR}(π) in analogy with \overline{V}^{LR}(π). Thus there is a representation of W on V(π) by the natural isomorphism

V(π) ≅ \overline{V}^{LR}(π)/V(π)_{+}. {(9.3.5) \atop \{eq:9.3.6\}}

Let O_{c} ⊂ 𝔤_{c} be a nilpotent orbit. We say that a Harish-Chandra cell is attached to a complex orbit O_{c} if

\overline{Ad G_{c}(AS(π))} = \overline{O_{c}}.

The sum of the Harish Chandra cells attached to O_{c} is denoted V(O_{c}).

Let 𝔥_{a} ⊂ 𝔤_{c} be an abstract Cartan subalgebra and let Π_{a} be a set of (abstract) simple roots. For each irreducible representation L(γ), denote by τ(γ) the tau-invariant as defined in [V2]. Given a block B and disjoint orthogonal sets S_{1}, S_{2} ⊂ Π_{a}, define

B(S_{1}, S_{2}) = \{γ ∈ B | S_{1} ⊂ τ(γ), S_{2} ∩ τ(γ) = ∅\} . {(9.3.6) \atop \{eq:9.2.23\}}

If in addition we are given a nilpotent orbit O_{c} ⊂ 𝔤_{c}, we can also define

B(S_{1}, S_{2}, O_{c}) = \{γ ∈ B(S_{1}, S_{2})| AS(L(γ)) ⊂ \overline{O_{c}}\} . {(9.3.7) \atop \{eq:9.2.24\}}

The convention is that if f S_{i} = ∅ then no condition is imposed on the parameter. Let W_{i} = W (S_{i}), and define

\begin{aligned}m_{S}(σ) & = [σ : Ind_{W_{1}×W_{2}}^{W}(Sgn ⊗ T riv)], \\ m_{B}(σ) & = [σ : G_{B}(χ_{reg})] . \end{aligned}  (9.3.8)

In the case of G_{c} viewed as a real group, the cones defined by (9.3.1) are

parametrized by nilpotent orbits in 𝔤_{c}. In other words, π′ <^{LR} π is and only if AS(π′) ⊂ \overline{AS(π)}. So let C(O_{c}) be the cone corresponding to O_{c}. Note that in this case W_{c} ≅ W × W, and the representations are of the form σ ⊗ σ.

Theorem.

|B(S_{1}, S_{2}, O_{c})| = ∑_{σ⊗σ∈C(O_{c})} m_{B}(σ)m_{S}(σ) .

Proof. Consider ℤB(S_{1}, S_{2}, O_{c}) ⊂ ℤB(∅, ∅, O_{c}). Then B(∅, ∅, O_{c}) is a representation of W which consists of the representations in V(π) with AS(π) ⊂ \overline{O_{c}}. The fact that the representation V(π) is formed of σ with σ ⊗ σ ∈ C(O_{c}) follows from the argument before theorem 1 of [McG]. This accounts for m_{B}(σ) in the sum. The expressions of the action of W given by lemma 14.7 in [V2] and Frobenius reciprocity imply that the dimension of ℤB(S_{1}, S_{2}, O_{c}) equals the left hand side of the formula in the theorem, and it equals the cardinality of B(S_{1}, S_{2}, O_{c}). □

                                                                       {sec:9.4}

9.4. Assume that Ǒ is even. Then λ := ȟ/2 is integral, and it defines a set S_{2} by

S_{2} = S(λ) = {α ∈ Π_{a}|(α, λ) = 0} .  (9.4.1) {eq:9.2.25}

Let O_{c} be the nilpotent orbit attached to Ǒ by the duality in [BV3]. Then the special unipotent representations attached to Ǒ , called Unip(Ǒ), are defined to be the representations π with infinitesimal character λ and AS(π) ⊂ O_{c}. Via translation functors they are in 1-1 correspondence with the set

⋃_{B} B(∅, S(λ), O_{c}).  (9.4.2) {eq:9.2.26}

So we can use theorem 9.4 to count the number of unipotent representations. In the classical groups case, m_{B}(σ) is straightforward to compute. For the special unipotent case, m_{S}(σ) equals 0 except for the representations occuring in a particular left cell sometimes also called the Lusztig cell, which we denote C^{L}(O_{c}). The multiplicities of the representations occuring in \overline{C}^{L}(O_{c}) are all 1. These representations are in 1-1 correspondence with the conjugacy classes in Lusztig’s quotient of the component group \overline{A}(Ǒ). See [BV2] for details.

                                                                       {t:9.2.2}

Theorem (1).

|Unip(Ǒ)| = ∑_{B} ∑_{σ∈\overline{C}^{L}(O_{c})} m_{B}(σ) .

{t:9.3}

Theorem (2, [McG]). In the classical groups Sp(n), SO(p, q), each Harish-Chandra cell is of the form \overline{C}^{L}(O_{c}).

{d:9.2}

Definition. We say that a nilpotent orbit O_{c} is smoothly cuspidal if it satisfies

Type B, D: all odd sizes occur an even number of times, Type C: all even sizes occur an even number of times.

For O(ℝ), a real form of O_{c}, write A(O(ℝ)) for its (real) component group.

{p:9.2}

Proposition. For smoothly cuspidal orbits, A(Ǒ) = \overline{A}(Ǒ). In particular, |C^{L}(O_{c})| = |A(Ǒ)|. Furthermore,

|Unip(Ǒ))| = ∑_{O(ℝ)}|A(O(ℝ))|

where the sum is over all real forms O(ℝ) of O_{c}.

Proof. This is theorem 5.3 in [B2]. The proof consists of a direct calculation of multiplicities in the coherent continuation representation using the results developed earlier in this section. □

{sec:9.5}

9.5. We now return to type G = SO(2n + 1). Consider the spherical irreducible representation L(χǑ) with χǑ = ȟ/2 corresponding to a nilpotent orbit Ǒ in sp(n). If the orbit Ǒ meets a proper Levi component 𝔪̌, then L(Ǒ) is a subquotient of a representation which is unitarily induced from a unipotent representation on 𝔪. By induction, L(χǑ) is unitary. Thus we only consider the cases when Ǒ does not meet any proper Levi component. This means

Ǒ = (2x_{0}, …, 2x_{2m}), 0 ≤ x_{0} < ⋯ < x_{i} < x_{i+1} < ⋯ < x_{2m}, {(9.5.1) \atop \{eq:9.5.1\}}

so these orbits are even.

Because of assumption (9.5.1), the AS-set of L(χǑ) satisfies the property that

\overline{Ad G_{c}(AS(L(χǑ)))}

is the closure of the special orbit O_{c} dual to Ǒ. This is the orbit O_{c} with partition

(\underbrace{1, …, 1}_{r_{1}}, \underbrace{2, …, 2}_{r_{2}}, …, \underbrace{2m, …, 2m}_{r_{2m}}, \underbrace{2m + 1, …, 2m + 1}_{r_{2m+1}}), {(9.5.2) \atop \{eq:9.5.2\}}

where

\begin{aligned}r_{2i+1} & = 2(x_{2m−2i} − x_{2m−2i−1} + 1), \\ r_{2i} & = 2(x_{2m−2i+1} − x_{2m−2i} − 1), \\ r_{2m+1} & = 2x_{0} + 1. \end{aligned}

The columns of O_{c} are (2x_{2m} + 1, 2x_{2m−1} − 1, …, 2x_{0} + 1).

Definition. Given an orbit O_{c} with partition (9.5.2) or more generally a smoothly cuspidal orbit, we call the split real form O_{spl} the one which satisfies for each row size,

Type C,D: the number of rows starting with + and − is equal, Type B: in addition to the condition in types C,D for rows of size less than 2m + 1, for size 2m + 1, the number of starting with + is one more than those starting with −.

Theorem. The W F -set of the spherical representation L(χǑ) with Ǒ satisfying (9.5.1) is the closure of the split real form O_{spl} of the (complex) orbit O_{c} given by (9.5.2).

Proof. The main idea is outlined in [B2]. We use the fact that if π is a factor of π′, then W F (π) ⊂ W F (π′). We do an induction on m. The claim amounts to showing that if E occurs in AS(L(χǑ)), then the signatures of E, E^{2}, … are greater than the pairs

\begin{matrix}(x_{2m} + 1, x_{2m}), (x_{2m} + x_{2m−1}, x_{2m} + x_{2m−1}), …, \\ … (x_{2m} + ⋯ + x_{1}, x_{2m} + ⋯ + x_{1}), \\ (x_{2m} + ⋯ + x_{1} + x_{0} + 1, x_{2m} + ⋯ + x_{1} + x_{0}). \end{matrix}  (9.5.3) \{eq:9.5.3\}

The statement is clear when m = 0; L(χǑ) is the trivial representation. Let Ǒ_{1} be the nilpotent orbit corresponding to

(2x_{0}, …, 2x_{2m−2}).  (9.5.4) {eq:9.5.4}

By induction, W F (L(Ǒ_{1})) is the split real form of the nilpotent orbit corresponding to the partition

(\underbrace{1, …, 1}_{r′_{1}}, \underbrace{2, …, 2}_{r′_{2}}, …, \underbrace{2m − 2, …, 2m − 2}_{r′_{2m−2}}, \underbrace{2m − 1, …, 2m − 1}_{r′_{2m−1}}), (9.5.5) \{eq:9.5.5\}

where the columns are (2x_{2m−2}+1, 2x_{2m−3}−1, …, 2x_{0}+1). Let 𝔭 be the real parabolic subalgebra with Levi component 𝔤(n − x_{2m} − x_{2m−1}) × gl(x_{2m} + x_{2m−1}). There is a character χ of gl(x_{2m} + x_{2m−1}) such that π := L(χǑ) is a factor of π′ := Ind_{𝔭}^{𝔤_{c}}[L(χǑ_{1}) ⊗ χ]. But by section 8, W F (π′) is in the closure of nilpotent orbits corresponding to partitions

(\underbrace{2, …, 2}_{(r_{1}+r_{2})/2}, …, \underbrace{2m, …, 2m}_{r_{2m}}, \underbrace{2m + 1, …, 2m + 1}_{r_{2m+1}}), r_{1} + r_{2} even, (9.5.6) \{eq:9.5.6\}

(1, 1, \underbrace{2, …, 2}_{(r_{1}+r_{2}−1)/2}, …, \underbrace{2m, …, 2m}_{r_{2m}}, \underbrace{2m + 1, …, 2m + 1}_{r_{2m+1}}), r_{1} + r_{2} odd. (9.5.7) \{eq:9.5.7\}

It follows that the signatures for E^{k} in W F (L(χǑ)) are greater than the pairs

(a_{+}, a_{−}), (x_{2m} + x_{2m−1}, x_{2m} + x_{2m−1}), …,  (9.5.8) {eq:9.5.8}

for some a_{+} + a_{−} = x_{2m} + 1. Also, each row size greater than two and less than 2m + 1 has an equal number that start with + and −. For size 2m + 1 there is one more row starting with + than −.

The same argument with Ǒ_{2} corresponding to

(2x_{0}, … { \atop 2x_{2m−2}}, { \atop 2x_{2m−1}}, 2x_{2m})

shows that W F (L(χǑ)) is also contained in the closure of the nilpotent orbits with signatures

\begin{matrix}(x_{2m} + 1, x_{2m}), (x_{2m} + 1 + a_{+}, x_{2m} + a_{−}), \\ (x_{2m} + 1 + x_{2m−1} + x_{2m−2}, x_{2m} + 1 + x_{2m−1} + x_{2m−2}), …, \end{matrix} {(9.5.9) \atop \{eq:9.5.9\}}

for some a_{+} + a_{−} = x_{2m−1}. The claim follows. □

{sec:9.6}

9.6. Consider the special case when

x_{0} = x_{1} − 1 ≤ x_{2} = x_{3} − 1 ≤ ⋯ ≤ x_{2m−2} = x_{2m−1} − 1 ≤ x_{2m}. {(9.6.1) \atop \{eq:9.6.1\}}

The cell C^{L}(O_{c}) has size 2^{m}. We produce 2^{m} distinct irreducible representations with AS equal to the closure of O_{spl}. So 𝔤 is so(2p + 1, 2p). Let 𝔥 be the compact Cartan subalgebra. We write the coordinates

(a_{1}, …, a_{p} | b_{1}, …, b_{p}) {(9.6.2) \atop \{eq:9.6.2\}}

where the first p coordinates before the | are in the Cartan subalgebra of so(2p + 1) the last p coordinates are in so(2p). The roots ϵ_{i} ± ϵ_{j}, ϵ_{i} with 1 ≤ i, j ≤ p are all compact and so are ϵ_{p+k} ± ϵ_{p+l} with 1 ≤ k, l ≤ p. The roots ϵ_{i} ±ϵ_{p+k}, ϵ_{p+k} are noncompact. Let 𝔮_{c} = 𝔩_{c} +𝔲_{c} be a θ-stable parabolic subalgebra with Levi component

𝔩 = u(x_{2i_{1}+1}, x_{2i_{1}}) × u(x_{2i_{2}}, x_{2i_{2}+1}) × ⋯ × 𝔤(x_{2m}), {(9.6.3) \atop \{eq:9.6.3\}}

where the i_{j} are the numbers 0, …, m − 1 in some order. The parabolic subalgebra 𝔮_{c} corresponds to the weight

\begin{matrix}ξ = (m^{x_{2i_{1}+1}}, …, 1^{x_{2i_{m−1}}+1}, 0^{x_{2m}} | m^{x_{2i_{1}}} , …, 1^{x_{2i_{m−1}}} , 0^{x_{2m}}), \\ or \\ ξ = (m^{x_{2i_{1}+1}}, …, 1^{x_{2i_{m−1}}} , 0^{x_{2m}} | m^{x_{2i_{1}}} , …, 1^{x_{2i_{m−1}}+1}, 0^{x_{2m}}), \end{matrix} {(9.6.4) \atop \{eq:9.6.4\}}

depending whether m is odd or even.

The derived functor modules R_{𝔮_{c}}^{i}(ξ) from characters on 𝔩_{c} have AC-set contained in O_{spl}. To get infinitesimal character χǑ, these characters can only be

ξ_{i_{j}}^{±} := ±(1/2, …, 1/2), {(9.6.5) \atop \{eq:9.6.5\}}

on the unitary factors u(x_{2i_{j}+1}, x_{2i_{j}}) or u(x_{2i_{j}}, x_{2i_{j}+1}), and trivial on 𝔤(x_{2m}). We need to show that there are choices of parabolic subalgebras 𝔮_{c} as in (9.6.3) and characters as in (9.6.5) so that we get 2^{m} nonzero and distinct representations. For this we have to specify the Langlands parameters.

For each subset A := {k_{1}, …, k_{r}} ⊂ {0, …, m − 1}, k_{j} in decreasing order, label the complement A^{c} := {ℓ_{1}, …, ℓ_{t}}, and consider the θ−stable parabolic subalgebra 𝔮_{c,A} as in (9.6.3) and (9.6.4) corresponding to

{i_{1}, …, i_{m−1}} = {k_{1}, …, k_{r}, ℓ_{1}, …, ℓ_{t}}.  (9.6.6) {eq:9.6.6}

We will consider the representations R_{𝔮_{c,A}}(ξ_{A}), where ξ_{A} is the concatentation of the ξ_{i_{j}}^{±} with + for the first r, and − for the last t.

                                                                       {l:9.6} Lemma.

R_{𝔮_{c,A}}^{i}(ξ_{A}) = \{ \begin{matrix}0 if i ≠ dim(𝔲_{c,A} ∩ 𝔨_{c}), \\ nonzero irreducible if i = dim(𝔲_{c,A} ∩ 𝔨_{c}). \end{matrix}

Proof. For the vanishing part we check that the conditions in proposition 5.93 in [KnV] chapter V section 7 are satisfied. It is sufficient to show that

ind_{\overline{𝔮}_{c,A},L∩K}^{𝔤,K}(Z_{\overline{𝔮}_{c,A}}^{#}) := U(𝔤) ⊗_{\overline{𝔮}_{c,A}} Z_{\overline{𝔮}_{c,A}}^{#}  (9.6.7) \{eq:9.6.7\}

is irreducible. Here Z_{\overline{𝔮}_{c,A}}^{#} is the 1−dimensional module corresponding to ξ_{A} − ρ(𝔲_{c,A}), with

ρ(𝔲_{c,A}) := \frac{1}{2} ∑_{α∈Δ(𝔲_{A,c})}α.

The derived functors are normalized so that if W has infinitesimal character χ, then so do R_{𝔮_{c}}^{i}(W).

For generalized Verma modules of this kind we can apply the notions and results about associated cycles. The associated cycle of (9.6.8) is O_{c} from (9.5.2), and the multiplicity is 1. Any composition factor cannot have associated cycle formed of nilpotent orbits of strictly smaller dimension than O_{c} because the results in [BV2] apply. So if there is more than one factor, the multiplicity of O_{c} would be strictly larger than 1.

To show that R_{𝔮_{c,A}}^{dim(𝔲_{c,A}∩𝔨_{c})}(ξ_{A}) ≠ 0, we use the bottom layer K− types defined in chapter V section 6 of [KnV]. To simplify the notation slightly, we write

\begin{matrix}a_{1} = x_{2k_{1}+1}, b_{1} = x_{2k_{1}}, …, a_{r} = x_{2k_{r}}, b_{r} = x_{2k_{r}+1} r even, \\ a_{1} = x_{2k_{1}+1}, b_{1} = x_{2k_{1}}, …, a_{r} = x_{2k_{r}+1}, b_{r} = x_{2k_{r}} r odd. \end{matrix}  (9.6.8) \{eq:9.6.8\}

Let also a := ∑ a_{j}, b := ∑ b_{j}. Note that |a_{j} − b_{j}| = 1, and also |a − b| = 1. Then

μ := ξ + 2ρ(𝔲 ∩ 𝔰) − ρ(𝔲) = (1^{a}, 0^{p−a} | 1^{b}, 0^{p−a})  (9.6.9) {eq:9.6.9}

is dominant, therefore bottom layer. The aforementioned results then imply the nonvanishing. Finally the derived functor module is irreducible because the multiplicity in AV of the nilpotent orbit is 1. □

We now show that there are 2^{m} distinct representations. We will need to use the intermediate parabolic subalgebras

𝔮_{c,A} ⊂ 𝔮′_{c,A} ⊂ 𝔮c,A^{′′} ⊂ 𝔤_{c}  (9.6.10) {eq:9.6.10}

with Levi components

\begin{matrix}𝔩′_{c,A} = u(a_{1}, b_{1}) × ⋯ × u(a_{r}, b_{r}) × 𝔤(n − a − b), \\ 𝔩c,A^{′′} = u(a, b) × 𝔤(n − a − b), \end{matrix} {(9.6.11) \atop \{eq:9.6.11\}}

Apply induction in stages from 𝔮_{c,A} to 𝔮′_{c,A} first. On the factor 𝔤(n − a − b) the K−type μ in (9.6.9) is trivial, so the Langlands parameter is that of the spherical principal series. Similarly on the u(a_{j}, b_{j}) assume the infinitesimal character is χ_{j} := (max(a_{j}, b_{j}), …, min(a_{j}, b_{j})) with the coordinates going down by 1, and the Langlands parameter is that of a principal series with the appropriate 1-dimensional Langlands subquotient. Let 𝔥_{A} ⊂ 𝔩′_{c,A} be the the most split Cartan subalgebra. In particular the real roots are

α_{d} := ϵ_{d}+ϵ_{d+p}, ∑_{j≤s} a_{j} < d < ∑_{j≤s} a_{j}+min(a_{j}, b_{j}), 0 ≤ s ≤ r−1. {(9.6.12) \atop \{eq:9.6.12\}}

For each factor u(a_{j}, b_{j}) the Langlands parameter is of the form (λ_{j}, ν_{j}) where λ_{j} ∈ 𝔥_{A} ∩ 𝔨_{c}, and ν_{j} ∈ 𝔥_{A} ∩ 𝔰_{c}. Then

λ_{j} = (1/2^{a_{j}} | 1/2^{b_{j}}), {(9.6.13) \atop \{eq:9.6.13\}}

while

⟨ν_{j}, α_{d}⟩ = max(a_{j}, b_{j}) − (d − ∑_{j≤s} a_{j}) {(9.6.14) \atop \{eq:9.6.14\}}

{p:9.6}

Proposition. The representations R_{𝔮_{c,A}}^{dim(𝔲_{c,A}∩𝔨_{c})}(ξ_{A}) have Langlands parameters (λ^{G}, ν) where λ^{G} is obtained by concatenating the λ_{j} in (9.6.13) and ν satisfies (9.6.14).

Proof. There is a nonzero map X_{𝔩′_{c}}(λ^{G}, −ν) ⟶ L_{𝔩′}(λ^{G}, −ν) given by the Langlands classification. Thus there is a map

R_{𝔮′_{c},L′∩K}^{dim 𝔨_{c}∩𝔲′_{c}}[X_{𝔩′}(λ^{G}, −ν)] ⟶
⟶ R_{𝔮′_{c},L′∩K}^{dim 𝔨_{c}∩𝔲′_{c}}(L_{𝔩′_{c}}(λ^{G}, −ν)) = R_{𝔮_{c},L∩K}^{dim 𝔨_{c}∩𝔲_{c}}(ξ_{A}),

(9.6.15) {eq:9.6.15}

which is nonzero on the bottom layer K−type (9.6.9). On the other hand, because these are standard modules,

R_{𝔮}^{i}(X_{𝔩′}(λ^{G}, ν)) = \begin{cases}X(λ^{G}, ν) & if i = dim 𝔨_{c} ∩ 𝔲_{c}, \\ 0 & otherwise. \end{cases}

(9.6.16) {eq:9.6.16}

The proof follows. □

{sec:9.7}

9.7.

Theorem. The spherical unipotent representations L(χǑ) are unitary.

Proof. Write 𝔤(n) for the Lie algebra containing O. There is a (real) parabolic subalgebra 𝔭^{+} with Levi component 𝔪^{+} := gl(n_{1}) × ⋯ × gl(n_{k}) × 𝔤(n) in 𝔤^{+} of rank n_{1} + ⋯ + n_{k} + n, such that the split form O_{spl}^{+} of

O_{c}^{+} := (1, 1, 3, 3, …, 2m − 1, 2m − 1, 2m + 1)

is induced from O on 𝔤(n), trivial on the gl’s. We will consider the representation

I(π) := Ind_{𝔪^{+}}^{𝔤^{+}}[triv ⊗ ⋯ ⊗ triv ⊗ π].  (9.7.1) {eq:9.7.1}

We show that the form on I(π) induced from π is positive definite; this implies that the form on π is definite. We do this by showing that the possible factors of I(π) have to be unitary, and the forms on their lowest K−types are positive definite.

Combining proposition 9.4 with (9.2.3), we conclude that there are 3^{m}·2^{m} unipotent representations in the block of the spherical irreducible representation; all the factors of I(π) are in this block. The number 3^{m} also equals the number of real forms of O^{+}. We describe how to get 3^{m} ·2^{m} representations. For each O_{j}^{+}, we produce one representation π such that AC(π) = \overline{O_{j}^{+}}. Then theorem 9.4 implies that there is a Harish-Chandra cell with 2^{m} representations with this property. Since these cells must be disjoint, this gives the required number.

¿From section 9.1, each such form O_{j}^{+} is θ-stable induced from the trivial nilpotent orbit on a parabolic subalgebra with Levi component a real form of gl(1) × gl(3) × ⋯ × gl(2m − 1) × 𝔤_{c}(m). Using the results in [KnV], for each such parabolic subalgebra, we can find a derived functor induced module from an appropriate 1-dimensional character, that is nonzero and has associated variety equal to the closure of the given real form. Actually it is enough to construct this derived functor module at regular infinitesimal character where the fact that it is nonzero irreducibile is considerably easier.

So in this block, there is a cell for each real form of O^{+}, and each cell has 2^{m} irreducible representations with infinitesimal character χǑ. In particular for O_{spl}, the Levi component is u(1, 0) × u(1, 2) × u(3, 2) × ⋯ × so(m, m + 1). For this case, section 9.6 produced exactly 2^{m} parameters; their lowest K−types are of the form μ_{e}(n−k, k). These are the only possible constituents of the induced from L(χǑ). Since the constituents of the restriction of a μ_{e}(n − k, k) to a Levi component are again μ_{e}(m − l, l)’s, the only way L(χǑ) can fail to be unitary is if the form is negative on one of the K−types μ_{e}(n − k, k). But sections 5 and 6.2 show that the form is positive on the K−types μ_{e} of L(χǑ). □

10. Irreducibility

                                                                       {sec:10}

                                                                       {sec:10.1}

10.1. To complete the classification of the unitary dual we also need to prove the following irreducibility theorem. It is needed to show that the regions in theorem 3.1 are indeed unitary in the real case.

{t:10.1}

Theorem. Assume Ǒ is even, and such that x_{i−1} = x_{i} = x_{i+1} for some i. Let 𝔪 = gl(x_{i})×𝔤(n−x_{i}), and Ǒ_{1} ⊂ 𝔤(n−x_{i}) be the nilpotent orbit obtained from Ǒ by removing two rows of size x_{i}. Then

L(χǑ) = Ind_{GL(x_{i})×G(n−x_{i})}^{G(n)}[triv ⊗ L(χǑ_{1})].

In the p−adic case this follows from the work of Kazhdan-Lusztig ([BM1]). In the real case, it follows from the following proposition.

{p:10.1}

Proposition. The associated variety of a spherical representation L(χǑ) is given by the sum with multiplicity one of the following nilpotent orbits.

Type B, D: On the odd sized rows, the difference between the number

of +’s and number of −’s is 1, 0 or -1. Type C: On the even sized rows, the difference between the number of

+’s and number of −’s is 1, 0 or -1.

The proof of the proposition is lengthy, and follows from more general results which are unpublished ([B5]). We will give a different proof of theorem 10.1 in the next sections.

Remark. When Ǒ_{1} is even, but Ǒ is not, and just x_{i} = x_{i+1}, the proof follows from [BM1] in the p−adic case, and the Kazhdan-Lusztig conjectures for nonintegral infinitesimal character in the real case. We have already used these results in the course of the paper. □

The outline of the proof is as follows. In section 2, we prove some auxiliary reducibility results in the case when Ǒ is induced from the trivial nilpotent orbit of a maximal Levi component. In section 3, we combine these results with intertwining operator techniques to complete the proof of theorem 10.1.

{sec:10.2}

10.2. We need to study the ρ−induced modules from the trivial module on 𝔪 ⊂ 𝔤(n) where 𝔪 ≅ gl(n), or in some cases 𝔪 ≅ gl(a)×𝔤(b) with a+b = n.

Type B. Assume Ǒ corresponds to the partition 2x_{0} = 2x_{1} = 2a in sp(n, ℂ). The infinitesimal character is (−a+1/2, …, a−1/2) and the nilpotent orbit O_{c} corresponds to (1, 1, \underbrace{2, …, 2}_{2a−2}, 3). We are interested in the composition

series of

Ind_{GL(2a)}^{G(2a)}[triv]. {(10.2.1) \atop \{eq:10.2.1o\}}

There are three real forms of O_{c} in so(2a + 1, 2a),

\begin{matrix}+ & − & + \\ + & − \\ − & + \\ ⋮ & ⋮ \\ + & − \\ − & + \\ + \\ + \end{matrix}  \begin{matrix}+ & − & + \\ + & − \\ − & + \\ ⋮ & ⋮ \\ + & − \\ − & + \\ + \\ − \end{matrix}  \begin{matrix}− & + & − \\ + & − \\ − & + \\ ⋮ & ⋮ \\ + & − \\ − & + \\ + \\ + \end{matrix} {(10.2.2) \atop \{eq:10.2.1\}}

The associated cycle of (10.2.1) is the middle nilpotent orbit in (10.2.2) with multiplicity 2. Section 6 shows that there are at least two factors characterized by the fact that they contain the petite K−types which are the restrictions to S[O(2a + 1) × O(2a)] of

\begin{matrix}μ(0^{a}; +) ⊗ μ(0^{a}; +) \\ μ(1, 0^{a−1}; −) ⊗ μ(0^{a}; −). \end{matrix}  (10.2.3) \{eq:10.2.3\}

Thus because of multiplicity 2, there are exactly two factors. One of the factors is spherical. The nonspherical factor has Langlands parameter

\begin{aligned}λ^{G} & = (1/2, 0, …, 0 | 0, …, 0), \\ ν & = (0, a − 1/2, a − 1/2, …, 3/2, 3/2, 1/2). \end{aligned}  (10.2.4) \{eq:10.2.1p\}

The Cartan subalgebra for the nonspherical parameter is such that the root ϵ_{1} is noncompact imaginary, ϵ_{i}, ϵ_{i} ±ϵ_{j} with j > i ≥ 2, are real. The standard module X(λ^{G}, ν) which has \overline{X}(λ^{G}, ν) as quotient is the one for which ν is dominant. Thus we conjugate the Cartan subalgebra such that ϵ_{2a} is noncompact imaginary, ϵ_{i}, ϵ_{i} ± ϵ_{j} with i < j < 2a are real, and the usual positive system Δ^{+} = {ϵ_{i}, ϵ_{i} ± ϵ_{j}}_{i<j}.

Type C. Consider Ǒ which corresponds to the partition 2x_{0} = 2x_{1} = 2a + 1 < 2x_{2} = 2b + 1 in so(n, ℂ). The infinitesimal character is

(−a, …, a)(−b, …, −1)  (10.2.5) {eq:10.2.2}

The nilpotent orbit O_{c} is induced from the trivial one on gl(2a + 1) × 𝔤(b) and corresponds to

(\underbrace{1, …, 1}_{2b−2a−2}, 2, 2, \underbrace{3, …, 3}_{2a}).  (10.2.6) \{eq:10.2.3a\}

We are interested in the composition series of

Ind_{GL(2a+1)×G(b)]}^{G(2a+b+1)}[triv].  (10.2.7) {eq:10.2.4a}

There are three real forms of (10.2.6),

+ − + ; − + − ; ⋮  ⋮ ; + − + ; − + − ; + − ; + − ; + ; − ; ⋮ ; + ; −  + − + ; − + − ; ⋮  ⋮ ; + − + ; − + − ; + − ; − + ; + ; − ; ⋮ ; + ; −  + − + ; − + − ; ⋮  ⋮ ; + − + ; − + − ; − + ; − + ; + ; − ; ⋮ ; + ; −  (10.2.8) {eq:10.2.4}

The AC cycle of (10.2.7) consists of the middle nilpotent orbit in (10.2.8) with multiplicity 2. By a similar argument as for type B, we conclude that the composition series consists of two representations containing the petite K−types

\begin{matrix}μ(0^{n}), \\ μ(1^{a+1}, 0^{b−1}, (−1)^{a+1}). \end{matrix} {(10.2.9) \atop \{eq:10.2.4c\}}

These are also the lowest K−types of the represntations. The nonspherical representation has parameter

\begin{matrix}λ^{G} = (1/2^{a}, 0^{b}, −1/2)^{a}), \\ ν = (1/2^{a}, 0^{b}, 1/2^{a}). \end{matrix} {(10.2.10) \atop \{eq:10.2.4b\}}

Type D. Let Ǒ correspond to the partition 2x_{0} = 2x_{1} = 2a + 1 in so(n, ℂ). The infinitesimal character is (−a, …, a). The real forms of the nilpotent orbit O are

\begin{matrix}+ & − \\ − & + \\ ⋮ & ⋮ \\ + & − \\ − & + \end{matrix} {(10.2.11) \atop \{eq:10.2.5\}}

There are two nilpotent orbits with this partition labelled I, II. Each of them is induced from 𝔪 ≅ gl(2a), there are two such Levi components. We are interested in the induced modules

Ind_{GL(2a)}^{G(2a)}[triv]. {(10.2.12) \atop \{eq:10.2.6\}}

The multiplicity of the nilpotent orbit (10.2.11) in the AC cycle of (10.2.12) is 1, so the representations are irreducible.

We summarize these calculations in a proposition.

{p:10.2}

Proposition. The composition factors of the induced module from the trivial representation on 𝔪 all have relevant lowest K−types. In particular, the induced module is generated by spherically relevant K−types. Precisely, Type B: the representation is generated by the K−types of the form μ_{e}, Type C: the representation is generated by the K−-types of the form μ_{o}, Type D: the representation is generated by μ_{e}(0) = μ_{o}(0).

                                                                       {sec10:3}

10.3. We now prove the irreducibility result mentioned at the beginning of the section in the case of 𝔤 of type B; the other cases are similar. Let Ǒ_{1} be the nilpotent orbit where we have removed one string of size 2a. Let 𝔪 := gl(2a) × 𝔤(n − 2a). Then L(χǑ) is the spherical subquotient of the induced representation

I(a, L(χǑ_{1})) := Ind_{𝔪}^{𝔤}[(−a + 1/2, …, a − 1/2) ⊗ L(χǑ_{1})].  (10.3.1) {eq:10.3.1}

It is enough to show that if a parameter is unipotent, and satisfies x_{i−1} = x_{i} = x_{i+1} = a, then I(a, L(χǑ_{1})) is generated by its K−types μ_{e}. This is because by theorem 5.3, the K−types of type μ_{e} in (10.3.1) occur with full multiplicity in the spherical irreducible subquotient, and the module is unitary.

First, we reduce to the case when there are no 0 < x_{j} < a. Let ν be the dominant parameter of L(χǑ), and assume i is the smallest index so that x_{i−1} = a. There is an intertwining operator

X(ν) ⟶ I(1/2, …, x_{0} − 1/2; …; 1/2, …, x_{i−2} − 1/2; ν′)  (10.3.2) {eq:10.3.2}

where I is induced from gl(x_{0})×⋯×gl(x_{i−2})×g(n−∑_{j<i−1} x_{j}) with characters on the gl’s corresponding to the strings in (10.3.2) and the irreducible module L(ν′) on 𝔤(n − ∑_{j<i−1} x_{j}). The intertwining operator is onto, and thus the induced module is generated by its spherical vector. By the induction hypothesis, the induced module from (−a + 1/2, …, a − /2) ⊗ L(ν′′) on gl(2a) × 𝔤(n − ∑_{j≤i} x_{j}) is irreducible. But

\begin{matrix}I(1/2, …, x_{0} − 1/2; …; 1/2, …, x_{i−2} − 1/2; −a + 1/2, …, a − 1/2; ν′′) ≅ \\ I(−a + 1/2, …, a − 1/2; 1/2, …, x_{0} − 1/2; x …; 1/2, …, x_{i−2} − 1/2, ν′′) \end{matrix}

(10.3.3) {eq:10.3.3}

This module maps by an intertwining operator onto I(a, L(χǑ_{1})), so I(a, L(χǑ_{1})) is generated by its spherical vector.

So we have reduced to the case when

\begin{matrix}x_{0} = x_{1} = x_{2} = a,  or \\ x_{0} = 0 < x_{1} = x_{2} = x_{3} = a. \end{matrix}  (10.3.4) \{eq:10.3.4\}

Suppose we are in the first case of (10.3.4) and m = 1. The infinitesimal character is

(a − 1/2, a − 1/2, a − 1/2, …, 1/2, 1/2, 1/2),

each coordinate occuring three times. The induced module

I(−a + 1/2, …, a − 1/2)  (10.3.5) {eq:10.3.5}

of 𝔤(2a) is a direct sum of irreducible factors computed in section 10.2; in particular it is generated by K−types of the form μ_{e}(2a − k, k) (with k = 0, 1). Consider the module

I(a − 1/2; …; 1/2; −a + 1/2, …, a − 1/2), {(10.3.6) \atop \{eq:10.3.6\}}

induced from characters on GL(1) × ⋯ × GL(1) × GL(2a). It is a direct sum of induced modules from the two factors of (10.3.5). Each such induced module is a homomorphic image of the corresponding standard module with dominant parameter. So (10.3.6) is also generated by its μ_{e} isotypic components. But then

\begin{matrix}I(a − 1/2; …; 1/2; −a + 1/2, …, a − 1/2) ≅ \\ I(−a + 1/2, …, a − 1/2; a − 1/2; …; 1/2) \end{matrix} {(10.3.7) \atop \{eq:10.3.7\}}

so the latter is also generated by its μ_{e} isotypic components. Finally, the intertwining operator

I(a − 1/2; …; 1/2) ⟶ I(1/2, …, a − 1/2) {(10.3.8) \atop \{eq:10.3.8\}}

is onto, and the image of the intertwining operator

I(1/2, …, a − 1/2) ⟶ I(−a + 1/2, …, −1/2) {(10.3.9) \atop \{eq:10.3.9\}}

is onto L(−a+1/2, …, −1/2). Thus the module induced from gl(2a)×𝔤(a),

I(−a + 1/2, …, a − 1/2; L(−a + 1/2, …, −1/2)), {(10.3.10) \atop \{eq:10.3.10\}}

is generated by its μ_{e} isotypic components. Since the multiplicity of these K−types in (10.3.10) is the same as in the irreducible spherical module, it follows that they must be equal.

Now suppose that we are in the first case of (10.3.4) and m > 1, or in the second case, and m > 2. The parameter has another x_{2m−1} ≤ x_{2m}. We use an argument similar to the one above to show that the module

I(−x_{2m−1} + 1/2, …, x_{2m} − 1/2, L(χǑ_{2})), {(10.3.11) \atop \{eq:10.3.11\}}

where Ǒ_{2} is the nilpotent orbit with partition obtained from Ǒ by removing 2x_{2m−1}, 2x_{2m}, is generated by its μ_{e} isotypic components. The claim then follows because the induced module is a homomorphic image of (10.3.11). Precisely, X(ν) maps onto

\begin{matrix}I(x_{2m−1} + 1/2, …, & x_{2m} − 1/2; 1/2, …, x_{0} − 1/2; …; 1/2, …, x_{2m−2} − 1/2; \\ L(−x_{2m−1} + 1/2, −x_{2m−1} + 1/2, …, −1/2, −1/2)) \end{matrix}

(10.3.12) {eq:10.3.12} So this module is generated by its spherical vector. Replace L(−x_{2m−1} + 1/2, −x_{2m−1} + 1/2, …, −1/2, −1/2) by I(−x_{2m−1} + 1/2, …, x_{2m−1} − 1/2). The ensuing module is a direct sum of two induced modules by section 10.2.

They are both homomorphic images of standard modules, so generated by their lowest K−types, which are of type μ_{e}. Next observe that the map

I(x_{2m−1} + 1/2, …, x_{2m} − 1/2;1/2, …, x_{0} − 1/2; …; 1/2, …, x_{2m−2} − 1/2;
− x_{2m−1} + 1/2, …, x_{2m−1} − 1/2) ⟶
I(−x_{2m−1} + 1/2, … x_{2m} − 1/2;1/2, …, x_{0} − 1/2; …; 1/2, …, x_{2m−2} − 1/2)

(10.3.13) {eq:10.3.13} is onto. So the target module is generated by its μ_{e} isotypic components. The module

I(1/2, …, x_{0} − 1/2; …; 1/2, …, x_{2m−2} − 1/2)  (10.3.14) {eq:10.3.14}

(the string (−x_{2m−1}+1/2, …, x_{2m}−1/2) has been removed) has L(−x_{2m−2}+ 1/2, …, 1/2) as its unique irreducible quotient, because it is the homomorphic image of an X(ν) with ν dominant. Therefore it is generated by its spherical vector. Combining this with the induction assumption, we conclude that

I(−x_{2m−1} + 1/2, …, x_{2m} − 1/2; −a + 1/2, …, a − 1/2; L(Ǒ_{3})) (10.3.15) {eq:10.3.15}

is generated by its μ_{e} isotypic components. It is isomorphic to

I(−a + 1/2, …, a − 1/2; −x_{2m−1} + 1/2, …, x_{2m} − 1/2; L(Ǒ_{3})). (10.3.16) {eq:10.3.16}

Finally, the multiplicities of the μ_{e} isotypic components of I(−x_{2m−1} + 1/2, …, x_{2m} − 1/2; L(Ǒ_{3})) are the same as for the irreducible subquotient L(Ǒ_{1})). This completes the proof of the claim in this case.

Remains to consider the case when m = 2 and x_{0} = 0 < x_{1} = x_{2} = x_{3} = a ≤ x_{4}. In this case, the module

I(a+1/2, …, x_{4}−1/2; −a+1/2, …, a−1/2; −a+1/2, …, a−1/2) (10.3.17) {eq:10.3.17}

is generated by its μ_{e} isotypic components because of proposition 10.2, and arguments similar to the above. Therefore the same holds for

I(−a + 1/2, …, x_{4} − 1/2; −a + 1/2, …, a − 1/2),  (10.3.18) {eq:10.3.18}

which is a homomorphic image via the intertwining operator which interchanges the first two strings. But this is isomorphic to

I(−a + 1/2, …, a − 1/2, −a + 1/2, …, x_{4} − 1/2).  (10.3.19) {eq:10.3.19}

Then I(−a + 1/2, …, a − 1/2, L(−x_{4} + 1/2, …, −1/2, −1/2) is a homomorphic image of (10.3.19) so it is generated by its μ_{e} isotypic components. By section 5.3, the multiplicities of the μ_{e} isotypic components are the same in I(−a + 1/2, …, a − 1/2, L(−x_{4} + 1/2, …, −1/2, −1/2)) as in L(χǑ).

This completes the proof of theorem 10.1. □

References

[ABV] J. Adams, D. Barbasch, D. Vogan, The Langlands classification and irreducible characters of real reductive groups, Progress in Mathematics, Birkhäuser, Boston-Basel-Berlin, (1992), vol. 104.

[BB] D. Barbasch, M. Bozicevic The associated variety of an induced representationproceedings of the AMS 127 no. 1 (1999), 279-288

[B1] D. Barbasch, The unitary dual of complex classical groups, Inv. Math. 96 (1989), 103–176.

[B2] D. Barbasch, Unipotent representations for real reductive groups, Proceedings of ICM, Kyoto 1990, Springer-Verlag, The Mathematical Society of Japan, 1990, pp. 769–777.

[B3] D. Barbasch, The spherical unitary dual for split classical p−adic groups, Geometry and representation theory of real and p−adic groups (J. Tirao, D. Vogan, and J. Wolf, eds.), Birkhauser-Boston, Boston-Basel-Berlin, 1996, pp. 1–2.

[B4] D. Barbasch, Orbital integrals of nilpotent orbits , Proceedings of Symposia in Pure Mathematics, vol. 68, (2000) 97-110.

[B5] D. Barbasch, The associated variety of a unipotent representation preprint

[B6] D. Barbasch Relevant and petite K−types for split groups, Functional Analysis VIII, D. Bakić et al, Various publication series no 47, Ny Munkegade, bldg530, 800 Aarhus C, Denmark.

[B7] D. Barbasch A reduction theorem for the unitary dual of U(p,q) in volume in honor of J. Carmona, Birkhäuser, 2003, 21-60

[B] A. Borel Admissible representations of a semi-simple group over a local field with vectors fixed under an Iwahori subgroup , Inv. Math., vol. 35, 233-259, 1976

[BC1] D. Barbasch, D. Ciubotaru Spherical unitary principal series, preprint to appear in Quarterly Journal of Mathematics.

[BC2] Spherical unitary dual for exceptional groups of type E, preprint

[BM1] D. Barbasch and A. Moy A unitarity criterion for p−adic groups, Inv. Math. 98 (1989), 19–38.

[BM2] , Reduction to real infinitesimal character in affine Hecke algebras, Journal of the AMS 6 no. 3 (1993), 611-635.

[BM3] , Unitary spherical spectrum for p−adic classical groups, Acta Applicandae Math 5 no. 1 (1996), 3-37.

[BS] D. Barbasch, M. Sepanski Closure ordering and the Kostant-Sekiguchi correspondence, Proceedings of the AMS 126 no. 1 (1998), 311-317.

[BV1] D. Barbasch, D. Vogan The local structure of characters J. of Funct. Anal. 37 no. 1 (1980) 27-55

[BV2] D. Barbasch, D. Vogan Unipotent representations of complex semisimple groups Ann. of Math., 121, (1985), 41-110

[BV3] D. Barbasch, D. Vogan Weyl group representation and nilpotent orbits Representation theory of reductive groups (Park City, Utah, 1982), Progr. Math., 40, Birkhuser Boston, Boston, MA, (1983), 21-33.

[CM] D. Collingwood, M. McGovern Nilpotent orbits in semisimple Lie algebras, Van Nostrand Reinhold Co., New York, (1993).

[D] D. Djokovic Closures of conjugacy classes in classical real linear Lie groups II Trans. Amer. Math. Soc. 270 no. 1, (1982), 217-252.

[K] A. Knapp Representation theory of real semisimple groups: an overview based on examples Princeton University Press, Princeton, New Jersey (1986)

[KnV] A. Knapp, D. Vogan Cohomological induction and unitary representations Princeton University Press, Princeton Mathematical Series vol. 45, 1995.

[L1] G. Lusztig Characters of reductive groups over a finite field Annals of Math. Studies, Princeton University Press vol. 107.

[LS] G. Lusztig, N. Spaltenstein Induced unipotent classes J. of London Math. Soc. (2), 19 (1979), 41-52

[McG] W. McGovern Cells of Harish-Chandra modules for real classical groups Amer. Jour. of Math., 120, (1998), 211-228.

[SV] W. Schmid, K. Vilonen Characteristic cycles and wave front cycles of representations of reductive groups, Ann. of Math., 151 (2000), 1071-1118.

[Stein] E. Stein Analysis in matrix space and some new representations of SL(n, ℂ) Ann. of Math. 86 (1967) 461-490

[T] M. Tadic Classification of unitary representations in irreducible representations of general linear groups, Ann. Sci. École Norm. Sup. (4) 19, (1986) no. 3, 335-382.

[V1] D. Vogan The unitary dual of GL(n) over an archimedean field, Inv. Math., 83 (1986), 449-505.

[V2] Irreducible characters of semisimple groups IV Duke Math. J. 49, (1982), 943-1073

[V3] Representations of real reductive Lie groups Progress in Mathematics, vol. 15, (1981) , Birkhäuser, Boston-Basel-Stuttgart

[Weyl] H. Weyl The classical groups: Their invariants and Representations, Princeton Landmarks in Mathematics, Princeton University Press, Princeton NJ, 1997

[ZE] A. Zelevinsky Induced representations of reductive 𝔭-adic groups II. On irreducible representations of GL(n), Ann. Sci. cole Norm. Sup. (4) 13 no. 2 165-210

(D. Barbasch) Dept. of Mathematics, Cornell University, Ithaca, NY 14850

   E-mail address: barbasch@math.cornell.edu