LECTURE NOTES 10/13/05

JOSHUA P. BOWMAN

1. Abstract vector spaces

What is a vector? Loose definition we’ve been using so far: an object that can be added and scalar multiplied. Even this makes clear that the definition of a vector requires a context, i.e., a vector space. The examples we’ve seen:

• ℝ^{n}

• Mat_{k×n}

Here are other examples of sets where we can add and multiply by real numbers:

• C[0, 1], the set of continuous functions on the interval [0, 1]. To add two functions together, we just add their values at each point, and likewise for scalar multiplication:

(f + g)(x) = f(x) + g(x), (cf)(x) = c(f(x)) for all x ∈ [0, 1].

We know that adding two continuous functions gives another continuous function, and multiplying a function by a real number doesn’t change its continuity.

• ℓ^{0}, the set of sequences (functions ℕ → ℝ).

(a_{0}, a_{1}, a_{2}, …) + (b_{0}, b_{1}, b_{2}, …) = (a_{0} + b_{0}, a_{1} + b_{1}, a_{2} + b_{2}, …),

c(a_{0}, a_{1}, a_{2}, …) = (ca_{0}, ca_{1}, ca_{2}, …).

• P oly_{x}^{3}, the set of polynomials in x with degree at most 3. To add, we add coefficients, and likewise to multiply by a real. The sum of two such polynomials cannot have degree more than 3, so P oly_{x}^{3} is closed under addition.

How can we talk about all these different things at once, and see what’s the same about them? They are unified by the concept of a vector space.

Definition 1.1. A (real) vector space is a set V with addition and scalar multiplication defined that satisfy the following axioms:

(1)  ∃ 0 ∈ V such that ∀ v ∈ V, v + 0 = v  (identity)
(2)  ∀ v ∈ V, ∃ −v ∈ V such that v + (−v) = 0  (inverses)
(3)  ∀ v, w ∈ V, v + w = w + v  (commutativity)
(4) ∀ u, v, w ∈ V, c, d ∈ ℝ, \{ \begin{matrix}(u + v) + w = u + (v + w) \\ c(dv) = (cd)v \end{matrix}  (associativity laws)

(5)  ∀ v, w ∈ V, c, d ∈ ℝ, \{ \begin{matrix}c(v + w) = cv + cw \\ (c + d)v = cv + dv \end{matrix}  (distributive laws)
(6)  ∀ v ∈ V, 1v = v  (normalization)

These axioms are chosen so that almost everything you know about ℝ^{n} is true for any vector space. For example,

• A subspace of a vector space W is a subset W that is also itself a vector space; when adding and scalar multiplying things in W, you don’t go outside of W. Looking at the examples above, we see that P oly_{x}^{3} is a subspace of C[0, 1], because a polynomial is certainly continuous on [0, 1]. You should convince yourself that the addition and scalar multiplication we’ve defined for P oly_{x}^{3} is the same as the one for C[0, 1].

• Linear combination and span work just as before. A function in P oly_{x}^{3} is a linear combination of the functions 1, x, x^{2}, and x^{3}; these four functions span P oly_{x}^{3}. The two sequences (1, 0, 1, 0, 1, 0, …) and (0, 1, 0, 1, 0, 1, …) in ℓ^{0} span the set of sequences whose terms have alternating values.

• A finite set of vectors {v_{1}, …, v_{n}} is linearly independent if

c_{1}v_{1} + ⋯ + c_{n}v_{n} = 0 ⟹ c_{1} = ⋯ = c_{n} = 0.

A non-zero polynomial has only finitely many points on which it equals zero, so a + bx + cx^{2} + dx^{3} ∈ P oly_{x}^{3} can only equal 0 for all x if a = b = c = d = 0. Thus {1, x, x^{2}, x^{3}} is a linearly indepedent spanning set for P oly_{x}^{3}—hey, we have a basis!

As the last example above suggests, bases also work as before, and the “dimension” of a space is simply a measure of the size of a basis. Unfortunately, in “infinite-dimensional” spaces, some of what we know breaks down. This is mostly due to the fact that linear combinations must be finite. For example, in ℓ^{0}, let

e_{1} = (1, 0, 0, 0, …)
e_{2} = (0, 1, 0, 0, …)
e_{3} = (0, 0, 1, 0, …)
⋮

These vectors are linearly independent, because obviously

∑_{j=1}^{k} a_{j}e_{i_{j}} = (0, 0, 0, …) ⟺ a_{1} = ⋯ = a_{k} = 0.

But the vectors {e_{i}} do not span ℓ^{0}, because any linear combination of them will only have finitely many nonzero terms.

You may ask, why don’t we simply allow infinite linear combinations? The reason is, once we write something like

∑_{i=1}^{∞} a_{i}e_{i},

we’re no longer talking about a sum; we’re talking about a limit, which means we need a way of measuring convergence. An abstract vector space doesn’t come equipped with such a thing. There are infinite-dimensional spaces, called Hilbert spaces, in which one can make sense of the above expression, and these are a possible topic for the term project. We will not deal with questions of basis in infinite-dimensional spaces in this class.

2. Finite-dimensional spaces

Okay, let’s step back from seeing what can go wrong. We should at least be able to understand completely our usual vector space notions when the dimension is finite. And such is the case, by the following proposition:

Proposition 2.1. A vector space V is finite-dimensional, with dimension n, iff there exists a linear map Φ : ℝ^{n} → V that is 1-to-1 and onto. Moreover, such a map sends a basis of ℝ^{n} to a basis of V.

(The book calls Φ a “concrete-to-abstract” function.) The point of the above proposition is that a vector space that can be spanned by only finitely many elements can be thought of as ℝ^{n} in a very specific way. The map Φ tells us exactly which vectors in V to think of as our basis.

Let’s look at our standby example, P oly_{x}^{3}. We’ve already seen that {1, x, x^{2}, x^{3}} is a basis for this space, so the first map ℝ^{4} → P oly_{x}^{3} that should come to mind is

Φ : [\begin{matrix}a \\ b \\ c \\ d \end{matrix}] ↦ a + bx + cx^{2} + dx^{3}.

This almost looks stupid: it looks like the basis we’ve chosen for P oly_{x}^{3} is just acting as “place-holders” for the ℝ^{4} coordinates. That’s what Φ does, in a sense: once we’ve chosen a set of linearly independent vectors in our abstract space, Φ associates them to the standard basis in ℝ^{n}. Once we’ve done that, we can treat V (or whatever subspace is spanned by the vectors we chose) just like ℝ^{n}.

Let’s look at another example, which allows us to exploit the “bigness” of ℓ^{0}: As before, let e_{i} ∈ ℓ^{0} be the sequence that is zero everywhere except in the (i − 1)st term, which is 1. Fix n, and define Φ : ℝ^{n} → ℓ^{0} by

Φ : [a_{1} ; a_{2} ; ⋮ ; a_{n}] ↦ a_{1}e_{1} + a_{2}e_{2} + ⋯ + a_{n}e_{n} = (a_{1}, a_{2}, …, a_{n}, 0, 0, …).

Thus, we can think of ℝ^{n} as being the subspace of ℓ^{0} composed of sequences that have 0 for the nth and higher terms. That is, ℓ^{0} contains a copy of ℝ^{n} for every n! This gives us a useful and accurate way to say ℝ^{m} ⊂ ℝ^{n} when m ≤ n: just tack on enough zeroes to vectors in ℝ^{m} to make them have n entries.

3. Linear transformations

The axioms for a vector space also allow us to define linear transformations between vector spaces exactly as we always have. To emphasize:

Definition 3.1. Let V and W be vector spaces. A map T : V → W is linear iff

(7)  T (v + w) = T (v) + T (w)  for all v, w ∈ V
(8)  and T (cv) = cT (v)  for all v ∈ V, c ∈ ℝ.

We’ve already seen examples in the case of the “concrete-to-abstract” functions. Here’s one that gives a name to basic properties of the derivative that you’ve known for ages:

Example 3.2. Let C^{1}(0, 1) be the space of continuously differentiable functions on (0, 1). (This space, C[0, 1], and other spaces whose elements are functions are sometimes called function spaces.) Define D : C^{1}(0, 1) → C(0, 1) by

D(f) = f′.

Recall that (f + g)′ = f′ + g′, and (cf)′ = cf′. Hence D, the differentiation map, is linear!

The kernel of a linear map is, as before, a subspace of the domain. In the previous example, the kernel is the set of constant functions, which is certainly a subspace: it’s spanned by the function 1. But D is also onto: every continuous function has an antiderivative. Thus we don’t have an analogue of the dimension formula (without more careful definitions, which we won’t go into here). Once again, the problem is that C^{1}(0, 1) and C(0, 1) are infinite-dimensional (for example, they both contain all the polynomials). One of next week’s exercises will probably ask you to think a bit more about this example, and to write a matrix for D just on the finite-dimensional space of polynomials of degree d or less.

4. Inner products

So what about length, orthogonality, and all that? The definition of a vector space doesn’t give us any way to handle these things. And sometimes there’s just no reasonable way to define them. We need some extra structure. Here’s an example that generalizes ℝ^{n}.

Example 4.1 (An important subspace of ℓ^{0}). Let’s use the sequence structure of ℓ^{0} to define the following:

ℓ^{2} = {(a_{i})_{i=0}^{∞} ∈ ℓ^{0} | ∑_{i=0}^{∞} |a_{i}|^{2} < ∞} .

This is called the space of square summable sequences. To show that it is a subspace of ℓ^{0}, we need to show that it’s closed under addition and scalar multiplication. Scalar multiplication is easy: if (a_{i}) ∈ ℓ^{2}, c ∈ ℝ, then

∑_{i=0}^{∞} |ca_{i}|^{2} = ∑_{i=0}^{∞} |c|^{2}|a_{i}|^{2} = |c|^{2} ∑_{i=0}^{∞} |a_{i}|^{2} < ∞.

For addition, we first recall that |ab| ≤ \frac{1}{2}(a^{2} + b^{2}). Thus, if (a_{i}), (b_{i}) ∈ ℓ^{2},

∑_{i=0}^{∞} |a_{i}+b_{i}|^{2} ≤ ∑_{i=0}^{∞} (|a_{i}|^{2} + 2|a_{i}b_{i}| + |b_{i}|^{2}) ≤ ∑_{i=0}^{∞} (2|a_{i}|^{2} + 2|b_{i}|^{2}) = 2 ∑_{i=0}^{∞} |a_{i}|^{2}+2 ∑_{i=0}^{∞} |b_{i}|^{2} < ∞.

If (a_{i}) only had n non-zero terms, then ∑_{i=0}^{∞} |a_{i}|^{2} would just be the square of the length of (a_{i}) as a vector in ℝ^{n}. Since the length in ℝ^{n} is defined by the dot product, we’d like to extend the dot product to ℓ^{2}. For (a_{i}), (b_{i}) ∈ ℓ^{2},

⟨(a_{i}), (b_{i})⟩ = ∑_{i=0}^{∞} a_{i}b_{i}

is called the inner product (or sometimes dot product) of (a_{i}) and (b_{i}). With it, the notions of length, orthogonality, and convergence once again become accessible. ℓ^{2} is a Hilbert space.