### Cayley-Hamilton theorem

This is to detail a much easier proof (at least, I find it so) of Cayley-Hamilton than the ones which appear on the Wikipedia page. It only applies in the case of complex vector spaces; most of the post is taken up with a proof of a lemma about complex matrices that is very useful in many contexts.

The idea is as follows: given an arbitrary square matrix, upper-triangularise it (looking at it in basis $$B$$). Then consider how $$A-\lambda I$$ acts on the vectors of $$B$$; in particular, how it deals with the subspace spanned by $$b_1, \dots, b_i$$.

# Lemma: upper-triangulation

Given a square matrix $$A$$, there is a basis with respect to which $$A$$ is upper-triangular.

Proof: by induction. It’s obviously true for $$1 \times 1$$ matrices, as they’re already triangular. Now, let’s take an arbitrary $$n \times n$$ matrix $$A$$. We want to make it upper-triangular. In particular, thinking about the top-left element, we need $$A$$ to have an eigenvector (since if $$A$$ is upper-triangular with respect to basis $$B$$, then $$A(b_1) = \lambda b_1$$, where $$\lambda$$ is the top-left element). OK, let’s grab an eigenvector $$v_1$$ with eigenvalue $$\lambda$$.

We’d love to be done by induction at this point - if we extend our eigenvector to a basis, that extension itself forms a smaller space, on which $$A$$ is upper-triangulable. We have that every subspace has a complement, so let’s call pick a complement of $$\text{span}(v_1)$$ and call it $$W$$.

Now, we want $$A$$ to be upper-triangulable on $$W$$. It makes sense, then, to restrict it to $$W$$ - we’ll call the restriction $$\tilde{A}$$, and that’s a linear map from $$W$$ to $$V$$. Our inductive hypothesis requires a square matrix, so we need to throw out one of the rows of this linear map - but in order that we’re working with an endomorphism (rather than just a linear map) we need $$A$$’s domain to be $$W$$. That means we have to throw out the top row as well - that is, we compose with $$\pi$$ the projection map onto $$W$$.

Then $$\pi \cdot \tilde{A}$$ is $$(n-1)\times(n-1)$$, and so we can induct to state that there is a basis of $$W \leq V$$ with respect to which $$\pi \cdot \tilde{A}$$ is upper-triangular. Let’s take that basis of $$W$$ as our extension to $$v_1$$, to make a basis of $$V$$. (These are $$n-1$$ length-$$n$$ vectors.)

Then we construct $$A$$’s matrix as $$A(v_1), A(v_2), \dots, A(v_n)$$. (That’s how we construct a matrix for a map in a basis: state where the basis vectors go under the map.)

Now, with respect to this basis $$v_1, \dots, v_n$$, what does $$A$$ look like? Certainly $$A(v_1) = \lambda v_1$$ by definition. $$\pi(A(v_2)) = \pi(\tilde{A}(v_2))$$ because $$\tilde{A}$$ acts just the same as $$A$$ on $$W$$; by upper-triangularity of $$\pi \cdot \tilde{A}$$, we have that $$\pi \cdot \tilde{A}(\pi(v_2)) = k v_2$$ for some $$k$$. The first element (the $$v_1$$ coefficient) of $$A(v_2)$$, who knows? (We threw that information away by taking $$\pi$$.) But that doesn’t matter - we’re looking for upper-triangulability rather than diagonalisability, so we’re allowed to have spare elements sitting at the top of the matrix.

And so forth: $$A$$ is upper-triangular with respect to some basis.

## Note

Remember that we threw out some information by projecting onto $$W$$. If it turned out that we didn’t throw out any information - if it turned out that if we could always “fill in with zeros” - then we’d find that we’d constructed a basis of eigenvectors, and that the matrix was diagonalisable. (This is how the two ideas are related.)

# Theorem

Recall the statement of the theorem:

Every square matrix satisfies its characteristic polynomial.

Now, this would be absolutely trivial if our matrix $$A$$ were diagonalisable - just look at it in a basis with respect to which $$A$$ is diagonal (recalling that change-of-basis doesn’t change characteristic polynomial), and we end up with $$n$$ simultaneous equations which are conveniently decoupled from each other (by virtue of the fact that $$A$$ is diagonal).

We can’t assume diagonalisability - but we’ve shown that there is something nearly as good, namely upper-triangulability. Let’s assume (by picking an appropriate basis) that $$A$$ is upper-triangular. Now, let’s say the characteristic polynomial is $$\chi(x) = (x - \lambda_1)(x-\lambda_2) \dots (x-\lambda_n)$$. What does $$\chi(A)$$ do to the basis vectors?

Well, let’s consider the first basis vector, $$e_1$$. We have that $$A(e_1) = \lambda_1 e_i$$ because $$A$$ is upper-triangular with top-left element $$\lambda_1$$, so we have $$(A-\lambda_1 I)(e_1) = 0$$. If we look at the characteristic polynomial as $$(x-\lambda_n)\dots (x-\lambda_1)$$, then, we see that $$\chi(A)(e_1) = 0$$.

What about the second basis vector? $$A(e_2) = k e_1 + \lambda_2 e_2$$; so $$(A - \lambda_2 I)(e_2) = k e_1$$. We’ve pulled the $$2$$nd basis vector into an earlier-considered subspace, and happily we can kill it by applying $$(A-\lambda_1 I)$$. That is, $$\chi(A)(e_2) = (A-\lambda_n I)\dots (A-\lambda_1 I)(A-\lambda_2 I)(e_2) = (A-\lambda_n I)\dots (A-\lambda_1 I) (k e_1) = 0$$.

Keep going: the final case is the $$n$$th basis vector, $$e_n$$. $$A-\lambda_n I$$ has a zero in the bottom-right entry, and is upper-triangular, so it must take $$e_n$$ to the subspace spanned by $$e_1, \dots, e_{n-1}$$. Hence $$(A-\lambda_1 I)\dots (A-\lambda_n I)(e_n) = 0$$.

Since $$\chi(A)$$ is zero on a basis, it must be zero on the whole space, and that is what we wanted to prove.