Linear Algebra

0 - Vector Spaces, Linear Maps, Rank, Kernel

Definition (Vector space): A set $V$ and a field $F$ with $(V,+)$ being an abelian group, and a scalar multiplication $(\cdot , \cdot) : F \times V \longrightarrow V$ That is associative, with $\mathbb 1 \in F$ producing the identity map $\mathbb 1 v = v$, and distributive laws both ways between $+$ on $V$ and $+$ on $F$, and the scalar multiplication of a product in $F$ is the composition of the scalar multiplications. Elements in $V$ are called vectors, and elements of $F$ are called scalars. $U \subset V$ is a subspace of $V$ if $U$ is closed under vector addition and scalar multiplication, inherited from $V$, and also contains the $0$ vector.

I don’t know why I wrote the definition like this, maybe I am lazy to type

Definition (Linear combination): Let $V$ over $F$ be a vector space, $<\mathbf{v_{i}>}$ a list of vectors, and $<a_{i}>$ a list of scalars. The linear combination of these lists is a vector $\mathbf{u} = \sum_{i=1}^n a_{i} \mathbf{v_{i}}$. The set of all linear combinations of $<\mathbf{v_{i}}>$ is called $\text{Span}(<\mathbf{v_{i}}>)$ and is a subspace of $V$, as choosing all scalars to be zero gives the zero vector in the span, and also by definition, the span is closed under both operations. if there exists a list of scalars $<a_i>$ not all zero for which $\mathbf{0} =\sum_{i=1}a_{i} \mathbf{v_{i}}$, then the list of vectors $<\mathbf{v_{i}}>$ is called linearly dependent. On the other hand if $\mathbf{0} =\sum_{i=1}a_{i} \mathbf{v_{i}}$ if and only if each $a_i = 0$, then the list of vectors is linearly independent. The dimension of a vector space is the length of the smallest list of vectors that spans it. The same applies for subspaces. for a linearly dependent list of vectors, the dimension of the subspace induced by its span is at most the length of the list minus one, as one of these vectors is already in the span, so we need not include it in the list, and we can still span this subspace. if $V$ is a vector space, any smallest list of linearly independent vectors that span $V$ are called “basis” vectors.

Definition (Linear map): if $U,V$ are vector spaces, (on fields $F_U, F_V$) then a function $f: U \rightarrow V$ is a linear map if:

• $f(\mathbf{u_{1}+ u_{2}}) = f(\mathbf{u_{1}}) +f(\mathbf{u_{2}})$
• $F_U = F_V$ (upto isomorphism).
• $f(\lambda \mathbf{u}) = \lambda f(\mathbf{u})$

The linear map is a homomorphism between two vector spaces. let $<\mathbf{e_{i}}>$ be a basis list of $U$. Then for any vector $\mathbf{v} \in \text{RANGE}(f)$ we have $\mathbf{u} = \sum_{i=1}^na_{i}\mathbf{e_{i}}$ such that $f(\mathbf{u}) = \mathbf{v}$ Therefore, $f\left( \sum_{i=1}^na_{i}\mathbf{e_{i}} \right) =\sum_{i=1}^n a_{i}f(\mathbf{e_{i}})$. This allows us to write a matrix

$$M = \begin{bmatrix} \uparrow \\ f(\mathbf{e_{1}} ) \dots f(\mathbf{e_{n}}) \\ \downarrow \end{bmatrix}$$

Whose columns are the image of each basis, and when we write $\mathbf{u}$ as a column vector $[a_{1},a_{2}, \dots ,a_{n}]^T$, with components wrt the same basis, The application $f(\mathbf{u})$ is equivalent to the matmul $M \mathbf{u}$.

• Also $f(U)$ (also called $\text{RANGE}(f)$) is a subspace of $V$, simple due to the structure preserving nature of the linear map: if $f(\mathbf{u}) \in f(U)$, then $\lambda f(\mathbf{u}) = f(\lambda \mathbf{u}) \in f(U)$. Similiary, if $f(\mathbf{u_{1}}), f(\mathbf{u_{2}}) \in f(U)$, then $f(\mathbf{u_{1}}) + f(\mathbf{u_{2}}) = f(\mathbf{u_{1} + u_{2}}) \in f(U)$. Zero vector is in there too.

Hence, we are concerned about the dimension of $f(U) \le dim(V)$. To see if our linear map $f$ squashes the space $U$ into a lower dimension than $U$.

Definition (Rank of a linear map): Let $U$ be a vector space with $dim(U) = n$, and $<\mathbf{e_{i}}>$ a basis of $U$, and let $V$ also be a vector space, both over a scalar field $F$. then the rank of the linear map $f: U \rightarrow V$ is the dimension of $f(U)$. By definition, the list $<f(\mathbf{e_{i}})>$ spans $f(U)$ and has length $n$. Therefore, $dim(f(U))$ is at most $n$ ( We may find smaller spanning lists for $f(U)$ ).
In other words, $\text{rank}(f) \le dim(U)$.

Definition (Kernel of a linear map): for linear map $f: U \rightarrow V$, the kernel of $f$, is the subset of $U$ that maps to the zero vector in $V$. $\text{Kern(f)} \subset U$ such that for all $\mathbf{u } \in \text{Kern}(f)$, $f(\mathbf{u}) = \mathbf{0} \in V$. We notice that $\text{Kern}(f)$ is a subspace of $U$. if $f(\mathbf{u}) = \mathbf{0}$ then, $f(\mathbf{\lambda u}) = \lambda f(\mathbf{u}) = \mathbf{0}$ . Similarly, if $f(\mathbf{u_{1}}) = \mathbf{0}$ and $f(\mathbf{u_{2}}) = \mathbf{0}$, then $f(\mathbf{u_{1} + u_{2}}) = f(\mathbf{u_{1}})+ f(\mathbf{u_{2}}) = \mathbf{0}$. Finally, notice that $f(\mathbf{ 0 + 0}) = f(\mathbf{0}) + f(\mathbf{0})$ hence $f(\mathbf{0}) = f(\mathbf{0}) + f(\mathbf{0})$, but $f(\mathbf{0})$ has some additive inverse in $V$, therefore $\mathbf{0} \in V = f(\mathbf{0})$.

---

Linear Dependence Lemma

We need some nice spanning list and linear independent list lemmas from LADR for this one

Lemma 2.19 (Linear Dependence Lemma): Suppose $v_1, \ldots, v_m$ is a linearly dependent list in $V$. Then there exists $k \in \{1, 2, \ldots, m\}$ such that $v_k \in \text{span}(v_1, \ldots, v_{k-1})$. Furthermore, if $k$ satisfies the condition above and the $k$th term is removed from $v_1, \ldots, v_m$, then the span of the remaining list equals $\text{span}(v_1, \ldots, v_m)$.

Proof: Because the list $v_1, \ldots, v_m$ is linearly dependent, there exist numbers $a_1, \ldots, a_m \in F$, not all 0, such that

$$a_1v_1 + \cdots + a_mv_m = 0.$$

Let $k$ be the largest element of $\{1, \ldots, m\}$ such that $a_k \neq 0$. Then

$$v_k = -\frac{a_1}{a_k} v_1 - \cdots - \frac{a_{k-1}}{a_k} v_{k-1},$$

which proves that $v_k \in \text{span}(v_1, \ldots, v_{k-1})$, as desired. $\square$

---

Lemma (Length of linearly independent list $\le$ length of spanning list): let $U$ be a vector space, $<\mathbf{u_{i}}>$ a linearly independent list of length $m$ in $U$, $<\mathbf{w_{i}}>$ a spanning list of length $n$ in $U$. Then, $m \le n$.

Proof: Consider the following process:

• step 1: create the list $L_{1} = \mathbf{u_{1}}, \mathbf{w_{1}}, \dots, \mathbf{w_{n}}$ of length $n+1$. This list is linearly dependent because $\mathbf{u_{1}} \in \text{Span}<\mathbf{w_{i}>}$, but $\mathbf{u_{1}} \neq \mathbf{0}$ as it comes from a linearly independent list. Hence there exist some $\mathbf{w_{j}} \in \text{Span} \ L_{1}$, which we can remove to get $L^*_1$ which is still a spanning list of $U$
• step k: Repeating this process to get $L^*_{k-1}$ which contains $\mathbf{u_{k-1}}, \dots \mathbf{u_{1}}$ and the rest are the remaining $\mathbf{w's}$. make $L_k$ by adding $\mathbf{u_{k}}$ to $L^*_{k-1}$. $L_k$ is a list of length $n+1$ and is linearly dependent as (by induction) $L^*_{k-1}$ was a spanning list. Yet again, there is some removable $\mathbf{w_{j}} \in \text{span} \ L_{k}[1..j-1]$ (linear dependence lemma) and it can’t be any one of the $\mathbf{u's}$ as they are all preceding by other $u's$ which are all linearly independent. Remove $\mathbf{w_{j}}$ to obtain $L^*_{k}$.
• At the end of this process, we obtain $L^*_m$, which is a spanning list, obtained by removing (at least) $m$ number of $\mathbf{w's}$. This means than $m \le n$. $\square$

---

Theorem (All bases have the same length): if $V$ is a vector space, and $L = <\mathbf{u_{i}}>$ and $L' = <\mathbf{v_{i}}>$ and $L, L'$ are both linearly independent and spanning lists (they’re both basis), then they have the same length. Moreover, any linearly independent list of this length is a basis of $V$.

Proof: let $|L| = m$, $|L'| = n$. Apply the above lemma, treating $L$ as a linearly independent list and $L'$ as a spanning list, to get $m \le n$. Now do it the other way around to get $m \le n$ and hence $m = n :=dim(V)$. For the second part, let $L^* = <\mathbf{e_{i}}>$ be a linearly independent list with length $n = dim(V)$. Do the replacement process, treating $L^*$ are linearly independent, and a basis $L$ as the spanning list, both of the same length. This allows us to add an element of $L^*$ to $L$ and remove an element which was in $L$, and still maintaining that the list is spanning throughout the process. so we modify $L$ to become $L^*$ while maintaning “spanning Ness” at each step. Therefore $L^*$ is spanning. Hence $L^*$ is a basis. $\square$

---

Rank Nullity Theorem

Theorem (Rank Nullity): Let $U,V$ be vector spaces over $F$ and $f: U \rightarrow V$ a linear map. Then, $dim(U) = \text{rank(f)} + dim(\text{kern}(f))$.

Proof: let $K =<\mathbf{k_{i}}>$ be a basis for $\text{Kern}(f)$ of length $m$. let $E = <\mathbf{e_{i}}>$ be a basis of $U$ of length $n = dim(U)$. Using the process described in the above lemma, treating $K$ as the linearly independent list, $E$ as the spanning list, create $L^* = \mathbf{k_{1}}, \dots, \mathbf{k_{m}}, \mathbf{e_{m+1}} \dots \mathbf{e_{n}}$. This list spans $U$. moreover, since $E$ and $K$ are both linearly independent, at each step, after adding a $\mathbf{k_{i}}$ when we remove an $\mathbf{e_{j}}$ we have a linearly independent list (this can be again shown by induction on the same process). Therefore $L^*$ is a linearly independent list. Since it is also of length $n = dim(U)$, $L^*$ is a basis of $U$. Hence $f(L^*) := f(\mathbf{k_{1}}), \dots f(\mathbf{k_{m}}), f(\mathbf{e_{m+1}}), \dots f(\mathbf{e_{n}})$ spans $V$ (refer to def of linear map). Since any $\mathbf{k_{t}} \in \text{Kern}(f)$ each $f(\mathbf{k_{i}} = \mathbf{ 0})$. Therefore, $fL^{**} := f(\mathbf{e_{m+1}}), \dots f(\mathbf{e_{n}})$ still spans $V$. We claim $fL^{**}$ is linearly independent. Otherwise we have $\mathbf{0} \in V = \sum_{j=m+1}^n a_{j}f(\mathbf{e_{j}}) = f\left( \sum_{j=m+1}^{n} a_{j} \mathbf{e_{j}}\right)$. This means that $\mathbf{t}^* =\sum_{j=m+1}^{n} a_{j} \mathbf{e_{j}} \in \text{Kern}(f)$, and hence, $\mathbf{t^*} \in span(K)$.If $\mathbf{t^*} \neq \mathbf{0}\in U$, then $\mathbf{t^*}$ can be written as a linear combination in two ways, one using just $K$ and other one using just $E[m+1 .. n]$. Subtracting the two, we get $\mathbf{0} = \sum_{j=1}^m \lambda_{j}\mathbf{k_{j}} - \sum_{j = m+1}^n a_{j}\mathbf{e_{j}}$. so if even one $a_j \neq 0 \in F$, then we contradict the linear independence of $L^*$ (and even $E$ for that matter). Hence, bubbling back, $fL^{**}$ is linearly independent, as each $a_j$ has to be zero whenever $\mathbf{0} \in V = \sum_{j=m+1}^n a_{j}f(\mathbf{e_{j}})$. $\square$

In pratice, if $M$ is a matrix representing a linear map from $U$ to $V$, then by solving for the kernel/nullspace of $U$ by $M\mathbf{x} = \mathbf{0}$ and finding the dimension of this nullspace, we can say that the rank/dimension of the image of this linear map $M$ is equal to the dimension of $U$ minus the dimension of the nullspace/kernel of $M$.

---

Invertible Linear Maps

Theorem (Smaller to larger is not surjective): Let $U,V$ be vector spaces, $dim(U) = m < dim(V) = n$, $f: U \rightarrow V$ a linear map. Since $dim(f(U)) = \text{rank}(f) \le dim(U) = m < dim(V) = n$, $f(U)$ cannot span $V$. Hence $f$ is not surjective. (refer definition of rank of linear map, or use rank-nullity theorem to conclude $dim(f(U))\le dim(U)$).

Theorem (Larger to smaller is not injective): Let $U,V$ be vector spaces, $dim(U) > dim(V)$ and $f: U \rightarrow V$ a linear map. Using rank-nullity theorem, we have $dim(U) = dim(f(U)) + dim(\text{kern}(f))$ Hence, $dim(\text{kern}(f)) = dim(U) - dim(f(U)) >dim(V) - dim(f(U)) \ge 0$ (the rightmost inequality owing to the fact that f(U) lives in V so its dimension is at most V), Finally, $dim(kern(f)) > 0$, which means that the kernel of $f$ is not the trivial subspace {0}. Therefore, there are at least two vectors in the kernel, both of which (by definition) map to zero in $V$

Combining the two above ideas, and stating with a little flair, we have:

Theorem (Invertible linear maps): A linear map $f: U \rightarrow V$ is bijective if and only if $dim(U) = dim(V)$ and $\text{Kern}(f) = \{\mathbf{0}\}$.

Proof: Necessity of the two conditions are already clear from the above two theorems. We just have to show sufficiency. Well, given $\text{Kern}(f) = \{ 0 \}$ and $dim(U) = dim(V)$, suppose $f$ is not injective. then there exists vectors $\mathbf{u} \neq \mathbf{v}$ such that $f(\mathbf{u}) = f(\mathbf{v})$, since the two vectors are not equal, at least one of them is non zero. However, it follows that $f(\mathbf{u-v}) = \mathbf{0} \in V$. hence $\mathbf{u-v} \neq \mathbf{0}\in \text{Kern}(f)$, contradicting that kern(f) = {0}. Moreover, using rank nullity, along with $dim(\text{Kern}(f)) = 0$, We see that $dim(f(U)) = dim(U) = dim(V)$. It is a trivial fact that the $0$ and highest dimensional subspaces, are uniquely the trivial subspace and the original space itself respectively. Therefore $f(U) = V$. showing surjectivity. $\square$

---

1 - Operators, Eigenvalues, Invariant Subspaces

Definition (Eigenvectors of a linear map): Let $f$ be a linear map on a vector space $V$ (to itself). Then the Eigenvectors of $f$ : $E^f =\{ \mathbf{v} \in V : f(\mathbf{v}) = \lambda \mathbf{v}, \exists \lambda \in F \}$. The $\lambda$ here is called an eigenvalue of $f$. The set of all eigenvalues of $f$ is called the spectrum of $f$ and denoted $\text{Spec}(f)$. If we define $E^f_\lambda$ as the set that contains all vectors $\mathbf{v}$ for which $f(\mathbf{v}) = \lambda \mathbf{v}$ for that particular lambda, then we have $E^f = \bigcup_{\lambda \in \text{spec}(f)} E^f_{\lambda}$. For any particular $\lambda \in \text{spec}(f)$ and a linear map $f$, $E_\lambda^f \cup \{ \mathbf{0} \}$ is a subspace of $V$: Notice that $f(\mathbf{v_{1}}) = \lambda \mathbf{v_{1}}, \ f(\mathbf{v_{2}}) = \lambda \mathbf{v_{2}} \implies f(\mathbf{v_{1} + v_{2}}) = \lambda(\mathbf{v_{1} + v_{2}})$, and $f(\mathbf{v}) = \lambda \mathbf{v} \implies f(c\mathbf{v}) =c(\lambda \mathbf{v}) = \lambda(c \mathbf{v})$.

Warning: The set of eigenvectors of f, $E^f$ itself, without fixing those eigenvectors with a particular eigenvalue IS NOT ALWAYS A SUBSPACE of V!!! For example just think about $f(\mathbf{v_{1}}) = \lambda_{1} \mathbf{v_{1}}, f(\mathbf{v_{2}})= \lambda_{2}\mathbf{v_{2}}$ then, $f(\mathbf{v_{1} + v_{2}}) = \lambda_{1}\mathbf{v_{1}}+ \lambda_{2}\mathbf{v_{2}}$.

This is pretty cool, so when we transform a vector space $V$, there are subspaces of $V$ which contain vectors that each just get scaled by a particular amount $\lambda$ under the transform. The eigenvalue and eigenvector sections are pure gold. Read them :) (LADR)

The idea is to have Invariant subspaces:

Definition (Operator, invariant subspace): If $V$ is a vector space, a linear map $f$ on $V$ (to $V$) is called an operator. A subspace $U$ is invariant under $f$ if $f(\mathbf{u}) \in U$ for each $\mathbf{u} \in U$. In other words, the restriction $f|_{U}$ is an operator on $U$.

Let’s talk about one dimensional Invariant subspaces, pick a particular $\mathbf{v} \in V$. Let $U = \{ \lambda \mathbf{v} : \lambda \in F \}$ . Then, if $f$ is an invariant on $U$, then for any $\mathbf{u} \in U$ , $f\mathbf{u} \in U$. But any $\mathbf{u} = \lambda \mathbf{v}$ for some $\lambda \in F$. since $f$ is linear, $f\mathbf{\lambda \mathbf{v}} = \lambda f(\mathbf{v})$. This means that the map $f$ is defined by the image of $\mathbf{v}$ itself, and that image is in $U$ therefore there has to be some unique scalar $\kappa$ for which $f(\mathbf{v}) = \kappa \mathbf{v}$. (note that $U =\text{span}(\mathbf{v})$).
Conversely if there exists an operator $f$ such that for some $\mathbf{v} \in V$, $f(\mathbf{v}) = \kappa \mathbf{v}$ for some scalar $\kappa \in F$, then $\text{span}(\mathbf{v})$ is an invariant subspace under $f$.

Definition (Eigenvalue): if $f$ is an operator on $V$, then $\lambda$ is an eigenvalue of $f$ if there exists a non zero $\mathbf{v}\in V$ such that $f(\mathbf{v}) = \lambda \mathbf{v}$. Writing $f$ as a matrix $T$, We have $(T - \lambda I)\mathbf{v} = \mathbf{0}$. Since $\mathbf{v}$ is non zero, the linear map $(T- \lambda I) \equiv f-\lambda \iota$ has a non trivial kernel/null space. As we know, (from the discussion on invertibility), This is equivalent to saying, $f - \lambda \iota$ is not injective. Using rank nullity theorem, Since the kernel is non trivial, the dimension of $(f-\lambda\iota)(V)$ is smaller than the dimension $V$ which means that this map is also not surjective. Hence finally this map is not invertible.

---

Theorem (Linearly independent eigenvectors): If $f$ is an operator on $V$, if we pick a list of distinct eigenvectors, each associated with it’s own distinct eigen value, then that list is linearly independent. That is if $L = <\mathbf{v_{i}}>$ such that $\mathbf{v_{i}} \in E^f_{\lambda_{i}}$, then $L$ is linearly independent.

Proof: since each $\mathbf{v_{i}}$ is non zero (by definition), pick the smallest subset of $L$, $L^* = <\mathbf{v^*_{i}}>$ of length $m \ge 2$, such that there exists a linear combination $G = \sum_{i=1}^ma_{i}\mathbf{v^*_{i}} = \mathbf{0}$ such that each $a_i$ is non zero (if an $a_i$ is zero, just drop it). Using matrix notation apply $T -\lambda_{m}$ to $G$, since $(T - \lambda_{m})\mathbf{v^*}_{m} = \mathbf{ 0}$, we obtain a smaller sublist $G'$ (of length m-1) that is also linearly dependent, contradicting the minimality of $G$. $\square$

---

Theorem (Number of eigenvalues $\le$ dimension): if $f$ is an operator on $V$, then the number of distinct eigenvalues of $f$ is at most $dim(V)$

Proof: Pick a list $L$ of eigenvectors, one for each distinct eigenvalue. Then since this list is linearly independent (from the theorem above), it’s length is at most $dim(V)$. $\square$

---

2 - Polynomials of Operators

Definition (Field polynomials): let $F$ be a field. A function $p : F \rightarrow F$ is a polynomial (of degree $n$) if there exists $a_i \in F$, ($a_{n-1} \neq 0$) for each $i = 0$ to $n-1$ such that $p(z) = \sum_{i=0}^{n-1}a_{i}z^i$ for each $z \in F$. The set of all polynomials of $F$ is written $\mathscr P(F)$. This set is a vector space, where addition of two polynomial functions $p_1 + p_{2}$ is defined as the function that maps $z \mapsto p_1(z) + p_{2}(z)$ for each $z \in F$. Similarly, the product of two polynomials $p_{1} \cdot p_{2}$ is defined as the map $z \mapsto p_{1}(z) \cdot p_{2}(z)$. Finally, the scalar multiplication of a polynomial $\lambda p$, $\lambda \in F$ is defined as the map $z \mapsto \lambda p(z)$. It is left to the reader to verify that $\mathscr P(F)$ with the addition and scalar multiplication defined above, is a vector space.

Definition (Powers of an operator): Let $V$ be a vector space over $F$ and $f$ an operator on $V$. Then $f^n$ is defined as $f$ composed on itself $n$ times. It’s pretty clear the the usual power laws hold good. And $f^0$ is the identity operator.

Now, if we want to apply the polynomial $p \in \mathscr P(F)$ to the operator $f$, if $p(z) = \sum_{i=0}^{n-1}a_{i}z^i$, then we define $p(f) := \sum_{i=0}^{n-1}a_{i}f^i$. This is well defined, because the set of operators on $V$, usually denoted $\mathscr L (V)$ is itself a vector space, over $F$, where (as usual) addition of linear maps is defined by pointwise addition, and scalar multiplication as pointwise scalar multiplication. We can also define multiplication on $\mathscr L(V)$ as function composition. We observe that $pq(f) = p(f)q(f) = q(f)p(f)$ (informally, replacing the symbol $z$ with another symbol $f$) doesn’t change anything. It is clear that for any field polynomial $p$, the operator polynomial $p(f)$ is also an operator on $V$, that is $p(f) \in \mathscr L(V)$ for any $p \in \mathscr P(F)$ and any $f \in \mathscr L(V)$. Notice that:

$$p(f)(\mathbf{u_{1}} + \mathbf{u_{2}}) = \sum_{i=1}^{n-1}a_{i}f^i(\mathbf{u_{1} + u_{2}})$$ $$= \sum_{i=1}^na_{i}(f^i(\mathbf{u_{1}})) +\sum_{i=1}^na_{i}(f^i(\mathbf{u_{2}})) = p(f)(\mathbf{u_{1}}) + p(f)(\mathbf{u_{2}})$$

(we are using the linearity of $f^i$) Similarly we can show that $p(f)(\lambda \mathbf{v}) = \lambda p(f)(\mathbf{v})$. Notice that using the identity polynomial $\mathbb 1(z) = z$, we can write any operator $f$ as the polynomial operator $\mathbb 1(f)$.

---

Theorem (Null space and range of polynomial operator are invariant): let $f$ be an operator on a vector space $V$, and $p(f)$ an operator polynomial. Then $\text{Kern}(p(f))$ and $p(f)(V)$ are both invariant under $f$.

Proof: take $\mathbf{u} \in \text{Kern}(p(f))$, then, $p(f)(\mathbf{u}) = \mathbf{0}$. We want to show that $f(\mathbf{u}) \in \text{Kern}(p(f))$, that is $p(f)(f(\mathbf{u})) = 0$ using polynomial notation, $p(f)(\mathbb 1 (f) (u)) = \mathbf{0}$ But the composition $p(f) \mathbb 1(f)$ is equal to $p \ \mathbb 1(f)$, where $p \ \mathbb 1$ is the product of $p$ and the identity polynomial. polynomial products are commutative hence $p \ \mathbb 1 (f)= \mathbb 1 \ p (f) = \mathbb 1(f) p(f)$. Hence $p(f)(\mathbb 1(f)(\mathbf{u})) = \mathbb{1}(f)(p(f)(\mathbf{u})) = \mathbb 1(f)(\mathbf{0}) = \mathbf{ 0}$. let $\mathbf{ u} \in p(f)(V)$, then for some $\mathbf{v}$, $p(f)(\mathbf{v}) = \mathbf{u}$. Taking $\mathbb 1(f)$ on both sides, and moving things around, we get $p(f)(f(\mathbf{v})) = f(\mathbf{u})$. $\square$

---

3 - Multilinear Algebra

Definition (Hom($V,W$)): The set of all linear maps from $V \to W$ where $V,W$ are vector spaces, is called $\text{Hom}(V,W)$. And is itself a vector space, where the zero function $\mathbf{O}(\mathbf{v}) = \mathbf{0_{w}}$ for each $\mathbf{v} \in V$ acts as the identity vector, and the map $f + g$ is defined as the map that $x \mapsto f(x)+g(x)$ and the map $\lambda f$ is defined as the one $x \mapsto \lambda f(x)$. Now, given $\text{Hom}(V_{1},V_{2})$ and $\text{Hom}(W_{1},W_{2})$ are vector spaces, we can have linear maps between them. This is the “multilinear nature”

---

Dual Spaces and Co-ordinate Projection Maps

Definition (Dual vector space): if $V$ is a vector space over $\mathbb R$, then $\text{Hom}(V, \mathbb R) = V^*$ is called the dual space of $V$. An element of $V^*$ is called a “co-vector”. For example if $P$ is the vector space of all polynomials with real entries and real coefficients, the definite integral of an element of $P$ from $0$ to $1$ is an operator call it $I$: $I(p) = \int_{0}^1p(x)dx$, then $I$ is a linear map from $P$ to $\mathbb R$, hence $I \in P^*$, so $I$ is a co-vector of $P$. In general if $V$ is a vector space over $F$, $\text{Hom}(V,F)$ is called the dual of $V$. An element of $V^*$ is often called a functional

Definition (Hamel Bases): if $V$ is a vector space (maybe of infinite dimension), then $B$ is a Hamel basis for $V$ if for all $\mathbf{v} \in V$, there exist a unique finite subset $\{v^1,v^2,\dots v^n\}$ of $B$, and $n$ non zero scalars $f_{1}, f_{2},\dots f_{n}$ such that $\mathbf{v} = \sum_{i=1}^nv^if_{i}$.

Definition (Co-ordinate projection maps): If $V$ is finite dimensional, with basis $<\mathbf{e_{i}}>_{i=1}^n$, Then the $j$‘th projection map $p_{j}: V \to F$ is the one that gives the scalar co-efficient of the $j$‘th basis vector when the input vector is written as a linear combination of the basis vectors. that is

$p_{j}(\lambda_{1}\mathbf{e_{1}}+ \lambda_{2}\mathbf{e_{2}} + \dots \lambda_{n}\mathbf{e_{n}}) = \lambda_{j}$

If $V$ is infinite dimensional, with Hamel basis $B$, then $p_b: V \to F$ gives the scalar coefficient of the basis vector $b \in B$, when the input vector is written as a unique linear combination of a subset of $B$. that is:

$$p_{b}\left(\sum_{i=1}^nv^if_{i}\right) = \begin{cases} f_{k} & b \in \{ v^1,v^2,\dots, v^n \}, \ \text{i.e.} \ b = v^k \\ 0 & b \not \in \{ v^1,v^2,\dots, v^n \} \end{cases}$$

In both cases, the co-ordinate projection maps are linear maps. In the finite dimensional case $p_{j}\left( a\sum_{i=1}^n\lambda_{i}\mathbf{e_{i}} +b\sum_{i=1}^n\mu_{i}\mathbf{e_{i}} \right) = a\lambda_{j} + b\mu_{j}$ and for the infinite dimensional case, I wont write the case-wise breakdown, but okay. Moreover, in both cases the set of all co-ordinate projection maps (also called co-ordinate functionals) are linearly independent vectors in $V^*$, for the infinite dimension case, consider the functional which is a linear combination of co-ordinate maps being equal to the zero functional. $L =\sum_{b \in B}f_{b}p_{b} = \mathbf{0}$ For any basis $b \in B$, $L(b) = 0\in F$ (as L is the zero functional) but $L(b) = f_{b}$ Hence $f_b = 0$, running through all $b \in B$, we notice that each $f_b = 0 \in F$. Hence $L$ is linearly independent Similarly, for the finite dimensional case let $T = \sum_{i=1}^n\lambda_{i}p_{i} = \mathbf{0}$, then $T(\mathbf{e_{j}})= \lambda_{j} = 0 \in F$, hence running through all $\mathbf{e_{i}}$,we have $\lambda_{i} = 0 \in F$ for each $i = 1 \to n$. so $<p_{i}>_{i=1}^n$ is a list of $n$ linearly independent vectors in $V^*$. let $f \in V^*$, for any $\mathbf{v} = \sum_{i=1}^n\lambda_{i} \mathbf{e_{i}} \in V$, Then $f(\mathbf{v})= \sum_{i=1}^n\lambda_{i}f(\mathbf{e_{i}})$ That is $f$ is completely determined by what it does to each of the basis vectors. Hence we see (beautifully!) that $f = \sum_{i=1}^nf(\mathbf{e_{i}})p_{i}$. Notice that $\left( \sum_{i=1}^nf(\mathbf{e_{i}})p_{i} \right)(\mu _{k} \mathbf{e_{k}}) =\sum_{i=1}^n (f(\mathbf{e_{i}}) \mu _k) \cdot (p_{j}(\mathbf{e_{k}})) =f(\mathbf{e_{k}})\mu _k =f(\mu_{k}\mathbf{e_{k}})$. So the co-ordinate projection functionals are a basis for $V^*$ when $V$ is finite dimensional. Moreover, even if $V$ is infinite dimensional, for any $\mathbf{v} \neq \mathbf{0} \in V$, there exists a functional $g$ for which $g(\mathbf{v}) \neq 0$. Notice $\mathbf{v} = \sum_{i=1}^nv^if_{i}$. then simple set $g = p_{v^k}$ then $g(\mathbf{v}) = f_{k} \neq 0 \in F$

Also, $dim(V) = dim(V^*)$

---

Injections into the Double Dual

Fix a particular vector $\mathbf{v} \in V$. Let $f \in V^*$. Define an evaluation map $ev_{\mathbf{v}}: V^* \to F$, as $ev_{\mathbf{}}(f) =f(\mathbf{v})$. Now, $ev_{\mathbf{v}}(f_{1}+f_{2}) = (f_{1}+f_{2})(\mathbf{v}) = f_{1}(\mathbf{v}) + f_{2}(\mathbf{v}) = ev_{\mathbf{v}}(f_{1}) + ev_{\mathbf{v}}(f_{2})$ and,

$$$$

Hence, for each $\mathbf{v} \in V$, $ev_{\mathbf{v}} \in \text{Hom}(V^*, F) = {V^*}^*$.

Now consider the map $\phi: V \to {V^*}^*$ given by $\phi(\mathbf{v}) = ev_{\mathbf{v}}$.

Is this map $\phi$ now linear? Notice that $\phi(\mathbf{v_{1}}+\mathbf{v_{2}})$ is equal to the function $ev_{\mathbf{v_{1}+v_{2}}}$. Now, for any $f \in V^*$, $ev_{\mathbf{v_{1}+v_{2}}}(f)=f(\mathbf{v_{1}}+\mathbf{v_{2}}) =f(\mathbf{v_{1}})+ f(\mathbf{v_{2}}) =ev_{\mathbf{v_{1}}}(f) + ev_{\mathbf{v_{2}}}(f)$.

So for any input, we see that the maps $ev_{\mathbf{v_{1} + v_{2}}}$ is equivalent to the map $ev_{\mathbf{v_{1}}} + ev_{\mathbf{v_{2}}}$, where addition of maps is defined as usual, $ev_{\mathbf{v_{1}}} + ev_{\mathbf{v_{2}}}$ is the map that takes $f$ to $ev_{\mathbf{v_{1}}}(f) + ev_{\mathbf{v_{2}}}(f)$.

Therefore $\phi(\mathbf{v_{1}+v_{2}}) = \phi(\mathbf{v_{1}}) + \phi(\mathbf{v_{2}})$. Similarly, it can be shown that $\phi(\lambda \mathbf{v}) = \lambda \phi(\mathbf{v})$.

therefore $\phi$ is a linear map from $V \to {V^*}^*$. Moreover this map is injective. Let us suppose $\phi(\mathbf{u}) = \phi(\mathbf{v})$. Then the two maps $ev_{\mathbf{u}} = ev_{\mathbf{v}}$ as functions. meaning that for all $f \in V^*$, $ev_{\mathbf{u}}(f) = ev_{\mathbf{v}}(f)$. Hence, for each $f \in V^*$, $f(\mathbf{u}) = f(\mathbf{v})$. Hence for all $f \in V^*$, $f(\mathbf{u-v}) = 0 \in F$, and as the result shown above, there exists at least one functional $f_{\mathbf{u-v}}$ for which $f_{\mathbf{u-v}}(\mathbf{u-v}) \neq 0$ unless $\mathbf{u} = \mathbf{v}$ hence the map is injective.

Moreover, if $V$ is of finite dimension, then $\phi$ is bijective. From the definition of co-ordinate projection maps, we know that $dim(V) = dim(V^*)$. Treating $V^*$ as the finite dimensional vector space, taking co-ordinate projection maps as the basis for $V^*$ and applying the same argument , we have that $dim(V^*) = dim({V^*}^*)$ Hence $\phi : V \to {V^*}^*$ is an injective function between two spaces of the same dimension. using rank nullity theorem, $dim(V) = dim(kern(\phi)) + dim(range(\phi))$, since $\phi$ is injective, $dim(kern(\phi)) = 0$. Hence $dim(V) = dim(range(\phi))$. Therefore $dim({V^*}^*) = dim(range(\phi))$, we know that there is only one subspace of ${V^*}^*$ of full dimension, namely itself, therefore $range(\phi) = {V^*}^*$.

Hence for finite dimensional vector spaces $V$, $V$ is isomorphic to its double dual ${V^*}^*$

---

Tensors

Definition (Tensor): An $r,s$ tensor on $V$ is a multilinear map $t: (V^*)^r \times (V)^s \to \mathbb R$, that is $t$ eats a tuple, whose first $r$ elements are dual vectors, and the remaining $s$ elements are vectors, and spits our a real number such that $t$ is linear in each entry of the tuple (or in each variable). Consider the $1,1$ tensors $t: V^* \times V \to \mathbb R$. Now consider a map $\phi: V \to (V^*)^*$ given by $\phi(\mathbf{v}) = t(\cdot, \mathbf{v})$, that is $\phi$ takes in a particular vector $\mathbf{v}$ and spits out the map defined by fixing the second entry of a $1,1$ tensor at $\mathbf{v}$, call it $t(\cdot, \mathbf{v})$, which is a function from $V^* \to \mathbb R$. Informally, this is basically just splitting the process of $t$ by first giving it a vector and let it spit out a map that eats a co vector on the left, and the right vector is fixed, equal to to the input vector. So if $V$ is finite dimensional, $1,1$ tensors can be seen as isomorphic to the set of all linear maps on $V$. In a similar fashion, $V^* = \text{Hom}(V, \mathbb R)$, therefore co-vectors are $0,1$ tensors. For finite dimension $V = {V^*}^* = \text{Hom}(V^*, \mathbb R)$ therefore vectors are $1,0$ tensors.

If $V$ is of finite dimension, then to specify a linear map from $V$, it is sufficient to know $f(\mathbf{e_{1}}), f(\mathbf{e_{2}}), \dots, f(\mathbf{e_{n}})$, so it is enough to know the evaluation of each basis vector to reconstruct the evaluation of the linear map on any vector

Definition (Components of a Tensor): Let $t$ be an $(r,s)$ tensor over a finite dimensional vector space $V$, with basis $<\mathbf{e_{i}}>_{i=1}^n$ and for the dual space, the basis of co-ordinate projection maps $<\mathbf{e^j}>_{j=1}^n$. Then, a component of $t$ is written

$$T^{j_{1},j_{2},\dots j_{r}}_{i_{1},i_{2},\dots i_{s}} = T(\mathbf{e^{j_{1}}}, \mathbf{e^{j_{2}}}, \dots, \mathbf{e^{j_{r}}},\mathbf{e_{i_{1}}}, \mathbf{e_{i_{2}}}, \dots, \mathbf{e_{i_{s}}})$$

So each component is the evaluation of: some $r$ length sub-sequence of the dual basis vectors, concatenated by $s$ length sub-sequence of the basis vectors. And this is all the info we need to evaluate $T$ for any input.

Sources and Remarks

https://www.youtube.com/channel/UCUHKG3S9N_QeIE2jQXd2-VQ - gravity and light lectures, I am working on cleaning up more content from here https://linear.axler.net/LADR4e.pdf - my favorite linear algebra book, again will clean up more content from here (my notes tend to be horribly messy, a majority of them being pen and paper) As always, if you find errors, hit me up on twitter, or just hit me up on twitter.