Linear Transformations

A linear transformation also often called linear map is a function between two vector spaces which we will come to later. For now you can think of a linear transformation as a function that takes in a vector and outputs a vector. The vectors can have the same dimension or different dimensions.

T: R^N \rightarrow R^M

Where $T$ is the transformation, $R^N$ is the input vector space so vectors of dimension $N$ and $R^M$ is the output vector space so vectors of dimension $M$ .

Importantly a linear transformation is operation preserving, meaning that it preserves the operations of addition and scalar multiplication. More formally the following two conditions must be satisfied for a function to be a linear transformation:

Additivity: $T(\boldsymbol{u} + \boldsymbol{v}) = T(\boldsymbol{u}) + T(\boldsymbol{v})$
Homogeneity: $T(c\boldsymbol{u}) = cT(\boldsymbol{u})$

linearTransformationAddition.png linearTransformationScaling.png

Valid Linear Transformation

Let's first look at an examples of a valid linear transformations where the vectors have the same dimension.

T: R^2 \rightarrow R^2, T \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 2x \\ 3y \end{bmatrix}

To show that this is a linear transformation we need to show that it satisfies the two conditions. Firstly we can show that it satisfies additivity:

\begin{align*} T(\boldsymbol{u} + \boldsymbol{v}) &= T \begin{bmatrix} x_1 + x_2 \\ y_1 + y_2 \end{bmatrix} = \begin{bmatrix} 2(x_1 + x_2) \\ 3(y_1 + y_2) \end{bmatrix} = \begin{bmatrix} 2x_1 + 2x_2 \\ 3y_1 + 3y_2 \end{bmatrix} \\ T(\boldsymbol{u}) + T(\boldsymbol{v}) &= \begin{bmatrix} 2x_1 \\ 3y_1 \end{bmatrix} + \begin{bmatrix} 2x_2 \\ 3y_2 \end{bmatrix} = \begin{bmatrix} 2x_1 + 2x_2 \\ 3y_1 + 3y_2 \end{bmatrix} \end{align*}

So we can see that $T(\boldsymbol{u} + \boldsymbol{v}) = T(\boldsymbol{u}) + T(\boldsymbol{v})$ so the transformation satisfies additivity. Now we can check that it satisfies homogeneity:

\begin{align*} T(c\boldsymbol{u}) &= T \begin{bmatrix} cx \\ cy \end{bmatrix} = \begin{bmatrix} 2(cx) \\ 3(cy) \end{bmatrix} = \begin{bmatrix} 2cx \\ 3cy \end{bmatrix} \\ cT(\boldsymbol{u}) &= c \begin{bmatrix} 2x \\ 3y \end{bmatrix} = \begin{bmatrix} 2cx \\ 3cy \end{bmatrix} \end{align*}

Because $T(c\boldsymbol{u}) = cT(\boldsymbol{u})$ the transformation satisfies homogeneity. Therefore the transformation is a linear transformation.

Linear Transformation with Different Dimensions

A linear transformation can also be between vectors of different dimensions. For example:

T: R^2 \rightarrow R^3, T \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 2x \\ 3y \\ 0 \end{bmatrix}

This transformation is still linear because it satisfies the two conditions. The additional dimension in the output vector is just simply set to 0 so it doesn't affect the linearity of the transformation.

\begin{align*} T(\boldsymbol{u} + \boldsymbol{v}) &= T \begin{bmatrix} x_1 + x_2 \\ y_1 + y_2 \end{bmatrix} = \begin{bmatrix} 2(x_1 + x_2) \\ 3(y_1 + y_2) \\ 0 \end{bmatrix} = \begin{bmatrix} 2x_1 + 2x_2 \\ 3y_1 + 3y_2 \\ 0 \end{bmatrix} \\ T(\boldsymbol{u}) + T(\boldsymbol{v}) &= \begin{bmatrix} 2x_1 \\ 3y_1 \\ 0 \end{bmatrix} + \begin{bmatrix} 2x_2 \\ 3y_2 \\ 0 \end{bmatrix} = \begin{bmatrix} 2x_1 + 2x_2 \\ 3y_1 + 3y_2 \\ 0 \end{bmatrix} \end{align*}

\begin{align*} T(c\boldsymbol{u}) &= T \begin{bmatrix} cx \\ cy \end{bmatrix} = \begin{bmatrix} 2(cx) \\ 3(cy) \\ 0 \end{bmatrix} = \begin{bmatrix} 2cx \\ 3cy \\ 0 \end{bmatrix} \\ cT(\boldsymbol{u}) &= c \begin{bmatrix} 2x \\ 3y \\ 0 \end{bmatrix} = \begin{bmatrix} 2cx \\ 3cy \\ 0 \end{bmatrix} \end{align*}

We can also do the same for a linear transformation where the input vector has more dimensions than the output vector.

T: R^3 \rightarrow R^2, T \begin{bmatrix} x \\ y \\ z \end{bmatrix} = \begin{bmatrix} 2x \\ 3y \end{bmatrix}

This transformation is still linear because it satisfies the two conditions. The additional dimensions in the input vector are simply ignored and don't affect the linearity of the transformation.

\begin{align*} T(\boldsymbol{u} + \boldsymbol{v}) &= T \begin{bmatrix} x_1 + x_2 \\ y_1 + y_2 \\ z_1 + z_2 \end{bmatrix} = \begin{bmatrix} 2(x_1 + x_2) \\ 3(y_1 + y_2) \end{bmatrix} = \begin{bmatrix} 2x_1 + 2x_2 \\ 3y_1 + 3y_2 \end{bmatrix} \\ T(\boldsymbol{u}) + T(\boldsymbol{v}) &= \begin{bmatrix} 2x_1 \\ 3y_1 \end{bmatrix} + \begin{bmatrix} 2x_2 \\ 3y_2 \end{bmatrix} = \begin{bmatrix} 2x_1 + 2x_2 \\ 3y_1 + 3y_2 \end{bmatrix} \end{align*}

\begin{align*} T(c\boldsymbol{u}) &= T \begin{bmatrix} cx \\ cy \\ cz \end{bmatrix} = \begin{bmatrix} 2(cx) \\ 3(cy) \end{bmatrix} = \begin{bmatrix} 2cx \\ 3cy \end{bmatrix} \\ cT(\boldsymbol{u}) &= c \begin{bmatrix} 2x \\ 3y \end{bmatrix} = \begin{bmatrix} 2cx \\ 3cy \end{bmatrix} \end{align*}

Invalid Linear Transformation

However, not all transformations are linear. Lets look at an example of a transformation that is not linear:

T: R^2 \rightarrow R^2, T \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} x^2 \\ y^2 \end{bmatrix}

This transformation is not linear because it doesn't satisfy the additivity condition:

\begin{align*} T(\boldsymbol{u} + \boldsymbol{v}) &= T \begin{bmatrix} x_1 + x_2 \\ y_1 + y_2 \end{bmatrix} = \begin{bmatrix} (x_1 + x_2)^2 \\ (y_1 + y_2)^2 \end{bmatrix} = \begin{bmatrix} x_1^2 + 2x_1x_2 + x_2^2 \\ y_1^2 + 2y_1y_2 + y_2^2 \end{bmatrix} \\ T(\boldsymbol{u}) + T(\boldsymbol{v}) &= \begin{bmatrix} x_1^2 \\ y_1^2 \end{bmatrix} + \begin{bmatrix} x_2^2 \\ y_2^2 \end{bmatrix} = \begin{bmatrix} x_1^2 + x_2^2 \\ y_1^2 + y_2^2 \end{bmatrix} \end{align*}

It also doesn't satisfy the homogeneity condition:

\begin{align*} T(c\boldsymbol{u}) &= T \begin{bmatrix} cx \\ cy \end{bmatrix} = \begin{bmatrix} (cx)^2 \\ (cy)^2 \end{bmatrix} = \begin{bmatrix} c^2x^2 \\ c^2y^2 \end{bmatrix} \\ cT(\boldsymbol{u}) &= c \begin{bmatrix} x^2 \\ y^2 \end{bmatrix} = \begin{bmatrix} cx^2 \\ cy^2 \end{bmatrix} \end{align*}

Matrices as Transformations

Also relates back to inverse matrices, as the inverse of a matrix is the matrix that undoes the transformation of the original matrix.

so linear maps are bijective, and thus invertible.

linearTransformationVisual.png

Kernel and Image

The image of a transformation is defiend just as with functions, it is the set of all possible outputs of the transformation. the null vector is always in the image of a transformation.

\operatorname{Im}(A) = \{ A\boldsymbol{x} \in R^M \mid \boldsymbol{x} \in R^N \}

The kernel of a transformation is the set of all inputs that map to the zero vector. This has a special name, the null space.

\operatorname{Ker}(A) = \{ \boldsymbol{x} \in R^N \mid A\boldsymbol{x} = \boldsymbol{o} \}

dimension of the domain = dimension of the kernel + dimension of the image

Bijective Linear Transformations

are invertible, and thus have a unique inverse.

Composition of Linear Transformations

Matrix Rank Determinants