# Linear Algebra - Orthogonality (Perpendicular)

Orthogonal is linear-algebra-ese for perpendicular.

## Definition

The squared length of the “hypotenuse” (the vector u + v) is

$\begin{array}{rcll} {\left \| u + v \right \|}^2 & = & \left \langle u+v,u+v \right \rangle & \\ & = & \left \langle u,u+v \right \rangle + \left \langle v,u+v \right \rangle & \text{by linearity of inner product} \\ & = & \left \langle u,u \right \rangle + \left \langle u,v \right \rangle + \left \langle v,u \right \rangle + \left \langle v,v \right \rangle & \text{by symmetry and linearity of inner product} \\ & = & {\left \| u \right \|}^2 + 2 \left \langle u,v \right \rangle + {\left \| v \right \|}^2 & \text{by symmetry of inner product} \\ \end{array}$

Thus following the Pythagorean Theorem u and v are orthogonal when $$\left \langle u,v \right \rangle = 0$$

then:

• the Pythagorean Theorem for vectors becomes:

$${\left \| u + v \right \|}^2 = {\left \| u \right \|}^2 + {\left \| v \right \|}^2$$

• every vector v is orthogonal to u when u is the zero vector.

## Property

• Property O1: If u is orthogonal to v then u is orthogonal to $\alpha v$ for every scalar $\alpha$ .
• Property O2: If u and v are both orthogonal to w then u + v is orthogonal to w.

## Example

• [1, 2] · [2,-1] = 0 so [1, 2] · [20,-10] = 0
• Second Example:
        [1, 2, 1]       . [1,-1, 1] = 0
[0, 1, 1]       . [1,-1, 1] = 0
----------------------------------------
([1, 2, 1] + [0, 1, 1]) . [1,-1, 1] = 0


## Type

### Mutual

$v_1, \dots, v_n$ are mutually orthogonal if $v_i$ is orthogonal to $v_j$ for every pair $i,j$ such that $i \neq j$ .

Example: $[1, 2, 1], [1,−1, 1], [1, 0,−1]$ are mutually orthogonal: $$\begin{array}{rrrr} \langle & [1, 2, 1], & [1,−1, 1] & \rangle & = & 0 \\ \langle & [1, 2, 1], & [1, 0,−1] & \rangle & = & 0 \\ \langle & [1,−1, 1], & [1, 0,−1] & \rangle & = & 0 \end{array}$$

Lemma: If $v_1, \dots, v_n$ are mutually orthogonal then, for any scalars $\alpha_1, \dots, \alpha_n$ : $${\| \alpha_1.v_1, \dots, \alpha_n.v_n \|}^2 = {\alpha_1}^2.{\| v_1 \|}^2 + \dots + {\alpha_n}^2.{\| v_n \|}^2$$

#### Mutually orthogonal nonzero are independent

Mutually orthogonal nonzero vectors are linearly independent

Proof: Let $v_1^* , \dots , v_n^*$ be mutually orthogonal nonzero vectors, then they are linearly independent if the coefficient $\alpha_0, \alpha_1 , \dots , \alpha_n$ are null in the following statement: $$0 = \alpha_0 v_0^* + \alpha_1 v_1^* + \dots + \alpha_n v_n^*$$

To show that $\alpha_0$ is zero, take inner product with $v_0^*$ on both sides:

$$\begin{array}{rrl} \langle v_0^*, 0 \rangle & = & \langle v_0^*, \alpha_0 v_0^* + \alpha_1 v_1^* + \dots + \alpha_n v_n^* \rangle \rangle \\ 0 & = & \alpha_0 \langle v_0^*, v_0^* \rangle + \alpha_1 \langle v_0^*, v_1^* \rangle + \dots + \alpha_n \langle v_0^*, v_n^* \rangle \\ 0 & = & \alpha_0 {\| v_0^* \|}^2 + \alpha_1 0 + \dots + \alpha_n 0 \\ 0 & = & \alpha_0 {\| v_0^* \|}^2 \\ 0 & = & \alpha_0 {\| v_0^* \|}^2 \\ \end{array}$$ Since $v_0^*$ is nonzero, the only solution is $\alpha_0 = 0$

## Operations

### Scaling

Scaling orthogonal vectors gives orthogonal vectors. If u is orthogonal to v then, for any scalars $\alpha, \beta$ : $${\left \|\alpha u + \beta v \right \|}^2 = \alpha^2 {\left \|\alpha u \right \| }^2 + \beta^2 {\left \|\alpha v \right \| }^2$$

Data Science
Data Analysis
Statistics
Data Science
Linear Algebra Mathematics
Trigonometry