Linear Algebra - Orthogonality (Perpendicular)

Card Puncher Data Processing

About

Orthogonal is linear-algebra-ese for perpendicular.

Definition

The squared length of the “hypotenuse” (the vector u + v) is

<math> \begin{array}{rcll} {\left \| u + v \right \|}^2 & = & \left \langle u+v,u+v \right \rangle & \\ & = & \left \langle u,u+v \right \rangle + \left \langle v,u+v \right \rangle & \text{by linearity of inner product} \\ & = & \left \langle u,u \right \rangle + \left \langle u,v \right \rangle + \left \langle v,u \right \rangle + \left \langle v,v \right \rangle & \text{by symmetry and linearity of inner product} \\ & = & {\left \| u \right \|}^2 + 2 \left \langle u,v \right \rangle + {\left \| v \right \|}^2 & \text{by symmetry of inner product} \\ \end{array} </math>

Thus following the Pythagorean Theorem u and v are orthogonal when <MATH>\left \langle u,v \right \rangle = 0</MATH>

then:

<MATH>{\left \| u + v \right \|}^2 = {\left \| u \right \|}^2 + {\left \| v \right \|}^2</MATH>

  • every vector v is orthogonal to u when u is the zero vector.

Property

  • Property O1: If u is orthogonal to v then u is orthogonal to <math>\alpha v </math> for every scalar <math>\alpha</math> .
  • Property O2: If u and v are both orthogonal to w then u + v is orthogonal to w.

Example

  • [1, 2] · [2,-1] = 0 so [1, 2] · [20,-10] = 0
  • Second Example:
[1, 2, 1]       . [1,-1, 1] = 0
        [0, 1, 1]       . [1,-1, 1] = 0
----------------------------------------
([1, 2, 1] + [0, 1, 1]) . [1,-1, 1] = 0

Type

Mutual

<math>v_1, \dots, v_n</math> are mutually orthogonal if <math>v_i</math> is orthogonal to <math>v_j</math> for every pair <math>i,j</math> such that <math>i \neq j</math> .

Example: <math>[1, 2, 1], [1,−1, 1], [1, 0,−1]</math> are mutually orthogonal: <MATH> \begin{array}{rrrr} \langle & [1, 2, 1], & [1,−1, 1] & \rangle & = & 0 \\ \langle & [1, 2, 1], & [1, 0,−1] & \rangle & = & 0 \\ \langle & [1,−1, 1], & [1, 0,−1] & \rangle & = & 0 \end{array} </MATH>

Lemma: If <math>v_1, \dots, v_n</math> are mutually orthogonal then, for any scalars <math>\alpha_1, \dots, \alpha_n</math> : <MATH> {\| \alpha_1.v_1, \dots, \alpha_n.v_n \|}^2 = {\alpha_1}^2.{\| v_1 \|}^2 + \dots + {\alpha_n}^2.{\| v_n \|}^2 </MATH>

Mutually orthogonal nonzero are independent

Mutually orthogonal nonzero vectors are linearly independent

Proof: Let <math>v_1^* , \dots , v_n^*</math> be mutually orthogonal nonzero vectors, then they are linearly independent if the coefficient <math>\alpha_0, \alpha_1 , \dots , \alpha_n</math> are null in the following statement: <MATH>0 = \alpha_0 v_0^* + \alpha_1 v_1^* + \dots + \alpha_n v_n^* </MATH>

To show that <math>\alpha_0</math> is zero, take inner product with <math>v_0^*</math> on both sides:

<MATH> \begin{array}{rrl} \langle v_0^*, 0 \rangle & = & \langle v_0^*, \alpha_0 v_0^* + \alpha_1 v_1^* + \dots + \alpha_n v_n^* \rangle \rangle \\ 0 & = & \alpha_0 \langle v_0^*, v_0^* \rangle + \alpha_1 \langle v_0^*, v_1^* \rangle + \dots + \alpha_n \langle v_0^*, v_n^* \rangle \\ 0 & = & \alpha_0 {\| v_0^* \|}^2 + \alpha_1 0 + \dots + \alpha_n 0 \\ 0 & = & \alpha_0 {\| v_0^* \|}^2 \\ 0 & = & \alpha_0 {\| v_0^* \|}^2 \\ \end{array} </MATH> Since <math>v_0^*</math> is nonzero, the only solution is <math>\alpha_0 = 0</math>

Operations

Scaling

Orthogonality Scaling

Scaling orthogonal vectors gives orthogonal vectors. If u is orthogonal to v then, for any scalars <math>\alpha, \beta</math> : <MATH> {\left \|\alpha u + \beta v \right \|}^2 = \alpha^2 {\left \|\alpha u \right \| }^2 + \beta^2 {\left \|\alpha v \right \| }^2 </MATH>





Discover More
Principal Component Pcr
Data Mining - Principal Component (Analysis|Regression) (PCA|PCR)

Principal Component Analysis (PCA) is a feature extraction method that use orthogonal linear projections to capture the underlying variance of the data. By far, the most famous dimension reduction approach...
Closest Point X Axis
Linear Algebra - Closest Point on a Line

How to find the closest point on a line from a point ? How to find the vector on the line that best approximates the given vector b (the closest point on the line) Letb be a vector Leta be a...
Card Puncher Data Processing
Linear Algebra - Closest point in higher dimension than a plane

Solving closest point in the span of many vectors Goal: An algorithm that, given a vector b and vectors v1, . . . , vn, finds the vector in Span {v1, . . . , vn} that is closest to b. Special case:...
Card Puncher Data Processing
Linear Algebra - Find a basis computation problem

Find a basis for a vector space Example: Find a basis for the null space of By the dot-product definition of matrix-vector multiplication, a vector v is in the null space of A if the dot-product...
Card Puncher Data Processing
Linear Algebra - Generators of a Vector Space

The generators for the set of vectors are the vectors in the following formula: where is a generating set for {[3, 0, 0], [0, 2, 0], [0, 0, 1]} is a generating set for . = Span {[3, 0,...
Card Puncher Data Processing
Linear Algebra - Inner product of two vectors

Inner products allow the rigorous introduction of intuitive geometrical notions such as the length of a vector or the angle between two vectors. They also provide the means of defining orthogonality between...
Card Puncher Data Processing
Linear Algebra - Orthogonal complement Vector Space

Let U be a of W. For each vector b in W, we can write b as the following projections: where: is in U, and is orthogonal to every vector in U. Let V be the set . V is the orthogonal complement...
Card Puncher Data Processing
Linear Algebra - Orthogonalization - Building an orthogonal set of generators

Original stated goal: Find the projection of b orthogonal to the space V spanned by arbitrary vectors Input: A list of vectors over the reals output: A list of mutually orthogonal vectors such...
Card Puncher Data Processing
Linear Algebra - Pythagorean Theorem

Following the definition of orthogonality, the Pythagorean Theorem for vectors becomes:
Image Vector
Linear Algebra - Vector

tuple in Linear algebra are called vector. A vector is a list of scalar (real number) used to represent a When the letters are in bold in a formula, it signifies that they're vectors, To represent...



Share this page:
Follow us:
Task Runner