# Linear Algebra - Closest Point on a Line

How to find the closest point on a line from a point ?

How to find the vector on the line that best approximates the given vector b (the closest point on the line)

## Lemma

• Let b be a vector
• Let a be a non-zero vector. The set $\{ \alpha.a : \alpha \in \mathbb{R}\}$ is a line L
• Let p be the point on L such that b-p is orthogonal to a

then p is the point on the line that is closest to b.

Among all the points on the line $\{ \alpha.a : \alpha \in \mathbb{R}\}$ , the closest to b is the point p on such that b − p is orthogonal to a

## Proof

### x-axis

Line is the x-axis, i.e. the set $\{(x, y) : y = 0\}$ , and point is (b1, b2).

Lemma states: closest point on the line is p = (b1, 0).

• For any other point q, the points (b1, b2), p, and q form a right triangle.
• Since q is different from p, the base is nonzero.
• By the Pythagorean Theorem, the hypotenuse’s length is greater than the height.
• This shows that q is farther from (b1, b2) than p is.

### Other Line

Let q be any point on L. The three points q, p, and b form a triangle.

• Since p and q are both on L, they are both multiples of a, so their difference p − q is also a multiple of a.
• Since b − p is orthogonal to a, therefore, it is also orthogonal to p − q
• Hence by the Pythagorean Theorem ${\| b-q \|}^2 = {\| p-q \|}^2 + {\| b-p \|}^2$
• if $q \neq p$ , then ${\| p-q \|}^2 > 0$ so $\| b-q \| < \| b-q \|$

## Definition

For any vector b and any vector a, define vectors $b^{||a}$ and $b^{\perp a}$ to be the projection of b onto Span {a} and the projection of b orthogonal to a if: $$b = b^{||a} + b^{\perp a}$$ where:

• the point $b^{||a}$ is the projection of b along a and then there is a scalar $\sigma \in \mathbb{R}$ such that $b^{||a} = \sigma a$
• the point $b^{\perp a}$ is a vector orthogonal to the line a. It's a projection orthogonal to a.

Then the following formulas are equivalent: $$\begin{array}{lrlll} b & = & b^{||a} & + & b^{\perp a} \\ b & = & \sigma a & + & b^{\perp a} \\ b^{\perp a} & = & b & - & \sigma a \\ \end{array}$$

### Lemma

One-dimensional Lemma: The point in Span {a} closest to b is $b^{||a}$ and the distance is the norm $\|b^{\perp a}\|$ .

### Corollary

For any vector b and vector a over the reals,

• the point in Span {a} that is closest to b is the projection $b^{||a}$ onto Span {a},
• and the distance between that point and b is $\| b^{\perp a} \|$ , the norm of the projection of b orthogonal to a.

### Computation

• To get $\sigma$

$\begin{array}{llll} \langle b^{\perp a}, a \rangle & = & 0 & \text{as } b^{\perp a} \text{ is orthogonal to a} \\ \langle b - b^{|| a}, a \rangle & = & 0 & \text{Substituion of b} \\ \langle b, a \rangle - sigma \langle a, a \rangle & = & 0 & \text{Using linearity and homogeneity of inner product} \\ \sigma & = & \frac{\displaystyle \langle b, a \rangle}{\displaystyle \langle a, a \rangle} & \text{Solving for } \sigma \end{array}$

• Function to get $b^{||a}$ :
# Find the vector (point) in Span a (line) that is closest to vector b (point):
# Projection along (or onto) a
# b || a
def project_along(b, a):
# because of floating-point roundoff error a*a > 1e-20 in place of a*a = 0
sigma = (b*a)/(a*a) if a*a > 1e-20 else 0
return sigma * a

• Function to get $b^{\perp a}$ :
# Projection to a
# b perp a
def project_orthogonal(b, a):
return b - project_along(b, a)  


### Example

a = [6, 2] and b = [2, 4].

The closest point on the line $\{ \alpha a : \alpha \in \mathbb{R}\}$ is the point $b^{||a} = \sigma a$ where:

$$\begin{array}{rrr} \sigma & = & \frac{a.b}{a.a} & = & \frac{6.2 + 2.4}{6.6 + 2.2} & = & \frac{20}{40} & = & \frac{1}{2} \end{array}$$

thus:

• the closest point is $\frac{1}{2} [6,2]$
• the distance to b is: $\| b^{\perp a} \| = \| [2,4]-[3,1] \| = \|[-1,3]\| = \sqrt{(-1.-1+3.3)} = \sqrt{10} = 3,5$