Chapter 2 Vectors

The word “vector” goes back to a name Hamilton gave in 1846 while organizing his theory of quaternions. He borrowed a Latin root meaning “carrier” to refer to the imaginary components that represent a geometric displacement in three-dimensional space. Although Hamilton gave the thing its name, he held to the end of his life that three-dimensional vectors could never exist apart from the quaternions that contained them. History, however, took exactly the opposite road: this geometric component, once attached to the quaternion system, went on to become the basic language of all of linear algebra.

The geometric intuition behind vectors, however, is two thousand years older than the name.

Around the fourth to third century BCE, the Aristotelian Mechanical Problems already observed that when an object takes part in two straight-line motions in different directions at once, its actual path is the diagonal of the parallelogram formed by the directions and distances of those two motions. The geometric intuition is correct, but it is in essence a kinematic composition based on the theory of proportions: it is not restricted to right angles, and it does not yet have the dynamical notion of “force.” In the first century CE, Hero of Alexandria used the construction of a parallelogram more explicitly to analyze compound motion and the decomposition of collisions—but this was still purely the language of geometric line segments, with no algebraic structure at all.

The real turning point came in 1586. The Dutch mathematician Simon Stevin used a thought experiment with a closed chain of beads hung over a double inclined plane to turn “force” from a vague experience of pushing into a line-segment length that could be quantified and operated on geometrically—the first time a physical quantity and a geometric operation were bound together rigorously. Half a century later, Galileo decomposed projectile motion into two independent components, uniform horizontal motion and accelerated vertical motion, which is in essence an orthogonal decomposition of vectors; he simply did not yet have the term. In 1687, in Corollary I of the Principia, Newton formally established the parallelogram law of forces as a foundational corollary of the entire system of classical mechanics—a universal, axiomatized construction, though its derivations still relied purely on the language of Euclidean geometry rather than on algebraic symbols.

From Aristotle to Newton, what accumulated over two thousand years was geometric and physical intuition, not a general algebraic tool.

The algebraic tool arrived from an entirely different direction. In 1843, Hamilton carved the quaternion multiplication formula into Brougham Bridge in Dublin (that is Chapter 3’s story). Seeking an algebraic system for three-dimensional space in which division is possible, he wrote a quaternion as a real scalar plus a combination of three imaginary directions. In 1846 he named the part with the three imaginary directions the vector—from the Latin vehere; the word means “carrier” or “bearer,” since a vector “carries” a point in space from the origin to its destination. Hamilton coined the term, but what he cared about most was always the quaternion as a whole, complete under multiplication and division; in his eyes the vector could never be separated out on its own.

Forty years later, this component, bound up within the quaternion system, staged a revolt.

In the 1880s the American physicist Josiah Willard Gibbs and the British engineer Oliver Heaviside independently took the quaternion apart: they kept vector addition, “brutally” split quaternion multiplication, once a seamless whole, into two independent operations—the scalar part, with its sign reversed, became the dot product (the inner product), and the vector part became the cross product—and then threw away entirely the quaternion framework that had held them together. Their reason was thoroughly practical: for a physicist, being able to solve problems simply matters more than preserving the closure of an algebraic structure. Both of the symbols “ $\cdot$ ” and “ $\times$ ” were introduced by Gibbs at this time.

This set off a fierce controversy, the “Great Vector Debate.” Hamilton’s loyalists, led by Peter Tait, denounced it as a desecration of the master’s legacy: they charged that the cross product obtained by taking things apart was a defective pseudovector (its sign behaves abnormally under a reflection of coordinates), and they condemned Gibbs’s system as a “monstrosity” that destroyed the completeness of the algebra. Gibbs and Heaviside retorted that physicists need handy geometric tools, not the obscure and cumbersome baggage of quaternions carried for the sake of mathematicians’ algebraic purity.

The argument ran for nearly twenty years. In 1901 Gibbs’s student E. B. Wilson compiled his lecture notes and published them as Vector Analysis, formally fixing the standard form of vectors found in every physics and engineering textbook today—addition, scalar multiplication, dot product, cross product, together with the symbol system we now know so well. Hamilton’s quaternions did not disappear; they are active today in computer graphics and robotics, where they describe three-dimensional rotations precisely and without any risk of gimbal lock. But the winner of that argument over a common language for physics was, in the end, Gibbs.

The geometric intuition behind vectors has a two-thousand-year history; the word “vector” has less than two hundred years; and the dot product and the cross product were settled as independent algebraic operations even later than matrix multiplication (1858). The tools we take for granted on the blackboard today are, in essence, the spoils left behind by a great collision of mathematical and physical thought.

Chapter Structure and Learning Objectives¶

The vector toolkit that Gibbs settled on is the core content of this chapter. §2.1 establishes the basic definition of a vector, moving from geometric arrow to algebraic coordinate tuple, and explains how the two formulations complement each other.

§2.2 starts from the linear operations and builds up the full picture of vector algebra step by step. Addition obeys the parallelogram law (the diagonal that Aristotle saw), and scalar multiplication corresponds to stretching—the axioms for these two operations are precisely the core of the definition of a vector space in Chapter 4. On that basis, linear combinations let us see how large a space a set of vectors can span, and the standard basis sets up a coordinate system for that space. The choice of basis is arbitrary, but once it is fixed, the correspondence between a vector and its coefficients in that basis is unique—an idea that recurs in the change of basis in Chapter 3 and in eigenvectors in Chapter 8.

§2.3 introduces the inner product, equipping the vector space with a metric structure: length, angle, and projection all follow from it. The dot product connects algebraic computation with geometric angle, and it is the finite-dimensional prototype of the inner product spaces of Chapter 9.

§2.4 takes up the standard three-dimensional cross product, whose component formula does not carry over directly to arbitrary dimensions; this chapter uses it to describe vector area and normal vectors.

By the end of this chapter, “vector” will mean to you far more than “a row of numbers”: it will mean what two thousand years of physical intuition and two hundred years of algebraic argument have jointly deposited.

2.1 Introduction to Vectors¶

In the geometric formulation, a vector has both a magnitude (a length) and a direction; in the algebraic formulation, a vector is usually a list of numbers arranged in order. Vectors can represent displacement, velocity, and force in physics, and they can equally represent a color (an RGB color, say). In the sections that follow we introduce the basic notions and operations of vectors a few at a time.

Vectors of higher dimension follow the same pattern. In general, we define the space in which $n$ -dimensional real vectors live as the $n$ -fold Cartesian product of the set of real numbers.

Definition 3 (The

n

-Dimensional Real Vector Space

\mathbb{R}^n

)

\mathbb{R}^n = \underbrace{\mathbb{R}\times\mathbb{R}\times\cdots\times\mathbb{R}}_{n \text{ copies}} = \{(x_0, x_1, \dots, x_{n-1}) \mid x_i \in \mathbb{R}\}

(3)

That is, the set of all ordered tuples of $n$ real numbers, each element $\mathbf{v} = (x_0, x_1, \dots, x_{n-1})$ of which is an $n$ -dimensional vector. This definition carries over the Cartesian product of Definition 5 from Chapter 1, and it makes good on the preview of higher-dimensional coordinate spaces given in Exercise 3. Everything said about $\mathbb{R}^n$ in the rest of the book (the standard basis, the inner product, the exercises in $\mathbb{R}^4$ , and so on) rests on this definition.

§2.1 Summary

This section answered the question “what exactly is a vector?”—a question that looks simple and is in fact deep. A vector can be a geometric arrow, with a direction and a length, pointing at some location in space; it can equally be a neatly arranged column of numbers. Once coordinates are introduced, the two readings are completely equivalent. From $(x, y)$ in two dimensions to $(x, y, z)$ in three, and on to $\mathbb{R}^n$ , the definition of a vector extends naturally from geometric intuition to arbitrary dimension, so that colors, sounds, pixels, and even the hidden states of a neural network can all be brought into one and the same framework.

This dual identity as “geometric object” and “sequence of numbers” is no accident; it is the fundamental reason the language of vectors is so widely applicable. Geometric intuition supplies understanding, and algebraic structure supplies computation. This section fixed the notational conventions for vectors (bold lowercase $\mathbf{v}$ , and the Dirac notation $|\mathbf{v}\rangle$ ), laying the linguistic groundwork for every operation to come. The next section builds on that groundwork by introducing addition and scalar multiplication, and asks which operations can be performed between vectors and what rules those operations obey.

2.2 Linear Operations on Vectors¶

The most basic operations between vectors are addition and scalar multiplication. These operations constitute the foundation of linear algebra.

# This code block is not shown in the printed book; only the figure is displayed
import numpy as np
import matplotlib.pyplot as plt

np.set_printoptions(precision=4, suppress=True, linewidth=100)

# --- Visual hierarchy helpers ---
def print_header(title):
    print("=" * 60)
    print(f"  {title}")
    print("=" * 60)

def print_step(step, desc):
    print(f"\n▶ Step {step}: {desc}")
    print("-" * 40)

def compare_print(label, actual, expected_desc):
    print(f"\n[{label}]")
    print(f"  Computed value: {actual}")
    print(f"  Expected value: {expected_desc}")

# Fonts: the English edition needs no CJK font, so nothing is downloaded when _lang == 'en';
# the Chinese editions use this same block to fetch Noto Sans TC/SC where no CJK font is installed
import os, urllib.request
import matplotlib.font_manager as fm
_lang = 'en'
_cjk = ['Microsoft JhengHei', 'PingFang TC', 'Noto Sans CJK TC', 'Noto Sans TC']
_have = {f.name for f in fm.fontManager.ttflist}
if _lang != 'en' and not _have & set(_cjk):
    _font = os.path.join(os.path.expanduser('~'), '.cache', 'fonts', 'NotoSansTC.ttf')
    try:
        if not os.path.exists(_font):
            os.makedirs(os.path.dirname(_font), exist_ok=True)
            urllib.request.urlretrieve('https://github.com/google/fonts/raw/main/ofl/notosanstc/NotoSansTC%5Bwght%5D.ttf', _font + '.part')
            os.replace(_font + '.part', _font)
        fm.fontManager.addfont(_font)
        _have.add('Noto Sans TC')
    except OSError as err:
        print('Could not download the CJK font; Chinese text in figures may not display:', err)
plt.rcParams['font.family'] = [f for f in _cjk if f in _have] + ['DejaVu Sans']
plt.rcParams['axes.unicode_minus'] = False

# --- Color palette (Accent Mix) ---
C_BG   = "#F8F8F8"   # canvas background
C_GRID = "#D6D6D6"   # gridlines
C_AXIS = "#000000"   # axis text
C_V1   = "#57068C"   # vector a+b (violet)
C_V2   = "#006385"   # vector a (deep blue)
C_T1   = "#2AD2C9"   # vector b (teal)
C_AUX  = "#AB82C5"   # parallelogram guide lines (light violet)

# ============================================================
print_header("Example 2.1 | Geometric Picture of Vector Addition")
# ============================================================

# --- Step 0: define the vectors a, b and their sum ---
a = np.array([2, 3])
b = np.array([4, -1])
c = a + b   # a + b

print_step(0, "Define the vectors a, b and compute a+b")
compare_print("a + b", c, f"[{a[0]+b[0]}, {a[1]+b[1]}]")

# --- Step 1: set up the canvas ---
fig, ax = plt.subplots(figsize=(5, 5))
ax.set_facecolor(C_BG)
ax.grid(True, color=C_GRID, linewidth=0.8, alpha=0.7)
ax.set_aspect('equal', adjustable='box')
ax.set_box_aspect(1)

ax.set_xlim(-1, 8)
ax.set_ylim(-3, 6)

# --- estimate data_range to fix the arrow width for quiver ---
x_min, x_max = ax.get_xlim()
data_range = x_max - x_min   # xlim(-1, 8) → data_range = 9
kw = dict(angles='xy', scale_units='xy', scale=1, units='xy',
          width=data_range * 0.012)

# --- Step 2: draw the vectors a, b and a+b ---
ax.quiver(0, 0, a[0], a[1], color=C_V2, **kw, label=r'$\mathbf{a}$')
ax.quiver(0, 0, b[0], b[1], color=C_T1, **kw, label=r'$\mathbf{b}$')
ax.quiver(0, 0, c[0], c[1], color=C_V1, **kw, label=r'$\mathbf{a}+\mathbf{b}$')

# --- Step 3: draw the dashed parallelogram guide lines ---
ax.plot([a[0], c[0]], [a[1], c[1]], '--', color=C_AUX, linewidth=1.2, alpha=0.8)
ax.plot([b[0], c[0]], [b[1], c[1]], '--', color=C_AUX, linewidth=1.2, alpha=0.8)

ax.set_xlabel("$x_0$", fontsize=12, color=C_AXIS)
ax.set_ylabel("$x_1$", fontsize=12, color=C_AXIS)
ax.set_title("Geometric Picture of Vector Addition", fontsize=14, color=C_AXIS)
ax.legend(fontsize=12)
plt.tight_layout()
plt.show()

# This code block is not shown in the printed book; only the figure is displayed
import numpy as np
import matplotlib.pyplot as plt

np.set_printoptions(precision=4, suppress=True, linewidth=100)

# --- Visual hierarchy helpers ---
def print_header(title):
    print("=" * 60)
    print(f"  {title}")
    print("=" * 60)

def print_step(step, desc):
    print(f"\n▶ Step {step}: {desc}")
    print("-" * 40)

def compare_print(label, actual, expected_desc):
    print(f"\n[{label}]")
    print(f"  Computed value: {actual}")
    print(f"  Expected value: {expected_desc}")

# Fonts: the English edition needs no CJK font, so nothing is downloaded when _lang == 'en';
# the Chinese editions use this same block to fetch Noto Sans TC/SC where no CJK font is installed
import os, urllib.request
import matplotlib.font_manager as fm
_lang = 'en'
_cjk = ['Microsoft JhengHei', 'PingFang TC', 'Noto Sans CJK TC', 'Noto Sans TC']
_have = {f.name for f in fm.fontManager.ttflist}
if _lang != 'en' and not _have & set(_cjk):
    _font = os.path.join(os.path.expanduser('~'), '.cache', 'fonts', 'NotoSansTC.ttf')
    try:
        if not os.path.exists(_font):
            os.makedirs(os.path.dirname(_font), exist_ok=True)
            urllib.request.urlretrieve('https://github.com/google/fonts/raw/main/ofl/notosanstc/NotoSansTC%5Bwght%5D.ttf', _font + '.part')
            os.replace(_font + '.part', _font)
        fm.fontManager.addfont(_font)
        _have.add('Noto Sans TC')
    except OSError as err:
        print('Could not download the CJK font; Chinese text in figures may not display:', err)
plt.rcParams['font.family'] = [f for f in _cjk if f in _have] + ['DejaVu Sans']
plt.rcParams['axes.unicode_minus'] = False

# --- Color palette (Accent Mix) ---
C_BG   = "#F8F8F8"
C_GRID = "#D6D6D6"
C_AXIS = "#000000"
C_V1   = "#57068C"   # 2v, scaled up (violet)
C_V2   = "#006385"   # v, the original vector (deep blue)
C_T1   = "#2AD2C9"   # 0.5v, scaled down (teal)
C_WARN = "#FF5D47"   # −v, reversed (orange)
C_AUX  = "#AB82C5"   # dashed reference vector (light violet)

# ============================================================
print_header("Example 2.2 | Geometric Picture of Scalar Multiplication")
# ============================================================

# --- Step 0: define the original vector and the list of scalars ---
v       = np.array([2, 3])
scalars = [0.5,  1.0,  2.0,   -1.0]
colors  = [C_T1, C_V2, C_V1,   C_WARN]
labels  = ['0.5v (scaled down)', 'v (original)', '2v (scaled up)', '−v (reversed)']

print_step(0, "Define the vector v and the scalars")
print(f"  original vector v = {v}")

# --- Step 1: verify ‖λv‖ = |λ|·‖v‖ ---
print_step(1, "Verify the length formula for scalar multiplication: ‖λv‖ = |λ|·‖v‖")
norm_v = np.linalg.norm(v)
for lam in scalars:
    lv = lam * v
    compare_print(
        f"λ = {lam:+.1f}",
        f"‖λv‖ = {np.linalg.norm(lv):.4f}",
        f"|λ|·‖v‖ = {abs(lam) * norm_v:.4f}"
    )

# --- Step 2: set up a 2×2 grid of subplots ---
fig, axes = plt.subplots(2, 2, figsize=(10, 10))
axes_flat = axes.flatten()

# set the axis limits of each subplot first, then compute data_range
for ax in axes_flat:
    ax.set_xlim(-5, 5)
    ax.set_ylim(-7, 7)

ranges     = [a.get_xlim()[1] - a.get_xlim()[0] for a in axes_flat]
data_range = sum(ranges) / len(ranges)   # xlim(-5,5) → 10
kw = dict(angles='xy', scale_units='xy', scale=1, units='xy',
          width=data_range * 0.012)

# --- Step 3: draw the scalar multiples, one per subplot ---
for i, (lam, color, label) in enumerate(zip(scalars, colors, labels)):
    ax      = axes_flat[i]
    scaled_v = lam * v

    ax.set_facecolor(C_BG)
    ax.grid(True, color=C_GRID, linewidth=0.8, alpha=0.7)
    ax.set_aspect('equal', adjustable='box')
    ax.set_box_aspect(1)
    # reference vector v (shown in the auxiliary color when λ≠1)
    if lam != 1.0:
        ax.quiver(0, 0, v[0], v[1], color=C_AUX, alpha=0.5, **kw,
                  label=r'$\mathbf{v}$ (reference)')

    # λv, the principal vector
    ax.quiver(0, 0, scaled_v[0], scaled_v[1], color=color, **kw,
              label=rf'${lam}\mathbf{{v}}$')

    # label at the tip of the vector
    off_x = 0.3 if scaled_v[0] >= 0 else -0.8
    off_y = 0.3 if scaled_v[1] >= 0 else -1.0
    ax.text(scaled_v[0] + off_x, scaled_v[1] + off_y,
            f'{lam}v', fontsize=13, fontweight='bold', color=color,
            bbox=dict(boxstyle="round,pad=0.2", facecolor='white', alpha=0.8))

    # length information
    length = np.linalg.norm(scaled_v)
    ax.text(0.02, 0.98, f'‖λv‖ = {length:.2f}',
            transform=ax.transAxes, fontsize=11, verticalalignment='top',
            bbox=dict(boxstyle="round,pad=0.3", facecolor='white', alpha=0.8))

    ax.plot(0, 0, 'o', color=C_AXIS, markersize=7)
    ax.set_xlabel("$x_0$", fontsize=12, color=C_AXIS)
    ax.set_ylabel("$x_1$", fontsize=12, color=C_AXIS)
    ax.set_title(label, fontweight='bold', color=color, fontsize=14)
    ax.legend(fontsize=11, loc='lower right')

fig.suptitle('Geometric Picture of Scalar Multiplication', fontsize=18, fontweight='bold',
             color=C_AXIS, y=0.995)
plt.tight_layout()
plt.show()

2.2.1 Linear Combinations¶

A Preview of Two Related Notions: Span and Linear Independence

Linear combinations lead naturally to two notions that run through the entire book. We give the intuition here and leave the rigorous definitions to Chapter 4.

Given a set of vectors, the collection of all of their linear combinations is called the span of that set. For example, given two non-parallel vectors $\mathbf{u}$ and $\mathbf{v}$ in the plane, their linear combinations $\lambda_0\mathbf{u}+\lambda_1\mathbf{v}$ range over the whole plane, and we say that $\{\mathbf{u},\mathbf{v}\}$ spans $\mathbb{R}^2$ .

If no vector in a set can be written as a linear combination of the others (that is, if they are mutually non-redundant), the set is called linearly independent; otherwise it is linearly dependent. Two vectors in the plane are linearly independent exactly when, geometrically, they are not collinear. These two terms are used with this intuitive meaning in the remainder of the chapter and in the exercises; the rigorous definitions appear in Chapter 4.

Linear combinations can be applied to define line segments in vector form.

2.2.2 The Standard Basis¶

In $\mathbb{R}^n$ , the standard basis is a particular set of unit vectors that are mutually orthogonal and can represent any vector in the space.

Important

The properties below use the inner product $\mathbf{e}_i \cdot \mathbf{e}_j$ and the length $\|\mathbf{e}_i\|$ , which are not formally defined until §2.3. For now you may read them with the geometric intuition of “mutually perpendicular” and “of length 1,” and return to them once §2.3 has supplied the rigorous definitions.

Important properties of the standard basis:

Orthogonality: any two distinct standard basis vectors are perpendicular, that is, $\mathbf{e}_i \cdot \mathbf{e}_j = 0$ when $i \neq j$
Unit length: each standard basis vector has length 1, that is, $\|\mathbf{e}_i\| = 1$
Completeness: every vector can be expressed as a linear combination of the standard basis vectors
Uniqueness: the representation of each vector in the standard basis is unique

§2.2 Summary

This section answered the question of how vectors may be combined with one another, and brought out the geometric meaning and algebraic structure of those combinations. Vector addition rests intuitively on the parallelogram law, while scalar multiplication governs the scaling and the reversal of a vector; together they are called the linear operations, they satisfy commutativity, associativity, and distributivity, and they constitute the concrete prototype of the eight axioms of an abstract vector space in Chapter 4.

Built on top of them, the linear combination $\sum_i \lambda_i \mathbf{v}_i$ provides a systematic way of generating new vectors: varying the coefficients $\lambda_i$ lets us roam through the space. The parametric representation of a line segment, $\mathbf{p}=(1-t)\mathbf{a}+t\mathbf{b}$ , is the simplest linear combination, and the centroid of a triangle is the special case of equal weights; both foreshadow the emerging notion of a span. The introduction of the standard basis $\{\mathbf{e}_0, \dots, \mathbf{e}_{n-1}\}$ is more crucial still: it guarantees that every vector in $\mathbb{R}^n$ has a unique representation in components, $\mathbf{v}=\sum_{i=0}^{n-1}v_i\mathbf{e}_i$ , tying “decomposition” to “uniqueness.” The next section equips this algebraic structure with a metric—the inner product—so that we may speak of lengths, angles, and projections.

2.3 The Inner Product and Projection¶

Definition 10 (The

l_2

Norm, or Euclidean Norm)

For an $n$ -dimensional real vector $\mathbf{v} = \begin{bmatrix} v_0 \\ v_1 \\ \vdots \\ v_{n-1} \end{bmatrix}$ , the $l_2$ norm is defined as the square root of the inner product of the vector with itself:

\|\mathbf{v}\|_2 = \sqrt{\mathbf{v} \cdot \mathbf{v}}= \sqrt{\langle \mathbf{v} | \mathbf{v} \rangle} = \sqrt{\sum_{i=0}^{n-1} v_i^2}

(25)

This norm is also called the Euclidean norm; it represents the length of the vector in Euclidean space. Throughout this book the length of a vector is by default the $l_2$ norm, so that $\|\mathbf{v}\|= \|\mathbf{v}\|_2$ . In Chapters 9 and 11 we shall further consider norms of matrices, as well as the genuine $l_1$ norm (the sum of the absolute values of the components, $\sum_{i=0}^{n-1}|v_i|$ ) and the $l_\infty$ norm (the largest of the absolute values of the components).

The Cosine Formula, the Cauchy-Schwarz Inequality, and the Triangle Inequality

In the case of $\mathbb{R}^2$ , plane geometry gives us the cosine formula $\mathbf{u}\cdot\mathbf{v}=\|\mathbf{u}\|\|\mathbf{v}\|\cos\theta$ , from which it is easy to derive that the $l_2$ norm satisfies the Cauchy-Schwarz inequality and the triangle inequality. For a general real Cartesian coordinate system we shall instead start from the Cauchy-Schwarz inequality, show that the cosine formula is well-defined, and then derive the triangle inequality. In Chapter 9 we shall revisit this: for a general Euclidean space, two vectors in a high-dimensional space can be rotated into $\mathbb{R}^2$ with their angle preserved, so the cosine formula in high dimensions may be understood as the very same formula as in two dimensions.

The Cauchy-Schwarz inequality admits several different proofs. We give one of the simpler ones here.

Proof 1 (The Cauchy-Schwarz Inequality)

If $\mathbf{u}=\mathbf{0}$ , the conclusion holds trivially; assume from now on that $\mathbf{u}\neq\mathbf{0}$ . In $\mathbb{R}^n$ we have vectors $\mathbf{u}=(u_0,u_1,\dots,u_{n-1})$ and $\mathbf{v}=(v_0,v_1,\dots,v_{n-1})$ .
Consider the quadratic polynomial in $x$

(u_0 x + v_0)^2+(u_1 x + v_1)^2+ \dots + (u_{n-1} x + v_{n-1})^2= \left(\sum_{i=0}^{n-1} u_i^2\right)x^2 +2\left(\sum_{i=0}^{n-1} u_i v_i\right)x + \sum_{i=0}^{n-1} v_i^2 \geq 0

(35)

Since it has at most one real root, its discriminant is less than or equal to 0, so that

4\left(\sum_{i=0}^{n-1} u_i v_i\right)^2 - 4 \left(\sum_{i=0}^{n-1} u_i^2\right)\left(\sum_{i=0}^{n-1} v_i^2\right)\leq 0

(36)

Rearranging gives $|\mathbf{u}\cdot\mathbf{v}|\le\|\mathbf{u}\|\|\mathbf{v}\|$ . $\square$

Definition 12 (The Angle Between Vectors in Space)

Consider two nonzero real vectors $\mathbf{u}$ and $\mathbf{v}$ issuing from the origin in space. By the Cauchy-Schwarz inequality $|\langle \mathbf{u} |\mathbf{v} \rangle |\le\|\mathbf{u}\| \|\mathbf{v}\|$ , the quotient $\dfrac{\langle \mathbf{u} |\mathbf{v} \rangle}{\|\mathbf{u}\| \|\mathbf{v}\|}$ necessarily lies in $[-1,1]$ , so we may define the cosine formula

\cos \theta = \frac{\langle \mathbf{u} |\mathbf{v} \rangle}{\|\mathbf{u}\| \|\mathbf{v}\|}

(37)

and call $\theta$ (with $0\leq\theta \leq \pi$ ) the angle between the two vectors. The sign of the inner product reflects the type of angle directly: when $\langle \mathbf{u} |\mathbf{v} \rangle > 0$ we have $0\leq\theta<\pi/2$ ( $\theta=0$ meaning the same direction), and when $\langle \mathbf{u} |\mathbf{v} \rangle < 0$ we have $\pi/2<\theta\leq\pi$ ( $\theta=\pi$ meaning opposite directions). In particular, when $\cos \theta =0$ (that is, when the inner product vanishes) we write $\mathbf{u} \perp \mathbf{v}$ and say that the two vectors are perpendicular.

Example 10 (The Distance from a Point to a Line in Space)

Consider a point $\mathbf{x}_0$ in space and a line $l$ ,

\mathbf{x} = \mathbf{x}_1 + t\mathbf{a}

(41)

Dropping a perpendicular from the point $\mathbf{x}_0$ to the line $l$ gives a foot of the perpendicular at a point $\mathbf{p}$ , so that $\mathbf{x}_0$ , $\mathbf{x}_1$ , and $\mathbf{p}$ form a right triangle.

Moreover,

\mathbf{p} = \mathbf{x}_1 + \frac{\langle \mathbf{a} | \mathbf{x_0-x_1} \rangle}{\langle \mathbf{a} | \mathbf{a} \rangle}\mathbf{a}

(42)

and

\| \mathbf{x}_0 - \mathbf{x}_1 \|^2 = \| \mathbf{x}_0 - \mathbf{p}\|^2+ \| \mathbf{p}-\mathbf{x}_1 \|^2 = \| \mathbf{x}_0 - \mathbf{p}\|^2+ \frac{\langle \mathbf{a} | \mathbf{x_0-x_1} \rangle^2}{\langle \mathbf{a} | \mathbf{a} \rangle}

(43)

Note that $\| \mathbf{x}_0 - \mathbf{p}\|$ is the shortest distance from the point $\mathbf{x}_0$ to the line $l$ .

Example 11 (The Inner Product and Planes)

The inner product also gives a convenient way to define the equation of a plane. Consider a unit normal vector $\mathbf{n}$ and the set $\{ \mathbf{x} | \langle \mathbf{n} , \mathbf{x} \rangle= d_0 \}$ . One sees readily that this set constitutes a plane $S_0$ at distance $|d_0|$ from the origin, and that the points of this plane all have the same signed component $d_0$ along the direction $\mathbf{n}$ .

Consider a point $\mathbf{p}$ off the plane, and suppose $\langle \mathbf{n} , \mathbf{p} \rangle= d_1$ . Then $d_1$ may be used to define another plane $S_1$ : $\{ \mathbf{x} | \langle \mathbf{n} , \mathbf{x} \rangle= d_1 \}$ . Since $\mathbf{n}$ is a unit vector, the signed component of any point $\mathbf{x}$ along the direction $\mathbf{n}$ is exactly $\langle \mathbf{n} , \mathbf{x} \rangle$ , so the coordinates of the points of $S_0$ and $S_1$ along the direction $\mathbf{n}$ are fixed at $d_0$ and $d_1$ respectively. The two planes have the same normal vector $\mathbf{n}$ and are therefore parallel, and their shortest distance is the difference of those coordinates along the direction $\mathbf{n}$ , namely $|d_0-d_1|$ ; the distance from $\mathbf{p}\in S_1$ to the plane $S_0$ is measured along $\mathbf{n}$ in the same way and is therefore also $|d_0-d_1|$ .

Exercise 11 (◇The Parallelogram Law and the Polarization Identity)

Prove that the norm induced by an inner product satisfies the parallelogram law: any two vectors $\mathbf{u}$ and $\mathbf{v}$ in space satisfy

\|\mathbf{u+v} \|^2 + \|\mathbf{u-v}\|^2 = 2(\|\mathbf{u}\|^2 + \|\mathbf{v}\|^2)

(44)

Conversely, if a norm on a real space satisfies the parallelogram law, prove that we may define an inner product by

\langle \mathbf{u} | \mathbf{v} \rangle := \frac{1}{4}(|\mathbf{u+v}|^2 - |\mathbf{u-v}|^2)

(45)

Similarly, if a norm on a complex space satisfies the parallelogram law, prove that we may define a complex inner product by

\langle \mathbf{u} | \mathbf{v} \rangle := \frac{1}{4}(\|\mathbf{u+v}\|^2 - \|\mathbf{u-v}\|^2 - i\|\mathbf{u+iv}\|^2 + i\|\mathbf{u-iv}\|^2)

(46)

§2.3 Summary

This section answered the question of how distance and direction are to be measured in a vector space. The inner product $\langle \mathbf{u} | \mathbf{v} \rangle = \sum_{i=0}^{n-1} u_i v_i$ is a purely algebraic quantity, yet it conceals a wealth of geometric information: the $l_2$ norm it induces, $\|\mathbf{v}\|_2 = \sqrt{\langle\mathbf{v}|\mathbf{v}\rangle}$ , measures the length of a vector, while the Cauchy-Schwarz inequality $|\langle\mathbf{u}|\mathbf{v}\rangle| \le \|\mathbf{u}\|\|\mathbf{v}\|$ guarantees, when both vectors are nonzero, the legitimacy of the definition of angle—it is exactly this inequality that keeps the value of $\cos\theta = \langle\mathbf{u}|\mathbf{v}\rangle/(\|\mathbf{u}\|\|\mathbf{v}\|)$ inside $[-1, 1]$ , so that “angle” remains meaningful as a geometric notion in any dimension.

When $\mathbf u\neq\mathbf0$ , the orthogonal projection $\text{Proj}_{\mathbf{u}}\mathbf{v} = \frac{\langle\mathbf{u}|\mathbf{v}\rangle}{\|\mathbf{u}\|^2}\mathbf{u}$ is one of the most important applications of the inner product: it extracts “the component of a vector along a given direction” as an independent geometric operation, and computing the distance from a point to a line is a direct application of it. These notions will be generalized to complex inner product spaces in Chapter 9, while the exercise on the polarization identity at the end of this section previews the possibility of recovering the inner product from the norm. The next section turns to the standard three-dimensional cross product, which supplies another tool for measuring geometric relationships.

2.4 The Cross Product¶

This chapter discusses the standard three-dimensional cross product. It has wide application in physics, engineering, and computer graphics, and it provides a natural tool for describing rotations, torques, and normal vectors.

The Dimensional Restriction on the Cross Product

This chapter discusses the standard three-dimensional cross product; its component formula does not carry over directly to arbitrary dimensions. Seven dimensions admit a generalization that requires additional algebraic structure, and in general dimensions one usually turns to tools such as the wedge product instead. The exercises at the end of Chapter 3 contain further practice on the cross product.

Handling two-dimensional vectors. Two-dimensional vectors $(u_0,u_1)$ and $(v_0,v_1)$ may be regarded as the three-dimensional vectors $(u_0,u_1,0)$ and $(v_0,v_1,0)$ , after which the cross product may be computed. The result always has the form $(0,0,u_0v_1-u_1v_0)$ , that is, it points purely along the $z$ axis.

For this reason the “cross product” in two dimensions is sometimes defined as the scalar $u_0v_1-u_1v_0$ , but this is in substance the $z$ component of the three-dimensional cross product.

The Geometric Meaning of the Cross Product

Geometric properties.

Perpendicularity: the cross product $\mathbf{u}\times\mathbf{v}$ is perpendicular to the plane containing $\mathbf{u}$ and $\mathbf{v}$
Orientation: determined by the right-hand rule—curl the fingers of the right hand from $\mathbf{u}$ toward $\mathbf{v}$ , and the thumb points in the direction of the cross product
Magnitude: the length equals the area of the parallelogram formed by the two vectors

\|\mathbf{u}\times\mathbf{v}\|=\|\mathbf{u}\|\|\mathbf{v}\||\sin\theta|

(51)

where $\theta$ is the angle between the two nonzero real vectors. If either vector is zero, the component definition shows that the cross product is zero, and no angle needs to be defined.

Proof 2 (The Formula for the Magnitude of a Cross Product)

We must prove that $\|\mathbf{u}\times\mathbf{v}\|^2 = \|\mathbf{u}\|^2 \|\mathbf{v}\|^2 \sin^2\theta$
Use the trigonometric identity $\sin^2\theta = 1 - \cos^2\theta$
By the geometric meaning of the inner product, $\mathbf{u}\cdot\mathbf{v} = \|\mathbf{u}\|\|\mathbf{v}\|\cos\theta$
Therefore $\|\mathbf{u}\|^2 \|\mathbf{v}\|^2 \sin^2\theta = \|\mathbf{u}\|^2 \|\mathbf{v}\|^2 - (\mathbf{u}\cdot\mathbf{v})^2$
$\underline{\text{Substituting the definitions of the cross product and the inner product into the identity above and carrying out the algebra verifies it}}$

$\square$

# Here we use symbolic computation to help verify the cross product formula
import sympy as sp
from sympy.vector import CoordSys3D
u0,u1,u2,v0,v1,v2 = sp.symbols("u0,u1,u2,v0,v1,v2",real=True)

N = CoordSys3D('N')
u = u0 * N.i + u1 * N.j + u2 * N.k
v = v0 * N.i + v1 * N.j + v2 * N.k
print("inner product of u and v :",u.dot(v))
print("cross product of u and v :",u.cross(v))

assert sp.simplify(u.cross(v).dot(u.cross(v)) +u.dot(v) * u.dot(v)- u.dot(u) * v.dot(v) ) ==0 ,"The cross product formula is wrong!"
print("The cross product formula is correct!")

Example 14 (Extending a Line in Three-Dimensional Space to a Perpendicular Plane)

Suppose we have a line $l$ and a plane $S_0$ in three-dimensional space:

The line $l$ :

\mathbf{x} = \mathbf{p} + t\mathbf{d} \quad (-\infty < t < \infty)

(54)

where $\mathbf{p}$ is a point on the line and $\mathbf{d}$ is its direction vector.

The plane $S_0$ :

\mathbf{x} = \mathbf{q} + s\mathbf{a} + u\mathbf{b} \quad (-\infty < s,u < \infty)

(55)

whose normal vector is $\mathbf{n_0} = \mathbf{a} \times \mathbf{b}$ (after normalization).

If $\mathbf{d}$ is not parallel to $\mathbf{n_0}$ , that is, if $\mathbf{n_0}\times\mathbf{d}\neq\mathbf{0}$ , then there is a unique plane $S_1$ that contains $l$ and is perpendicular to $S_0$ .

Choose the normal vector of the plane $S_1$ to be

\mathbf{n_1} = \mathbf{n_0} \times \mathbf{d}

(56)

We see that $\mathbf{n_1} \perp \mathbf{n_0}$ , so the two planes are perpendicular (which means that the projection of either plane onto the other degenerates to a line, or equivalently that each of them contains a line passing perpendicularly through the other), and that $\mathbf{n_1}$ and $\mathbf{d}$ are linearly independent.

Set $\mathbf{e}= \mathbf{n_1} \times \mathbf{d}$

The plane $S_1$ :

\mathbf{x} = \mathbf{p} + t\mathbf{d} + v\mathbf{e} \quad (-\infty < t,v < \infty)

(57)

Example 15 (Extending Skew Lines to Parallel Planes)

For two skew lines $l_0$ and $l_1$ in three-dimensional space:

The line $l_0$ : $\mathbf{x} = \mathbf{p_0} + s\mathbf{d_0}$

The line $l_1$ : $\mathbf{x} = \mathbf{p_1} + t\mathbf{d_1}$

When $\mathbf{d_0} \times \mathbf{d_1} \neq \mathbf{0}$ , the two lines may be extended separately to parallel planes:

The plane $S_0$ (containing the line $l_0$ ):

\mathbf{x} = \mathbf{p_0} + s\mathbf{d_0} + t\mathbf{d_1} \quad (-\infty < s,t < \infty)

(58)

The plane $S_1$ (containing the line $l_1$ ):

\mathbf{x} = \mathbf{p_1} + s\mathbf{d_0} + t\mathbf{d_1} \quad (-\infty < s,t < \infty)

(59)

Proof of parallelism. Both planes have normal vector $\mathbf{n} = \mathbf{d_0} \times \mathbf{d_1}$ , so $S_0 \parallel S_1$ .

Computing the distance. The distance between the two parallel planes is the projected length of the vector $(\mathbf{p_1} - \mathbf{p_0})$ along the normal vector $\mathbf{n}$ :

d = \frac{|(\mathbf{p_1} - \mathbf{p_0}) \cdot \mathbf{n}|}{\|\mathbf{n}\|} = \frac{|(\mathbf{p_1} - \mathbf{p_0}) \cdot (\mathbf{d_0} \times \mathbf{d_1})|}{\|\mathbf{d_0} \times \mathbf{d_1}\|}

(60)

This distance is exactly the shortest distance between the two skew lines.

§2.4 Summary

This section answered the question of whether, besides the inner product, three-dimensional space carries any other natural multiplicative structure. The cross product $\mathbf{u} \times \mathbf{v}$ yields a new vector perpendicular to both $\mathbf{u}$ and $\mathbf{v}$ , whose magnitude $\|\mathbf{u} \times \mathbf{v}\| = \|\mathbf{u}\|\|\mathbf{v}\||\sin\theta|$ equals the area of the parallelogram the two vectors span, and whose orientation is fixed by the right-hand rule. This geometric reading makes the cross product a natural tool for computing normal vectors, torques, and areas.

The cross product does, however, carry one fundamental restriction: the cross product used in this chapter is the standard three-dimensional one; its component formula does not carry over directly to arbitrary dimensions, and seven dimensions admit a generalization that requires additional structure. This restriction is not a technical defect but a deep feature of three-dimensional geometry—behind it lie the subtle structures of quaternion algebra and Lie groups, a topic touched on further in the historical account opening Chapter 3. This section also displayed the power of the cross product in geometric problems: taking the cross product of the direction vectors as a common normal vector reduces the shortest distance between two skew lines to the distance between two parallel planes, unifying several geometric propositions elegantly. The next section puts all of the vector operations of this chapter into practice through Python code.

2.5 Vector Operations in Python¶

The NumPy library in Python provides powerful facilities for vector operations. Here are some basic examples:

import numpy as np

np.set_printoptions(precision=4, suppress=True, linewidth=100)

# ── Helper functions ────────────────────────────────────────
def print_header(title):
    print("=" * 60)
    print(f"  {title}")
    print("=" * 60)

def print_step(step, desc):
    print(f"\n▶ Step {step}: {desc}")
    print("-" * 40)

def compare_print(label, actual, expected_desc):
    print(f"  {label}")
    print(f"    computed = {actual}")
    print(f"    expected = {expected_desc}")

# ============================================================
print_header("Example 2.5.1 | Basic Vector Operations")
# ============================================================

# --- Step 0 ---
print_step(0, "Create the vectors 𝐚, 𝐛 ∈ ℝ³")
a = np.array([2, 3, 1], dtype=float)
b = np.array([4, -1, 2], dtype=float)
print(f"  𝐚 = {a}")
print(f"  𝐛 = {b}")

# --- Step 1 ---
print_step(1, "Vector addition and scalar multiplication")
compare_print("𝐚 + 𝐛", a + b, "entrywise addition")
compare_print("3𝐚",    3 * a,  "entrywise multiplication by the scalar λ = 3")
compare_print("0.3𝐚 + 0.7𝐛 (segment parameter t = 0.7)",
              0.3*a + 0.7*b,
              "(1−t)𝐚 + t𝐛, t ∈ [0,1]")

# --- Step 2 ---
print_step(2, "Parametric representation of a segment: 𝐩(t) = (1−t)𝐚 + t𝐛")
def line_segment(a, b, t):
    """Return the point on the segment 𝐚𝐛 with parameter t"""
    return (1 - t) * a + t * b

p0 = np.array([1, 2], dtype=float)
p1 = np.array([5, 4], dtype=float)
midpoint = line_segment(p0, p1, 0.5)
compare_print("midpoint 𝐩(0.5)", midpoint, "(𝐩₀ + 𝐩₁) / 2")

# ============================================================
print_header("Example 2.5.2 | Inner Product, Norm and the Cauchy-Schwarz Inequality")
# ============================================================

# --- Step 0 ---
print_step(0, "Create the vectors 𝐮, 𝐯 ∈ ℝ³")
u = np.array([1, 2, 3], dtype=float)
v = np.array([4, -1, 2], dtype=float)
print(f"  𝐮 = {u}")
print(f"  𝐯 = {v}")

# --- Step 1 ---
print_step(1, "Three equivalent ways to compute the inner product ⟨𝐮|𝐯⟩ = Σ uᵢvᵢ")
dot1 = np.dot(u, v)
dot2 = u @ v
dot3 = np.inner(u, v)
print(f"  np.dot(u, v)   = {dot1}")
print(f"  u @ v          = {dot2}")
print(f"  np.inner(u, v) = {dot3}")
print(f"  all three agree: {dot1 == dot2 == dot3}")

# --- Step 2 ---
print_step(2, "The L₂ norm and normalization")
def normalize(v):
    """Normalize a vector to the unit vector v̂ = v / ‖v‖"""
    norm = np.linalg.norm(v)
    return v / norm if norm != 0 else v

norm_v = np.linalg.norm(v)
v_hat  = normalize(v)
compare_print("‖𝐯‖₂",       norm_v, "√(Σ vᵢ²)")
compare_print("v̂ = 𝐯/‖𝐯‖₂", v_hat,  "‖v̂‖ should be 1.0")
print(f"  check ‖v̂‖ = {np.linalg.norm(v_hat):.4f}")

# --- Step 3 ---
print_step(3, "Angle between vectors: cos θ = ⟨𝐮|𝐯⟩ / (‖𝐮‖‖𝐯‖)")
def vector_angle(u, v, degrees=True):
    """Compute the angle between two vectors (in degrees by default)"""
    cos_theta = np.dot(u, v) / (np.linalg.norm(u) * np.linalg.norm(v))
    cos_theta = np.clip(cos_theta, -1, 1)          # numerical safeguard
    theta = np.arccos(cos_theta)
    return np.degrees(theta) if degrees else theta

angle = vector_angle(u, v)
print(f"  θ(𝐮, 𝐯) = {angle:.4f}°")

# --- Step 4 ---
print_step(4, "Cauchy-Schwarz inequality |⟨𝐮|𝐯⟩| ≤ ‖𝐮‖‖𝐯‖")
lhs = abs(np.dot(u, v))
rhs = np.linalg.norm(u) * np.linalg.norm(v)
compare_print("|⟨𝐮|𝐯⟩|",    lhs, "left-hand side")
compare_print("‖𝐮‖ · ‖𝐯‖", rhs, "right-hand side (upper bound)")
print(f"  inequality holds: {lhs <= rhs + 1e-10}")

# --- Step 5 ---
print_step(5, "Orthogonal projection Proj_𝐮(𝐯) = (⟨𝐯|𝐮⟩ / ⟨𝐮|𝐮⟩) 𝐮")
def vector_projection(v, u):
    """Compute the orthogonal projection of 𝐯 onto the direction of 𝐮"""
    return (np.dot(v, u) / np.dot(u, u)) * u

proj = vector_projection(v, u)
compare_print("Proj_𝐮(𝐯)", proj, "the projected component; the residual should be orthogonal to 𝐮")
residual = v - proj
print(f"  check orthogonality ⟨𝐮 | 𝐯 − Proj_𝐮(𝐯)⟩ = {np.dot(u, residual):.1e}")

# ============================================================
print_header("Example 2.5.3 | The Cross Product and Its Geometric Applications")
# ============================================================

# --- Step 0 ---
print_step(0, "The three-dimensional cross product 𝐚 × 𝐛")
a3d = np.array([1, 2, 3], dtype=float)
b3d = np.array([4, -1, 2], dtype=float)
cross = np.cross(a3d, b3d)
compare_print("𝐚 × 𝐛", cross, "expansion as a determinant: see the definition cross-product")

# --- Step 1 ---
print_step(1, "Verify perpendicularity of the cross product: (𝐚×𝐛)·𝐚 = 0, (𝐚×𝐛)·𝐛 = 0")
print(f"  (𝐚×𝐛)·𝐚 = {np.dot(cross, a3d):.1e}  (should be 0)")
print(f"  (𝐚×𝐛)·𝐛 = {np.dot(cross, b3d):.1e}  (should be 0)")

# --- Step 2 ---
print_step(2, "Area of the parallelogram = ‖𝐚 × 𝐛‖ = ‖𝐚‖‖𝐛‖|sin θ|")
area = np.linalg.norm(cross)
compare_print("‖𝐚 × 𝐛‖", f"{area:.4f}", "area of the parallelogram")

# --- Step 3 ---
print_step(3, "Application: the distance from a point to a line")
def point_to_line_distance(point, line_point, line_dir):
    """Compute the distance from a point in space to a line (using the projection residual)"""
    v    = point - line_point
    proj = vector_projection(v, line_dir)
    return np.linalg.norm(v - proj)

p      = np.array([5, 3, 1], dtype=float)
line_p = np.array([1, 1, 1], dtype=float)
line_d = np.array([1, 1, 0], dtype=float)
dist = point_to_line_distance(p, line_p, line_d)
compare_print("distance from point to line", f"{dist:.4f}", "‖𝐯 − Proj_𝐝(𝐯)‖")

# --- Step 4 ---
print_step(4, "Application: the unit normal vector of a plane (three points determine a plane)")
def plane_normal(q0, q1, q2):
    """Given three points on a plane, compute the unit normal vector 𝐧̂"""
    v0 = q1 - q0
    v1 = q2 - q0
    return normalize(np.cross(v0, v1))

q0 = np.array([1, 0, 0], dtype=float)
q1 = np.array([0, 1, 0], dtype=float)
q2 = np.array([0, 0, 1], dtype=float)
n_hat = plane_normal(q0, q1, q2)
compare_print("𝐧̂ (unit normal vector)", n_hat, "‖𝐧̂‖ should be 1.0")
print(f"  check ‖𝐧̂‖ = {np.linalg.norm(n_hat):.4f}")

# ============================================================
print_header("Example 2.5.4 | Gram-Schmidt Orthogonalization (a first look)")
# ============================================================

# --- Step 0 ---
print_step(0, "Given two linearly independent vectors 𝐯₀, 𝐯₁")
v0 = np.array([1.0, 1.0, 0.0])
v1 = np.array([1.0, 0.0, 1.0])
print(f"  𝐯₀ = {v0}")
print(f"  𝐯₁ = {v1}")

# --- Step 1 ---
print_step(1, "Gram-Schmidt: 𝐮₀ = v̂₀; 𝐮₁ = 𝐯₁ − Proj_𝐮₀(𝐯₁), then normalize")
def gram_schmidt(v0, v1):
    """Apply Gram-Schmidt orthogonalization to two vectors, returning the orthonormal pair (𝐮₀, 𝐮₁)"""
    u0 = normalize(v0)
    u1 = normalize(v1 - vector_projection(v1, u0))
    return u0, u1

u0, u1 = gram_schmidt(v0, v1)
print(f"  𝐮₀ = {u0}")
print(f"  𝐮₁ = {u1}")

# --- Step 2 ---
print_step(2, "Verify orthogonality and unit norm")
compare_print("⟨𝐮₀|𝐮₁⟩", f"{np.dot(u0, u1):.1e}", "should be 0 (orthogonal)")
compare_print("‖𝐮₀‖",    f"{np.linalg.norm(u0):.4f}", "should be 1.0 (unit norm)")
compare_print("‖𝐮₁‖",    f"{np.linalg.norm(u1):.4f}", "should be 1.0 (unit norm)")

§2.5 Summary

This section answered the question of how the mathematical notions of this chapter are turned into executable computation. NumPy takes np.array as its basic container and provides functions such as np.dot, np.cross, and np.linalg.norm that correspond directly to the definitions of the inner product, the cross product, and the norm, achieving very nearly a line-by-line translation between mathematical notation and code. It is worth noting that a one-dimensional NumPy array does not by default distinguish rows from columns, so shapes require particular care once matrix operations are involved; we shall confront this squarely when we take up matrix multiplication in Chapter 3.

The code also includes a short demonstration of Gram-Schmidt orthogonalization: it uses the projection formula to remove, step by step, the correlation between directions, producing an orthogonal set of vectors. The code is offered here only as a first look; the geometric mechanism behind it—that subtracting the projection removes the correlated component—and the full theory are left to the orthogonalization of Chapter 9 and the singular value decomposition (SVD) of Chapter 11. Through hands-on implementation, abstract geometric intuition and concrete numerical computation come together in this section, preparing the reader to enter the world of matrices.

2.6 Chapter Summary¶

Review of the Theoretical Thread¶

This chapter set out from a question that looks like an everyday one: “what is a vector?” The answer given in §2.1 is twofold—a vector is both a geometric arrow and an ordered list of numbers, and once coordinates are introduced the two readings are completely equivalent. This dual identity is not decoration; it is the basic tension running through the whole book: geometric intuition tells us “why,” and algebraic structure tells us “how to compute.”

§2.2 equipped the set of vectors with two operations, addition and scalar multiplication, and immediately displayed their geometric content: addition is the diagonal of a parallelogram, and scalar multiplication is a stretch along the original direction. More importantly, these two operations gave rise to the notion of a linear combination: given several vectors, taking a weighted sum with arbitrary coefficients lets us “reach” a space spanned jointly by those vectors. The unique-decomposition theorem for the standard basis is the summit of this section: $\mathbf{v} = \sum_{i=0}^{n-1} v_i \mathbf{e}_i$ is not merely a computational formula but the rigorous definition of the notion of a “coordinate.”

Addition and scalar multiplication, however, tell us only how to “move,” not how to “measure.” §2.3 introduced the inner product, hiding all of the information about length and angle inside the algebraic quantity $\langle\mathbf{u}|\mathbf{v}\rangle = \sum_i u_i v_i$ . The Cauchy-Schwarz inequality $|\langle\mathbf{u}|\mathbf{v}\rangle| \le \|\mathbf{u}\|\|\mathbf{v}\|$ is the core of that section: it guarantees the legitimacy of the definition of angle in any dimension and makes orthogonal projection a precise geometric operation rather than merely an intuitive impression. §2.4 turned to the cross product, peculiar to three-dimensional space, adding through $\mathbf{u} \times \mathbf{v}$ a mechanism for generating normal vectors; computing the shortest distance between skew lines displayed the surprising economy of the cross product in geometric problems. §2.5 then put all of the above into practice with Python and NumPy, including a first look at a code fragment for Gram-Schmidt orthogonalization as a preview of the theory of orthogonality to come.

Connections to Other Chapters¶

There is a clear bridge between this chapter and the language of sets from Chapter 1: set theory supplied precise language for “a set of elements” and for “functions,” and this chapter added operations on top of that language—vector addition and scalar multiplication—promoting the set $\mathbb{R}^n$ to a “vector space” carrying algebraic structure. That promotion will be formalized completely in the abstract linear spaces of Chapter 4: the eight computational rules observed in §2.2 (commutativity, associativity, distributivity, and the rest) are precisely the prototype of the vector space axioms, and readers will discover there that function spaces and polynomial spaces satisfy the very same axioms, so that the notion of a vector is enormously generalized.

Looking ahead, this chapter supplies almost all of the conceptual material for the matrices and linear transformations of Chapter 3: the change of basis presupposes the unique representation in the standard basis, the geometric meaning of matrix multiplication requires vector addition and scalar multiplication working together, and the introduction of quaternions in the historical opening of Chapter 3 will echo the three-dimensional restriction on the cross product. Further ahead, the real inner product of §2.3 is the finite-dimensional prototype of the inner product spaces of Chapter 9, where the complex inner product and the polarization identity are generalized; and the Gram-Schmidt code fragment first met in §2.5 foreshadows the orthogonalization techniques required by the singular value decomposition of Chapter 11.

The Role of This Chapter in the Book¶

This chapter solved a basic and unavoidable problem: before we may speak of matrices, transformations, and decompositions, we must first be clear about what a vector is, what may be done to it, and how it is to be measured. Addition and scalar multiplication establish the linear structure, the inner product establishes the metric structure, and together they constitute the complete framework of Euclidean space. Carrying that framework, a reader who meets the matrices of Chapter 3 is not merely manipulating blocks of numbers but tracking how the coordinates of a vector transform under different bases; and a reader who meets the symmetric matrices of Chapter 9 will see at once that the orthogonality of eigenvectors originates in the geometric meaning of the inner product.

The ubiquity of the vector toolkit in nature, engineering, and information science also gives this chapter a significance beyond the classroom. Force and displacement in physics, spectral components in signal processing, embedding vectors in machine learning—these fields each developed a need for “vectors” independently, in their own contexts, and yet all converged on one and the same mathematical language. Historically, the birth of that language was not smooth: the twenty-year “quaternion war” between Hamilton’s quaternion school and Gibbs’s vector school produced as its spoils exactly the addition, inner product, and cross product that a linear algebra course now takes for granted. A reader who carries this history forward may come to hold each definition in a little more regard, and to feel a little more impulse to ask why each definition is as it is.