Chapter 4 Abstract Linear Spaces

In 1926 quantum physics ran into a famous split in its formulation: Heisenberg’s “matrix mechanics” dealt with discrete arrays of numbers and transitions, while Schrödinger’s “wave mechanics” solved continuous partial differential equations. The two formulations looked worlds apart, yet physicists discovered to their astonishment that they gave exactly the same predictions. Before long the truth came into view: the language of vector spaces and linear operators supplied a common skeleton linking the two formulations—the discrete infinite-dimensional vectors and the continuous wave functions were, at bottom, only projections of one and the same abstract space onto different “bases.”

This story reveals the real driving force behind abstraction in modern mathematics: abstraction has never been a pursuit of formal vanity or elegance; it arises because objects that look completely different turn out to obey one and the same set of rules of operation—and that coincidence calls out for an explanation of what lies beneath it.

In fact, the germ of that explanation appeared nearly ninety years before quantum mechanics.

In 1844 the Prussian schoolteacher Hermann Grassmann published Die lineare Ausdehnungslehre. The central idea of the book was far ahead of its time: he held that the algebraic structure of “things that can be added and scaled” deserved study as a discipline in its own right, whatever objects it happened to carry—geometric line segments, polynomials, or mechanical quantities. He did not care what a vector “is in itself,” only what a vector “can do under the operations”—and this is precisely the essence of modern axiomatic thinking. The price of being ahead of one’s time, however, was enormous: Gauss privately acknowledged the insight but admitted that the book was “too laborious to read,” and the academic mainstream and university review committees shut it out because of its highly philosophical style. Grassmann waited nearly twenty years and in 1862 published, at his own expense, a heavily rewritten second edition, which still found almost no readers. Disheartened, he eventually turned his energies to the study of Sanskrit and achieved world-class results in comparative linguistics; he died in 1877 with his regret unresolved, never having seen his mathematical vision recognized by the academic world in his lifetime.

It was not until 1888 that the Italian mathematician Giuseppe Peano, deeply inspired by Grassmann, gave in his Calcolo geometrico the first rigorous axiom system for abstract vector spaces in human history—the very eight rules we know today: any set that is closed under addition and scalar multiplication and satisfies these eight rules of computation is, as an algebraic structure, a “vector space,” no matter whether what it holds is arrays of real numbers, geometric arrows, polynomials, matrices, or functions. Even so, Peano’s definition still seemed too far ahead of its time, and his contemporaries were widely puzzled: “We have concrete matrices and differential equations to compute with—why go to the trouble of this airy, insubstantial set of axioms?” The axiom system slept on the bookshelf for more than thirty years more, until the mathematical crisis set off by the explosion of quantum mechanics jolted the whole scientific community awake: mathematicians had long since built for them an ultimate framework capable of holding all of this at once.

From Grassmann to von Neumann, the tool was born nearly a century before the physical world urgently needed it.

The scope of this book is finite-dimensional linear spaces; we will not go deeply into the infinite-dimensional topological analysis of Hilbert spaces and Banach spaces—between finite and infinite dimension there are many fundamental differences of an analytic kind, and that is the home ground of functional analysis. But in the finite-dimensional world Peano’s eight axioms are just as complete, pure, and powerful. In the first three chapters we grew used to computing with vectors, matrices, and geometric transformations in $\mathbb{R}^n$, and we may have sensed dimly that polynomials, functions, and even spaces of matrices all follow “exactly the same rules of the game.”

The task of this chapter is to turn that vague intuition into rigorous mathematical language: What is a vector space? What is a linear mapping? What is dimension? Once these concepts are firmly established at the level of axioms, the theory acquires a universal penetrating power—it applies automatically to signal processing, quantum computing, dynamical systems in economics, and even the embedding spaces of learned representations in contemporary machine learning, without our having to reinvent the wheel in every new field.

Chapter Structure and Learning Objectives¶

Peano’s eight axioms are the starting point of this chapter, and the threshold of the book’s turn toward abstraction. §4.1 defines vector spaces at the level of axioms and pins down the precise meaning of subspace, span, and linear dependence. The concept of dimension receives its most general definition here—no longer “the $n$ in $\mathbb{R}^n$,” but “the size of a maximal linearly independent set,” a definition that applies equally well to polynomial spaces and matrix spaces.

§4.2 extends the linear transformations of Chapter 3 from $\mathbb{R}^n$ to mappings between arbitrary vector spaces. The kernel and the image of a linear mapping reveal how a transformation “compresses” space; the rank-nullity theorem is the central result of this section, and it is also the theoretical foundation for the structure of the solutions of systems of linear equations in Chapter 6.

§4.3 surveys systematically the typical examples of finite-dimensional vector spaces: the polynomial spaces $\mathcal{P}_n$, the matrix spaces $\mathcal{M}_{m \times n}$, finite-dimensional subspaces of function spaces, and the construction of direct sums. Together these examples show that vector spaces are far more varied than $\mathbb{R}^n$, and that the axiomatic framework handles all of these cases in a unified way.

Once you have read this chapter, what the word “vector” means to you will have changed completely—it is no longer “a list of numbers,” but any mathematical object that satisfies the eight rules. To the learner’s question “why must it be so abstract?” you will have an answer of your own.

4.1 The Abstract Theory of Vector Spaces¶

In linear algebra the vector space is a basic and important concept: it provides a unified framework for studying all kinds of linear structures.

4.1.1 The Abstract Definition of a Vector Space¶

4.1.2 The Dimension of a Vector Space¶

The Steinitz exchange lemma is the key tool for proving that the dimension of a finite-dimensional vector space is unique.

Using the Steinitz exchange lemma, we can prove that the dimension of a finite-dimensional vector space is unique.

4.2 Linear Mappings¶

A mapping goes from a set to a set; a linear mapping goes from a vector space to a vector space, and it preserves the operations of vector addition and scalar multiplication.

4.2.1 The Kernel and Image of a Linear Mapping¶

For mappings in general we consider only properties such as injectivity and surjectivity; for linear mappings we can look more closely, and the kernel and the image are two important concepts in their study.

4.2.2 Injections, Surjections, and Isomorphisms¶

Linear mappings can be classified according to the properties of their kernels and images.

4.2.3 Change of Basis in a Linear Space¶

In a vector space, a vector has only one representation with respect to a given basis, and matrices let us express its coordinates with respect to different bases.

When we have two different bases, the coordinates of the same vector with respect to the two bases are different, but they are related linearly.

4.2.4 The Matrix Representation of a Change of Coordinates¶

4.2.5 Linear Mappings between Different Finite-Dimensional Linear Spaces¶

We now go on to discuss linear mappings between different linear spaces.

This result generalizes the two-way correspondence between linear mappings and matrices that we discussed in [Section 3.2.2].

4.2.6 The Rank-Nullity Theorem¶

Recalling the classification of linear mappings discussed in Section 4.2.2, the rank-nullity theorem makes it far easier to determine how a linear mapping is classified:

4.3 Classic Finite-Dimensional Vector Spaces¶

This section introduces some classic finite-dimensional vector spaces in plain, accessible language. We apply a unified analytical framework to each vector space and discuss in detail its basic information, its typical applications, and its signature linear mappings.

The Unified Analytical Framework¶

4.3.1 Concrete Vector Spaces¶

Concrete vector spaces are the cornerstone of the theory of linear algebra and the starting point for learning it. They consist of concrete numerical coordinates and provide the most intuitive realization of the concept of a vector. Spaces of this kind are not only concrete models of the theory of abstract vector spaces but also the carriers of almost all practical computation and applications.

4.3.1.1 Euclidean Space $\mathbb{R}^n$¶

Basic Algebraic Structure¶

• Definition and notation: $\mathbb{R}^n = \{(x_0, x_1, \ldots, x_{n-1}) \mid x_i \in \mathbb{R}\}$

• Typical basis: the standard basis $\{\mathbf{e}_0, \mathbf{e}_1, \ldots, \mathbf{e}_{n-1}\}$

  • $\mathbf{e}_i = (0, \ldots, 0, \underbrace{1}_{\text{position } i}, 0, \ldots, 0)$

• Dimension: $n$ (each vector consists of $n$ independent real components)

• Linear operations: componentwise addition and scalar multiplication

• Important subspaces: coordinate hyperplanes, and lines and planes through the origin

Typical Applications¶

• Practical applications: 3D modeling in computer graphics, describing positions in robotics, state spaces in physical simulation

• Value in applications: provides the most intuitive framework for geometric computation and underlies all engineering computation

Geometric Intuition¶

• Geometric interpretation: $n$-dimensional Euclidean geometric space, the vector space most familiar to us

• Connections with familiar spaces: $\mathbb{R}^2$ (the plane) and $\mathbb{R}^3$ (three-dimensional space) provide intuitive geometric pictures

Core Linear Mappings¶

• Rotations: rigid transformations that preserve distances and angles $\mathbf{R}(\theta) = \begin{bmatrix}\cos\theta & -\sin\theta \\ \sin\theta & \cos\theta\end{bmatrix} \quad (\mathbb{R}^2)$

• Reflections: mirror transformations across a hyperplane

• Scalings: proportional scaling along each coordinate direction

• Geometric meaning of the mappings: they preserve the basic geometric properties of space and realize a variety of geometric transformations

Exercises¶

1. In $\mathbb{R}^3$, find the projection of the vector $(1,2,3)$ onto the $z$-axis

2. Prove that the determinant of a two-dimensional rotation matrix is 1

4.3.1.2 The Complex Vector Space $\mathbb{C}^n$¶

Basic Algebraic Structure¶

• Definition and notation: $\mathbb{C}^n = \{(z_0, z_1, \ldots, z_{n-1}) \mid z_i \in \mathbb{C}\}$

• Typical bases:

  • complex basis: $\{\mathbf{e}_0, \mathbf{e}_1, \ldots, \mathbf{e}_{n-1}\}$

  • real basis: $\{\mathbf{e}_0, i\mathbf{e}_0, \mathbf{e}_1, i\mathbf{e}_1, \ldots, \mathbf{e}_{n-1}, i\mathbf{e}_{n-1}\}$

• Dimension: $n$ as a complex vector space; $2n$ as a real vector space

Typical Applications¶

• Practical applications: describing states in quantum mechanics, frequency-domain analysis in signal processing, AC circuits in circuit analysis

• Value in applications: provides a natural mathematical framework for handling periodic and wave phenomena

Geometric Intuition¶

• Geometric interpretation: complex multiplication performs a rotation and a scaling at the same time, giving a richer geometric structure than real spaces have

• Connections with familiar spaces: each complex number carries two pieces of real information (its real part and its imaginary part)

Core Linear Mappings (the Field Must Be Specified)¶

• Complex conjugation (real-linear): $\mathbf z\mapsto\overline{\mathbf z}$; in general it is not complex-linear.

• Extracting the real/imaginary part (real-linear): $\operatorname{Re},\operatorname{Im}:\mathbb C^n\to\mathbb R^n$, regarding the domain as a real vector space.

• Unitary transformations (complex-linear): $\mathbf z\mapsto\mathbf U\mathbf z$, where $\mathbf U^*\mathbf U=\mathbf I$. Unitary transformations preserve the complex inner product and can be used to describe unitary evolution in quantum mechanics.

Exercises¶

1. Prove that the complex vector $(1+i, 2-i)$ has norm $\sqrt{7}$

2. Find a $2 \times 2$ unitary matrix that maps $(1,0)$ to $(i,0)$

4.3.1.3 ◆The Vector Space $\mathbb{F}_2^n$ over the Binary Field¶

Basic Algebraic Structure¶

• Definition and notation: $\mathbb{F}_2^n = \{(x_0, x_1, \ldots, x_{n-1}) \mid x_i \in \{0,1\}\}$

• Rules of operation:

  • Addition: $0 + 0 = 0$, $0 + 1 = 1$, $1 + 0 = 1$, $1 + 1 = 0$ (the XOR operation)

  • Multiplication: $0 \cdot 1 = 1 \cdot 0 = 0$, $1 \cdot 1 = 1$ (the AND operation)

• Typical basis: the standard basis $\{\mathbf{e}_0, \mathbf{e}_1, \ldots, \mathbf{e}_{n-1}\}$

• Dimension: $n$

• An interesting property: every vector is its own inverse: $\mathbf{v} + \mathbf{v} = \mathbf{0}$

Typical Applications¶

• Practical applications: the design of error-correcting codes, cryptography (for example AES encryption), the representation of computer memory

• Value in applications: provides the mathematical foundation for digital signal processing and error detection

Core Linear Mappings¶

• Cyclic shift: $(x_0,x_1,\ldots,x_{n-1})\mapsto(x_{n-1},x_0,\ldots,x_{n-2})$.

Other Important Operations¶

• Bit flip (affine): $\mathbf x\mapsto\mathbf x+\mathbf1$; when $n\ge1$ it does not fix the zero vector, so it is not a linear mapping.

Exercises¶

1. In $\mathbb{F}_2^3$, compute $(1,0,1) + (1,1,0)$

2. Prove that $\mathbb{F}_2^3$ has exactly 8 elements

4.3.1.4 ◆The Space of Quaternions $\mathbb{H}$¶

Basic Algebraic Structure¶

• Definition and notation: $\mathbb{H} = \{a + bi + cj + dk \mid a,b,c,d \in \mathbb{R}\}$

• Typical basis: $\{1, i, j, k\}$

• Dimension: 4 (as a real vector space)

• Important property: multiplication is noncommutative; in general $pq \neq qp$

Typical Applications¶

• Practical applications: representing three-dimensional rotations, computer graphics, attitude control in robotics

• Value in applications: avoids the problem of gimbal lock and provides smooth interpolation of rotations

Core Real-Linear Mappings¶

• Conjugation: $q\mapsto\bar q=a-bi-cj-dk$.

• Conjugation action of a fixed unit quaternion: $v\mapsto qv\bar q$ gives three-dimensional rotations on the space of pure imaginary quaternions.

Other Important Operations¶

• Squared norm (nonlinear): $|q|^2=q\bar q=a^2+b^2+c^2+d^2$.

Exercises¶

1. Compute the quaternion $(1 + i)(j + k)$

2. Verify that the quaternion $q = \frac{1}{\sqrt{2}}(1 + k)$ is a unit quaternion

4.3.2 Matrix Spaces and Their Subspaces¶

Matrix spaces are among the most important concrete vector spaces in linear algebra: they not only provide concrete representations of linear mappings but also play a central role in a wide range of mathematical and engineering applications.

4.3.2.1 The Matrix Space $M_{m \times n}(\mathbb{K})$¶

Basic Algebraic Structure¶

• Definition and notation: $M_{m \times n}(\mathbb{K})$, where $\mathbb{K}$ is usually $\mathbb{R}$ or $\mathbb{C}$

• Typical basis: the matrix units $\{\mathbf{E}_{ij} \mid 0 \leq i < m, 0 \leq j < n\}$

  • $\mathbf{E}_{ij}$ has a 1 in position $(i,j)$ and 0 in every other position

• Dimension: $m \times n$

• Linear operations: entrywise addition and scalar multiplication

• Important subspaces: symmetric matrices, skew-symmetric matrices, upper triangular matrices, lower triangular matrices

Typical Applications¶

• Practical applications: representing linear systems, data processing, image transformations, feature matrices in machine learning

• Value in applications: provides a concrete computational framework for linear transformations

Geometric Intuition¶

• Geometric interpretation: can be viewed as the “blueprint” of a linear transformation, or as a two-dimensional array of data

• Connections with familiar spaces: vectorization establishes an isomorphism with $\mathbb{K}^{mn}$

Core Linear Mappings¶

• Matrix transposition: $T(\mathbf{A}) = \mathbf{A}^{\top}$

• The trace: $\text{tr}(\mathbf{A}) = \sum_{i=0}^{\min(m,n)-1} a_{ii}$ (for square matrices)

• Left/right multiplication: $T_{\mathbf{B}}(\mathbf{A}) = \mathbf{BA}$ or $T_{\mathbf{C}}(\mathbf{A}) = \mathbf{AC}$

• Geometric meaning of the mappings: transposition corresponds to the dual of a linear transformation; the trace extracts information from the main diagonal

Exercises¶

1. Find the dimension of $M_{2 \times 3}(\mathbb{R})$ and a basis for it

2. Prove that matrix transposition is a linear mapping

4.3.2.2 The Space of Symmetric Matrices $Sym_n(\mathbb{R})$¶

Basic Algebraic Structure¶

• Definition and notation: $Sym_n(\mathbb{R}) = \{\mathbf{A} \in M_n(\mathbb{R}) \mid \mathbf{A} = \mathbf{A}^{\top}\}$

• Typical basis:

  • diagonal basis: $\{\mathbf{E}_{ii} \mid i = 0, 1, \ldots, n-1\}$

  • off-diagonal basis: $\{\mathbf{E}_{ij} + \mathbf{E}_{ji} \mid 0 \leq i < j < n\}$

• Dimension: $\frac{n(n+1)}{2}$

Typical Applications¶

• Practical applications: the theory of quadratic forms, principal component analysis, covariance matrices, optimization theory

• Value in applications: describes the energy and stability of systems, and is a central tool of statistical analysis

Geometric Intuition¶

• Geometric interpretation: a real symmetric matrix $\mathbf A$ defines the symmetric bilinear form $b(\mathbf x,\mathbf y)=\mathbf x^\top\mathbf A\mathbf y$; only when $\mathbf A$ is positive definite is this bilinear form an inner product, which then defines the length $\sqrt{\mathbf x^\top\mathbf A\mathbf x}$.

• Connections with familiar spaces: each symmetric matrix corresponds to a quadratic form

Core Linear Mappings¶

• Symmetrization operator: $S(\mathbf A)=\frac12(\mathbf A+\mathbf A^\top)$, which projects real square matrices onto the subspace of symmetric matrices.

Other Important Operations¶

• Taking eigenvalues (nonlinear): maps a symmetric matrix to the vector of its eigenvalues arranged in a fixed order; in general it does not preserve addition.

Exercises¶

1. Find the dimension of $Sym_3(\mathbb{R})$ and a basis for it

2. Verify that the symmetrization operator is indeed a linear mapping

4.3.2.3 The Space of Hermitian Matrices $Herm_n(\mathbb{C})$¶

Basic Algebraic Structure¶

• Definition and notation: $Herm_n(\mathbb{C}) = \{\mathbf{A} \in M_n(\mathbb{C}) \mid \mathbf{A} = \mathbf{A}^*\}$

• Typical basis: diagonal basis, real-part basis, imaginary-part basis

• Dimension: $n^2$ (as a real vector space)

• Important property: the diagonal entries must be real

Typical Applications¶

• Practical applications: observables in quantum mechanics, complex quadratic forms, covariance analysis in signal processing

• Value in applications: describes the physical properties of quantum systems, and is the mathematical foundation of quantum computing

Core Real-Linear Mappings¶

• Hermitian-part operator: $H(\mathbf A)=\frac12(\mathbf A+\mathbf A^*)$, a real-linear projection from $M_n(\mathbb C)$ (regarded as a real vector space) onto $Herm_n(\mathbb C)$.

• Extracting the real/imaginary part: $\operatorname{Re}(\mathbf A)$ and $\operatorname{Im}(\mathbf A)$ are real-linear mappings into $M_n(\mathbb R)$; in general they are not complex-linear mappings.

Exercises¶

1. Prove that the diagonal entries of a Hermitian matrix are all real

2. Find a basis for the $2 \times 2$ Hermitian matrices

4.3.2.4 The Space of Skew-Symmetric Matrices $Skew_n(\mathbb{R})$¶

Basic Algebraic Structure¶

• Definition and notation: $Skew_n(\mathbb{R}) = \{\mathbf{A} \in M_n(\mathbb{R}) \mid \mathbf{A} = -\mathbf{A}^{\top}\}$

• Typical basis: $\{\mathbf{E}_{ij} - \mathbf{E}_{ji} \mid 0 \leq i < j < n\}$

• Dimension: $\frac{n(n-1)}{2}$

• Important property: the diagonal entries must be zero

Typical Applications¶

• Practical applications: the angular velocity tensor, the electromagnetic field tensor, the theory of Lie groups, robotics

• Value in applications: describes the rotation of systems and antisymmetric fields, and is an important tool of differential geometry

Geometric Intuition¶

• Geometric interpretation: corresponds to skew-adjoint transformations, and in three-dimensional space is closely related to the cross product of vectors

• Connections with familiar spaces: $Skew_3(\mathbb{R})$ is isomorphic to $\mathbb{R}^3$

Core Linear Mappings¶

• Skew-symmetrization operator: $K(\mathbf A)=\frac12(\mathbf A-\mathbf A^\top)$.

• Commutator with a fixed $\mathbf B$: $\mathbf A\mapsto[\mathbf A,\mathbf B]=\mathbf A\mathbf B-\mathbf B\mathbf A$ is a linear mapping.

Other Important Operations¶

• Lie bracket (bilinear): $(\mathbf A,\mathbf B)\mapsto[\mathbf A,\mathbf B]$ is linear in each argument separately; it is not a jointly linear mapping on ordered pairs.

• Matrix exponential (nonlinear): $\exp(\mathbf A)$ maps skew-symmetric matrices to orthogonal matrices.

Exercises¶

1. Find the dimension of $Skew_3(\mathbb{R})$

2. Verify the correspondence between $Skew_3(\mathbb{R})$ and the cross product of vectors

4.3.3 Function Spaces¶

Function spaces extend the concept of a vector from finite-dimensional tuples of numbers to infinite-dimensional objects that are functions, a natural extension of the theory of linear algebra toward analysis. Although the full theory of function spaces involves infinite-dimensional analysis, many important function spaces can be approximated and understood through finite-dimensional subspaces.

4.3.3.1 The Polynomial Space $P_n(\mathbb{R})$¶

Basic Algebraic Structure¶

• Definition and notation: $P_n(\mathbb{R}) = \{a_0 + a_1x + \cdots + a_nx^n \mid a_i \in \mathbb{R}\}$

• Typical basis: the monomial basis $\{1, x, x^2, \ldots, x^n\}$

• Dimension: $n+1$

• Important subspaces: the even polynomials $\{p\in P_n(\mathbb R):p(-x)=p(x)\}$ and the odd polynomials $\{p\in P_n(\mathbb R):p(-x)=-p(x)\}$; the zero polynomial belongs to both.

• A nonexample: the set of monic polynomials is not a subspace. It does not contain the zero polynomial; for example, 1 is a monic polynomial but $1+1=2$ is not, so the set is not closed under addition.

Typical Applications¶

• Practical applications: numerical analysis, signal processing, approximation theory, data fitting

• Value in applications: provides the basic tools of function approximation, and is at the core of scientific computing

Geometric Intuition¶

• Geometric interpretation: a finite-dimensional approximation of a function space

• Connections with familiar spaces: the coefficient vectors establish an isomorphism with $\mathbb{R}^{n+1}$

Core Linear Mappings¶

• Differentiation operator: $D(p(x)) = p'(x)$

• Translation operator: $T_a(p(x)) = p(x+a)$

• Evaluation operator: $E_c(p(x)) = p(c)$

• Geometric meaning of the mappings: the differentiation operator lowers the degree of a polynomial; the evaluation operator maps a function to a number

Exercises¶

1. Find the matrix representation of the differentiation operator with respect to the basis $\{1, x, x^2\}$

2. Prove that the translation operator is linear

4.3.3.2 Spaces of Fourier Series (Finite-Dimensional Approximations)¶

Basic Algebraic Structure¶

• Typical basis: $\{1, \sin x, \cos x, \sin 2x, \cos 2x, \ldots, \sin nx, \cos nx\}$

• Dimension: $2n+1$

• Important property: the basis functions are orthogonal on the interval $[0, 2\pi]$

Typical Applications¶

• Practical applications: signal processing, image compression, spectral analysis

• Value in applications: converts time-domain signals into frequency-domain representations, and is the basis of digital signal processing

Core Linear Mappings¶

• Differentiation operator: maps $\sin kx$ to $k\cos kx$

• Fourier transform: the conversion between a function and its frequency-domain representation

• Geometric meaning of the mappings: reveals the frequency content of a signal

Exercises¶

1. Verify that $\sin x$ and $\cos x$ are orthogonal on $[0, 2\pi]$

2. Find the three-term Fourier approximation of the function $f(x) = x$ on $[-\pi, \pi]$

4.3.4 Composite Spaces¶

Composite spaces embody the constructive and unifying character of the theory of vector spaces: they show how, starting from known vector spaces, new vector spaces can be constructed by operations such as the direct sum and the tensor product. These constructions not only enrich the variety of vector spaces; more importantly, they provide a way of attacking complex problems by systematic decomposition.

4.3.4.1 The Direct Sum $V \oplus W$¶

Basic Algebraic Structure¶

• Definition and notation: $V \oplus W = \{(v,w) \mid v \in V, w \in W\}$

• Typical basis: if $\{v_i\}$ is a basis of $V$ and $\{w_j\}$ is a basis of $W$, then $\{(v_i, 0), (0, w_j)\}$ is a basis of $V \oplus W$

• Dimension: $\dim(V \oplus W) = \dim(V) + \dim(W)$

• Linear operations: $(v_0, w_0) + (v_1, w_1) = (v_0+v_1, w_0+w_1)$

Typical Applications¶

• Practical applications: system decomposition, parallel computing, modular design

• Value in applications: decomposes a complex system into independent subsystems that can be handled separately

Geometric Intuition¶

• Geometric interpretation: places two spaces “side by side,” keeping each space independent

• Connections with familiar spaces: like the cars of a train coupled together, each part relatively independent

Core Linear Mappings¶

• Projection mappings: $\pi_V(v,w) = v$, $\pi_W(v,w) = w$

• Inclusion mappings: $i_V(v) = (v,0)$, $i_W(w) = (0,w)$

• Geometric meaning of the mappings: the projections extract the components; the inclusions place the subspaces into the composite space

Exercises¶

1. Find the dimension of $\mathbb{R}^2 \oplus \mathbb{R}^3$ and a basis for it

2. Prove that the projection mappings are linear

4.3.4.2 ◆The Tensor Product $V \otimes W$¶

Basic Algebraic Structure¶

Representation of elements.

• Elementary tensors: $v \otimes w$ ($v \in V, w \in W$)

• General tensors: $\sum_{i=0}^{m-1}\sum_{j=0}^{n-1}c_{ij}(v_i\otimes w_j)$, where the $c_{ij}\in\mathbb K$ can be chosen independently.

Dimension relation.

Comparing examples.

• the dimension of $\mathbb{R}^2 \oplus \mathbb{R}^3$ = $2 + 3 = 5$ (direct sum)

• the dimension of $\mathbb{R}^2 \otimes \mathbb{R}^3$ = $2 \times 3 = 6$ (tensor product)

Typical basis. If $\{v_0, v_1\}$ is a basis of $V$ and $\{w_0, w_1, w_2\}$ is a basis of $W$, then

is a basis of $V \otimes W$ (6 basis vectors).

Rules of operation.

• Scalar multiplication: $c(v \otimes w) = (cv) \otimes w = v \otimes (cw)$

• Addition: $(v_0 \otimes w_0) + (v_1 \otimes w_1)$ in general cannot be written as a single elementary tensor

Typical Applications¶

• Practical applications: composite systems in quantum computing, separable filtering in image processing, feature interactions in machine learning

• Value in applications: captures all the interactions between systems, and is an important tool of modern scientific computing

Geometric Intuition¶

The fundamental difference from the direct sum.

Intuition.

• Direct sum: “side by side,” keeping each space independent

• Tensor product: “full interaction,” capturing every possible interaction

Connections with familiar spaces.

1. Matrices are tensor products.

   $$M_{m \times n}(\mathbb{R}) \cong \mathbb{R}^m \otimes \mathbb{R}^n$$

   (44)

Each matrix position $(i,j)$ corresponds to the basis element $e_i \otimes f_j$:

2. Polynomial spaces are also tensor products. $P_m(x) \otimes P_n(y) = P_{m,n}(x,y)$ (polynomials in two variables)

Core Linear Mappings¶

Tensor products of mappings. If $T: V \rightarrow V'$ and $S: W \rightarrow W'$, then:

Matrix representation. The tensor product corresponds to the Kronecker product of matrices:

Geometric meaning of the mappings. A tensor product of mappings acts on both subsystems at once, producing a coordinated global change

Case Studies from Practice¶

1. Two-qubit systems in quantum computing

• Basic concept: single-qubit states $|0⟩, |1⟩ \in \mathbb{C}^2$

• Two-qubit system: $\mathbb{C}^2 \otimes \mathbb{C}^2 = \mathbb{C}^4$

• Four basis states: $|00⟩, |01⟩, |10⟩, |11⟩$

• Entanglement: $(|00⟩ + |11⟩)/\sqrt{2}$ cannot be decomposed into a combination of single-qubit states

• Why the tensor product is needed: quantum entanglement embodies nonlocal correlations between particles

2. The elastic deformation of a rubber band

• Physical phenomenon: the deformation of a rubber band when it is stretched

• Tensor-product description: stress state = internal direction of the material ⊗ direction of the external force

• Practical significance: a rubber band shows different stiffness when stretched in different directions

• Why the tensor product is needed: the molecular chains of the material have an intrinsic orientation, which interacts with the direction of the applied force

3. How polarized sunglasses work

• Physical phenomenon: sunglasses can remove the glare from water surfaces and snow

• Tensor-product description: photon state = direction of propagation ⊗ direction of polarization

• Practical effect: light reflected from water is mainly horizontally polarized, and a vertical polarizer can block this glare

• Everyday experience: rotating polarized sunglasses lets you watch the glare disappear and reappear

Exercises¶

1. Compute the matrix representation and the vector representation of $(2,3) \otimes (1,4,5)$

2. Prove that the tensor product operation is bilinear

3. An advanced application: design a simple entangled quantum state and explain why it is not separable

📋 Summary: Direct Sum vs. Tensor Product¶

4.3.5 A Unified Summary¶

Through this systematic survey of the classic finite-dimensional vector spaces, we can establish a unified analytical framework. The table below summarizes the core features of each class of vector space:

Key points.

1. Concrete vector spaces provide intuitive geometric pictures and a computational foundation

2. Matrix spaces provide concrete representations of linear transformations

3. Function spaces extend the ideas of linear algebra to analysis

4. Composite spaces display the constructive and unifying character of the theory of vector spaces

Through the unified analytical framework, we have seen the structural features these classic finite-dimensional vector spaces share as well as the distinctive properties of each. This systematic way of understanding helps us to:

1. Build conceptual connections: understand the analogies between different spaces

2. Master the method of analysis: use the unified framework to analyze new vector spaces

3. Choose suitable tools: choose the vector space best suited to the features of a problem

4. Deepen theoretical understanding: abstract general principles from concrete examples

Practical significance. These classic spaces are not only important components of mathematical theory but also the mathematical foundation of modern science and technology: from quantum computing to machine learning, from signal processing to engineering design, none of them can do without these vector spaces.

4.4 Chapter Summary¶

Review of the Theoretical Thread¶

Each of the three sections of this chapter answered one question, and together they form a complete logical chain.

§4.1 asked: what exactly is a “vector”? The answer given by Peano’s eight axioms is that any set that is closed under addition and scalar multiplication and obeys the eight laws of operation is called a vector space—whether what it holds is tuples of coordinates, polynomials, or matrices. The basis is the central tool of this framework: it is at once a maximal linearly independent set and a minimal spanning set, and the Steinitz exchange lemma guarantees that, however a basis is chosen, its size never changes. Dimension thus becomes an intrinsic invariant of a vector space, and the starting point for all the quantitative discussion in the chapter.

§4.2 asked: how does a linear mapping “change” a space? The kernel and the image measure, respectively, the dimensions a mapping compresses and the dimensions it retains; the rank-nullity theorem $\dim V = \dim\ker(T) + \dim\operatorname{Im}(T)$ links the two precisely, revealing that a linear mapping is in essence a dimension-conserving redistribution: the dimensions compressed away and the dimensions passed on always add up to the dimension of the input space. The change-of-basis matrix (transition matrix) and the matrix representation of a mapping translate abstract relations between spaces into the computable language of matrices. Deciding whether a mapping is injective, surjective, or an isomorphism is thereby greatly simplified—between spaces of equal dimension the three are equivalent, and it suffices to confirm one of them.

§4.3 asked: how many kinds of spaces satisfy these axioms? From Euclidean spaces and complex spaces to matrix spaces, polynomial spaces, and Fourier spaces, and on to the composite spaces constructed by direct sums and tensor products, these objects, so different on the surface, display an orderly structure within the same framework. The specific values of their dimensions—$\frac{n(n+1)}{2}$, $\dim V \times \dim W$—are not things to memorize but readings of algebraic structure. The fundamental difference between the direct sum and the tensor product (dimensions add vs. dimensions multiply) runs through quantum computing, signal processing, and machine learning, and is the clearest footprint the axiomatic framework has left on modern science.

Connections to Other Chapters¶

This chapter builds on Chapter 3 and leads into the chapters that follow, playing the role of the book’s turn toward abstraction. Chapter 3 established the two-way correspondence between matrices and linear mappings on $\mathbb{R}^n$, and all of its conclusions depended on concrete coordinates; this chapter frees that correspondence from $\mathbb{R}^n$ and extends it to arbitrary finite-dimensional vector spaces. The idea of Chapter 3 that “a matrix represents a linear mapping” receives a precise meaning within the theorems of this chapter: once bases have been chosen, each linear mapping between finite-dimensional spaces corresponds to exactly one matrix, and vice versa. This correspondence is itself linear, so there is a natural linear isomorphism between the space of linear mappings and the matrix space—the examples of matrix spaces in §4.3 are a direct illustration of this fact.

Looking ahead, the rank-nullity theorem of §4.2 is the theoretical cornerstone for the structure of the solutions of systems of linear equations in Chapter 6: $\mathbf{Ax} = \mathbf{b}$ has a solution if and only if $\mathbf{b} \in \operatorname{Im}(T_\mathbf{A})$, and the number of degrees of freedom in the solutions is exactly $\dim\ker(T_\mathbf{A})$. The block matrices of Chapter 5 correspond, at the algebraic level, to the direct-sum decompositions of §4.3—decomposing a complex space into relatively independent subspaces is precisely the geometric motivation for block operations. In the eigenvalue problem of Chapter 8, the concept of an invariant subspace deepens the theory of subspaces of this chapter; the inner product spaces of Chapter 9 add the structure of geometric distance on top of the axiomatic framework of §4.1, and Gram-Schmidt orthogonalization is a concrete application of the theory of bases in spaces equipped with a metric.

The Role of This Chapter in the Book¶

Grassmann published his Die lineare Ausdehnungslehre in 1844 and brought out a new edition in 1862; his work long went without wide recognition, although Hankel had already acknowledged its contribution in 1867. Peano proposed an axiomatic description of real linear spaces in 1888. This history reminds us that the value of an abstract structure may take time to become apparent; the common language established in this chapter allows matrices, polynomials, and functions to be studied within a single framework.

What this chapter has done is precisely to understand the structure of this common language. The axioms of a vector space, the kernel-image decomposition of a linear mapping, the rank-nullity theorem—once these concepts are established at the level of axioms, they apply automatically to basis-function expansions in signal processing, the state spaces of quantum physics, the linear approximations of economic models, and the embedding spaces of machine learning, with no need to prove them again for each field. This is the true power of abstraction: not to make mathematics harder, but to let many problems that would otherwise have to be solved separately be answered all at once.

What you take away from this chapter is not just a set of definitions and theorems but a way of looking at structure—faced with a new mathematical object, the first question is not “what do its elements look like?” but “which laws of operation does it satisfy, what is its dimension, and how do its linear mappings decompose the space?” This habit of questioning is the ticket of admission to all of the more advanced linear algebra that follows.

Concept Map¶