Chapter 3 Linear Mappings and Matrices

If you are seeing matrix multiplication for the first time, it is almost impossible to find it “natural.” When two matrices are multiplied, why must the rows of the first matrix be multiplied by the columns of the second and the products then summed term by term? Why not simply multiply the entries in corresponding positions, as we do for addition? The definition looks like an arbitrary rule laid down on a whim by some mathematician, and one suspects that some secret is being concealed behind it.

The suspicion is entirely reasonable—but the secret runs deeper than you expect: this seemingly odd definition of matrix multiplication took more than a hundred years, and the thought and clashing ideas of several first-rate mathematicians, before it finally settled into place. In form it is a convention of operation; in substance it is the most basic, and the most profound, algebraic cornerstone of linear algebra.

The story has to begin at the end of the eighteenth century. Leibniz, Gauss, and Binet, while solving systems of linear equations and studying determinants, all arranged coefficients in rectangular arrays of numbers, and in the course of their computations all of them implicitly carried out elimination steps resembling matrix multiplication. But nobody at the time stopped to ask: “What sort of thing is this array itself?” In their eyes the array was merely scaffolding to assist a computation; once the house was built and the equations solved, the scaffolding came down.

The breakthrough arrived unexpectedly in 1843. The Irish mathematician William Rowan Hamilton had been trying to extend the complex numbers to three-dimensional space, and had pondered the problem for ten years without success. On October 16 of that year, while walking across Brougham Bridge in Dublin, he saw in a flash that rotations of space required four dimensions, and that multiplication would have to give up commutativity altogether. On the spot he carved the fundamental formula $i^2 = j^2 = k^2 = ijk = -1$ into the stone of the bridge. This was the first time in the history of mathematics that a scholar seriously declared: the failure of multiplication to commute is not a flaw but an intrinsic feature of rotations in real space.

At almost the same time, in 1844, the twenty-one-year-old German prodigy Gotthold Eisenstein, studying linear transformations, had already begun to use a single letter to stand for an entire system of linear substitutions; he wrote down rules resembling matrix multiplication and pointed out explicitly that this operation is not commutative. Gauss held him in the highest esteem, even ranking him alongside Niels Henrik Abel—sadly, both were geniuses who lit up the mathematical sky and died young: Abel of tuberculosis at twenty-six, and Eisenstein of illness in Berlin at twenty-nine (1852). His profound ideas lay scattered here and there, never integrated into a systematic algebra.

It was Arthur Cayley who took the decisive step. Inspired by the symbolic treatment of substitutions in the work of Eisenstein and others, he published his epoch-making A Memoir on the Theory of Matrices in 1858. Cayley did something none of his predecessors had done: he stopped treating the array of numbers as a tool to assist computation and treated it instead as an algebraic entity with operations of its own. He declared explicitly: “I wish to represent a transformation by a single symbol.” The seemingly awkward “row times column” definition of matrix multiplication was by no means set arbitrarily—it is uniquely determined by the structure of composing substitutions, namely the requirement that the combined effect of first applying the linear mapping $\mathbf{A}$ and then applying the linear mapping $\mathbf{B}$ must equal exactly the matrix product $\mathbf{BA}$ .

Sixty-seven years later, in 1925, the young Heisenberg, working on quantum mechanics on the island of Helgoland and guided by physical intuition in computing the intensities of radiation in atomic spectra, ran independently into the same wall. He found that to fit the spectral regularities he had to adopt a strange algebra in which the order of multiplication changes the result ( $XY \neq YX$ ). In a letter to Wolfgang Pauli he wrote uneasily that this strange kind of multiplication filled him with dread. Heisenberg did not even know what this algebra was—until his mentor Max Born saw the paper and recognized, to his surprise, that it was exactly the matrix algebra Cayley had built up completely in pure mathematics seventy years before! Only then did physicists, who had long ignored progress in pure algebra, turn back in a scramble and open the yellowed pages of the mathematicians.

More intriguing still is this: even at that point, the geometric face of the matrix as a “linear transformation of space” remained absent from classical physics for a long time—and the absence runs deeper than one would imagine. Newton’s Mathematical Principles of Natural Philosophy of 1687 is from beginning to end a combination of Euclidean geometric proportion and a newborn calculus; Lagrange made mechanics thoroughly algebraic in his Analytical Mechanics of 1788, and one may search his manuscript from end to end without finding a single matrix enclosed in square brackets; Euler, deriving the equations for the rotation of a rigid body, treated the nine components of the inertia tensor as independent coefficients and struggled through intricate trigonometric substitutions rather than using the concise multiplication of matrices. The whole magnificent edifice of classical mechanics—the Newtonian, Lagrangian, and Hamiltonian systems—was finished and roofed over before the notion of a matrix appeared.

The irony is that when students in engineering or physics today solve problems in classical mechanics and vibration, matrices are simply everywhere. That is because only after quantum mechanics broke the deadlock in 1925 did physicists turn back and rewrite the long-finished classical theory in the exceptionally tidy modern language of matrices. First the avant-garde microscopic world of quanta put matrices to work; then matrices returned the favor by beautifying and unifying macroscopic classical mechanics.

The path of historical development was a winding one: from the early arithmetic of elimination in systems of equations, to the purely symbolic matrix algebra of the mid-nineteenth century, to the adoption under duress by quantum mechanics in 1925, which in turn set off a thorough recasting of classical mechanics in matrix form; and only in the 1930s, through the axiomatic foundations laid by von Neumann and others in operator theory and infinite-dimensional Hilbert space, did the “linear mapping” that matrix multiplication represents finally attain its full elevation into modern geometry.

A convention of operation that looks unremarkable took a hundred and seventy years to travel from Leibniz’s tables of coefficients to Cayley’s matrix algebra, and only with the axiomatized spaces of modern mathematics did it finally show its geometric hand. If matrix multiplication feels awkward the first time you meet it, that is not because you lack mathematical intuition; it is because you are touching, for the first time, a mathematical reality that puzzled the finest minds in history for a long time.

Chapter Structure and Learning Objectives¶

The story told in the introduction points in the end to a single question: why is matrix multiplication defined in the unnatural way of “multiplying rows by columns and summing”? The five sections of this chapter answer that question together, and build up the whole geometric vocabulary of matrices along the way.

§3.1 begins with rotation in two dimensions. The angle addition formulas supply clues to the matrix and to the rule of multiplication; it is by extending the requirement to all compatible linear mappings, so that $(\mathbf{AB})\mathbf{x}=\mathbf{A}(\mathbf{B}\mathbf{x})$ holds for every input, that standard matrix multiplication is uniquely determined. Rotation is a concrete starting point for understanding this general requirement.

§3.2 generalizes the matrix from a special case about rotations into a general algebraic object, and builds the complete system of operations: addition, multiplication, transposition, and the inverse. The four interpretations of matrix multiplication (the inner product, combinations of columns, combinations of rows, and a sum of rank-one matrices) are equivalent to one another, and you will find yourself using different ones on different occasions. The central result of this section is the representation theorem for linear mappings: the column vectors of a matrix are exactly where the standard basis vectors land under the mapping. This reduces understanding “the effect of a mapping on an entire space” to reading off a few column vectors, and it is the insight in this chapter most worth chewing over again and again.

§3.3 uses this tool to take systematic stock of the linear mappings commonly met in two and three dimensions: the zero mapping, scaling, projection, reflection, and rotation, each with its own definite matrix form and its own determinant signature. Rodrigues’ rotation formula in three dimensions and the gimbal lock of the Euler angles are the high point of the section, and the noncommutativity of composed rotations is exhibited here in concrete form.

§3.4 introduces a quantitative tool alongside these geometric descriptions: the determinant measures how much a mapping scales volume, $|\det \mathbf{A}|$ is the volume scale factor, and the sign records whether orientation has been reversed. A determinant of zero means that space has been compressed into a lower dimension and that the mapping has lost its invertibility.

§3.5 takes the determinant criterion as a key and interprets the solution of the linear system $\mathbf{Ax} = \mathbf{b}$ as “finding a preimage of the mapping.” When $\det(\mathbf{A}) \neq 0$ , the inverse mapping exists and the solution is given by the formula for the inverse matrix; when the determinant is zero, the system may have no solution at all or infinitely many, and that suspense is left for Chapter 6 to treat systematically.

By the end of this chapter, matrix multiplication will no longer look to you like an algorithm to be memorized, but like a faithful algebraic translation of the geometric operation of composition; and every column vector of a matrix will be a new direction into which space is sent by that mapping.

3.1 From Rotation Mappings to the Notion of a Matrix¶

This section introduces the notion of a matrix by way of linear mappings in two dimensions; we shall come to understand how a matrix arises naturally by working through a concrete rotation mapping.

3.1.1 Introducing the Matrix through Rotation Mappings¶

We first consider a vector $\mathbf{x} = \begin{bmatrix} x \\ y \end{bmatrix}$ in the two-dimensional plane $\mathbb{R}^2$ , which may be expressed in polar form as:

\mathbf{x} = r \begin{bmatrix} \cos\gamma \\ \sin\gamma \end{bmatrix}

(3)

Here $r = \|\mathbf{x}\|$ is the length of the vector and $\gamma$ is the directed polar angle of the nonzero vector, measured counterclockwise from the positive $x$ half-axis and understood modulo $2\pi$ ; for the zero vector the polar angle may be chosen arbitrarily.

Now we consider rotating this vector counterclockwise through $\alpha$ degrees. By the angle addition formulas for the trigonometric functions, the rotated vector is:

\mathbf{x}_{\text{new}} = r \begin{bmatrix} \cos(\alpha+\gamma) \\ \sin(\alpha+\gamma) \end{bmatrix} = r \begin{bmatrix} \cos \alpha \cos \gamma - \sin \alpha \sin \gamma \\ \sin \alpha \cos \gamma + \cos \alpha \sin \gamma \end{bmatrix}

(4)

Substituting $r\cos\gamma = x$ and $r\sin\gamma = y$ into the expression above, we obtain:

\mathbf{x}_{\text{new}} = \begin{bmatrix} \cos\alpha \cdot x - \sin \alpha \cdot y \\ \sin \alpha \cdot x + \cos \alpha \cdot y \end{bmatrix}

(5)

We note several important features:

Since $\alpha$ is a fixed constant, the final expression is a linear function of the vector $\begin{bmatrix} x \\ y \end{bmatrix}$
Since $\begin{bmatrix} x \\ y \end{bmatrix}$ stands for an arbitrary point of the plane, this mapping rotates every point of the plane through $\alpha$ degrees

We may separate the constants from the variables to obtain a more compact matrix form:

\begin{bmatrix} x_{\text{new}} \\ y_{\text{new}} \end{bmatrix} = \begin{bmatrix} \cos\alpha & - \sin \alpha \\ \sin\alpha & \cos \alpha \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix}

(6)

We extract the $2 \times 2$ array of numbers in the middle, call it a matrix (or a square matrix), and give it the symbol $\mathbf{A}$ :

\mathbf{A} = \begin{bmatrix} \cos\alpha & - \sin \alpha \\ \sin\alpha & \cos \alpha \end{bmatrix}

(7)

The original mapping may now be written compactly as:

\mathbf{x}_{\text{new}} = \mathbf{A}\mathbf{x} = T_{\mathbf{A}}(\mathbf{x})

(8)

Here $T_{\mathbf{A}}$ denotes the mapping by which the matrix $\mathbf{A}$ acts on a vector.

The question now is: how should the operation between a matrix and a vector be defined so that it yields the result we want?

Observing the structure of the rotation mapping, we find that if the entries of the matrix and the entries of the vector are paired and multiplied in the following way, the original mapping is recovered:

\begin{bmatrix} \rightarrow \\ \rightarrow \end{bmatrix} \begin{bmatrix} ↓ \end{bmatrix}

(9)

Specifically, the inner product of row 0 of the matrix with the vector gives component 0 of the result, and the inner product of row 1 of the matrix with the vector gives component 1. This is the central idea of matrix-vector multiplication.

3.1.2 A Natural Route to Matrix Multiplication¶

Before giving a rigorous definition of a matrix, let us look more closely at the rotation mapping. Clearly, besides rotating all at once, we may also rotate in two stages. If, for instance, we want to rotate through $\alpha + \beta$ degrees in the end, we may first rotate through $\alpha$ degrees and then through $\beta$ degrees.

From the point of view of mappings this is entirely reasonable: the composite of the two mappings ought to agree with the result of carrying out the rotation in one step. By the angle addition formulas, we want:

\begin{bmatrix} \cos(\alpha+\beta) & - \sin (\alpha+\beta) \\ \sin(\alpha+\beta) & \cos (\alpha+\beta) \end{bmatrix}

(10)

Expanding by the angle addition formulas gives:

\begin{bmatrix} \cos\alpha\cos\beta - \sin\alpha\sin\beta & -\sin\alpha\cos\beta - \cos\alpha\sin\beta \\ \sin\alpha\cos\beta + \cos\alpha\sin\beta & \cos\alpha\cos\beta - \sin\alpha\sin\beta \end{bmatrix}

(11)

and this ought to equal the result of first rotating through $\alpha$ degrees and then through $\beta$ degrees:

\begin{bmatrix} \cos\beta & - \sin \beta \\ \sin\beta & \cos \beta \end{bmatrix} \begin{bmatrix} \cos\alpha & - \sin \alpha \\ \sin\alpha & \cos \alpha \end{bmatrix}

(12)

There is an important observation to be made here: note that the two column vectors of the rotation matrix are essentially the same, differing only by a rotation through ninety degrees:

\begin{bmatrix} - \sin \alpha \\ \cos \alpha \end{bmatrix}=\begin{bmatrix} \cos(\alpha+\pi/2) \\ \sin(\alpha+\pi/2) \end{bmatrix}

(13)

This means that we may regard matrix multiplication as the matrix on the left acting separately on each of the column vectors of the matrix on the right:

\begin{bmatrix} \rightarrow \\ \rightarrow \end{bmatrix} \begin{bmatrix} ↓ & ↓ \end{bmatrix}

(14)

This gives us an important clue for defining matrix multiplication, a notion we shall develop in detail in the sections that follow.

3.1.3 Verifying Rotation Matrix Multiplication in Python¶

import sympy as sp

def print_header(title):
    print("=" * 60)
    print(f"  {title}")
    print("=" * 60)

def print_step(step, desc):
    print(f"\n▶ Step {step}: {desc}")
    print("-" * 40)

def compare_print(label, actual, expected_desc):
    print(f"\n[{label}]")
    print(f"  Computed value: {actual}")
    print(f"  Expected value: {expected_desc}")

# ============================================================
print_header("Example 3.1 | Symbolic Verification of Composed Rotations: R(θ)R(φ) = R(θ+φ)")
# ============================================================

# --- Step 0 ---
print_step(0, "Define the symbolic variables and the rotation matrices R(θ), R(φ)")
theta, phi = sp.symbols('theta phi', real=True)

R_theta = sp.Matrix([
    [sp.cos(theta), -sp.sin(theta)],
    [sp.sin(theta),  sp.cos(theta)]
])
R_phi = sp.Matrix([
    [sp.cos(phi), -sp.sin(phi)],
    [sp.sin(phi),  sp.cos(phi)]
])
print("R(θ) ="); sp.pprint(R_theta)
print("\nR(φ) ="); sp.pprint(R_phi)

# --- Step 1 ---
print_step(1, "Compute the matrix product R(θ) × R(φ) and simplify it")
product = R_theta * R_phi
simplified = sp.Matrix([
    [sp.trigsimp(product[i, j]) for j in range(2)]
    for i in range(2)
])
print("R(θ)R(φ) after simplification ="); sp.pprint(simplified)

# --- Step 2 ---
print_step(2, "Build the theoretical prediction R(θ+φ) and compare it entry by entry")
R_sum = sp.Matrix([
    [sp.cos(theta + phi), -sp.sin(theta + phi)],
    [sp.sin(theta + phi),  sp.cos(theta + phi)]
])
print("Theoretical prediction R(θ+φ) ="); sp.pprint(R_sum)

all_equal = all(
    sp.simplify(simplified[i, j] - R_sum[i, j]) == 0
    for i in range(2) for j in range(2)
)
compare_print(
    "R(θ)R(φ) = R(θ+φ)?",
    "✓ identical entry by entry" if all_equal else "✗ not equal",
    "R(θ+φ) (expanded by the angle addition formulas)"
)

# --- Step 3 ---
print_step(3, "Exhibit the correspondence with the angle addition identities")
print(f"  entry (1,1): cos(θ)cos(φ) − sin(θ)sin(φ) = {sp.trigsimp(product[0,0])}")
print(f"  entry (2,1): sin(θ)cos(φ) + cos(θ)sin(φ) = {sp.trigsimp(product[1,0])}")
print("\n  → Composing rotations supplies clues to the multiplication rule; only the composition requirement for all compatible linear mappings determines the general rule uniquely.")

# This code block is not shown in the printed book; only the figure is displayed
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.patches import Arc

np.set_printoptions(precision=4, suppress=True, linewidth=100)

# Fonts: the English edition needs no CJK font, so nothing is downloaded when _lang == 'en';
# the Chinese editions use this same block to fetch Noto Sans TC/SC where no CJK font is installed
import os, urllib.request
import matplotlib.font_manager as fm
_lang = 'en'
_cjk = ['Microsoft JhengHei', 'PingFang TC', 'Noto Sans CJK TC', 'Noto Sans TC']
_have = {f.name for f in fm.fontManager.ttflist}
if _lang != 'en' and not _have & set(_cjk):
    _font = os.path.join(os.path.expanduser('~'), '.cache', 'fonts', 'NotoSansTC.ttf')
    try:
        if not os.path.exists(_font):
            os.makedirs(os.path.dirname(_font), exist_ok=True)
            urllib.request.urlretrieve('https://github.com/google/fonts/raw/main/ofl/notosanstc/NotoSansTC%5Bwght%5D.ttf', _font + '.part')
            os.replace(_font + '.part', _font)
        fm.fontManager.addfont(_font)
        _have.add('Noto Sans TC')
    except OSError as err:
        print('Could not download the CJK font; Chinese text in figures may not display:', err)
plt.rcParams['font.family'] = [f for f in _cjk if f in _have] + ['DejaVu Sans']
plt.rcParams['axes.unicode_minus'] = False

# --- Color palette (Accent Mix) ---
C_BG   = "#F8F8F8"
C_GRID = "#D6D6D6"
C_AXIS = "#000000"
C_V1   = "#57068C"   # the rotated vector (violet)
C_V2   = "#006385"   # the original vector (deep blue)
C_AUX  = "#AB82C5"   # the arc of rotation (light violet)

def create_rotation_visualization():
    """Visualizing the rotation of a two-dimensional vector: the geometric effect of the rotation matrix R(45°)"""

    theta_deg = 45
    theta_rad = np.deg2rad(theta_deg)
    R = np.array([
        [np.cos(theta_rad), -np.sin(theta_rad)],
        [np.sin(theta_rad),  np.cos(theta_rad)]
    ])

    v_orig    = np.array([1.0, 0.0])
    v_rotated = R @ v_orig

    fig, ax = plt.subplots(figsize=(6, 6))
    ax.set_facecolor(C_BG)
    ax.grid(True, color=C_GRID, linewidth=0.8, alpha=0.7)
    ax.set_aspect('equal', adjustable='box')
    ax.set_box_aspect(1)

    ax.set_xlim([-0.3, 1.4])
    ax.set_ylim([-0.3, 1.2])
    ax.axhline(0, color=C_AXIS, linewidth=0.6)
    ax.axvline(0, color=C_AXIS, linewidth=0.6)

    # --- estimate data_range in order to set the quiver arrow width dynamically ---
    x_min, x_max = ax.get_xlim()
    data_range = x_max - x_min
    kw = dict(angles='xy', scale_units='xy', scale=1,
              units='xy', width=data_range * 0.012)

    ax.quiver(0, 0, v_orig[0],    v_orig[1],    color=C_V2, **kw,
              label=r'original vector $\mathbf{v} = [1,\,0]^\top$')
    ax.quiver(0, 0, v_rotated[0], v_rotated[1], color=C_V1, **kw,
              label=r'after rotation $R(45°)\mathbf{v}$')

    arc = Arc((0, 0), 0.55, 0.55, angle=0, theta1=0, theta2=theta_deg,
              color=C_AUX, linestyle='--', linewidth=1.8)
    ax.add_patch(arc)
    ax.text(0.38, 0.12, f'{theta_deg}°', fontsize=12, color=C_AUX)

    ax.set_title('Rotation of a Two-Dimensional Vector', fontsize=15, fontweight='bold', color=C_AXIS)
    ax.set_xlabel('X axis', fontsize=12, color=C_AXIS)
    ax.set_ylabel('Y axis', fontsize=12, color=C_AXIS)
    ax.legend(loc='upper left', fontsize=10)

    plt.tight_layout()
    plt.show()

    print(f"original vector: {v_orig}")
    print(f"rotation matrix R(45°):\n{R}")
    print(f"rotated vector: {v_rotated}")
    print(f"theoretical value: [cos45°, sin45°] = [{np.cos(theta_rad):.4f}, {np.sin(theta_rad):.4f}]")

create_rotation_visualization()

§3.1 Summary

This section used two-dimensional rotation to answer where the rule of matrix multiplication comes from: the angle addition formulas write the result of a rotation as an operation between a table of coefficients and a vector, and the composition of rotations supplies clues to the rule of multiplication. Considering rotations of the plane alone, however, is not enough to determine general matrix multiplication uniquely. Requiring that all matrices of compatible sizes satisfy $(\mathbf{AB})\mathbf{x}=\mathbf{A}(\mathbf{B}\mathbf{x})$ for every input $\mathbf{x}$ is what uniquely determines each column of the product matrix and yields standard matrix multiplication.

This derivation reveals what matrix multiplication really is: the column vectors of a matrix are, in essence, the images of the standard basis vectors under the mapping. This insight will be raised to a general theorem in §3.2, and will become the central tool with which the whole book analyzes the geometric meaning of a matrix.

3.2 Introduction to Matrices¶

The matrix is one of the most basic notions in linear algebra, rich in theoretical significance and in practical application. This section introduces the basic definition of a matrix and the basic operation of multiplication.

3.2.1 The Definition of a Matrix and Its Basic Operations¶

Definition 1 (Matrix)

A matrix is a rectangular array of $m \times n$ numbers arranged in $m$ rows and $n$ columns, usually written as:

\mathbf{A} = [a_{ij}]_{m \times n} = \begin{bmatrix} a_{00} & a_{01} & \cdots & a_{0,n-1} \\ a_{10} & a_{11} & \cdots & a_{1,n-1} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m-1,0} & a_{m-1,1} & \cdots & a_{m-1,n-1} \end{bmatrix}

(15)

Here $a_{ij}= (\mathbf{A})_{ij}$ denotes the entry in the $i$ -th row and the $j$ -th column of the matrix $\mathbf{A}$ , with indices $i \in \{0, 1, \ldots, m-1\}$ and $j \in \{0, 1, \ldots, n-1\}$ .

In a computer implementation, A[i,j] denotes the entry in the $i$ -th row and the $j$ -th column directly.

When the number of rows of a matrix equals the number of columns ( $m=n$ ), the matrix is called a square matrix.

The rows and the columns of a matrix may be singled out and regarded as vectors in their own right.

We are now in a position to introduce matrix multiplication. Matrix multiplication is one of the most basic and most important operations in linear algebra, and it is closely bound up with linear mappings.

Definition 3 (Matrix Multiplication)

For matrices $\mathbf{A} = [a_{ij}]_{m \times n}$ and $\mathbf{B} = [b_{jk}]_{n \times p}$ , their product $\mathbf{C} = \mathbf{AB} = [c_{ik}]_{m \times p}$ is defined by:

c_{ik} = \sum_{j=0}^{n-1} a_{ij}b_{jk}= \text{row}_i(\mathbf{A})\text{col}_k(\mathbf{B})

(19)

For real matrices this is the inner-product view: the $(i,k)$ entry of $\mathbf{C}$ is the inner product of the $i$ -th row of $\mathbf{A}$ with the $k$ -th column of $\mathbf{B}$ . The multiplication of complex matrices still uses the sum of products of entries above, without complex conjugation, so it does not coincide directly with this book’s complex inner product; the complex inner product is taken up in Chapter 9.

Several Other Views of Matrix Multiplication

Besides the inner-product view, there are several other ways of looking at matrix multiplication that are in common use

The column-combination view: the $k$ -th column of $\mathbf{C}$ is the linear combination of the column vectors of $\mathbf{A}$ whose coefficients are the corresponding entries of the $k$ -th column of $\mathbf{B}$ . $\text{col}_k(\mathbf{C})= \sum_i b_{ik} \text{col}_i(\mathbf{A})$
The row-combination view: the $k$ -th row of $\mathbf{C}$ is the linear combination of the row vectors of $\mathbf{B}$ whose coefficients are the corresponding entries of the $k$ -th row of $\mathbf{A}$ . $\text{row}_k(\mathbf{C})= \sum_i a_{ki} \text{row}_i(\mathbf{B})$
A sum of matrices of rank at most 1: multiplying the $k$ -th column of $\mathbf{A}$ by the $k$ -th row of $\mathbf{B}$ produces a matrix of rank at most 1. Its rank is exactly 1 when both factors are nonzero, and 0 when either factor is zero. $\mathbf{C}=\sum_k\text{col}_k(\mathbf{A})\text{row}_k(\mathbf{B})$ .

All of these may in fact be regarded as special cases of block matrix multiplication, and further examples will be given in the second section of Chapter 5.

Exercise 1

Compute the following matrix products:

$\begin{bmatrix}3&-2&1 \\ 1 &-1&2\end{bmatrix}\begin{bmatrix}-1&5\\-2&4\\3&-1 \end{bmatrix}$
$\begin{bmatrix}3&1&2&-1 \\ 0 &3&1&0 \\1&0&1&0 \end{bmatrix}\begin{bmatrix}0&5\\2&0\\0&-1\\-3&0 \end{bmatrix}$
$\begin{bmatrix}1 \\ 2 \\ 3 \\ 4\end{bmatrix}\begin{bmatrix}4&3&2&1 \end{bmatrix}$
$\begin{bmatrix}4&3&2&1 \end{bmatrix}\begin{bmatrix}1 \\ 2 \\ 3 \\ 4\end{bmatrix}$
$\begin{bmatrix}0&-1&0 \\ 1 &0&1 \\ 0 &1&0 \end{bmatrix}^4$
$\begin{bmatrix}2&1&0 \\ 0 &2&1 \\ 0 &0&2 \end{bmatrix}^4$
$\begin{bmatrix}a&0&0 \\ 0&b&0 \\ 0 &0&c \end{bmatrix}^n$
$\begin{bmatrix}1&1 \\ 1 &1 \end{bmatrix}^n$

3.2.2 The Two-Way Correspondence between Linear Mappings and Matrices¶

Much as in the previous section, we may use a matrix to define a linear mapping in the Cartesian coordinate system.

We now face a natural question in the reverse direction: if we start from a linear mapping, can we always find a matrix that represents it?

An important way of understanding a linear mapping is to observe how it acts on the standard basis vectors. As described in 2.1.3 on the standard basis, linear combinations of the standard basis of $\mathbb{R}^n$ generate any vector whatsoever in $\mathbb{R}^n$ . From the properties of a linear mapping it follows that a linear mapping is completely determined by its action on the basis vectors. If we know $T(\mathbf{e}_0), T(\mathbf{e}_1), \ldots, T(\mathbf{e}_{n-1})$ , then we can determine the action of $T$ on any vector at all. This shows that although a linear mapping is on the face of it a mapping from an infinite set to an infinite set, most of the time it behaves more like a mapping from a finite set to a finite set.

3.2.3 Basic Matrix Operations in Python¶

import numpy as np

np.set_printoptions(precision=4, suppress=True, linewidth=100)

def print_header(title):
    print("=" * 60)
    print(f"  {title}")
    print("=" * 60)

def print_step(step, desc):
    print(f"\n▶ Step {step}: {desc}")
    print("-" * 40)

def compare_print(label, actual, expected_desc):
    print(f"\n[{label}]")
    print(f"  Computed value: {actual}")
    print(f"  Expected value: {expected_desc}")

# ============================================================
print_header("Example 3.2 | Basic Matrix Operations: Addition, Multiplication, Transposition, and Algebraic Properties")
# ============================================================

# --- Step 0 ---
print_step(0, "Build the test matrices A, B, C")
A = np.array([[1, 2], [3, 4]])
B = np.array([[5, 6], [7, 8]])
C = np.array([[1, 0], [0, 2]])
print(f"A =\n{A}\nB =\n{B}\nC =\n{C}")

# --- Step 1 ---
print_step(1, "Addition and scalar multiplication")
print(f"A + B =\n{A + B}")
print(f"2A =\n{2 * A}")

# --- Step 2 ---
print_step(2, "Matrix multiplication AB (the @ operator) and the Hadamard product (∘)")
AB = A @ B
H  = A * B    # Hadamard product: entry-by-entry multiplication
print(f"AB = A @ B =\n{AB}")
print(f"A ∘ B (Hadamard product) =\n{H}")

# --- Step 3 ---
print_step(3, "Verify associativity: (AB)C = A(BC)")
left  = (A @ B) @ C
right = A @ (B @ C)
compare_print("(AB)C vs A(BC)", np.allclose(left, right),
              "associativity always holds (True)")
print(f"(AB)C =\n{left}")

# --- Step 4 ---
print_step(4, "Verify the failure of commutativity: AB ≠ BA (in general)")
BA = B @ A
compare_print("AB == BA?", np.array_equal(AB, BA),
              "matrices do not commute in general (False)")
print(f"AB =\n{AB}\nBA =\n{BA}")

# --- Step 5 ---
print_step(5, "Verify the transposition property (AB)^T = B^T A^T")
lhs = (A @ B).T
rhs = B.T @ A.T
compare_print("(AB)^T = B^T A^T?", np.allclose(lhs, rhs),
              "the reversal law for transposes always holds (True)")
print(f"tr(A) = {np.trace(A)} (the sum of the diagonal entries)")

3.2.4 ◆Higher-Order Matrix Operations with einsum in Python¶

Before demonstrating einsum, we first supply two definitions that will be needed below and that recur in later chapters.

Definition 5 (The Trace and the Frobenius Inner Product)

The trace of a square matrix $\mathbf{A} = [a_{ij}]_{n\times n}$ is defined as the sum of the entries on the main diagonal:

\text{tr}(\mathbf{A}) = \sum_{i=0}^{n-1} a_{ii}

(23)

The Frobenius inner product of two real matrices $\mathbf{A}, \mathbf{B}$ of the same size is defined by:

\langle \mathbf{A}, \mathbf{B}\rangle_F = \text{tr}(\mathbf{A}^\top \mathbf{B}) = \sum_{i}\sum_{j} a_{ij} b_{ij}

(24)

When $\mathbf{A}$ is $m\times n$ and $\mathbf{B}$ is $n\times m$ , the trace has an important property—cyclic invariance, $\text{tr}(\mathbf{AB}) = \text{tr}(\mathbf{BA})$ —whose deeper meaning (that the trace equals the sum of the eigenvalues) will be revealed in Chapter 8, and which will appear again with the density matrix in Chapter 10.

import numpy as np

np.set_printoptions(precision=4, suppress=True, linewidth=100)

def print_header(title):
    print("=" * 60)
    print(f"  {title}")
    print("=" * 60)

def print_step(step, desc):
    print(f"\n▶ Step {step}: {desc}")
    print("-" * 40)

def compare_print(label, actual, expected_desc):
    print(f"\n[{label}]")
    print(f"  Computed value: {actual}")
    print(f"  Expected value: {expected_desc}")

# ============================================================
print_header("Example 3.2◆ | Einstein Summation with einsum: Matrix Multiplication, the Trace, and the Frobenius Inner Product")
# ============================================================

# --- Step 0 ---
print_step(0, "Build the matrices A, B")
A = np.array([[1, 2], [3, 4]])
B = np.array([[5, 6], [7, 8]])
print(f"A =\n{A}\nB =\n{B}")

# --- Step 1 ---
print_step(1, "Matrix multiplication: einsum 'ik,kj->ij' = A @ B")
C_einsum = np.einsum('ik,kj->ij', A, B)
C_normal = A @ B
compare_print("einsum vs @", np.allclose(C_einsum, C_normal), "exactly the same (True)")
print(f"AB =\n{C_einsum}")

# --- Step 2 ---
print_step(2, "The trace of a matrix: einsum 'ii->' = tr(A) (summing along the diagonal)")
tr_einsum = int(np.einsum('ii->', A))
tr_normal = int(np.trace(A))
compare_print("tr(A)", tr_einsum, f"np.trace(A) = {tr_normal}")

# --- Step 3 ---
print_step(3, "Verify the commuting property of the trace: tr(AB) = tr(BA)")
tr_AB = int(np.einsum('ij,ji->', A, B))
tr_BA = int(np.einsum('ij,ji->', B, A))
compare_print("tr(AB) = tr(BA)?", tr_AB == tr_BA,
              "always holds (True); note that AB ≠ BA and yet the traces are equal")
print(f"  tr(AB) = {tr_AB}, tr(BA) = {tr_BA}")

# --- Step 4 ---
print_step(4, "The Frobenius inner product: einsum 'ij,ij->' = tr(A^T B)")
frob = int(np.einsum('ij,ij->', A, B))
frob_check = int(np.trace(A.T @ B))
compare_print("⟨A, B⟩_F = tr(A^T B)", frob, f"tr(A^T B) = {frob_check}")

§3.2 Summary

This section answered the question of what a complete system of operations a matrix, considered as an algebraic object, ought to possess. Starting from the formal definition, §3.2 built the complete algebraic structure of matrix addition, scalar multiplication, matrix multiplication, and transposition. Matrix multiplication in particular repays close study: the inner-product view, the column-combination view, the row-combination view, and the rank-one-sum view are four interpretations, equivalent to one another and yet each with its own uses—the inner-product view makes entry-by-entry computation convenient, while the column-combination view reveals how a mapping acts on an entire space.

The central result of §3.2 is the two-way correspondence theorem between linear mappings and matrices: every linear mapping $\mathbb{R}^n \to \mathbb{R}^m$ corresponds to exactly one $m \times n$ matrix, and the column vectors of that matrix are precisely the images of the standard basis vectors. This reduces “studying a mapping with infinitely many inputs” to “reading off finitely many column vectors,” and it links geometric intuition with algebraic computation. With this correspondence in hand, the matrix form of every geometric transformation in §3.3 becomes legible—one need only see where the column vectors go, and the geometric behavior of the whole mapping is plain at a glance.

3.3 Typical Linear Mappings in Two and Three Dimensions¶

3.3.1 The Zero Mapping¶

3.3.2 The Identity Mapping¶

3.3.3 The Scaling Mapping¶

Example 7 (Standard Basis Analysis of the Scaling Mapping)

The scaling matrix: $\mathbf{S} = \begin{bmatrix} s_x & 0 \\ 0 & s_y \end{bmatrix}$ , with $s_x, s_y \neq 0$

Action on the standard basis:

$T(\mathbf{e}_0) = \begin{bmatrix} s_x \\ 0 \end{bmatrix} = s_x\mathbf{e}_0$
$T(\mathbf{e}_1) = \begin{bmatrix} 0 \\ s_y \end{bmatrix} = s_y\mathbf{e}_1$

Geometric effect: scaling by a factor of $s_x$ in the x direction and by a factor of $s_y$ in the y direction

Analysis of properties:

Injective: distinct vectors are mapped to distinct positions
Surjective: any target vector can be reached
Bijective: the inverse mapping is $\begin{bmatrix} 1/s_x & 0 \\ 0 & 1/s_y \end{bmatrix}$
◇Determinant: $\det(\mathbf{S}) = s_x s_y \neq 0$

3.3.4 The Projection Matrix¶

Projection is an important notion in linear algebra: it “casts” a vector of a higher-dimensional space onto a lower-dimensional subspace.

Analysis of the Action of a Projection Matrix on the Standard Basis¶

Let us come to understand the properties of a projection matrix by observing how it acts on the standard basis vectors:

Example 8 (Standard Basis Analysis of the Projection onto the x Axis)

The projection matrix: $\mathbf{P}_x = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}$

Action on the standard basis:

$\mathbf{P}_x(\mathbf{e}_0) = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}\begin{bmatrix} 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 1 \\ 0 \end{bmatrix} = \mathbf{e}_0$
$\mathbf{P}_x(\mathbf{e}_1) = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}\begin{bmatrix} 0 \\ 1 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix} = \mathbf{0}$

Geometric interpretation:

$\mathbf{e}_0$ (the x direction) is left unchanged, because it already lies in the target of the projection
$\mathbf{e}_1$ (the y direction) is mapped to the zero vector, because it is perpendicular to the target of the projection

Analysis of properties:

Not injective: $\mathbf{P}_x(\mathbf{e}_1) = \mathbf{0}$ , and every vector of the form $\begin{bmatrix} a \\ b \end{bmatrix}$ is projected to $\begin{bmatrix} a \\ 0 \end{bmatrix}$
Not surjective: the image is only the x axis, and no nonzero point on the y axis can be reached
◇Kernel: $\ker(\mathbf{P}_x) = \text{span}\{\mathbf{e}_1\}$ (the y axis)
◇Image: $\text{Im}(\mathbf{P}_x) = \text{span}\{\mathbf{e}_0\}$ (the x axis)
◇Determinant: $\det(\mathbf{P}_x) = 1 \times 0 - 0 \times 0 = 0$

Example 9 (Computing a Projection Matrix and Analyzing It on the Basis Vectors)

Let $\mathbf{u} = \begin{bmatrix} 1/\sqrt{2} \\ 1/\sqrt{2} \end{bmatrix}$ be the unit vector making an angle of 45° with the x axis. The matrix of the projection onto the direction of $\mathbf{u}$ is:

\mathbf{P}_{\mathbf{u}} = \begin{bmatrix} 1/\sqrt{2} \\ 1/\sqrt{2} \end{bmatrix} \begin{bmatrix} 1/\sqrt{2} & 1/\sqrt{2} \end{bmatrix} = \begin{bmatrix} 1/2 & 1/2 \\ 1/2 & 1/2 \end{bmatrix}

(26)

Action on the standard basis:

$\mathbf{P}_{\mathbf{u}}(\mathbf{e}_0) = \begin{bmatrix} 1/2 & 1/2 \\ 1/2 & 1/2 \end{bmatrix}\begin{bmatrix} 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 1/2 \\ 1/2 \end{bmatrix}$
$\mathbf{P}_{\mathbf{u}}(\mathbf{e}_1) = \begin{bmatrix} 1/2 & 1/2 \\ 1/2 & 1/2 \end{bmatrix}\begin{bmatrix} 0 \\ 1 \end{bmatrix} = \begin{bmatrix} 1/2 \\ 1/2 \end{bmatrix}$

Observation: both standard basis vectors are projected onto one and the same vector $\begin{bmatrix} 1/2 \\ 1/2 \end{bmatrix} = \frac{1}{\sqrt{2}}\mathbf{u}$ !

This matrix projects any vector at all onto the line $y = x$ .

3.3.5 The Reflection Matrix¶

A reflection mapping flips a figure across some line or plane, just like the image one sees in a mirror.

Definition 7 (Reflection Matrix)

A reflection matrix reflects a vector across some line (in two dimensions) or plane (in three dimensions).

Common two-dimensional reflection matrices:

Reflection across the x axis: $\mathbf{R}_x = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix}$
Reflection across the y axis: $\mathbf{R}_y = \begin{bmatrix} -1 & 0 \\ 0 & 1 \end{bmatrix}$
Reflection across the line $y = x$ : $\mathbf{R}_{y=x} = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}$

In general, the matrix of the reflection across a line through the origin with unit normal vector $\mathbf{n}$ is:

\mathbf{R} = \mathbf{I} - 2\mathbf{n}\mathbf{n}^\top = \mathbf{I} - 2| \mathbf{n} \rangle \langle \mathbf{n} |

(27)

Analysis of the Action of a Reflection Matrix on the Standard Basis¶

Example 10 (Standard Basis Analysis of the Reflection across the x Axis)

The reflection matrix: $\mathbf{R}_x = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix}$

Action on the standard basis:

$\mathbf{R}_x(\mathbf{e}_0) = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix}\begin{bmatrix} 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 1 \\ 0 \end{bmatrix} = \mathbf{e}_0$
$\mathbf{R}_x(\mathbf{e}_1) = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix}\begin{bmatrix} 0 \\ 1 \end{bmatrix} = \begin{bmatrix} 0 \\ -1 \end{bmatrix} = -\mathbf{e}_1$

Geometric interpretation:

$\mathbf{e}_0$ (the x direction) is left unchanged, because it lies on the axis of reflection
$\mathbf{e}_1$ (the y direction) becomes its opposite, because it is perpendicular to the axis of reflection

Analysis of properties:

Injective: the images of the two basis vectors are linearly independent, so only the zero vector is mapped to the zero vector. That the images are distinct is not by itself enough to imply injectivity.
Surjective: any target vector can be reached by a reflection
Bijective: the inverse mapping is the mapping itself, $\mathbf{R}_x^{-1} = \mathbf{R}_x$
◇Determinant: $\det(\mathbf{R}_x) = 1 \times (-1) - 0 \times 0 = -1$ (a negative value indicates a reversal of orientation)

Example 11 (Standard Basis Analysis of the Reflection across the Line

y = x

)

The reflection matrix: $\mathbf{R}_{y=x} = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}$

Action on the standard basis:

$\mathbf{R}_{y=x}(\mathbf{e}_0) = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}\begin{bmatrix} 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 0 \\ 1 \end{bmatrix} = \mathbf{e}_1$
$\mathbf{R}_{y=x}(\mathbf{e}_1) = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}\begin{bmatrix} 0 \\ 1 \end{bmatrix} = \begin{bmatrix} 1 \\ 0 \end{bmatrix} = \mathbf{e}_0$

Geometric interpretation: this mapping is in fact a coordinate swap, sending $(x,y)$ to $(y,x)$

Analysis of properties:

Injective: the images of the two basis vectors are linearly independent, so only the zero vector is mapped to the zero vector. That the images are distinct is not by itself enough to imply injectivity.
Surjective: the two basis vectors still span the whole plane
Bijective: the inverse mapping is again the mapping itself
◇Determinant: $\det(\mathbf{R}_{y=x}) = 0 \times 0 - 1 \times 1 = -1$

3.3.6 The Rotation Mapping¶

We introduced the two-dimensional rotation matrix in detail in Section 3.1. Let us now review it and extend it to the three-dimensional case.

Example 12 (Standard Basis Analysis of the Two-Dimensional Rotation Mapping)

Counterclockwise rotation through $\theta$ (also called a Givens matrix): $\mathbf{R}(\theta) = \begin{bmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{bmatrix}$

Action on the standard basis:

$T_{\mathbf{R}}(\mathbf{e}_0) = \begin{bmatrix} \cos\theta \\ \sin\theta \end{bmatrix}$
$T_{\mathbf{R}}(\mathbf{e}_1) = \begin{bmatrix} -\sin\theta \\ \cos\theta \end{bmatrix}$

Geometric effect: $\mathbf{e}_0$ is rotated to the angle $\theta$ , and $\mathbf{e}_1$ is rotated to the angle $\theta + \pi/2$

Analysis of properties:

Injective: a rotation is a rigid motion, and preserves distances and angles
Surjective: any vector may be obtained by rotating some vector
Bijective: the inverse mapping is $\mathbf{R}(-\theta)$
◇Determinant: $\det(\mathbf{R}) = \cos^2\theta + \sin^2\theta = 1$
◇Orthogonality: $\mathbf{R}^\top \mathbf{R} = \mathbf{I}$

Three-Dimensional Rotation Matrices¶

Rotation in three-dimensional space is a good deal more complicated than in two dimensions, because we must specify an axis of rotation. The most basic three-dimensional rotations are those about the coordinate axes:

Definition 8 (Elementary Rotation Matrices in Three Dimensions)

The elementary rotation matrices for rotations about the coordinate axes:

Rotation about the z axis through the angle $\theta$ :

\mathbf{R}_z(\theta) = \begin{bmatrix} \cos\theta & -\sin\theta & 0 \\ \sin\theta & \cos\theta & 0 \\ 0 & 0 & 1 \end{bmatrix}

(28)

Rotation about the x axis through the angle $\theta$ :

\mathbf{R}_x(\theta) = \begin{bmatrix} 1 & 0 & 0 \\ 0 & \cos\theta & -\sin\theta \\ 0 & \sin\theta & \cos\theta \end{bmatrix}

(29)

Rotation about the y axis through the angle $\theta$ :

\mathbf{R}_y(\theta) = \begin{bmatrix} \cos\theta & 0 & \sin\theta \\ 0 & 1 & 0 \\ -\sin\theta & 0 & \cos\theta \end{bmatrix}

(30)

For linear mappings in three-dimensional space the method of analysis is entirely analogous:

Example 13 (Analysis of a Three-Dimensional Rotation Mapping)

Rotation about the z axis: $\mathbf{R}_z(\theta) = \begin{bmatrix} \cos\theta & -\sin\theta & 0 \\ \sin\theta & \cos\theta & 0 \\ 0 & 0 & 1 \end{bmatrix}$

Action on the standard basis:

$T_{\mathbf{R}}(\mathbf{e}_0) = \begin{bmatrix} \cos\theta \\ \sin\theta \\ 0 \end{bmatrix}$
$T_{\mathbf{R}}(\mathbf{e}_1) = \begin{bmatrix} -\sin\theta \\ \cos\theta \\ 0 \end{bmatrix}$
$T_{\mathbf{R}}(\mathbf{e}_2) = \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} = \mathbf{e}_2$

Analysis of properties:

the three image vectors are linearly independent
$\det(\mathbf{R}_z) = 1 \neq 0$
the mapping is bijective
the z direction is left unchanged, while a rotation is carried out within the xy plane

Any three-dimensional rotation whatsoever may be expressed as a combination of these three elementary rotations, and this leads to the notion of the Euler angles.

For a rotation about an arbitrary axis, we may use Rodrigues’ rotation formula:

Proof 2 (Rodrigues’ Rotation Formula (Sketch))

Write $\mathbf{K}$ for the skew-symmetric matrix above; its action is exactly the cross product, $\mathbf{K}\mathbf{w} = \mathbf{u}\times\mathbf{w}$ (that is, $\mathbf{K}$ is the cross product in matrix form). We need only verify the action of $\mathbf{R}$ on a basis and then extend by linearity.

The axis of rotation is left fixed: $\mathbf{K}\mathbf{u} = \mathbf{u}\times\mathbf{u} = \mathbf{0}$ , and hence $\mathbf{R}\mathbf{u} = \mathbf{u}$ —the axis vector is held fixed.
Rotation within the perpendicular plane: take any unit vector $\mathbf{w}$ perpendicular to $\mathbf{u}$ . Since $\mathbf{u}$ is a unit vector and $\mathbf{w}\perp\mathbf{u}$ , we have $\mathbf{K}\mathbf{w} = \mathbf{u}\times\mathbf{w}$ and $\mathbf{K}^2\mathbf{w} = \mathbf{u}\times(\mathbf{u}\times\mathbf{w}) = -\mathbf{w}$ . Substituting into the formula gives
$\mathbf{R}\mathbf{w} = \mathbf{w} + \sin\theta\,(\mathbf{u}\times\mathbf{w}) + (1-\cos\theta)(-\mathbf{w}) = \cos\theta\,\mathbf{w} + \sin\theta\,(\mathbf{u}\times\mathbf{w}),$
(33)
which is exactly the result of rotating $\mathbf{w}$ counterclockwise through $\theta$ within the plane perpendicular to $\mathbf{u}$ .
Extension by linearity: $\{\mathbf{u},\ \mathbf{w},\ \mathbf{u}\times\mathbf{w}\}$ form a basis of $\mathbb{R}^3$ , the action of $\mathbf{R}$ on these three vectors has already been completely determined, and extending by linearity we obtain that the action of $\mathbf{R}$ on the whole space agrees with the formula as stated. $\square$

The complete step-by-step computation (including the correspondence between $\mathbf{K}$ and the cross product of vectors) may be found in the special topic on skew-symmetric matrices in the exercise notebook, Exercise 61.

By means of this method of analysis based on the standard basis vectors, we can systematically understand and classify the various linear mappings, and this lays a solid foundation for the study of more complicated notions of linear algebra to come.

3.3.7 Verifying the Gimbal Lock of the Euler Angles in Python¶

import sympy as sp

def print_header(title):
    print("=" * 60)
    print(f"  {title}")
    print("=" * 60)

def print_step(step, desc):
    print(f"\n▶ Step {step}: {desc}")
    print("-" * 40)

def compare_print(label, actual, expected_desc):
    print(f"\n[{label}]")
    print(f"  Computed value: {actual}")
    print(f"  Expected value: {expected_desc}")

# ============================================================
print_header("Example 3.3 | Gimbal Lock of the Euler Angles: Verifying the Degeneracy of the Degrees of Freedom at θ = 90°")
# ============================================================

# --- Step 0 ---
print_step(0, "Define the symbols for the Euler angles: ψ (yaw), θ (pitch), φ (roll)")
psi, theta, phi = sp.symbols('psi theta phi', real=True)

# --- Step 1 ---
print_step(1, "Build the three elementary rotation matrices Rx(φ), Ry(θ), Rz(ψ)")
Rx = sp.Matrix([
    [1, 0, 0],
    [0, sp.cos(phi), -sp.sin(phi)],
    [0, sp.sin(phi),  sp.cos(phi)]
])
Ry = sp.Matrix([
    [ sp.cos(theta), 0, sp.sin(theta)],
    [0, 1, 0],
    [-sp.sin(theta), 0, sp.cos(theta)]
])
Rz = sp.Matrix([
    [sp.cos(psi), -sp.sin(psi), 0],
    [sp.sin(psi),  sp.cos(psi), 0],
    [0, 0, 1]
])
print("Rx(φ) ="); sp.pprint(Rx)

# --- Step 2 ---
print_step(2, "Compute the total rotation matrix R = Rz(ψ) Ry(θ) Rx(φ)")
R = sp.trigsimp(Rz * Ry * Rx)
print("R = Rz × Ry × Rx ="); sp.pprint(R)

# --- Step 3 ---
print_step(3, "Substitute θ = π/2 and observe the gimbal lock")
R_lock = sp.simplify(R.subs(theta, sp.pi / 2))
print("the rotation matrix R at θ = 90° ="); sp.pprint(R_lock)

# --- Step 4 ---
print_step(4, "Analyze the degenerate structure: at θ = π/2 the matrix depends only on φ − ψ")
for i in range(3):
    for j in range(3):
        print(f"  R_lock[{i},{j}] = {sp.trigsimp(R_lock[i, j])}")
print("  With θ = π/2 fixed, φ and ψ determine the orientation only through the difference φ − ψ.")
compare_print(
    "Independent degrees of freedom",
    "3 angles → only 2 effective degrees of freedom remain",
    "θ = 90° makes Ry carry the Z axis onto the X axis, so the axes of rotation for ψ and φ coincide"
)
print("\n  → Gimbal lock: one independent direction of rotation is lost, and a quaternion parametrization must be used instead.")

# Colab: enable the custom widget manager so that FigureWidget and the sliders are interactive; skipped elsewhere
try:
    from google.colab import output
    output.enable_custom_widget_manager()
except ImportError:
    pass

# This code block is not shown in the printed book; only the figure is displayed
import numpy as np
import plotly.graph_objects as go
import ipywidgets as widgets
from IPython.display import display

np.set_printoptions(precision=4, suppress=True, linewidth=100)

def print_header(title):
    print("=" * 60)
    print(f"  {title}")
    print("=" * 60)

def print_step(step, desc):
    print(f"\n▶ Step {step}: {desc}")
    print("-" * 40)

def compare_print(label, actual, expected_desc):
    print(f"\n[{label}]")
    print(f"  Computed value: {actual}")
    print(f"  Expected value: {expected_desc}")

# --- Color palette (Accent Mix) ---
C_BG   = "#F8F8F8"
C_GRID = "#D6D6D6"
C_V1   = "#57068C"   # the rotated vector (violet)
C_V2   = "#006385"   # the original vector (deep blue)
C_T1   = "#2AD2C9"   # the axis of rotation (teal)

def rodrigues_rotation_matrix(axis, theta):
    """
    Rodrigues' rotation formula: R = I + sin(θ)K + (1 − cos(θ))K²
    axis: the axis of rotation [x, y, z] (normalized automatically); theta: the angle of rotation (in radians)
    """
    axis = np.array(axis, dtype=float)
    norm = np.linalg.norm(axis)
    if norm == 0:
        raise ValueError("The axis of rotation must not be the zero vector.")
    axis = axis / norm
    K = np.array([
        [0,       -axis[2],  axis[1]],
        [axis[2],  0,       -axis[0]],
        [-axis[1], axis[0],  0      ]
    ])
    return np.eye(3) + np.sin(theta) * K + (1 - np.cos(theta)) * (K @ K)

def demo_rodrigues_formula():
    # ============================================================
    print_header("Example 3.3◆ | Rodrigues' Rotation Formula: Verifying the Properties of the Rotation Matrix about an Arbitrary Axis")
    # ============================================================

    # --- Step 0 ---
    print_step(0, "Rotation through 45° about the z axis")
    R_z45 = rodrigues_rotation_matrix([0, 0, 1], np.pi / 4)
    print(f"R(z, 45°) =\n{R_z45}")

    # --- Step 1 ---
    print_step(1, "Rotation through 60° about the arbitrary axis [1, 1, 1]")
    R_arb = rodrigues_rotation_matrix([1, 1, 1], np.pi / 3)
    print(f"R([1,1,1], 60°) =\n{R_arb}")

    # --- Step 2 ---
    print_step(2, "Verify the properties of a rotation matrix: det(R)=1, R^T R = I")
    det_val   = np.linalg.det(R_z45)
    ortho_err = np.linalg.norm(R_z45.T @ R_z45 - np.eye(3))
    compare_print("det(R(z,45°))", f"{det_val:.6f}", "always 1")
    compare_print("‖R^T R − I‖", f"{ortho_err:.2e}", "always 0 (an orthogonal matrix)")

def create_3d_rotation_visualization():
    """Interactive 3D visualization of rotation: FigureWidget is used so that the camera view is preserved"""

    v0 = np.array([1.0, 0.0, 0.0])

    # --- Step 0: build the initial FigureWidget (the view state is held by the widget) ---
    fig = go.FigureWidget(data=[
        go.Scatter3d(  # trace 0: the original vector
            x=[0, v0[0]], y=[0, v0[1]], z=[0, v0[2]],
            mode='lines+markers',
            line=dict(color=C_V2, width=8), marker=dict(size=4),
            name='original vector v₀'
        ),
        go.Scatter3d(  # trace 1: the rotated vector
            x=[0, v0[0]], y=[0, v0[1]], z=[0, v0[2]],
            mode='lines+markers',
            line=dict(color=C_V1, width=8), marker=dict(size=4),
            name='after rotation Rv₀'
        ),
        go.Scatter3d(  # trace 2: the axis of rotation
            x=[-0, 0], y=[-0, 0], z=[-1, 1],
            mode='lines+markers',
            line=dict(color=C_T1, width=5, dash='dash'), marker=dict(size=3),
            name='axis of rotation'
        ),
    ])
    fig.update_layout(
        plot_bgcolor=C_BG, paper_bgcolor=C_BG,
        scene=dict(
            xaxis=dict(range=[-1.5, 1.5], title='X', gridcolor=C_GRID),
            yaxis=dict(range=[-1.5, 1.5], title='Y', gridcolor=C_GRID),
            zaxis=dict(range=[-1.5, 1.5], title='Z', gridcolor=C_GRID),
            aspectmode='cube'
        ),
        title='Rodrigues Rotation (drag the sliders; zooming the view is not reset)',
        scene_camera=dict(eye=dict(x=1.5, y=1.5, z=1.5))
    )

    # --- Step 1: define the update function (only the data are changed; the figure is not rebuilt) ---
    def update_plot(x, y, z, theta_deg):
        axis = np.array([x, y, z], dtype=float)
        norm = np.linalg.norm(axis)
        if norm == 0:
            print("The axis of rotation must not be the zero vector; set at least one axis component to a nonzero value. The figure keeps the last valid rotation.")
            return
        ax_n = axis / norm
        R = rodrigues_rotation_matrix(ax_n, np.radians(theta_deg))
        v1 = R @ v0

        with fig.batch_update():   # update in one batch, to avoid flicker
            # trace 1: the rotated vector
            fig.data[1].x = [0, v1[0]]
            fig.data[1].y = [0, v1[1]]
            fig.data[1].z = [0, v1[2]]
            # trace 2: the axis of rotation
            fig.data[2].x = [-ax_n[0], ax_n[0]]
            fig.data[2].y = [-ax_n[1], ax_n[1]]
            fig.data[2].z = [-ax_n[2], ax_n[2]]
            # update only the title, leaving layout.scene alone (this preserves the camera)
            fig.layout.title.text = (
                f'Rodrigues Rotation  unit axis=[{ax_n[0]:.3f},{ax_n[1]:.3f},{ax_n[2]:.3f}]  angle={theta_deg}°'
            )

    # --- Step 2: build the sliders and connect them ---
    controls = dict(
        x=widgets.FloatSlider(min=-1, max=1, step=0.1, value=0,   description='Axis X:'),
        y=widgets.FloatSlider(min=-1, max=1, step=0.1, value=0,   description='Axis Y:'),
        z=widgets.FloatSlider(min=-1, max=1, step=0.1, value=1,   description='Axis Z:'),
        theta_deg=widgets.FloatSlider(min=0, max=360, step=10, value=45, description='Angle (°):')
    )
    ui = widgets.VBox(list(controls.values()))
    out = widgets.interactive_output(update_plot, controls)
    display(fig, ui, out)

demo_rodrigues_formula()
create_3d_rotation_visualization()

§3.3 Summary

This section answered the question of what the most common linear mappings in two- and three-dimensional space look like, and how their matrix forms are to be read. By means of the principle established in §3.2 that “a column vector is the image of a standard basis vector,” the matrix of each kind of mapping can be written down directly from the geometric picture. The zero mapping flattens space to a single point (determinant zero); a rotation preserves distances and angles (determinant one); a reflection reverses orientation (determinant negative one); and a projection matrix satisfies $\mathbf{P}^2 = \mathbf{P}$ , the geometric embodiment of idempotence.

The discussion of Rodrigues’ formula and of the Euler angles for three-dimensional rotations reveals a deeper phenomenon: the composition of three-dimensional rotation matrices is not commutative, while gimbal lock shows that the Euler angle parametrization loses a degree of freedom in certain configurations—which hints that the topological structure of the rotation group is far more complicated than in two dimensions, and is a first intimation of the Lie groups to be studied later. The classification of mappings built up in this section leads naturally to the next question: can the effect of a mapping on the “size” of space be described quantitatively? That is precisely what the determinant of §3.4 will answer.

3.4 Determinants and Mappings of Space¶

In what we have studied so far we have come to understand the geometric effect of various linear mappings, but one important quantitative tool is still missing—how are we to measure precisely the effect of a mapping on the “size” of space? The determinant is exactly the notion that resolves this question.

3.4.1 The Column Vectors of a Square Matrix Form an Area¶

Let us begin with a concrete geometric question: if we have a unit square, how does its area change after a linear mapping?

Example 14 (The Two-Dimensional Determinant)

For a $2 \times 2$ matrix:

\mathbf{A} = \begin{bmatrix} a & c \\ b & d \end{bmatrix}

(34)

the formula for computing its determinant is obtained as follows:

We consider the two vectors $\begin{bmatrix} a \\ b \end{bmatrix}$ and $\begin{bmatrix} c \\ d \end{bmatrix}$ , embed them in three-dimensional space, and use the cross-product formula of the previous chapter to obtain:

\det(\mathbf{A}) = \text{the area of the parallelogram formed by the column vectors of the square matrix} = ad - bc

(35)

Geometric meaning: the absolute value of the determinant equals the ratio of the area of the image figure to the area of the original figure.

3.4.2 The Column Vectors of a Square Matrix Form a Volume¶

In the three-dimensional case, what the determinant describes is the volume formed by the column vectors of the square matrix.

Recall from Chapter 2: the scalar triple product $\mathbf{a}\cdot(\mathbf{b}\times\mathbf{c})$ of three vectors $\mathbf{a}, \mathbf{b}, \mathbf{c}$ is numerically equal to the signed volume of the parallelepiped they span (the sign recording the orientation given by the right-hand rule). This conclusion was established by the Chapter 2 exercises Exercise 30 and Exercise 31. Substituting the three column vectors of a $3\times 3$ matrix into the scalar triple product therefore yields exactly the volume of the parallelepiped they span—the natural generalization to three dimensions of the two-dimensional “signed area.”

Example 15 (Computing a Three-Dimensional Determinant)

Compute the determinant of the matrix below:

\mathbf{A} = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{bmatrix}

(39)

Expanding along cofactors:

\det(\mathbf{A}) = 1(5 \times 9 - 6 \times 8) - 2(4 \times 9 - 6 \times 7) + 3(4 \times 8 - 5 \times 7)

(40)

= 1(45 - 48) - 2(36 - 42) + 3(32 - 35)

(41)

= 1(-3) - 2(-6) + 3(-3) = -3 + 12 - 9 = 0

(42)

Therefore $\det(\mathbf{A}) = 0$ , which shows that the column vectors of this matrix are linearly dependent and that its image cannot span the whole of three-dimensional space.

3.4.3 Determinants of Some Special Matrices¶

Let us compute the determinants of some mapping matrices with which we are already familiar:

Example 16 (Determinants of Common Mappings)

The identity matrix: $\mathbf{I} = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}$
$\det(\mathbf{I}) = 1 \times 1 - 0 \times 0 = 1$
Geometric interpretation: the area is not changed, in accordance with intuition.
A scaling matrix: $\mathbf{S} = \begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix}$
$\det(\mathbf{S}) = 2 \times 3 - 0 \times 0 = 6$
Geometric interpretation: the area is magnified by a factor of 6 (a factor of 2 in the x direction and 3 in the y direction).
A rotation matrix: $\mathbf{R} = \begin{bmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{bmatrix}$
$\det(\mathbf{R}) = \cos^2\theta + \sin^2\theta = 1$
Geometric interpretation: a rotation does not change area.
A reflection matrix: $\mathbf{M} = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix}$
$\det(\mathbf{M}) = 1 \times (-1) - 0 \times 0 = -1$
Geometric interpretation: the size of the area is unchanged, but the orientation is changed (the negative sign indicates a flip).

3.4.4 Checking the Formula for the Determinant in Python¶

import sympy as sp

def print_header(title):
    print("=" * 60)
    print(f"  {title}")
    print("=" * 60)

def print_step(step, desc):
    print(f"\n▶ Step {step}: {desc}")
    print("-" * 40)

def compare_print(label, actual, expected_desc):
    print(f"\n[{label}]")
    print(f"  Computed value: {actual}")
    print(f"  Expected value: {expected_desc}")

# ============================================================
print_header("Example 3.4 | Symbolic Verification of the Determinant: Equivalence of the Cross-Product/Inner-Product Formula and det(M)")
# ============================================================

# --- Step 0 ---
print_step(0, "Build a symbolic 3×3 matrix M (arranged by columns: columns 0, 1, 2 are col0, col1, col2)")
a,b,c,d,e,f,g,h,m = sp.symbols('a b c d e f g h m', real=True)
# the matrix is arranged by columns: column 0 of M = [a,b,c], column 1 = [d,e,f], column 2 = [g,h,m]
M = sp.Matrix([
    [a, d, g],
    [b, e, h],
    [c, f, m]
])
print("M ="); sp.pprint(M)

# --- Step 1 ---
print_step(1, "Compute det(M) and det(M^T) directly with sympy")
det_M  = M.det()
det_MT = M.T.det()
print(f"det(M)   = {det_M}")
compare_print("det(M) = det(M^T)?",
              sp.simplify(det_M - det_MT) == 0,
              "transposition does not change the determinant (True)")

# --- Step 2 ---
print_step(2, "Compute det(M) by the cross-product and inner-product formula: col0 · (col1 × col2)")
col0 = sp.Matrix([a, b, c])
col1 = sp.Matrix([d, e, f])
col2 = sp.Matrix([g, h, m])
det_cross = col0.dot(col1.cross(col2))
print(f"col0 · (col1 × col2) = {det_cross}")

# --- Step 3 ---
print_step(3, "Verify that the two ways of computing agree")
diff = sp.simplify(det_cross - det_M)
compare_print(
    "det(M) = col0·(col1×col2)?",
    f"difference = {diff}",
    "determinant = the signed volume spanned by the three column vectors (the difference is always 0)"
)
assert diff == 0, "Verification failed!"
print("\n  → Geometric interpretation of the determinant: the signed volume of the parallelepiped spanned by the three column vectors.")

3.4.5 Computing Determinants and Verifying Their Geometric Meaning in Python¶

import math
import numpy as np

np.set_printoptions(precision=4, suppress=True, linewidth=100)

def print_header(title):
    print("=" * 60)
    print(f"  {title}")
    print("=" * 60)

def print_step(step, desc):
    print(f"\n▶ Step {step}: {desc}")
    print("-" * 40)

def compare_print(label, actual, expected_desc):
    print(f"\n[{label}]")
    print(f"  Computed value: {actual}")
    print(f"  Expected value: {expected_desc}")

# ============================================================
print_header("Example 3.5 | Determinants of Typical Linear Mappings and Verification of Their Geometric Meaning")
# ============================================================

# --- Step 0 ---
print_step(0, "Define helper functions for the typical mapping matrices")

def scaling_matrix(sx, sy):
    """Scaling matrix: scales by a factor of sx in the x direction and by a factor of sy in the y direction"""
    return np.array([[sx, 0.0], [0.0, sy]])

def projection_to_line(u):
    """Projection matrix: P = uu^T (projection onto the direction of the unit vector u)"""
    u = u / np.linalg.norm(u)
    return np.outer(u, u)

def rotation_matrix(theta):
    """Counterclockwise rotation matrix R(θ)"""
    c, s = math.cos(theta), math.sin(theta)
    return np.array([[c, -s], [s, c]])

# --- Step 1 ---
print_step(1, "Build six typical mapping matrices")
I2          = np.eye(2)
scale_23    = scaling_matrix(2, 3)
proj_x      = np.array([[1.0, 0.0], [0.0, 0.0]])          # projection onto the x axis
proj_45     = projection_to_line(np.array([1.0, 1.0]))     # projection onto y=x
refl_x      = np.array([[1.0, 0.0], [0.0, -1.0]])          # reflection across the x axis
rot_45      = rotation_matrix(math.pi / 4)

matrices = {
    "Identity I":        I2,
    "Scaling (2,3)":         scale_23,
    "Projection→x":          proj_x,
    "Projection→y=x":           proj_45,
    "Reflection x":          refl_x,
    "Rotation 45°":           rot_45,
}

print(f"\n{'Mapping type':<14} {'det(A)':>10}  {'Geometric meaning'}")
print("-" * 55)
meanings = {
    "Identity I":  "area scale factor = 1, orientation unchanged",
    "Scaling (2,3)":   "area scaled by 2×3 = 6",
    "Projection→x":    "collapsed to 1D, the area becomes zero",
    "Projection→y=x":     "collapsed to 1D, the area becomes zero",
    "Reflection x":    "area unchanged, orientation reversed (det < 0)",
    "Rotation 45°":     "area unchanged, orientation preserved (det = 1)",
}
for name, mat in matrices.items():
    d = np.linalg.det(mat)
    print(f"{name:<14} {d:>10.4f}  {meanings[name]}")

# --- Step 2 ---
print_step(2, "Verify that the determinant of a rotation matrix is always 1 (for every angle)")
angles = np.linspace(0, 2 * math.pi, 9)
dets   = [np.linalg.det(rotation_matrix(a)) for a in angles]
all_one = all(abs(d - 1.0) < 1e-10 for d in dets)
compare_print(
    "det(R(θ)) = 1 for every θ?",
    f"largest error = {max(abs(d-1) for d in dets):.2e}",
    "rotation preserves area and orientation (True)"
)
for a, d in zip(angles, dets):
    print(f"  θ = {math.degrees(a):6.1f}°  det = {d:.6f}")

# --- Step 3 ---
print_step(3, "Verify the idempotence of a projection matrix: P² = P")
compare_print("P_x² = P_x?", np.allclose(proj_x @ proj_x, proj_x),
              "idempotence: projecting twice is the same as projecting once (True)")
compare_print("P_45² = P_45?", np.allclose(proj_45 @ proj_45, proj_45),
              "idempotence: this holds for the projection matrix onto any direction (True)")

§3.4 Summary

This section answered the question of how a single number may measure precisely the effect of a linear mapping on the volume of space. The determinant is exactly that number. The two-dimensional determinant equals the signed area of the parallelogram spanned by the two column vectors; the three-dimensional determinant equals the signed volume of the parallelepiped spanned by the three column vectors; $|\det \mathbf{A}|$ gives the factor by which the mapping scales volume; and the sign records whether orientation has been reversed.

The most important conclusion is the criterion for invertibility: $\det(\mathbf{A}) \neq 0$ is equivalent to the mapping compressing no nonzero part of space down to dimension zero (that is, to its being bijective), and hence equivalent to the matrix being invertible. Conversely, $\det(\mathbf{A}) = 0$ means that the mapping compresses the whole of $\mathbb{R}^n$ into a subspace of lower dimension, so the input cannot be recovered uniquely: a target in the image has infinitely many preimages, and a target outside the image has none. This geometric story about the determinant lays the groundwork for §3.5: whether an inverse matrix exists depends on whether the determinant is zero. The complete theory of the determinant will be developed systematically, in axiomatic form, in Chapter 7.

3.5 Linear Systems and Linear Mappings¶

We now examine the relationship between systems of linear equations and some typical linear mappings from two dimensions to two dimensions and from three dimensions to three dimensions. In fact, solving a system of linear equations is nothing other than searching for a preimage of a linear mapping.

3.5.1 Understanding Linear Systems Geometrically¶

The Need for an Inverse Mapping, and the Notion of an Inverse Matrix¶

Starting from the system $\mathbf{Ax} = \mathbf{b}$ , we naturally ask: how are we to “undo” the mapping $\mathbf{A}$ in order to find the original vector $\mathbf{x}$ ?

This is like reasoning backward in everyday life:

if we know the result of subjecting an object to some transformation, we want to know what it looked like originally
if we know the ciphertext, we want to recover the plaintext by an inverse operation
if we know the output of a function, we want to find the corresponding input

In linear algebra, this operation of “undoing” is exactly the role played by the inverse matrix.

When the inverse matrix of $\mathbf{A}$ exists, solving the linear system $\mathbf{Ax} = \mathbf{b}$ becomes entirely straightforward:

\mathbf{x} = \mathbf{A}^{-1}\mathbf{b}

(44)

In the remainder of this section we concentrate first on the ideal case—that of an invertible square matrix—and give concrete methods for computing the inverse matrix in two and three dimensions. These basic skills will lay important groundwork for understanding the more complicated situations.

3.5.2 The Geometric Meaning of a Two-Dimensional Linear System¶

3.5.3 The Inverse Matrix and the Solution of a System¶

When the matrix of the mapping is invertible, we can “undo” the mapping in order to find the original vector:

Example 18 (Solving by Means of the Inverse Matrix)

Solve the system $\begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 5 \\ 4 \end{bmatrix}$

Step 1. Compute the determinant

\det(\mathbf{A}) = 2 \times 2 - 1 \times 1 = 3 \neq 0

(49)

Step 2. Compute the inverse matrix

\mathbf{A}^{-1} = \frac{1}{3}\begin{bmatrix} 2 & -1 \\ -1 & 2 \end{bmatrix}

(50)

Step 3. Solve

\mathbf{x} = \mathbf{A}^{-1}\mathbf{b} = \frac{1}{3}\begin{bmatrix} 2 & -1 \\ -1 & 2 \end{bmatrix}\begin{bmatrix} 5 \\ 4 \end{bmatrix} = \frac{1}{3}\begin{bmatrix} 6 \\ 3 \end{bmatrix} = \begin{bmatrix} 2 \\ 1 \end{bmatrix}

(51)

Verification: $\begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix}\begin{bmatrix} 2 \\ 1 \end{bmatrix} = \begin{bmatrix} 5 \\ 4 \end{bmatrix}$ ✓

3.5.4 Generalization to Three Dimensions¶

A three-dimensional linear system may likewise be written in the matrix form $\mathbf{Ax} = \mathbf{b}$ , and so long as $\det(\mathbf{A}) \neq 0$ there is a unique solution $\mathbf{x} = \mathbf{A}^{-1}\mathbf{b}$ .

Example 19 (Solving a Three-Dimensional System by Means of the Inverse Matrix)

Solve the system:

\begin{cases} 2x + y \phantom{{}+z} = 4 \\ \phantom{2}x + 3y + z = 8 \\ \phantom{2x + {}}y + 2z = 4 \end{cases}

(54)

Step 1. Write it in the matrix form $\mathbf{Ax} = \mathbf{b}$

\mathbf{A} = \begin{bmatrix} 2 & 1 & 0 \\ 1 & 3 & 1 \\ 0 & 1 & 2 \end{bmatrix}, \quad \mathbf{b} = \begin{bmatrix} 4 \\ 8 \\ 4 \end{bmatrix}

(55)

Matching against the symbols of the formula: $a=2,\,d=1,\,g=0,\;b=1,\,e=3,\,h=1,\;c=0,\,f=1,\,m=2$ .

Step 2. Compute the determinant

\det(\mathbf{A}) = 2(3 \cdot 2 - 1 \cdot 1) - 1(1 \cdot 2 - 0 \cdot 1) + 0 = 2(5) - 1(2) = 8

(56)

Step 3. Apply the formula to compute each entry, and assemble the inverse matrix

\mathbf{A}^{-1} = \frac{1}{8}\begin{bmatrix} em-hf & -(dm-fg) & dh-eg \\ -(bm-ch) & am-cg & -(ah-bg) \\ bf-ce & -(af-cd) & ae-bd \end{bmatrix} = \frac{1}{8}\begin{bmatrix} 5 & -2 & 1 \\ -2 & 4 & -2 \\ 1 & -2 & 5 \end{bmatrix}

(57)

Step 4. Solve

\mathbf{x} = \mathbf{A}^{-1}\mathbf{b} = \frac{1}{8}\begin{bmatrix} 5 & -2 & 1 \\ -2 & 4 & -2 \\ 1 & -2 & 5 \end{bmatrix} \begin{bmatrix} 4 \\ 8 \\ 4 \end{bmatrix} = \frac{1}{8}\begin{bmatrix} 8 \\ 16 \\ 8 \end{bmatrix} = \begin{bmatrix} 1 \\ 2 \\ 1 \end{bmatrix}

(58)

Verification: $\mathbf{Ax} = \begin{bmatrix}2&1&0\\1&3&1\\0&1&2\end{bmatrix}\begin{bmatrix}1\\2\\1\end{bmatrix} = \begin{bmatrix}4\\8\\4\end{bmatrix}$ ✓

Remark 3 (The Determinant and the Solutions of a System (Geometric Intuition))

Both the formula and the geometric meaning show that the value of the determinant settles directly what happens to the solutions of the system. When the determinant is nonzero, the images of the $n$ basis vectors “spread out” in space into a nondegenerate figure (a parallelogram in two dimensions, a parallelepiped in three) that fills the whole space; the mapping is therefore a bijection, and the system must have a unique solution. Conversely, when the determinant is zero, the mapping compresses space into a lower dimension; in that case a target in the image has infinitely many preimages, and a target outside the image has none.

The argument above is the geometric intuition version. The step from “compressed into a lower dimension $\Rightarrow$ the input cannot be recovered uniquely for every target” is an intuitive leap that we accept provisionally in this chapter; its rigorous proof relies on the structure theory of solutions and on the theory of determinants, treated in Chapter 6 and Chapter 7 respectively.

3.5.5 Solving Linear Systems in Python¶

import numpy as np

np.set_printoptions(precision=4, suppress=True, linewidth=100)

def print_header(title):
    print("=" * 60)
    print(f"  {title}")
    print("=" * 60)

def print_step(step, desc):
    print(f"\n▶ Step {step}: {desc}")
    print("-" * 40)

def compare_print(label, actual, expected_desc):
    print(f"\n[{label}]")
    print(f"  Computed value: {actual}")
    print(f"  Expected value: {expected_desc}")

# ============================================================
print_header("Example 3.5 | Solving a Linear System: the Inverse-Matrix Method and Direct Solution")
# ============================================================

# --- Step 0 ---
print_step(0, "Build the system Ax = b")
# the system: 2x + y = 5
#              x + 2y = 4
A = np.array([[2.0, 1.0],
              [1.0, 2.0]])
b = np.array([5.0, 4.0])
print("the system:\n  2x + y = 5\n   x + 2y = 4")
print(f"\nA =\n{A}")
print(f"b = {b}")

# --- Step 1 ---
print_step(1, "Test for invertibility: det(A)")
det_A = np.linalg.det(A)
compare_print("det(A)", f"{det_A:.4f}",
              "det(A) ≠ 0 → the mapping is invertible → the system has a unique solution")

# --- Step 2 ---
print_step(2, "Method 1: the inverse-matrix method,  x = A^{-1} b")
A_inv = np.linalg.inv(A)
x_inv = A_inv @ b
print(f"A^{{-1}} =\n{A_inv}")
print(f"x = A^{{-1}}b = {x_inv}")
residual = np.linalg.norm(A @ x_inv - b)
compare_print("residual ‖Ax − b‖", f"{residual:.2e}", "numerically this should approach 0")

# --- Step 3 ---
print_step(3, "Method 2: np.linalg.solve (numerically stable; recommended)")
x_solve = np.linalg.solve(A, b)
compare_print(
    "do the two methods give the same solution vector?",
    np.allclose(x_inv, x_solve),
    "the results agree (True); solve has smaller numerical error than the inverse-matrix method"
)
print(f"x = {x_solve}  →  x₀ = {x_solve[0]:.4f}, x₁ = {x_solve[1]:.4f}")
print(f"\n  Geometric meaning: the vector b = {b} is a linear combination of")
print(f"  the column vectors [{A[0,0]},{A[1,0]}] and [{A[0,1]},{A[1,1]}] of A,")
print(f"  whose coefficients are exactly the solution x₀ = {x_solve[0]:.4f}, x₁ = {x_solve[1]:.4f}.")

§3.5 Summary

This section answered the question of what solving the linear system $\mathbf{Ax} = \mathbf{b}$ has to do with the theory of linear mappings. The answer is this: solving the system is searching for a preimage of the mapping $T_{\mathbf{A}}$ —which vector $\mathbf{x}$ is carried to $\mathbf{b}$ by the mapping $\mathbf{A}$ ? When $\det(\mathbf{A}) \neq 0$ (that is, when the mapping is invertible), the preimage exists and is unique, and the solution is $\mathbf{x} = \mathbf{A}^{-1}\mathbf{b}$ ; the two- and three-dimensional formulas for the inverse matrix are given directly by dividing the adjugate matrix by the determinant.

With this, the chapter has completed a full circuit of ideas: the rotation mapping leads naturally to matrix multiplication (§3.1) → the algebraic system of matrices establishes the one-to-one correspondence between mappings and matrices (§3.2) → the typical geometric mappings are analyzed one by one (§3.3) → the determinant characterizes invertibility quantitatively (§3.4) → solving a system returns us to the inverse problem for a mapping (§3.5). The discussion in this chapter has, however, been confined to the case $\det(\mathbf{A}) \neq 0$ ; when the determinant is zero, the system may have no solution at all or infinitely many, and that is the starting point for the systematic theory of Chapter 6.

3.6 Chapter Summary¶

Review of the Theoretical Thread¶

This chapter began from rotation in two dimensions, using the angle addition formulas to introduce tables of coefficients and matrix–vector multiplication. The composition of rotations supplies clues to the rule of matrix multiplication; requiring that all compatible linear mappings satisfy $(\mathbf{AB})\mathbf{x}=\mathbf{A}(\mathbf{B}\mathbf{x})$ is what uniquely determines standard matrix multiplication. Taking $\mathbf{x}$ to be each standard basis vector in turn determines the product matrix column by column.

§3.2 builds a complete algebraic system on this foundation. The four equivalent interpretations of matrix multiplication (inner products of rows with columns, combinations of columns, combinations of rows, and a sum of rank-one matrices) reveal the rich geometric structure hidden behind the rule of multiplication. The central result is the representation theorem for linear mappings: every linear mapping $\mathbb{R}^n \to \mathbb{R}^m$ corresponds to exactly one $m \times n$ matrix, and the column vectors of the matrix are precisely the images of the standard basis vectors. This reduces “understanding the behavior of a mapping of infinite dimension” to “reading off where finitely many column vectors go,” and is the epistemological foundation of Chapter 3.

§3.3 takes this as its tool and organizes systematically the matrix forms of the linear mappings commonly met in two and three dimensions. The zero mapping, scaling, projection ( $\mathbf{P}^2 = \mathbf{P}$ ), reflection ( $\det = -1$ , $\mathbf{R}^2 = \mathbf{I}$ ), and rotation ( $\det = 1$ , preserving distances and angles) each have their own distinct determinant signature. Rodrigues’ formula for three-dimensional rotations and the gimbal lock of the Euler angles reveal further the noncommutativity of composed rotations and the nontrivial topology of the rotation group—foreshadowings of the deeper discussions to come.

§3.4 introduces the determinant as a precise characterization of “the factor by which a mapping scales the size of space”: $|\det \mathbf{A}|$ measures the scaling of volume, the sign records a reversal of orientation, and $\det(\mathbf{A}) = 0$ is equivalent to space being compressed into a lower dimension, and hence equivalent to the mapping not being invertible. §3.5 takes this as its criterion and closes the circuit: solving the system $\mathbf{Ax} = \mathbf{b}$ is finding a preimage of the mapping $T_{\mathbf{A}}$ , and $\det(\mathbf{A}) \neq 0$ guarantees that the preimage exists and is unique, given by the formula for the inverse matrix.

Connections to Other Chapters¶

This chapter takes up the geometric understanding of vectors in $\mathbb{R}^n$ from Chapter 2—length, angle, linear dependence—and converts those geometric notions into the language of matrices. In particular, the linear combination of vectors becomes in this chapter the notion of “the space spanned by the column vectors of a matrix,” laying the linguistic groundwork for the range and the kernel to come.

This chapter prepares directly for the abstract linear spaces of Chapter 4. The principle established in §3.2 that “a linear mapping is completely determined by the images of the basis vectors” is precisely the central tool of Chapter 4—once a space is allowed to have an arbitrary basis, what a change of basis means, and how a similarity transformation may represent one and the same mapping, will both unfold naturally from the matrix representation theorem of this chapter. The geometric intuition for the determinant is only introduced in this chapter; Chapter 7 will place it on the rigorous axioms of multilinearity and antisymmetry, and will develop the theory of its computation systematically. The eigenvalue problem of Chapter 8 then asks: which basis makes the matrix representation of one and the same mapping simplest? The roots of that question lie deep in §3.2 and §3.4 of this chapter.

The Role of This Chapter in the Book¶

The identification of linear mappings with matrices is one of the most important conceptual bridges in linear algebra. It turns geometric problems into algebraic computations, and it gives algebraic computations their geometric meaning back. The central problem this chapter solves is: what is a matrix, and where does it come from?—and the answer is not “it is an array of numbers,” but “it is the coordinate representation of a linear mapping with respect to a basis.” This shift of viewpoint matters more than any computational technique.

Seen from the standpoint of nature and of engineering, almost every linear approximation (the linearization of a mechanical system, the linear filtering of a signal, the fully connected layer of a neural network) takes the matrix as its central language. Only by understanding “why matrix multiplication is defined the way it is” can one understand why the stacked matrix multiplications of deep learning are, geometrically, the same thing as the composition of a series of transformations of space. By the end of this chapter, the matrix you see is no longer merely an array of numbers waiting to be computed with, but a geometric operation on space—one that distorts, rotates, projects, or compresses it—and this point of view will be the most important background for reading every chapter that follows.

Chapter Structure and Learning Objectives¶

3.1 From Rotation Mappings to the Notion of a Matrix¶

3.1.1 Introducing the Matrix through Rotation Mappings¶

3.1.2 A Natural Route to Matrix Multiplication¶

3.1.3 Verifying Rotation Matrix Multiplication in Python¶

3.2 Introduction to Matrices¶

3.2.1 The Definition of a Matrix and Its Basic Operations¶

3.2.2 The Two-Way Correspondence between Linear Mappings and Matrices¶

3.2.3 Basic Matrix Operations in Python¶

3.2.4 ◆Higher-Order Matrix Operations with einsum in Python¶

3.3 Typical Linear Mappings in Two and Three Dimensions¶

3.3.1 The Zero Mapping¶

3.3.2 The Identity Mapping¶

3.3.3 The Scaling Mapping¶

3.3.4 The Projection Matrix¶

Analysis of the Action of a Projection Matrix on the Standard Basis¶

3.3.5 The Reflection Matrix¶

Analysis of the Action of a Reflection Matrix on the Standard Basis¶

3.3.6 The Rotation Mapping¶

Three-Dimensional Rotation Matrices¶

3.3.7 Verifying the Gimbal Lock of the Euler Angles in Python¶

3.4 Determinants and Mappings of Space¶

3.4.1 The Column Vectors of a Square Matrix Form an Area¶

3.4.2 The Column Vectors of a Square Matrix Form a Volume¶

3.4.3 Determinants of Some Special Matrices¶

3.4.4 Checking the Formula for the Determinant in Python¶

3.4.5 Computing Determinants and Verifying Their Geometric Meaning in Python¶

3.5 Linear Systems and Linear Mappings¶

3.5.1 Understanding Linear Systems Geometrically¶

The Need for an Inverse Mapping, and the Notion of an Inverse Matrix¶

3.5.2 The Geometric Meaning of a Two-Dimensional Linear System¶

3.5.3 The Inverse Matrix and the Solution of a System¶

3.5.4 Generalization to Three Dimensions¶

3.5.5 Solving Linear Systems in Python¶

3.6 Chapter Summary¶

Review of the Theoretical Thread¶

Connections to Other Chapters¶

The Role of This Chapter in the Book¶

Concept Map¶