The first 67 exercises are compiled from the original manuscript of Chapter 9 exercises, in the same order as in the manuscript, and are divided by topic into nine groups. Every exercise comes with a hint and a complete solution that can be expanded; computational exercises are followed by a NumPy verification, while pure proof exercises have no accompanying program. The last exercise, on the Jacobi method, is an extension exercise.
This part uses 0-based indexing. Unless stated otherwise, the inner product on is the standard inner product . Unifying the terminology of the manuscript and changing its subscripts to 0-based indexing are not noted exercise by exercise; every change that alters the meaning of an exercise (adding or removing conditions, correcting data or notation, supplying definitions), as well as errors and ambiguities in the original manuscript, is recorded in the “Note on the original manuscript” of the corresponding exercise.
import numpy as np
np.set_printoptions(precision=4, suppress=True, linewidth=100)
def print_header(title):
print("=" * 60)
print(f" {title}")
print("=" * 60)
def print_step(step, desc):
print(f"\n▶ Step {step}: {desc}")
print("-" * 40)
def compare_print(label, actual, expected_desc):
print(f"\n[{label}]")
print(f" Computed value: {actual}")
print(f" Expected value: {expected_desc}")
def check(label, actual, expected):
compare_print(label, actual, expected)
assert np.allclose(actual, expected), label
print(" check passed")Definition and Examples of Inner Products¶
# ============================================================
print_header("Exercise 9-1 | Axioms of a Real Inner Product")
# ============================================================
def ip_01(x, y, k):
"""<x, y>_k = x^T G_k y"""
G = np.array([[k, -3.0], [-3.0, 1.0]])
return np.asarray(x) @ G @ np.asarray(y)
# --- Step 1 ---
print_step(1, "Matrix form agrees with the expansion; (i) symmetry, (ii) linearity in the second argument")
rng = np.random.default_rng(1)
k = 2.7 # an arbitrary real number k
x, y, u, v = rng.standard_normal((4, 2))
a, b = 1.3, -0.4
expand = k*x[0]*y[0] - 3*x[0]*y[1] - 3*x[1]*y[0] + x[1]*y[1]
check("x^T G_k y vs. the expansion", ip_01(x, y, k), expand)
check("(i) symmetry", ip_01(x, y, k), ip_01(y, x, k))
check("(ii) linearity in the second argument", ip_01(x, a*u + b*v, k), a*ip_01(x, u, k) + b*ip_01(x, v, k))
# --- Step 2 ---
print_step(2, "Range for (iii): k > 9")
x13 = np.array([1.0, 3.0])
check("<(1,3),(1,3)> for k = 9", ip_01(x13, x13, 9), 0.0)
check("<(1,3),(1,3)> for k = 8 (negative)", ip_01(x13, x13, 8), -1.0)
for kk in (9.5, 10, 20):
lam = np.linalg.eigvalsh(np.array([[kk, -3.0], [-3.0, 1.0]]))
check(f"k = {kk}: smallest eigenvalue of G_k > 0", lam.min() > 0, True)
for kk in (9, 5, -1):
lam = np.linalg.eigvalsh(np.array([[kk, -3.0], [-3.0, 1.0]]))
check(f"k = {kk}: G_k is not positive definite", lam.min() > 1e-12, False)
# --- Step 3 ---
print_step(3, "k = 10: length and perpendicular direction")
u0 = np.array([1.0, 0.0])
w = np.array([3.0, 10.0])
check("||u||", np.sqrt(ip_01(u0, u0, 10)), np.sqrt(10))
check("<u, (3,10)>_10", ip_01(u0, w, 10), 0.0)
check("||(3,10)||", np.sqrt(ip_01(w, w, 10)), np.sqrt(10))
check("<u, (0,1)>_10 (the perpendicular direction under the standard inner product)", ip_01(u0, [0.0, 1.0], 10), -3.0)============================================================
Exercise 9-1 | Axioms of a Real Inner Product
============================================================
▶ Step 1: Matrix form agrees with the expansion; (i) symmetry, (ii) linearity in the second argument
----------------------------------------
[x^T G_k y vs. the expansion]
Computed value: -0.22580195849905899
Expected value: -0.22580195849905882
check passed
[(i) symmetry]
Computed value: -0.22580195849905899
Expected value: -0.22580195849905893
check passed
[(ii) linearity in the second argument]
Computed value: -2.2066737806193535
Expected value: -2.2066737806193535
check passed
▶ Step 2: Range for (iii): k > 9
----------------------------------------
[<(1,3),(1,3)> for k = 9]
Computed value: 0.0
Expected value: 0.0
check passed
[<(1,3),(1,3)> for k = 8 (negative)]
Computed value: -1.0
Expected value: -1.0
check passed
[k = 9.5: smallest eigenvalue of G_k > 0]
Computed value: True
Expected value: True
check passed
[k = 10: smallest eigenvalue of G_k > 0]
Computed value: True
Expected value: True
check passed
[k = 20: smallest eigenvalue of G_k > 0]
Computed value: True
Expected value: True
check passed
[k = 9: G_k is not positive definite]
Computed value: False
Expected value: False
check passed
[k = 5: G_k is not positive definite]
Computed value: False
Expected value: False
check passed
[k = -1: G_k is not positive definite]
Computed value: False
Expected value: False
check passed
▶ Step 3: k = 10: length and perpendicular direction
----------------------------------------
[||u||]
Computed value: 3.1622776601683795
Expected value: 3.1622776601683795
check passed
[<u, (3,10)>_10]
Computed value: 0.0
Expected value: 0.0
check passed
[||(3,10)||]
Computed value: 3.1622776601683795
Expected value: 3.1622776601683795
check passed
[<u, (0,1)>_10 (the perpendicular direction under the standard inner product)]
Computed value: -3.0
Expected value: -3.0
check passed
# ============================================================
print_header("Exercise 9-2 | An Integral Inner Product on Polynomials")
# ============================================================
# --- Step 1 ---
print_step(1, "Gram matrix of the basis {1, x, x^2}: H[i, j] = ∫_0^1 x^(i+j) dx = 1/(i+j+1)")
H = np.array([[1.0/(i + j + 1) for j in range(3)] for i in range(3)])
print(H)
check("H is positive definite (smallest eigenvalue > 0)", np.linalg.eigvalsh(H).min() > 0, True)
def ip_02(p, q):
"""p, q are coefficient vectors (a0, a1, a2); returns ∫_0^1 p(x) q(x) dx"""
return np.asarray(p, float) @ H @ np.asarray(q, float)
# independent check of ip_02 by numerical integration (trapezoidal rule)
xs = np.linspace(0, 1, 200001)
def ip_num_02(p, q):
fx = np.polyval(p[::-1], xs) * np.polyval(q[::-1], xs)
return np.sum((fx[1:] + fx[:-1]) / 2) * (xs[1] - xs[0])
rng = np.random.default_rng(2)
p_r, q_r = rng.standard_normal((2, 3))
check("coefficient form = numerical integration", ip_02(p_r, q_r), ip_num_02(p_r, q_r))
# --- Step 2 ---
print_step(2, "<x, 3x-2> and lengths")
one, x1, x2 = [1, 0, 0], [0, 1, 0], [0, 0, 1]
check("<x, 3x-2>", ip_02(x1, [-2, 3, 0]), 0.0)
check("(||1||, ||x||, ||x^2||)",
[np.sqrt(ip_02(e, e)) for e in (one, x1, x2)], [1, 1/np.sqrt(3), 1/np.sqrt(5)])
# --- Step 3 ---
print_step(3, "Angle between 1 and x")
cos_t = ip_02(one, x1) / np.sqrt(ip_02(one, one) * ip_02(x1, x1))
check("cos θ", cos_t, np.sqrt(3)/2)
check("θ", np.arccos(cos_t), np.pi/6)
# --- Step 4 ---
print_step(4, "Subspace orthogonal to x")
row = H[1] # <x, p> = H[1] @ (a0, a1, a2)
check("coefficients of <x, p> ×12", 12*row, [6, 4, 3])
basis = np.array([[-2, 3, 0], [-1, 0, 2]], float)
check("<x, 3x-2>, <x, 2x^2-1>", [ip_02(x1, b) for b in basis], [0, 0])
check("the two basis polynomials are linearly independent", np.linalg.matrix_rank(basis), 2)
check("dimension of the orthogonal complement = 3 - rank", 3 - np.linalg.matrix_rank(row[None, :]), 2)
# --- Step 5 ---
print_step(5, "Cauchy-Schwarz inequality")
for _ in range(5):
f, g = rng.standard_normal((2, 3))
check("|<f,g>| <= ||f|| ||g||",
abs(ip_02(f, g)) <= np.sqrt(ip_02(f, f) * ip_02(g, g)) + 1e-12, True)
f = rng.standard_normal(3)
check("equality for f and 2f", abs(ip_02(f, 2*f)), np.sqrt(ip_02(f, f) * ip_02(2*f, 2*f)))============================================================
Exercise 9-2 | An Integral Inner Product on Polynomials
============================================================
▶ Step 1: Gram matrix of the basis {1, x, x^2}: H[i, j] = ∫_0^1 x^(i+j) dx = 1/(i+j+1)
----------------------------------------
[[1. 0.5 0.3333]
[0.5 0.3333 0.25 ]
[0.3333 0.25 0.2 ]]
[H is positive definite (smallest eigenvalue > 0)]
Computed value: True
Expected value: True
check passed
[coefficient form = numerical integration]
Computed value: 0.011455724284581886
Expected value: 0.011455724273442361
check passed
▶ Step 2: <x, 3x-2> and lengths
----------------------------------------
[<x, 3x-2>]
Computed value: 0.0
Expected value: 0.0
check passed
[(||1||, ||x||, ||x^2||)]
Computed value: [np.float64(1.0), np.float64(0.5773502691896257), np.float64(0.4472135954999579)]
Expected value: [1, np.float64(0.5773502691896258), np.float64(0.4472135954999579)]
check passed
▶ Step 3: Angle between 1 and x
----------------------------------------
[cos θ]
Computed value: 0.8660254037844387
Expected value: 0.8660254037844386
check passed
[θ]
Computed value: 0.5235987755982987
Expected value: 0.5235987755982988
check passed
▶ Step 4: Subspace orthogonal to x
----------------------------------------
[coefficients of <x, p> ×12]
Computed value: [6. 4. 3.]
Expected value: [6, 4, 3]
check passed
[<x, 3x-2>, <x, 2x^2-1>]
Computed value: [np.float64(0.0), np.float64(0.0)]
Expected value: [0, 0]
check passed
[the two basis polynomials are linearly independent]
Computed value: 2
Expected value: 2
check passed
[dimension of the orthogonal complement = 3 - rank]
Computed value: 2
Expected value: 2
check passed
▶ Step 5: Cauchy-Schwarz inequality
----------------------------------------
[|<f,g>| <= ||f|| ||g||]
Computed value: True
Expected value: True
check passed
[|<f,g>| <= ||f|| ||g||]
Computed value: True
Expected value: True
check passed
[|<f,g>| <= ||f|| ||g||]
Computed value: True
Expected value: True
check passed
[|<f,g>| <= ||f|| ||g||]
Computed value: True
Expected value: True
check passed
[|<f,g>| <= ||f|| ||g||]
Computed value: True
Expected value: True
check passed
[equality for f and 2f]
Computed value: 0.05069029428408314
Expected value: 0.05069029428408314
check passed
# ============================================================
print_header("Exercise 9-3 | Frobenius Inner Product")
# ============================================================
def frob_03(A, B):
"""<A, B> = tr(A^T B)"""
return np.trace(np.asarray(A).T @ np.asarray(B))
# --- Step 1 ---
print_step(1, "tr(A^T B) = Σ a_ij b_ij, and the inner product axioms")
rng = np.random.default_rng(3)
X, Y, Z = rng.standard_normal((3, 2, 2))
a, b = 0.7, -1.9
check("tr(X^T Y) = Σ x_ij y_ij", frob_03(X, Y), np.sum(X * Y))
check("symmetry", frob_03(X, Y), frob_03(Y, X))
check("linearity in the second argument", frob_03(X, a*Y + b*Z), a*frob_03(X, Y) + b*frob_03(X, Z))
check("<X, X> = Σ x_ij^2 > 0", frob_03(X, X) > 0, True)
# --- Step 2 ---
print_step(2, "Inner product and norms of the given matrices")
A = np.array([[1, 2], [0, 3]])
B = np.array([[2, 0], [1, 1]])
check("A^T B", A.T @ B, [[2, 0], [7, 3]])
check("<A, B>", frob_03(A, B), 5)
check("(||A||, ||B||)", [np.sqrt(frob_03(A, A)), np.sqrt(frob_03(B, B))], [np.sqrt(14), np.sqrt(6)])
check("agrees with np.linalg.norm(·, 'fro')", np.linalg.norm(A, 'fro'), np.sqrt(14))
# --- Step 3 ---
print_step(3, "Column-wise vec(A) = (a, b, c, d): Frobenius inner product = standard inner product on R^4")
vecA = A.flatten(order='F') # stack column 0, then column 1
check("vec(A)", vecA, [1, 0, 2, 3])
check("<A, B> = vec(A)^T vec(B)", frob_03(A, B), vecA @ B.flatten(order='F'))
# --- Step 4 ---
print_step(4, "Symmetric and skew-symmetric matrices are orthogonal")
for n in (2, 4):
M = rng.standard_normal((n, n))
S = (M + M.T) / 2
K = (M - M.T) / 2
check(f"n = {n}: <S, K>", frob_03(S, K), 0.0)
check(f"n = {n}: S + K = M (orthogonal decomposition)", S + K, M)============================================================
Exercise 9-3 | Frobenius Inner Product
============================================================
▶ Step 1: tr(A^T B) = Σ a_ij b_ij, and the inner product axioms
----------------------------------------
[tr(X^T Y) = Σ x_ij y_ij]
Computed value: -1.0856961812858006
Expected value: -1.0856961812858006
check passed
[symmetry]
Computed value: -1.0856961812858006
Expected value: -1.0856961812858006
check passed
[linearity in the second argument]
Computed value: 18.171022543353594
Expected value: 18.17102254335359
check passed
[<X, X> = Σ x_ij^2 > 0]
Computed value: True
Expected value: True
check passed
▶ Step 2: Inner product and norms of the given matrices
----------------------------------------
[A^T B]
Computed value: [[2 0]
[7 3]]
Expected value: [[2, 0], [7, 3]]
check passed
[<A, B>]
Computed value: 5
Expected value: 5
check passed
[(||A||, ||B||)]
Computed value: [np.float64(3.7416573867739413), np.float64(2.449489742783178)]
Expected value: [np.float64(3.7416573867739413), np.float64(2.449489742783178)]
check passed
[agrees with np.linalg.norm(·, 'fro')]
Computed value: 3.7416573867739413
Expected value: 3.7416573867739413
check passed
▶ Step 3: Column-wise vec(A) = (a, b, c, d): Frobenius inner product = standard inner product on R^4
----------------------------------------
[vec(A)]
Computed value: [1 0 2 3]
Expected value: [1, 0, 2, 3]
check passed
[<A, B> = vec(A)^T vec(B)]
Computed value: 5
Expected value: 5
check passed
▶ Step 4: Symmetric and skew-symmetric matrices are orthogonal
----------------------------------------
[n = 2: <S, K>]
Computed value: 0.0
Expected value: 0.0
check passed
[n = 2: S + K = M (orthogonal decomposition)]
Computed value: [[-0.2813 -0.668 ]
[-1.0552 -0.3908]]
Expected value: [[-0.2813 -0.668 ]
[-1.0552 -0.3908]]
check passed
[n = 4: <S, K>]
Computed value: 0.0
Expected value: 0.0
check passed
[n = 4: S + K = M (orthogonal decomposition)]
Computed value: [[ 0.4819 -0.2386 0.9578 -0.1998]
[ 0.0243 1.5458 0.5451 -0.5052]
[-0.1828 0.5405 1.9351 -0.2696]
[-0.2436 1.0023 -0.8865 -0.2917]]
Expected value: [[ 0.4819 -0.2386 0.9578 -0.1998]
[ 0.0243 1.5458 0.5451 -0.5052]
[-0.1828 0.5405 1.9351 -0.2696]
[-0.2436 1.0023 -0.8865 -0.2917]]
check passed
# ============================================================
print_header("Exercise 9-4 | The Matrix of a Bilinear Form")
# ============================================================
# --- Step 1 ---
print_step(1, "Recover the matrix A from a_ij = <e_i, e_j>")
def form_04(x, y):
return x[0]*y[0] - x[0]*y[1] - x[1]*y[0] + 4*x[1]*y[1]
E = np.eye(2)
A = np.array([[form_04(E[i], E[j]) for j in range(2)] for i in range(2)])
check("A", A, [[1, -1], [-1, 4]])
check("A is symmetric", A, A.T)
rng = np.random.default_rng(4)
x, y = rng.standard_normal((2, 2))
check("x^T A y = original expression", x @ A @ y, form_04(x, y))
# --- Step 2 ---
print_step(2, "The matrix is not unique given only the quadratic form; A is positive definite")
A2 = np.array([[1, -2], [0, 4]])
check("x^T A2 x = x^T A x", x @ A2 @ x, x @ A @ x)
check("x^T A2 y ≠ x^T A y (different bilinear forms)", np.isclose(x @ A2 @ y, x @ A @ y), False)
check("all eigenvalues of A are positive", np.linalg.eigvalsh(A).min() > 0, True)
# --- Step 3 ---
print_step(3, "Order n: a_ij = e_i^T A e_j; spot-check ⇒ via the contrapositive and ⇐ with random vectors")
n = 4
En = np.eye(n)
M = rng.standard_normal((n, n))
check("[e_i^T M e_j] recovers M", [[En[i] @ M @ En[j] for j in range(n)] for i in range(n)], M)
# ⇒ (contrapositive): when M is not symmetric, x = e_0, y = e_1 already break the symmetry of the form
check("<e_0, e_1> ≠ <e_1, e_0> when M is not symmetric", np.isclose(En[0] @ M @ En[1], En[1] @ M @ En[0]), False)
# ⇐: when S is symmetric, the form is symmetric
S = (M + M.T) / 2
u, v = rng.standard_normal((2, n))
check("u^T S v = v^T S u when S is symmetric", u @ S @ v, v @ S @ u)
# --- Step 4 ---
print_step(4, "A = [[2,3],[3,4]] does not give an inner product")
C = np.array([[2, 3], [3, 4]])
check("<(1,-1),(1,-1)>", np.array([1, -1]) @ C @ np.array([1, -1]), 0)
check("<(3,-2),(3,-2)>", np.array([3, -2]) @ C @ np.array([3, -2]), -2)
check("det", np.linalg.det(C), -1)
check("eigenvalues 3 ± √10", np.linalg.eigvalsh(C), [3 - np.sqrt(10), 3 + np.sqrt(10)])============================================================
Exercise 9-4 | The Matrix of a Bilinear Form
============================================================
▶ Step 1: Recover the matrix A from a_ij = <e_i, e_j>
----------------------------------------
[A]
Computed value: [[ 1. -1.]
[-1. 4.]]
Expected value: [[1, -1], [-1, 4]]
check passed
[A is symmetric]
Computed value: [[ 1. -1.]
[-1. 4.]]
Expected value: [[ 1. -1.]
[-1. 4.]]
check passed
[x^T A y = original expression]
Computed value: -0.8247505957919986
Expected value: -0.8247505957919985
check passed
▶ Step 2: The matrix is not unique given only the quadratic form; A is positive definite
----------------------------------------
[x^T A2 x = x^T A x]
Computed value: 0.31917786488105554
Expected value: 0.31917786488105554
check passed
[x^T A2 y ≠ x^T A y (different bilinear forms)]
Computed value: False
Expected value: False
check passed
[all eigenvalues of A are positive]
Computed value: True
Expected value: True
check passed
▶ Step 3: Order n: a_ij = e_i^T A e_j; spot-check ⇒ via the contrapositive and ⇐ with random vectors
----------------------------------------
[[e_i^T M e_j] recovers M]
Computed value: [[np.float64(-1.6413972945846467), np.float64(-0.005203264171931977), np.float64(-0.6234637409883934), np.float64(0.14863152325202633)], [np.float64(-1.608187784186389), np.float64(0.2417718768768513), np.float64(0.23538091873745476), np.float64(1.5756260314314627)], [np.float64(0.3166450164719021), np.float64(0.5105466616976417), np.float64(-1.4931166849642326), np.float64(2.2527291247240275)], [np.float64(-1.9156455579583005), np.float64(1.1018018558224842), np.float64(-0.3299040738738497), np.float64(-0.8806465179277969)]]
Expected value: [[-1.6414 -0.0052 -0.6235 0.1486]
[-1.6082 0.2418 0.2354 1.5756]
[ 0.3166 0.5105 -1.4931 2.2527]
[-1.9156 1.1018 -0.3299 -0.8806]]
check passed
[<e_0, e_1> ≠ <e_1, e_0> when M is not symmetric]
Computed value: False
Expected value: False
check passed
[u^T S v = v^T S u when S is symmetric]
Computed value: 1.6725345994053922
Expected value: 1.6725345994053922
check passed
▶ Step 4: A = [[2,3],[3,4]] does not give an inner product
----------------------------------------
[<(1,-1),(1,-1)>]
Computed value: 0
Expected value: 0
check passed
[<(3,-2),(3,-2)>]
Computed value: -2
Expected value: -2
check passed
[det]
Computed value: -1.0000000000000004
Expected value: -1
check passed
[eigenvalues 3 ± √10]
Computed value: [-0.1623 6.1623]
Expected value: [np.float64(-0.16227766016837952), np.float64(6.16227766016838)]
check passed
# ============================================================
print_header("Exercise 9-5 | Change of Basis and Inner Products")
# ============================================================
# --- Step 1 ---
print_step(1, "2-dimensional case: M = B^T B")
B = np.array([[1, -1], [2, -1]], float) # column 0 is v_0, column 1 is v_1
M = B.T @ B
check("M", M, [[5, -3], [-3, 2]])
check("M is symmetric", M, M.T)
check("diagonal = (||v_0||^2, ||v_1||^2)", np.diag(M), [B[:, 0] @ B[:, 0], B[:, 1] @ B[:, 1]])
check("cos θ(v_0, v_1)", M[0, 1] / np.sqrt(M[0, 0] * M[1, 1]), -3/np.sqrt(10))
# --- Step 2 ---
print_step(2, "Inner product computed in the new coordinates = standard inner product")
rng = np.random.default_rng(5)
x, y = rng.standard_normal((2, 2))
xb = np.linalg.solve(B, x) # [x]_B
yb = np.linalg.solve(B, y)
check("[x]_B^T M [y]_B = x^T y", xb @ M @ yb, x @ y)
expand = 5*xb[0]*yb[0] - 3*xb[0]*yb[1] - 3*xb[1]*yb[0] + 2*xb[1]*yb[1]
check("expanded form", expand, x @ y)
# --- Step 3 ---
print_step(3, "n-dimensional case: M_ij = <f_i, f_j>, and M is positive definite")
n = 4
Bn = rng.standard_normal((n, n))
check("random B is invertible", np.linalg.matrix_rank(Bn), n)
Mn = Bn.T @ Bn
gram = np.array([[Bn[:, i] @ Bn[:, j] for j in range(n)] for i in range(n)])
check("B^T B = [<f_i, f_j>]", Mn, gram)
u, v = rng.standard_normal((2, n))
ub, vb = np.linalg.solve(Bn, u), np.linalg.solve(Bn, v)
check("[u]_B^T M [v]_B = u^T v", ub @ Mn @ vb, u @ v)
check("M is positive definite", np.linalg.eigvalsh(Mn).min() > 0, True)============================================================
Exercise 9-5 | Change of Basis and Inner Products
============================================================
▶ Step 1: 2-dimensional case: M = B^T B
----------------------------------------
[M]
Computed value: [[ 5. -3.]
[-3. 2.]]
Expected value: [[5, -3], [-3, 2]]
check passed
[M is symmetric]
Computed value: [[ 5. -3.]
[-3. 2.]]
Expected value: [[ 5. -3.]
[-3. 2.]]
check passed
[diagonal = (||v_0||^2, ||v_1||^2)]
Computed value: [5. 2.]
Expected value: [np.float64(5.0), np.float64(2.0)]
check passed
[cos θ(v_0, v_1)]
Computed value: -0.9486832980505138
Expected value: -0.9486832980505138
check passed
▶ Step 2: Inner product computed in the new coordinates = standard inner product
----------------------------------------
[[x]_B^T M [y]_B = x^T y]
Computed value: -0.35765144361598944
Expected value: -0.35765144361598955
check passed
[expanded form]
Computed value: -0.35765144361598955
Expected value: -0.35765144361598955
check passed
▶ Step 3: n-dimensional case: M_ij = <f_i, f_j>, and M is positive definite
----------------------------------------
[random B is invertible]
Computed value: 4
Expected value: 4
check passed
[B^T B = [<f_i, f_j>]]
Computed value: [[ 2.7765 -0.0872 -0.5653 -0.1143]
[-0.0872 6.5977 1.4419 -4.3061]
[-0.5653 1.4419 0.817 0.053 ]
[-0.1143 -4.3061 0.053 5.3754]]
Expected value: [[ 2.7765 -0.0872 -0.5653 -0.1143]
[-0.0872 6.5977 1.4419 -4.3061]
[-0.5653 1.4419 0.817 0.053 ]
[-0.1143 -4.3061 0.053 5.3754]]
check passed
[[u]_B^T M [v]_B = u^T v]
Computed value: 0.42200040886616597
Expected value: 0.4220004088661687
check passed
[M is positive definite]
Computed value: True
Expected value: True
check passed
Orthogonal Projection and the Method of Least Squares¶
# ============================================================
print_header("Exercise 9-7 | Expansion in an Orthogonal Basis")
# ============================================================
U = np.array([[1, 0, 1],
[-1, 4, 1],
[2, 1, -2]], dtype=float) # row i is u_i
u0, u1, u2 = U
w = np.array([1, 2, 3], dtype=float)
v = np.array([5, 6, 1], dtype=float)
# --- Step 1 ---
print_step(1, "Pairwise inner products and the basis test")
check("<u0,u1>, <u0,u2>, <u1,u2>", [u0 @ u1, u0 @ u2, u1 @ u2], [0, 0, 0])
check("rank([u0 u1 u2])", np.linalg.matrix_rank(U), 3)
# --- Step 2 ---
print_step(2, "Cosine of the angle between u0 and w")
cos_theta = (u0 @ w) / (np.linalg.norm(u0) * np.linalg.norm(w))
check("cos θ", cos_theta, 2 * np.sqrt(7) / 7)
# --- Step 3 ---
print_step(3, "Projection coefficients c_i = <u_i, v> / <u_i, u_i>")
c = np.array([(ui @ v) / (ui @ ui) for ui in U])
check("(c0, c1, c2)", c, [3, 10/9, 14/9])
check("c0 u0 + c1 u1 + c2 u2", c @ U, v)============================================================
Exercise 9-7 | Expansion in an Orthogonal Basis
============================================================
▶ Step 1: Pairwise inner products and the basis test
----------------------------------------
[<u0,u1>, <u0,u2>, <u1,u2>]
Computed value: [np.float64(0.0), np.float64(0.0), np.float64(0.0)]
Expected value: [0, 0, 0]
check passed
[rank([u0 u1 u2])]
Computed value: 3
Expected value: 3
check passed
▶ Step 2: Cosine of the angle between u0 and w
----------------------------------------
[cos θ]
Computed value: 0.7559289460184544
Expected value: 0.7559289460184545
check passed
▶ Step 3: Projection coefficients c_i = <u_i, v> / <u_i, u_i>
----------------------------------------
[(c0, c1, c2)]
Computed value: [3. 1.1111 1.5556]
Expected value: [3, 1.1111111111111112, 1.5555555555555556]
check passed
[c0 u0 + c1 u1 + c2 u2]
Computed value: [5. 6. 1.]
Expected value: [5. 6. 1.]
check passed
# ============================================================
print_header("Exercise 9-8 | Projecting a Vector onto a Plane")
# ============================================================
w0 = np.array([1, 1, 0, 2], dtype=float)
w1 = np.array([1, 0, 2, -1], dtype=float)
u = np.array([-1, 0, 0, 7], dtype=float)
B = np.column_stack([w0, w1]) # 4×2; columns 0 and 1 are w0 and w1
# --- Step 1 ---
print_step(1, "Normal equations: coefficient matrix B^T B and right-hand side B^T u")
check("B^T B", B.T @ B, [[6, -1], [-1, 6]])
check("B^T u", B.T @ u, [13, -8])
# --- Step 2 ---
print_step(2, "Solve for (α, β)")
alpha_beta = np.linalg.solve(B.T @ B, B.T @ u)
check("(α, β)", alpha_beta, [2, -1])
# --- Step 3 ---
print_step(3, "Projection vector and check of perpendicularity")
u_p = B @ alpha_beta
check("u_p", u_p, [1, 2, -2, 5])
check("B^T (u - u_p)", B.T @ (u - u_p), [0, 0])============================================================
Exercise 9-8 | Projecting a Vector onto a Plane
============================================================
▶ Step 1: Normal equations: coefficient matrix B^T B and right-hand side B^T u
----------------------------------------
[B^T B]
Computed value: [[ 6. -1.]
[-1. 6.]]
Expected value: [[6, -1], [-1, 6]]
check passed
[B^T u]
Computed value: [13. -8.]
Expected value: [13, -8]
check passed
▶ Step 2: Solve for (α, β)
----------------------------------------
[(α, β)]
Computed value: [ 2. -1.]
Expected value: [2, -1]
check passed
▶ Step 3: Projection vector and check of perpendicularity
----------------------------------------
[u_p]
Computed value: [ 1. 2. -2. 5.]
Expected value: [1, 2, -2, 5]
check passed
[B^T (u - u_p)]
Computed value: [0. 0.]
Expected value: [0, 0]
check passed
# ============================================================
print_header("Exercise 9-12 | Best-Fit Line")
# ============================================================
def fit_line_by_means_12(a, b):
"""compute (m, k) with the averages formula of part 3"""
a_bar, b_bar = a.mean(), b.mean()
ab_bar, a2_bar = (a * b).mean(), (a ** 2).mean()
D = a2_bar - a_bar ** 2
return np.array([(ab_bar - a_bar * b_bar) / D,
(a2_bar * b_bar - ab_bar * a_bar) / D])
# --- Step 1 ---
print_step(1, "Part 2: build A = (a 1) and the normal equations")
a = np.array([1, 2, 2, 3], dtype=float)
b = np.array([1, 1, 2, 2], dtype=float)
A = np.column_stack([a, np.ones_like(a)])
check("A^T A", A.T @ A, [[18, 8], [8, 4]])
check("A^T b", A.T @ b, [13, 6])
# --- Step 2 ---
print_step(2, "Solve the normal equations for (m, k)")
mk = np.linalg.solve(A.T @ A, A.T @ b)
check("(m, k)", mk, [0.5, 0.5])
check("agrees with np.linalg.lstsq", np.linalg.lstsq(A, b, rcond=None)[0], mk)
# --- Step 3 ---
print_step(3, "Residual perpendicular to Col(A); minimum sum of squared errors")
r = b - A @ mk
check("residual r", r, [0, -0.5, 0.5, 0])
check("A^T r", A.T @ r, [0, 0])
check("‖r‖²", r @ r, 0.5)
# --- Step 4 ---
print_step(4, "Parts 3 and 4: averages formula and passing through the center (including random data)")
check("averages formula (data of part 2)", fit_line_by_means_12(a, b), [0.5, 0.5])
check("m·ā + k = b̄ (data of part 2)", mk[0] * a.mean() + mk[1], b.mean())
rng = np.random.default_rng(12)
a_r = rng.normal(size=20)
b_r = 3 * a_r - 1 + rng.normal(scale=0.5, size=20)
A_r = np.column_stack([a_r, np.ones_like(a_r)])
mk_r = np.linalg.lstsq(A_r, b_r, rcond=None)[0]
check("averages formula = lstsq (random data)", fit_line_by_means_12(a_r, b_r), mk_r)
check("m·ā + k = b̄ (random data)", mk_r[0] * a_r.mean() + mk_r[1], b_r.mean())============================================================
Exercise 9-12 | Best-Fit Line
============================================================
▶ Step 1: Part 2: build A = (a 1) and the normal equations
----------------------------------------
[A^T A]
Computed value: [[18. 8.]
[ 8. 4.]]
Expected value: [[18, 8], [8, 4]]
check passed
[A^T b]
Computed value: [13. 6.]
Expected value: [13, 6]
check passed
▶ Step 2: Solve the normal equations for (m, k)
----------------------------------------
[(m, k)]
Computed value: [0.5 0.5]
Expected value: [0.5, 0.5]
check passed
[agrees with np.linalg.lstsq]
Computed value: [0.5 0.5]
Expected value: [0.5 0.5]
check passed
▶ Step 3: Residual perpendicular to Col(A); minimum sum of squared errors
----------------------------------------
[residual r]
Computed value: [-0. -0.5 0.5 0. ]
Expected value: [0, -0.5, 0.5, 0]
check passed
[A^T r]
Computed value: [-0. -0.]
Expected value: [0, 0]
check passed
[‖r‖²]
Computed value: 0.5
Expected value: 0.5
check passed
▶ Step 4: Parts 3 and 4: averages formula and passing through the center (including random data)
----------------------------------------
[averages formula (data of part 2)]
Computed value: [0.5 0.5]
Expected value: [0.5, 0.5]
check passed
[m·ā + k = b̄ (data of part 2)]
Computed value: 1.5
Expected value: 1.5
check passed
[averages formula = lstsq (random data)]
Computed value: [ 3.0027 -1.097 ]
Expected value: [ 3.0027 -1.097 ]
check passed
[m·ā + k = b̄ (random data)]
Computed value: -0.5217863913811589
Expected value: -0.5217863913811593
check passed
# ============================================================
print_header("Exercise 9-13 | Best-Fit Plane")
# ============================================================
pts = np.array([[0, 0, 0],
[1, 0, 1],
[0, 1, 0],
[-1, 0, -1],
[0, -1, -1]], dtype=float) # row i is (x_i, y_i, z_i)
x, y, z = pts[:, 0], pts[:, 1], pts[:, 2]
A = np.column_stack([x, y, np.ones(5)]) # 5×3: the columns are x, y, 1, in order
# --- Step 1 ---
print_step(1, "Normal equations: A^T A and A^T z")
check("A^T A", A.T @ A, np.diag([2, 2, 5]))
check("A^T z", A.T @ z, [2, 1, -1])
# --- Step 2 ---
print_step(2, "Solve for (m, n, k)")
mnk = np.linalg.solve(A.T @ A, A.T @ z)
check("(m, n, k)", mnk, [1, 1/2, -1/5])
check("agrees with np.linalg.lstsq", np.linalg.lstsq(A, z, rcond=None)[0], mnk)
# --- Step 3 ---
print_step(3, "Residual perpendicular to Col(A), minimum sum of squared errors, passing through the center")
r = z - A @ mnk
check("residual r", r, [1/5, 1/5, -3/10, 1/5, -3/10])
check("A^T r", A.T @ r, [0, 0, 0])
check("‖r‖²", r @ r, 3/10)
check("m·x̄ + n·ȳ + k = z̄", mnk @ [x.mean(), y.mean(), 1], z.mean())============================================================
Exercise 9-13 | Best-Fit Plane
============================================================
▶ Step 1: Normal equations: A^T A and A^T z
----------------------------------------
[A^T A]
Computed value: [[2. 0. 0.]
[0. 2. 0.]
[0. 0. 5.]]
Expected value: [[2 0 0]
[0 2 0]
[0 0 5]]
check passed
[A^T z]
Computed value: [ 2. 1. -1.]
Expected value: [2, 1, -1]
check passed
▶ Step 2: Solve for (m, n, k)
----------------------------------------
[(m, n, k)]
Computed value: [ 1. 0.5 -0.2]
Expected value: [1, 0.5, -0.2]
check passed
[agrees with np.linalg.lstsq]
Computed value: [ 1. 0.5 -0.2]
Expected value: [ 1. 0.5 -0.2]
check passed
▶ Step 3: Residual perpendicular to Col(A), minimum sum of squared errors, passing through the center
----------------------------------------
[residual r]
Computed value: [ 0.2 0.2 -0.3 0.2 -0.3]
Expected value: [0.2, 0.2, -0.3, 0.2, -0.3]
check passed
[A^T r]
Computed value: [ 0. 0. -0.]
Expected value: [0, 0, 0]
check passed
[‖r‖²]
Computed value: 0.3
Expected value: 0.3
check passed
[m·x̄ + n·ȳ + k = z̄]
Computed value: -0.2
Expected value: -0.2
check passed
Gram-Schmidt Orthogonalization and QR Decomposition¶
# ============================================================
print_header("Exercise 9-14 | Gram-Schmidt and QR")
# ============================================================
# --- Step 1 ---
print_step(1, "Build x_0, x_1, x_2 and set w_0 = x_0")
x0 = np.array([1., 2., 2.])
x1 = np.array([1., 0., 1.])
x2 = np.array([0., 0., 1.])
w0 = x0.copy()
check("||w_0||", np.linalg.norm(w0), 3)
# --- Step 2 ---
print_step(2, "w_1 = x_1 - Proj_{W_0}(x_1)")
a01 = (w0 @ x1) / (w0 @ w0)
w1 = x1 - a01 * x0
check("a_01", a01, 1/3)
check("w_1", w1, [2/3, -2/3, 1/3])
check("||w_1||", np.linalg.norm(w1), 1)
# --- Step 3 ---
print_step(3, "w_2 = x_2 - Proj_{W_1}(x_2), then rewrite it as a combination of x_0, x_1")
proj = (w0 @ x2) / (w0 @ w0) * w0 + (w1 @ x2) / (w1 @ w1) * w1
w2 = x2 - proj
check("Proj_{W_1}(x_2)", proj, [4/9, 2/9, 5/9])
check("w_2", w2, [-4/9, -2/9, 4/9])
check("||w_2||", np.linalg.norm(w2), 2/3)
a02, a12 = np.linalg.solve(np.column_stack([x0, x1])[:2], proj[:2]) # the first two components already determine the coefficients
check("Proj = a_02 x_0 + a_12 x_1", a02 * x0 + a12 * x1, proj)
check("(a_02, a_12)", [a02, a12], [1/9, 1/3])
# --- Step 4 ---
print_step(4, "Orthogonality and X = W U^{-1}")
X = np.column_stack([x0, x1, x2])
W = np.column_stack([w0, w1, w2])
G = W.T @ W
check("off-diagonal entries of W^T W", G - np.diag(np.diag(G)), np.zeros((3, 3)))
U = np.array([[1, -a01, -a02], [0, 1, -a12], [0, 0, 1]])
check("W = X U", X @ U, W)
check("U^{-1}", np.linalg.inv(U), [[1, 1/3, 2/9], [0, 1, 1/3], [0, 0, 1]])
check("X = W U^{-1}", W @ np.linalg.inv(U), X)
# --- Step 5 ---
print_step(5, "Normalize to get Q; R = D U^{-1} = Q^T X")
Q = W / np.linalg.norm(W, axis=0)
R = np.diag(np.linalg.norm(W, axis=0)) @ np.linalg.inv(U)
check("Q", Q, np.array([[1, 2, -2], [2, -2, -1], [2, 1, 2]]) / 3)
check("R", R, [[3, 1, 2/3], [0, 1, 1/3], [0, 0, 2/3]])
check("R = Q^T X", Q.T @ X, R)
check("Q^T Q = I", Q.T @ Q, np.eye(3))
check("QR = X", Q @ R, X)============================================================
Exercise 9-14 | Gram-Schmidt and QR
============================================================
▶ Step 1: Build x_0, x_1, x_2 and set w_0 = x_0
----------------------------------------
[||w_0||]
Computed value: 3.0
Expected value: 3
check passed
▶ Step 2: w_1 = x_1 - Proj_{W_0}(x_1)
----------------------------------------
[a_01]
Computed value: 0.3333333333333333
Expected value: 0.3333333333333333
check passed
[w_1]
Computed value: [ 0.6667 -0.6667 0.3333]
Expected value: [0.6666666666666666, -0.6666666666666666, 0.3333333333333333]
check passed
[||w_1||]
Computed value: 1.0
Expected value: 1
check passed
▶ Step 3: w_2 = x_2 - Proj_{W_1}(x_2), then rewrite it as a combination of x_0, x_1
----------------------------------------
[Proj_{W_1}(x_2)]
Computed value: [0.4444 0.2222 0.5556]
Expected value: [0.4444444444444444, 0.2222222222222222, 0.5555555555555556]
check passed
[w_2]
Computed value: [-0.4444 -0.2222 0.4444]
Expected value: [-0.4444444444444444, -0.2222222222222222, 0.4444444444444444]
check passed
[||w_2||]
Computed value: 0.6666666666666666
Expected value: 0.6666666666666666
check passed
[Proj = a_02 x_0 + a_12 x_1]
Computed value: [0.4444 0.2222 0.5556]
Expected value: [0.4444 0.2222 0.5556]
check passed
[(a_02, a_12)]
Computed value: [np.float64(0.11111111111111109), np.float64(0.33333333333333337)]
Expected value: [0.1111111111111111, 0.3333333333333333]
check passed
▶ Step 4: Orthogonality and X = W U^{-1}
----------------------------------------
[off-diagonal entries of W^T W]
Computed value: [[ 0. 0. 0.]
[ 0. 0. -0.]
[ 0. -0. 0.]]
Expected value: [[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]]
check passed
[W = X U]
Computed value: [[ 1. 0.6667 -0.4444]
[ 2. -0.6667 -0.2222]
[ 2. 0.3333 0.4444]]
Expected value: [[ 1. 0.6667 -0.4444]
[ 2. -0.6667 -0.2222]
[ 2. 0.3333 0.4444]]
check passed
[U^{-1}]
Computed value: [[1. 0.3333 0.2222]
[0. 1. 0.3333]
[0. 0. 1. ]]
Expected value: [[1, 0.3333333333333333, 0.2222222222222222], [0, 1, 0.3333333333333333], [0, 0, 1]]
check passed
[X = W U^{-1}]
Computed value: [[1. 1. 0.]
[2. 0. 0.]
[2. 1. 1.]]
Expected value: [[1. 1. 0.]
[2. 0. 0.]
[2. 1. 1.]]
check passed
▶ Step 5: Normalize to get Q; R = D U^{-1} = Q^T X
----------------------------------------
[Q]
Computed value: [[ 0.3333 0.6667 -0.6667]
[ 0.6667 -0.6667 -0.3333]
[ 0.6667 0.3333 0.6667]]
Expected value: [[ 0.3333 0.6667 -0.6667]
[ 0.6667 -0.6667 -0.3333]
[ 0.6667 0.3333 0.6667]]
check passed
[R]
Computed value: [[3. 1. 0.6667]
[0. 1. 0.3333]
[0. 0. 0.6667]]
Expected value: [[3, 1, 0.6666666666666666], [0, 1, 0.3333333333333333], [0, 0, 0.6666666666666666]]
check passed
[R = Q^T X]
Computed value: [[ 3. 1. 0.6667]
[ 0. 1. 0.3333]
[-0. -0. 0.6667]]
Expected value: [[3. 1. 0.6667]
[0. 1. 0.3333]
[0. 0. 0.6667]]
check passed
[Q^T Q = I]
Computed value: [[ 1. 0. -0.]
[ 0. 1. -0.]
[-0. -0. 1.]]
Expected value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
check passed
[QR = X]
Computed value: [[ 1. 1. 0.]
[ 2. 0. -0.]
[ 2. 1. 1.]]
Expected value: [[1. 1. 0.]
[2. 0. 0.]
[2. 1. 1.]]
check passed
# ============================================================
print_header("Exercise 9-15 | Stepwise Orthogonalization")
# ============================================================
def gram_schmidt_stepwise_15(X):
# normalize as we orthogonalize; R[j, k] = <x_k, q_j>
n = X.shape[1]
Q = np.zeros_like(X, dtype=float)
R = np.zeros((n, n))
for k in range(n):
w = X[:, k].astype(float)
for j in range(k):
R[j, k] = X[:, k] @ Q[:, j]
w = w - R[j, k] * Q[:, j]
R[k, k] = np.linalg.norm(w)
Q[:, k] = w / R[k, k]
return Q, R
# --- Step 1 ---
print_step(1, "Build q_0, q_1, q_2 one at a time")
X = np.array([[1., 1., 0.], [2., 0., 0.], [2., 1., 1.]])
Q, R = gram_schmidt_stepwise_15(X)
check("q_0", Q[:, 0], np.array([1, 2, 2]) / 3)
check("q_1", Q[:, 1], np.array([2, -2, 1]) / 3)
check("q_2", Q[:, 2], np.array([-2, -1, 2]) / 3)
# --- Step 2 ---
print_step(2, "Check <x_k, q_k> = ||w_k|| and X = QR")
check("diagonal entries (||w_0||, ||w_1||, ||w_2||)", np.diag(R), [3, 1, 2/3])
check("<x_k, q_k>", [X[:, k] @ Q[:, k] for k in range(3)], [3, 1, 2/3])
check("R", R, [[3, 1, 2/3], [0, 1, 1/3], [0, 0, 2/3]])
check("QR = X", Q @ R, X)
# --- Step 3 ---
print_step(3, "Compare with the previous exercise and with numpy's QR (diagonal made positive)")
Q_np, R_np = np.linalg.qr(X)
S = np.diag(np.sign(np.diag(R_np))) # make the diagonal entries of R positive
check("Q agrees with numpy", Q_np @ S, Q)
check("R agrees with numpy", S @ R_np, R)============================================================
Exercise 9-15 | Stepwise Orthogonalization
============================================================
▶ Step 1: Build q_0, q_1, q_2 one at a time
----------------------------------------
[q_0]
Computed value: [0.3333 0.6667 0.6667]
Expected value: [0.3333 0.6667 0.6667]
check passed
[q_1]
Computed value: [ 0.6667 -0.6667 0.3333]
Expected value: [ 0.6667 -0.6667 0.3333]
check passed
[q_2]
Computed value: [-0.6667 -0.3333 0.6667]
Expected value: [-0.6667 -0.3333 0.6667]
check passed
▶ Step 2: Check <x_k, q_k> = ||w_k|| and X = QR
----------------------------------------
[diagonal entries (||w_0||, ||w_1||, ||w_2||)]
Computed value: [3. 1. 0.6667]
Expected value: [3, 1, 0.6666666666666666]
check passed
[<x_k, q_k>]
Computed value: [np.float64(3.0), np.float64(1.0), np.float64(0.6666666666666666)]
Expected value: [3, 1, 0.6666666666666666]
check passed
[R]
Computed value: [[3. 1. 0.6667]
[0. 1. 0.3333]
[0. 0. 0.6667]]
Expected value: [[3, 1, 0.6666666666666666], [0, 1, 0.3333333333333333], [0, 0, 0.6666666666666666]]
check passed
[QR = X]
Computed value: [[ 1. 1. 0.]
[ 2. 0. -0.]
[ 2. 1. 1.]]
Expected value: [[1. 1. 0.]
[2. 0. 0.]
[2. 1. 1.]]
check passed
▶ Step 3: Compare with the previous exercise and with numpy's QR (diagonal made positive)
----------------------------------------
[Q agrees with numpy]
Computed value: [[ 0.3333 0.6667 -0.6667]
[ 0.6667 -0.6667 -0.3333]
[ 0.6667 0.3333 0.6667]]
Expected value: [[ 0.3333 0.6667 -0.6667]
[ 0.6667 -0.6667 -0.3333]
[ 0.6667 0.3333 0.6667]]
check passed
[R agrees with numpy]
Computed value: [[3. 1. 0.6667]
[0. 1. 0.3333]
[0. 0. 0.6667]]
Expected value: [[3. 1. 0.6667]
[0. 1. 0.3333]
[0. 0. 0.6667]]
check passed
# ============================================================
print_header("Exercise 9-16 | QR Decomposition of Square Matrices")
# ============================================================
def gram_schmidt_qr_16(A):
# classical Gram-Schmidt; returns Q and an R with positive diagonal entries
A = np.asarray(A, dtype=float)
n = A.shape[1]
Q = np.zeros_like(A)
R = np.zeros((n, n))
for k in range(n):
w = A[:, k].copy()
for j in range(k):
R[j, k] = Q[:, j] @ A[:, k]
w -= R[j, k] * Q[:, j]
R[k, k] = np.linalg.norm(w)
assert R[k, k] > 1e-12, "zero vector encountered: this column is linearly dependent on the preceding columns"
Q[:, k] = w / R[k, k]
return Q, R
s = 1 / np.sqrt(2)
# --- Step 1 ---
print_step(1, "Part 1: QR decomposition of A")
A = np.array([[1, 0, 0], [0, 1, 0], [1, 0, 1]])
QA, RA = gram_schmidt_qr_16(A)
check("Q", QA, [[s, 0, -s], [0, 1, 0], [s, 0, s]])
check("R", RA, [[np.sqrt(2), 0, s], [0, 1, 0], [0, 0, s]])
check("Q^T Q = I", QA.T @ QA, np.eye(3))
check("QR = A", QA @ RA, A)
# --- Step 2 ---
print_step(2, "Part 2: first confirm that B is invertible")
B = np.array([[1, 0, 1, -2], [1, -2, 3, -2], [1, 0, -1, 2], [1, -2, 1, -2]])
check("rank(B)", np.linalg.matrix_rank(B), 4)
check("det(B)", np.linalg.det(B), -16)
# --- Step 3 ---
print_step(3, "Part 2: QR decomposition of B")
QB, RB = gram_schmidt_qr_16(B)
check("Q", QB, np.array([[1, 1, 1, -1], [1, -1, 1, 1], [1, 1, -1, 1], [1, -1, -1, -1]]) / 2)
check("R", RB, [[2, -2, 2, -2], [0, 2, -2, 2], [0, 0, 2, -2], [0, 0, 0, 2]])
check("Q^T Q = I", QB.T @ QB, np.eye(4))
check("QR = B", QB @ RB, B)
check("det(Q)·det(R)", np.linalg.det(QB) * np.linalg.det(RB), -16)============================================================
Exercise 9-16 | QR Decomposition of Square Matrices
============================================================
▶ Step 1: Part 1: QR decomposition of A
----------------------------------------
[Q]
Computed value: [[ 0.7071 0. -0.7071]
[ 0. 1. 0. ]
[ 0.7071 0. 0.7071]]
Expected value: [[np.float64(0.7071067811865475), 0, np.float64(-0.7071067811865475)], [0, 1, 0], [np.float64(0.7071067811865475), 0, np.float64(0.7071067811865475)]]
check passed
[R]
Computed value: [[1.4142 0. 0.7071]
[0. 1. 0. ]
[0. 0. 0.7071]]
Expected value: [[np.float64(1.4142135623730951), 0, np.float64(0.7071067811865475)], [0, 1, 0], [0, 0, np.float64(0.7071067811865475)]]
check passed
[Q^T Q = I]
Computed value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
Expected value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
check passed
[QR = A]
Computed value: [[ 1. 0. -0.]
[ 0. 1. 0.]
[ 1. 0. 1.]]
Expected value: [[1 0 0]
[0 1 0]
[1 0 1]]
check passed
▶ Step 2: Part 2: first confirm that B is invertible
----------------------------------------
[rank(B)]
Computed value: 4
Expected value: 4
check passed
[det(B)]
Computed value: -15.999999999999998
Expected value: -16
check passed
▶ Step 3: Part 2: QR decomposition of B
----------------------------------------
[Q]
Computed value: [[ 0.5 0.5 0.5 -0.5]
[ 0.5 -0.5 0.5 0.5]
[ 0.5 0.5 -0.5 0.5]
[ 0.5 -0.5 -0.5 -0.5]]
Expected value: [[ 0.5 0.5 0.5 -0.5]
[ 0.5 -0.5 0.5 0.5]
[ 0.5 0.5 -0.5 0.5]
[ 0.5 -0.5 -0.5 -0.5]]
check passed
[R]
Computed value: [[ 2. -2. 2. -2.]
[ 0. 2. -2. 2.]
[ 0. 0. 2. -2.]
[ 0. 0. 0. 2.]]
Expected value: [[2, -2, 2, -2], [0, 2, -2, 2], [0, 0, 2, -2], [0, 0, 0, 2]]
check passed
[Q^T Q = I]
Computed value: [[1. 0. 0. 0.]
[0. 1. 0. 0.]
[0. 0. 1. 0.]
[0. 0. 0. 1.]]
Expected value: [[1. 0. 0. 0.]
[0. 1. 0. 0.]
[0. 0. 1. 0.]
[0. 0. 0. 1.]]
check passed
[QR = B]
Computed value: [[ 1. 0. 1. -2.]
[ 1. -2. 3. -2.]
[ 1. 0. -1. 2.]
[ 1. -2. 1. -2.]]
Expected value: [[ 1 0 1 -2]
[ 1 -2 3 -2]
[ 1 0 -1 2]
[ 1 -2 1 -2]]
check passed
[det(Q)·det(R)]
Computed value: -15.999999999999998
Expected value: -16
check passed
# ============================================================
print_header("Exercise 9-17 | Solving a System with QR")
# ============================================================
def back_substitution_17(R, c):
# upper triangular system R x = c, solved from the last row upward
n = len(c)
x = np.zeros(n)
for i in range(n - 1, -1, -1):
x[i] = (c[i] - R[i, i + 1:] @ x[i + 1:]) / R[i, i]
return x
A = np.array([[2., -2., 2.], [2., 0., -3.], [1., 1., 5.]])
Q = np.array([[2, -2, 1], [2, 1, -2], [1, 2, 2]]) / 3
R = np.array([[3., -1., 1.], [0., 2., 1.], [0., 0., 6.]])
b = np.array([4., -2., 5.])
# --- Step 1 ---
print_step(1, "Check the given QR decomposition")
check("Q^T Q = I", Q.T @ Q, np.eye(3))
check("QR = A", Q @ R, A)
# --- Step 2 ---
print_step(2, "c = Q^T b")
c = Q.T @ b
check("c", c, [3, 0, 6])
# --- Step 3 ---
print_step(3, "Solve R x = c by back substitution and verify A x = b")
x = back_substitution_17(R, c)
check("x", x, [1/2, -1/2, 1])
check("A x = b", A @ x, b)
check("agrees with np.linalg.solve", np.linalg.solve(A, b), x)============================================================
Exercise 9-17 | Solving a System with QR
============================================================
▶ Step 1: Check the given QR decomposition
----------------------------------------
[Q^T Q = I]
Computed value: [[ 1. -0. 0.]
[-0. 1. -0.]
[ 0. -0. 1.]]
Expected value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
check passed
[QR = A]
Computed value: [[ 2. -2. 2.]
[ 2. 0. -3.]
[ 1. 1. 5.]]
Expected value: [[ 2. -2. 2.]
[ 2. 0. -3.]
[ 1. 1. 5.]]
check passed
▶ Step 2: c = Q^T b
----------------------------------------
[c]
Computed value: [3. 0. 6.]
Expected value: [3, 0, 6]
check passed
▶ Step 3: Solve R x = c by back substitution and verify A x = b
----------------------------------------
[x]
Computed value: [ 0.5 -0.5 1. ]
Expected value: [0.5, -0.5, 1]
check passed
[A x = b]
Computed value: [ 4. -2. 5.]
Expected value: [ 4. -2. 5.]
check passed
[agrees with np.linalg.solve]
Computed value: [ 0.5 -0.5 1. ]
Expected value: [ 0.5 -0.5 1. ]
check passed
Orthogonal Matrices and Orthogonal Transformations¶
# ============================================================
print_header("Exercise 9-20 | Completing an Orthonormal Basis")
# ============================================================
thetas = np.linspace(0, 2 * np.pi, 7, endpoint=False)
# --- Step 1 ---
print_step(1, "Part 1: q_0 = (1,0,0), verified for several θ and ε")
q0 = np.array([1., 0., 0.])
for th in thetas:
for eps in (1, -1):
q1 = np.array([0, np.cos(th), np.sin(th)])
q2 = eps * np.array([0, -np.sin(th), np.cos(th)])
Qm = np.column_stack([q0, q1, q2])
assert np.allclose(Qm.T @ Qm, np.eye(3))
assert np.isclose(np.linalg.det(Qm), eps)
Q_ex = np.column_stack([q0, [0, 1/2, np.sqrt(3)/2], [0, -np.sqrt(3)/2, 1/2]]) # θ = π/3, ε = 1
check("Q^T Q for θ = π/3, ε = 1", Q_ex.T @ Q_ex, np.eye(3))
check("det(Q) for θ = π/3, ε = 1", np.linalg.det(Q_ex), 1)
print(f" verified {len(thetas) * 2} pairs (θ, ε) in total")
# --- Step 2 ---
print_step(2, "Part 2: q_0 = (1,2,-2)/3, build an orthonormal basis {u, v} of W")
q0 = np.array([1., 2., -2.]) / 3
u = np.array([2., 1., 2.]) / 3
v = np.cross(q0, u)
check("v = q_0 × u", v, np.array([2, -2, -1]) / 3)
check("Gram matrix of {q_0, u, v}", np.array([q0, u, v]) @ np.array([q0, u, v]).T, np.eye(3))
# --- Step 3 ---
print_step(3, "Part 2: verify that the parametric and expanded forms agree and all give orthonormal bases")
for th in thetas:
c, s = np.cos(th), np.sin(th)
for eps in (1, -1):
q1 = c * u + s * v
q2 = eps * (-s * u + c * v)
assert np.allclose(q1, np.array([2*c + 2*s, c - 2*s, 2*c - s]) / 3)
assert np.allclose(q2, eps * np.array([2*c - 2*s, -2*c - s, -c - 2*s]) / 3)
Qm = np.column_stack([q0, q1, q2])
assert np.allclose(Qm.T @ Qm, np.eye(3))
th0, eps0 = 0.0, 1
q1_0 = np.cos(th0) * u + np.sin(th0) * v # computed from the parametric form
q2_0 = eps0 * (-np.sin(th0) * u + np.cos(th0) * v)
check("q_1 given by the parametric form for θ = 0, ε = 1", q1_0, np.array([2, 1, 2]) / 3)
check("q_2 given by the parametric form for θ = 0, ε = 1", q2_0, np.array([2, -2, -1]) / 3)
check("Gram matrix of [q_0 q_1 q_2] for θ = 0, ε = 1",
np.column_stack([q0, q1_0, q2_0]).T @ np.column_stack([q0, q1_0, q2_0]), np.eye(3))
print(f" verified {len(thetas) * 2} pairs (θ, ε) in total")============================================================
Exercise 9-20 | Completing an Orthonormal Basis
============================================================
▶ Step 1: Part 1: q_0 = (1,0,0), verified for several θ and ε
----------------------------------------
[Q^T Q for θ = π/3, ε = 1]
Computed value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
Expected value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
check passed
[det(Q) for θ = π/3, ε = 1]
Computed value: 0.9999999999999999
Expected value: 1
check passed
verified 14 pairs (θ, ε) in total
▶ Step 2: Part 2: q_0 = (1,2,-2)/3, build an orthonormal basis {u, v} of W
----------------------------------------
[v = q_0 × u]
Computed value: [ 0.6667 -0.6667 -0.3333]
Expected value: [ 0.6667 -0.6667 -0.3333]
check passed
[Gram matrix of {q_0, u, v}]
Computed value: [[ 1. 0. -0.]
[ 0. 1. 0.]
[-0. 0. 1.]]
Expected value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
check passed
▶ Step 3: Part 2: verify that the parametric and expanded forms agree and all give orthonormal bases
----------------------------------------
[q_1 given by the parametric form for θ = 0, ε = 1]
Computed value: [0.6667 0.3333 0.6667]
Expected value: [0.6667 0.3333 0.6667]
check passed
[q_2 given by the parametric form for θ = 0, ε = 1]
Computed value: [ 0.6667 -0.6667 -0.3333]
Expected value: [ 0.6667 -0.6667 -0.3333]
check passed
[Gram matrix of [q_0 q_1 q_2] for θ = 0, ε = 1]
Computed value: [[ 1. 0. -0.]
[ 0. 1. 0.]
[-0. 0. 1.]]
Expected value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
check passed
verified 14 pairs (θ, ε) in total
# ============================================================
print_header("Exercise 9-23 | 2×2 Orthogonal Matrices")
# ============================================================
def rot_23(t):
return np.array([[np.cos(t), -np.sin(t)], [np.sin(t), np.cos(t)]])
def refl_23(t):
return np.array([[np.cos(t), np.sin(t)], [np.sin(t), -np.cos(t)]])
# --- Step 1 ---
print_step(1, "A random orthogonal matrix is always R(θ) or S(θ)")
rng = np.random.default_rng(23)
for k in range(4):
Q, _r = np.linalg.qr(rng.standard_normal((2, 2)))
if (np.linalg.det(Q) > 0) != (k % 2 == 0): # alternate between rotations and reflections
Q[:, 1] *= -1
theta = np.arctan2(Q[1, 0], Q[0, 0]) % (2 * np.pi) # column 0 = (cosθ, sinθ)
if np.isclose(np.linalg.det(Q), 1):
check(f"rotation R(θ), θ = {theta:.4f}", Q, rot_23(theta))
else:
check(f"reflection S(θ), θ = {theta:.4f}", Q, refl_23(theta))
# --- Step 2 ---
print_step(2, "Determinants: det R = 1, det S = -1")
t = np.pi / 3
check("det R(π/3)", np.linalg.det(rot_23(t)), 1)
check("det S(π/3)", np.linalg.det(refl_23(t)), -1)
# --- Step 3 ---
print_step(3, "Eigenvalues of the rotation matrix: e^{±iθ}")
lam = np.sort_complex(np.linalg.eigvals(rot_23(t)))
check("eigenvalues of R(π/3)", lam, np.sort_complex(np.array([np.exp(1j * t), np.exp(-1j * t)])))
v = np.array([1, -1j])
check("R(θ)(1,-i)ᵀ = e^{iθ}(1,-i)ᵀ", rot_23(t) @ v, np.exp(1j * t) * v)
check("R(π) = -I (every nonzero vector is an eigenvector)", rot_23(np.pi), -np.eye(2))
check("eigenvalues of R(π)", np.linalg.eigvals(rot_23(np.pi)).real, [-1, -1])
# --- Step 4 ---
print_step(4, "Eigenvalues and eigenvectors of the reflection matrix")
S = refl_23(t)
check("eigenvalues of S(π/3)", np.sort(np.linalg.eigvalsh(S)), [-1, 1])
u = np.array([np.cos(t / 2), np.sin(t / 2)])
n = np.array([-np.sin(t / 2), np.cos(t / 2)])
check("S u = u", S @ u, u)
check("S n = -n", S @ n, -n)
check("S = |u⟩⟨u| - |n⟩⟨n|", np.outer(u, u) - np.outer(n, n), S)
check("S = I - 2|n⟩⟨n|", np.eye(2) - 2 * np.outer(n, n), S)
============================================================
Exercise 9-23 | 2×2 Orthogonal Matrices
============================================================
▶ Step 1: A random orthogonal matrix is always R(θ) or S(θ)
----------------------------------------
[rotation R(θ), θ = 3.0372]
Computed value: [[-0.9946 -0.1042]
[ 0.1042 -0.9946]]
Expected value: [[-0.9946 -0.1042]
[ 0.1042 -0.9946]]
check passed
[reflection S(θ), θ = 4.2696]
Computed value: [[-0.4285 -0.9036]
[-0.9036 0.4285]]
Expected value: [[-0.4285 -0.9036]
[-0.9036 0.4285]]
check passed
[rotation R(θ), θ = 3.4868]
Computed value: [[-0.941 0.3384]
[-0.3384 -0.941 ]]
Expected value: [[-0.941 0.3384]
[-0.3384 -0.941 ]]
check passed
[reflection S(θ), θ = 4.2271]
Computed value: [[-0.4665 -0.8845]
[-0.8845 0.4665]]
Expected value: [[-0.4665 -0.8845]
[-0.8845 0.4665]]
check passed
▶ Step 2: Determinants: det R = 1, det S = -1
----------------------------------------
[det R(π/3)]
Computed value: 1.0
Expected value: 1
check passed
[det S(π/3)]
Computed value: -1.0
Expected value: -1
check passed
▶ Step 3: Eigenvalues of the rotation matrix: e^{±iθ}
----------------------------------------
[eigenvalues of R(π/3)]
Computed value: [0.5-0.866j 0.5+0.866j]
Expected value: [0.5-0.866j 0.5+0.866j]
check passed
[R(θ)(1,-i)ᵀ = e^{iθ}(1,-i)ᵀ]
Computed value: [0.5 +0.866j 0.866-0.5j ]
Expected value: [0.5 +0.866j 0.866-0.5j ]
check passed
[R(π) = -I (every nonzero vector is an eigenvector)]
Computed value: [[-1. -0.]
[ 0. -1.]]
Expected value: [[-1. -0.]
[-0. -1.]]
check passed
[eigenvalues of R(π)]
Computed value: [-1. -1.]
Expected value: [-1, -1]
check passed
▶ Step 4: Eigenvalues and eigenvectors of the reflection matrix
----------------------------------------
[eigenvalues of S(π/3)]
Computed value: [-1. 1.]
Expected value: [-1, 1]
check passed
[S u = u]
Computed value: [0.866 0.5 ]
Expected value: [0.866 0.5 ]
check passed
[S n = -n]
Computed value: [ 0.5 -0.866]
Expected value: [ 0.5 -0.866]
check passed
[S = |u⟩⟨u| - |n⟩⟨n|]
Computed value: [[ 0.5 0.866]
[ 0.866 -0.5 ]]
Expected value: [[ 0.5 0.866]
[ 0.866 -0.5 ]]
check passed
[S = I - 2|n⟩⟨n|]
Computed value: [[ 0.5 0.866]
[ 0.866 -0.5 ]]
Expected value: [[ 0.5 0.866]
[ 0.866 -0.5 ]]
check passed
# ============================================================
print_header("Exercise 9-24 | Householder Reflections")
# ============================================================
# --- Step 1 ---
print_step(1, "Q for n = (1,2,1)ᵀ/√6")
n = np.array([1, 2, 1]) / np.sqrt(6)
Q = np.eye(3) - 2 * np.outer(n, n)
check("Q", Q, np.array([[2, -2, -1], [-2, -1, -2], [-1, -2, 2]]) / 3)
check("QᵀQ", Q.T @ Q, np.eye(3))
check("det Q", np.linalg.det(Q), -1)
# --- Step 2 ---
print_step(2, "A general unit vector n (ℝ⁵): T preserves inner products")
rng = np.random.default_rng(24)
m = rng.standard_normal(5)
m = m / np.linalg.norm(m)
Qm = np.eye(5) - 2 * np.outer(m, m)
x, y = rng.standard_normal(5), rng.standard_normal(5)
check("⟨Tx, Ty⟩ = ⟨x, y⟩", (Qm @ x) @ (Qm @ y), x @ y)
# --- Step 3 ---
print_step(3, "Algebraic check: Qᵀ = Q, QᵀQ = I")
check("Qᵀ = Q", Qm.T, Qm)
check("QᵀQ = I₅", Qm.T @ Qm, np.eye(5))
# --- Step 4 ---
print_step(4, "Eigenvalues ±1 and eigenspaces")
check("eigenvalues of Q (increasing)", np.linalg.eigvalsh(Q), [-1, 1, 1])
check("Q n = -n", Q @ n, -n)
H = np.array([[2, -1, 0], [1, 0, -1]], dtype=float) # basis of E_1 (one vector per row)
check("nᵀh = 0 (h ∈ E_1)", H @ n, [0, 0])
check("Q h = h", (Q @ H.T).T, H)
============================================================
Exercise 9-24 | Householder Reflections
============================================================
▶ Step 1: Q for n = (1,2,1)ᵀ/√6
----------------------------------------
[Q]
Computed value: [[ 0.6667 -0.6667 -0.3333]
[-0.6667 -0.3333 -0.6667]
[-0.3333 -0.6667 0.6667]]
Expected value: [[ 0.6667 -0.6667 -0.3333]
[-0.6667 -0.3333 -0.6667]
[-0.3333 -0.6667 0.6667]]
check passed
[QᵀQ]
Computed value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
Expected value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
check passed
[det Q]
Computed value: -1.0000000000000004
Expected value: -1
check passed
▶ Step 2: A general unit vector n (ℝ⁵): T preserves inner products
----------------------------------------
[⟨Tx, Ty⟩ = ⟨x, y⟩]
Computed value: -2.9397317989147935
Expected value: -2.939731798914793
check passed
▶ Step 3: Algebraic check: Qᵀ = Q, QᵀQ = I
----------------------------------------
[Qᵀ = Q]
Computed value: [[-0.0592 -0.269 0.912 0.1467 0.2662]
[-0.269 0.9317 0.2316 0.0373 0.0676]
[ 0.912 0.2316 0.2148 -0.1263 -0.2292]
[ 0.1467 0.0373 -0.1263 0.9797 -0.0369]
[ 0.2662 0.0676 -0.2292 -0.0369 0.9331]]
Expected value: [[-0.0592 -0.269 0.912 0.1467 0.2662]
[-0.269 0.9317 0.2316 0.0373 0.0676]
[ 0.912 0.2316 0.2148 -0.1263 -0.2292]
[ 0.1467 0.0373 -0.1263 0.9797 -0.0369]
[ 0.2662 0.0676 -0.2292 -0.0369 0.9331]]
check passed
[QᵀQ = I₅]
Computed value: [[ 1. -0. 0. 0. 0.]
[-0. 1. 0. 0. 0.]
[ 0. 0. 1. -0. -0.]
[ 0. 0. -0. 1. 0.]
[ 0. 0. -0. 0. 1.]]
Expected value: [[1. 0. 0. 0. 0.]
[0. 1. 0. 0. 0.]
[0. 0. 1. 0. 0.]
[0. 0. 0. 1. 0.]
[0. 0. 0. 0. 1.]]
check passed
▶ Step 4: Eigenvalues ±1 and eigenspaces
----------------------------------------
[eigenvalues of Q (increasing)]
Computed value: [-1. 1. 1.]
Expected value: [-1, 1, 1]
check passed
[Q n = -n]
Computed value: [-0.4082 -0.8165 -0.4082]
Expected value: [-0.4082 -0.8165 -0.4082]
check passed
[nᵀh = 0 (h ∈ E_1)]
Computed value: [0. 0.]
Expected value: [0, 0]
check passed
[Q h = h]
Computed value: [[ 2. -1. 0.]
[ 1. 0. -1.]]
Expected value: [[ 2. -1. 0.]
[ 1. 0. -1.]]
check passed
# ============================================================
print_header("Exercise 9-25 | Reflection Across a Subspace")
# ============================================================
def proj_25(B):
B = np.asarray(B, dtype=float)
return B @ np.linalg.inv(B.T @ B) @ B.T
# --- Step 1 ---
print_step(1, "A random 2-dimensional subspace of ℝ⁵: Q = 2P - I is an orthogonal matrix")
rng = np.random.default_rng(25)
B = rng.standard_normal((5, 2))
P = proj_25(B)
Q = 2 * P - np.eye(5)
check("P² = P", P @ P, P)
check("Pᵀ = P", P.T, P)
check("QᵀQ = I₅", Q.T @ Q, np.eye(5))
# --- Step 2 ---
print_step(2, "Eigenvalues ±1, E_1 = W, E_{-1} = W⊥")
check("eigenvalues of Q (increasing)", np.linalg.eigvalsh(Q), [-1, -1, -1, 1, 1])
check("Q w = w (basis of W)", Q @ B, B)
_, _, Vt = np.linalg.svd(B.T)
Wperp = Vt[2:].T # orthonormal basis of W⊥ (5×3)
check("Bᵀ W⊥ = 0", B.T @ Wperp, np.zeros((2, 3)))
check("Q v = -v (basis of W⊥)", Q @ Wperp, -Wperp)
# --- Step 3 ---
print_step(3, "W = span{e₀, e₁} ⊂ ℝ⁴")
B4 = np.array([[1, 0], [0, 1], [0, 0], [0, 0]])
P4 = proj_25(B4)
Q4 = 2 * P4 - np.eye(4)
check("P_W", P4, np.diag([1, 1, 0, 0]))
check("Q", Q4, np.diag([1, 1, -1, -1]))
check("det Q", np.linalg.det(Q4), 1)
============================================================
Exercise 9-25 | Reflection Across a Subspace
============================================================
▶ Step 1: A random 2-dimensional subspace of ℝ⁵: Q = 2P - I is an orthogonal matrix
----------------------------------------
[P² = P]
Computed value: [[ 0.0318 0.0542 -0.0397 0.1171 0.1122]
[ 0.0542 0.6012 -0.2743 0.3155 -0.2491]
[-0.0397 -0.2743 0.1335 -0.1933 0.0386]
[ 0.1171 0.3155 -0.1933 0.4571 0.3123]
[ 0.1122 -0.2491 0.0386 0.3123 0.7763]]
Expected value: [[ 0.0318 0.0542 -0.0397 0.1171 0.1122]
[ 0.0542 0.6012 -0.2743 0.3155 -0.2491]
[-0.0397 -0.2743 0.1335 -0.1933 0.0386]
[ 0.1171 0.3155 -0.1933 0.4571 0.3123]
[ 0.1122 -0.2491 0.0386 0.3123 0.7763]]
check passed
[Pᵀ = P]
Computed value: [[ 0.0318 0.0542 -0.0397 0.1171 0.1122]
[ 0.0542 0.6012 -0.2743 0.3155 -0.2491]
[-0.0397 -0.2743 0.1335 -0.1933 0.0386]
[ 0.1171 0.3155 -0.1933 0.4571 0.3123]
[ 0.1122 -0.2491 0.0386 0.3123 0.7763]]
Expected value: [[ 0.0318 0.0542 -0.0397 0.1171 0.1122]
[ 0.0542 0.6012 -0.2743 0.3155 -0.2491]
[-0.0397 -0.2743 0.1335 -0.1933 0.0386]
[ 0.1171 0.3155 -0.1933 0.4571 0.3123]
[ 0.1122 -0.2491 0.0386 0.3123 0.7763]]
check passed
[QᵀQ = I₅]
Computed value: [[ 1. 0. -0. -0. -0.]
[ 0. 1. -0. 0. 0.]
[-0. -0. 1. -0. 0.]
[-0. 0. -0. 1. -0.]
[-0. 0. 0. -0. 1.]]
Expected value: [[1. 0. 0. 0. 0.]
[0. 1. 0. 0. 0.]
[0. 0. 1. 0. 0.]
[0. 0. 0. 1. 0.]
[0. 0. 0. 0. 1.]]
check passed
▶ Step 2: Eigenvalues ±1, E_1 = W, E_{-1} = W⊥
----------------------------------------
[eigenvalues of Q (increasing)]
Computed value: [-1. -1. -1. 1. 1.]
Expected value: [-1, -1, -1, 1, 1]
check passed
[Q w = w (basis of W)]
Computed value: [[ 0.354 -0.005 ]
[-0.5316 -2.2766]
[ 0.0187 0.9274]
[ 1.0433 -0.5362]
[ 2.2294 1.9437]]
Expected value: [[ 0.354 -0.005 ]
[-0.5316 -2.2766]
[ 0.0187 0.9274]
[ 1.0433 -0.5362]
[ 2.2294 1.9437]]
check passed
[Bᵀ W⊥ = 0]
Computed value: [[-0. -0. 0.]
[-0. 0. -0.]]
Expected value: [[0. 0. 0.]
[0. 0. 0.]]
check passed
[Q v = -v (basis of W⊥)]
Computed value: [[-0.0591 0.5005 0.8451]
[-0.3552 0.4054 -0.329 ]
[-0.9245 -0.0998 0.0415]
[-0.1243 -0.6798 0.2554]
[-0.0094 0.3362 -0.3325]]
Expected value: [[-0.0591 0.5005 0.8451]
[-0.3552 0.4054 -0.329 ]
[-0.9245 -0.0998 0.0415]
[-0.1243 -0.6798 0.2554]
[-0.0094 0.3362 -0.3325]]
check passed
▶ Step 3: W = span{e₀, e₁} ⊂ ℝ⁴
----------------------------------------
[P_W]
Computed value: [[1. 0. 0. 0.]
[0. 1. 0. 0.]
[0. 0. 0. 0.]
[0. 0. 0. 0.]]
Expected value: [[1 0 0 0]
[0 1 0 0]
[0 0 0 0]
[0 0 0 0]]
check passed
[Q]
Computed value: [[ 1. 0. 0. 0.]
[ 0. 1. 0. 0.]
[ 0. 0. -1. 0.]
[ 0. 0. 0. -1.]]
Expected value: [[ 1 0 0 0]
[ 0 1 0 0]
[ 0 0 -1 0]
[ 0 0 0 -1]]
check passed
[det Q]
Computed value: 1.0
Expected value: 1
check passed
# ============================================================
print_header("Exercise 9-26 | Orthogonal Transformations under a Nonstandard Inner Product")
# ============================================================
G = np.array([[2, 1], [1, 1]])
def ip_26(x, y):
return x @ G @ y # ⟨x, y⟩_G = xᵀ G y
# --- Step 1 ---
print_step(1, "G is positive definite (leading principal minors)")
check("(D_1, D_2): determinants of the upper-left 1×1 and 2×2 submatrices", [G[0, 0], np.linalg.det(G)], [2, 1])
# --- Step 2 ---
print_step(2, "B is orthonormal for ⟨·,·⟩_G")
b0, b1 = np.array([0, 1]), np.array([1, -1])
check("(⟨b0,b0⟩, ⟨b1,b1⟩, ⟨b0,b1⟩)", [ip_26(b0, b0), ip_26(b1, b1), ip_26(b0, b1)], [1, 1, 0])
# --- Step 3 ---
print_step(3, "T preserves ⟨·,·⟩_G: AᵀGA = G")
A = np.array([[-1, 0], [2, 1]])
check("AᵀGA", A.T @ G @ A, G)
rng = np.random.default_rng(26)
u, v = rng.standard_normal(2), rng.standard_normal(2)
check("⟨Tu, Tv⟩_G = ⟨u, v⟩_G", ip_26(A @ u, A @ v), ip_26(u, v))
check("AᵀA (not orthogonal for the standard inner product)", A.T @ A, [[5, 2], [2, 1]])
# --- Step 4 ---
print_step(4, "Matrix representation [T]_B")
M = np.column_stack([b0, b1])
Q = np.linalg.inv(M) @ A @ M
check("[T]_B", Q, [[1, 0], [0, -1]])
check("QᵀQ", Q.T @ Q, np.eye(2))
============================================================
Exercise 9-26 | Orthogonal Transformations under a Nonstandard Inner Product
============================================================
▶ Step 1: G is positive definite (leading principal minors)
----------------------------------------
[(D_1, D_2): determinants of the upper-left 1×1 and 2×2 submatrices]
Computed value: [np.int64(2), np.float64(1.0)]
Expected value: [2, 1]
check passed
▶ Step 2: B is orthonormal for ⟨·,·⟩_G
----------------------------------------
[(⟨b0,b0⟩, ⟨b1,b1⟩, ⟨b0,b1⟩)]
Computed value: [np.int64(1), np.int64(1), np.int64(0)]
Expected value: [1, 1, 0]
check passed
▶ Step 3: T preserves ⟨·,·⟩_G: AᵀGA = G
----------------------------------------
[AᵀGA]
Computed value: [[2 1]
[1 1]]
Expected value: [[2 1]
[1 1]]
check passed
[⟨Tu, Tv⟩_G = ⟨u, v⟩_G]
Computed value: -15.656094408098612
Expected value: -15.65609440809861
check passed
[AᵀA (not orthogonal for the standard inner product)]
Computed value: [[5 2]
[2 1]]
Expected value: [[5, 2], [2, 1]]
check passed
▶ Step 4: Matrix representation [T]_B
----------------------------------------
[[T]_B]
Computed value: [[ 1. 0.]
[ 0. -1.]]
Expected value: [[1, 0], [0, -1]]
check passed
[QᵀQ]
Computed value: [[1. 0.]
[0. 1.]]
Expected value: [[1. 0.]
[0. 1.]]
check passed
# ============================================================
print_header("Exercise 9-27 | Matrix Representations of Orthogonal Transformations")
# ============================================================
# --- Step 1 ---
print_step(1, "The matrix representation with respect to an orthonormal basis is orthogonal (random example in ℝ⁴)")
rng = np.random.default_rng(27)
U, _ = np.linalg.qr(rng.standard_normal((4, 4))) # orthogonal transformation T(x) = U x
Pb, _ = np.linalg.qr(rng.standard_normal((4, 4))) # orthonormal basis q_0..q_3 = columns of Pb
TB = np.linalg.inv(Pb) @ U @ Pb # [T]_B
check("[T]_Bᵀ [T]_B", TB.T @ TB, np.eye(4))
# --- Step 2 ---
print_step(2, "Counterexample with the non-orthogonal basis A = {(1,0)ᵀ, (1,1)ᵀ}")
T = np.array([[0, -1], [1, 0]]) # rotation by 90°
S = np.array([[1, 1], [0, 1]]) # columns are a_0, a_1
TA = np.linalg.inv(S) @ T @ S
check("[T]_A", TA, [[-1, -2], [1, 1]])
check("[T]_Aᵀ [T]_A (≠ I)", TA.T @ TA, [[2, 3], [3, 5]])
GA = S.T @ S # Gram matrix
check("[T]_Aᵀ G_A [T]_A = G_A", TA.T @ GA @ TA, GA)
============================================================
Exercise 9-27 | Matrix Representations of Orthogonal Transformations
============================================================
▶ Step 1: The matrix representation with respect to an orthonormal basis is orthogonal (random example in ℝ⁴)
----------------------------------------
[[T]_Bᵀ [T]_B]
Computed value: [[ 1. -0. -0. 0.]
[-0. 1. -0. 0.]
[-0. -0. 1. -0.]
[ 0. 0. -0. 1.]]
Expected value: [[1. 0. 0. 0.]
[0. 1. 0. 0.]
[0. 0. 1. 0.]
[0. 0. 0. 1.]]
check passed
▶ Step 2: Counterexample with the non-orthogonal basis A = {(1,0)ᵀ, (1,1)ᵀ}
----------------------------------------
[[T]_A]
Computed value: [[-1. -2.]
[ 1. 1.]]
Expected value: [[-1, -2], [1, 1]]
check passed
[[T]_Aᵀ [T]_A (≠ I)]
Computed value: [[2. 3.]
[3. 5.]]
Expected value: [[2, 3], [3, 5]]
check passed
[[T]_Aᵀ G_A [T]_A = G_A]
Computed value: [[1. 1.]
[1. 2.]]
Expected value: [[1 1]
[1 2]]
check passed
# ============================================================
print_header("Exercise 9-28 | Orientation of Bases")
# ============================================================
rng = np.random.default_rng(28)
# --- Step 1 ---
print_step(1, "Same orientation is an equivalence relation: multiplicativity of the determinant")
M, N = rng.standard_normal((3, 3)), rng.standard_normal((3, 3))
check("det(M⁻¹) = 1/det M", np.linalg.det(np.linalg.inv(M)), 1 / np.linalg.det(M))
check("det(NM) = det N · det M", np.linalg.det(N @ M), np.linalg.det(N) * np.linalg.det(M))
Fs = [rng.standard_normal((3, 3)) for _ in range(6)] # 6 random bases (columns are the basis vectors)
same = [[np.linalg.det(np.linalg.solve(F, Fp)) > 0 for Fp in Fs] for F in Fs]
sign = [[np.sign(np.linalg.det(F)) == np.sign(np.linalg.det(Fp)) for Fp in Fs] for F in Fs]
check("same orientation ⇔ dets of the same sign (only two classes)", np.array(same), np.array(sign))
# --- Step 2 ---
print_step(2, "Positively oriented ⇔ det F > 0: verify against another positively oriented basis G")
G = rng.standard_normal((3, 3))
if np.linalg.det(G) < 0:
G[:, 0] *= -1 # give G the same orientation as the standard basis
Fs2 = [rng.standard_normal((3, 3)) for _ in range(20)]
by_def = [np.linalg.det(np.linalg.solve(G, F)) > 0 for F in Fs2] # by definition: F has the same orientation as G
by_det = [np.linalg.det(F) > 0 for F in Fs2] # criterion: det F > 0
print(f" {sum(by_det)} of the 20 random bases are positively oriented")
check("'same orientation as the positively oriented G' ⇔ 'det F > 0'", np.array(by_def), np.array(by_det))
# --- Step 3 ---
print_step(3, "ℝ²: det(f₀ f₁) = r₀ r₁ sin φ, where φ is the signed angle from f₀ to f₁")
for alpha, phi, r0, r1 in [(0.3, np.pi / 2, 1, 2), (1.0, 2.5, 3, 1), (2.0, -1.2, 1, 1)]:
f0 = r0 * np.array([np.cos(alpha), np.sin(alpha)])
f1 = r1 * np.array([np.cos(alpha + phi), np.sin(alpha + phi)])
d = np.linalg.det(np.column_stack([f0, f1]))
turn = "counterclockwise → positively oriented" if phi > 0 else "clockwise → negatively oriented"
check(f"φ = {phi:+.4f} ({turn})", d, r0 * r1 * np.sin(phi))
check("det(e₀ e₁)", np.linalg.det(np.eye(2)), 1)
check("det(e₁ e₀)", np.linalg.det(np.eye(2)[:, ::-1]), -1)
# --- Step 4 ---
print_step(4, "ℝ³: det(u, v, u×v) = ‖u×v‖² > 0")
u, v = rng.standard_normal(3), rng.standard_normal(3)
w = np.cross(u, v)
check("det(u v u×v)", np.linalg.det(np.column_stack([u, v, w])), w @ w)
# --- Step 5 ---
print_step(5, "Orthogonal matrices in ℝ³: det Q = 1 ⇔ q₀ × q₁ = q₂ (right-handed), det Q = -1 ⇔ q₀ × q₁ = -q₂")
for k in range(4):
Q, _ = np.linalg.qr(rng.standard_normal((3, 3)))
if (np.linalg.det(Q) > 0) != (k % 2 == 0): # alternate between det = 1 and det = -1
Q[:, 0] *= -1
d = np.linalg.det(Q)
check(f"det Q = {d:+.0f}: q₀ × q₁ = (det Q)·q₂", np.cross(Q[:, 0], Q[:, 1]), d * Q[:, 2])
# --- Step 6 ---
print_step(6, "Change-of-basis matrix between the two bases of W")
A = np.array([[1, 0], [0, 1], [1, 0]], dtype=float) # columns are a_0, a_1
C = np.array([[1, 1], [1, 2], [1, 1]], dtype=float) # columns are c_0, c_1
M6, *_ = np.linalg.lstsq(A, C, rcond=None)
check("A M = C", A @ M6, C)
check("M", M6, [[1, 1], [1, 2]])
check("det M", np.linalg.det(M6), 1)
============================================================
Exercise 9-28 | Orientation of Bases
============================================================
▶ Step 1: Same orientation is an equivalence relation: multiplicativity of the determinant
----------------------------------------
[det(M⁻¹) = 1/det M]
Computed value: -0.6314246303908053
Expected value: -0.6314246303908053
check passed
[det(NM) = det N · det M]
Computed value: 2.2698124482564226
Expected value: 2.2698124482564226
check passed
[same orientation ⇔ dets of the same sign (only two classes)]
Computed value: [[ True True False True True False]
[ True True False True True False]
[False False True False False True]
[ True True False True True False]
[ True True False True True False]
[False False True False False True]]
Expected value: [[ True True False True True False]
[ True True False True True False]
[False False True False False True]
[ True True False True True False]
[ True True False True True False]
[False False True False False True]]
check passed
▶ Step 2: Positively oriented ⇔ det F > 0: verify against another positively oriented basis G
----------------------------------------
12 of the 20 random bases are positively oriented
['same orientation as the positively oriented G' ⇔ 'det F > 0']
Computed value: [ True True False True False True False True True True False True False True False False
False True True True]
Expected value: [ True True False True False True False True True True False True False True False False
False True True True]
check passed
▶ Step 3: ℝ²: det(f₀ f₁) = r₀ r₁ sin φ, where φ is the signed angle from f₀ to f₁
----------------------------------------
[φ = +1.5708 (counterclockwise → positively oriented)]
Computed value: 1.9999999999999998
Expected value: 2.0
check passed
[φ = +2.5000 (counterclockwise → positively oriented)]
Computed value: 1.7954164323118693
Expected value: 1.7954164323118693
check passed
[φ = -1.2000 (clockwise → negatively oriented)]
Computed value: -0.9320390859672263
Expected value: -0.9320390859672263
check passed
[det(e₀ e₁)]
Computed value: 1.0
Expected value: 1
check passed
[det(e₁ e₀)]
Computed value: -1.0
Expected value: -1
check passed
▶ Step 4: ℝ³: det(u, v, u×v) = ‖u×v‖² > 0
----------------------------------------
[det(u v u×v)]
Computed value: 5.820236498742807
Expected value: 5.820236498742807
check passed
▶ Step 5: Orthogonal matrices in ℝ³: det Q = 1 ⇔ q₀ × q₁ = q₂ (right-handed), det Q = -1 ⇔ q₀ × q₁ = -q₂
----------------------------------------
[det Q = +1: q₀ × q₁ = (det Q)·q₂]
Computed value: [0.9289 0.3283 0.1716]
Expected value: [0.9289 0.3283 0.1716]
check passed
[det Q = -1: q₀ × q₁ = (det Q)·q₂]
Computed value: [ 0.2487 0.4286 -0.8686]
Expected value: [ 0.2487 0.4286 -0.8686]
check passed
[det Q = +1: q₀ × q₁ = (det Q)·q₂]
Computed value: [-0.0554 -0.1586 0.9858]
Expected value: [-0.0554 -0.1586 0.9858]
check passed
[det Q = -1: q₀ × q₁ = (det Q)·q₂]
Computed value: [ 0.6804 -0.676 0.283 ]
Expected value: [ 0.6804 -0.676 0.283 ]
check passed
▶ Step 6: Change-of-basis matrix between the two bases of W
----------------------------------------
[A M = C]
Computed value: [[1. 1.]
[1. 2.]
[1. 1.]]
Expected value: [[1. 1.]
[1. 2.]
[1. 1.]]
check passed
[M]
Computed value: [[1. 1.]
[1. 2.]]
Expected value: [[1, 1], [1, 2]]
check passed
[det M]
Computed value: 0.9999999999999997
Expected value: 1
check passed
# ============================================================
print_header("Exercise 9-29 | Rotation Axis and Rotation Angle")
# ============================================================
Q = np.array([[2, -1, 2], [2, 2, -1], [-1, 2, 2]]) / 3
# --- Step 1 ---
print_step(1, "Orthogonal with det = 1")
check("QᵀQ", Q.T @ Q, np.eye(3))
check("det Q", np.linalg.det(Q), 1)
# --- Step 2 ---
print_step(2, "Rotation axis: Ker(Q - I)")
check("rank(Q - I)", np.linalg.matrix_rank(Q - np.eye(3)), 2)
_, _, Vt = np.linalg.svd(Q - np.eye(3))
axis = Vt[-1] * np.sign(Vt[-1].sum())
q2 = np.ones(3) / np.sqrt(3)
check("unit axis vector", axis, q2)
check("Q q₂ = q₂", Q @ q2, q2)
# --- Step 3 ---
print_step(3, "Rotation angle: 1 + 2cosθ = tr Q")
check("tr Q", np.trace(Q), 2)
theta = np.arccos((np.trace(Q) - 1) / 2)
check("θ", theta, np.pi / 3)
q0 = np.array([1, -1, 0]) / np.sqrt(2)
q1 = np.cross(q2, q0)
check("q₁ = q₂ × q₀", q1, np.array([1, 1, -2]) / np.sqrt(6))
P = np.column_stack([q0, q1, q2])
check("det P (positively oriented basis)", np.linalg.det(P), 1)
c, s = np.cos(np.pi / 3), np.sin(np.pi / 3)
Rz = np.array([[c, -s, 0], [s, c, 0], [0, 0, 1]])
check("PᵀQP = R_z(π/3)", P.T @ Q @ P, Rz)
dirac = np.outer(q2, q2) + c * (np.outer(q0, q0) + np.outer(q1, q1)) + s * (np.outer(q1, q0) - np.outer(q0, q1))
check("Dirac expansion = Q", dirac, Q)
============================================================
Exercise 9-29 | Rotation Axis and Rotation Angle
============================================================
▶ Step 1: Orthogonal with det = 1
----------------------------------------
[QᵀQ]
Computed value: [[ 1. -0. 0.]
[-0. 1. -0.]
[ 0. -0. 1.]]
Expected value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
check passed
[det Q]
Computed value: 1.0
Expected value: 1
check passed
▶ Step 2: Rotation axis: Ker(Q - I)
----------------------------------------
[rank(Q - I)]
Computed value: 2
Expected value: 2
check passed
[unit axis vector]
Computed value: [0.5774 0.5774 0.5774]
Expected value: [0.5774 0.5774 0.5774]
check passed
[Q q₂ = q₂]
Computed value: [0.5774 0.5774 0.5774]
Expected value: [0.5774 0.5774 0.5774]
check passed
▶ Step 3: Rotation angle: 1 + 2cosθ = tr Q
----------------------------------------
[tr Q]
Computed value: 2.0
Expected value: 2
check passed
[θ]
Computed value: 1.0471975511965979
Expected value: 1.0471975511965976
check passed
[q₁ = q₂ × q₀]
Computed value: [ 0.4082 0.4082 -0.8165]
Expected value: [ 0.4082 0.4082 -0.8165]
check passed
[det P (positively oriented basis)]
Computed value: 1.0
Expected value: 1
check passed
[PᵀQP = R_z(π/3)]
Computed value: [[ 0.5 -0.866 0. ]
[ 0.866 0.5 0. ]
[-0. -0. 1. ]]
Expected value: [[ 0.5 -0.866 0. ]
[ 0.866 0.5 0. ]
[ 0. 0. 1. ]]
check passed
[Dirac expansion = Q]
Computed value: [[ 0.6667 -0.3333 0.6667]
[ 0.6667 0.6667 -0.3333]
[-0.3333 0.6667 0.6667]]
Expected value: [[ 0.6667 -0.3333 0.6667]
[ 0.6667 0.6667 -0.3333]
[-0.3333 0.6667 0.6667]]
check passed
# ============================================================
print_header("Exercise 9-30 | Rotation Decomposition of an Orthogonal Matrix")
# ============================================================
def Rz_30(t):
c, s = np.cos(t), np.sin(t)
return np.array([[c, -s, 0], [s, c, 0], [0, 0, 1]])
def Ry_30(t): # signs as in the original manuscript
c, s = np.cos(t), np.sin(t)
return np.array([[c, 0, -s], [0, 1, 0], [s, 0, c]])
def Rx_30(t):
c, s = np.cos(t), np.sin(t)
return np.array([[1, 0, 0], [0, c, -s], [0, s, c]])
Q = np.array([[-2, 1, -2], [-2, -2, 1], [1, -2, -2]]) / 3
Q29 = np.array([[2, -1, 2], [2, 2, -1], [-1, 2, 2]]) / 3
R = np.diag([1, 1, -1])
# --- Step 1 ---
print_step(1, "Q = -Q' (the matrix of Exercise 9-29), det Q = -1")
check("Q = -Q'", Q, -Q29)
check("QᵀQ", Q.T @ Q, np.eye(3))
check("det Q", np.linalg.det(Q), -1)
# --- Step 2 ---
print_step(2, "A = Q R (column 2 negated), det A = 1")
A = Q @ R
check("A", A, np.array([[-2, 1, 2], [-2, -2, -1], [1, -2, 2]]) / 3)
check("det A", np.linalg.det(A), 1)
# --- Step 3 ---
print_step(3, "Solve for α, β, γ from the entries of A")
alpha, beta, gamma = -3 * np.pi / 4, np.arctan(np.sqrt(2) / 4), -np.pi / 4
check("sin β = 1/3", np.sin(beta), 1 / 3)
check("β = arcsin A[2,0]", np.arcsin(A[2, 0]), beta)
check("γ = atan2(A[2,1], A[2,2])", np.arctan2(A[2, 1], A[2, 2]), gamma)
check("α = atan2(A[1,0], A[0,0])", np.arctan2(A[1, 0], A[0, 0]), alpha)
print(f" β ≈ {beta:.4f} rad ≈ {np.degrees(beta):.2f}°")
# --- Step 4 ---
print_step(4, "Check the product")
check("R_y(β) R_x(γ)", Ry_30(beta) @ Rx_30(gamma),
[[2 * np.sqrt(2) / 3, 1 / (3 * np.sqrt(2)), -1 / (3 * np.sqrt(2))],
[0, 1 / np.sqrt(2), 1 / np.sqrt(2)],
[1 / 3, -2 / 3, 2 / 3]])
check("R_z(α) R_y(β) R_x(γ) R = Q", Rz_30(alpha) @ Ry_30(beta) @ Rx_30(gamma) @ R, Q)
# --- Step 5 ---
print_step(5, "Another view: Q = Q' R_z(π) R, a rotoreflection")
check("-I = R_z(π) R", Rz_30(np.pi) @ R, -np.eye(3))
check("Q = Q' R_z(π) R", Q29 @ Rz_30(np.pi) @ R, Q)
q = np.ones(3) / np.sqrt(3)
def rodrigues_30(axis, t): # rotation by t about the unit axis `axis` (right-hand rule)
K = np.array([[0, -axis[2], axis[1]], [axis[2], 0, -axis[0]], [-axis[1], axis[0], 0]])
return np.cos(t) * np.eye(3) + np.sin(t) * K + (1 - np.cos(t)) * np.outer(axis, axis)
check("Q' = rotation by π/3 about q", rodrigues_30(q, np.pi / 3), Q29)
check("Q = rotation by 4π/3 about q, then reflection across q⊥", rodrigues_30(q, 4 * np.pi / 3) @ (np.eye(3) - 2 * np.outer(q, q)), Q)
check("Q q = -q", Q @ q, -q)
============================================================
Exercise 9-30 | Rotation Decomposition of an Orthogonal Matrix
============================================================
▶ Step 1: Q = -Q' (the matrix of Exercise 9-29), det Q = -1
----------------------------------------
[Q = -Q']
Computed value: [[-0.6667 0.3333 -0.6667]
[-0.6667 -0.6667 0.3333]
[ 0.3333 -0.6667 -0.6667]]
Expected value: [[-0.6667 0.3333 -0.6667]
[-0.6667 -0.6667 0.3333]
[ 0.3333 -0.6667 -0.6667]]
check passed
[QᵀQ]
Computed value: [[ 1. -0. 0.]
[-0. 1. -0.]
[ 0. -0. 1.]]
Expected value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
check passed
[det Q]
Computed value: -1.0
Expected value: -1
check passed
▶ Step 2: A = Q R (column 2 negated), det A = 1
----------------------------------------
[A]
Computed value: [[-0.6667 0.3333 0.6667]
[-0.6667 -0.6667 -0.3333]
[ 0.3333 -0.6667 0.6667]]
Expected value: [[-0.6667 0.3333 0.6667]
[-0.6667 -0.6667 -0.3333]
[ 0.3333 -0.6667 0.6667]]
check passed
[det A]
Computed value: 1.0
Expected value: 1
check passed
▶ Step 3: Solve for α, β, γ from the entries of A
----------------------------------------
[sin β = 1/3]
Computed value: 0.33333333333333337
Expected value: 0.3333333333333333
check passed
[β = arcsin A[2,0]]
Computed value: 0.3398369094541219
Expected value: 0.339836909454122
check passed
[γ = atan2(A[2,1], A[2,2])]
Computed value: -0.7853981633974483
Expected value: -0.7853981633974483
check passed
[α = atan2(A[1,0], A[0,0])]
Computed value: -2.356194490192345
Expected value: -2.356194490192345
check passed
β ≈ 0.3398 rad ≈ 19.47°
▶ Step 4: Check the product
----------------------------------------
[R_y(β) R_x(γ)]
Computed value: [[ 0.9428 0.2357 -0.2357]
[ 0. 0.7071 0.7071]
[ 0.3333 -0.6667 0.6667]]
Expected value: [[np.float64(0.9428090415820635), np.float64(0.2357022603955158), np.float64(-0.2357022603955158)], [0, np.float64(0.7071067811865475), np.float64(0.7071067811865475)], [0.3333333333333333, -0.6666666666666666, 0.6666666666666666]]
check passed
[R_z(α) R_y(β) R_x(γ) R = Q]
Computed value: [[-0.6667 0.3333 -0.6667]
[-0.6667 -0.6667 0.3333]
[ 0.3333 -0.6667 -0.6667]]
Expected value: [[-0.6667 0.3333 -0.6667]
[-0.6667 -0.6667 0.3333]
[ 0.3333 -0.6667 -0.6667]]
check passed
▶ Step 5: Another view: Q = Q' R_z(π) R, a rotoreflection
----------------------------------------
[-I = R_z(π) R]
Computed value: [[-1. -0. 0.]
[ 0. -1. 0.]
[ 0. 0. -1.]]
Expected value: [[-1. -0. -0.]
[-0. -1. -0.]
[-0. -0. -1.]]
check passed
[Q = Q' R_z(π) R]
Computed value: [[-0.6667 0.3333 -0.6667]
[-0.6667 -0.6667 0.3333]
[ 0.3333 -0.6667 -0.6667]]
Expected value: [[-0.6667 0.3333 -0.6667]
[-0.6667 -0.6667 0.3333]
[ 0.3333 -0.6667 -0.6667]]
check passed
[Q' = rotation by π/3 about q]
Computed value: [[ 0.6667 -0.3333 0.6667]
[ 0.6667 0.6667 -0.3333]
[-0.3333 0.6667 0.6667]]
Expected value: [[ 0.6667 -0.3333 0.6667]
[ 0.6667 0.6667 -0.3333]
[-0.3333 0.6667 0.6667]]
check passed
[Q = rotation by 4π/3 about q, then reflection across q⊥]
Computed value: [[-0.6667 0.3333 -0.6667]
[-0.6667 -0.6667 0.3333]
[ 0.3333 -0.6667 -0.6667]]
Expected value: [[-0.6667 0.3333 -0.6667]
[-0.6667 -0.6667 0.3333]
[ 0.3333 -0.6667 -0.6667]]
check passed
[Q q = -q]
Computed value: [-0.5774 -0.5774 -0.5774]
Expected value: [-0.5774 -0.5774 -0.5774]
check passed
Orthogonal Diagonalization of Symmetric Matrices¶
# ============================================================
print_header("Exercise 9-31 | Orthogonal Diagonalization")
# ============================================================
def gram_schmidt_31(V):
"""Apply Gram-Schmidt to the columns of V in order; return the matrix whose columns are the resulting orthonormal vectors."""
V = np.array(V, dtype=float)
Q = np.zeros_like(V)
for j in range(V.shape[1]):
w = V[:, j] - Q[:, :j] @ (Q[:, :j].T @ V[:, j])
Q[:, j] = w / np.linalg.norm(w)
return Q
A = np.array([[1, 2], [2, 4]], dtype=float)
B = np.array([[2, 1, 1], [1, 2, 1], [1, 1, 2]], dtype=float)
C = np.fliplr(np.eye(4)) # matrix with 1s on the antidiagonal
# --- Step 1 ---
print_step(1, "Eigenvalues (in increasing order) and eigenspace bases")
check("eigenvalues of A", np.linalg.eigvalsh(A), [0, 5])
check("eigenvalues of B", np.linalg.eigvalsh(B), [1, 1, 4])
check("eigenvalues of C", np.linalg.eigvalsh(C), [-1, -1, 1, 1])
eig_data = [
("A", A, 0, [[2, -1]]), ("A", A, 5, [[1, 2]]),
("B", B, 1, [[1, -1, 0], [1, 0, -1]]), ("B", B, 4, [[1, 1, 1]]),
("C", C, -1, [[1, 0, 0, -1], [0, 1, -1, 0]]), ("C", C, 1, [[1, 0, 0, 1], [0, 1, 1, 0]]),
]
for name, M, lam, basis in eig_data:
for v in np.array(basis, dtype=float):
check(f"{name}v = {lam}v, v = {v}", M @ v, lam * v)
# --- Step 2 ---
print_step(2, "Gram-Schmidt and the orthogonal matrix Q")
QB_E1 = gram_schmidt_31(np.array([[1, -1, 0], [1, 0, -1]], dtype=float).T)
check("E_1 of B after Gram-Schmidt", QB_E1,
np.column_stack([np.array([1, -1, 0]) / np.sqrt(2), np.array([1, 1, -2]) / np.sqrt(6)]))
V_naive = np.column_stack([np.array([1, -1, 0]) / np.sqrt(2), np.array([1, 0, -1]) / np.sqrt(2), np.ones(3) / np.sqrt(3)])
print("\nV^T V when we only normalize, without Gram-Schmidt =\n", V_naive.T @ V_naive)
assert not np.allclose(V_naive.T @ V_naive, np.eye(3))
QA = np.array([[2, 1], [-1, 2]]) / np.sqrt(5)
QB = np.column_stack([QB_E1, np.ones(3) / np.sqrt(3)])
QC = np.array([[1, 0, 1, 0], [0, 1, 0, 1], [0, -1, 0, 1], [-1, 0, 1, 0]]) / np.sqrt(2)
cases = [("A", A, QA, [0, 5]), ("B", B, QB, [1, 1, 4]), ("C", C, QC, [-1, -1, 1, 1])]
for name, M, Q, lams in cases:
n = len(lams)
check(f"Q^T Q = I ({name})", Q.T @ Q, np.eye(n))
check(f"Q^T {name} Q = Λ", Q.T @ M @ Q, np.diag(lams))
# --- Step 3 ---
print_step(3, "Fifth power M^5 = Q Λ^5 Q^T")
expected5 = {
"A": np.array([[625, 1250], [1250, 2500]]),
"B": np.eye(3) + 341 * np.ones((3, 3)),
"C": C,
}
for name, M, Q, lams in cases:
M5 = np.linalg.matrix_power(M, 5)
check(f"{name}^5 (direct multiplication)", M5, expected5[name])
check(f"{name}^5 (Q Λ^5 Q^T)", Q @ np.diag(np.array(lams, dtype=float) ** 5) @ Q.T, expected5[name])============================================================
Exercise 9-31 | Orthogonal Diagonalization
============================================================
▶ Step 1: Eigenvalues (in increasing order) and eigenspace bases
----------------------------------------
[eigenvalues of A]
Computed value: [0. 5.]
Expected value: [0, 5]
check passed
[eigenvalues of B]
Computed value: [1. 1. 4.]
Expected value: [1, 1, 4]
check passed
[eigenvalues of C]
Computed value: [-1. -1. 1. 1.]
Expected value: [-1, -1, 1, 1]
check passed
[Av = 0v, v = [ 2. -1.]]
Computed value: [0. 0.]
Expected value: [ 0. -0.]
check passed
[Av = 5v, v = [1. 2.]]
Computed value: [ 5. 10.]
Expected value: [ 5. 10.]
check passed
[Bv = 1v, v = [ 1. -1. 0.]]
Computed value: [ 1. -1. 0.]
Expected value: [ 1. -1. 0.]
check passed
[Bv = 1v, v = [ 1. 0. -1.]]
Computed value: [ 1. 0. -1.]
Expected value: [ 1. 0. -1.]
check passed
[Bv = 4v, v = [1. 1. 1.]]
Computed value: [4. 4. 4.]
Expected value: [4. 4. 4.]
check passed
[Cv = -1v, v = [ 1. 0. 0. -1.]]
Computed value: [-1. 0. 0. 1.]
Expected value: [-1. -0. -0. 1.]
check passed
[Cv = -1v, v = [ 0. 1. -1. 0.]]
Computed value: [ 0. -1. 1. 0.]
Expected value: [-0. -1. 1. -0.]
check passed
[Cv = 1v, v = [1. 0. 0. 1.]]
Computed value: [1. 0. 0. 1.]
Expected value: [1. 0. 0. 1.]
check passed
[Cv = 1v, v = [0. 1. 1. 0.]]
Computed value: [0. 1. 1. 0.]
Expected value: [0. 1. 1. 0.]
check passed
▶ Step 2: Gram-Schmidt and the orthogonal matrix Q
----------------------------------------
[E_1 of B after Gram-Schmidt]
Computed value: [[ 0.7071 0.4082]
[-0.7071 0.4082]
[ 0. -0.8165]]
Expected value: [[ 0.7071 0.4082]
[-0.7071 0.4082]
[ 0. -0.8165]]
check passed
V^T V when we only normalize, without Gram-Schmidt =
[[ 1. 0.5 -0. ]
[ 0.5 1. -0. ]
[-0. -0. 1. ]]
[Q^T Q = I (A)]
Computed value: [[ 1. -0.]
[-0. 1.]]
Expected value: [[1. 0.]
[0. 1.]]
check passed
[Q^T A Q = Λ]
Computed value: [[0. 0.]
[0. 5.]]
Expected value: [[0 0]
[0 5]]
check passed
[Q^T Q = I (B)]
Computed value: [[ 1. 0. -0.]
[ 0. 1. -0.]
[-0. -0. 1.]]
Expected value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
check passed
[Q^T B Q = Λ]
Computed value: [[ 1. 0. 0.]
[ 0. 1. -0.]
[ 0. -0. 4.]]
Expected value: [[1 0 0]
[0 1 0]
[0 0 4]]
check passed
[Q^T Q = I (C)]
Computed value: [[1. 0. 0. 0.]
[0. 1. 0. 0.]
[0. 0. 1. 0.]
[0. 0. 0. 1.]]
Expected value: [[1. 0. 0. 0.]
[0. 1. 0. 0.]
[0. 0. 1. 0.]
[0. 0. 0. 1.]]
check passed
[Q^T C Q = Λ]
Computed value: [[-1. 0. 0. 0.]
[ 0. -1. 0. 0.]
[ 0. 0. 1. 0.]
[ 0. 0. 0. 1.]]
Expected value: [[-1 0 0 0]
[ 0 -1 0 0]
[ 0 0 1 0]
[ 0 0 0 1]]
check passed
▶ Step 3: Fifth power M^5 = Q Λ^5 Q^T
----------------------------------------
[A^5 (direct multiplication)]
Computed value: [[ 625. 1250.]
[1250. 2500.]]
Expected value: [[ 625 1250]
[1250 2500]]
check passed
[A^5 (Q Λ^5 Q^T)]
Computed value: [[ 625. 1250.]
[1250. 2500.]]
Expected value: [[ 625 1250]
[1250 2500]]
check passed
[B^5 (direct multiplication)]
Computed value: [[342. 341. 341.]
[341. 342. 341.]
[341. 341. 342.]]
Expected value: [[342. 341. 341.]
[341. 342. 341.]
[341. 341. 342.]]
check passed
[B^5 (Q Λ^5 Q^T)]
Computed value: [[342. 341. 341.]
[341. 342. 341.]
[341. 341. 342.]]
Expected value: [[342. 341. 341.]
[341. 342. 341.]
[341. 341. 342.]]
check passed
[C^5 (direct multiplication)]
Computed value: [[0. 0. 0. 1.]
[0. 0. 1. 0.]
[0. 1. 0. 0.]
[1. 0. 0. 0.]]
Expected value: [[0. 0. 0. 1.]
[0. 0. 1. 0.]
[0. 1. 0. 0.]
[1. 0. 0. 0.]]
check passed
[C^5 (Q Λ^5 Q^T)]
Computed value: [[0. 0. 0. 1.]
[0. 0. 1. 0.]
[0. 1. 0. 0.]
[1. 0. 0. 0.]]
Expected value: [[0. 0. 0. 1.]
[0. 0. 1. 0.]
[0. 1. 0. 0.]
[1. 0. 0. 0.]]
check passed
# ============================================================
print_header("Exercise 9-32 | Spectral Decomposition of a 2×2 Matrix")
# ============================================================
q0 = np.array([1, 2]) / np.sqrt(5)
q1 = np.array([2, -1]) / np.sqrt(5)
P0 = np.outer(q0, q0) # |q0⟩⟨q0|
P1 = np.outer(q1, q1) # |q1⟩⟨q1|
# --- Step 1 ---
print_step(1, "Parts 1–2: properties of the projection matrices (using q0, q1 from part 5 as an example)")
for name, P, qa, qb in [("P0", P0, q0, q1), ("P1", P1, q1, q0)]:
check(f"{name}^2 = {name}", P @ P, P)
check(f"{name}^T = {name}", P.T, P)
check(f"{name} leaves its own q unchanged", P @ qa, qa)
check(f"{name} sends the other q to 0", P @ qb, np.zeros(2))
check("P0 P1 = 0", P0 @ P1, np.zeros((2, 2)))
check("P1 P0 = 0", P1 @ P0, np.zeros((2, 2)))
check("P0 + P1 = I_2", P0 + P1, np.eye(2))
# --- Step 2 ---
print_step(2, "Part 3: combination of outer products of the natural basis")
e0, e1 = np.eye(2)
A3 = 3 * np.outer(e0, e0) + (-5) * np.outer(e1, e1)
check("3 e0 e0^T - 5 e1 e1^T", A3, [[3, 0], [0, -5]])
check("its eigenvalues (in increasing order)", np.linalg.eigvalsh(A3), [-5, 3])
# --- Step 3 ---
print_step(3, "Parts 4–5: assemble A from the spectral decomposition and diagonalize it")
A5 = 4 * P0 + (-1) * P1
check("A = 4 q0 q0^T - q1 q1^T", A5, [[0, 2], [2, 3]])
check("A is symmetric", A5.T, A5)
check("A q0 = 4 q0", A5 @ q0, 4 * q0)
check("A q1 = -q1", A5 @ q1, -q1)
Q = np.column_stack([q0, q1])
check("Q^T A Q", Q.T @ A5 @ Q, np.diag([4, -1]))
# --- Step 4 ---
print_step(4, "Part 6: spectral decomposition of A = [[5,-1],[-1,5]]")
A6 = np.array([[5, -1], [-1, 5]], dtype=float)
check("eigenvalues of A", np.linalg.eigvalsh(A6), [4, 6])
u0 = np.array([1, 1]) / np.sqrt(2)
u1 = np.array([1, -1]) / np.sqrt(2)
check("4|q0⟩⟨q0| + 6|q1⟩⟨q1|", 4 * np.outer(u0, u0) + 6 * np.outer(u1, u1), A6)============================================================
Exercise 9-32 | Spectral Decomposition of a 2×2 Matrix
============================================================
▶ Step 1: Parts 1–2: properties of the projection matrices (using q0, q1 from part 5 as an example)
----------------------------------------
[P0^2 = P0]
Computed value: [[0.2 0.4]
[0.4 0.8]]
Expected value: [[0.2 0.4]
[0.4 0.8]]
check passed
[P0^T = P0]
Computed value: [[0.2 0.4]
[0.4 0.8]]
Expected value: [[0.2 0.4]
[0.4 0.8]]
check passed
[P0 leaves its own q unchanged]
Computed value: [0.4472 0.8944]
Expected value: [0.4472 0.8944]
check passed
[P0 sends the other q to 0]
Computed value: [0. 0.]
Expected value: [0. 0.]
check passed
[P1^2 = P1]
Computed value: [[ 0.8 -0.4]
[-0.4 0.2]]
Expected value: [[ 0.8 -0.4]
[-0.4 0.2]]
check passed
[P1^T = P1]
Computed value: [[ 0.8 -0.4]
[-0.4 0.2]]
Expected value: [[ 0.8 -0.4]
[-0.4 0.2]]
check passed
[P1 leaves its own q unchanged]
Computed value: [ 0.8944 -0.4472]
Expected value: [ 0.8944 -0.4472]
check passed
[P1 sends the other q to 0]
Computed value: [0. 0.]
Expected value: [0. 0.]
check passed
[P0 P1 = 0]
Computed value: [[0. 0.]
[0. 0.]]
Expected value: [[0. 0.]
[0. 0.]]
check passed
[P1 P0 = 0]
Computed value: [[0. 0.]
[0. 0.]]
Expected value: [[0. 0.]
[0. 0.]]
check passed
[P0 + P1 = I_2]
Computed value: [[1. 0.]
[0. 1.]]
Expected value: [[1. 0.]
[0. 1.]]
check passed
▶ Step 2: Part 3: combination of outer products of the natural basis
----------------------------------------
[3 e0 e0^T - 5 e1 e1^T]
Computed value: [[ 3. 0.]
[ 0. -5.]]
Expected value: [[3, 0], [0, -5]]
check passed
[its eigenvalues (in increasing order)]
Computed value: [-5. 3.]
Expected value: [-5, 3]
check passed
▶ Step 3: Parts 4–5: assemble A from the spectral decomposition and diagonalize it
----------------------------------------
[A = 4 q0 q0^T - q1 q1^T]
Computed value: [[0. 2.]
[2. 3.]]
Expected value: [[0, 2], [2, 3]]
check passed
[A is symmetric]
Computed value: [[0. 2.]
[2. 3.]]
Expected value: [[0. 2.]
[2. 3.]]
check passed
[A q0 = 4 q0]
Computed value: [1.7889 3.5777]
Expected value: [1.7889 3.5777]
check passed
[A q1 = -q1]
Computed value: [-0.8944 0.4472]
Expected value: [-0.8944 0.4472]
check passed
[Q^T A Q]
Computed value: [[ 4. 0.]
[ 0. -1.]]
Expected value: [[ 4 0]
[ 0 -1]]
check passed
▶ Step 4: Part 6: spectral decomposition of A = [[5,-1],[-1,5]]
----------------------------------------
[eigenvalues of A]
Computed value: [4. 6.]
Expected value: [4, 6]
check passed
[4|q0⟩⟨q0| + 6|q1⟩⟨q1|]
Computed value: [[ 5. -1.]
[-1. 5.]]
Expected value: [[ 5. -1.]
[-1. 5.]]
check passed
# ============================================================
print_header("Exercise 9-33 | Projection Matrices and Spectral Decomposition")
# ============================================================
B = np.array([[1, 0], [0, 1], [1, 1]], dtype=float)
n = np.array([-1, -1, 1], dtype=float)
I3 = np.eye(3)
# --- Step 1 ---
print_step(1, "Part 1: n ⊥ W and the two projection matrices")
check("B^T n = 0 (n ⊥ W)", B.T @ n, [0, 0])
P0 = B @ np.linalg.inv(B.T @ B) @ B.T
P1 = np.outer(n, n) / (n @ n)
check("P0 = Proj_W", P0, np.array([[2, -1, 1], [-1, 2, 1], [1, 1, 2]]) / 3)
check("P1 = Proj_n", P1, np.array([[1, 1, -1], [1, 1, -1], [-1, -1, 1]]) / 3)
for name, P in [("P0", P0), ("P1", P1)]:
check(f"{name}^T = {name}", P.T, P)
check(f"{name}^2 = {name}", P @ P, P)
check("P0 P1 = 0", P0 @ P1, np.zeros((3, 3)))
check("P1 P0 = 0", P1 @ P0, np.zeros((3, 3)))
check("P0 + P1 = I_3", P0 + P1, I3)
# --- Step 2 ---
print_step(2, "Part 2: orthogonal diagonalization of A = P0 + 4 P1")
A = P0 + 4 * P1
check("A", A, [[2, 1, -1], [1, 2, -1], [-1, -1, 2]])
check("eigenvalues of A", np.linalg.eigvalsh(A), [1, 1, 4])
q0 = np.array([1, 0, 1]) / np.sqrt(2)
q1 = np.array([-1, 2, 1]) / np.sqrt(6)
q2 = n / np.linalg.norm(n)
w1 = np.array([0, 1, 1]) - (q0 @ np.array([0, 1, 1])) * q0
check("Gram-Schmidt yields q1", w1 / np.linalg.norm(w1), q1)
Q = np.column_stack([q0, q1, q2])
check("Q^T Q = I_3", Q.T @ Q, I3)
check("Q^T A Q = Λ", Q.T @ A @ Q, np.diag([1, 1, 4]))
# --- Step 3 ---
print_step(3, "Comparison: projection matrix = Σ|q⟩⟨q| over the eigenspace")
check("|q0⟩⟨q0| + |q1⟩⟨q1| = P0", np.outer(q0, q0) + np.outer(q1, q1), P0)
check("|q2⟩⟨q2| = P1", np.outer(q2, q2), P1)============================================================
Exercise 9-33 | Projection Matrices and Spectral Decomposition
============================================================
▶ Step 1: Part 1: n ⊥ W and the two projection matrices
----------------------------------------
[B^T n = 0 (n ⊥ W)]
Computed value: [0. 0.]
Expected value: [0, 0]
check passed
[P0 = Proj_W]
Computed value: [[ 0.6667 -0.3333 0.3333]
[-0.3333 0.6667 0.3333]
[ 0.3333 0.3333 0.6667]]
Expected value: [[ 0.6667 -0.3333 0.3333]
[-0.3333 0.6667 0.3333]
[ 0.3333 0.3333 0.6667]]
check passed
[P1 = Proj_n]
Computed value: [[ 0.3333 0.3333 -0.3333]
[ 0.3333 0.3333 -0.3333]
[-0.3333 -0.3333 0.3333]]
Expected value: [[ 0.3333 0.3333 -0.3333]
[ 0.3333 0.3333 -0.3333]
[-0.3333 -0.3333 0.3333]]
check passed
[P0^T = P0]
Computed value: [[ 0.6667 -0.3333 0.3333]
[-0.3333 0.6667 0.3333]
[ 0.3333 0.3333 0.6667]]
Expected value: [[ 0.6667 -0.3333 0.3333]
[-0.3333 0.6667 0.3333]
[ 0.3333 0.3333 0.6667]]
check passed
[P0^2 = P0]
Computed value: [[ 0.6667 -0.3333 0.3333]
[-0.3333 0.6667 0.3333]
[ 0.3333 0.3333 0.6667]]
Expected value: [[ 0.6667 -0.3333 0.3333]
[-0.3333 0.6667 0.3333]
[ 0.3333 0.3333 0.6667]]
check passed
[P1^T = P1]
Computed value: [[ 0.3333 0.3333 -0.3333]
[ 0.3333 0.3333 -0.3333]
[-0.3333 -0.3333 0.3333]]
Expected value: [[ 0.3333 0.3333 -0.3333]
[ 0.3333 0.3333 -0.3333]
[-0.3333 -0.3333 0.3333]]
check passed
[P1^2 = P1]
Computed value: [[ 0.3333 0.3333 -0.3333]
[ 0.3333 0.3333 -0.3333]
[-0.3333 -0.3333 0.3333]]
Expected value: [[ 0.3333 0.3333 -0.3333]
[ 0.3333 0.3333 -0.3333]
[-0.3333 -0.3333 0.3333]]
check passed
[P0 P1 = 0]
Computed value: [[ 0. 0. -0.]
[ 0. 0. -0.]
[ 0. 0. -0.]]
Expected value: [[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]]
check passed
[P1 P0 = 0]
Computed value: [[ 0. 0. 0.]
[ 0. 0. 0.]
[-0. -0. -0.]]
Expected value: [[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]]
check passed
[P0 + P1 = I_3]
Computed value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
Expected value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
check passed
▶ Step 2: Part 2: orthogonal diagonalization of A = P0 + 4 P1
----------------------------------------
[A]
Computed value: [[ 2. 1. -1.]
[ 1. 2. -1.]
[-1. -1. 2.]]
Expected value: [[2, 1, -1], [1, 2, -1], [-1, -1, 2]]
check passed
[eigenvalues of A]
Computed value: [1. 1. 4.]
Expected value: [1, 1, 4]
check passed
[Gram-Schmidt yields q1]
Computed value: [-0.4082 0.8165 0.4082]
Expected value: [-0.4082 0.8165 0.4082]
check passed
[Q^T Q = I_3]
Computed value: [[ 1. -0. 0.]
[-0. 1. -0.]
[ 0. -0. 1.]]
Expected value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
check passed
[Q^T A Q = Λ]
Computed value: [[ 1. -0. 0.]
[-0. 1. 0.]
[ 0. 0. 4.]]
Expected value: [[1 0 0]
[0 1 0]
[0 0 4]]
check passed
▶ Step 3: Comparison: projection matrix = Σ|q⟩⟨q| over the eigenspace
----------------------------------------
[|q0⟩⟨q0| + |q1⟩⟨q1| = P0]
Computed value: [[ 0.6667 -0.3333 0.3333]
[-0.3333 0.6667 0.3333]
[ 0.3333 0.3333 0.6667]]
Expected value: [[ 0.6667 -0.3333 0.3333]
[-0.3333 0.6667 0.3333]
[ 0.3333 0.3333 0.6667]]
check passed
[|q2⟩⟨q2| = P1]
Computed value: [[ 0.3333 0.3333 -0.3333]
[ 0.3333 0.3333 -0.3333]
[-0.3333 -0.3333 0.3333]]
Expected value: [[ 0.3333 0.3333 -0.3333]
[ 0.3333 0.3333 -0.3333]
[-0.3333 -0.3333 0.3333]]
check passed
# ============================================================
print_header("Exercise 9-34 | Computing Powers via the Spectral Decomposition")
# ============================================================
A = np.array([[1, 2], [2, 4]], dtype=float)
B = np.array([[2, 1, 1], [1, 2, 1], [1, 1, 2]], dtype=float)
C = np.fliplr(np.eye(4))
J = np.ones((3, 3))
QA = np.array([[2, 1], [-1, 2]]) / np.sqrt(5)
QB = np.column_stack([np.array([1, -1, 0]) / np.sqrt(2),
np.array([1, 1, -2]) / np.sqrt(6),
np.ones(3) / np.sqrt(3)])
QC = np.array([[1, 0, 1, 0], [0, 1, 0, 1], [0, -1, 0, 1], [-1, 0, 1, 0]]) / np.sqrt(2)
cases = [("A", A, QA, [0, 5]), ("B", B, QB, [1, 1, 4]), ("C", C, QC, [-1, -1, 1, 1])]
def spectral_sum_34(Q, lams, power=1):
"""Return Σ λ_i^power |q_i⟩⟨q_i|."""
return sum(lam ** power * np.outer(Q[:, i], Q[:, i]) for i, lam in enumerate(lams))
# --- Step 1 ---
print_step(1, "Part 1: spectral decomposition M = Σ λ_i |q_i⟩⟨q_i|")
for name, M, Q, lams in cases:
check(f"Σ λ_i |q_i⟩⟨q_i| = {name}", spectral_sum_34(Q, lams), M)
check("the λ=5 term of A, |q1⟩⟨q1|", np.outer(QA[:, 1], QA[:, 1]), np.array([[1, 2], [2, 4]]) / 5)
PB1 = np.outer(QB[:, 0], QB[:, 0]) + np.outer(QB[:, 1], QB[:, 1])
check("B: projection onto E_1 = I - J/3", PB1, np.eye(3) - J / 3)
check("B: projection onto E_4 = J/3", np.outer(QB[:, 2], QB[:, 2]), J / 3)
# Switch to another orthonormal basis of E_1: the individual terms change, but their sum does not
v0 = np.array([1, 0, -1]) / np.sqrt(2)
w = np.array([0, 1, -1]) - (v0 @ np.array([0, 1, -1])) * v0
v1 = w / np.linalg.norm(w)
check("B: sum of projections for another basis of E_1", np.outer(v0, v0) + np.outer(v1, v1), PB1)
Pm = np.outer(QC[:, 0], QC[:, 0]) + np.outer(QC[:, 1], QC[:, 1])
Pp = np.outer(QC[:, 2], QC[:, 2]) + np.outer(QC[:, 3], QC[:, 3])
check("C: P_- = (I - C)/2", Pm, (np.eye(4) - C) / 2)
check("C: P_+ = (I + C)/2", Pp, (np.eye(4) + C) / 2)
# --- Step 2 ---
print_step(2, "Part 2: M^5 = Σ λ_i^5 |q_i⟩⟨q_i|")
expected5 = {"A": [[625, 1250], [1250, 2500]], "B": np.eye(3) + 341 * J, "C": C}
for name, M, Q, lams in cases:
S5 = spectral_sum_34(Q, lams, power=5)
check(f"Σ λ_i^5 |q_i⟩⟨q_i| ({name})", S5, expected5[name])
check(f"agrees with direct computation of {name}^5", np.linalg.matrix_power(M, 5), S5)============================================================
Exercise 9-34 | Computing Powers via the Spectral Decomposition
============================================================
▶ Step 1: Part 1: spectral decomposition M = Σ λ_i |q_i⟩⟨q_i|
----------------------------------------
[Σ λ_i |q_i⟩⟨q_i| = A]
Computed value: [[1. 2.]
[2. 4.]]
Expected value: [[1. 2.]
[2. 4.]]
check passed
[Σ λ_i |q_i⟩⟨q_i| = B]
Computed value: [[2. 1. 1.]
[1. 2. 1.]
[1. 1. 2.]]
Expected value: [[2. 1. 1.]
[1. 2. 1.]
[1. 1. 2.]]
check passed
[Σ λ_i |q_i⟩⟨q_i| = C]
Computed value: [[0. 0. 0. 1.]
[0. 0. 1. 0.]
[0. 1. 0. 0.]
[1. 0. 0. 0.]]
Expected value: [[0. 0. 0. 1.]
[0. 0. 1. 0.]
[0. 1. 0. 0.]
[1. 0. 0. 0.]]
check passed
[the λ=5 term of A, |q1⟩⟨q1|]
Computed value: [[0.2 0.4]
[0.4 0.8]]
Expected value: [[0.2 0.4]
[0.4 0.8]]
check passed
[B: projection onto E_1 = I - J/3]
Computed value: [[ 0.6667 -0.3333 -0.3333]
[-0.3333 0.6667 -0.3333]
[-0.3333 -0.3333 0.6667]]
Expected value: [[ 0.6667 -0.3333 -0.3333]
[-0.3333 0.6667 -0.3333]
[-0.3333 -0.3333 0.6667]]
check passed
[B: projection onto E_4 = J/3]
Computed value: [[0.3333 0.3333 0.3333]
[0.3333 0.3333 0.3333]
[0.3333 0.3333 0.3333]]
Expected value: [[0.3333 0.3333 0.3333]
[0.3333 0.3333 0.3333]
[0.3333 0.3333 0.3333]]
check passed
[B: sum of projections for another basis of E_1]
Computed value: [[ 0.6667 -0.3333 -0.3333]
[-0.3333 0.6667 -0.3333]
[-0.3333 -0.3333 0.6667]]
Expected value: [[ 0.6667 -0.3333 -0.3333]
[-0.3333 0.6667 -0.3333]
[-0.3333 -0.3333 0.6667]]
check passed
[C: P_- = (I - C)/2]
Computed value: [[ 0.5 0. 0. -0.5]
[ 0. 0.5 -0.5 0. ]
[ 0. -0.5 0.5 -0. ]
[-0.5 0. -0. 0.5]]
Expected value: [[ 0.5 0. 0. -0.5]
[ 0. 0.5 -0.5 0. ]
[ 0. -0.5 0.5 0. ]
[-0.5 0. 0. 0.5]]
check passed
[C: P_+ = (I + C)/2]
Computed value: [[0.5 0. 0. 0.5]
[0. 0.5 0.5 0. ]
[0. 0.5 0.5 0. ]
[0.5 0. 0. 0.5]]
Expected value: [[0.5 0. 0. 0.5]
[0. 0.5 0.5 0. ]
[0. 0.5 0.5 0. ]
[0.5 0. 0. 0.5]]
check passed
▶ Step 2: Part 2: M^5 = Σ λ_i^5 |q_i⟩⟨q_i|
----------------------------------------
[Σ λ_i^5 |q_i⟩⟨q_i| (A)]
Computed value: [[ 625. 1250.]
[1250. 2500.]]
Expected value: [[625, 1250], [1250, 2500]]
check passed
[agrees with direct computation of A^5]
Computed value: [[ 625. 1250.]
[1250. 2500.]]
Expected value: [[ 625. 1250.]
[1250. 2500.]]
check passed
[Σ λ_i^5 |q_i⟩⟨q_i| (B)]
Computed value: [[342. 341. 341.]
[341. 342. 341.]
[341. 341. 342.]]
Expected value: [[342. 341. 341.]
[341. 342. 341.]
[341. 341. 342.]]
check passed
[agrees with direct computation of B^5]
Computed value: [[342. 341. 341.]
[341. 342. 341.]
[341. 341. 342.]]
Expected value: [[342. 341. 341.]
[341. 342. 341.]
[341. 341. 342.]]
check passed
[Σ λ_i^5 |q_i⟩⟨q_i| (C)]
Computed value: [[0. 0. 0. 1.]
[0. 0. 1. 0.]
[0. 1. 0. 0.]
[1. 0. 0. 0.]]
Expected value: [[0. 0. 0. 1.]
[0. 0. 1. 0.]
[0. 1. 0. 0.]
[1. 0. 0. 0.]]
check passed
[agrees with direct computation of C^5]
Computed value: [[0. 0. 0. 1.]
[0. 0. 1. 0.]
[0. 1. 0. 0.]
[1. 0. 0. 0.]]
Expected value: [[0. 0. 0. 1.]
[0. 0. 1. 0.]
[0. 1. 0. 0.]
[1. 0. 0. 0.]]
check passed
# ============================================================
print_header("Exercise 9-36 | The Rayleigh Quotient of a 2×2 Matrix")
# ============================================================
A = np.array([[3, 2], [2, 0]], dtype=float)
def rayleigh_36(x):
"""Rayleigh quotient R_A(x) = x^T A x / x^T x."""
x = np.asarray(x, dtype=float)
return (x @ A @ x) / (x @ x)
# --- Step 1 ---
print_step(1, "Part 1: eigenvalues and eigenvectors")
check("eigenvalues of A (in increasing order)", np.linalg.eigvalsh(A), [-1, 4])
q0 = np.array([1, -2]) / np.sqrt(5)
q1 = np.array([2, 1]) / np.sqrt(5)
check("A q0 = -q0", A @ q0, -q0)
check("A q1 = 4 q1", A @ q1, 4 * q1)
check("spectral decomposition -|q0⟩⟨q0| + 4|q1⟩⟨q1|", -np.outer(q0, q0) + 4 * np.outer(q1, q1), A)
# --- Step 2 ---
print_step(2, "Part 2: homogeneity R_A(c x) = R_A(x)")
x = np.array([0.7, -1.3])
for c in (-3.0, 0.5, 10.0):
check(f"R_A({c} x) = R_A(x)", rayleigh_36(c * x), rayleigh_36(x))
# --- Step 3 ---
print_step(3, "Part 3: scan the directions on the unit circle")
theta = np.linspace(0, np.pi, 200001)
U = np.vstack([np.cos(theta), np.sin(theta)]) # each column is a unit vector
R = 3 * U[0] ** 2 + 4 * U[0] * U[1]
check("max R_A", R.max(), 4)
check("min R_A", R.min(), -1)
u_max = U[:, R.argmax()]
u_min = U[:, R.argmin()]
check("direction of the maximum ∥ (2,1) (|det(u, (2,1))| < 1e-4, limited by the grid spacing)", abs(u_max[0] * 1 - u_max[1] * 2) < 1e-4, True)
check("direction of the minimum ∥ (1,-2) (|det(u, (1,-2))| < 1e-4, limited by the grid spacing)", abs(u_min[0] * (-2) - u_min[1] * 1) < 1e-4, True)
check("R_A((2,1)) = 4", rayleigh_36([2, 1]), 4)
check("R_A((1,-2)) = -1", rayleigh_36([1, -2]), -1)
# --- Step 4 ---
print_step(4, "Part 4: q_A = R_A on unit vectors")
for u in (q1, -q1, q0, -q0):
check(f"q_A({u}) = R_A({u})", u @ A @ u, rayleigh_36(u))
rng = np.random.default_rng(36)
X = rng.normal(size=(1000, 2))
vals = np.array([rayleigh_36(v) for v in X])
assert vals.min() >= -1 - 1e-12 and vals.max() <= 4 + 1e-12
print("\nR_A of all 1000 random vectors lies in [-1, 4]: check passed")============================================================
Exercise 9-36 | The Rayleigh Quotient of a 2×2 Matrix
============================================================
▶ Step 1: Part 1: eigenvalues and eigenvectors
----------------------------------------
[eigenvalues of A (in increasing order)]
Computed value: [-1. 4.]
Expected value: [-1, 4]
check passed
[A q0 = -q0]
Computed value: [-0.4472 0.8944]
Expected value: [-0.4472 0.8944]
check passed
[A q1 = 4 q1]
Computed value: [3.5777 1.7889]
Expected value: [3.5777 1.7889]
check passed
[spectral decomposition -|q0⟩⟨q0| + 4|q1⟩⟨q1|]
Computed value: [[3. 2.]
[2. 0.]]
Expected value: [[3. 2.]
[2. 0.]]
check passed
▶ Step 2: Part 2: homogeneity R_A(c x) = R_A(x)
----------------------------------------
[R_A(-3.0 x) = R_A(x)]
Computed value: -0.9954128440366973
Expected value: -0.9954128440366972
check passed
[R_A(0.5 x) = R_A(x)]
Computed value: -0.9954128440366972
Expected value: -0.9954128440366972
check passed
[R_A(10.0 x) = R_A(x)]
Computed value: -0.9954128440366973
Expected value: -0.9954128440366972
check passed
▶ Step 3: Part 3: scan the directions on the unit circle
----------------------------------------
[max R_A]
Computed value: 3.999999999905701
Expected value: 4
check passed
[min R_A]
Computed value: -0.9999999999057013
Expected value: -1
check passed
[direction of the maximum ∥ (2,1) (|det(u, (2,1))| < 1e-4, limited by the grid spacing)]
Computed value: True
Expected value: True
check passed
[direction of the minimum ∥ (1,-2) (|det(u, (1,-2))| < 1e-4, limited by the grid spacing)]
Computed value: True
Expected value: True
check passed
[R_A((2,1)) = 4]
Computed value: 4.0
Expected value: 4
check passed
[R_A((1,-2)) = -1]
Computed value: -1.0
Expected value: -1
check passed
▶ Step 4: Part 4: q_A = R_A on unit vectors
----------------------------------------
[q_A([0.8944 0.4472]) = R_A([0.8944 0.4472])]
Computed value: 3.9999999999999996
Expected value: 4.0
check passed
[q_A([-0.8944 -0.4472]) = R_A([-0.8944 -0.4472])]
Computed value: 3.9999999999999996
Expected value: 4.0
check passed
[q_A([ 0.4472 -0.8944]) = R_A([ 0.4472 -0.8944])]
Computed value: -0.9999999999999999
Expected value: -1.0
check passed
[q_A([-0.4472 0.8944]) = R_A([-0.4472 0.8944])]
Computed value: -0.9999999999999999
Expected value: -1.0
check passed
R_A of all 1000 random vectors lies in [-1, 4]: check passed
Complex Inner Product Spaces¶
We now turn to complex inner product spaces. Unless stated otherwise, the inner product on is always the standard inner product , which is linear in the second argument and conjugate linear in the first (see Note).
# ============================================================
print_header("Exercise 9-38 | Length of a Complex Vector")
# ============================================================
# --- Step 1 ---
print_step(1, "Keep the real inner product formula x^T x")
x = np.array([1, 1j])
check("x^T x", x @ x, 0)
# --- Step 2 ---
print_step(2, "Complex inner product x^* x")
check("x^* x", np.vdot(x, x), 2) # np.vdot conjugates its first argument
check("||x||", np.linalg.norm(x), np.sqrt(2))
# --- Step 3 ---
print_step(3, "Map C^n to R^{2n}: preserves length and addition; random check of the triangle inequality")
def phi_38(z):
return np.concatenate([z.real, z.imag])
rng = np.random.default_rng(38)
ok_norm, ok_add, ok_tri = True, True, True
for _ in range(1000):
n = rng.integers(1, 6)
a = rng.normal(size=n) + 1j * rng.normal(size=n)
b = rng.normal(size=n) + 1j * rng.normal(size=n)
ok_norm &= np.isclose(np.linalg.norm(a), np.linalg.norm(phi_38(a)))
ok_add &= np.allclose(phi_38(a + b), phi_38(a) + phi_38(b))
ok_tri &= np.linalg.norm(a + b) <= np.linalg.norm(a) + np.linalg.norm(b) + 1e-12
check("||z|| = ||phi(z)|| (1000 trials)", ok_norm, True)
check("phi(x+y) = phi(x)+phi(y) (1000 trials)", ok_add, True)
check("Triangle inequality (1000 trials)", ok_tri, True)============================================================
Exercise 9-38 | Length of a Complex Vector
============================================================
▶ Step 1: Keep the real inner product formula x^T x
----------------------------------------
[x^T x]
Computed value: 0j
Expected value: 0
check passed
▶ Step 2: Complex inner product x^* x
----------------------------------------
[x^* x]
Computed value: (2+0j)
Expected value: 2
check passed
[||x||]
Computed value: 1.4142135623730951
Expected value: 1.4142135623730951
check passed
▶ Step 3: Map C^n to R^{2n}: preserves length and addition; random check of the triangle inequality
----------------------------------------
[||z|| = ||phi(z)|| (1000 trials)]
Computed value: True
Expected value: True
check passed
[phi(x+y) = phi(x)+phi(y) (1000 trials)]
Computed value: True
Expected value: True
check passed
[Triangle inequality (1000 trials)]
Computed value: True
Expected value: True
check passed
# ============================================================
print_header("Exercise 9-39 | The Pythagorean Theorem in Complex Space")
# ============================================================
# --- Step 1 ---
print_step(1, "Find k from <u,v> = (1-i)(1+k) = 0")
k = -1
u = np.array([1, 1 + 1j])
v = np.array([1 - 1j, k])
check("<u,v> = u^* v", np.vdot(u, v), 0)
k_wrong = -(1 - 1j) / (1 + 1j)
check("k obtained by mistakenly using u^T v", k_wrong, 1j)
check("Then u^* v is not 0", np.isclose(np.vdot(u, np.array([1 - 1j, k_wrong])), 0), False)
# --- Step 2 ---
print_step(2, "Legs and length of the hypotenuse")
check("||u||^2", np.vdot(u, u).real, 3)
check("||v||^2", np.vdot(v, v).real, 3)
check("u - v", u - v, [1j, 2 + 1j])
check("||u - v||", np.linalg.norm(u - v), np.sqrt(6))
check("||u + v||", np.linalg.norm(u + v), np.sqrt(6))
# --- Step 3 ---
print_step(3, "||u-v||^2 = ||u||^2 + ||v||^2 - 2 Re<u,v> (random vectors)")
rng = np.random.default_rng(39)
a = rng.normal(size=4) + 1j * rng.normal(size=4)
b = rng.normal(size=4) + 1j * rng.normal(size=4)
lhs = np.linalg.norm(a - b) ** 2
rhs = np.linalg.norm(a) ** 2 + np.linalg.norm(b) ** 2 - 2 * np.vdot(a, b).real
check("Expansion", lhs, rhs)
uu, vv = np.array([1, 0]), np.array([1j, 0])
check("Counterexample to the converse: <u,v>", np.vdot(uu, vv), 1j)
check("Counterexample to the converse: ||u-v||^2", np.linalg.norm(uu - vv) ** 2, 2)============================================================
Exercise 9-39 | The Pythagorean Theorem in Complex Space
============================================================
▶ Step 1: Find k from <u,v> = (1-i)(1+k) = 0
----------------------------------------
[<u,v> = u^* v]
Computed value: 0j
Expected value: 0
check passed
[k obtained by mistakenly using u^T v]
Computed value: 1j
Expected value: 1j
check passed
[Then u^* v is not 0]
Computed value: False
Expected value: False
check passed
▶ Step 2: Legs and length of the hypotenuse
----------------------------------------
[||u||^2]
Computed value: 3.0
Expected value: 3
check passed
[||v||^2]
Computed value: 3.0
Expected value: 3
check passed
[u - v]
Computed value: [0.+1.j 2.+1.j]
Expected value: [1j, (2+1j)]
check passed
[||u - v||]
Computed value: 2.449489742783178
Expected value: 2.449489742783178
check passed
[||u + v||]
Computed value: 2.449489742783178
Expected value: 2.449489742783178
check passed
▶ Step 3: ||u-v||^2 = ||u||^2 + ||v||^2 - 2 Re<u,v> (random vectors)
----------------------------------------
[Expansion]
Computed value: 12.190631774138552
Expected value: 12.190631774138549
check passed
[Counterexample to the converse: <u,v>]
Computed value: 1j
Expected value: 1j
check passed
[Counterexample to the converse: ||u-v||^2]
Computed value: 2.0000000000000004
Expected value: 2
check passed
# ============================================================
print_header("Exercise 9-40 | Unitary Matrices of Order 2")
# ============================================================
# --- Step 1 ---
print_step(1, "Unit vectors orthogonal to (1,0): (0, e^{i phi})")
e0 = np.array([1, 0])
for phi in [0, np.pi / 2, np.pi / 4, 2.0]:
x = np.array([0, np.exp(1j * phi)])
check(f"phi={phi:.4f}: (<e0,x>, ||x||)", [np.vdot(e0, x), np.linalg.norm(x)], [0, 1])
# --- Step 2 ---
print_step(2, "Unitary matrices of the form [[1,z],[0,w]]: z=0, |w|=1")
U2 = np.array([[1, 0], [0, np.exp(0.7j)]])
check("diag(1,e^{0.7i})^* diag(1,e^{0.7i})", U2.conj().T @ U2, np.eye(2))
U2_bad = np.array([[1, 0.1], [0, np.sqrt(1 - 0.01)]]) # with z ≠ 0, column 1 is still a unit vector
check("Not unitary when z=0.1", np.allclose(U2_bad.conj().T @ U2_bad, np.eye(2)), False)
# --- Step 3 ---
print_step(3, "Parametrization: every choice of z,w,theta gives a unitary matrix; every unitary matrix has this form")
def U_40(z, w, lam):
return np.array([[z, -np.conj(w) * lam], [w, np.conj(z) * lam]])
rng = np.random.default_rng(40)
zw = rng.normal(size=2) + 1j * rng.normal(size=2)
z, w = zw / np.linalg.norm(zw)
lam = np.exp(1j * 1.234)
U = U_40(z, w, lam)
check("U^* U", U.conj().T @ U, np.eye(2))
M = rng.normal(size=(2, 2)) + 1j * rng.normal(size=(2, 2))
V, _ = np.linalg.qr(M) # random unitary matrix
z1, w1, lam1 = V[0, 0], V[1, 0], np.linalg.det(V)
check("Column 1 of the random unitary matrix = lambda (-conj(w), conj(z))", V[:, 1], lam1 * np.array([-np.conj(w1), np.conj(z1)]))
check("Random unitary matrix = U(z,w,lambda)", U_40(z1, w1, lam1), V)
# --- Step 4 ---
print_step(4, "Length preservation: ||U u|| = 1")
uvec = rng.normal(size=2) + 1j * rng.normal(size=2)
uvec /= np.linalg.norm(uvec)
check("||U u||", np.linalg.norm(U @ uvec), 1)
# --- Step 5 ---
print_step(5, "det(U) = lambda")
check("det(U)", np.linalg.det(U), lam)
check("det = 1 when lambda=1", np.linalg.det(U_40(z, w, 1)), 1)
# --- Step 6 ---
print_step(6, "A real unitary matrix is orthogonal: lambda = ±1 gives rotations and reflections")
t = 0.9
R = U_40(np.cos(t), np.sin(t), 1).real
F = U_40(np.cos(t), np.sin(t), -1).real
check("Rotation R^T R", R.T @ R, np.eye(2))
check("Reflection F^T F", F.T @ F, np.eye(2))
check("(det R, det F)", [np.linalg.det(R), np.linalg.det(F)], [1, -1])============================================================
Exercise 9-40 | Unitary Matrices of Order 2
============================================================
▶ Step 1: Unit vectors orthogonal to (1,0): (0, e^{i phi})
----------------------------------------
[phi=0.0000: (<e0,x>, ||x||)]
Computed value: [np.complex128(0j), np.float64(1.0)]
Expected value: [0, 1]
check passed
[phi=1.5708: (<e0,x>, ||x||)]
Computed value: [np.complex128(0j), np.float64(1.0)]
Expected value: [0, 1]
check passed
[phi=0.7854: (<e0,x>, ||x||)]
Computed value: [np.complex128(0j), np.float64(1.0)]
Expected value: [0, 1]
check passed
[phi=2.0000: (<e0,x>, ||x||)]
Computed value: [np.complex128(0j), np.float64(1.0)]
Expected value: [0, 1]
check passed
▶ Step 2: Unitary matrices of the form [[1,z],[0,w]]: z=0, |w|=1
----------------------------------------
[diag(1,e^{0.7i})^* diag(1,e^{0.7i})]
Computed value: [[1.+0.j 0.+0.j]
[0.+0.j 1.+0.j]]
Expected value: [[1. 0.]
[0. 1.]]
check passed
[Not unitary when z=0.1]
Computed value: False
Expected value: False
check passed
▶ Step 3: Parametrization: every choice of z,w,theta gives a unitary matrix; every unitary matrix has this form
----------------------------------------
[U^* U]
Computed value: [[ 1.+0.j -0.+0.j]
[-0.+0.j 1.+0.j]]
Expected value: [[1. 0.]
[0. 1.]]
check passed
[Column 1 of the random unitary matrix = lambda (-conj(w), conj(z))]
Computed value: [ 0.1259+0.2737j -0.9527+0.0398j]
Expected value: [ 0.1259+0.2737j -0.9527+0.0398j]
check passed
[Random unitary matrix = U(z,w,lambda)]
Computed value: [[-0.7145-0.6315j 0.1259+0.2737j]
[-0.2703+0.1331j -0.9527+0.0398j]]
Expected value: [[-0.7145-0.6315j 0.1259+0.2737j]
[-0.2703+0.1331j -0.9527+0.0398j]]
check passed
▶ Step 4: Length preservation: ||U u|| = 1
----------------------------------------
[||U u||]
Computed value: 0.9999999999999999
Expected value: 1
check passed
▶ Step 5: det(U) = lambda
----------------------------------------
[det(U)]
Computed value: (0.33046510807172985+0.943818209374634j)
Expected value: (0.33046510807172985+0.9438182093746337j)
check passed
[det = 1 when lambda=1]
Computed value: (1.0000000000000002+0j)
Expected value: 1
check passed
▶ Step 6: A real unitary matrix is orthogonal: lambda = ±1 gives rotations and reflections
----------------------------------------
[Rotation R^T R]
Computed value: [[ 1. -0.]
[-0. 1.]]
Expected value: [[1. 0.]
[0. 1.]]
check passed
[Reflection F^T F]
Computed value: [[1. 0.]
[0. 1.]]
Expected value: [[1. 0.]
[0. 1.]]
check passed
[(det R, det F)]
Computed value: [np.float64(0.9999999999999999), np.float64(-0.9999999999999999)]
Expected value: [1, -1]
check passed
# ============================================================
print_header("Exercise 9-41 | The Polarization Identity")
# ============================================================
nsq = lambda z: np.vdot(z, z).real # ||z||^2
def polar_book_41(x, y):
"""Convention of this book (linear in the second argument): (1/4) Σ_{k=0}^{3} i^{-k} ||x + i^k y||^2"""
return sum((1j) ** (-k) * nsq(x + (1j) ** k * y) for k in range(4)) / 4
def polar_orig_41(x, y):
"""Formula of the original manuscript: (1/4)(||x+y||^2 - ||x-y||^2 + i||x+iy||^2 - i||x-iy||^2)"""
return (nsq(x + y) - nsq(x - y) + 1j * nsq(x + 1j * y) - 1j * nsq(x - 1j * y)) / 4
rng = np.random.default_rng(41)
# --- Step 1 ---
print_step(1, "Real inner product space: <x,y> = (||x+y||^2 - ||x-y||^2)/4")
xr, yr = rng.normal(size=5), rng.normal(size=5)
check("Real polarization", (nsq(xr + yr) - nsq(xr - yr)) / 4, xr @ yr)
# --- Step 2 ---
print_step(2, "Counterexample x=(1,0), y=(i,0)")
x = np.array([1, 0]); y = np.array([1j, 0])
check("<x,y> = x^* y", np.vdot(x, y), 1j)
check("Right-hand side of the real formula", (nsq(x + y) - nsq(x - y)) / 4, 0)
# --- Step 3 ---
print_step(3, "Complex polarization identity under the convention of this book")
check("Counterexample vectors: formula of this book", polar_book_41(x, y), 1j)
check("Counterexample vectors: formula of the original manuscript", polar_orig_41(x, y), -1j)
ok_book, ok_orig = True, True
for _ in range(200):
a = rng.normal(size=3) + 1j * rng.normal(size=3)
b = rng.normal(size=3) + 1j * rng.normal(size=3)
ok_book &= np.isclose(polar_book_41(a, b), np.vdot(a, b)) # = <a,b>
ok_orig &= np.isclose(polar_orig_41(a, b), np.vdot(b, a)) # = <b,a> = conj<a,b>
check("Formula of this book = <x,y> (200 trials)", ok_book, True)
check("Formula of the original manuscript = <y,x> = conj<x,y> (200 trials)", ok_orig, True)
# --- Step 4 ---
print_step(4, "||x + c y||^2 = S + 2 Re(c <x,y>), c = 1, -1, i, -i")
a = rng.normal(size=3) + 1j * rng.normal(size=3)
b = rng.normal(size=3) + 1j * rng.normal(size=3)
S, w = nsq(a) + nsq(b), np.vdot(a, b)
for c in [1, -1, 1j, -1j]:
check(f"c = {c}", nsq(a + c * b), S + 2 * (c * w).real)============================================================
Exercise 9-41 | The Polarization Identity
============================================================
▶ Step 1: Real inner product space: <x,y> = (||x+y||^2 - ||x-y||^2)/4
----------------------------------------
[Real polarization]
Computed value: -0.2789963481704243
Expected value: -0.2789963481704246
check passed
▶ Step 2: Counterexample x=(1,0), y=(i,0)
----------------------------------------
[<x,y> = x^* y]
Computed value: 1j
Expected value: 1j
check passed
[Right-hand side of the real formula]
Computed value: 0.0
Expected value: 0
check passed
▶ Step 3: Complex polarization identity under the convention of this book
----------------------------------------
[Counterexample vectors: formula of this book]
Computed value: 1j
Expected value: 1j
check passed
[Counterexample vectors: formula of the original manuscript]
Computed value: -1j
Expected value: (-0-1j)
check passed
[Formula of this book = <x,y> (200 trials)]
Computed value: True
Expected value: True
check passed
[Formula of the original manuscript = <y,x> = conj<x,y> (200 trials)]
Computed value: True
Expected value: True
check passed
▶ Step 4: ||x + c y||^2 = S + 2 Re(c <x,y>), c = 1, -1, i, -i
----------------------------------------
[c = 1]
Computed value: 30.402869408160406
Expected value: 30.402869408160402
check passed
[c = -1]
Computed value: 12.106362181738684
Expected value: 12.106362181738685
check passed
[c = 1j]
Computed value: 6.678720945713077
Expected value: 6.678720945713078
check passed
[c = (-0-1j)]
Computed value: 35.830510644186006
Expected value: 35.830510644186006
check passed
# ============================================================
print_header("Exercise 9-42 | Complex Projections and the QR Decomposition")
# ============================================================
rng = np.random.default_rng(42)
H_ = lambda M: M.conj().T
# --- Step 1 ---
print_step(1, "P = u u^*: P^* = P, P^2 = P, Im ⟂ Ker, direct sum decomposition")
n = 4
u = rng.normal(size=n) + 1j * rng.normal(size=n)
u /= np.linalg.norm(u)
P = np.outer(u, u.conj()) # |u><u|
check("P^* = P", H_(P), P)
check("P^2 = P", P @ P, P)
x = rng.normal(size=n) + 1j * rng.normal(size=n)
check("P x = <u,x> u", P @ x, np.vdot(u, x) * u)
check("x - Px ∈ Ker(P)", P @ (x - P @ x), np.zeros(n))
check("<Px, x - Px> = 0", np.vdot(P @ x, x - P @ x), 0)
check("rank(P)", np.linalg.matrix_rank(P), 1)
# --- Step 2 ---
print_step(2, "Projection onto a k-dimensional subspace: B (B^*B)^{-1} B^* = Q Q^*")
k = 2
B = rng.normal(size=(n, k)) + 1j * rng.normal(size=(n, k))
PW = B @ np.linalg.inv(H_(B) @ B) @ H_(B)
Qw, _ = np.linalg.qr(B)
check("B(B^*B)^{-1}B^* = Q Q^*", PW, Qw @ H_(Qw))
check("P_W^* = P_W", H_(PW), PW)
check("P_W^2 = P_W", PW @ PW, PW)
check("P_W B = B (vectors in W are fixed)", PW @ B, B)
check("B^*(x - P_W x) = 0 (x - P_W x ∈ W^⟂)", H_(B) @ (x - PW @ x), np.zeros(k))
dists = [np.linalg.norm(x - B @ c) for c in rng.normal(size=(200, k)) + 1j * rng.normal(size=(200, k))]
check("P_W x is the best approximation (200 vectors in W)", min(dists) >= np.linalg.norm(x - PW @ x) - 1e-12, True)
# --- Step 3 ---
print_step(3, "Gram-Schmidt: v0 = (i,1), v1 = (0,i)")
v0 = np.array([1j, 1]); v1 = np.array([0, 1j])
r00 = np.linalg.norm(v0); q0 = v0 / r00
r01 = np.vdot(q0, v1) # q0^* v1
w1 = v1 - r01 * q0
r11 = np.linalg.norm(w1); q1 = w1 / r11
check("r00", r00, np.sqrt(2))
check("r01 = <q0, v1>", r01, 1j / np.sqrt(2))
check("w1", w1, [0.5, 0.5j])
check("q1", q1, np.array([1, 1j]) / np.sqrt(2))
check("<q0, q1>", np.vdot(q0, q1), 0)
# --- Step 4 ---
print_step(4, "A = Q R, where the diagonal of R is positive real")
A = np.array([[1j, 0], [1, 1j]])
Q = np.column_stack([q0, q1])
R = np.array([[r00, r01], [0, r11]])
check("Q", Q, np.array([[1j, 1], [1, 1j]]) / np.sqrt(2))
check("R", R, [[np.sqrt(2), 1j / np.sqrt(2)], [0, 1 / np.sqrt(2)]])
check("Q^* Q", H_(Q) @ Q, np.eye(2))
check("Q R = A", Q @ R, A)
Qn, Rn = np.linalg.qr(A) # the diagonal from numpy need not be positive real; after adjusting the phases the factors should agree
D = np.diag(np.diag(Rn) / np.abs(np.diag(Rn)))
check("Q from numpy QR after phase adjustment", Qn @ D, Q)
check("R from numpy QR after phase adjustment", H_(D) @ Rn, R)============================================================
Exercise 9-42 | Complex Projections and the QR Decomposition
============================================================
▶ Step 1: P = u u^*: P^* = P, P^2 = P, Im ⟂ Ker, direct sum decomposition
----------------------------------------
[P^* = P]
Computed value: [[ 0.4732+0.j 0.2698+0.2944j -0.0025-0.1824j 0.1097-0.211j ]
[ 0.2698-0.2944j 0.337 +0.j -0.1149-0.1024j -0.0687-0.1885j]
[-0.0025+0.1824j -0.1149+0.1024j 0.0703+0.j 0.0807+0.0434j]
[ 0.1097+0.211j -0.0687+0.1885j 0.0807-0.0434j 0.1195-0.j ]]
Expected value: [[ 0.4732-0.j 0.2698+0.2944j -0.0025-0.1824j 0.1097-0.211j ]
[ 0.2698-0.2944j 0.337 -0.j -0.1149-0.1024j -0.0687-0.1885j]
[-0.0025+0.1824j -0.1149+0.1024j 0.0703-0.j 0.0807+0.0434j]
[ 0.1097+0.211j -0.0687+0.1885j 0.0807-0.0434j 0.1195+0.j ]]
check passed
[P^2 = P]
Computed value: [[ 0.4732+0.j 0.2698+0.2944j -0.0025-0.1824j 0.1097-0.211j ]
[ 0.2698-0.2944j 0.337 +0.j -0.1149-0.1024j -0.0687-0.1885j]
[-0.0025+0.1824j -0.1149+0.1024j 0.0703+0.j 0.0807+0.0434j]
[ 0.1097+0.211j -0.0687+0.1885j 0.0807-0.0434j 0.1195+0.j ]]
Expected value: [[ 0.4732-0.j 0.2698+0.2944j -0.0025-0.1824j 0.1097-0.211j ]
[ 0.2698-0.2944j 0.337 -0.j -0.1149-0.1024j -0.0687-0.1885j]
[-0.0025+0.1824j -0.1149+0.1024j 0.0703-0.j 0.0807+0.0434j]
[ 0.1097+0.211j -0.0687+0.1885j 0.0807-0.0434j 0.1195+0.j ]]
check passed
[P x = <u,x> u]
Computed value: [-0.5829-0.3356j -0.5412+0.1713j 0.1325-0.2229j 0.0146-0.3377j]
Expected value: [-0.5829-0.3356j -0.5412+0.1713j 0.1325-0.2229j 0.0146-0.3377j]
check passed
[x - Px ∈ Ker(P)]
Computed value: [ 0.+0.j 0.-0.j -0.+0.j 0.+0.j]
Expected value: [0. 0. 0. 0.]
check passed
[<Px, x - Px> = 0]
Computed value: (-5.551115123125783e-17-2.220446049250313e-16j)
Expected value: 0
check passed
[rank(P)]
Computed value: 1
Expected value: 1
check passed
▶ Step 2: Projection onto a k-dimensional subspace: B (B^*B)^{-1} B^* = Q Q^*
----------------------------------------
[B(B^*B)^{-1}B^* = Q Q^*]
Computed value: [[ 0.5507+0.j -0.0913+0.1265j 0.2078+0.3559j 0.0034-0.2308j]
[-0.0913-0.1265j 0.2384-0.j 0.103 -0.2295j 0.2524-0.174j ]
[ 0.2078-0.3559j 0.103 +0.2295j 0.4019-0.j 0.0736+0.0434j]
[ 0.0034+0.2308j 0.2524+0.174j 0.0736-0.0434j 0.8089+0.j ]]
Expected value: [[ 0.5507+0.j -0.0913+0.1265j 0.2078+0.3559j 0.0034-0.2308j]
[-0.0913-0.1265j 0.2384-0.j 0.103 -0.2295j 0.2524-0.174j ]
[ 0.2078-0.3559j 0.103 +0.2295j 0.4019+0.j 0.0736+0.0434j]
[ 0.0034+0.2308j 0.2524+0.174j 0.0736-0.0434j 0.8089+0.j ]]
check passed
[P_W^* = P_W]
Computed value: [[ 0.5507-0.j -0.0913+0.1265j 0.2078+0.3559j 0.0034-0.2308j]
[-0.0913-0.1265j 0.2384+0.j 0.103 -0.2295j 0.2524-0.174j ]
[ 0.2078-0.3559j 0.103 +0.2295j 0.4019+0.j 0.0736+0.0434j]
[ 0.0034+0.2308j 0.2524+0.174j 0.0736-0.0434j 0.8089-0.j ]]
Expected value: [[ 0.5507+0.j -0.0913+0.1265j 0.2078+0.3559j 0.0034-0.2308j]
[-0.0913-0.1265j 0.2384-0.j 0.103 -0.2295j 0.2524-0.174j ]
[ 0.2078-0.3559j 0.103 +0.2295j 0.4019-0.j 0.0736+0.0434j]
[ 0.0034+0.2308j 0.2524+0.174j 0.0736-0.0434j 0.8089+0.j ]]
check passed
[P_W^2 = P_W]
Computed value: [[ 0.5507+0.j -0.0913+0.1265j 0.2078+0.3559j 0.0034-0.2308j]
[-0.0913-0.1265j 0.2384+0.j 0.103 -0.2295j 0.2524-0.174j ]
[ 0.2078-0.3559j 0.103 +0.2295j 0.4019-0.j 0.0736+0.0434j]
[ 0.0034+0.2308j 0.2524+0.174j 0.0736-0.0434j 0.8089+0.j ]]
Expected value: [[ 0.5507+0.j -0.0913+0.1265j 0.2078+0.3559j 0.0034-0.2308j]
[-0.0913-0.1265j 0.2384-0.j 0.103 -0.2295j 0.2524-0.174j ]
[ 0.2078-0.3559j 0.103 +0.2295j 0.4019-0.j 0.0736+0.0434j]
[ 0.0034+0.2308j 0.2524+0.174j 0.0736-0.0434j 0.8089+0.j ]]
check passed
[P_W B = B (vectors in W are fixed)]
Computed value: [[ 0.3688-0.4283j -0.9589-0.3521j]
[ 0.8785+0.5323j -0.0499+0.3654j]
[-0.1849+0.4127j -0.6809+0.4308j]
[ 1.2225+2.1416j -0.1545-0.4064j]]
Expected value: [[ 0.3688-0.4283j -0.9589-0.3521j]
[ 0.8785+0.5323j -0.0499+0.3654j]
[-0.1849+0.4127j -0.6809+0.4308j]
[ 1.2225+2.1416j -0.1545-0.4064j]]
check passed
[B^*(x - P_W x) = 0 (x - P_W x ∈ W^⟂)]
Computed value: [-0.-0.j -0.+0.j]
Expected value: [0. 0.]
check passed
[P_W x is the best approximation (200 vectors in W)]
Computed value: True
Expected value: True
check passed
▶ Step 3: Gram-Schmidt: v0 = (i,1), v1 = (0,i)
----------------------------------------
[r00]
Computed value: 1.4142135623730951
Expected value: 1.4142135623730951
check passed
[r01 = <q0, v1>]
Computed value: 0.7071067811865475j
Expected value: 0.7071067811865475j
check passed
[w1]
Computed value: [0.5+0.j 0. +0.5j]
Expected value: [0.5, 0.5j]
check passed
[q1]
Computed value: [0.7071+0.j 0. +0.7071j]
Expected value: [0.7071+0.j 0. +0.7071j]
check passed
[<q0, q1>]
Computed value: 1.6653345369377348e-16j
Expected value: 0
check passed
▶ Step 4: A = Q R, where the diagonal of R is positive real
----------------------------------------
[Q]
Computed value: [[0. +0.7071j 0.7071+0.j ]
[0.7071+0.j 0. +0.7071j]]
Expected value: [[0. +0.7071j 0.7071+0.j ]
[0.7071+0.j 0. +0.7071j]]
check passed
[R]
Computed value: [[1.4142+0.j 0. +0.7071j]
[0. +0.j 0.7071+0.j ]]
Expected value: [[np.float64(1.4142135623730951), 0.7071067811865475j], [0, np.float64(0.7071067811865475)]]
check passed
[Q^* Q]
Computed value: [[1.+0.j 0.+0.j]
[0.-0.j 1.+0.j]]
Expected value: [[1. 0.]
[0. 1.]]
check passed
[Q R = A]
Computed value: [[0.+1.j 0.+0.j]
[1.+0.j 0.+1.j]]
Expected value: [[0.+1.j 0.+0.j]
[1.+0.j 0.+1.j]]
check passed
[Q from numpy QR after phase adjustment]
Computed value: [[ 0. +0.7071j 0.7071-0.j ]
[ 0.7071+0.j -0. +0.7071j]]
Expected value: [[0. +0.7071j 0.7071+0.j ]
[0.7071+0.j 0. +0.7071j]]
check passed
[R from numpy QR after phase adjustment]
Computed value: [[1.4142+0.j 0. +0.7071j]
[0. +0.j 0.7071+0.j ]]
Expected value: [[1.4142+0.j 0. +0.7071j]
[0. +0.j 0.7071+0.j ]]
check passed
# ============================================================
print_header("Exercise 9-43 | Diagonalization of a Hermitian Matrix")
# ============================================================
A = np.array([[2, 1 - 1j], [1 + 1j, 3]])
# --- Step 1 ---
print_step(1, "A^* = A")
check("A^*", A.conj().T, A)
# --- Step 2 ---
print_step(2, "Characteristic polynomial and eigenvalues")
check("Coefficients of the characteristic polynomial λ^2 - 5λ + 4", np.poly(A), [1, -5, 4])
eig = np.linalg.eigvals(A)
check("Eigenvalues (in increasing order)", np.sort(eig.real), [1, 4])
check("Imaginary parts of the eigenvalues", eig.imag, [0, 0])
# --- Step 3 ---
print_step(3, "Eigenvectors and orthogonality")
v0 = np.array([-1 + 1j, 1]); v1 = np.array([1 - 1j, 2])
check("A v0 = 1·v0", A @ v0, v0)
check("A v1 = 4·v1", A @ v1, 4 * v1)
check("<v0, v1> = v0^* v1", np.vdot(v0, v1), 0)
# --- Step 4 ---
print_step(4, "U^* A U = Λ")
U = np.column_stack([v0 / np.sqrt(3), v1 / np.sqrt(6)])
Lam = np.diag([1, 4])
check("(||v0||^2, ||v1||^2)", [np.vdot(v0, v0).real, np.vdot(v1, v1).real], [3, 6])
check("U^* U", U.conj().T @ U, np.eye(2))
check("U^* A U", U.conj().T @ A @ U, Lam)
check("U Λ U^*", U @ Lam @ U.conj().T, A)============================================================
Exercise 9-43 | Diagonalization of a Hermitian Matrix
============================================================
▶ Step 1: A^* = A
----------------------------------------
[A^*]
Computed value: [[2.-0.j 1.-1.j]
[1.+1.j 3.-0.j]]
Expected value: [[2.+0.j 1.-1.j]
[1.+1.j 3.+0.j]]
check passed
▶ Step 2: Characteristic polynomial and eigenvalues
----------------------------------------
[Coefficients of the characteristic polynomial λ^2 - 5λ + 4]
Computed value: [ 1.+0.j -5.+0.j 4.-0.j]
Expected value: [1, -5, 4]
check passed
[Eigenvalues (in increasing order)]
Computed value: [1. 4.]
Expected value: [1, 4]
check passed
[Imaginary parts of the eigenvalues]
Computed value: [-0. 0.]
Expected value: [0, 0]
check passed
▶ Step 3: Eigenvectors and orthogonality
----------------------------------------
[A v0 = 1·v0]
Computed value: [-1.+1.j 1.+0.j]
Expected value: [-1.+1.j 1.+0.j]
check passed
[A v1 = 4·v1]
Computed value: [4.-4.j 8.+0.j]
Expected value: [4.-4.j 8.+0.j]
check passed
[<v0, v1> = v0^* v1]
Computed value: 0j
Expected value: 0
check passed
▶ Step 4: U^* A U = Λ
----------------------------------------
[(||v0||^2, ||v1||^2)]
Computed value: [np.float64(3.0), np.float64(6.0)]
Expected value: [3, 6]
check passed
[U^* U]
Computed value: [[1.+0.j 0.+0.j]
[0.+0.j 1.+0.j]]
Expected value: [[1. 0.]
[0. 1.]]
check passed
[U^* A U]
Computed value: [[1.+0.j 0.+0.j]
[0.+0.j 4.+0.j]]
Expected value: [[1 0]
[0 4]]
check passed
[U Λ U^*]
Computed value: [[2.-0.j 1.-1.j]
[1.+1.j 3.+0.j]]
Expected value: [[2.+0.j 1.-1.j]
[1.+1.j 3.+0.j]]
check passed
# ============================================================
print_header("Exercise 9-44 | Spectral Decomposition of a Hermitian Matrix")
# ============================================================
A = np.array([[2, 1 - 1j], [1 + 1j, 3]])
lam0, lam1 = 1, 4
u0 = np.array([-1 + 1j, 1]) / np.sqrt(3)
u1 = np.array([1 - 1j, 2]) / np.sqrt(6)
# --- Step 1 ---
print_step(1, "P_j = |u_j><u_j| and the spectral decomposition")
P0 = np.outer(u0, u0.conj())
P1 = np.outer(u1, u1.conj())
check("P0", P0, np.array([[2, -1 + 1j], [-1 - 1j, 1]]) / 3)
check("P1", P1, np.array([[1, 1 - 1j], [1 + 1j, 2]]) / 3)
check("λ0 P0 + λ1 P1 = A", lam0 * P0 + lam1 * P1, A)
check("Independent of the phase: (e^{iφ}u0)(e^{iφ}u0)^*", np.outer(np.exp(0.8j) * u0, (np.exp(0.8j) * u0).conj()), P0)
# --- Step 2 ---
print_step(2, "P0 + P1 = I, P0 P1 = 0, P_j^2 = P_j")
check("P0 + P1", P0 + P1, np.eye(2))
check("P0 P1", P0 @ P1, np.zeros((2, 2)))
check("P1 P0", P1 @ P0, np.zeros((2, 2)))
check("P0^2 = P0", P0 @ P0, P0)
check("P1^2 = P1", P1 @ P1, P1)============================================================
Exercise 9-44 | Spectral Decomposition of a Hermitian Matrix
============================================================
▶ Step 1: P_j = |u_j><u_j| and the spectral decomposition
----------------------------------------
[P0]
Computed value: [[ 0.6667-0.j -0.3333+0.3333j]
[-0.3333-0.3333j 0.3333+0.j ]]
Expected value: [[ 0.6667+0.j -0.3333+0.3333j]
[-0.3333-0.3333j 0.3333+0.j ]]
check passed
[P1]
Computed value: [[0.3333-0.j 0.3333-0.3333j]
[0.3333+0.3333j 0.6667+0.j ]]
Expected value: [[0.3333+0.j 0.3333-0.3333j]
[0.3333+0.3333j 0.6667+0.j ]]
check passed
[λ0 P0 + λ1 P1 = A]
Computed value: [[2.-0.j 1.-1.j]
[1.+1.j 3.+0.j]]
Expected value: [[2.+0.j 1.-1.j]
[1.+1.j 3.+0.j]]
check passed
[Independent of the phase: (e^{iφ}u0)(e^{iφ}u0)^*]
Computed value: [[ 0.6667+0.j -0.3333+0.3333j]
[-0.3333-0.3333j 0.3333+0.j ]]
Expected value: [[ 0.6667-0.j -0.3333+0.3333j]
[-0.3333-0.3333j 0.3333+0.j ]]
check passed
▶ Step 2: P0 + P1 = I, P0 P1 = 0, P_j^2 = P_j
----------------------------------------
[P0 + P1]
Computed value: [[1.-0.j 0.+0.j]
[0.+0.j 1.+0.j]]
Expected value: [[1. 0.]
[0. 1.]]
check passed
[P0 P1]
Computed value: [[ 0.+0.j 0.+0.j]
[-0.+0.j 0.+0.j]]
Expected value: [[0. 0.]
[0. 0.]]
check passed
[P1 P0]
Computed value: [[ 0.+0.j -0.+0.j]
[ 0.+0.j 0.+0.j]]
Expected value: [[0. 0.]
[0. 0.]]
check passed
[P0^2 = P0]
Computed value: [[ 0.6667+0.j -0.3333+0.3333j]
[-0.3333-0.3333j 0.3333+0.j ]]
Expected value: [[ 0.6667-0.j -0.3333+0.3333j]
[-0.3333-0.3333j 0.3333+0.j ]]
check passed
[P1^2 = P1]
Computed value: [[0.3333+0.j 0.3333-0.3333j]
[0.3333+0.3333j 0.6667+0.j ]]
Expected value: [[0.3333-0.j 0.3333-0.3333j]
[0.3333+0.3333j 0.6667+0.j ]]
check passed
# ============================================================
print_header("Exercise 9-46 | Eigenvalues of Unitary Matrices")
# ============================================================
# --- Step 1 ---
print_step(1, "Random unitary matrix: every eigenvalue has modulus 1")
rng_46 = np.random.default_rng(46)
M = rng_46.normal(size=(4, 4)) + 1j * rng_46.normal(size=(4, 4))
U, _ = np.linalg.qr(M) # the Q factor of a QR decomposition is a unitary matrix
check("U^* U = I", U.conj().T @ U, np.eye(4))
lam_U = np.linalg.eigvals(U)
print("Eigenvalues of U:", lam_U)
check("|λ| (unitary matrix U)", np.abs(lam_U), np.ones(4))
# --- Step 2 ---
print_step(2, "Real orthogonal matrix (cyclic permutation): the complex eigenvalues also have modulus 1")
Q = np.array([[0, 0, 1],
[1, 0, 0],
[0, 1, 0]], dtype=float)
check("Q^T Q = I", Q.T @ Q, np.eye(3))
lam_Q = np.linalg.eigvals(Q)
print("Eigenvalues of Q (the cube roots of 1):", lam_Q)
check("|λ| (orthogonal matrix Q)", np.abs(lam_Q), np.ones(3))
# --- Step 3 ---
print_step(3, "Eigenvalues e^{±iθ} of the rotation matrix R_θ (θ = π/3)")
theta = np.pi / 3
R = np.array([[np.cos(theta), -np.sin(theta)],
[np.sin(theta), np.cos(theta)]])
lam_R = np.linalg.eigvals(R)
lam_R = lam_R[np.argsort(lam_R.imag)] # sort by imaginary part: e^{-iθ}, e^{iθ}
check("Eigenvalues", lam_R, [np.exp(-1j * theta), np.exp(1j * theta)])
print("Discriminant 4cos²θ - 4 =", 4 * np.cos(theta)**2 - 4, "< 0, so there are no real roots")
# --- Step 4 ---
print_step(4, "Eigenvectors (1, -i), (1, i) in C^2 and unitary diagonalization")
q_plus = np.array([1, -1j]) / np.sqrt(2)
q_minus = np.array([1, 1j]) / np.sqrt(2)
check("R q₊ = e^{iθ} q₊", R @ q_plus, np.exp(1j * theta) * q_plus)
check("R q₋ = e^{-iθ} q₋", R @ q_minus, np.exp(-1j * theta) * q_minus)
check("⟨q₊|q₋⟩ = 0", np.vdot(q_plus, q_minus), 0)
R_rebuilt = (np.exp(1j * theta) * np.outer(q_plus, q_plus.conj())
+ np.exp(-1j * theta) * np.outer(q_minus, q_minus.conj()))
check("e^{iθ}|q₊⟩⟨q₊| + e^{-iθ}|q₋⟩⟨q₋| = R", R_rebuilt, R)
# --- Step 5 ---
print_step(5, "No real eigenvectors: for every real unit vector x, det(x, Rx) = sinθ ≠ 0")
phis = np.linspace(0, 2 * np.pi, 361)
X = np.vstack([np.cos(phis), np.sin(phis)]) # each column is a real unit vector
RX = R @ X
dets = X[0] * RX[1] - X[1] * RX[0]
check("Maximum error over 361 directions, max|det(x, Rx) - sinθ|", np.max(np.abs(dets - np.sin(theta))), 0)
for th, expected in [(0.0, np.eye(2)), (np.pi, -np.eye(2))]:
R_th = np.array([[np.cos(th), -np.sin(th)], [np.sin(th), np.cos(th)]])
check(f"θ = {th:.4f}: R_θ = ±I (every nonzero vector is an eigenvector)", R_th, expected)============================================================
Exercise 9-46 | Eigenvalues of Unitary Matrices
============================================================
▶ Step 1: Random unitary matrix: every eigenvalue has modulus 1
----------------------------------------
[U^* U = I]
Computed value: [[ 1.+0.j 0.-0.j 0.+0.j 0.+0.j]
[ 0.+0.j 1.+0.j -0.-0.j -0.-0.j]
[ 0.-0.j -0.+0.j 1.+0.j -0.-0.j]
[ 0.-0.j -0.+0.j -0.+0.j 1.+0.j]]
Expected value: [[1. 0. 0. 0.]
[0. 1. 0. 0.]
[0. 0. 1. 0.]
[0. 0. 0. 1.]]
check passed
Eigenvalues of U: [ 0.0872+0.9962j 0.5241-0.8517j -0.4938-0.8695j -0.948 +0.3184j]
[|λ| (unitary matrix U)]
Computed value: [1. 1. 1. 1.]
Expected value: [1. 1. 1. 1.]
check passed
▶ Step 2: Real orthogonal matrix (cyclic permutation): the complex eigenvalues also have modulus 1
----------------------------------------
[Q^T Q = I]
Computed value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
Expected value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
check passed
Eigenvalues of Q (the cube roots of 1): [-0.5+0.866j -0.5-0.866j 1. +0.j ]
[|λ| (orthogonal matrix Q)]
Computed value: [1. 1. 1.]
Expected value: [1. 1. 1.]
check passed
▶ Step 3: Eigenvalues e^{±iθ} of the rotation matrix R_θ (θ = π/3)
----------------------------------------
[Eigenvalues]
Computed value: [0.5-0.866j 0.5+0.866j]
Expected value: [np.complex128(0.5000000000000001-0.8660254037844386j), np.complex128(0.5000000000000001+0.8660254037844386j)]
check passed
Discriminant 4cos²θ - 4 = -2.9999999999999996 < 0, so there are no real roots
▶ Step 4: Eigenvectors (1, -i), (1, i) in C^2 and unitary diagonalization
----------------------------------------
[R q₊ = e^{iθ} q₊]
Computed value: [0.3536+0.6124j 0.6124-0.3536j]
Expected value: [0.3536+0.6124j 0.6124-0.3536j]
check passed
[R q₋ = e^{-iθ} q₋]
Computed value: [0.3536-0.6124j 0.6124+0.3536j]
Expected value: [0.3536-0.6124j 0.6124+0.3536j]
check passed
[⟨q₊|q₋⟩ = 0]
Computed value: 0j
Expected value: 0
check passed
[e^{iθ}|q₊⟩⟨q₊| + e^{-iθ}|q₋⟩⟨q₋| = R]
Computed value: [[ 0.5 +0.j -0.866+0.j]
[ 0.866+0.j 0.5 +0.j]]
Expected value: [[ 0.5 -0.866]
[ 0.866 0.5 ]]
check passed
▶ Step 5: No real eigenvectors: for every real unit vector x, det(x, Rx) = sinθ ≠ 0
----------------------------------------
[Maximum error over 361 directions, max|det(x, Rx) - sinθ|]
Computed value: 2.220446049250313e-16
Expected value: 0
check passed
[θ = 0.0000: R_θ = ±I (every nonzero vector is an eigenvector)]
Computed value: [[ 1. -0.]
[ 0. 1.]]
Expected value: [[1. 0.]
[0. 1.]]
check passed
[θ = 3.1416: R_θ = ±I (every nonzero vector is an eigenvector)]
Computed value: [[-1. -0.]
[ 0. -1.]]
Expected value: [[-1. -0.]
[-0. -1.]]
check passed
# ============================================================
print_header("Exercise 9-48 | True or False: Spectral Theory")
# ============================================================
# --- Step 1 ---
print_step(1, "Part 1 (true): A = P D P^T is necessarily symmetric")
c, s = np.cos(0.7), np.sin(0.7)
P = np.array([[c, -s, 0], [s, c, 0], [0, 0, 1.0]]) # real orthogonal matrix
D = np.diag([3.0, -1.0, 2.0])
A1 = P @ D @ P.T
check("P^T P = I", P.T @ P, np.eye(3))
check("A^T = A", A1.T, A1)
# --- Step 2 ---
print_step(2, "Part 2 (false): the orthogonal matrix R_{π/2} is not symmetric and has no real eigenvalues")
A2 = np.array([[0.0, -1.0], [1.0, 0.0]])
check("A^T A = I (orthogonal)", A2.T @ A2, np.eye(2))
check("A^T = -A (not symmetric)", A2.T, -A2)
lam2 = np.linalg.eigvals(A2)
lam2 = lam2[np.argsort(lam2.imag)]
check("Eigenvalues ±i (non-real, so not orthogonally diagonalizable)", lam2, [-1j, 1j])
# --- Step 3 ---
print_step(3, "Part 3 (true): all eigenvalues positive ⟹ x^T A x > 0 for real vectors")
A3 = np.array([[2.0, 1.0, 0.0], [1.0, 3.0, 1.0], [0.0, 1.0, 2.0]])
lam3, P3 = np.linalg.eigh(A3)
check("Eigenvalues", lam3, [1.0, 2.0, 4.0])
rng_48 = np.random.default_rng(48)
xs = rng_48.normal(size=(1000, 3))
quad = np.einsum("ij,jk,ik->i", xs, A3, xs)
check("1000 random real vectors: x^T A x > 0", bool(quad.min() > 0), True)
x = xs[0]
y = P3.T @ x # y_i = ⟨q_i|x⟩
check("x^T A x = Σ λ_i y_i²", x @ A3 @ x, np.sum(lam3 * y**2))
z = np.array([1, 1j])
check("Reminder: transposing a complex vector without conjugating gives (1,i)^T I (1,i) = 0", z @ np.eye(2) @ z, 0)
# --- Step 4 ---
print_step(4, "Part 4 (false): the eigenvalues of A^* = -A always have real part 0, but they can be 0 (counterexample O)")
A4 = np.array([[1j, 1 + 2j], [-1 + 2j, 0]])
check("A^* = -A", A4.conj().T, -A4)
lam4 = np.linalg.eigvals(A4)
lam4 = lam4[np.argsort(lam4.imag)]
check("Eigenvalues", lam4, [(1 - np.sqrt(21)) / 2 * 1j, (1 + np.sqrt(21)) / 2 * 1j])
check("Re λ = 0", lam4.real, [0, 0])
check("Counterexample O: O^* = -O", np.zeros((2, 2)).conj().T, -np.zeros((2, 2)))
check("The eigenvalues of counterexample O are 0 (real part 0, but not purely imaginary)", np.linalg.eigvals(np.zeros((2, 2))), [0, 0])
# --- Step 5 ---
print_step(5, "Part 5 (true): x^* y = 0 ⟹ the Pythagorean identity; the converse fails")
x5 = np.array([1, 1j]); y5 = np.array([1j, 1])
check("x^* y = ⟨x|y⟩", np.vdot(x5, y5), 0)
check("‖x+y‖² = ‖x‖² + ‖y‖²", np.linalg.norm(x5 + y5)**2,
np.linalg.norm(x5)**2 + np.linalg.norm(y5)**2)
xa = np.array([1, 0]); ya = np.array([1j, 0])
check("Converse direction: ‖x+y‖² = ‖x‖² + ‖y‖² = 2", np.linalg.norm(xa + ya)**2, 2)
check("but x^* y = i ≠ 0", np.vdot(xa, ya), 1j)
# --- Step 6 ---
print_step(6, "Part 6 (false): all eigenvalues real, but not symmetric")
A6 = np.array([[1.0, 1.0], [0.0, 2.0]])
lam6 = np.sort(np.linalg.eigvals(A6).real)
check("Eigenvalues (all real)", lam6, [1.0, 2.0])
print("A^T == A ?", np.allclose(A6.T, A6))
assert not np.allclose(A6.T, A6)
check("Inner product of the eigenvectors (1,0), (1,1) ≠ 0", np.array([1, 0]) @ np.array([1, 1]), 1)
# --- Step 7 ---
print_step(7, "Part 7 (true): B = U^* A U is still Hermitian")
H = rng_48.normal(size=(3, 3)) + 1j * rng_48.normal(size=(3, 3))
A7 = H + H.conj().T # Hermitian matrix
U7, _ = np.linalg.qr(rng_48.normal(size=(3, 3)) + 1j * rng_48.normal(size=(3, 3)))
B = U7.conj().T @ A7 @ U7
check("B^* = B", B.conj().T, B)
check("B and A have the same eigenvalues", np.linalg.eigvalsh(B), np.linalg.eigvalsh(A7))============================================================
Exercise 9-48 | True or False: Spectral Theory
============================================================
▶ Step 1: Part 1 (true): A = P D P^T is necessarily symmetric
----------------------------------------
[P^T P = I]
Computed value: [[ 1. -0. 0.]
[-0. 1. 0.]
[ 0. 0. 1.]]
Expected value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
check passed
[A^T = A]
Computed value: [[1.3399 1.9709 0. ]
[1.9709 0.6601 0. ]
[0. 0. 2. ]]
Expected value: [[1.3399 1.9709 0. ]
[1.9709 0.6601 0. ]
[0. 0. 2. ]]
check passed
▶ Step 2: Part 2 (false): the orthogonal matrix R_{π/2} is not symmetric and has no real eigenvalues
----------------------------------------
[A^T A = I (orthogonal)]
Computed value: [[1. 0.]
[0. 1.]]
Expected value: [[1. 0.]
[0. 1.]]
check passed
[A^T = -A (not symmetric)]
Computed value: [[ 0. 1.]
[-1. 0.]]
Expected value: [[-0. 1.]
[-1. -0.]]
check passed
[Eigenvalues ±i (non-real, so not orthogonally diagonalizable)]
Computed value: [0.-1.j 0.+1.j]
Expected value: [(-0-1j), 1j]
check passed
▶ Step 3: Part 3 (true): all eigenvalues positive ⟹ x^T A x > 0 for real vectors
----------------------------------------
[Eigenvalues]
Computed value: [1. 2. 4.]
Expected value: [1.0, 2.0, 4.0]
check passed
[1000 random real vectors: x^T A x > 0]
Computed value: True
Expected value: True
check passed
[x^T A x = Σ λ_i y_i²]
Computed value: 5.417957870729893
Expected value: 5.417957870729891
check passed
[Reminder: transposing a complex vector without conjugating gives (1,i)^T I (1,i) = 0]
Computed value: 0j
Expected value: 0
check passed
▶ Step 4: Part 4 (false): the eigenvalues of A^* = -A always have real part 0, but they can be 0 (counterexample O)
----------------------------------------
[A^* = -A]
Computed value: [[ 0.-1.j -1.-2.j]
[ 1.-2.j 0.-0.j]]
Expected value: [[-0.-1.j -1.-2.j]
[ 1.-2.j -0.-0.j]]
check passed
[Eigenvalues]
Computed value: [0.-1.7913j 0.+2.7913j]
Expected value: [np.complex128(-0-1.79128784747792j), np.complex128(2.79128784747792j)]
check passed
[Re λ = 0]
Computed value: [0. 0.]
Expected value: [0, 0]
check passed
[Counterexample O: O^* = -O]
Computed value: [[0. 0.]
[0. 0.]]
Expected value: [[-0. -0.]
[-0. -0.]]
check passed
[The eigenvalues of counterexample O are 0 (real part 0, but not purely imaginary)]
Computed value: [0. 0.]
Expected value: [0, 0]
check passed
▶ Step 5: Part 5 (true): x^* y = 0 ⟹ the Pythagorean identity; the converse fails
----------------------------------------
[x^* y = ⟨x|y⟩]
Computed value: 0j
Expected value: 0
check passed
[‖x+y‖² = ‖x‖² + ‖y‖²]
Computed value: 4.0
Expected value: 4.000000000000001
check passed
[Converse direction: ‖x+y‖² = ‖x‖² + ‖y‖² = 2]
Computed value: 2.0000000000000004
Expected value: 2
check passed
[but x^* y = i ≠ 0]
Computed value: 1j
Expected value: 1j
check passed
▶ Step 6: Part 6 (false): all eigenvalues real, but not symmetric
----------------------------------------
[Eigenvalues (all real)]
Computed value: [1. 2.]
Expected value: [1.0, 2.0]
check passed
A^T == A ? False
[Inner product of the eigenvectors (1,0), (1,1) ≠ 0]
Computed value: 1
Expected value: 1
check passed
▶ Step 7: Part 7 (true): B = U^* A U is still Hermitian
----------------------------------------
[B^* = B]
Computed value: [[-3.5778-0.j -2.668 -0.7551j 0.9573-0.9689j]
[-2.668 +0.7551j -2.9931-0.j -0.313 +1.8387j]
[ 0.9573+0.9689j -0.313 -1.8387j -0.3631-0.j ]]
Expected value: [[-3.5778+0.j -2.668 -0.7551j 0.9573-0.9689j]
[-2.668 +0.7551j -2.9931+0.j -0.313 +1.8387j]
[ 0.9573+0.9689j -0.313 -1.8387j -0.3631+0.j ]]
check passed
[B and A have the same eigenvalues]
Computed value: [-6.1141 -2.6702 1.8503]
Expected value: [-6.1141 -2.6702 1.8503]
check passed
# ============================================================
print_header("Exercise 9-49 | Complex Least Squares")
# ============================================================
# --- Step 1 ---
print_step(1, "The system z h = w has no solution")
z = np.array([1, 1j])
w = np.array([1 + 2j, -1.5 + 1j])
h_from_row = w / z # h solved from each equation separately
print("h solved from the first and the second equation, respectively:", h_from_row)
check("h given by the two equations", h_from_row, [1 + 2j, 1 + 1.5j])
assert not np.isclose(h_from_row[0], h_from_row[1])
# --- Step 2 ---
print_step(2, "Why z^*: z^T z = 0, but z^* z = 2")
check("z^T z (transpose only)", z @ z, 0)
check("z^* z = ⟨z|z⟩", np.vdot(z, z), 2)
# --- Step 3 ---
print_step(3, "Normal equations z^* z ĥ = z^* w")
zw = np.vdot(z, w) # z^* w = ⟨z|w⟩
check("z^* w", zw, 2 + 3.5j)
h_hat = zw / np.vdot(z, z)
check("ĥ", h_hat, 1 + 1.75j)
h_lstsq = np.linalg.lstsq(z.reshape(2, 1), w, rcond=None)[0][0]
check("ĥ (np.linalg.lstsq)", h_lstsq, 1 + 1.75j)
# --- Step 4 ---
print_step(4, "Residual and least-squares error")
r = w - z * h_hat
check("Residual r = w - z ĥ", r, [0.25j, 0.25])
check("z^* r = 0 (the residual is orthogonal to z)", np.vdot(z, r), 0)
err = np.linalg.norm(r)**2
check("‖w - z ĥ‖²", err, 1 / 8)
check("‖w‖² - |z^* w|² / ‖z‖²", np.linalg.norm(w)**2 - abs(zw)**2 / 2, 1 / 8)============================================================
Exercise 9-49 | Complex Least Squares
============================================================
▶ Step 1: The system z h = w has no solution
----------------------------------------
h solved from the first and the second equation, respectively: [1.+2.j 1.+1.5j]
[h given by the two equations]
Computed value: [1.+2.j 1.+1.5j]
Expected value: [(1+2j), (1+1.5j)]
check passed
▶ Step 2: Why z^*: z^T z = 0, but z^* z = 2
----------------------------------------
[z^T z (transpose only)]
Computed value: 0j
Expected value: 0
check passed
[z^* z = ⟨z|z⟩]
Computed value: (2+0j)
Expected value: 2
check passed
▶ Step 3: Normal equations z^* z ĥ = z^* w
----------------------------------------
[z^* w]
Computed value: (2+3.5j)
Expected value: (2+3.5j)
check passed
[ĥ]
Computed value: (1+1.75j)
Expected value: (1+1.75j)
check passed
[ĥ (np.linalg.lstsq)]
Computed value: (0.9999999999999998+1.7499999999999996j)
Expected value: (1+1.75j)
check passed
▶ Step 4: Residual and least-squares error
----------------------------------------
[Residual r = w - z ĥ]
Computed value: [0. +0.25j 0.25+0.j ]
Expected value: [0.25j, 0.25]
check passed
[z^* r = 0 (the residual is orthogonal to z)]
Computed value: 0j
Expected value: 0
check passed
[‖w - z ĥ‖²]
Computed value: 0.12500000000000003
Expected value: 0.125
check passed
[‖w‖² - |z^* w|² / ‖z‖²]
Computed value: 0.12500000000000178
Expected value: 0.125
check passed
Testing and Factoring Positive Definite Matrices¶
# ============================================================
print_header("Exercise 9-50 | Positive Definite Matrix of Order 2")
# ============================================================
A = np.array([[4., 2.], [2., 5.]])
# --- Step 1 ---
print_step(1, "Complete the square: x^T A x = (2x_0 + x_1)^2 + 4x_1^2")
rng = np.random.default_rng(50)
X = rng.standard_normal((1000, 2))
quad = np.einsum('ni,ij,nj->n', X, A, X)
sos = (2 * X[:, 0] + X[:, 1])**2 + 4 * X[:, 1]**2
check("max |quadratic form - completed square| (1000 random vectors)", np.max(np.abs(quad - sos)), 0)
check("minimum of the quadratic form > 0", bool(quad.min() > 0), True)
# --- Step 2 ---
print_step(2, "Eigenvalues (9 ∓ √17)/2")
lam = np.linalg.eigvalsh(A)
check("eigenvalues λ_0, λ_1", lam, [(9 - np.sqrt(17)) / 2, (9 + np.sqrt(17)) / 2])
check("all eigenvalues positive", bool(np.all(lam > 0)), True)
# --- Step 3 ---
print_step(3, "Q = [[2, 1], [0, 2]] with A = Q^T Q")
Q = np.array([[2., 1.], [0., 2.]])
check("Q^T Q", Q.T @ Q, A)
check("det(Q)", np.linalg.det(Q), 4.0)
check("Q equals the transpose of the numpy Cholesky factor", Q, np.linalg.cholesky(A).T)
# Another choice: A = P Λ P^T, Q' = Λ^{1/2} P^T
lam, P = np.linalg.eigh(A)
Q2 = np.diag(np.sqrt(lam)) @ P.T
check("(Q')^T Q'", Q2.T @ Q2, A)============================================================
Exercise 9-50 | Positive Definite Matrix of Order 2
============================================================
▶ Step 1: Complete the square: x^T A x = (2x_0 + x_1)^2 + 4x_1^2
----------------------------------------
[max |quadratic form - completed square| (1000 random vectors)]
Computed value: 1.4210854715202004e-14
Expected value: 0
check passed
[minimum of the quadratic form > 0]
Computed value: True
Expected value: True
check passed
▶ Step 2: Eigenvalues (9 ∓ √17)/2
----------------------------------------
[eigenvalues λ_0, λ_1]
Computed value: [2.4384 6.5616]
Expected value: [np.float64(2.4384471871911697), np.float64(6.561552812808831)]
check passed
[all eigenvalues positive]
Computed value: True
Expected value: True
check passed
▶ Step 3: Q = [[2, 1], [0, 2]] with A = Q^T Q
----------------------------------------
[Q^T Q]
Computed value: [[4. 2.]
[2. 5.]]
Expected value: [[4. 2.]
[2. 5.]]
check passed
[det(Q)]
Computed value: 4.0
Expected value: 4.0
check passed
[Q equals the transpose of the numpy Cholesky factor]
Computed value: [[2. 1.]
[0. 2.]]
Expected value: [[2. 1.]
[0. 2.]]
check passed
[(Q')^T Q']
Computed value: [[4. 2.]
[2. 5.]]
Expected value: [[4. 2.]
[2. 5.]]
check passed
# ============================================================
print_header("Exercise 9-52 | Leading Principal Minor Test")
# ============================================================
def leading_minors_52(M):
# D_k = determinant of the upper-left k×k submatrix, k = 1, ..., n
return np.array([np.linalg.det(M[:k, :k]) for k in range(1, M.shape[0] + 1)])
A = np.array([[2, -1, 0], [-1, 2, -1], [0, -1, 2]], dtype=float)
B = np.array([[1, 2, 3], [2, 2, 4], [3, 4, 3]], dtype=float)
# --- Step 1 ---
print_step(1, "Leading principal minors of A and the sum-of-squares form")
check("(D_1, D_2, D_3) of A", leading_minors_52(A), [2, 3, 4])
rng = np.random.default_rng(52)
X = rng.standard_normal((1000, 3))
quad = np.einsum('ni,ij,nj->n', X, A, X)
sos = X[:, 0]**2 + (X[:, 0] - X[:, 1])**2 + (X[:, 1] - X[:, 2])**2 + X[:, 2]**2
check("max |x^T A x - sum of squares| (1000 random vectors)", np.max(np.abs(quad - sos)), 0)
check("eigenvalues of A", np.linalg.eigvalsh(A), [2 - np.sqrt(2), 2, 2 + np.sqrt(2)])
# --- Step 2 ---
print_step(2, "Leading principal minors of B")
check("(D_1, D_2, D_3) of B", leading_minors_52(B), [1, -2, 8])
# --- Step 3 ---
print_step(3, "Counterexample x = (-2, 1, 0)^T")
x = np.array([-2., 1., 0.])
check("B x", B @ x, [0, -2, -2])
check("x^T B x", x @ B @ x, -2)============================================================
Exercise 9-52 | Leading Principal Minor Test
============================================================
▶ Step 1: Leading principal minors of A and the sum-of-squares form
----------------------------------------
[(D_1, D_2, D_3) of A]
Computed value: [2. 3. 4.]
Expected value: [2, 3, 4]
check passed
[max |x^T A x - sum of squares| (1000 random vectors)]
Computed value: 7.105427357601002e-15
Expected value: 0
check passed
[eigenvalues of A]
Computed value: [0.5858 2. 3.4142]
Expected value: [np.float64(0.5857864376269049), 2, np.float64(3.414213562373095)]
check passed
▶ Step 2: Leading principal minors of B
----------------------------------------
[(D_1, D_2, D_3) of B]
Computed value: [ 1. -2. 8.]
Expected value: [1, -2, 8]
check passed
▶ Step 3: Counterexample x = (-2, 1, 0)^T
----------------------------------------
[B x]
Computed value: [ 0. -2. -2.]
Expected value: [0, -2, -2]
check passed
[x^T B x]
Computed value: -2.0
Expected value: -2
check passed
# ============================================================
print_header("Exercise 9-53 | A Pitfall in Testing Semidefiniteness")
# ============================================================
# --- Step 1 ---
print_step(1, "Counterexample A = diag(0, -1)")
A = np.array([[0., 0.], [0., -1.]])
check("(D_1, D_2)", [A[0, 0], np.linalg.det(A)], [0, 0])
x = np.array([0., 1.])
check("x^T A x, x = (0, 1)^T", x @ A @ x, -1)
check("eigenvalues of A", np.linalg.eigvalsh(A), [-1, 0])
# The original conditions a ≥ 0, b ≥ 0, det ≥ 0 all hold for this A, yet it is not positive semidefinite
check("original conditions a≥0, b≥0, det≥0 hold", bool(A[0, 0] >= 0 and A[0, 1] >= 0 and np.linalg.det(A) >= 0), True)
# --- Step 2 ---
print_step(2, "Corrected statement: symmetric, a≥0, d≥0, det≥0 ⇒ positive semidefinite (random test)")
rng = np.random.default_rng(53)
min_eigs = []
for _ in range(2000):
a, d = rng.uniform(0, 3, size=2)
if rng.random() < 0.2:
a = 0.0 # cover the case a = 0
b = rng.uniform(-1, 1) * np.sqrt(a * d) # guarantees b^2 ≤ ad
M = np.array([[a, b], [b, d]])
min_eigs.append(np.linalg.eigvalsh(M).min())
check("smallest eigenvalue over 2000 samples ≥ -1e-12", bool(min(min_eigs) >= -1e-12), True)
# --- Step 3 ---
print_step(3, "Symmetry cannot be dropped: C = [[1, 0], [-4, 1]]")
C = np.array([[1., 0.], [-4., 1.]])
check("det(C)", np.linalg.det(C), 1)
x = np.array([1., 1.])
check("x^T C x, x = (1, 1)^T", x @ C @ x, -2)
check("symmetric part (C + C^T)/2", (C + C.T) / 2, [[1, -2], [-2, 1]])============================================================
Exercise 9-53 | A Pitfall in Testing Semidefiniteness
============================================================
▶ Step 1: Counterexample A = diag(0, -1)
----------------------------------------
[(D_1, D_2)]
Computed value: [np.float64(0.0), np.float64(0.0)]
Expected value: [0, 0]
check passed
[x^T A x, x = (0, 1)^T]
Computed value: -1.0
Expected value: -1
check passed
[eigenvalues of A]
Computed value: [-1. 0.]
Expected value: [-1, 0]
check passed
[original conditions a≥0, b≥0, det≥0 hold]
Computed value: True
Expected value: True
check passed
▶ Step 2: Corrected statement: symmetric, a≥0, d≥0, det≥0 ⇒ positive semidefinite (random test)
----------------------------------------
[smallest eigenvalue over 2000 samples ≥ -1e-12]
Computed value: True
Expected value: True
check passed
▶ Step 3: Symmetry cannot be dropped: C = [[1, 0], [-4, 1]]
----------------------------------------
[det(C)]
Computed value: 1.0
Expected value: 1
check passed
[x^T C x, x = (1, 1)^T]
Computed value: -2.0
Expected value: -2
check passed
[symmetric part (C + C^T)/2]
Computed value: [[ 1. -2.]
[-2. 1.]]
Expected value: [[1, -2], [-2, 1]]
check passed
# ============================================================
print_header("Exercise 9-54 | Computing Cholesky Decompositions")
# ============================================================
def cholesky_upper_54(A):
# Compute the upper triangular L row by row so that A = L^T L
A = np.asarray(A, dtype=float)
n = A.shape[0]
L = np.zeros_like(A)
for i in range(n):
L[i, i] = np.sqrt(A[i, i] - np.sum(L[:i, i]**2))
for j in range(i + 1, n):
L[i, j] = (A[i, j] - np.sum(L[:i, i] * L[:i, j])) / L[i, i]
return L
A = np.array([[1, -3], [-3, 13]], dtype=float)
B = np.array([[4, -2, 4], [-2, 10, -5], [4, -5, 9]], dtype=float)
# --- Step 1 ---
print_step(1, "Decompose A")
L_A = cholesky_upper_54(A)
check("L_A", L_A, [[1, -3], [0, 2]])
check("L_A^T L_A", L_A.T @ L_A, A)
# --- Step 2 ---
print_step(2, "Decompose B")
L_B = cholesky_upper_54(B)
check("L_B", L_B, [[2, -1, 2], [0, 3, -1], [0, 0, 2]])
check("L_B^T L_B", L_B.T @ L_B, B)
# --- Step 3 ---
print_step(3, "Compare with numpy (numpy gives a lower triangular G with A = G G^T, so L = G^T)")
check("L_A = cholesky(A)^T", L_A, np.linalg.cholesky(A).T)
check("L_B = cholesky(B)^T", L_B, np.linalg.cholesky(B).T)============================================================
Exercise 9-54 | Computing Cholesky Decompositions
============================================================
▶ Step 1: Decompose A
----------------------------------------
[L_A]
Computed value: [[ 1. -3.]
[ 0. 2.]]
Expected value: [[1, -3], [0, 2]]
check passed
[L_A^T L_A]
Computed value: [[ 1. -3.]
[-3. 13.]]
Expected value: [[ 1. -3.]
[-3. 13.]]
check passed
▶ Step 2: Decompose B
----------------------------------------
[L_B]
Computed value: [[ 2. -1. 2.]
[ 0. 3. -1.]
[ 0. 0. 2.]]
Expected value: [[2, -1, 2], [0, 3, -1], [0, 0, 2]]
check passed
[L_B^T L_B]
Computed value: [[ 4. -2. 4.]
[-2. 10. -5.]
[ 4. -5. 9.]]
Expected value: [[ 4. -2. 4.]
[-2. 10. -5.]
[ 4. -5. 9.]]
check passed
▶ Step 3: Compare with numpy (numpy gives a lower triangular G with A = G G^T, so L = G^T)
----------------------------------------
[L_A = cholesky(A)^T]
Computed value: [[ 1. -3.]
[ 0. 2.]]
Expected value: [[ 1. -3.]
[ 0. 2.]]
check passed
[L_B = cholesky(B)^T]
Computed value: [[ 2. -1. 2.]
[ 0. 3. -1.]
[ 0. 0. 2.]]
Expected value: [[ 2. -1. 2.]
[ 0. 3. -1.]
[ 0. 0. 2.]]
check passed
# ============================================================
print_header("Exercise 9-55 | Cholesky Diagonal Entries and Leading Principal Minors")
# ============================================================
def leading_minors_55(M):
# Return (D_0, D_1, ..., D_n), where D_0 = 1 and D_k = determinant of the upper-left k×k submatrix
n = M.shape[0]
return np.array([1.0] + [np.linalg.det(M[:k, :k]) for k in range(1, n + 1)])
A = np.array([[1, -3], [-3, 13]], dtype=float)
B = np.array([[4, -2, 4], [-2, 10, -5], [4, -5, 9]], dtype=float)
# --- Step 1 ---
print_step(1, "Random positive definite matrix: L_kk = sqrt(D_{k+1} / D_k), D_k = Π L_ii^2")
rng = np.random.default_rng(55)
G = rng.standard_normal((6, 6))
M = G.T @ G + 0.5 * np.eye(6) # positive definite symmetric matrix of order 6
L = np.linalg.cholesky(M).T # upper triangular, M = L^T L
D = leading_minors_55(M)
check("diagonal entries of L", np.diag(L), np.sqrt(D[1:] / D[:-1]))
check("D_k = Π_{i<k} L_ii^2 (k = 0..6)", D, np.concatenate([[1.0], np.cumprod(np.diag(L)**2)]))
# --- Step 2 ---
print_step(2, "Redo A: D = (1, 1, 4)")
D_A = leading_minors_55(A)
check("(D_0, D_1, D_2)", D_A, [1, 1, 4])
diag_A = np.sqrt(D_A[1:] / D_A[:-1])
check("(L_00, L_11)", diag_A, [1, 2])
L_A = np.array([[diag_A[0], A[0, 1] / diag_A[0]], [0, diag_A[1]]])
check("L_A", L_A, [[1, -3], [0, 2]])
check("L_A^T L_A", L_A.T @ L_A, A)
# --- Step 3 ---
print_step(3, "Redo B: D = (1, 4, 36, 144)")
D_B = leading_minors_55(B)
check("(D_0, D_1, D_2, D_3)", D_B, [1, 4, 36, 144])
d = np.sqrt(D_B[1:] / D_B[:-1])
check("(L_00, L_11, L_22)", d, [2, 3, 2])
L01 = B[0, 1] / d[0]
L02 = B[0, 2] / d[0]
L12 = (B[1, 2] - L01 * L02) / d[1]
L_B = np.array([[d[0], L01, L02], [0, d[1], L12], [0, 0, d[2]]])
check("L_B", L_B, [[2, -1, 2], [0, 3, -1], [0, 0, 2]])
check("L_B^T L_B", L_B.T @ L_B, B)
check("det(L_B) = sqrt(D_3)", np.linalg.det(L_B), 12)============================================================
Exercise 9-55 | Cholesky Diagonal Entries and Leading Principal Minors
============================================================
▶ Step 1: Random positive definite matrix: L_kk = sqrt(D_{k+1} / D_k), D_k = Π L_ii^2
----------------------------------------
[diagonal entries of L]
Computed value: [3.2684 3.4051 2.2012 2.0533 1.7037 1.0058]
Expected value: [3.2684 3.4051 2.2012 2.0533 1.7037 1.0058]
check passed
[D_k = Π_{i<k} L_ii^2 (k = 0..6)]
Computed value: [ 1. 10.6827 123.864 600.1598 2530.1867 7344.1015 7430.1299]
Expected value: [ 1. 10.6827 123.864 600.1598 2530.1867 7344.1015 7430.1299]
check passed
▶ Step 2: Redo A: D = (1, 1, 4)
----------------------------------------
[(D_0, D_1, D_2)]
Computed value: [1. 1. 4.]
Expected value: [1, 1, 4]
check passed
[(L_00, L_11)]
Computed value: [1. 2.]
Expected value: [1, 2]
check passed
[L_A]
Computed value: [[ 1. -3.]
[ 0. 2.]]
Expected value: [[1, -3], [0, 2]]
check passed
[L_A^T L_A]
Computed value: [[ 1. -3.]
[-3. 13.]]
Expected value: [[ 1. -3.]
[-3. 13.]]
check passed
▶ Step 3: Redo B: D = (1, 4, 36, 144)
----------------------------------------
[(D_0, D_1, D_2, D_3)]
Computed value: [ 1. 4. 36. 144.]
Expected value: [1, 4, 36, 144]
check passed
[(L_00, L_11, L_22)]
Computed value: [2. 3. 2.]
Expected value: [2, 3, 2]
check passed
[L_B]
Computed value: [[ 2. -1. 2.]
[ 0. 3. -1.]
[ 0. 0. 2.]]
Expected value: [[2, -1, 2], [0, 3, -1], [0, 0, 2]]
check passed
[L_B^T L_B]
Computed value: [[ 4. -2. 4.]
[-2. 10. -5.]
[ 4. -5. 9.]]
Expected value: [[ 4. -2. 4.]
[-2. 10. -5.]
[ 4. -5. 9.]]
check passed
[det(L_B) = sqrt(D_3)]
Computed value: 12.0
Expected value: 12
check passed
# ============================================================
print_header("Exercise 9-56 | Complex Cholesky Decomposition")
# ============================================================
A = np.array([[2, 1 - 1j], [1 + 1j, 2]])
# --- Step 1 ---
print_step(1, "Hermitian property and leading principal minors")
check("A^* = A", A.conj().T, A)
D = np.array([A[0, 0], np.linalg.det(A)])
check("(D_1, D_2)", D, [2, 2])
check("eigenvalues 2 ∓ √2", np.linalg.eigvalsh(A), [2 - np.sqrt(2), 2 + np.sqrt(2)])
# --- Step 2 ---
print_step(2, "Upper triangular L with A = L^* L")
L = np.array([[np.sqrt(2), (1 - 1j) / np.sqrt(2)], [0, 1]])
check("L^* L", L.conj().T @ L, A)
check("L_11 = sqrt(D_2 / D_1)", L[1, 1], np.sqrt((D[1] / D[0]).real))
# numpy gives a lower triangular G with A = G G^*, so L = G^*
check("L = cholesky(A)^*", L, np.linalg.cholesky(A).conj().T)
# --- Step 3 ---
print_step(3, "x^* A x = ||L x||^2 (random complex vector)")
rng = np.random.default_rng(56)
x = rng.standard_normal(2) + 1j * rng.standard_normal(2)
check("x^* A x", (x.conj() @ A @ x).real, np.linalg.norm(L @ x)**2)============================================================
Exercise 9-56 | Complex Cholesky Decomposition
============================================================
▶ Step 1: Hermitian property and leading principal minors
----------------------------------------
[A^* = A]
Computed value: [[2.-0.j 1.-1.j]
[1.+1.j 2.-0.j]]
Expected value: [[2.+0.j 1.-1.j]
[1.+1.j 2.+0.j]]
check passed
[(D_1, D_2)]
Computed value: [2.+0.j 2.+0.j]
Expected value: [2, 2]
check passed
[eigenvalues 2 ∓ √2]
Computed value: [0.5858 3.4142]
Expected value: [np.float64(0.5857864376269049), np.float64(3.414213562373095)]
check passed
▶ Step 2: Upper triangular L with A = L^* L
----------------------------------------
[L^* L]
Computed value: [[2.+0.j 1.-1.j]
[1.+1.j 2.+0.j]]
Expected value: [[2.+0.j 1.-1.j]
[1.+1.j 2.+0.j]]
check passed
[L_11 = sqrt(D_2 / D_1)]
Computed value: (1+0j)
Expected value: 1.0
check passed
[L = cholesky(A)^*]
Computed value: [[1.4142+0.j 0.7071-0.7071j]
[0. +0.j 1. +0.j ]]
Expected value: [[1.4142-0.j 0.7071-0.7071j]
[0. -0.j 1. -0.j ]]
check passed
▶ Step 3: x^* A x = ||L x||^2 (random complex vector)
----------------------------------------
[x^* A x]
Computed value: 9.552299241830477
Expected value: 9.552299241830479
check passed
Bilinear Forms and Quadratic Forms¶
# ============================================================
print_header("Exercise 9-57 | Matrix Representation of a Quadratic Form")
# ============================================================
rng = np.random.default_rng(57)
# --- Step 1 ---
print_step(1, "Real quadratic form: compare coefficients to get the symmetric matrix A")
A = np.array([[3., 2.], [2., 5.]])
check("A is symmetric", A, A.T)
for _ in range(3):
x = rng.standard_normal(2)
q_direct = 3*x[0]**2 + 4*x[0]*x[1] + 5*x[1]**2
check("x^T A x agrees with q(x)", x @ A @ x, q_direct)
# --- Step 2 ---
print_step(2, "Hermitian quadratic form: compare coefficients to get B")
B = np.array([[2, 1+1j], [1-1j, 4]])
check("B is a Hermitian matrix (B^* = B)", B, B.conj().T)
for _ in range(3):
z = rng.standard_normal(2) + 1j*rng.standard_normal(2)
qz = (2*abs(z[0])**2 + (1+1j)*np.conj(z[0])*z[1]
+ (1-1j)*z[0]*np.conj(z[1]) + 4*abs(z[1])**2)
check("z^* B z agrees with q(z)", z.conj() @ B @ z, qz)
check("Imaginary part of q(z) is 0", np.imag(qz), 0.0)============================================================
Exercise 9-57 | Matrix Representation of a Quadratic Form
============================================================
▶ Step 1: Real quadratic form: compare coefficients to get the symmetric matrix A
----------------------------------------
[A is symmetric]
Computed value: [[3. 2.]
[2. 5.]]
Expected value: [[3. 2.]
[2. 5.]]
check passed
[x^T A x agrees with q(x)]
Computed value: 2.0012286123012757
Expected value: 2.0012286123012757
check passed
[x^T A x agrees with q(x)]
Computed value: 24.48958840003455
Expected value: 24.48958840003455
check passed
[x^T A x agrees with q(x)]
Computed value: 0.975927973372148
Expected value: 0.9759279733721479
check passed
▶ Step 2: Hermitian quadratic form: compare coefficients to get B
----------------------------------------
[B is a Hermitian matrix (B^* = B)]
Computed value: [[2.+0.j 1.+1.j]
[1.-1.j 4.+0.j]]
Expected value: [[2.-0.j 1.+1.j]
[1.-1.j 4.-0.j]]
check passed
[z^* B z agrees with q(z)]
Computed value: (9.143377095140135+0j)
Expected value: (9.143377095140135+0j)
check passed
[Imaginary part of q(z) is 0]
Computed value: 0.0
Expected value: 0.0
check passed
[z^* B z agrees with q(z)]
Computed value: (9.177125045551456+0j)
Expected value: (9.177125045551456+0j)
check passed
[Imaginary part of q(z) is 0]
Computed value: 0.0
Expected value: 0.0
check passed
[z^* B z agrees with q(z)]
Computed value: (6.6839606658787325+0j)
Expected value: (6.683960665878732+0j)
check passed
[Imaginary part of q(z) is 0]
Computed value: 0.0
Expected value: 0.0
check passed
# ============================================================
print_header("Exercise 9-58 | Trace under Similarity and Congruence")
# ============================================================
A = np.array([[1., 2.], [2., 1.]])
P = np.array([[1., 1.], [0., 2.]])
# --- Step 1 ---
print_step(1, "Linear operator: similarity transformation B = P^{-1} A P")
Pinv = np.linalg.inv(P)
check("P^{-1}", Pinv, [[1, -0.5], [0, 0.5]])
B = Pinv @ A @ P
check("B", B, [[0, 3], [1, 2]])
check("tr(B) = tr(A) = 2", [np.trace(B), np.trace(A)], [2, 2])
check("Eigenvalues of B and A (in increasing order)",
[np.sort(np.linalg.eigvals(B).real), np.linalg.eigvalsh(A)], [[-1, 3], [-1, 3]])
# --- Step 2 ---
print_step(2, "Quadratic form: congruence transformation C = P^T A P")
C = P.T @ A @ P
check("C", C, [[1, 5], [5, 13]])
check("tr(C) = 14, not equal to tr(A) = 2", np.trace(C), 14)
y = np.array([0.3, -1.2])
check("q_A(Py) = y^T C y", (P @ y) @ A @ (P @ y), y @ C @ y)
# --- Step 3 ---
print_step(3, "Quantities preserved by congruence: signs of the eigenvalues and of the determinant")
eigC = np.linalg.eigvalsh(C)
check("Eigenvalues of C: 7 ± sqrt(61)", eigC, [7 - np.sqrt(61), 7 + np.sqrt(61)])
check("Numbers of positive and negative eigenvalues of C and A: (1, 1)",
[np.sum(eigC > 0), np.sum(eigC < 0)], [1, 1])
check("det C = det A · (det P)^2 = -12", np.linalg.det(C), np.linalg.det(A) * np.linalg.det(P)**2)============================================================
Exercise 9-58 | Trace under Similarity and Congruence
============================================================
▶ Step 1: Linear operator: similarity transformation B = P^{-1} A P
----------------------------------------
[P^{-1}]
Computed value: [[ 1. -0.5]
[ 0. 0.5]]
Expected value: [[1, -0.5], [0, 0.5]]
check passed
[B]
Computed value: [[0. 3.]
[1. 2.]]
Expected value: [[0, 3], [1, 2]]
check passed
[tr(B) = tr(A) = 2]
Computed value: [np.float64(2.0), np.float64(2.0)]
Expected value: [2, 2]
check passed
[Eigenvalues of B and A (in increasing order)]
Computed value: [array([-1., 3.]), array([-1., 3.])]
Expected value: [[-1, 3], [-1, 3]]
check passed
▶ Step 2: Quadratic form: congruence transformation C = P^T A P
----------------------------------------
[C]
Computed value: [[ 1. 5.]
[ 5. 13.]]
Expected value: [[1, 5], [5, 13]]
check passed
[tr(C) = 14, not equal to tr(A) = 2]
Computed value: 14.0
Expected value: 14
check passed
[q_A(Py) = y^T C y]
Computed value: 15.209999999999997
Expected value: 15.209999999999997
check passed
▶ Step 3: Quantities preserved by congruence: signs of the eigenvalues and of the determinant
----------------------------------------
[Eigenvalues of C: 7 ± sqrt(61)]
Computed value: [-0.8102 14.8102]
Expected value: [np.float64(-0.810249675906654), np.float64(14.810249675906654)]
check passed
[Numbers of positive and negative eigenvalues of C and A: (1, 1)]
Computed value: [np.int64(1), np.int64(1)]
Expected value: [1, 1]
check passed
[det C = det A · (det P)^2 = -12]
Computed value: -11.999999999999995
Expected value: -11.999999999999998
check passed
# ============================================================
print_header("Exercise 9-60 | Congruence Classification of 2×2 Symmetric Matrices")
# ============================================================
def congruence_canon_60(A, tol=1e-12):
# Orthogonal diagonalization → sort in the order "positive, negative, zero" (consistent with the six canonical forms of the exercise) → scale; return P and P^T A P
lam, Q = np.linalg.eigh(A)
sgn = np.where(lam > tol, 1, np.where(lam < -tol, -1, 0))
rank_key = np.where(sgn == 1, 0, np.where(sgn == -1, 1, 2))
order = np.argsort(rank_key, kind="stable") # +1 first, -1 in the middle, 0 last
lam, Q, sgn = lam[order], Q[:, order], sgn[order]
d = 1/np.sqrt(np.where(sgn != 0, np.abs(lam), 1.0))
P = Q @ np.diag(d)
return P, P.T @ A @ P
samples = [
("Positive definite", [[2, 1], [1, 2]], np.diag([1, 1])),
("Negative definite", [[-3, 1], [1, -2]], np.diag([-1, -1])),
("Positive semidefinite (one positive, one zero)", [[1, 1], [1, 1]], np.diag([1, 0])),
("Negative semidefinite (one negative, one zero)", [[-1, 1], [1, -1]], np.diag([-1, 0])),
("Indefinite", [[1, 2], [2, 1]], np.diag([1, -1])),
("Zero matrix", [[0, 0], [0, 0]], np.diag([0, 0])),
]
# --- Step 1 ---
print_step(1, "Take a representative of each class and construct P so that P^T A P is the canonical form")
for name, A, target in samples:
A = np.array(A, dtype=float)
P, canon = congruence_canon_60(A)
check(f"{name}: P^T A P", canon, target)
assert abs(np.linalg.det(P)) > 1e-12 # P is invertible
# --- Step 2 ---
print_step(2, "Independent classification: predict the canonical form from the signs of det A and tr A alone, then compare with the constructed result")
def classify_by_det_tr_60(A, tol=1e-12):
# Order 2: if det > 0, the two eigenvalues have the same sign (look at tr); if det < 0, one is positive and one is negative; if det = 0, at least one is zero (look at tr)
det, tr = np.linalg.det(A), np.trace(A)
if det > tol:
return np.diag([1, 1]) if tr > 0 else np.diag([-1, -1])
if det < -tol:
return np.diag([1, -1])
if tr > tol:
return np.diag([1, 0])
if tr < -tol:
return np.diag([-1, 0])
return np.diag([0, 0])
rng = np.random.default_rng(60)
tests = [rng.standard_normal((2, 2)) for _ in range(6)]
tests = [M + M.T for M in tests]
v = rng.standard_normal(2)
tests += [np.outer(v, v), -np.outer(v, v), 3*np.eye(2), -2*np.eye(2)] # add semidefinite and positive/negative definite cases
for A in tests:
P, canon = congruence_canon_60(A)
check("P^T A P agrees with the det/tr classification", canon, classify_by_det_tr_60(A))
assert abs(np.linalg.det(P)) > 1e-12============================================================
Exercise 9-60 | Congruence Classification of 2×2 Symmetric Matrices
============================================================
▶ Step 1: Take a representative of each class and construct P so that P^T A P is the canonical form
----------------------------------------
[Positive definite: P^T A P]
Computed value: [[1. 0.]
[0. 1.]]
Expected value: [[1 0]
[0 1]]
check passed
[Negative definite: P^T A P]
Computed value: [[-1. 0.]
[ 0. -1.]]
Expected value: [[-1 0]
[ 0 -1]]
check passed
[Positive semidefinite (one positive, one zero): P^T A P]
Computed value: [[1. 0.]
[0. 0.]]
Expected value: [[1 0]
[0 0]]
check passed
[Negative semidefinite (one negative, one zero): P^T A P]
Computed value: [[-1. 0.]
[ 0. 0.]]
Expected value: [[-1 0]
[ 0 0]]
check passed
[Indefinite: P^T A P]
Computed value: [[ 1. 0.]
[ 0. -1.]]
Expected value: [[ 1 0]
[ 0 -1]]
check passed
[Zero matrix: P^T A P]
Computed value: [[0. 0.]
[0. 0.]]
Expected value: [[0 0]
[0 0]]
check passed
▶ Step 2: Independent classification: predict the canonical form from the signs of det A and tr A alone, then compare with the constructed result
----------------------------------------
[P^T A P agrees with the det/tr classification]
Computed value: [[-1. 0.]
[ 0. -1.]]
Expected value: [[-1 0]
[ 0 -1]]
check passed
[P^T A P agrees with the det/tr classification]
Computed value: [[ 1. 0.]
[-0. -1.]]
Expected value: [[ 1 0]
[ 0 -1]]
check passed
[P^T A P agrees with the det/tr classification]
Computed value: [[ 1. -0.]
[-0. -1.]]
Expected value: [[ 1 0]
[ 0 -1]]
check passed
[P^T A P agrees with the det/tr classification]
Computed value: [[ 1. 0.]
[ 0. -1.]]
Expected value: [[ 1 0]
[ 0 -1]]
check passed
[P^T A P agrees with the det/tr classification]
Computed value: [[-1. 0.]
[-0. -1.]]
Expected value: [[-1 0]
[ 0 -1]]
check passed
[P^T A P agrees with the det/tr classification]
Computed value: [[ 1. -0.]
[-0. -1.]]
Expected value: [[ 1 0]
[ 0 -1]]
check passed
[P^T A P agrees with the det/tr classification]
Computed value: [[1. 0.]
[0. 0.]]
Expected value: [[1 0]
[0 0]]
check passed
[P^T A P agrees with the det/tr classification]
Computed value: [[-1. -0.]
[ 0. 0.]]
Expected value: [[-1 0]
[ 0 0]]
check passed
[P^T A P agrees with the det/tr classification]
Computed value: [[1. 0.]
[0. 1.]]
Expected value: [[1 0]
[0 1]]
check passed
[P^T A P agrees with the det/tr classification]
Computed value: [[-1. 0.]
[ 0. -1.]]
Expected value: [[-1 0]
[ 0 -1]]
check passed
# ============================================================
print_header("Exercise 9-62 | Eliminating Cross Terms (Real Quadratic Forms)")
# ============================================================
rng = np.random.default_rng(62)
# --- Step 1 ---
print_step(1, "Two variables: orthogonal matrix Q and Q^T A Q")
A = np.array([[2., 1.], [1., 2.]])
Q = np.array([[1., -1.], [1., 1.]]) / np.sqrt(2)
check("Q^T Q = I", Q.T @ Q, np.eye(2))
check("det Q = 1 (rotation)", np.linalg.det(Q), 1.0)
check("Q^T A Q = diag(3, 1)", Q.T @ A @ Q, np.diag([3, 1]))
# --- Step 2 ---
print_step(2, "Substitute x = Q y: q(x) = 3 y0^2 + y1^2")
for _ in range(3):
y = rng.standard_normal(2)
x = Q @ y
q = 2*x[0]**2 + 2*x[0]*x[1] + 2*x[1]**2
check("q(x) vs. 3y0^2 + y1^2", q, 3*y[0]**2 + y[1]**2)
# --- Step 3 ---
print_step(3, "General n variables: Q^T = Q^{-1} makes congruence coincide with similarity")
M = rng.standard_normal((4, 4)); A4 = (M + M.T) / 2
lam, Q4 = np.linalg.eigh(A4)
check("Q^T = Q^{-1}", Q4.T, np.linalg.inv(Q4))
check("Q^T A Q = Λ", Q4.T @ A4 @ Q4, np.diag(lam))
x = rng.standard_normal(4)
y = Q4.T @ x
check("x^T A x = Σ λ_i <q_i|x>^2", x @ A4 @ x, np.sum(lam * y**2))============================================================
Exercise 9-62 | Eliminating Cross Terms (Real Quadratic Forms)
============================================================
▶ Step 1: Two variables: orthogonal matrix Q and Q^T A Q
----------------------------------------
[Q^T Q = I]
Computed value: [[1. 0.]
[0. 1.]]
Expected value: [[1. 0.]
[0. 1.]]
check passed
[det Q = 1 (rotation)]
Computed value: 0.9999999999999998
Expected value: 1.0
check passed
[Q^T A Q = diag(3, 1)]
Computed value: [[3. 0.]
[0. 1.]]
Expected value: [[3 0]
[0 1]]
check passed
▶ Step 2: Substitute x = Q y: q(x) = 3 y0^2 + y1^2
----------------------------------------
[q(x) vs. 3y0^2 + y1^2]
Computed value: 5.310109708859807
Expected value: 5.310109708859809
check passed
[q(x) vs. 3y0^2 + y1^2]
Computed value: 0.842607582274886
Expected value: 0.8426075822748862
check passed
[q(x) vs. 3y0^2 + y1^2]
Computed value: 2.611367428461757
Expected value: 2.611367428461758
check passed
▶ Step 3: General n variables: Q^T = Q^{-1} makes congruence coincide with similarity
----------------------------------------
[Q^T = Q^{-1}]
Computed value: [[ 0.2592 0.6353 -0.543 0.4841]
[ 0.7283 0.3574 0.5083 -0.2887]
[-0.0738 0.2177 -0.5161 -0.8251]
[ 0.63 -0.6491 -0.4247 0.038 ]]
Expected value: [[ 0.2592 0.6353 -0.543 0.4841]
[ 0.7283 0.3574 0.5083 -0.2887]
[-0.0738 0.2177 -0.5161 -0.8251]
[ 0.63 -0.6491 -0.4247 0.038 ]]
check passed
[Q^T A Q = Λ]
Computed value: [[-2.4894 -0. 0. -0. ]
[-0. -1.5035 -0. -0. ]
[ 0. -0. 1.4873 0. ]
[-0. -0. 0. 1.7897]]
Expected value: [[-2.4894 0. 0. 0. ]
[ 0. -1.5035 0. 0. ]
[ 0. 0. 1.4873 0. ]
[ 0. 0. 0. 1.7897]]
check passed
[x^T A x = Σ λ_i <q_i|x>^2]
Computed value: -4.917191772937468
Expected value: -4.917191772937469
check passed
# ============================================================
print_header("Exercise 9-63 | Eliminating Cross Terms (Hermitian Quadratic Forms)")
# ============================================================
rng = np.random.default_rng(63)
# --- Step 1 ---
print_step(1, "H is a Hermitian matrix with eigenvalues ±1")
H = np.array([[0, 1j], [-1j, 0]])
check("H^* = H", H.conj().T, H)
check("Eigenvalues (in increasing order)", np.linalg.eigvalsh(H), [-1, 1])
# --- Step 2 ---
print_step(2, "Unitary matrix U and U^* H U")
U = np.array([[1j, -1j], [1, 1]]) / np.sqrt(2)
check("U^* U = I", U.conj().T @ U, np.eye(2))
check("U^* H U = diag(1, -1)", U.conj().T @ H @ U, np.diag([1, -1]))
# --- Step 3 ---
print_step(3, "Random z: q(z) = |w0|^2 - |w1|^2, w = U^* z")
for _ in range(3):
z = rng.standard_normal(2) + 1j*rng.standard_normal(2)
q = 1j*np.conj(z[0])*z[1] - 1j*z[0]*np.conj(z[1])
w = U.conj().T @ z
check("q(z) vs. |w0|^2 - |w1|^2", q, abs(w[0])**2 - abs(w[1])**2)
# --- Step 4 ---
print_step(4, "General n: U^* H U = Λ (real diagonal)")
M = rng.standard_normal((3, 3)) + 1j*rng.standard_normal((3, 3))
H3 = (M + M.conj().T) / 2
lam, U3 = np.linalg.eigh(H3)
check("U^* H U = Λ", U3.conj().T @ H3 @ U3, np.diag(lam))
z = rng.standard_normal(3) + 1j*rng.standard_normal(3)
check("z^* H z = Σ λ_i |<u_i|z>|^2", z.conj() @ H3 @ z, np.sum(lam * abs(U3.conj().T @ z)**2))============================================================
Exercise 9-63 | Eliminating Cross Terms (Hermitian Quadratic Forms)
============================================================
▶ Step 1: H is a Hermitian matrix with eigenvalues ±1
----------------------------------------
[H^* = H]
Computed value: [[ 0.-0.j -0.+1.j]
[ 0.-1.j 0.-0.j]]
Expected value: [[ 0.+0.j 0.+1.j]
[-0.-1.j 0.+0.j]]
check passed
[Eigenvalues (in increasing order)]
Computed value: [-1. 1.]
Expected value: [-1, 1]
check passed
▶ Step 2: Unitary matrix U and U^* H U
----------------------------------------
[U^* U = I]
Computed value: [[1.+0.j 0.+0.j]
[0.+0.j 1.+0.j]]
Expected value: [[1. 0.]
[0. 1.]]
check passed
[U^* H U = diag(1, -1)]
Computed value: [[ 1.+0.j 0.+0.j]
[ 0.+0.j -1.+0.j]]
Expected value: [[ 1 0]
[ 0 -1]]
check passed
▶ Step 3: Random z: q(z) = |w0|^2 - |w1|^2, w = U^* z
----------------------------------------
[q(z) vs. |w0|^2 - |w1|^2]
Computed value: (-7.586351917374334+0j)
Expected value: -7.586351917374331
check passed
[q(z) vs. |w0|^2 - |w1|^2]
Computed value: (2.5036005711435116+0j)
Expected value: 2.5036005711435116
check passed
[q(z) vs. |w0|^2 - |w1|^2]
Computed value: (1.2903091506469337+0j)
Expected value: 1.290309150646931
check passed
▶ Step 4: General n: U^* H U = Λ (real diagonal)
----------------------------------------
[U^* H U = Λ]
Computed value: [[-1.4976+0.j -0. +0.j 0. +0.j]
[-0. -0.j 1.3924-0.j 0. +0.j]
[ 0. +0.j 0. -0.j 2.3218+0.j]]
Expected value: [[-1.4976 0. 0. ]
[ 0. 1.3924 0. ]
[ 0. 0. 2.3218]]
check passed
[z^* H z = Σ λ_i |<u_i|z>|^2]
Computed value: (3.02819130959171+0j)
Expected value: 3.0281913095917092
check passed
Applications of the Principal Axis Theorem: Conics, Quadric Surfaces, and Data Science¶
# ============================================================
print_header("Exercise 9-64 | Principal Axes of a Conic")
# ============================================================
# --- Step 1 ---
print_step(1, "Matrix form")
A = np.array([[5., -2.], [-2., 8.]])
xy = np.array([1.3, -0.7])
check("x^T A x vs. 5x^2 - 4xy + 8y^2", xy @ A @ xy, 5*xy[0]**2 - 4*xy[0]*xy[1] + 8*xy[1]**2)
# --- Step 2 ---
print_step(2, "Orthogonal diagonalization: P^T A P = diag(4, 9)")
check("Eigenvalues", np.linalg.eigvalsh(A), [4, 9])
P = np.array([[2., -1.], [1., 2.]]) / np.sqrt(5)
check("P^T P = I", P.T @ P, np.eye(2))
check("det P = 1", np.linalg.det(P), 1.0)
check("P^T A P", P.T @ A @ P, np.diag([4, 9]))
check("Rotation angle (degrees)", np.degrees(np.arctan2(P[1, 0], P[0, 0])), np.degrees(np.arctan(0.5)))
# --- Step 3 ---
print_step(3, "Semi-axis lengths and a check of the endpoints")
semi = np.sqrt(36 / np.array([4., 9.]))
check("Semi-major axis, semi-minor axis", semi, [3, 2])
check("Length of the major axis, length of the minor axis", 2*semi, [6, 4])
for v in (3*P[:, 0], -3*P[:, 0], 2*P[:, 1], -2*P[:, 1]):
check("Endpoint lies on the curve", v @ A @ v, 36)============================================================
Exercise 9-64 | Principal Axes of a Conic
============================================================
▶ Step 1: Matrix form
----------------------------------------
[x^T A x vs. 5x^2 - 4xy + 8y^2]
Computed value: 16.01
Expected value: 16.009999999999998
check passed
▶ Step 2: Orthogonal diagonalization: P^T A P = diag(4, 9)
----------------------------------------
[Eigenvalues]
Computed value: [4. 9.]
Expected value: [4, 9]
check passed
[P^T P = I]
Computed value: [[1. 0.]
[0. 1.]]
Expected value: [[1. 0.]
[0. 1.]]
check passed
[det P = 1]
Computed value: 1.0
Expected value: 1.0
check passed
[P^T A P]
Computed value: [[ 4. -0.]
[ 0. 9.]]
Expected value: [[4 0]
[0 9]]
check passed
[Rotation angle (degrees)]
Computed value: 26.56505117707799
Expected value: 26.56505117707799
check passed
▶ Step 3: Semi-axis lengths and a check of the endpoints
----------------------------------------
[Semi-major axis, semi-minor axis]
Computed value: [3. 2.]
Expected value: [3, 2]
check passed
[Length of the major axis, length of the minor axis]
Computed value: [6. 4.]
Expected value: [6, 4]
check passed
[Endpoint lies on the curve]
Computed value: 36.0
Expected value: 36
check passed
[Endpoint lies on the curve]
Computed value: 36.0
Expected value: 36
check passed
[Endpoint lies on the curve]
Computed value: 36.0
Expected value: 36
check passed
[Endpoint lies on the curve]
Computed value: 36.0
Expected value: 36
check passed
# ============================================================
print_header("Exercise 9-65 | Classification of a Quadric Surface")
# ============================================================
rng = np.random.default_rng(65)
# --- Step 1 ---
print_step(1, "x^T A x = 2(xy + yz + zx)")
A = np.ones((3, 3)) - np.eye(3)
v = rng.standard_normal(3)
x, y, z = v
check("x^T A x", v @ A @ v, 2*(x*y + y*z + z*x))
# --- Step 2 ---
print_step(2, "Eigenvalues and type of surface")
lam = np.linalg.eigvalsh(A)
check("Eigenvalues (in increasing order)", lam, [-1, -1, 2])
check("tr A, det A", [np.trace(A), np.linalg.det(A)], [0, 2])
check("Numbers of positive and negative eigenvalues of (1/2)A: (1, 2) → hyperboloid of two sheets",
[np.sum(lam/2 > 0), np.sum(lam/2 < 0)], [1, 2])
vertex = np.ones(3) / np.sqrt(3)
check("Vertex (1,1,1)/sqrt(3) lies on the surface", vertex[0]*vertex[1] + vertex[1]*vertex[2] + vertex[0]*vertex[2], 1)
# --- Step 3 ---
print_step(3, "Rotation matrix P and the canonical form")
p0 = np.array([1., 1., 1.]) / np.sqrt(3)
p1 = np.array([1., -1., 0.]) / np.sqrt(2)
p2 = np.cross(p0, p1)
check("p2 = (1,1,-2)/sqrt(6)", p2, np.array([1., 1., -2.]) / np.sqrt(6))
P = np.column_stack([p0, p1, p2])
check("P^T P = I", P.T @ P, np.eye(3))
check("det P = 1", np.linalg.det(P), 1.0)
check("P^T A P = diag(2, -1, -1)", P.T @ A @ P, np.diag([2, -1, -1]))
# Two sheets: points on the surface satisfy |x'| >= 1
for _ in range(3):
yz = rng.standard_normal(2)
xp = np.sqrt(1 + (yz[0]**2 + yz[1]**2) / 2) * rng.choice([-1, 1])
pt = P @ np.array([xp, yz[0], yz[1]])
check("Point constructed from x' satisfies xy+yz+zx = 1", pt[0]*pt[1] + pt[1]*pt[2] + pt[0]*pt[2], 1)============================================================
Exercise 9-65 | Classification of a Quadric Surface
============================================================
▶ Step 1: x^T A x = 2(xy + yz + zx)
----------------------------------------
[x^T A x]
Computed value: -2.7104428224938504
Expected value: -2.7104428224938504
check passed
▶ Step 2: Eigenvalues and type of surface
----------------------------------------
[Eigenvalues (in increasing order)]
Computed value: [-1. -1. 2.]
Expected value: [-1, -1, 2]
check passed
[tr A, det A]
Computed value: [np.float64(0.0), np.float64(2.0)]
Expected value: [0, 2]
check passed
[Numbers of positive and negative eigenvalues of (1/2)A: (1, 2) → hyperboloid of two sheets]
Computed value: [np.int64(1), np.int64(2)]
Expected value: [1, 2]
check passed
[Vertex (1,1,1)/sqrt(3) lies on the surface]
Computed value: 1.0000000000000002
Expected value: 1
check passed
▶ Step 3: Rotation matrix P and the canonical form
----------------------------------------
[p2 = (1,1,-2)/sqrt(6)]
Computed value: [ 0.4082 0.4082 -0.8165]
Expected value: [ 0.4082 0.4082 -0.8165]
check passed
[P^T P = I]
Computed value: [[ 1. -0. -0.]
[-0. 1. 0.]
[-0. 0. 1.]]
Expected value: [[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
check passed
[det P = 1]
Computed value: 1.0
Expected value: 1.0
check passed
[P^T A P = diag(2, -1, -1)]
Computed value: [[ 2. 0. -0.]
[ 0. -1. 0.]
[ 0. 0. -1.]]
Expected value: [[ 2 0 0]
[ 0 -1 0]
[ 0 0 -1]]
check passed
[Point constructed from x' satisfies xy+yz+zx = 1]
Computed value: 1.0000000000000004
Expected value: 1
check passed
[Point constructed from x' satisfies xy+yz+zx = 1]
Computed value: 1.0
Expected value: 1
check passed
[Point constructed from x' satisfies xy+yz+zx = 1]
Computed value: 1.000000000000001
Expected value: 1
check passed
# ============================================================
print_header("Exercise 9-66 | Covariance Matrix and Principal Component Analysis")
# ============================================================
rng = np.random.default_rng(66)
# --- Step 1 ---
print_step(1, "Random centered data: Ω = A^T A is symmetric positive semidefinite")
n = 50
a = rng.standard_normal(n); a -= a.mean()
b = 0.6*a + rng.standard_normal(n); b -= b.mean()
Omega = np.array([[a @ a, a @ b], [a @ b, b @ b]]) / n
A = np.column_stack([a, b]) / np.sqrt(n)
check("Ω = A^T A", A.T @ A, Omega)
check("Ω is symmetric", Omega, Omega.T)
check("a, b linearly independent → Ω positive definite (all eigenvalues positive)", np.all(np.linalg.eigvalsh(Omega) > 0), True)
b_dep = 2*a
Omega_dep = np.array([[a @ a, a @ b_dep], [a @ b_dep, b_dep @ b_dep]]) / n
u_null = np.array([2., -1.]) / np.sqrt(5)
check("When b = 2a, det Ω = 0", np.linalg.det(Omega_dep), 0.0)
check("When b = 2a, u = (2,-1)/sqrt(5) gives u^T Ω u = 0", u_null @ Omega_dep @ u_null, 0.0)
a2 = rng.standard_normal(2); a2 -= a2.mean()
b2 = rng.standard_normal(2); b2 -= b2.mean()
Omega2 = np.array([[a2 @ a2, a2 @ b2], [a2 @ b2, b2 @ b2]]) / 2
check("Centered data with n = 2: det Ω = 0 (cannot be positive definite)", np.linalg.det(Omega2), 0.0)
# --- Step 2 ---
print_step(2, "Projected data have mean 0 and variance u^T Ω u")
theta = rng.uniform(0, 2*np.pi)
u = np.array([np.cos(theta), np.sin(theta)])
c = a*u[0] + b*u[1]
check("Mean", c.mean(), 0.0)
check("Variance", (c @ c) / n, u @ Omega @ u)
# --- Step 3 ---
print_step(3, "Eigenvalues and eigenvectors of Ω = [[4,2],[2,7]]")
W = np.array([[4., 2.], [2., 7.]])
check("Eigenvalues (in increasing order)", np.linalg.eigvalsh(W), [3, 8])
q0 = np.array([1., 2.]) / np.sqrt(5)
q1 = np.array([2., -1.]) / np.sqrt(5)
check("W q0 = 8 q0", W @ q0, 8*q0)
check("W q1 = 3 q1", W @ q1, 3*q1)
# --- Step 4 ---
print_step(4, "Scan u^T Ω u over the unit circle: maximum 8, minimum 3")
ts = np.linspace(0, np.pi, 200001)
U = np.vstack([np.cos(ts), np.sin(ts)])
vals = np.einsum("ij,ik,kj->j", U, W, U)
u_max = U[:, np.argmax(vals)]
u_min = U[:, np.argmin(vals)]
check("Maximum variance", vals.max(), 8)
check("Minimum variance", vals.min(), 3)
check("Direction of maximum variance = ±(1,2)/sqrt(5)", abs(u_max @ q0), 1.0)
check("Direction of minimum variance = ±(2,-1)/sqrt(5)", abs(u_min @ q1), 1.0)
check("Total variance = tr Ω", 8 + 3, np.trace(W))============================================================
Exercise 9-66 | Covariance Matrix and Principal Component Analysis
============================================================
▶ Step 1: Random centered data: Ω = A^T A is symmetric positive semidefinite
----------------------------------------
[Ω = A^T A]
Computed value: [[0.9292 0.3621]
[0.3621 1.1168]]
Expected value: [[0.9292 0.3621]
[0.3621 1.1168]]
check passed
[Ω is symmetric]
Computed value: [[0.9292 0.3621]
[0.3621 1.1168]]
Expected value: [[0.9292 0.3621]
[0.3621 1.1168]]
check passed
[a, b linearly independent → Ω positive definite (all eigenvalues positive)]
Computed value: True
Expected value: True
check passed
[When b = 2a, det Ω = 0]
Computed value: 0.0
Expected value: 0.0
check passed
[When b = 2a, u = (2,-1)/sqrt(5) gives u^T Ω u = 0]
Computed value: 0.0
Expected value: 0.0
check passed
[Centered data with n = 2: det Ω = 0 (cannot be positive definite)]
Computed value: -2.223753220512612e-18
Expected value: 0.0
check passed
▶ Step 2: Projected data have mean 0 and variance u^T Ω u
----------------------------------------
[Mean]
Computed value: 2.886579864025407e-17
Expected value: 0.0
check passed
[Variance]
Computed value: 1.2299369638017024
Expected value: 1.2299369638017024
check passed
▶ Step 3: Eigenvalues and eigenvectors of Ω = [[4,2],[2,7]]
----------------------------------------
[Eigenvalues (in increasing order)]
Computed value: [3. 8.]
Expected value: [3, 8]
check passed
[W q0 = 8 q0]
Computed value: [3.5777 7.1554]
Expected value: [3.5777 7.1554]
check passed
[W q1 = 3 q1]
Computed value: [ 2.6833 -1.3416]
Expected value: [ 2.6833 -1.3416]
check passed
▶ Step 4: Scan u^T Ω u over the unit circle: maximum 8, minimum 3
----------------------------------------
[Maximum variance]
Computed value: 7.999999999905701
Expected value: 8
check passed
[Minimum variance]
Computed value: 3.000000000094298
Expected value: 3
check passed
[Direction of maximum variance = ±(1,2)/sqrt(5)]
Computed value: 0.9999999999905701
Expected value: 1.0
check passed
[Direction of minimum variance = ±(2,-1)/sqrt(5)]
Computed value: 0.9999999999905701
Expected value: 1.0
check passed
[Total variance = tr Ω]
Computed value: 11
Expected value: 11.0
check passed
# ============================================================
print_header("Exercise 9-67 | Energy of a Quantum System")
# ============================================================
rng = np.random.default_rng(67)
H = np.array([[5, 2j], [-2j, 2]])
# --- Step 1 ---
print_step(1, "Eigenvalues and eigenvectors")
check("H^* = H", H.conj().T, H)
check("Eigenvalues (in increasing order)", np.linalg.eigvalsh(H), [1, 6])
u0 = np.array([1, 2j]) / np.sqrt(5)
u1 = np.array([2j, 1]) / np.sqrt(5)
check("H u0 = 1·u0", H @ u0, u0)
check("H u1 = 6·u1", H @ u1, 6*u1)
# --- Step 2 ---
print_step(2, "Unitary matrix U and the canonical form")
U = np.column_stack([u0, u1])
check("<u0|u1> = 0", np.vdot(u0, u1), 0)
check("U^* U = I", U.conj().T @ U, np.eye(2))
check("U^* H U = diag(1, 6)", U.conj().T @ H @ U, np.diag([1, 6]))
check("H = |u0><u0| + 6|u1><u1|", np.outer(u0, u0.conj()) + 6*np.outer(u1, u1.conj()), H)
# --- Step 3 ---
print_step(3, "Random states: h(z) is real and equals |w0|^2 + 6|w1|^2")
for _ in range(3):
z = rng.standard_normal(2) + 1j*rng.standard_normal(2)
h = z.conj() @ H @ z
w = U.conj().T @ z
check("Imaginary part of h(z)", h.imag, 0.0)
check("h(z) vs. |w0|^2 + 6|w1|^2", h.real, abs(w[0])**2 + 6*abs(w[1])**2)
check("h(z) vs. 5|z0|^2 + 2|z1|^2 - 4 Im(conj(z0) z1)",
h.real, 5*abs(z[0])**2 + 2*abs(z[1])**2 - 4*np.imag(np.conj(z[0])*z[1]))
zn = z / np.linalg.norm(z)
e = (zn.conj() @ H @ zn).real
check("Energy of a unit state lies in [1, 6]", 1 - 1e-12 <= e <= 6 + 1e-12, True)============================================================
Exercise 9-67 | Energy of a Quantum System
============================================================
▶ Step 1: Eigenvalues and eigenvectors
----------------------------------------
[H^* = H]
Computed value: [[ 5.-0.j -0.+2.j]
[ 0.-2.j 2.-0.j]]
Expected value: [[ 5.+0.j 0.+2.j]
[-0.-2.j 2.+0.j]]
check passed
[Eigenvalues (in increasing order)]
Computed value: [1. 6.]
Expected value: [1, 6]
check passed
[H u0 = 1·u0]
Computed value: [0.4472+0.j 0. +0.8944j]
Expected value: [0.4472+0.j 0. +0.8944j]
check passed
[H u1 = 6·u1]
Computed value: [0. +5.3666j 2.6833+0.j ]
Expected value: [0. +5.3666j 2.6833+0.j ]
check passed
▶ Step 2: Unitary matrix U and the canonical form
----------------------------------------
[<u0|u1> = 0]
Computed value: 0j
Expected value: 0
check passed
[U^* U = I]
Computed value: [[1.+0.j 0.+0.j]
[0.+0.j 1.+0.j]]
Expected value: [[1. 0.]
[0. 1.]]
check passed
[U^* H U = diag(1, 6)]
Computed value: [[1.+0.j 0.+0.j]
[0.+0.j 6.+0.j]]
Expected value: [[1 0]
[0 6]]
check passed
[H = |u0><u0| + 6|u1><u1|]
Computed value: [[5.+0.j 0.+2.j]
[0.-2.j 2.+0.j]]
Expected value: [[ 5.+0.j 0.+2.j]
[-0.-2.j 2.+0.j]]
check passed
▶ Step 3: Random states: h(z) is real and equals |w0|^2 + 6|w1|^2
----------------------------------------
[Imaginary part of h(z)]
Computed value: 0.0
Expected value: 0.0
check passed
[h(z) vs. |w0|^2 + 6|w1|^2]
Computed value: 1.1195074675943746
Expected value: 1.1195074675943748
check passed
[h(z) vs. 5|z0|^2 + 2|z1|^2 - 4 Im(conj(z0) z1)]
Computed value: 1.1195074675943746
Expected value: 1.1195074675943748
check passed
[Energy of a unit state lies in [1, 6]]
Computed value: True
Expected value: True
check passed
[Imaginary part of h(z)]
Computed value: -8.881784197001252e-16
Expected value: 0.0
check passed
[h(z) vs. |w0|^2 + 6|w1|^2]
Computed value: 11.890370344271124
Expected value: 11.890370344271124
check passed
[h(z) vs. 5|z0|^2 + 2|z1|^2 - 4 Im(conj(z0) z1)]
Computed value: 11.890370344271124
Expected value: 11.890370344271123
check passed
[Energy of a unit state lies in [1, 6]]
Computed value: True
Expected value: True
check passed
[Imaginary part of h(z)]
Computed value: 0.0
Expected value: 0.0
check passed
[h(z) vs. |w0|^2 + 6|w1|^2]
Computed value: 6.673983950416356
Expected value: 6.673983950416355
check passed
[h(z) vs. 5|z0|^2 + 2|z1|^2 - 4 Im(conj(z0) z1)]
Computed value: 6.673983950416356
Expected value: 6.673983950416355
check passed
[Energy of a unit state lies in [1, 6]]
Computed value: True
Expected value: True
check passed
The Jacobi Method¶
The Jacobi method is one of the oldest numerical algorithms for finding the eigenvalues of a real symmetric matrix. It was proposed by Carl Gustav Jacob Jacobi in 1846, more than a century before the modern QR algorithm.
Its central idea comes from a simple observation: the eigenvalues of a diagonal matrix can be read off at a glance—they are the entries on the diagonal. So if a matrix can be “reduced” step by step to diagonal form by a sequence of similarity transformations that leave the eigenvalues unchanged, the problem is solved.
Recall the spectral theorem of Chapter 9: a real symmetric matrix is always orthogonally diagonalizable, that is, there exists an orthogonal matrix such that
The Jacobi method approximates this constructively—at each step it chooses a plane rotation (a Givens rotation) that eliminates one off-diagonal entry exactly, while guaranteeing that the overall “off-diagonal energy”
decreases strictly monotonically. Although after one entry is eliminated, other entries in the same row and column may be disturbed and no longer be zero, we will see in the exercise below that each rotation reduces by exactly ; this “conservation of energy” viewpoint is the key to analyzing the convergence of the algorithm.
✓ All exercises completed.