Contents
Preface
1 Matrices
1.1 Basic Terminology
1.2 Basic Operations
1.3 Some Basic Types of Matrices
Exercises
2 Submatrices and Partitioned Matrices
2.1 Some Terminology and Basic Results
2.2 Scalar Multiples, Transposes, Sums, and Products of Partitioned Matrices
2.3 Some Results on the Product of a Matrix and a Column Vector
2.4 Expansion of a Matrix in Terms of Its Rows, Columns, or Elements
Exercises
3 Linear Dependence and Independence
3.1 Definitions
3.2 Some Basic Results
Exercises
4 Linear Spaces: Row and Column Spaces
4.1 Some Definitions, Notation, and Basic Relationships and Properties
4.2 Subspaces
4.3 Bases
4.4 Rank of a Matrix
4.5 Some Basic Results on Partitioned Matrices and on Sums of Matrices
Exercises
5 Trace of a (Square) Matrix
5.1 Definition and Basic Properties
5.2 Trace of a Product
5.3 Some Equivalent Conditions
Exercises
6 Geometrical Considerations
6.1 Definitions: Norm, Distance, Angle, Inner Product, and Orthogonality
6.2 Orthogonal and Orthonormal Sets
6.3 Schwarz Inequality
6.4 Orthonormal Bases
Exercises
7 Linear Systems: Consistency and Compatibility
7.1 Some Basic Terminology
7.2 Consistency
7.3 Compatibility
7.4 Linear Systems of the Form A\\u0027 AX = A\\u0027B
Exercise
8 Inverse Matrices
8.1 Some Definitions and Basic Results
8.2 Properties of Inverse Matrices
8.3 Premultiplication or Postmultiplication by a Matrix of Full Column or Row Rank
8.4 Orthogonal Matrices
8.5 Some Basic Results on the Ranks and Inverses of Partitioned Matrices
Exercises
9 Generalized Inverses
9.1 Definition, Existence, and a Connection to the Solution of Linear Systems
9.2 Some Alternative Characterizations
9.3 Some Elementary Properties
9.4 Invariance to the Choice of a Generalized Inverse
9.5 A Necessary and Sufficient Condition for the Consistency of a Linear System
9.6 Some Results on the Ranks and Generalized Inverses of Partitioned Matrices
9.7 Extension of Some Results on Systems of the Form AX = B to Systems of the Form AXC = B
Exercises
10 Idempotent Matrices
10.1 Definition and Some Basic Properties
10.2 Some Basic Results
Exercises
11 Linear Systems: Solutions
11.1 Some Terminology, Notation, and Basic Results
11.2 General Form of a Solution
11.3 Number of Solutions
11.4 A Basic Result on Null Spaces
11.5 An Alternative Expression for the General Form of a Solution
11.6 Equivalent Linear Systems
11.7 Null and Column Spaces of Idempotent Matrices
11.8 Linear Systems With Nonsingular Triangular or Block-Triangular Coefficient Matrices
11.9 A Computational Approach
11.10 Linear Combinations of the Unknowns
11.11 Absorption
11.12 Extensions to Systems of the Form AXC = B
Exercises
12 Projections and Projection Matrices
12.1 Some General Results, Terminology, and Notation
12.2 Projection of a Column Vector
12.3 Projection Matrices
12.4 Least Squares Problem
12.5 Orthogonal Complements
Exercises
13 Determinants
13.1 Some Definitions, Notation, and Special Cases
13.2 Some Basic Properties of Determinants
13.3 Partitioned Matrices, Products of Matrices, and Inverse Matrices
13.4 A Computational Approach
13.5 Cofactors
13.6 Vandermonde Matrices
13.7 Some Results on the Determinant of the Sum of Two Matrices
13.8 Laplace\\u0027s Theorem and the Binet-Cauchy Formula
Exercises
14 Linear, Bilinear, and Quadratic Forms
14.1 Some Terminology and Basic Results
14.2 Nonnegative Definite Quadratic Forms and Matrices
14.3 Decomposition of Symmetric and Symmetric Nonnegative Definite Matrices
14.4 Generalized Inverses of Symmetric Nonnegative Definite Matrices
14.5 LDU, U\\u0027DU, and Cholesky Decompositions
14.6 Skew-Symmetric Matrices
14.7 Trace of a Nonnegative Definite Matrix
14.8 Partitioned Nonnegative Definite Matrices
14.9 Some Results on Determinants
14.10 Geometrical Considerations
14.11 Some Results on Ranks and Row and Column Spaces and on Linear Systems
14.12 Projections, Projection Matrices, and Orthogonal Complements
Exercises
15 Matrix Differentiation
15.1 Definitions, Notation, and Other Preliminaries
15.2 Differentiation of (Scalar-Valued) Functions: Some Elementary Results
15.3 Differentiation of Linear and Quadratic Forms
15.4 Differentiation of Matrix Sums, Products, and Transposes (and of Matrices of Constants)
15.5 Differentiation of a Vector or an (Unrestricted or Symmetric) Matrix With Respect to Its Elements
15.6 Differentiation of a Trace of a Matrix
15.7 The Chain Rule
15.8 First-Order Partial Derivatives of Determinants and Inverse and Adjoint Matrices
15.9 Second-Order Partial Derivatives of Determinants and Inverse Matrices
15.10 Differentiation of Generalized Inverses
15.11 Differentiation of Projection Matrices
15.12 Evaluation of Some Multiple Integrals
Exercises
Bibliographic and Supplementary Notes
16 Kronecker Products and the Vec and Vech Operators
16.1 The Kronecker Product of Two or More Matrices: Definition and Some Basic Properties
16.2 The Vec Operator: Definition and Some Basic Properties
16.3 Vec-Permutation Matrix
16.4 The Vech Operator
16.5 Reformulation of a Linear System
16.6 Some Results on Jacobian Matrices
Exercises
Bibliographic and Supplementary Notes
17 Intersections and Sums of Subspaces
17.1 Definitions and Some Basic Properties
17.2 Some Results on Row and Column Spaces and on the Ranks of Partitioned Matrices
17.3 Some Results on Linear Systems and on Generalized Inverses of Partitioned Matrices
17.4 Subspaces: Sum of Their Dimensions Versus Dimension of Their Sum
17.5 Some Results on the Rank of a Product of Matrices
17.6 Projections Along a Subspace
17.7 Some Further Results on the Essential Disjointness and Orthogonality of Subspaces and on Projections and Projection Matrices
Exercises
Bibliographic and Supplementary Notes
18 Sums (and Differences) of Matrices
18.1 Some Results on Determinants
18.2 Some Results on Inverses and Generalized Inverses and on Linear Systems
18.3 Some Results on Positive (and Nonnegative) Definiteness
18.4 Some Results on Idempotency
18.5 Some Results on Ranks
Exercises
Bibliographic and Supplementary Notes
19 Minimization of a Second-Degree Polynomial (in n Variables) Subject to Linear Constraints
19.1 Unconstrained Minimization
19.2 Constrained Minimization
19.3 Explicit Expressions for Solutions to the Constrained Minimization Problem
19.4 Some Results on Generalized Inverses of Partitioned Matrices
19.5 Some Additional Results on the Form of Solutions to the Constrained Minimization Problem
19.6 Transformation of the Constrained Minimization Problem to an Unconstrained Minimization Problem
19.7 The Effect of Constraints on the Generalized Least Squares Problem
Exercises
Bibliographic and Supplementary Notes
20 The Moore-Penrose Inverse
20.1 Definition, Existence, and Uniqueness (of the Moore-Penrose Inverse)
20.2 Some Special Cases
20.3 Special Types of Generalized Inverses
20.4 Some Alternative Representations and Characterizations
20.5 Some Basic Properties and Relationships
20.6 Minimum Norm Solution to the Least Squares Problem
20.7 Expression of the Moore-Penrose Inverse as a Limit
20.8 Differentiation of the Moore-Penrose Inverse
Exercises
Bibliographic and Supplementary Notes
21 Eigenvalues and Eigenvectors
21.1 Definitions, Terminology, and Some Basic Results
21.2 Eigenvalues of Triangular or Block-Triangular Matrices and of Diagonal or Block-Diagonal Matrices
21.3 Similar Matrices
21.4 Linear Independence of Eigenvectors
21.5 Diagonalization
21.6 Expressions for the Trace and Determinant of a Matrix
21.7 Some Results on the Moore-Penrose Inverse of a Symmetric Matrix
21.8 Eigenvalues of Orthogonal, Idempotent, and Nonnegative Definite Matrices
21.9 Square Root of a Symmetric Nonnegative Definite Matrix
21.10 Some Relationships
21.11 Eigenvalues and Eigenvectors of Kronecker Products of (Square) Matrices
21.12 Singular Value Decomposition
21.13 Simultaneous Diagonalization
21.14 Generalized Eigenvalue Problem
21.15 Differentiation of Eigenvalues and Eigenvectors
21.16 An Equivalence (Involving Determinants and Polynomials)
Appendix: Some Properties of Polynomials (in a Single Variable)
Exercises
Bibliographic and Supplementary Notes
22 Linear Transformations
22.1 Some Definitions, Terminology, and Basic Results
22.2 Scalar Multiples, Sums, and Products of Linear Transformations
22.3 Inverse Transformations and Isomorphic Linear Spaces
22.4 Matrix Representation of a Linear Transformation
22.5 Terminology and Properties Shared by a Linear Transformation and Its Matrix Representation
22.6 Linear Functionals and Dual Transformations
Exercises
References
Index
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W