Linear Algebra and Matrix Analysis for Statistics.pdf
This beautifully written text is unlike any other in statistical science. It starts at the level of a first undergraduate course in linear algebra, and takes the student all the way up to the graduate level, including Hilbert spaces. It is extremely well crafted and proceeds up through that theory at a very good pace. The statistics chapters are added at just the right places to motivate the reader and illustrate the theory. The book is compactly written and mathematically rigorous, yet the style is lively as well as engaging. This elegant, sophisticated work will serve upper level and graduate statistics education well. All and all a book I wish I could have written. -Jim Zidek, University of British Columbia, Vancouver, Canada
University of Minnesota, Minneapolis, USA Department of Math and Statistics, University of Maryland Baltimore County, USA
Basic Operations Basic definitions and notations Matrix addition and scalar-matrix multiplication Matrix multiplication Partitioned matrices The "trace" of a square matrix Some special matrices Systems of Linear Equations Introduction Gaussian elimination Gauss-Jordan elimination Elementary matrices Homogeneous linear systems The inverse of a matrix More on Linear Equations The LU decomposition Crout's Algorithm LU decomposition with row interchanges The LDU and Cholesky factorizations Inverse of partitioned matrices The LDU decomposition for partitioned matrices The Sherman-Woodbury-Morrison formula Euclidean Spaces Introduction Vector addition and scalar multiplication Linear spaces and subspaces Intersection and sum of subspaces Linear combinations and spans Four fundamental subspaces Linear independence Basis and dimension The Rank of a Matrix Rank and nullity of a matrix Bases for the four fundamental subspaces Rank and inverse Rank factorization The rank-normal form Rank of a partitioned matrix Bases for the fundamental subspaces using the rank normal form Complementary Subspaces Sum of subspaces The dimension of the sum of subspaces Direct sums and complements Projectors Orthogonality, Orthogonal Subspaces, and Projections Inner product, norms, and orthogonality Row rank = column rank: A proof using orthogonality Orthogonal projections Gram-Schmidt orthogonalization Orthocomplementary subspaces The fundamental theorem of linear algebra More on Orthogonality Orthogonal matrices The QR decomposition Orthogonal projection and projector Orthogonal projector: Alternative derivations Sum of orthogonal projectors Orthogonal triangularization Revisiting Linear Equations Introduction Null spaces and the general solution of linear systems Rank and linear systems Generalized inverse of a matrix Generalized inverses and linear systems The Moore-Penrose inverse Determinants Definitions Some basic properties of determinants Determinant of products Computing determinants The determinant of the transpose of a matrix - revisited Determinants of partitioned matrices Cofactors and expansion theorems The minor and the rank of a matrix The Cauchy-Binet formula The Laplace expansion Eigenvalues and Eigenvectors Characteristic polynomial and its roots Spectral decomposition of real symmetric matrices Spectral decomposition of Hermitian and normal matrices Further results on eigenvalues Singular value decomposition Quadratic Forms Introduction Quadratic forms Matrices in quadratic forms Positive and nonnegative definite matrices Congruence and Sylvester's law of inertia Nonnegative definite matrices and minors Extrema of quadratic forms Simultaneous diagonalization Matrix and Vector Norms Matrix norms Matrix approximation Principal component analysis Hilbert Spaces Orthogonal projection in Hilbert spaces Some common Hilbert spaces in statistics Sobolov spaces Reproducing kernel Hilbert space References Exercises appear at the end of each chapter.
Linear algebra and the study of matrix algorithms have become fundamental to the development of statistical models. Using a vector-space approach, this book provides an understanding of the major concepts that underlie linear algebra and matrix analysis. Each chapter introduces a key topic, such as infinite-dimensional spaces, and provides illustrative examples. The authors examine recent developments in diverse fields such as spatial statistics, machine learning, data mining, and social network analysis. Complete in its coverage and accessible to students without prior knowledge of linear algebra, the text also includes results that are useful for traditional statistical applications.