Preface xiii Part One -- Matrices 1 Basic properties of vectors and matrices 3 1 Introduction 3 2 Sets 3 3 Matrices: addition and multiplication 4 4 The transpose of a matrix 6 5 Square matrices 6 6 Linear forms and quadratic forms 7 7 The rank of a matrix 9 8 The inverse 10 9 The determinant 10 10 The trace 11 11 Partitioned matrices 12 12 Complex matrices 14 13 Eigenvalues and eigenvectors 14 14 Schur''s decomposition theorem 17 15 The Jordan decomposition 18 16 The singular-value decomposition 20 17 Further results concerning eigenvalues 20 18 Positive (semi)definite matrices 23 19 Three further results for positive definite matrices 25 20 A useful result 26 21 Symmetric matrix functions 27 Miscellaneous exercises 28 Bibliographical notes 30 2 Kronecker products, vec operator, and Moore-Penrose inverse 31 1 Introduction 31 2 The Kronecker product 31 3 Eigenvalues of a Kronecker product 33 4 The vec operator 34 5 The Moore-Penrose (MP) inverse 36 6 Existence and uniqueness of the MP inverse 37 7 Some properties of the MP inverse 38 8 Further properties 39 9 The solution of linear equation systems 41 Miscellaneous exercises 43 Bibliographical notes 45 3 Miscellaneous matrix results 47 1 Introduction 47 2 The adjoint matrix 47 3 Proof of Theorem 3.1 49 4 Bordered determinants 51 5 The matrix equation AX = 0 51 6 The Hadamard product 52 7 The commutation matrix Kmn 54 8 The duplication matrix Dn 56 9 Relationship between Dn+1 and Dn , I 58 10 Relationship between Dn+1 and Dn , II 59 11 Conditions for a quadratic form to be positive (negative) subject to linear constraints 60 12 Necessary and sufficient conditions for r(A : B) = r(A) + r(B) 63 13 The bordered Gramian matrix 65 14 The equations X1A + X2B' = G1,X1B = G2 67 Miscellaneous exercises 69 Bibliographical notes 70 Part Two -- Differentials: the theory 4 Mathematical preliminaries 73 1 Introduction 73 2 Interior points and accumulation points 73 3 Open and closed sets 75 4 The Bolzano-Weierstrass theorem 77 5 Functions 78 6 The limit of a function 79 7 Continuous functions and compactness 80 8 Convex sets 81 9 Convex and concave functions 83 Bibliographical notes 86 5 Differentials and differentiability 87 1 Introduction 87 2 Continuity 88 3 Differentiability and linear approximation 90 4 The differential of a vector function 91 5 Uniqueness of the differential 93 6 Continuity of differentiable functions 94 7 Partial derivatives 95 8 The first identification theorem 96 9 Existence of the differential, I 97 10 Existence of the differential, II 99 11 Continuous differentiability 100 12 The chain rule 100 13 Cauchy invariance 102 14 The mean-value theorem for real-valued functions 103 15 Differentiable matrix functions 104 16 Some remarks on notation 106 17 Complex differentiation 108 Miscellaneous exercises 110 Bibliographical notes 110 6 The second differential 111 1 Introduction 111 2 Second-order partial derivatives 111 3 The Hessian matrix 112 4 Twice differentiability and second-order approximation, I 113 5 Definition of twice differentiability 114 6 The second differential 115 7 Symmetry of the Hessian matrix 117 8 The second identification theorem 119 9 Twice differentiability and second-order approximation, II 119 10 Chain rule for Hessian matrices 121 11 The analog for second differentials 123 12 Taylor''s theorem for real-valued functions 124 13 Higher-order differentials 125 14 Real analytic functions 125 15 Twice differentiable matrix functions 126 Bibliographical notes 127 7 Static optimization 129 1 Introduction 129 2 Unconstrained optimization 130 3 The existence of absolute extrema 131 4 Necessary conditions for a local minimum 132 5 Sufficient conditions for a local minimum: first-derivative test 134 6 Sufficient conditions for a local minimum: second-derivative test 136 7 Characterization of differentiable convex functions 138 8 Characterization of twice differentiable convex functions 141 9 Sufficient conditions for an absolute minimum 142 10 Monotonic transformations 143 11 Optimization subject to constraints 144 12 Necessary conditions for a local minimum under constraints 145 13 Sufficient conditions for a local minimum under constraints 149 14 Sufficient conditions for an absolute minimum under constraints 154 15 A note on constraints in matrix form 155 16 Economic interpretation of Lagrange multipliers 155 Appendix: the implicit function theorem 157 Bibliographical notes 159 Part Three -- Differentials: the practice 8 Some important differentials 163 1 Introduction 163 2 Fundamental rules of differential calculus 163 3 The differential of a determinant 165 4 The differential of an inverse 168 5 Differential of the Moore-Penrose inverse 169 6 The differential of the adjoint matrix 172 7 On differentiating eigenvalues and eigenvectors 174 8 The continuity of eigenprojections 176 9 The differential of eigenvalues and eigenvectors: symmetric case 180 10 Two alternative expressions for dλ 183 11 Second differential of the eigenvalue function 185 Miscellaneous exercises 186 Bibliographical notes 189 9 First-order differentials and Jacobian matrices 191 1 Introduction 191 2 Classification 192 3 Derisatives 192 4 Derivatives 194 5 Identification of Jacobian matrices 196 6 The first identification table 197 7 Partitioning of the derivative 197 8 Scalar functions of a scalar 198 9 Scalar functions of a vector 198 10 Scalar functions of a matrix, I: trace 199 11 Scalar functions of a matrix, II: determinant 201 12 Scalar functions of a matrix, III: eigenvalue 202 13 Two examples of vector functions 203 14 Matrix functions 204 15 Kronecker products 206 16 Some other problems 208 17 Jacobians of transformations 209 Bibliographical notes 210 10 Second-order differentials and Hessian matrices 211 1 Introduction 211 2 The second identification table 211 3 Linear and quadratic forms 212 4 A useful theorem 213 5 The determinant function 214 6 The eigenvalue function 215 7 Other examples 215 8 Composite functions 217 9 The eigenvector function 218 10 Hessian of matrix functions, I 219 11 Hessian of matrix functions, II 219 Miscellaneous exercises 220 Part Four -- Inequalities 11 Inequalities 225 1 Introduction 225 2 The Cauchy-Schwarz inequality 226 3 Matrix analogs of the Cauchy-Schwarz inequality 227 4 The theorem of the arithmetic and geometric means 228 5 The Rayleigh quotient 230 6 Concavity of λ 1 and convexity of λ n 232 7 Variational description of eigenvalues 232 8 Fischer''s min-max theorem 234 9 Monotonicity of the eigenvalues 236 10 The Poincar´e separation theorem 236 11 Two corollaries of Poincar´e''s theorem 237 12 Further consequences of the Poincar´e theorem 238 13 Multiplicative version 239 14 The maximum of a bilinear form 241 15 Hadamard''s inequality 242 16 An interlude: Karamata''s inequality 242 17 Karamata''s inequality and eigenvalues 244 18 An inequality concerning positive semidefinite matrices 245 19 A representation theorem for ( ∑api )1/p 246 20 A representation theorem for (tr A p)1/p 247 21 Hölder''s inequality 248 22 Concavity of log A 250 23 Minkowski''s inequality 251 24 Quasilinear representation of A 1/n 253 25 Minkowski''s determinant theorem 255 26 Weighted means of order p 256 27 Schlömilch''s inequality 258 28 Curvature properties of Mp ( x , a ) 259 29 Least squares 260 30 Generalized least squares 261 31 Restricted least squares 262 32 Restricted least squares: matrix version 264 Miscellaneous exercises 265 Bibliographical notes 269 Part Five -- The linear model 12 Statistical preliminaries 273 1 Introduction 273 2 The cumulative distribution function 273 3 The joint density function 274 4 Expectations 274 5 Variance and covariance 275 6 Independence of two random variables 277 7 Independence of n random variables 279 8 Sampling 279 9 The one-dimensional normal distribution 279 10 The multivariate normal distribution 280 11 Estimation 282 Miscellaneous exercises 282 Bibliographical notes 283 13 The linear re.