Why I Wrote This Book 1 Discrete random variables 1 1.1 Motivating example 1 1.2 Bernoulli random variable 2 1.3 General discrete random variable 4 1.4 Mean and variance 6 1.4.1 Mechanical interpretation of the mean 7 1.4.
2 Variance 12 1.5 R basics 15 1.5.1 Scripts/functions 16 1.5.2 Text editing in R 17 1.5.3 Saving your R code 18 1.
5.4 for loop 18 1.5.5 Vectorized computations 19 1.5.6 Graphics 23 1.5.7 Coding and help in R 25 1.
6 Binomial distribution 26 1.7 Poisson distribution 32 1.8 Random number generation using sample 38 1.8.1 Generation of a discrete random variable 38 1.8.2 Random Sudoku 39 2 Continuous random variables 43 2.1 Distribution and density functions 43 2.
1.1 Cumulative distribution function 43 2.1.2 Empirical cdf 45 2.1.3 Density function 46 2.2 Mean, variance, and other moments 48 2.2.
1 Quantiles, quartiles, and the median 54 2.2.2 The tight confidence range 55 2.3 Uniform distribution 59 2.4 Exponential distribution 63 2.4.1 Laplace or double-exponential distribution 67 2.4.
2 R functions 67 2.5 Moment generating function 69 2.5.1 Fourier transform and characteristic function 72 2.6 Gamma distribution 75 2.6.1 Relationship to Poisson distribution 77 2.6.
2 Computing the gamma distribution in R 79 2.6.3 The tight confidence range 79 2.7 Normal distribution 82 2.8 Chebyshev''s inequality 91 2.9 The law of large numbers 93 2.9.1 Four types of stochastic convergence 94 2.
9.2 Integral approximation using simulations 99 2.10 The central limit theorem 104 2.10.1 Why the normal distribution is the most natural symmetric distribution 112 2.10.2 CLT on the relative scale 113 2.11 Lognormal distribution 116 2.
11.1 Computation of the tight confidence range 118 2.12 Transformations and the delta method 120 2.12.1 The delta method 124 2.13 Random number generation 126 2.13.1 Cauchy distribution 130 2.
14 Beta distribution 132 2.15 Entropy 134 2.16 Benford''s law: the distribution of the first digit 138 2.16.1 Distributions that almost obey Benford''s law 142 2.17 The Pearson family of distributions 145 2.18 Major univariate continuous distributions 147 3 Multivariate random variables 149 3.1 Joint cdf and density 149 3.
1.1 Expectation 154 3.1.2 Bivariate discrete distribution 154 3.2 Independence 156 3.2.1 Convolution 159 3.3 Conditional density 168 3.
3.1 Conditional mean and variance 171 3.3.2 Mixture distribution and Bayesian statistics 179 3.3.3 Random sum 182 3.3.4 Cancer tumors grow exponentially 184 3.
4 Correlation and linear regression 189 3.5 Bivariate normal distribution 198 3.5.1 Regression as conditional mean 206 3.5.2 Variance decomposition and coefficient of determination 208 3.5.3 Generation of dependent normal observations 209 3.
5.4 Copula 214 3.6 Joint density upon transformation 218 3.7 Geometric probability 223 3.7.1 Meeting problem 224 3.7.2 Random objects on the square 225 3.
8 Optimal portfolio allocation 230 3.8.1 Stocks do not correlate 231 3.8.2 Correlated stocks 232 3.8.3 Markowitz bullet 233 3.8.
4 Probability bullet 234 3.9 Distribution of order statistics 236 3.10 Multidimensional random vectors 239 3.10.1 Multivariate conditional distribution 245 3.10.2 Multivariate MGF 247 3.10.
3 Multivariate delta method 248 3.10.4 Multinomial distribution 251 4 Four important distributions in statistics 255 4.1 Multivariate normal distribution 255 4.1.1 Generation of multivariate normal variables 259 4.1.2 Conditional distribution 261 4.
1.3 Multivariate CLT 268 4.2 Chi-square distribution 270 4.2.1 Noncentral chi-square distribution 276 4.2.2 Expectations and variances of quadratic forms 277 4.2.
3 Kronecker product and covariance matrix 277 4.3 t -distribution 280 4.3.1 Noncentral t -distribution 284 4.4 F -distribution 286 5 Preliminary data analysis and visualization 291 5.1 Comparison of random variables using the cdf 291 5.1.1 ROC curve 294 5.
1.2 Survival probability 305 5.2 Histogram 312 5.3 Q-Q plot 315 5.3.1 The q-q confidence bands 319 5.4 Box plot 324 5.5 Kernel density estimation 325 5.
5.1 Density movie 331 5.5.2 3D scatterplots 333 5.6 Bivariate normal kernel density 335 5.6.1 Bivariate kernel smoother for images 339 5.6.
2 Smoothed scatterplot 341 5.6.3 Spatial statistics for disease mapping 342 6 Parameter estimation 347 6.1 Statistics as inverse probability 349 6.2 Method of moments 350 6.2.1 Generalized method of moments 353 6.3 Method of quantiles 357 6.
4 Statistical properties of an estimator 358 6.4.1 Unbiasedness 359 6.4.2 Mean Square Error 365 6.4.3 Multidimensional MSE 371 6.4.
4 Consistency of estimators 373 6.5 Linear estimation 378 6.5.1 Estimation of the mean using linear estimator 379 6.5.2 Vector representation 383 6.6 Estimation of variance and correlation coefficient 385 6.6.
1 Quadratic estimation of the variance 386 6.6.2 Estimation of the covariance and correlation coefficient 389 6.7 Least squares for simple linear regression 398 6.7.1 Gauss--Markov theorem 402 6.7.2 Statistical properties of the OLS estimator under the normal assumption 404 6.
7.3 The lm function and prediction by linear regression 406 6.7.4 Misinterpretation of the coefficient of determination 410 6.8 Sufficient statistics and the exponential family of distributions 415 6.8.1 Uniformly minimum-variance unbiased estimator 419 6.8.
2 Exponential family of distributions 422 6.9 Fisher information and the Cramér--Rao bound 433 6.9.1 One parameter 434 6.9.2 Multiple parameters 440 6.10 Maximum likelihood 453 6.10.
1 Basic definitions and examples 453 6.10.2 Circular statistics and the von Mises distribution 471 6.10.3 Maximum likelihood, sufficient statistics and the exponential family 475 6.10.4 Asymptotic properties of ML 477 6.10.
5 When maximum likelihood breaks down 485 6.10.6 Algorithms for log-likelihood function maximization 498 6.11 Estimating equations and the M-estimator 510 6.11.1 Robust statistics 516 7 Hypothesis testing and confidence intervals 523 7.1 Fundamentals of statistical testing 523 7.1.
1 The p -value and its interpretation 525 7.1.2 Ad hoc statistical testing 528 7.2 Simple hypothesis 531 7.3 The power function of the Z -test 536 7.3.1 Type II error and the power function 536 7.3.
2 Optimal significance level and the ROC curve 542 7.3.3 One-sided hypothesis 545 7.4 The t -test for the means 549 7.4.1 One-sample t -test 549 7.4.2 Two-sample t -test 552 7.
4.3 One-sided t -test 557 7.4.4 Paired versus unpaired t -test 558 7.4.5 Parametric versus nonparametric tests 560 7.5 Variance test 562 7.5.
1 Two-sided variance test 562 7.5.2 One-sided variance test 565 7.6 Inverse-cdf test 566 7.6.1 General formulation 567 7.6.2 The F -test for variances 569 7.
6.3 Binomial proportion 573 7.6.4 Poisson rate 577 7.7 Testing for correlation coefficient 580 7.8 Confidence interval 583 7.8.1 Unbiased CI and its connection to hypothesis testing 588 7.
8.2 Inverse cdf CI 589 7.8.3 CI for the normal variance and SD 591 7.8.4 CI for other major statistical parameters 592 7.8.5 Confidence region 594 7.
9 Three asymptotic tests and confidence intervals 597 7.9.1 Pearson chi-square test 605 7.9.2 Handwritten digit recognition 608 7.10 Limitations of classical hypothesis testing and the d -value 612 7.10.1 What the p -value means? 613 7.
10.2 Why α = 0.05? 614 7.10.3 The null hypothesis is always rejected with a large enough sample size 616 7.10.4 Parameter-based inference 618 7.10.
5 The d -value for individual inference 619 8 Linear model and its extensions 627 8.1 Basic definitions and linear least squares 627 8.1.1 Linear model with the intercept term 632 8.1.2 The vector-space geometry of least squares 633 8.1.3 Coefficient of determination 636 8.
2 The Gauss--Markov theorem 639 8.2.1 Estimation of regression variance 641 8.3 Properties of OLS estimators under the normal assumption 643 8.3.1 The sensitivity of statistical inference to violation of the normal assumption 646 8.4 Statistical inference with linear models 650 8.4.
1 Confidence interval and region 650 8.4.2 Linear hypothesis testing and the F -test 653 8.4.3 Prediction by linear regression and simultaneous confidence band 661 8.4.4 Testing the null hypothesis and the coefficient of determination 664 8.4.
5 Is X fixed or random? 665 8.5 The one-sided p - and d -value for regression coefficients 671 8.5.1 The one-sided p -value for interpretation on the population level 672 8.5.2 The d -value for interpretation on the indi.