Probability Theory p. 1
Set Theory p. 1
Basics of Probability Theory p. 5
Axiomatic Foundations p. 5
The Calculus of Probabilities p. 9
Counting p. 13
Enumerating Outcomes p. 16
Conditional Probability and Independence p. 20
Random Variables p. 27
Distribution Functions p. 29
Density and Mass Functions p. 34
Exercises p. 37
Miscellanea p. 44
Transformations and Expectations p. 47
Distributions of Functions of a Random Variable p. 47
Expected Values p. 55
Moments and Moment Generating Functions p. 59
Differentiating Under an Integral Sign p. 68
Exercises p. 76
Miscellanea p. 82
Common Families of Distributions p. 85
Introduction p. 85
Discrete Distributions p. 85
Continuous Distributions p. 98
Exponential Families p. 111
Location and Scale Families p. 116
Inequalities and Identities p. 121
Probability Inequalities p. 122
Identities p. 123
Exercises p. 127
Miscellanea p. 135
Multiple Random Variables p. 139
Joint and Marginal Distributions p. 139
Conditional Distributions and Independence p. 147
Bivariate Transformations p. 156
Hierarchical Models and Mixture Distributions p. 162
Covariance and Correlation p. 169
Multivariate Distributions p. 177
Inequalities p. 186
Numerical Inequalities p. 186
Functional Inequalities p. 189
Exercises p. 192
Miscellanea p. 203
Properties of a Random Sample p. 207
Basic Concepts of Random Samples p. 207
Sums of Random Variables from a Random Sample p. 211
Sampling from the Normal Distribution p. 218
Properties of the Sample Mean and Variance p. 218
The Derived Distributions: Student's t and Snedecor's F p. 222
Order Statistics p. 226
Convergence Concepts p. 232
Convergence in Probability p. 232
Almost Sure Convergence p. 234
Convergence in Distribution p. 235
The Delta Method p. 240
Generating a Random Sample p. 245
Direct Methods p. 247
Indirect Methods p. 251
The Accept/Reject Algorithm p. 253
Exercises p. 255
Miscellanea p. 267
Principles of Data Reduction p. 271
Introduction p. 271
The Sufficiency Principle p. 272
Sufficient Statistics p. 272
Minimal Sufficient Statistics p. 279
Ancillary Statistics p. 282
Sufficient, Ancillary, and Complete Statistics p. 284
The Likelihood Principle p. 290
The Likelihood Function p. 290
The Formal Likelihood Principle p. 292
The Equivariance Principle p. 296
Exercises p. 300
Miscellanea p. 307
Point Estimation p. 311
Introduction p. 311
Methods of Finding Estimators p. 312
Method of Moments p. 312
Maximum Likelihood Estimators p. 315
Bayes Estimators p. 324
The EM Algorithm p. 326
Methods of Evaluating Estimators p. 330
Mean Squared Error p. 330
Best Unbiased Estimators p. 334
Sufficiency and Unbiasedness p. 342
Loss Function Optimality p. 348
Exercises p. 355
Miscellanea p. 367
Hypothesis Testing p. 373
Introduction p. 373
Methods of Finding Tests p. 374
Likelihood Ratio Tests p. 374
Bayesian Tests p. 379
Union-Intersection and Intersection-Union Tests p. 380
Methods of Evaluating Tests p. 382
Error Probabilities and the Power Function p. 382
Most Powerful Tests p. 387
Sizes of Union-Intersection and Intersection-Union Tests p. 394
p-Values p. 397
Loss Function Optimality p. 400
Exercises p. 402
Miscellanea p. 413
Interval Estimation p. 417
Introduction p. 417
Methods of Finding Interval Estimators p. 420
Inverting a Test Statistic p. 420
Pivotal Quantities p. 427
Pivoting the CDF p. 430
Bayesian Intervals p. 435
Methods of Evaluating Interval Estimators p. 440
Size and Coverage Probability p. 440
Test-Related Optimality p. 444
Bayesian Optimality p. 447
Loss Function Optimality p. 449
Exercises p. 451
Miscellanea p. 463
Asymptotic Evaluations p. 467
Point Estimation p. 467
Consistency p. 467
Efficiency p. 470
Calculations and Comparisons p. 473
Bootstrap Standard Errors p. 478
Robustness p. 481
The Mean and the Median p. 482
M-Estimators p. 484
Hypothesis Testing p. 488
Asymptotic Distribution of LRTs p. 488
Other Large-Sample Tests p. 492
Interval Estimation p. 496
Approximate Maximum Likelihood Intervals p. 496
Other Large-Sample Intervals p. 499
Exercises p. 504
Miscellanea p. 515
Analysis of Variance and Regression p. 521
Introduction p. 521
Oneway Analysis of Variance p. 522
Model and Distribution Assumptions p. 524
The Classic ANOVA Hypothesis p. 525
Inferences Regarding Linear Combinations of Means p. 527
The ANOVA F Test p. 530
Simultaneous Estimation of Contrasts p. 534
Partitioning Sums of Squares p. 536
Simple Linear Regression p. 539
Least Squares: A Mathematical Solution p. 542
Best Linear Unbiased Estimators: A Statistical Solution p. 544
Models and Distribution Assumptions p. 548
Estimation and Testing with Normal Errors p. 550
Estimation and Prediction at a Specified x = x[subscript 0] p. 557
Simultaneous Estimation and Confidence Bands p. 559
Exercises p. 563
Miscellanea p. 572
Regression Models p. 577
Introduction p. 577
Regression with Errors in Variables p. 577
Functional and Structural Relationships p. 579
A Least Squares Solution p. 581
Maximum Likelihood Estimation p. 583
Confidence Sets p. 588
Logistic Regression p. 591
The Model p. 591
Estimation p. 593
Robust Regression p. 597
Exercises p. 602
Miscellanea p. 608
Computer Algebra p. 613
Table of Common Distributions p. 621
References p. 629
Author Index p. 645
Subject Index p. 649