• Distribution and Statistic
• Chebyshev’s Theorem
3
Outline
• Probability Concepts
– Basic Concepts – Conditioning Concepts
• Law of Total Expectation/Variance, Bayes’ Theorem
Outline
• Probability Concepts
– Basic Concepts – Conditioning Concepts
• Law of Total Expectation/Variance, Bayes’ Theorem
• Limit Theorems
– Modes of Convergence – Law of Large Numbers (LLN) – Central Limit Theorem (CLT)
probability of the event – the probability of A, P(A), regardless of whether B has occurred.
• Joint probability 联合概率: the probability of two
events in conjunction – P(A∩B), P(AB), P(A,B) – If there are two possible outcomes for r.v. X with events B and Bc, then P(A)=P(A∩B)+ P(A∩Bc) Law of Total Probability
16
Conditioning Concepts
• Conditional expectation 条件期望
– the expected value of a r.v. with respect to a conditional probability distribution p ( xy ) E[Y | X x] y pY | X ( y | X x) y p X ( x) y y
– P(A) = prior probability 先验概率
• the probability of A given only initial information
– P(A|B) = posterior probability 后验概率
• the updated conditional probability of A, given initial information and the outcome of B
– the occurrence of A/B makes it neither more nor less probable that B/A occurs,
• P(A ∩ B) = P(A) * P(B) • P(A U B) = P(A) + P(B) – P(A ∩B) – Independent events A = heads on one toss of fair coin B = heads on second toss of same coin – Dependent events A = rain forecasted on the news B = take umbrella to work
• Random Experiment 随机试验
– procedure whose outcome cannot be predicted in advance
• e.g., toss a coin twice
• Sample Space (S or
) 样本空间
– the set of all possible outcomes
• Intersection (and) 交集 & &&
– e.g., A=heads on first, B=heads on second – A ∩ B = {H,H}
• Complement 补集 !
– sets of all outcomes not in A – e.g., A={T,T}, Ac={H,H},{H,T},{T,H}
p
y
Y|X
( y | x) 1
f
y
Y|X
( y | x) 1
15
Conditioning Concepts
• Bayes’ theorem 贝叶斯定理
– relates conditional and marginal probabilities of events A and B, where B has a non-zero probability:
13
Conditioning Concepts
• Conditional probability
– the probability of A, given the occurrence of B
• P(A|B) = P(A∩B)/P(B) → P(A∩B) = P(A|B)P(B) Rule of multiplication 乘法原则 • Independence P(A|B)=? Disjoint events P(A|B)=?
Random Variable
Discrete Random Variable Continuous Random Variable
11
Conditioning Concepts
• Conditioning Concepts
– Conditional Probability
• Law of Total Probability • Bayes’ Theorem
9
Basic Concepts
• If A and B are disjoint,
– they have no element in common. – and if A occurs, then B cannot occur.
• P(A ∩B) =0 • P(A U B) = P(A) + P(B) A Black Cards B Red Cards
– i.e., a function from events to probability levels
6
Basic Concepts
• Union (or) 并集 | ||
– e.g., A=heads on first, B=heads on second – A U B= {H,T},{H,H},{T,H}
E[XY] = E[X]E[Y|X] = E[Y]E[X|Y]
17
Conditioning Concepts
• Law of Total/Iterated Expectation (LIE) 全/重期望律
E[X]=E[E[X|Y]]
• Applications of LIE – Other versions: E[X|I1]=E[E[X|I2]|I1] where the value of I1 is determined by that of I2.
f XY ( x, y ) E[Y | X x] y fY | X ( y | x)dy y dy f X ( x) y y
– Conditional expectation & regression 回归:
regression line: y = E[Y|X]
– Independence & Disjoint: E[Y|X=x] = ? E[XY] = ?
• Probability 概率论
(mathematical foundations of statistics)
Inductive 归纳 (particular → general)
Given the information in the box, what is in your hand?
– Assume known population and parameters, and compute the probability of drawing a particular sample. Deductive 演绎 (general → particular) 2
• conditional probability density function (pdf) of Y given X
f XY ( x, y ) fY | X ( y | x ) f X ( x)
14
Conditioning Concepts
• Comments:
– Conditional distribution can be viewed as a probability distribution defined on a reduced sample space.
A card cannot be Black and Red at the same time.
10
Basic Concepts
• Random Variable (X) 随机变量 r.v.
– represents a possible numerical value from a random event.
– Conditional Expectation
• Law of Total Expectation
– Conditional Variance
• Law of Total Variance
12
Conditioning Concepts
• Marginal probability 边际概率: the unconditional