The mean squared error (MSE):
2 ˆ ˆ ˆ MSE{f h (x)} = Var{fh (x)} + [Bias{fh (x)}] 4 1 h = K 2 [f (x)]2 [µ2 (K )]2 2 f (x) + nh 4 1 + o(h4 ) + o . nh as h → 0, nh → ∞.
Kernel density estimation
Chapter 4: Nonparametric density estimation Baisen Liu Contents Histogram Smoother univariate density estimation The choice of smoothing parameter The constructing the confidence intervals Univariate cumulative distribution function ess)ds and K
2 2
=
K 2 (s)ds.
The optimal local bandwidth hopt (x): ˆ hopt (x) = arg min MSE{f h (x)}.
The statistical properties
Chapter 4: Nonparametric density estimation Baisen Liu Contents Histogram Smoother univariate density estimation The choice of smoothing parameter The constructing the confidence intervals Univariate cumulative distribution function estimation
Chapter 4: Nonparametric density estimation
Baisen Liu School of Statistics, Dongbei University of Finance & Economics September 23, 2014
Contents
Chapter 4: Nonparametric density estimation Baisen Liu Contents Histogram Smoother univariate density estimation The choice of smoothing parameter The constructing the confidence intervals Univariate cumulative distribution function estimation
The mean integrated squared error (MISE): ˆ MISE(f h) = = ˆ MSE{f h (x)}dx h4 1 K 2 + [µ2 (K )]2 f (x) 2 nh 4 1 +o + o(h4 ), nh as h → 0, nh → ∞.
2 2
where f (x)
The definition of the density f (x):
f (x) ≡ d F (x + h) − F (x − h) F (x) ≡ limh→0 . dx 2h
The histogram estimate of f (x): ˆ(x) = #{xi ∈ (x − h, x + h]} . f 2nh The kernel density estimator of f (x): ˆ(x) = 1 f nh where K (u) =
i=1
1 K (·/h). h K (·) is called kernel function, and h is call bandwith. Kh (·) =
The construction of kernel density estimate
Chapter 4: Nonparametric density estimation Baisen Liu Contents Histogram Smoother univariate density estimation The choice of smoothing parameter The constructing the confidence intervals Univariate cumulative distribution function estimation
L: Kernel functions
Kernel Uniform Triangle Epanechnikov Quartic(Biweight) Triweight Gaussian Cosine
π 4
K (u)
1 2 I (|u|
≤ 1)
(1 − |u|)I (|u| ≤ 1)
3 4 (1 15 16 (1 35 32 (1
The statistical properties
Chapter 4: Nonparametric density estimation Baisen Liu Contents Histogram Smoother univariate density estimation The choice of smoothing parameter The constructing the confidence intervals Univariate cumulative distribution function estimation
Bias: ˆ Bias{f h (x)} = =
h→0
ˆ E{f h (x) − f (x)}
1 n n
E{Kh (x − Xi )} − f (x) f (x) s2 K (s)ds + o(h2 ).
=
i=1 h2
2
Variance: ˆ Var{f h (x)} = =
nh→∞
Var
=
1 n Var{Kh (x − Xi )} n2 i=1 1 1 f (x) K 2 (s)ds + o nh nh
Let X1 , ..., Xn be a random sample from a population X with density f (x). The kernel density estimator of f (x) is: 1 ˆ f h (x) = n where
n
Kh (x − Xi ),
2
3
4
5
6
Histogram
Chapter 4: Nonparametric density estimation Baisen Liu Contents Histogram Smoother univariate density estimation The choice of smoothing parameter The constructing the confidence intervals Univariate cumulative distribution function estimation
1 n Kh (x − Xi ) n i=1
.
The statistical properties
Chapter 4: Nonparametric density estimation Baisen Liu Contents Histogram Smoother univariate density estimation The choice of smoothing parameter The constructing the confidence intervals Univariate cumulative distribution function estimation
The definition of the density f (x): F (x + h) − F (x) d F (x) ≡ limh→0 . dx h The empirical distribution: f (x) ≡ ˆ (x) = #{xi ≤ x} . F n The histogram estimate of f (x): ˆ(x) = (#{xi ≤ bj +1 } − #{xi ≤ bj })/n , x ∈ (bj , bj +1 ], f h where h = bj +1 − bj is called binwidth.
Chapter 4: Nonparametric density estimation Baisen Liu Contents Histogram Smoother univariate density estimation The choice of smoothing parameter The constructing the confidence intervals Univariate cumulative distribution function estimation
n
K
i=1
x − xi h
,
1 2,
0
if − 1 < u ≤ 1, otherwise.
is called the uniform kernel function.
Chapter 4: Nonparametric density estimation Baisen Liu Contents Histogram Smoother univariate density estimation The choice of smoothing parameter The constructing the confidence intervals Univariate cumulative distribution function estimation