– Let a be an arbitrary Mx1 vector and let
Then
– Since
,
17
Properties of the Correlation Matrix
• Property 3: The correlation matrix of a w.s.s. process is nonsingular due to the unavoidable presence of additive noise.
1
Reference Books
• Simon Haykin , Adaptive Filter Theory, 4th edition, Prentice Hall, 2002. 《自适应滤波器原理》，电子工业出版社, 2010.
• John G. Proakis , Algorithms for Statistical Signal Processing, Prentice Hall, 2002. 《统计信号处理算法》，清华大学出版社, 2003. • James V. Candy, Bayesian Signal Processing, John Wiley & Sons, 2009. • Ali H. Sayed , Fundamentals of Adaptive Filtering, Wiley-IEEE Press, 2003. • Paulo S. R. Diniz, Adaptive filtering: algorithms and practical implementations, Kluwer Academic Publishers, 3rd edition, 2008. • Digital Signal Processing and Statistical Classification, by George J. Miao and Mark A. Clements, Artech House, 2002. • 张贤达，《现代信号处理》，第二版，清华大学出版社, 2002. • 叶中付，《统计信号处理》，中国科学技术大学出版社，2009. • 王永良, 丁前军, 李荣锋,《自适应阵列处理》,清华大学出版社, 2009.
2
Definition of filtering
• A filter is commonly used to refer to a system that is designed to extract information about a prescribed quantity of interest from noisy data.
• We will generally assume that
14
Correlation Matrix
• Let u(n) be the Mx1 observation vector
15
Properties of the Correlation Matrix
• Property 1: The correlation matrix of a stationary discrete-time stochastic process is Hermitian symmetric (Toeplitz).
• u(n), u(n-1), ... , u(n-M)
11.5 11 10.5
10
9.5
9
8.5
8
0
20
40
60
80
100
120
140
160
8
Stochastic Processes
• A stochastic process is strictly stationary,
if its statistical properties are invariant to a time shift • Joint pdf of {u(n), u(n-1), ... , u(n-M)} remain the same regardless of n. • Joint pdf is not easy to obtain,
4
Applications
!!! Noise and errors are statistical in nature !!! We will use statistical tools.
5
Basic Kinds of Estimation
• Filtering (real-time operation) • Smoothing (off-line operation) • Prediction (real-time operation)
3
Applications
• Communications; radar, sonar, • Control Systems; navigation, • Speech/Image Processing; echo and noise cancellation, biomedical engineering • Others; seismology, financial engineering, etc.
– None of the eigenvalues of R is zero due to noise, i.e.
• Property 4: If the order of the elements of the vector u(n) is (time)-reversed, the effect is the transposition of the autocorrelation matrix.
Introduction
BACKGROUND REVIEW Discrete-time and random signals Power spectral density OPTIMAL LINEAR FILTERS Linear stochastic models Wiener filter Adaptive filter State-space model Introduction Kalman filter BAYESIAN SIGNAL PROCESSINF Bayesian approach to state-space model Particle filter Hidden models APPLICATIONS Array processing Speech/image processing
– First and second moments are used frequently.
9
Mean and Covariance
– Mean (Expected) Value of u(n) (1st order)
– Autocorrelation Function of u(n) (2nd order)
u(t) Filter u(n) y(n) Non-linear or y(t)
Non-Linear
Otherwise, it is non-linear.
Linear
or
!!! Non-linear filters may be hard to analyse!!!
7
Stochastic Processes
– We call the process u(n) as ergodic.
• If the process is ergodic, i.e. then • We can estimate the mean value of the process as
13
Ergodic Processes
• For a w.s.s. process, the autocorrelation can be estimated
18
Properties of the Correlation Matrix
19
Properties of the Correlation Matrix
• Property 6: The correlation matrices RM and RM+1 of a stationary discrete-time stochastic process, pertaining to M and M+1observations of the process, respectively are related by
r (n, n k ) c (n, n k )
variance
μ=0 →
11
Ergodic Processes
• Ensemble averages (expectations) are across the process -> can be obtained analytically (actual expectation) • Sample (time) averages are along the process -> can be obtained emprically (from realizations of a process) (estimated expectation)
Statistical Signal Processing - Advanced Filtering
Dr. Weifeng Li Graduate School at Shenzhen, Tsinghua University li.weifeng@
Course Content
– Autocovariance Function of u(n) (2nd order)
–10Biblioteka Mean and Covariance
• Stationary (strictly/w.s.s) processes