Adding the known bias, we get the final estimate as:
7
Experiment Result
Emotion label preprocessing: gray dots indicate individual second-bysecond labels, red ellipses indicate the estimates of the distribution, and blue ellipses indicate the predictions using Kalman filter
Much relaxation of the independence assumptions Avoid label bias problem
Disadvantage:
much complexity
17
References
1. /wiki/Emotion_and_memory 2. Y. E. Kim, E. M. Schmiddt, R. Migneco, B. G. Morton, P.Richardson, J. Scott, J. A. Speck and D. Turnbull, “Music emotion recognition: A state of the art review”, in ISMIR, Utrecht, Netherlands, 2010 3. Y. E. Kim, E. M. Schmiddt, and D. Turnbullm, “Feature selection for content-based, time-varying musical emotion regression”, in ACM MIR, Philadelphia, PA, 2010
Where Z(x) is a normalization factor
Maximum Likelihood Parameter Inference The log likelihood is given by:
Differentiating the log-likelihood with respect
So
Where
14
CRF in Musical Emotion Recognition
Label: A-V modeled acoustic data
Observation: Mel-frequency cepstral coefficients (MFCC) Transition probabilities: emotions tend to change smoothly
3
Linear Gauss-Markov model
4
Statistical Assumptions
Driving noise w and observation noise v are zero mean Gaussian
W and v are independent of X and Y
4. Y. E. Kim, E. M. Schmiddt, “Prediction of time-varying musical mood distributions from audio”, in ISMIR, Utrecht, Netherlands, 2010
5. Y. E. Kim, E. M. Schmiddt, “Prediction of time-varying musical mood distributions using Kalman filtering”, in IEEE ICMLA, Washington, D.C., 2010
State probabilities: state of emotion Recognition: emotion changes over time
15
Emotion Space Heatmap Prediction
16
Advantages and Disadvantages
Advantages:
8
Advantages and Limitations
Advantages:
Smooth and robust estimates
Distribution evolves over time
Limitations:
Limited model complexity was unable to cover a wide variance in emotion space dynamics all three become darker as time progresses, i.e. the estimation becomes indeterminism
6. Y. E. Kim, E. M. Schmiddt, “Modeling musical emotion dynamics with conditional random fields”, in ISMIR 2011
7. Hanna M. Wallach, “Conditional Random Fields: An Introduction”, in 2004 8. Greg Welch, and Gary Bishop, “An Introduction to the Kalman Filter”, in 2006
The process x is Markov, i.e.,
5
Kalman Filtering
First performing the forward recursions:
6
Kalman filtering (Cont.)
Then performing the backward recursions:
2
Data Training with Kalman Filtering
AKA: Linear Quardratic Estimation (LQE)
Recursive Estimator Predict: uses the state estimate from the previous timestep to produce an estimate of the state at the current timestep, aka priori state estimate
Modeling Musical Emotion Dynamics
senter: SHUMIN XU
1
Arousal-Valence Model
2-dimensions representation of emotional memory Arousal: High- vs. low-energy (e.g. energetic vs. calm) Valence: positive vs. negative (e.g. happy vs. sad)
Update: the current a priori prediction is combined with current observation information to refine the state estimate, termed as posteriori state estimate
10
Conditional Random Fields (Cont.)
Potential functions:
Joint Probability:
Let Rewrite the probability function as:
11
CRF: Max Likelihood Parameter Inference
12
CRF: Dynamic Programming
Right side could be rewritten as
Using forward-backward algorithm, define
13
CRF: Dynamic Programming (Cont.)
And the recursion relations:
9
Conditional Random Fields (CRF)
Definition: For observations sequence X and label sequence Y, Let G = (V, E) be a graph such that , so that Y is indexed by the vertices of G. Then (X, Y) is a conditional random field when the random variables Yv conditioned on X, obey the Markov property with respect to the graph: where w~v means that w and v are neighbors in G.