computer vision - Intuitive explanation of tracking with Kalman filters

I would much appreciate an intuitive explanation for (visual) tracking with Kalman filters. what I know:

Prediction step:

• Dynamic system state $\mathbf x_t$: target location at time $t$


• Measurement $\mathbf z_t$: the image at time index $t$ (??)

Based on images/measurements $1\rightarrow(t-1)$ I want to predict state $\mathbf x_t$? (using dynamic equation) Is that correct ?

How can I interpret the correction step into those terms (image, target location) ?

Answer

First you have to assume a motion model. Let's say you wish to track a ball flying through the air. The ball has a downward acceleration due to gravity of 9.8m/s^2. So in this case the constant acceleration motion model is appropriate.

Under this model, your state is position, velocity, and acceleration. Given the previous state you can easily predict the next state.

You also have a notion of detection. You have a video of the ball moving, and you somehow have to detect the ball in each video frame (e. g. by using background subtraction).

Your detections are noisy. Also, the motion of the ball does not exactly fit the constant acceleration model because of air resistance, wind, cosmic rays, etc. The Kalman filter needs two matrices describing this: one is the covariance of the measurement noise (your detection inaccuracy), and one for the covariance of the process noise (how the motion of the ball deviates from the model you have specified).

If you are tracking a single object, then the Kalman filter lets you smooth out some of the noise, and also predict where the object is when a detection is missing (e. g. if the object is occluded). Here is an example of tracking a single object with a Kalman filter using the Computer Vision System Toolbox for MATLAB.

If you are tracking multiple objects, then the Kalman filter predictions let you decide which detection goes with which object. A good way to do that is to use the log likelihood of the detection given the error covariance of the prediction. Here is an example of tracking multiple objects with a Kalman filter.