The KLT Tracker

In [14], Shi and Tomasi present the KLT (Kanade-Lucas-Tomasi) Tracker, an algorithm that selects features which are optimal for tracking, and keeps track of these features. The basic principle of the KLT is that a good feature is one that can be tracked well, so tracking should not be separated from feature extraction. If a feature is lost in a subsequent frame, the user can optionally ask the procedure to find another one to keep the number of features constant.

A good feature is a textured patch with high intensity variation in both $x$and $y$ directions, such as a corner. Denote the intensity function by $g(x,y)$and consider the local intensity variation matrix

$$Z = \begin{bmatrix}g_x^2 & g_xg_y \\ g_xg_y & g_y^2\end{bmatrix}.$$

A patch defined by a $25\times 25$ window is accepted as a candidate feature if in the center of the window both eigenvalues of $Z$, $\lambda_1$ and $\lambda_2$, exceed a predefined threshold $\lambda$: $\min(\lambda_1,\lambda_2) > \lambda$. A separate parameter sets the minimum distance between (centers of) features.

The (maximum) number of features to be tracked, $N_{f}$, is specified by the user. In the first frame of a sequence, the candidate features are ranked according to their strength defined by $\min(\lambda_1,\lambda_2)$, then $N_{f}$ strongest features are selected if available. (If not, all candidates are used.)

Figure 1 illustrates feature selection by the KLT. A few most dominant features are shown as black points overlaid on the corresponding part of a standard PIV image.

  

Figure 1: Dominant features selected by the KLT and the IPAN Trackers.

The KLT defines a measure of dissimilarity that quantifies the change of appearance of a feature between the first and the current image frame, allowing for affine image changes. At the same time, a pure translation model of motion is used to track the selected best features through the sequence. For reliable and fast processing, the maximum interframe displacement is limited, although it can be quite large compared to that of the conventional optical flow approaches.

When applied to PIV sequences, the KLT Tracker does not provide uniform sampling of the velocity vector field needed for proper visualisation of a flow. However, the spatial distribution of the tracked features tends to be relatively regular because of the above mentioned minimum distance between the features. Most parts of a flow are measured, with the exception of the borders and the regions of poor contrast and visibility. For better visualisation, we resample the KLT results using the procedure discussed in section 2.2.3.

The source code of the KLT Tracker can be downloaded from the web site [3] and tested on arbitrary image sequences.