Machine Perception and Interaction Group (MPIG)

Pyramid Implementation
Initial guess
Machine Perception and Interaction Group (MPIG)

Feature Extraction
Feature matching
Compute E or F for [R|t]
Drawing path
Machine Perception and Interaction Group (MPIG)

second
Feature matching
Machine Perception and Interaction Group (MPIG)

rich feature descriptors
example
code FlannBasedMatcher matcher; std::vector< DMatch > matches; matcher.match(descriptors1,descriptors2,matches); Mat img_matches; drawMatches( img1, keypoints1, img2, keypoints2, matches, img_matches ); //-- Draw matches imshow("Matches", img_matches ); //-- Show detected matches
FLANN
Machine Perception and Interaction Group (MPIG)

rich feature descriptors
example
FLANN
Machine Perception and Interaction Group (MPIG)
rich feature descriptors
brute-force Comparing each feature in the first set to each feature in the second set
FLANN (Fast Library for Approximate Nearest Neighbors) K-means tree K-dimension tree
We write as follow for convenience
Machine Perception and Interaction Group (MPIG)

Standard KLT algorithm
Machine Perception and Interaction Group (MPIG)
KLT algorithm
u v
gray value u is I(x,y)
The goal is to find v on J, where I(u) and J(v) are similar The way is to compute d
Machine Perception and Interaction Group (MPIG)

Standard KLT algorithm
Machine Perception and Interaction Group (MPIG)

Standard KLT algorithm
Machine Perception and Interaction Group (MPIG)

Code of LK based openCV
Address: /sunzuolei/vo_basis
calcOpticalFlowPyrLK ()
vector<float> err; vector<uchar> status; Size winSize=Size(21,21); //设定金字塔层搜索窗口尺寸

optical flow
Lucas-Kanade (LK or KLT) [1]
three assumptions
• Brightness constancy • Temporal persistence or small movements
• Spatial coherence
[1] B. D. Lucas, T. Kanade, An Iterative Image Registration Technique with an Application to Stereo Vision .
Machine Perception and Interaction Group (MPIG)

Eliminate errors
code double max_dist = 0; double min_dist = 100; for( int i=0; i<descriptors1.rows; i++ ) { double dist = matches[i].distance; if( dist < min_dist ) min_dist = dist; if( dist > max_dist ) max_dist = dist; } //-- Draw only "good" matches (i.e. whose distance is less than 2*min_dist ) std::vector< DMatch > good_matches; for( int i = 0; i < descriptors1.rows; i++ ) { if( matches[i].distance < 2*min_dist ) { good_matches.push_back( matches[i]); } }
Machine Perception and Interaction Group (MPIG)
example
Machine Perception and Interaction Group (MPIG)

thanks
Machine Perception and Interaction Group (MPIG)
TermCriteria termcrit=TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, 30, 0.01);//指定光 流法搜索算法收敛迭代的类型 calcOpticalFlowPyrLK(img1, img2, points1, points2, status, err, winSize, 3, termcrit, 0, 0.001);
Assumptions
Brightness Constancy Assumption translational model:
Machine Perception and Interaction Group (MPIG)

Assumptions
Taylor
2nd assumption
Machine Perception and Interaction Group (MPIG)
//-- Draw only "good" matches Mat img_matches; drawMatches( img1, keypoints1, img2, keypoints2, good_matches, img_matches, Scalar::all(- 1), Scalar::all(-1),vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS ); //-- Show detected matches imshow( "Good Matches", img_matches );
Point matching using
rich feature descriptors
brute-force
FLANN
optical flow
Horn-Schunck
Lucas-Kanade
Machine Perception and Interaction Group (Meption and Interaction Group (MPIG)

Assumptions
3rd assumption
Machine Perception and Interaction Group (MPIG)


Machine Perception and Interaction Group (MPIG)

Eliminate errors
Getting rid of points for which the KLT tracking failed or those who have gone outside the frame int indexCorrection = 0;//初始化参数 for( int i=0; i<status.size(); i++) { Point2f pt = points2.at(i- indexCorrection); if ((status.at(i) == 0)||(pt.x<0)||(pt.y<0)) { if((pt.x<0)||(pt.y<0)) { status.at(i) = 0;//将对应的这组光流置零,等同于未发现该光流 } points1.erase (points1.begin() + (i - indexCorrection));//删除该点 points2.erase (points2.begin() + (i - indexCorrection)); indexCorrection++; } }