Surface-based 3D matching is robust – even with occlusion
Surface-based 3D matching is robust – even at (outside) the image border
Surface-based 3D matching is fast and offers the full pose range
What do I want to measure and how accurately? What are the characteristics of the object? What are the setup requirements?
3D matching: Model generation
Best basis for identification applications
Simply faster development
HALCON 10 in a nutshell
Powerful 3D vision
First class matching in any dimension
Faster and smarter
Surface-based 3D matching
Multi-view stereo
HALCON supports multi-view stereo
Reconstruct 3D scenes from multiple images
Reconstruct 3D scenes from multiple images
Best basis for identification applications
Simply faster development
Powerful 3D vision
3D vision
New 3D features
Multi-view calibration
3D object model
Primitives fitting
Picking of piled tubes with cylinder fitting
Surface-based matching
HALCON offers 3D to 3D surface-based matching
HALCON offers a new 3D matching technology: Surface-based 3D matching
Surface-based 3D matching is robust – even with low resolution
Surface-based 3D matching is robust – even with occlusion
Surface-based 3D matching is robust – even with occlusion
© 2003-2010 MVTec Software GmbH
3D Vision and Applications in Robotics
Dr. Olaf Munkelt Harbin, China, 2011
Overview
3D is required in many application areas in different industries
Surface-based 3D matching using Kinect
Image © Microsoft
Bin picking of smoothly shaped objects using surface-based 3D matching
Pose
Process
There is a variety of methods to measure surfaces in 3D
Stereo
Sheet of Light
Time of Flight
Photometric Stereo
Interferometry
Focus Determination
Surface-based 3D matching is robust – even with low resolution
Engine part
Depth image from TOF camera (176 x 144) Object size about 70 x 45
Surface-based 3D matching is robust – even with low resolution (TOF data)
Reconstruction
Matching
… or both approches can be combined!
HALCON 10 in a nutshell
Powerful 3D vision
First class matching in any dimension
Faster and smarter
These 3D methods are also known, but not very common in machine vision
Depth from … Defocus Motion
Shadows
Texture
Perspective
Moiré
Sheet of light
Sheet of light
Localization of a known object and control of the gripping process of the robot
Match 2D representation Refine position for high precision Control robot movement
Most 3D methods are based on the same principles as human 3D vision
Binocular stereo offers a flexible solution for 3D surface measurement
Bin picking of metal cylinders
iStock_000002103108Small We.wmv
The Control Loop
Machine vision application
HALCON
Standard PC Intelligent camera embedded system
result
hole 1 2 3 radius 3.15 mm 3.05 mm 2.79 mm
3D object model
HALCON offers a new data structure for 3D data
HALCON offers a new data structure for 3D data
Object model 3D Points (X,Y,Z) Point normals Triangles Polygons Primitive parameters xyz-Mapping Matching information
Pose from Points
Pose from Contours
Matching
HALCON supports descriptor-based matching
What do I want to measure and how accurately?
What are the characteristics of the object?
Primitive par Radius
Cylinder
Center Direction Radius
Plane
aX + bY + cZ + d = 0
The XYZ-mapping is needed for triangulation
3D sensors
HALCON supports various 3D sensors that can be used with surface-based 3D matching
Time of flight Laser triangulation (incl. Laser) Fringe projection
HALCON allows to use the full range of sensors to extract 3D information
Game industry will provide cheap depth sensors to the vision industry
Image © Microsoft
What are the setup requirements?
Pose recognition based on features
Pose recognition of metal parts with only one camera
Pose recognition based on edge information
xyz_to_object_model_3d
prepare_object_model_3d
segment_object_model_3d
fit_primitives_object_model_3d
Sheet of light combined with the segmentation and fitting of the 3D data allows flexible picking
Surface-based 3D matching is robust
Surface-based 3D matching is robust – even with erroneous data