3/31/2005 CSE 576: Computer Vision 18
Virtualized RealityTM
Takeo Kanade, CMU
• generate new video
• steerable version used for SuperBowl XXV “eye vision” system
3/31/2005
CSE 576: Computer Vision
20
Image Enhancement
High dynamic range photography [Debevec et al.’97; Mitsunaga & Nayar’99]
• combine several different exposures together
Many applications
• • • • small images faster to process good for multiresolution processing compression progressive transmission(传送)
Known as “MIP-maps” in graphics community Precursor(ancestor) to wavelets
3/31/2005 CSE 576: Computer Vision 19
Edge detection and editing
Elder, J. H. and R. M. Goldberg. "Image Editing in the Contour Domain," Proc. IEEE: Computer Vision and Pattern Recognition, pp. 374-381, June, 1998.
filter mask
“Gaussian” Pyramid “Laplacian” Pyramid
• Created from Gaussian pyramid by subtraction Ll = Gl – expand(Gl+1)
3/31/2005 CSE 576: Computer Vision 26
Debevec, Taylor, and Malik, SIGGRAPH 1996
3/31/2005
CSE 576: Computer Vision
15
Face Modeling
3/31/2005
CSE 576: Computer Vision
16
View Morphing
Morph between pair of images using epipolar geometry [Seitz & Dyer, SIGGRAPH’96]
• Project 1description and demo [Ian Simon]
3/31/2005 CSE 576: Computer Vision 23
Image Pyramids
Image Pyramids
3/31/2005
CSE 576: Computer Vision
25
Pyramid Creation
Image Features
CSE 576, Spring 2005
About me
• Ph. D., Carnegie Mellon, 1988 • Researcher, Cambridge Research Lab at DEC, 1990-1995 • Senior Researcher, Interactive Visual Media Group, Microsoft, 1995• Research interests:
Computer Vision [Trucco&Verri’98]
3/31/2005
CSE 576: Computer Vision
7
Image-Based Modeling
image processing graphics Images (2D) vision 3 Image processing 4 Feature extraction 7 Image alignment 8 Mosaics 9 Stereo correspondence 11 Model-based reconstruction 14 Image-based rendering 12 Photometric recovery 2.1 Geometric image formation 2.2 Photometric image formation Geometry (3D) shape
3/31/2005
CSE 576: Computer Vision
11
Applications
• • • • • • Tracking and surveillance (Sarnoff) Fingerprint recognition (Digital Persona) Biometrics / iris scans (Iridian Technologies) Vehicle safety (MobilEye) Drowning people (VisionIQ Inc) Optical motion capture (Vicon)
• • • • • • What is computer vision? Scale-space and pyramids What are good features? Feature detection Feature descriptors (Next lecture: feature matching)
right pyramid
bd Blending
3/31/2005
CSE 576: Computer Vision
30
original
smoothed (5x5 Gaussian)
why does this work?
smoothed – original
Scale space: insights
As the scale is increased
• edge position can change • edges can disappear • new edges are not created
Bottom line[key] need to consider edges at different scales (or else know what scale you care about)
+
Photometry appearance
5 Camera calibration
6 Structure from motion
3/31/2005
CSE 576: Computer Vision
8
Applications
• Geometric reconstruction: modeling, forensics, special effects (ILM, RealVis,2D3) • Image and video editing (Avid, Adobe) • Webcasting and Indexing Digital Video (Virage) • Scientific / medical applications (GE)
• Project 1description and demo [Ian Simon]
3/31/2005 CSE 576: Computer Vision 3
What is Computer Vision?
What is Computer Vision?
• • • • Image Understanding (AI, behavior) A sensor modality for robotics Computer emulation of human vision Inverse of Computer Graphics
• computer vision (stereo, motion), computer graphics (image-based rendering), data-parallel programming
3/31/2005 CSE 576: Computer Vision 2
Today’s lecture
3/31/2005
CSE 576: Computer Vision
12
Image Morphing
3/31/2005
CSE 576: Computer Vision
13
Panoramic Mosaics
+
+ … +
=
3/31/2005
CSE 576: Computer Vision
14
3D Shape Reconstruction
3/31/2005
CSE 576: Computer Vision
17
Virtualized RealityTM
Takeo Kanade, CMU
• collect video from 50+ stream
reconstruct 3D model sequences
/afs/cs/project/VirtualizedR/www/VirtualizedR.html
Octaves in the Spatial Domain
Lowpass Images
Bandpass Images
3/31/2005 CSE 576: Computer Vision 27
Pyramids
Advantages of pyramids
• Faster than Fourier transform • Avoids “ringing” artifacts