CAP 6938-02

Visual Simulation

Professor Hassan Foroosh

Dept. of Computer Science

University of Central Florida

Course Rationale

Photo-realistic view synthesis is a central goal in visual simulation of real world.  The impact, and the increasing demand is visibly reflected in the entertainment industry, e.g. games and thrilling movies such as Matrix, Star Wars, Jurassic Park, etc. The challenge is to augment real scenes seamlessly with virtual objects or to blend real world with a virtual world. However, the current state of the art is highly manual-intensive and requires a great deal of artistry.  Motivated by overcoming these limitations, a new area has recently emerged on the intersection of computer vision and computer graphics, which is referred to as “Image-Based Modeling and Rendering’’ (IBM/R). Currently only a few major institutions nationwide (e.g. MIT, Stanford, Berkeley, CMU, Washington, Penn State) have introduced this new emerging bi-disciplinary research area in their curriculum. However, the area is rapidly expanding and it’s impact is expected to be felt in many other disciplines and applications.

The main challenge raised by IBM/R is to explore the endless possibilities that arise by merging vision and graphics techniques, which have traditionally evolved almost independently. For instance, non-photo realistic modeling has been a major area of research in computer graphics for years. Computer vision provides new promising possibilities by capturing photo-realistic representations of the real world directly from photographs and videos.  Vision based techniques also provide new human-computer interaction tools that promise to revolutionize graphics industry in terms of affecting artistic and media productions by facilitating image and video editing. Hardware and software architectures are also believed to take advantage of IBM/R representations in near future in order to render complex scenes more efficiently.

In this course, we will survey many of the computer vision techniques that are applicable to the field of computer graphics, with the aim of performing visual simulation and augmentation of real world. The topics covered include visual processing, image composition, motion, single-view and multi-view geometry, pose/structure estimation, depth/shape recovery, layers, 2.5D/3D, image-based rendering, and light. Prior background in computer vision is assumed.  However, the fundamental concepts and mathematics that underlie these approaches will be covered in addition to the algorithms themselves.

Prerequisites:  a prior course in computer vision or computer graphics.  The course is open to graduate students.

Grading Policy:  Students are primarily assessed based on laboratory work, case studies, class presentations, and a final term project. The grading will be equally distributed among these components.

Course web site: http://www.cs.ucf.edu/courses/cap6938-02/

Course email list: send a mail to majordomo@cs.ucf.edu and put the following

subscribe cap6938-02

in the body of the email, NOT in the subject.

To print a copy of the Syllabus use this link.

Schedule

Week	Topic
1	· Introductory Notes · Lecture Notes on Projective Geometry and Single View Geometry · This weeks Reading assignment: Chapter 2 of “Three Dimensional Computer Vision”, by O.D. Faugeras. The book should be on the reserved shelf for this course.
2	· Lecture Notes on Multi-view Geometry · Programming Assignment 1 · This weeks Reading assignment: Chapter 3 & 4 of “Multiple View Geometry”, · By Hartley and Zisserman. The book should be on the reserved shelf
3	· Lecture Notes on Structure From Motion
4	· Lecture Notes on Calibration & Bundle ADjustment
5	· Lecture Notes on Visual Motion Estimation
6	Programming Assignment 2
7	Lecture Notes on Texture Synthesis
8	Break
9	Final Project suggestions: Project 1 Project 2 Project 3 Project 4

Texts & References

There is no single textbook for this course. However, below is a list of some of the books used. In addition, important research papers appeared in the literature will be used as the reference material, some of which are given below under the corresponding topics.

Books:

Computer vision texts:

• R. Hartley and A. Zisserman, Multiple View Geometry, Cambridge University Press, 2000.
• Faugeras, O.D., Three-Dimensional Computer Vision, MIT Press, 1993.
• E. Trucco and A. Verri, Introductory Techniques for 3-D Computer Vision, Prentice Hall,1998
• Nalwa, V., A Guided Tour of Computer Vision, Addison-Wesley, 1993.

Special topics:

• J. A. Sethian. Level Set Methods. Cambridge University Press, 1996.
• A. Blake and M. Isard, Active Contours, Springer-Verlag, 1998.
• G. Wolberg, Digital Image Warping, IEEE Computer Society Press, 1990.

Computer graphics texts:

• Foley, van Dam, Feiner, and Hughes. Computer Graphics: Principles and Practice, 2nd edition in C. ISBN 0-201-84840-6.
• Alan Watt, 3D Computer Graphics, 2nd ed. ISBN 0-20-163186-5 .
• Hearn and Baker, Computer Graphics, C Version, 2nd ed. ISBN 0-13-530924-7.
• Andrew S. Glassner et. al. ed., Graphics Gems I-V, ISBN 0122861663 .
• Woo, Neider, and Davis. OpenGL Programming Guide,, 2nd edition. ISBN 0-201-46138-2.

Research Papers:

Blending and Composition:

• J. F. Blinn, Jim Blinn's Corner: Compositing, Part 1: Theory, IEEE Computer Graphics and Application, 14(5), Sept. 1994, pp 83-87
• Burt, P. J. and Adelson, E. H., A multiresolution spline with applications to image mosaics, ACM Transactions on Graphics, 42(4), October 1983, 217-236.
• Edge detection
• John Canny, A Computational Approach to Edge Detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(6), November, 1986, 679-698.

Image warping:

• Paul Heckbert, Survey of Texture Mapping (text, figures), IEEE Computer Graphics and Applications, 6(11), November 1986, 56-67. 
• Beier, T. and Neely, S., Feature-Based Image Metamorphosis, Computer Graphics (SIGGRAPH'92), 26(2), July 1992, 35-42

Motion, Registration, Mosaic:

• Bergen, J. R. and Anandan, P. and Hanna, K. J. and Hingorani, R., Hierarchical Model-Based Motion Estimation, ECCV'92, May 1992, 237-252, Springer-Verlag.
• H. Foroosh, “Adaptive Dense Optical Flow Using a Non-stationary Model”, Int. Journal of Computer Vision, under revision, submitted in 2002;
• H. Foroosh, J. Zerubia and M. Berthod, “Extension of Phase Correlation to Sub-pixel Registration”, IEEE Trans. Image Processing, vol. 11, Issue 3, pp. 188-200, 2002.
• Lucas, B. D., and Kanade, T., An Iterative Image Registration Technique with an Application to Stereo Vision, Proceedings of 7th International Conference on Artificial Intelligence, pp. 121-130, 1981.