Create a system that reliably detects activities. Add activities. Improve the output. This could be graphical, musical, or voice output. Consider incorporating a tracker. Make your system truly interactive, i.e., adapt your image analysis modules (e.g., thresholds on intensity changes) based on the user's interaction.
In our research, we recorded hundreds of images of living cells in time-lapse phase-contrast microscopy video. You could help automate the interpretation of these vast amounts of data, which is too time-consuming, costly, and prone to human error when done by hand. You could try to develop a segmentation method based on adaptive thresholding and active contours. You could also try variations of the basic active contour algorithm that we discussed in class.
The lungs deform during inspiration and expiration. Modeling the deformations is of clinical interest as it facilitates the diagnosis and treatment of lung disease. For example, lung cancer is often treated with radiotherapy. Research in medical image analysis has focused on methods to determine the position of a tumor that moves with respiration during radiation treatment and thus reduce the amount of undesirable radiation to surrounding healthy tissue. Here is a paper that describes rigid-body registration of lung surfaces: M. Betke, H. Hong, D. Thomas, C. Prince, J. P. Ko, "Landmark Detection in the Chest and Registration of Lung Surfaces with an Application to Nodule Registration." Medical Image Analysis, 7:3, pp. 265-281, September 2003. pdf. The goal of the project is to extend this work to deformable registration.
A simple 2D face tracker can be implemented by detecting skin color and motion in the video. See, for example, S. Birchfield, "Elliptical Head Tracking Using Intensity Gradients." In Proceedings of the IEEE Computer Vision and Pattern Recognition Conference, pp. 232-237, June 2000. pdf. More interesting would be a generalization of the task to 3D: Implementation of a method that estimates the position and orientation of a head in three-dimensions, for example, Q. Ji and R. Hu, "3D Face pose estimation and tracking from a monocular camera," Image and Vision Computing, Volume 20, Issue 7, 1 May 2002, Pages 499-511. pdf.
See previous project. Here you will use the view of the face from cameras. I could give you a paper to implement (or extend). Please ask me for references.
Implement a method to recognize different hand shapes or gestures for human-computer interaction, or simply tracks hands for behavior understanding.
- We recently developed a powerful hand and finger tracker: Z. Wu, M. Betke, J. Wang, V. Athitsos, and S. Sclaroff, "Tracking with Dynamic Hidden-State Shape Models," Computer Vision - ECCV 2008, 10th European Conference on Computer Vision, Marseille, France, October 12-18, 2008, Proceedings, Part I, LNCS 5302, pages 643-656, Springer-Verlag. pdf. The system can identify the curling degree of each finger from certain view points. Your project could use our system and extend it. Your extension could focus on making the system more general to allow recognition from more view points. Or your contribution could be to add a 2nd camera and combine the detection results from both views. This would be a very exciting, publishable result!
- If you would like to try an easier (but very respectable) project on hand detection and tracking, check out a previous CS 585 course project: S. Crampton, M. Betke, and S. Sclaroff. "Counting Fingers in Real Time Using Computer-Vision Techniques," 2004. pdf. You could reimplement the approach and make it part of the Student Village art installation project.
- Another relevant paper on hand posture recognition is: J. Triesch and C. von der Malsburg. "Classification of hand postures against complex backgrounds using elastic graph matching," Image and Vision Computing, Volume 20, Issues 13-14, 1 December 2002, Pages 937-943. pdf.
- Hand tracking is also very important for developing an automated sign language translator: Y. Quan, S. Sclaroff and V. Athitsos, "Automatic 2D Hand Tracking in Video Sequences," IEEE Workshop on Applications of Computer Vision, 2005. pdf. Other papers on ASL hand recognition can be found at http://www.cs.bu.edu/groups/ivc
- Here is a another hand tracking paper: J. Letessier and F. Berard. Visual tracking of bare fingers for interactive surfaces. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (UIST 04)}, pages 119-122. ACM Press, 2004. pdf.
Please see our webpage on Thermal Video Analysis of Bats. Censusing natural populations of bats is important for understanding the ecological and economic impact of these animals on terrestrial ecosystems. It is challenging to census bats accurately, since they emerge in large numbers at night from their day-time roosting sites. We have used infrared thermal cameras to record Brazilian free-tailed bats in California, Massachusetts, New Mexico, and Texas and have developed an automated image analysis system that detects, tracks, and counts the emerging bats.
We need help in improving our detection and tracking methods. In particular, we would like to automatically evaluate the shape of flying bats. Using the material we covered in CS 585 on thresholding, segmentation, and active contour methods, this project would build a system that analyzes the shapes of flying bats.
Training shapes:
Test shapes:
Please see our webpage on Thermal Video Analysis of Bats. This project uses thermal videos of wind turbines. The goal is to detect the bats and birds that might be flying by. Bats and birds have been killed by wind turbines, and maybe there is a way to help wildlife avoid the deadly blades. Here is an example of a sequence of thermal videos that you could analyze for this project (the blade is moving upwards, the bat is not hit):
We have used Boston University's MRI and CT scanners to image a bat in three dimensions. We need somebody to help design an algorithm to visualize the bat in these data sets and to analyze its anatomical features. (To my knowledge, no one has done this with bats! This would be an exciting research project!)
Develop a system that can detect facial features and/or their motion, for example, eyebrow raises, in video sequences of American Sign Language (ASL) communications. An eyebrow raise is an important grammatical tool in American Sign Language to indicate a question. You could built upon a system that was developed by one of my students in CS 585. The ASL data is available from three view points:
You could use SignStream's ASL video annotations that include events such as "eyebrow raises," which were manually determined by linguists and can serve as your "ground truth." Check out a previous CS 585 course project: T. Castelli, M. Betke, C. Neidle, "Facial feature tracking and occlusion recovery in American Sign Language." In A. Fred and A. Lourenço, editors, Pattern Recognition in Information Systems: Proceedings of the 6th International Workshop on Pattern Recogntion in Information Systems - PRIS 2006, pages 81-90, Paphos, Cyprus, May 2006. INSTICC Press. pdf. See also Technical Report BU-CS-2005-024.
Develop a people tracking program. Videotape people walking on campus (you can use our cameras). Can you automatically detect which moving "blobs" are people and not cars, bikes, dogs etc? Could you apply your method to improve airport security? You may reimplement: P. K. TrakulPong and R. Bowden. "A real time adaptive visual surveillance system for tracking low-resolution colour targets in dynamically changing scenes,"Image and Vision Computing, Volume 21, Issue 10, September 2003, Pages 913-929. pdf. There are some very interesting newer papers. Please ask me for references.
Images often must be resized to fit in PDA or cell phone displays. If the aspect ratio changes, image content may be deformed or important objects in the image may be cropped. For example, if image A must be resized into a 100-pixel height image, a simple downsampling algorithm would result in image B. A content-aware resizing algorithm would produce image C instead. If the original image must be resized into 200x150-pixel image, a simple downsampling algorithm would deform the image content and yield image D. A content-aware resizing algorithm would produce image E instead.
A 
B
C
D
E
For ideas how to approach the problem of designing a context-aware resizing algorithm, you may read the 2007 SIGGRAPH paper by Avidan and Shamir or the 2008 SIGGRAPH paper by Rubinstein et al. about seam craving.
This project researches efficient ways to store and retrieve images and video in a large databases. Various applications are possible: web image retrieval, CT databases for lung cancer screening, video surveillance databases. See, for example, the paper by Petrakis and Faloutsos on medical image databases that we discussed in class.
The goal of this project is to develop a vision-based system that detects if a driver is falling asleep behind the wheel. You may take one of our video cameras and film a friend behind the wheel. Try to detect the blinking eyes of a driver using image differencing techniques. Can you detect the difference between "normal" blinking and closing the eyes for a longer period of time? Test this only when the car is parked! The challenge of this project is to detect eye closures under various lighting conditions. So park your car in various locations and capture images at different times of the day and in various weather situations. Please ask me for references.
Driver Face Analysis for Intelligent Vehicles
The car industry is curious to find out as much as possible about driver behavior. What does a driver focus on while driving? How long does a driver look at the dashboard or the rear mirror? The goal is to design smarter, safer cars. Drivers are videotaped in simulators and in the real world. The videos are then annotated by hand in a painstaking process. Automation of this process is needed, because it would allow a larger test population. In addition, automatic recognition of driver intentions or mistakes may trigger safety features in our future "intelligent cars." The goal of this project is to implement a system that automatically analyzes the driver's face. The system should detect and track facial features. This would be a good group project, since it can be combined with the "Head Tracker" and "Warning System for Tired Drivers" projects. Please ask me for references.
