Advances in Imaging: emerging devices and visual mining
Date: TBD 2016 | Tuition: TBD | Continuing Education Units (CEUs): TBD
*This course has limited enrollment. Apply early to guarantee your spot.
Application Deadline »
The course provides an overview of computational imaging and visual mining techniques, including novel imaging platforms to sample light in radically new ways and emerging topics in image analysis and exploitation. Participants will discuss emerging new cameras that can sample the high dynamic range (HDR), light field, or wide spectrum. The course will also address the ultra-fast optics for femto-photography and diffraction-beating technologies for microscopy that are bringing unprecedented resolution in time and space and nanophotography. We will explore visual mining techniques using image processing and computer vision methods that are allowing scalable and human-in-loop exploitation of visual data.
The course surveys the landscape of imaging hardware, optics, sensors, and computational techniques through a mix of theory, hands-on open-ended exercises (rapid prototyping), and practical applications. Participants will learn about high-end imaging devices and observe them in up-close demonstrations. In addition, participants will explore computer vision using installed cameras, connected cameras in the cloud, and OpenCV on mobile platforms. Participants will explore emerging solutions that are opening up new research and commercial opportunities in immediate as well as future applications. This course includes group discussions where participants can share their industry or academic experiences on what imaging technology trends are emerging. As a group, at the end of each day, we will try to predict the evolution of imaging technology on 5 and 10+ year timeframes. Participants will get exposure to research at the MIT Media Lab by engaging with members of the Camera Culture Group. They will tour the Media Lab, observe demos, and learn its philosophy and how it works through case studies and discussions.
Key topics include light fields, high-dynamic range imaging, time of flight imaging, signal processing, applied optics, Fourier optics, ultrafast and multi-spectral imaging, compressive sensing, computer vision, web crawling on visual data, image analysis on mobile phones, and social photo collections.
Fundamentals: Core concepts, understandings, and tools (30%)
Latest Developments: Recent advances and future trends (40%)
Industry Applications: Linking theory and real-world (30%)
Lecture: Delivery of material in a lecture format (60%)
Discussion or Groupwork: Participatory learning (25%)
Labs: Demonstrations, experiments, simulations (15%)
Introductory: Appropriate for a general audience (50%)
Specialized: Assumes experience in practice area or field (30%)
Advanced: In-depth explorations at the graduate level (20%)
The participants of this course will:
- Understand the basics of a variety of computational imaging and visual mining techniques, both those used in industry today and cutting-edge techniques from the laboratory.
- Observe demonstrations of various imaging hardware and visual analysis software.
- Learn new methods for overcoming the traditional constraints in imaging using optics, sensors, and computer vision.
- Explore new emerging solutions that are opening up new research and commercial opportunities in current and future applications.
- Participate in small group discussions about the future of imaging, including future products, services, and societal impact.
- Meet and network with the Boston area imaging community via tours and social events planned each evening.
Who Should Attend
The course is suitable for decision-makers and planners for the next generation of imaging solutions, engineers and designers of imaging systems, and anyone interested in reviewing existing and emerging solutions in optics, sensors, image analysis, and computer vision. Application areas include consumer photography (including mobile phones), industrial machine vision, and scientific and medical imaging.
Background: There are no pre-requisites but a general knowledge of technologies that involve image processing, optics, and sensors is a plus.
Day 1 – Morning: Beyond a 2D Image
A computational camera attempts to digitally capture the essence of visual information by exploiting the synergistic combination of task-specific optics, illumination, sensors, and processing. Demonstrations and discussions will center on thermal cameras, multi-spectral cameras, high-speed cameras, and 3D range-sensing time-of-flight cameras and camera arrays. Participants will address opportunities in scientific and medical imaging, mobile phone-based photography, cameras for human-computer interaction (HCI), and sensors mimicking animal eyes.
Day 1 – Afternoon: Rethinking Cameras and New Devices in the Market
Several new imaging devices are now in the market. They include: Google Glass, Microsoft Kinect, Leap Motion, lifelogging cameras, time-of-flight cameras, Google Tango phones, Lytro light field cameras, and FLIR thermal IR cameras. They are being paired with new interaction and display solutions such as Oculus Rift VR, E-Ink, and glassesfree 3D displays.
This session covers the complete camera pipeline. The instructor will explore several physical imaging prototypes plus commercial devices and help participants understand how each stage of the imaging process can be manipulated.
The day will end with a tour of the MIT Media Lab where many of these solutions are being used and new technologies are being developed.
Day 2 – Morning: Impact of Other Fields
This field—at the intersection of signal processing, applied optics, computer graphics and vision, electronics, art, and online sharing through social networks—is emerging as truly multidisciplinary. It cannot be studied in isolation. This session will examine several multidisciplinary impacts, such as whether innovative camera-like sensors can overcome the tough problems in scene understanding and generate insightful awareness, creating actionable information. In addition, participants will see how new algorithms are emerging to exploit unusual optics, programmable wavelength control, and femtosecond-accurate photon counting to decompose the sensed values into perceptually critical elements.
Day 2 – Afternoon: Computational Imaging and Computational Photography
Participants will explore modern methods for capturing and sharing visual information. If novel cameras can be designed to sample light in radically new ways, then rich and useful forms of visual information may be recorded—beyond those present in traditional photographs. Furthermore, if the computational process can be made aware of these novel imaging models, then the scene can be analyzed in higher dimensions (beyond 2D and 3D) and novel aesthetic renderings of the visual information can be synthesized. As an introduction to computer vision lectures on day three, participants will study the abilities of human vision versus machine vision and computer vision.
The day will end with a networking event with the Boston area imaging community including students, researchers, entrepreneurs, and professionals. This is part of the monthly meetup group called Imaging Café (http://tinyurl.com/imagingcafe).
Day 3 – Morning: Computer Vision, Low-level Image Features, and Analysis
We don’t see with our eyes. We record with our eyes and see with our brains. Machines don’t see with cameras. Cameras record the photons and computers reconstruct the photo and also make a sense of the world. Computer vision techniques include low-level methods such as image processing and analysis that including filtering (think Instagram) and image statistics. Participants will study the range of low-level techniques and perform some hands-on experiments.
Day 3 – Afternoon: Computer Vision, Mid-level and High Level, Shape Recovery, Object Recognition, and Machine Learning
More sophisticated computer vision methods mimic visual analysis in the human brain. In fact, human vision is less concerned with pixel values and the majority of processing is dedicated to mid-level and high-level vision. Mid-level processing includes segmentation and shape understanding and this course will look at techniques and standard methods for both. In addition, 3D shape recovery is a very important operation in industrial and scientific imaging. Participants will survey the state-of-the-art techniques. High-level computer vision deals with object/face detection and recognition and generic scene understanding. Instructors will explore these topics and also introduce new topics in machine learning, clustering, compressive sensing, and sparse representations. The emerging new field of visual social computing is at the intersection of Internet vision, human computing, and social computing.
The day will end with an optional tour of an MIT imaging startup where participants will meet the team and hear their stories of development and innovation of MIT technologies in a commercial setting.
Day 4 – Morning: Roadmap for Imaging Applications
The final day explores the impact of new imaging technology and applications on society—how imaging will change our world in the next five years. A series of short presentations will be followed by discussions as a class or in small groups.
With more than a billion people now using networked mobile cameras, we are seeing a rapid evolution in activities based on visual exchange. The capture and analysis of visual information plays an important role in photography, art, medical imaging, tele-presence, worker safety, scene understanding, and robotics. But current computational approaches analyze images from cameras with limited abilities. The goal of MIT’s Camera Culture Group is to go beyond post-capture software methods and exploit unusual optics, modern sensors, programmable illumination, and bio-inspired processing to decompose sensed values into perceptually critical elements. A significant enhancement in the next billion cameras to support scene analysis and mechanisms for superior metadata tagging for effective sharing will bring about a revolution in visual communication. We will discuss strategies the Camera Culture Group has developed and have an interactive exercise to map all the imaging technologies and how they are being used in various sectors, followed by working in small teams to create a roadmap for each sector.
Day 4 – Afternoon: The Future of Imaging
Guided by the questions that follow, the course will end by addressing the future of imaging. What will a camera/display look like in coming years? How will the next billion cameras change social culture? How can we augment the camera to support best “image search?” How will portable health diagnostics impact health care? Will we live mostly in virtual/augmented reality? How will ultra-high-speed/resolution imaging change us? How can we improve “trust” in imaging? Can we print anything? What are the opportunities in pervasive recording? What features will be in Photoshop in coming years? What is the future of moviemaking, news reporting, or sports viewing? These questions will shape the future of imaging products and services.
Course schedule and registration times
Class runs 9:30 am - 5:15 pm all days.
- Morning lecture and demonstrations, 9:30 am – 1:00 pm (Session 1: 9:30 – 11:00 am, Session 2: 11:30 am – 1:00 pm)
- Afternoon lectures and demonstrations or discussion, 2:00 – 5:15 pm (Session 3: 2:00 – 3:30 pm, Session 4: 3:45 – 5:15 pm)
- Evening activities, 6:00 – 7:30 pm
There will be a reception and dinner for faculty and participants on Monday evening. On Tuesday and Wednesday evenings, there will be networking opportunities with the Boston area imaging community.
Laptops are required for this course.
project manager, u.s. navy
"A lot of the concepts were used immediately to start interactions with my team."
vice president for research and development, nexterra foundation
"The instructors seemed to have a goal of exposing the class to the spirit of innovation in which they perform their research at MIT, and the interactions really seemed driven by that goal. It was good."
"The course was very enlightening and stretched my imagination alot."
professor, university of new brunswick
"The course provided us with a platform to not just learn latest technologies but also meet top leaders in imaging. What I learned from the course is beyond what I could learn from a normal course."
"This is an excellent course not just for learning new knowledge but also for finding new collaboration opportunities."
About The Lecturers
Ramesh Raskar is an Associate Professor at the MIT Media Lab and heads the Lab’s Camera Culture research group. He joined MIT from Mitsubishi Electric Research Laboratories (MERL) in 2008.
Raskar's research interests span the fields of computational light transport, computational photography, inverse problems in imaging, and human-computer interaction. Recent projects and inventions include transient imaging to look around a corner (CORNAR), low-cost eye care devices (NETRA, CATRA), a next-generation CAT-scan machine, imperceptible markers for motion capture (Prakash), long distance barcodes (Bokode), touch+hover 3D interaction displays (BiDi screen), new theoretical models to augment light fields (ALF) to represent wave phenomena and algebraic rank constraints for 3D displays (HR3D).
Raskar is a recipient of the TR100 award from Technology Review magazine (2004), Global Indus Technovator Award to recognize the top 20 Indian technology innovators world wide (2003), Alfred P. Sloan Research Fellowship (2009), and DARPA Young Faculty award (2010). Other awards include the Marr Prize honorable mention (2009), LAUNCH Health Innovation Award (2010), Vodafone Wireless Innovation Award (first place, 2011), and the Edison Award (2012). He holds over 40 U.S. patents and has received four Mitsubishi Electric Invention Awards. He is currently co-authoring a book on computational photography.
This course takes place on the MIT campus in Cambridge, Massachusetts. We can also offer this course for groups of employees at your location. Please contact the Short Programs office for further details.
Links & Resources
- 1,000,000,000,000 Frames/Second Photography - Ramesh Raskar
- TEDxBeaconStreet: How to Think Like an MIT Media Lab Inventor: Ramesh Raskar
- YouTube: Ramesh Raskar, MIT Media Lab
- TEDxBeaconStreet: How to Think Like an MIT Media Lab Inventor | Ramesh Raskar
- Compressive Displays
- Camera Culture | MIT Media Lab - CORNAR: Femtosecond Transient Imaging
- Camera Culture | MIT Media Lab - Femto-Photography: Visualizing Photons in Motion at a Trillion Frames per Second
- Livemint/Wall Street Journal article: Hardware is becoming the new software: Ramesh Raskar
- PDF (download) - Computational Photography: Epsilon to Coded Photography