PhD Student working with Antonio Torralba Computer Science and Artificial Intelligence Laboratory (CSAIL) Department of Electrical Engineering and Computer Science (EECS) Massachusetts Institute of Technology (MIT) Address: 32-D428, Stata Center, 32 Vassar Street, Cambridge, MA 02139 Email: Homepage: http://mit.edu/jxiao/

Interests

My research interests are in computer vision, with a focus on scene understanding. I am interested in scene and object recognition, data-driven approach and dataset issue, image matching and retrieval, visual feature and similarity, semantic segmentation and co-segmentation, 3D reconstruction and modeling, human visual perception, vision for graphics, learning for vision, etc.

Highlights

• Scene recognition can now be evaluated more realistically. [Project Webpage]
We construct a scene recognition dataset that exhaustively covers most of the places encountered by humans. Now we can test the performance of global features for scene classification realistically, evaluate numerous state-of-the-art algorithms, design the best known algorithm to establish new bounds of computer performance, and compare computer and human performance.
• Image memorability can be measured and predicted. [Project Webpage]
We show that we can measure and predict image memorability from a scientific point of view, which is usually difficult for subjective image attributes. We are able to conclude that the most memorable photos are those that contain people, followed by static indoor scenes and human-scale objects. On the other hand, landscape photos are mostly forgettable.
• Semantic-aware building reconstruction is robust enough to deploy massively. [Project Webpage]
By introducing some recognition into 3D reconstruction, we are able to build an automatic pipeline to reconstruct clean 3D mesh models for buildings robustly, and deploy it massively at city scale using images captured by a camera mounted on a car.
• Semantic segmentation for streetview images is mature enough for real world applications. [Project Webpage]
We show that semantic segmentation for street view images seems to be quite accurate and robust, and ready to use massively at city scale. This enables many potential applications for street-level scene understanding, such as autonomous driving.
• Seminal works on co-segmentation.
Our project on segmentation is one of the seminal works on co-segmentation, including multi-view image segmentation [ICCV07, ECCV08], 3D point cloud segmentation [ICCV07], semantic co-segmentation [ICCV09], segmentation with SFM [ICCV07, ECCV08, ICCV09].

Popular Downloads

• SUN Database for Scene Understanding (Dataset and Source Code)
• DrawMe: A light-weight Javascript library for line drawing on a picture

Publications

	K. Ehinger, J. Xiao, A. Torralba and A. Oliva Estimating scene typicality from human ratings and image features Proceedings of 33rd Annual Meeting of the Cognitive Science Society (CogSci2011) Oral Presentation Paper Slides
	P. Isola, J. Xiao, A. Torralba and A. Oliva What makes an image memorable? Proceedings of 24th IEEE Conference on Computer Vision and Pattern Recognition (CVPR2011) Paper Project Webpage (Dataset and Source Code) Appear on MIT News
	H. Zhang, J. Xiao, and L. Quan Supervised Label Transfer for Semantic Segmentation of Street Scenes Proceedings of the 11th European Conference on Computer Vision (ECCV2010)
	J. Xiao, J. Hays, K. Ehinger, A. Oliva, and A. Torralba SUN Database: Large-scale Scene Recognition from Abbey to Zoo Proceedings of 23rd IEEE Conference on Computer Vision and Pattern Recognition (CVPR2010) Paper Project Webpage with Dataset and Source Code Poster DrawMe
	P. Zhao, T. Fang, J. Xiao, H. Zhang, Q. Zhao, and L. Quan Rectilinear Parsing of Architecture in Urban Environment Proceedings of 23rd IEEE Conference on Computer Vision and Pattern Recognition (CVPR2010) Oral presentation
	J. Xiao, T. Fang, P. Zhao, M. Lhuillier, and L. Quan Image-based Street-side City Modeling ACM Transaction on Graphics (TOG), Volume 28, Number 5 Proceedings of ACM SIGGRAPH Asia 2009 Project Webpage High Resolution Paper Video of System Explanation Video of Results
	J. Xiao and L. Quan Multiple View Semantic Segmentation for Street View Images Proceedings of 12th IEEE International Conference on Computer Vision (ICCV2009) Project Webpage High Resolution Paper High Resolution Poster (126M) Low Resolution Poster Video in YouTube
	J. Xiao Image-based Building Modeling Thesis for Master of Philosophy in Computer Science and Engineering The Hong Kong University of Science and Technology M.Phil. Thesis (27MB)
	J. Xiao, T. Fang, P. Tan, P. Zhao, E. Ofek, and L. Quan Image-based Facade Modeling ACM Transaction on Graphics (TOG), Volume 27, Number 5 Proceedings of ACM SIGGRAPH Asia 2008 [Image is selected for Back Cover of ToG. Video is selected for Papers Preview.] Low Resolution Paper High Resolution Paper Video in YouTube Slides Image 1 Image 2
	P. Tan, T. Fang, J. Xiao, P. Zhao, and L. Quan Single Image Tree Modeling ACM Transaction on Graphics (TOG), Volume 27, Number 5 Proceedings of ACM SIGGRAPH Asia 2008 [Video is selected for Papers Preview.] Low Resolution Paper High Resolution Paper Video in YouTube
	J. Xiao, J. Chen, D.-Y. Yeung, and L. Quan Learning Two-view Stereo Matching Proceedings of the 10th European Conference on Computer Vision (ECCV2008) Springer Lecture Notes in Computer Science (LNCS), Pages 15-27 Oral Presentation Low Resolution Paper High Resolution Paper Slides
	J. Xiao, J. Chen, D.-Y. Yeung, and L. Quan Structuring Visual Words in 3D for Arbitrary-view Object Localization Proceedings of the 10th European Conference on Computer Vision (ECCV2008) Springer Lecture Notes in Computer Science (LNCS), Pages 725-737 Low Resolution Paper High Resolution Paper Slides Data Set (742MB)
	J. Xiao, J. Wang, P. Tan, and L. Quan Joint Affinity Propagation for Multiple View Segmentation Proceedings of 11th IEEE International Conference on Computer Vision (ICCV2007) Oral presentation Low Resolution Paper High Resolution Paper Slides
	J. Xiao Segmentation for Image-Based Modeling Final Year Thesis for Bachelor of Engineering in Computer Science The Hong Kong University of Science and Technology

Professional Activities

I am a program committee member / reviewer for the following conferences and journals:

• ACM SIGGRAPH 2012
• KSII Transactions on Internet and Information Systems
• Annual Conference of the European Association for Computer Graphics (Eurographics) 2012
• IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2012
• International Conference on Digital Information Management (ICDIM) 2011
• Pattern Recognition Letters
• The Imaging Science Journal
• IEEE International Conference on Computer Vision (ICCV) 2011
• IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2011
• ACM Multimedia 2010
• Neural Information Processing Systems (NIPS) 2010
• ACM Transactions on Graphics (ToG)
• ACM SIGGRAPH ASIA 2010
• European Conference on Computer Vision (ECCV) 2010
• ACM SIGGRAPH 2010
• IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2010
• International Journal of Computer Vision (IJCV)
• ACM SIGGRAPH ASIA 2009
• IEEE International Conference on Computer Vision (ICCV) 2009
• IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2009
• Machine Vision and Applications Journal (MVA) by Springer and IAPR
• IPSJ Transactions on Computer Vision and Applications
• IAPR International Conference on Pattern Recognition (ICPR) 2008
• Image and Vision Computing Journal (IVC) by Elsevier
• Asian Conference on Computer Vision (ACCV) 2007
• IEEE International Conference on Computer Vision (ICCV) 2007

Code and Resources