Foundations of Spatial Perception for Robotics: Hierarchical Representations and Real-Time Systems

The paper (Hughes et al., 2023) lays the foundations of hierarchical metric-semantic map representations for robotics. While hierarchical representations have been popular in robotics since its inception, the paper shows that these representations (i) lead to graphs with small treewidth (hence enabling efficient inference), (ii) allow combining traditional SLAM techniques with neuro-symbolic reasoning based on graph neural networks (GNN), and (iii) enable novel GNN architectures, namely Neural Trees, that are provably expressive and efficient. The paper also provides the first approach to build hierarchical representations, namely 3D scene graphs, of indoor environments in real-time. The approach combines 3D geometry, topology, and geometric deep learning, and reconstructs a hierarchical map of objects, places, rooms, buildings, and their relations.

The code has been released open-source at: https://github.com/MIT-SPARK/Hydra.

References

1. Hughes, N., Chang, Y., Hu, S., Talak, R., Abdulhai, R., Strader, J., & Carlone, L. (2023). Foundations of Spatial Perception for Robotics: Hierarchical Representations and Real-time Systems. ArXiv Preprint: 2305.07154.
   Bibtex

   @article{Hughes23arxiv-hydraFoundations,
     author = {Hughes, N. and Chang, Y. and Hu, S. and Talak, R. and Abdulhai, R. and Strader, J. and Carlone, L.},
     title = {Foundations of Spatial Perception for Robotics: Hierarchical Representations and Real-time Systems},
     journal = {arXiv preprint: 2305.07154},
     note = {\linkToPdf{https://arxiv.org/pdf/2305.07154.pdf},\linkToVideo{https://youtu.be/AEaBq2-FeY0}},
     pdf = {https://arxiv.org/pdf/2305.07154.pdf},
     funding = {DCIST, ONRRAIDER, AIA, LL4FRESCO, Lockheed, ARA2},
     year = {2023}
   }