Task Based Semantic Representation

Here is another great piece of research out of the lab of Luca Carlone at MIT involving hierarchical representation of segmented 3D maps where the representation is compressed via an information theoretic framework known as the Information Bottleneck Method. They are calling this approach Clio and the code has been open sourced on GitHub.

One of the more novel contributions of this paper is the task specific representations. The system is seeded with natural language tasks specifications and it uses these task goals while creating the hierarchical map. Using this in combination with the information bottleneck approach allows a type of map compression while preserving the fidelity needed to complete the given tasks. So 3D point cloud representations of objects not relevant to the task can be represented with lower fidelity and data relevant to the task can be stored in higher fidelity. This fidelity is essentially semantic objects that are retained in the map with corresponding point cloud data. As you can imagine much of this point cloud data is redundant, and much of it is irrelevant to the tasks at hand.

The final interesting method presented is a incremental agglomerative information bottleneck algorithm which allows clustering to be performed independently on connected components with no potential merges in the graph, therefore different sections of the graph can be optimized in parallel.

References

• Clio code @ Github
• Paper @ Arxiv