Radu Alexandru Rosu (University of Bonn); Peer Schütt (University of Bonn); Jan Quenzel (University of Bonn); Sven Behnke (University of Bonn)
 |
 |
 |
 |
Deep convolutional neural networks (CNNs) have shown outstanding performance in the task of semantically segmenting images. Applying the same methods on 3D data still poses challenges due to the heavy memory requirements and the lack of structured data. Here, we propose LatticeNet, a novel approach for 3D semantic segmentation, which takes raw point clouds as input. A PointNet describes the local geometry which we embed into a sparse permutohedral lattice. The lattice allows for fast convolutions while keeping a low memory footprint. Further, we introduce DeformSlice, a novel learned data-dependent interpolation for projecting lattice features back onto the point cloud. We present results of 3D segmentation on multiple datasets where our method achieves state-of-the-art performance.
 | Start Time | End Time |
|---|---|---|
| 07/14 15:00 UTC | 07/14 17:00 UTC |
This paper extends the well noticed Splatnet [31] method with original contributions by introducing learned operations for splatting and slicing. It is shown in the paper that this extensions lead to significantly better performance in point cloud segmentation. The paper is clearly written and reads very well. The important concepts are illustrated as figures. The new method is thoroughly evaluated on relevant data sets and compared to other state-of-the-art methods. The proposed method achieves state-of-the-art performance at faster speeds and with a lower memory footprint. On the KITTI data set it outperforms all other state-of-the-art methods. The contribution is significant in improving 3D point cloud segmentation which is an important component in robotics. However, now real link to robotics is provided in the paper except by using the KITTI data set. I would be nice to have experiments directly pointing out the importance for robotics scenarios. Point cloud processing is I guess more interesting in the robotics community as there is a larger number of sensors producing point clouds. So, nevertheless such an algorithm is an important component for robotics.
The authors introduce four new operations on permutohedral lattices that are novel and significant for reasons explained in the following: Instead of summing the vertex features features are concatenated. A different distribuet function does the same for coordinates. The "distribute" significance is that it enables the learning of the splatting with a Pointnet. The second is a downsampler (strided convolution) that facilitates more context. The third is an upsampler (transposed convolution) and the fourth is a deformslicing which maps back to point clouds but bu subtracting from the maximum of the weighted values, an operation that guarantees equivariance to point permutations. The authors beautifully visualize the new splatter and slicer. Extensive experiments are performed in ShapeNet, ScanNet and Semantic KITTI. The experiments contain extensive ablation studies as well as comparison to other point cloud segmentation algorithms. In summary, the experimentation is thorough, the methodological contributions are novel and the problem is hard.
 |
 |
 |