Paper reading | point voxel CNN for efficient 3D deep learning

The paper ：Point-Voxel CNN for Efficient 3D Deep Learning
Code ：code
source ： NeurIPS2019
Company ： Massachusetts institute of technology, Shanghai Jiaotong University

Preface

At present, when I am making point cloud, most of my attention is focused on the direct processing method on point , Some of the voxel Methods . This article is about voxel-based and point-based The advantages and disadvantages of the method are analyzed , And combine the advantages of both and put forward a combined method . The article idea It is relatively simple and easy to understand , The code is not very complex . But the article is very logical , I even think this article is a summary of the problem , Previous overview , It is more wonderful than the later experiment .

The first part ： introduction

This paper introduces two common methods in the field of point cloud ： Conventional voxel-based and New type point-based

Conventional voxel-based Method features ：
Grid ： The mesh is finely divided with high accuracy , But memory consumption （ Triple growth ）; Coarse mesh generation with low accuracy （ There are some points divided into the same grid ）, But you don't need to consume too much memory
Overall advantage ： Be regular , convenient 3D Convolution
Overall shortcomings ： Waste space , There is redundant information （ Sparse part ） And the loss of information （ Dense part ）

New type point-based Method features
Convolute the existing points , Not many volumes, not many volumes , There is no waste of space and loss of information
shortcoming ： Need to find a neighbor , That is, random memory access is required random memory access

Here the author explains the advantages and disadvantages of these two methods , The implementation of the underlying code is analyzed （ Although I haven't understood it yet , But compare the results of time and memory occupation directly with other papers , Here the article explains the general principle , This part is very persuasive ）
chart a The comparison is the consumption of energy and bandwidth by occupying too much memory . Arithmetic operations （ Add and multiply , Such as a Left ） Low energy consumption , High bandwidth , Access to memory （ Such as a Right ） High energy consumption , Small bandwidth utilization , Therefore, try to reduce the consumption of memory
chart b Compare the time consumption of random memory access , There is a bus conflict , It will waste waiting time （ I don't quite understand this one , So a little ）

The second part ： Previous work

A little

The third part ： motivation

voxel Method's impact on memory usage and accuracy trande-off chart ：
point Method ： Irregular memory access and the proportion of dynamic kernel overhead and other operations
Here is our method >90% All the time is spent on actual calculation , Other methods waste computing time 30-50% Random memory access and 2-50% On the dynamic kernel

The fourth part ： Method

Two branches ：

• voxel Branch with low precision voxel mesh （ Low memory usage ）
• point Branch with the simplest MLP（pointNet）, Do not find adjacent points and convolute with kernel points

The key ：

• Devoxelization Part uses trilinear interpolation , That is, the distance weighting of voxels around the point
• fuse Feature addition is performed directly on the part of

The fifth part ： experiment

Object Part Segmentation

Indoor Scene Segmentation
3D Object Detection
Performance is maintained and even improved , But the running time and memory consumption have been greatly improved ！