Abstract
Constructing a robust and discriminative local descriptor for 3D shapes is a key component of many computer vision applications. Although existing learning-based approaches can achieve good performance in specific benchmarks, they usually fail to learn sufficient information from shapes with different types and structures (e.g., spatial resolution, connectivity, and transformation).
To solve this issue, we present a more discriminative local descriptor for deformable 3D shapes with incompatible structures. Based on spectral embedding using the Laplace-Beltrami framework on the surface, we construct a novel local spectral feature that exhibits high resilience to changes in mesh resolution, triangulation, and transformation. The multiscale local spectral features around each vertex are then encoded into a geometry image called vertex spectral image in a compact manner. Such vertex spectral images can be efficiently trained to learn local descriptors using a triplet neural network. Then, we present a new benchmark dataset for training and evaluation by extending the widely used FAUST dataset. We utilize a remeshing approach to generate modified shapes with different structures.Furthermore, we evaluate the proposed approach thoroughly and conduct an in-depth comparison to demonstrate that our approach outperforms recent state-of-the-art methods on this benchmark.
Overview
Given an input shape (left), we first compute a 16-dimensional local spectral feature for each vertex of the mesh to construct feature models, and the scalar value of each dimension is color-coded in the middle images. For a point of interest, we then generate vertex spectral images by packing the 16-dimensional LPS in a compact manner. Finally, we send the vertex spectral images to the convolutional neural network to obtain our descriptor.
Dataset
Sample shapes from our dataset. The three models on the left have different resolutions (6890, 8K, and 15K) and triangulations. We show the models with different scalings (middle) and rotations (right). [Download Dataset]
Experimental Results
Impact of resolution and triangulation on the proposed LPS feature. Left: PCA results on the LPS feature. Right: shape matching results of multi-resolution using the LPS feature.
Impact of resolution and triangulation on our newly-learned descriptor. Left: CMC curve. Right: correspondence quality of the geodesic error.
Bibtex
@InProceedings{Wang_2019_CVPR,
author = {Wang, Yiqun and Guo, Jianwei and Yan, Dong-Ming and Wang, Kai and Zhang, Xiaopeng},
title = {A Robust Local Spectral Descriptor for Matching Non-Rigid Shapes With Incompatible Shape Structures},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
pages={6231-6240},
year = {2019} }