We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:


References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computer Vision and Pattern Recognition

Title: NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation

Abstract: 3D pose estimation is a challenging but important task in computer vision. In this work, we show that standard deep learning approaches to 3D pose estimation are not robust when objects are partially occluded or viewed from a previously unseen pose. Inspired by the robustness of generative vision models to partial occlusion, we propose to integrate deep neural networks with 3D generative representations of objects into a unified neural architecture that we term NeMo. In particular, NeMo learns a generative model of neural feature activations at each vertex on a dense 3D mesh. Using differentiable rendering we estimate the 3D object pose by minimizing the reconstruction error between NeMo and the feature representation of the target image. To avoid local optima in the reconstruction loss, we train the feature extractor to maximize the distance between the individual feature representations on the mesh using contrastive learning. Our extensive experiments on PASCAL3D+, occluded-PASCAL3D+ and ObjectNet3D show that NeMo is much more robust to partial occlusion and unseen pose compared to standard deep networks, while retaining competitive performance on regular data. Interestingly, our experiments also show that NeMo performs reasonably well even when the mesh representation only crudely approximates the true object geometry with a cuboid, hence revealing that the detailed 3D geometry is not needed for accurate 3D pose estimation. The code is publicly available at this https URL
Comments: Accepted by ICLR 2021. Code is publicly available
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Cite as: arXiv:2101.12378 [cs.CV]
  (or arXiv:2101.12378v3 [cs.CV] for this version)

Submission history

From: Angtian Wang [view email]
[v1] Fri, 29 Jan 2021 03:23:12 GMT (8136kb,D)
[v2] Tue, 2 Feb 2021 17:59:10 GMT (8137kb,D)
[v3] Thu, 4 Feb 2021 05:01:51 GMT (8136kb,D)

Link back to: arXiv, form interface, contact.