Name
Abstract | ||
|---|---|---|
AbstractNeural representations have emerged as a new paradigm for applications in rendering, imaging, geometric modeling, and simulation. Compared to traditional representations such as meshes, point clouds, or volumes they can be flexibly incorporated into differentiable learning-based pipelines. While recent improvements to neural representations now make it possible to represent signals with fine details at moderate resolutions (e.g., for images and 3D shapes), adequately representing large-scale or complex scenes has proven a challenge. Current neural representations fail to accurately represent images at resolutions greater than a megapixel or 3D scenes with more than a few hundred thousand polygons. Here, we introduce a new hybrid implicit-explicit network architecture and training strategy that adaptively allocates resources during training and inference based on the local complexity of a signal of interest. Our approach uses a multiscale block-coordinate decomposition, similar to a quadtree or octree, that is optimized during training. The network architecture operates in two stages: using the bulk of the network parameters, a coordinate encoder generates a feature grid in a single forward pass. Then, hundreds or thousands of samples within each block can be efficiently evaluated using a lightweight feature decoder. With this hybrid implicit-explicit network architecture, we demonstrate the first experiments that fit gigapixel images to nearly 40 dB peak signal-to-noise ratio. Notably this represents an increase in scale of over 1000X compared to the resolution of previously demonstrated image-fitting experiments. Moreover, our approach is able to represent 3D shapes significantly faster and better than previous techniques; it reduces training times from days to hours or minutes and memory requirements by over an order of magnitude. |
Year | DOI | Venue |
|---|---|---|
2021 | 10.1145/3450626.3459785 | ACM Transactions on Graphics |
Keywords | DocType | Volume |
Neural Signal Representation | Conference | 40 |
Issue | ISSN | Citations |
4 | 0730-0301 | 1 |
PageRank | References | Authors |
0.38 | 0 | 6 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Julien N P Martel | 1 | 28 | 7.46 |
| David B. Lindell | 2 | 46 | 7.19 |
| Connor Z. Lin | 3 | 1 | 0.72 |
| Eric R. Chan | 4 | 1 | 1.06 |
| Marco Monteiro | 5 | 1 | 0.72 |
| Gordon Wetzstein | 6 | 945 | 72.47 |