- SHELLS uses a shared DinoV2 backbone with LoRA adaptation to extract per-view feature maps from input images.
- A graph stage projectively samples features and processes them with an XCiT-based transformer to regress a coarse mesh.
- The coarse mesh is displaced along normals to create sampling shells for surface-aware feature sampling.
- A shared transformer aggregates shell-based features with a full-resolution tokenized template to predict a high-fidelity mesh.
- SHELLS can be applied frame-by-frame to dynamic facial performances, producing temporally smooth and expressive registrations.
- It handles occluded regions like the inner mouth cavity by correlating them with visible areas to regress all vertices holistically.
- Random camera dropout during training and mean-variance feature fusion make SHELLS robust to the number of input views at inference time.
- Reconstructions remain plausible and detailed even with as few as two input views and scale gracefully with more views.