RefinedFields

Abstract

Planar scene representations have recently witnessed increased interests for modeling scenes from images, as their lightweight planar structure enables compatibility with image-based models. Notably, K-Planes have gained particular attention as they extend planar scene representations to support in-the-wild scenes, in addition to object-level scenes. However, their visual quality has recently lagged behind that of state-of-the-art techniques. To reduce this gap, we propose RefinedFields, a method that leverages pre-trained networks to refine K-Planes scene representations via optimization guidance using an alternating training procedure. We carry out extensive experiments and verify the merit of our method on synthetic data and real tourism photo collections. RefinedFields enhances rendered scenes with richer details and improves upon its base representation on the task of novel view synthesis. Our code is publicly available as open-source.

Method

We learn a scene through two alternating stages. Scene fitting optimizes our K-Planes representation to reproduce the images in the training set, as traditionally done in neural rendering techniques. Scene refining finetunes a pre-trained prior and infers a new refined K-Planes representation, which will subsequently be corrected by scene fitting.

Results

In-the-wild renders

Synthetic renders

BibTeX


      @article{kassab2023refinedfields,
        title={RefinedFields: Radiance Fields Refinement for Unconstrained Scenes},
        author={Kassab, Karim and Schnepf, Antoine and Franceschi, Jean-Yves and Caraffa, Laurent and Mary, Jeremie and Gouet-Brunet, Val{\'e}rie},
        journal={arXiv preprint arXiv:2312.00639},
        year={2023}
      }