Abstract
Reposing generic objects without the use of 3D models poses a significant challenge due to the absence of a standardized pose definition. Although previous works have targeted specific classes such as humans, a generic framework that is class agnostic is still missing in the literature. In response to this challenge, we introduce EOPose an end-to-end framework designed to address this problem and create a new dataset of paired objects using Objaverse. We utilize generalized pose correspondences of objects obtained using local-global correspondence matching algorithms to establish class-agnostic correlation. Afterward, we propose a novel architecture EOPose to generate the image in a new pose in 2 stages by i) warping the source image to move the point correspondences to their respective location and ii) employing a GAN-based architecture to inpaint the occluded information and harmonize the warped output. EOPose achieves state-of-the-art results as observed qualitatively and on quantitative benchmarks of image quality (PSNR, SSIM, and FID). The paper presents extensive comparisons with other existing solutions, including a detailed user study and ablation studies to gauge the effect of each of our contributions on the object-reposing problem.
Sarthak Mehrotra
Research Associate
Rishabh Jain
Research Scientist
Mayur Hemani
Senior ML Research Scientist
Balaji Krishnamurthy
Senior Principal Scientist and Senior Director
