Our method extends naturally to 4D by adopting the idea in 3D, that is, to have a raw inference from the seed image, and correct its details with stable diffusion... frames, we apply multiple existing video editing methods to the input video... video will be combined with the animated novel object to build up a raw inpainted video that is temporally consistent...
To determine the motion for the new object, since there exists no stable method for generating a background-consistent movement for a generated foreground object... CoTracker enables us to track self-defined key points on the original video precisely... projected to the generated object with consistent background information...
Collection
[
|
...
]