This is a unified architecture for video generation tasks that can respond to a variety of inputs: text only, image only, and both text and image. We've split the process into two steps: first, generating images conditioned on a text prompt, and then generating video conditioned on both the text and the generated image. This "factorized" or split approach to video generation lets us train video generation models efficiently."
"In human evaluations, our video generations are strongly preferred compared to prior work - in fact, this model was preferred over [Meta's previous generative video project] by 96% of respondents based on quality and by 85% of respondents based on faithfulness to the text prompt. Finally, the same model can "animate" user-provided images based on a text prompt where it once again sets a new state-of-the-art outperforming prior work by a significant margin. "
[
Collection
]
[
|
...
]