VACE: All-in-One Video Creation and Editing
Abstract
VACE, an all-in-one framework for video creation and editing, integrates multiple tasks within a unified model using a Video Condition Unit and Context Adapter for flexible and consistent video synthesis.
Diffusion Transformer has demonstrated powerful capability and scalability in generating high-quality images and videos. Further pursuing the unification of generation and editing tasks has yielded significant progress in the domain of image content creation. However, due to the intrinsic demands for consistency across both temporal and spatial dynamics, achieving a unified approach for video synthesis remains challenging. We introduce VACE, which enables users to perform Video tasks within an All-in-one framework for Creation and Editing. These tasks include reference-to-video generation, video-to-video editing, and masked video-to-video editing. Specifically, we effectively integrate the requirements of various tasks by organizing video task inputs, such as editing, reference, and masking, into a unified interface referred to as the Video Condition Unit (VCU). Furthermore, by utilizing a Context Adapter structure, we inject different task concepts into the model using formalized representations of temporal and spatial dimensions, allowing it to handle arbitrary video synthesis tasks flexibly. Extensive experiments demonstrate that the unified model of VACE achieves performance on par with task-specific models across various subtasks. Simultaneously, it enables diverse applications through versatile task combinations. Project page: https://ali-vilab.github.io/VACE-Page/.
Community
We are delighted to announce that the VACE-Preview version has been officially released (for more details, please refer to Huggingface and Modelscope). You can now download and enjoy this preview version. If you have any questions or feedback, please feel free to let us know to help us further improve.
make the child moving
make the child and ball moving
Good
ะกะพะทะดะฐะน ะผะฝะต ะฟัะพะผ ะธะท Dะฝั ัะตะบััะฐ: ะะฐะฟะธัะธ ัะฐะทะฒััะฝัััะน ะฟะพัั ะดะปั ัะพััะตัะตะน (ะะะพะฝัะฐะบัะต, Telegram) ะพ ะทะฐะฟััะบะต ะฝะพะฒะพะน ะผะฝะพะณะพััะฝะบัะธะพะฝะฐะปัะฝะพะน ะฟะปะฐััะพัะผั ะดะปั ะะฐะณะตััะฐะฝะฐ. ะกะพั ัะฐะฝัะน ะดััะถะตะปัะฑะฝัะน, ะฒะดะพั ะฝะพะฒะปัััะธะน ัะพะฝ, ะดะพะฑะฐะฒั ะฟัะธะทัะฒ ะบ ะดะตะนััะฒะธั. ะะฑััะผ: 120โ150 ัะปะพะฒ.
ะกะพะทะดะฐะน ะบะพัะพัะบะธะน ะฐะฝะพะฝั (ะดะพ 60 ัะปะพะฒ) ะดะปั ัะฐัััะปะบะธ ะฒ ะผะตััะตะฝะดะถะตัะฐั ะพ ััะฐััะต ะฟะปะฐััะพัะผั. ะกะดะตะปะฐะน ะฐะบัะตะฝั ะฝะฐ ัะดะพะฑััะฒะต ะธ ะตะดะธะฝััะฒะต ัะตัะฒะธัะฐ. ะะบะปััะธ ะฟัะธะทัะฒ ะฟะพะดะฟะธัะฐัััั ะฝะฐ ะบะฐะฝะฐะปั.
ะะฐะฟะธัะธ ัะตะบัั ะดะปั ะฒะธะดะตะพโะฐะฝะพะฝัะฐ ะฝะฐ YouTube (1โ1,5 ะผะธะฝััั). ะะฟะธัะธ ะบะปััะตะฒัะต ะฟัะตะธะผััะตััะฒะฐ ะฟะปะฐััะพัะผั, ะฟะตัะตัะธัะปะธ ัะพัะณะพะฒัะต ัะพัะผะฐัั, ะดะพะฑะฐะฒั ัะผะพัะธะพะฝะฐะปัะฝัะน ะฟัะธะทัะฒ ะบ ััะฐััะธั.
ะกะพััะฐะฒั ัะฟะธัะพะบ ะธะท 7โ10 ะฒะพะฟัะพัะพะฒ ะดะปั ะพะฟัะพัะฐ ะฐัะดะธัะพัะธะธ ะฒ ะบะพะผะผะตะฝัะฐัะธัั : ััะพ ะฟะพะปัะทะพะฒะฐัะตะปะธ ั ะพััั ะฒะธะดะตัั ะฝะฐ ะฟะปะฐััะพัะผะต, ะบะฐะบะธะต ััะฝะบัะธะธ ะดะปั ะฝะธั ะฒะฐะถะฝั. ะคะพัะผัะปะธััะน ะบัะฐัะบะพ ะธ ะฟะพะฝััะฝะพ.
ะะฐะฟะธัะธ ัะตะบัั ะดะปั ะฑะฐะฝะฝะตัะฐ (ะดะพ 20 ัะปะพะฒ) ะพ ะทะฐะฟััะบะต ะฟะปะฐััะพัะผั. ะะพะปะถะตะฝ ะทะฐัะตะฟะธัั ะฒะฝะธะผะฐะฝะธะต, ะฟะตัะตะดะฐัั ัััั ะธ ะฒัะทะฒะฐัั ะถะตะปะฐะฝะธะต ัะทะฝะฐัั ะฑะพะปััะต.
ะกะพะทะดะฐะน ะฟะพััโะพะฑัััะฝะตะฝะธะต (100โ120 ัะปะพะฒ) ะดะปั ะฝะพะฒะธัะบะพะฒ: ััะพ ััะพ ะทะฐ ะฟะปะฐััะพัะผะฐ, ะบะฐะบ ะพะฝะฐ ัะฐะฑะพัะฐะตั, ะทะฐัะตะผ ะฝัะถะฝะฐ. ะัะฟะพะปัะทัะน ะฟัะพัััะต ะฐะฝะฐะปะพะณะธะธ (ะฝะฐะฟัะธะผะตั, ยซัะพะฒัะตะผะตะฝะฝัะน ััะฝะพะบ 2.0ยป).
ะะฐะฟะธัะธ ะผะพัะธะฒะฐัะธะพะฝะฝะพะต ะพะฑัะฐัะตะฝะธะต ะบ ะฟัะตะดะฟัะธะฝะธะผะฐัะตะปัะผ ะะฐะณะตััะฐะฝะฐ (80โ100 ัะปะพะฒ) ั ะฟัะธะทัะฒะพะผ ะฟัะธัะพะตะดะธะฝะธัััั ะบ ะฟะปะฐััะพัะผะต. ะะพะดัะตัะบะฝะธ ะฒัะณะพะดั ะดะปั ะฑะธะทะฝะตัะฐ: ัะฐััะธัะตะฝะธะต ะฐัะดะธัะพัะธะธ, ะฝะพะฒัะต ะฒะพะทะผะพะถะฝะพััะธ ะฟัะพะดะฐะถ.
ะกะพััะฐะฒั ัะตะบัั ะดะปั emailโัะฐัััะปะบะธ ะฟะพะดะฟะธััะธะบะฐะผ ะพ ะทะฐะฟััะบะต ะฟะปะฐััะพัะผั. ะะบะปััะธ: ะฟัะธะฒะตัััะฒะธะต, ะบัะฐัะบะพะต ะพะฟะธัะฐะฝะธะต, ะบะปััะตะฒัะต ัะธัะบะธ, ัััะปะบะธ ะฝะฐ ัะพััะตัะธ, ะฟัะธะทัะฒ ะบ ะดะตะนััะฒะธั. ะะฑััะผ: 150โ180 ัะปะพะฒ.
ะะฐะฟะธัะธ ะบะพัะพัะบะธะน ะดะธะฐะปะพะณ (5โ7 ัะตะฟะปะธะบ) ะผะตะถะดั ะฟะพะปัะทะพะฒะฐัะตะปะตะผ ะธ ะฟะพะดะดะตัะถะบะพะน ะฟะปะฐััะพัะผั. ะะพะปัะทะพะฒะฐัะตะปั ัะฟัะฐัะธะฒะฐะตั, ะบะฐะบ ะฝะฐัะฐัั ะฟะพะปัะทะพะฒะฐัััั ัะตัะฒะธัะพะผ, ะบะฐะบะธะต ัะพัะผะฐัั ะดะพัััะฟะฝั. ะัะฒะตัั ะฟะพะฝััะฝะพ ะธ ะดััะถะตะปัะฑะฝะพ.
ะกะพะทะดะฐะน ัะตะบัั ะดะปั ะฟัะพะผะพโััะพัะธั ะฒ Telegram (3โ4 ะบะฐััะพัะบะธ, ะดะพ 40 ัะปะพะฒ ะฝะฐ ะบะฐะถะดัั). ะะฐ ะฟะตัะฒะพะน โ ะฐะฝะพะฝั ะทะฐะฟััะบะฐ, ะฝะฐ ะฒัะพัะพะน โ ะบะปััะตะฒัะต ะฟัะตะธะผััะตััะฒะฐ, ะฝะฐ ััะตััะตะน โ ะฟัะธะทัะฒ ะฟะพะดะฟะธัะฐัััั ะธ ััะฐััะฒะพะฒะฐัั.
Get this paper in your agent:
hf papers read 2503.07598 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash