Hugging Face's logo

Diffusers documentation

AllegroTransformer3DModel

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v0.42.1).
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

AllegroTransformer3DModel

Um modelo Diffusion Transformer para dados 3D da Allegro foi introduzido em Allegro: Open the Black Box of Commercial-Level Video Generation Model pela RhymesAI.

O modelo pode ser carregado com o seguinte trecho de código.

from diffusers import AllegroTransformer3DModel

transformer = AllegroTransformer3DModel.from_pretrained("rhymes-ai/Allegro", subfolder="transformer", dtype=torch.bfloat16).to("cuda")  # or "mps", "xpu", "cpu"

AllegroTransformer3DModel

classe diffusers.AllegroTransformer3DModel

< >

( patch_size: int = 2patch_size_t: int = 1num_attention_heads: int = 24attention_head_dim: int = 96in_channels: int = 4out_channels: int = 4num_layers: int = 32dropout: float = 0.0cross_attention_dim: int = 2304attention_bias: bool = Truesample_height: int = 90sample_width: int = 160sample_frames: int = 22activation_fn: str = 'gelu-approximate'norm_elementwise_affine: bool = Falsenorm_eps: float = 1e-06caption_channels: int = 4096interpolation_scale_h: float = 2.0interpolation_scale_w: float = 2.0interpolation_scale_t: float = 2.2 )

forward

< >

( hidden_states: Tensorencoder_hidden_states: Tensortimestep: LongTensorattention_mask: typing.Optional[torch.Tensor] = Noneencoder_attention_mask: typing.Optional[torch.Tensor] = Noneimage_rotary_emb: tuple[torch.Tensor, torch.Tensor] | None = Nonereturn_dict: bool = True )

Parâmetros

  • hidden_states (torch.Tensor de forma (batch_size, num_channels, num_frames, height, width)) — hidden_states de entrada.
  • encoder_hidden_states (torch.Tensor de forma (batch_size, sequence_len, embed_dims)) — Incorporações condicionais (incorporações calculadas a partir das condições de entrada, como prompts) a serem usadas.
  • timestep (torch.LongTensor) — Usado para indicar o passo de desruido.
  • attention_mask (torch.Tensor, opcional) — Máscara de autoatenção aplicada a hidden_states.
  • encoder_attention_mask (torch.Tensor, opcional) — Máscara de atenção cruzada aplicada a encoder_hidden_states.
  • image_rotary_emb (tuple de torch.Tensor, opcional) — Incorporações posicionais rotativas pré-computadas.
  • return_dict (bool, opcional, padrão True) — Se deve ou não retornar um ~models.transformer_2d.Transformer2DModelOutput em vez de uma tupla simples.

O método forward do AllegroTransformer3DModel.

Transformer2DModelOutput

classe diffusers.models.modeling_outputs.Transformer2DModelOutput

< >

( sample: torch.Tensor )

Parâmetros

  • sample (torch.Tensor de forma (batch_size, num_channels, height, width) ou (batch size, num_vector_embeds - 1, num_latent_pixels) se Transformer2DModel for discreto) — A saída dos estados ocultos condicionada à entrada encoder_hidden_states. Se discreto, retorna distribuições de probabilidade para os pixels latentes sem ruído.

A saída de Transformer2DModel.

Update on GitHub

←AceStepTransformer1DModel AnyFlowFARTransformer3DModel→