Model Overview

  • Model Architecture: MuseSparkForConditionalGeneration
    • Input: Text, Image, Video, Audio
    • Output: Text
  • Supported Hardware Microarchitecture: AMD MI300/MI325/MI350/MI355
  • Operating System(s): Linux
  • Inference Engine: SGLang / vLLM
  • Model Optimizer: AMD-Quark(v0.12)
    • Quantized layers: MoE experts and self-attention, all layers.
    • Weight quantization:
      • FP8 (E4M3), Per-Channel, Static
    • Activation quantization:
      • FP8 (E4M3), Per-Token, Dynamic

This checkpoint quantizes the routed MoE experts and self-attention projections to PTPC FP8 (E4M3) across all layers and keeps vision, audio, routing, and lm_head in bf16.

Model Quantization

The model was quantized from meta-models/Muse-Spark-Open-816B-A42B using AMD-Quark. The weights and activations are quantized to FP8.

Quantization scripts:

from quark.torch import ModelQuantizer
from quark.torch.quantization.config.template import LLMTemplate
from quark.torch.quantization.weight_convert import SplitFusedExperts, WeightConverter

LLMTemplate.register_template(
    LLMTemplate(
        model_type="muse_spark",
        kv_layers_name=["*k_proj", "*v_proj"],
        q_layer_name="*q_proj",
        exclude_layers_name=[
            "lm_head",
            "*vision*",
            "*audio*",
            "model.language_model.layers.*.mlp.pre_expert_proj",
            "model.language_model.layers.*.mlp.post_expert_proj",
            "model.language_model.layers.*.mlp.gate",
        ],
        f2f_weight_converters=[
            WeightConverter(
                "gate_up_proj",
                ["gate_proj.weight", "up_proj.weight"],
                operations=[SplitFusedExperts(split_axis=0)],
            ),
            WeightConverter(
                "down_proj",
                ["down_proj.weight"],
                operations=[SplitFusedExperts(split_axis=0)],
            ),
        ],
    )
)

template = LLMTemplate.get("muse_spark")
quant_config = template.get_config(scheme="ptpc_fp8")

quantizer = ModelQuantizer(quant_config)
quantizer.direct_quantize_checkpoint(
    pretrained_model_path="meta-models/Muse-Spark-Open-816B-A42B",
    save_path="amd/Muse-Spark-Open-816B-A42B-Quark-PTPC-FP8",
    weight_converters=template.f2f_weight_converters,
)

Deployment

Todo

License

Modifications Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.

Downloads last month
1,100
Safetensors
Model size
822B params
Tensor type
F32
路
BF16
路
F8_E4M3
路
Inference Providers NEW
This model isn't deployed by any Inference Provider. 馃檵 Ask for provider support

Model tree for amd/Muse-Spark-Open-816B-A42B-Quark-PTPC-FP8

Quantized
(25)
this model
Quantizations
1 model