aws-neuron/torch-neuronx-model-support
Viewer • Updated • 89.6k • 20.6k • 2
AI & ML interests
None defined yet.
Recent Activity
View all activity
dacorvo
updated a
dataset 2 days ago
dacorvo
updated a
Space 3 days ago
dacorvo
updated a
bucket 3 days ago
Jingya
updated a
Space 6 days ago
Jingya
updated a
dataset 6 days ago
dacorvo
published a
Space 18 days ago
dacorvo
published a
dataset 18 days ago
dacorvo
published a
bucket 18 days ago
Post
2,527
432 GB of ultra-fast HBM4 and up to 23.3 TB/s of memory bandwidth on a single GPU 🤯.
Two weeks ago, we got early access to AMD's new Instinct MI455X, and our first goal was simple: make sure 🤗 Transformers works on day one.
Over the past few weeks, we worked closely with the AMD team to validate the platform, enable Flash Attention, add torchcodec support for multimodal models, and resolve issues uncovered during testing.
The result:
✅ 99.5% success rate across our 24 core Transformers model architectures - already on par with our daily CI on previous AMD and NVIDIA platforms.
The hardware is just as exciting. With 432 GB of HBM per GPU, our early capacity experiments showed more than 3× the concurrent long-context requests compared to MI300, thanks to the much larger KV cache capacity.
A huge thanks to the AMD team for the early access and the great collaboration!
Read the full blog 👇
https://huggingface.co/blog/badaoui/transformers-on-amd-mi455
Two weeks ago, we got early access to AMD's new Instinct MI455X, and our first goal was simple: make sure 🤗 Transformers works on day one.
Over the past few weeks, we worked closely with the AMD team to validate the platform, enable Flash Attention, add torchcodec support for multimodal models, and resolve issues uncovered during testing.
The result:
✅ 99.5% success rate across our 24 core Transformers model architectures - already on par with our daily CI on previous AMD and NVIDIA platforms.
The hardware is just as exciting. With 432 GB of HBM per GPU, our early capacity experiments showed more than 3× the concurrent long-context requests compared to MI300, thanks to the much larger KV cache capacity.
A huge thanks to the AMD team for the early access and the great collaboration!
Read the full blog 👇
https://huggingface.co/blog/badaoui/transformers-on-amd-mi455
- 1 reply
jeffboudier
posted an update 4 months ago
Post
812
@alvarobartt published a step-by-step guide to deploy zai-org/GLM-5.2 on AMD GPUs, using newly released features in Microsoft Foundry. The FP8 model fits on a single node of MI300X GPUs, which cuts the bill in half vs. H100.
https://alvarobartt.com/goal-glm-5.2-on-foundry/
https://alvarobartt.com/goal-glm-5.2-on-foundry/
Post
580
Building high-performance, reproducible kernels for AMD ROCm just got a lot easier.
I've put together a guide on building, testing, and sharing ROCm-compatible kernels using the Hugging Face
Learn how to:
- Use Nix for reproducible builds
- Integrate kernels as native PyTorch operators
- Share your kernels on the Hub for anyone to use with
We use the 🏆 award-winning RadeonFlow GEMM kernel as a practical example.
📜 Check out the full guide here : https://huggingface.co/blog/build-rocm-kernels
I've put together a guide on building, testing, and sharing ROCm-compatible kernels using the Hugging Face
kernel-builder and kernels libraries; so you can focus on optimizing performance rather than spending time on setup.Learn how to:
- Use Nix for reproducible builds
- Integrate kernels as native PyTorch operators
- Share your kernels on the Hub for anyone to use with
kernels.get_kernel()We use the 🏆 award-winning RadeonFlow GEMM kernel as a practical example.
📜 Check out the full guide here : https://huggingface.co/blog/build-rocm-kernels
Post
3,182
🚀 Big news for AI builders!
We’re thrilled to announce that the Qwen3-VL family of vision-language models is now available on Azure AI Foundry, thanks to our collaboration with Microsoft.
We bring open-source innovation to enterprise-grade AI infrastructure, making it easier than ever for enterprise to deploy and scale the latest and greatest from models from hugging Face securely within Azure.
🔍 Highlights:
- Deploy Qwen3-VL instantly via managed endpoints
- Built-in governance, telemetry, and lifecycle management
- True multimodal reasoning — vision, language, and code understanding
- State-of-the-art performance, outperforming closed-source models like Gemini 2.5 Pro and GPT-5
- Available in both *Instruct* and *Thinking* modes, across 24 model sizes
👉 Get started today: search for Qwen3-VL in the Hugging Face Collection on Azure AI Foundry.
We’re thrilled to announce that the Qwen3-VL family of vision-language models is now available on Azure AI Foundry, thanks to our collaboration with Microsoft.
We bring open-source innovation to enterprise-grade AI infrastructure, making it easier than ever for enterprise to deploy and scale the latest and greatest from models from hugging Face securely within Azure.
🔍 Highlights:
- Deploy Qwen3-VL instantly via managed endpoints
- Built-in governance, telemetry, and lifecycle management
- True multimodal reasoning — vision, language, and code understanding
- State-of-the-art performance, outperforming closed-source models like Gemini 2.5 Pro and GPT-5
- Available in both *Instruct* and *Thinking* modes, across 24 model sizes
👉 Get started today: search for Qwen3-VL in the Hugging Face Collection on Azure AI Foundry.
- 1 reply
Post
897
What’s your biggest headache deploying Hugging Face models to the cloud—and how can we fix it for you?
- 8 replies
Post
539
Post
4,034
🤝 Collaborating with AMD to ensure Hugging Face Transformers runs smoothly on AMD GPUs!
We run daily CI on AMD MI325 to track the health of the most important model architectures and we’ve just made our internal dashboard public.
By making this easily accessible, we hope to spark community contributions and improve support for everyone!
We run daily CI on AMD MI325 to track the health of the most important model architectures and we’ve just made our internal dashboard public.
By making this easily accessible, we hope to spark community contributions and improve support for everyone!
- 2 replies
Post
450
🚀 Optimum libraries keep growing, and Optimum v2 is just around the corner!
I recently added ONNX export support for a bunch of new models in the optimum-onnx library, including: DeepSeek-V3, Cohere, Nemotron, Arcee, StableLM … and more!
⚡ With ONNX export, you can run your favorite models faster and more efficiently across different hardware backends, making deployment and experimentation much smoother.
💡 Have a model you’d love to see supported? Contributions are super welcome — let’s make Optimum even better together!
#ONNX #Optimum #HuggingFace #OpenSource #AI
I recently added ONNX export support for a bunch of new models in the optimum-onnx library, including: DeepSeek-V3, Cohere, Nemotron, Arcee, StableLM … and more!
⚡ With ONNX export, you can run your favorite models faster and more efficiently across different hardware backends, making deployment and experimentation much smoother.
💡 Have a model you’d love to see supported? Contributions are super welcome — let’s make Optimum even better together!
#ONNX #Optimum #HuggingFace #OpenSource #AI
jeffboudier
posted an update about 1 year ago
Post
3,483
Quick 30s demo of the new Hub > Azure AI integration to deploy HF models in your own Azure account. Now with Py and CLI!
GG @alvarobartt @kramp @pagezyhf
GG @alvarobartt @kramp @pagezyhf
Post
3,249
We've improved the Deploy button on Hugging Face model pages for Microsoft Azure
1/ no more long waits before seeing model support status
2/ ready-to-use CLI and Python snippets
3/ redirection to Azure AI Foundry rather than Azure ML
✋ if you see any bugs or have feedback, open an issue on our repo:
https://github.com/huggingface/Microsoft-Azure
1/ no more long waits before seeing model support status
2/ ready-to-use CLI and Python snippets
3/ redirection to Azure AI Foundry rather than Azure ML
✋ if you see any bugs or have feedback, open an issue on our repo:
https://github.com/huggingface/Microsoft-Azure
Post
3,277
Is there a "one-size-fits-all" recipe for quantizing Large Language Models? 🤔
As part of my ongoing work in mixed-precision quantization, I've been exploring this question by measuring layer-by-layer sensitivity. The goal is to see if we can find universal rules for which layers can be quantized aggressively without impacting performance.The results are fascinating and reveal two key insights:
1️⃣ Sensitivity profiles are like architectural "fingerprints." Models from the same family share strikingly similar sensitivity patterns. As you can see in the charts below for the Gemma and SmolLM families, the ranking and relative sensitivity of the layers remain remarkably consistent. This suggests that the underlying architecture is a primary driver of a model's quantization behavior.
2️⃣ A "universal" mixed-precision quantization strategy is challenging. While models within a family are similar, these "fingerprints" change dramatically when comparing different architectures like LLaMA, Qwen, and StableLM. This highlights the difficulty in creating a generalized mixed-precision configuration that works optimally across all model families.
However, there is one near-universal truth we uncovered: the mlp.down_proj layer consistently emerges as one of the most sensitive components across all models studied.
This finding strongly resonates with the work in "The Super Weight in Large Language Models" (by Mengxia Yu et al.). The paper identifies that functionally critical parameters, or "super weights," are concentrated in these down_proj layers. Our empirical results provide clear validation for this theory, showing these layers are highly intolerant to precision loss.
In short, while every architecture has a unique sensitivity profile, a fingerprint shaped not only by its core design but also by its specific training dataset and optimization approach, some components remain universally critical!
What are your thoughts?
As part of my ongoing work in mixed-precision quantization, I've been exploring this question by measuring layer-by-layer sensitivity. The goal is to see if we can find universal rules for which layers can be quantized aggressively without impacting performance.The results are fascinating and reveal two key insights:
1️⃣ Sensitivity profiles are like architectural "fingerprints." Models from the same family share strikingly similar sensitivity patterns. As you can see in the charts below for the Gemma and SmolLM families, the ranking and relative sensitivity of the layers remain remarkably consistent. This suggests that the underlying architecture is a primary driver of a model's quantization behavior.
2️⃣ A "universal" mixed-precision quantization strategy is challenging. While models within a family are similar, these "fingerprints" change dramatically when comparing different architectures like LLaMA, Qwen, and StableLM. This highlights the difficulty in creating a generalized mixed-precision configuration that works optimally across all model families.
However, there is one near-universal truth we uncovered: the mlp.down_proj layer consistently emerges as one of the most sensitive components across all models studied.
This finding strongly resonates with the work in "The Super Weight in Large Language Models" (by Mengxia Yu et al.). The paper identifies that functionally critical parameters, or "super weights," are concentrated in these down_proj layers. Our empirical results provide clear validation for this theory, showing these layers are highly intolerant to precision loss.
In short, while every architecture has a unique sensitivity profile, a fingerprint shaped not only by its core design but also by its specific training dataset and optimization approach, some components remain universally critical!
What are your thoughts?
- 4 replies
Post
2,219
Deploy GPT OSS models with Hugging Face on Azure AI!
We’re thrilled to enable OpenAI GPT OSS models on Azure AI Model Catalog for Azure users to try the model securely the day of its release.
In our official launch blogpost, there’s a section on how to deploy the model to your Azure AI Hub. Get started today!
https://huggingface.co/blog/welcome-openai-gpt-oss#azure
We’re thrilled to enable OpenAI GPT OSS models on Azure AI Model Catalog for Azure users to try the model securely the day of its release.
In our official launch blogpost, there’s a section on how to deploy the model to your Azure AI Hub. Get started today!
https://huggingface.co/blog/welcome-openai-gpt-oss#azure