CoreML Speech Models
Collection
Speech AI models for Apple Neural Engine via CoreML. iOS/macOS ready. ASR, TTS, VAD, diarization. • 47 items • Updated • 7
Nemotron 3.5 Core ML encoder: FP16 mask experiment
Current main contains a second encoder experiment for speech-swift issue 503. It changes internal audio-length and attention-cache mask calculations from Int32 to FP16 (16-bit floating point). Runtime integer operations drop from 25 to four input/output boundary casts. The public integer feature interface and all 296 published encoder palette lookup tables and index arrays are preserved.
Affected A16/iOS 27.2 validation is pending. The published model and this candidate both transcribe correctly on the available iPhone 16 Pro.
The repository name comes from the first experiment. The earlier iOS 17 candidate changed the deployment target while retaining the integer graph. The reporter confirmed that it still failed. That revision remains available. The current encoder targets iOS 18 and changes the mask calculation types.
| Property | Value |
|---|---|
| Architecture | Cache-aware FastConformer encoder and RNN-T decoder |
| Parameters | 600 million |
| Format | Three compiled Core ML .mlmodelc models |
| Encoder weights | Published 8-bit palette indices and FP16 lookup tables |
| Decoder and joint | Published FP16 models, unchanged |
| Runtime file size | 612.4 MiB |
| Audio | 16 kHz mono |
| Streaming chunk / lookahead | 320 ms / 240 ms |
| Minimum deployment target | iOS 18 |
| Baseline revision | 447095fe87b480b5e6a15367f135303d479de8ac |
| Status | Experimental; test on the affected A16 device before adopting |
Files
| File | Size | Purpose |
|---|---|---|
| encoder.mlmodelc/ | 565.3 MiB | Encoder with FP16 length/mask calculations |
| decoder.mlmodelc/ | 28.5 MiB | Unchanged published decoder |
| joint.mlmodelc/ | 18.0 MiB | Unchanged published joint network |
| config.json | 589 B | Unchanged streaming configuration |
| vocab.json, languages.json, tokenizer.model | Unchanged | Vocabulary, language prompts and tokenizer |
| experiment.json | JSON | Source revisions, graph change and artifact hashes |
| TESTING.md | Guide | Revision-pinned device comparison |
| testing/LoadProbe.swift | Swift source | Standalone Core ML load/reload and compute-plan report |
| testing/test_audio.wav | 625 KiB | Same 20-second mono 16 kHz speech sample as the earlier test |
| testing/fixture-provenance.json | JSON | Speech sample provenance and hashes |
| testing/validation.json | JSON | Results for this FP16 mask candidate |
Validation
- 17 exporter unit checks passed.
- All six CPU encoder outputs matched the published encoder bit for bit across 201 calls: speech, silence, partial chunks, all audio lengths 0 through 32 at cache lengths 0, 1, 55 and 56, and two extra language-prompt slots.
- On M5 Pro/macOS 26.6.2, the English speech sample produced identical transcription tokens with both bundles in CPU-only and CPU+ANE modes.
- On iPhone 16 Pro/A18 Pro/iOS 26.6, all four native Core ML speech comparisons also produced the same transcript and 29 reference tokens.
These tests used standalone Core ML streaming encoders and greedy RNN-T decoders. The iPhone tests used frozen mel features from the speech sample. The Swift SDK test suite was not rebuilt for this candidate. One English sample and synthetic language prompts do not establish multilingual accuracy.
Observed speech timing
Each row is one pass of the same 20-second sample with CPU+ANE requested. Model loading is excluded. Real-time factor is processing time divided by audio duration; smaller values mean faster processing.
| Hardware / OS | Bundle | Processing time | Real-time factor |
|---|---|---|---|
| M5 Pro / macOS 26.6.2 | Published | 0.690 s | 0.035 |
| M5 Pro / macOS 26.6.2 | FP16 mask candidate | 0.839 s | 0.042 |
| A18 Pro / iOS 26.6 | Published | 1.185 s | 0.059 |
| A18 Pro / iOS 26.6 | FP16 mask candidate | 1.529 s | 0.076 |
The candidate was slower in these single passes. This experiment tests the ANE compiler compatibility hypothesis. A performance improvement has not been established. The published CPU-only phone run logged a BNNS compilation storage error, so its timing is excluded from comparisons. Additional app-restart cache timings on the phone were not completed.
See testing/validation.json for measurements and limitations. Successful
tests on M5 Pro and A18 do not establish compatibility with A16/iOS 27.2.
Usage
Resolve and record the test revision before downloading. Keep the published
baseline in a separate directory. Follow TESTING.md for the device checks.
import coremltools as ct
from huggingface_hub import HfApi, snapshot_download
repo = "aufklarer/Nemotron-3.5-ASR-Streaming-0.6B-CoreML-INT8-iOS17-Encoder-Test"
revision = HfApi().model_info(repo).sha
bundle = snapshot_download(repo, revision=revision, local_dir="nemotron-fp16-mask-test")
print("Candidate revision:", revision)
encoder = ct.models.CompiledMLModel(
f"{bundle}/encoder.mlmodelc",
compute_units=ct.ComputeUnit.CPU_AND_NE,
)
On Apple Silicon macOS, compile the standalone load probe and obtain a JSON report:
swiftc -O -swift-version 6 -target arm64-apple-macosx15.0 -parse-as-library -D LOAD_PROBE_CLI nemotron-fp16-mask-test/testing/LoadProbe.swift -o /tmp/nemotron-load-probe
/tmp/nemotron-load-probe nemotron-fp16-mask-test ane candidate-ane.json
Source and license
Upstream: NVIDIA Nemotron 3.5 ASR Streaming 0.6B, revision f3d333391852ba876df169dcc9ba902d25b6ab0b. The model retains the upstream OpenMDW 1.1 license.
Baseline: published Core ML INT8 bundle, revision 447095fe87b480b5e6a15367f135303d479de8ac.
Links
- Downloads last month
- 31
Model tree for aufklarer/Nemotron-3.5-ASR-Streaming-0.6B-CoreML-INT8-iOS17-Encoder-Test
Base model
nvidia/nemotron-3.5-asr-streaming-0.6b