Nemotron-Labs-Unified-ASR-0.6b: Core ML + Core AI
Core ML and Core AI conversion of NVIDIA's nemotron-labs-unified-asr-0.6b: one Unified FastConformer-RNNT checkpoint (600M) for offline and streaming speech recognition in 25 European languages, with punctuation and capitalization. Unofficial port, not affiliated with or endorsed by NVIDIA or Apple.
Same architecture as parakeet-unified-en-0.6b plus a
language-ID prompt: both encoders take an int32 prompt_id and apply NVIDIA's prompt adapter inside the graph.
Prompt ids are in coreml/metadata.json["prompt_dictionary"] (e.g. en 0, de 9, fr 8, pl 17, auto 127).
Files
| Path | What |
|---|---|
coreml/nemotron_unified_encoder[_int8] |
offline encoder, 15 s window: mel (1, 128, 1501) + mel_length + prompt_id β encoder (1, 1024, 188) |
coreml/nemotron_unified_encoder_streaming_{L}_{C}_{R}[_int8] |
streaming encoder with the [left, chunk, right] attention context baked in (80 ms frames) |
coreml/nemotron_unified_decoder |
RNNT prediction net: one token + LSTM state (2 x 1 x 640) |
coreml/nemotron_unified_joint_decision_single_step |
joint + argmax / prob / top-64 for one encoder frame |
coreml/nemotron_unified_joint |
full-grid joint logits |
coreml/vocab.json, coreml/metadata.json |
4,095 SentencePiece pieces (blank = 4095); shapes, contexts, prompt dictionary |
coreai/*.aimodel |
Core AI (beta): offline encoder, 1.12 s streaming encoder, decoder, joint decision |
Each Core ML model ships as .mlpackage and compiled .mlmodelc. Encoders are fp16 (~1.1 GB) or int8 (_int8,
per-channel linear symmetric weights, ~570 MB). ML Programs for iOS 17 / macOS 14+. Mel is not included as a model:
compute NeMo's log-mel (128 bins, n_fft 512, hop 160, window 400, preemph 0.97) with per-feature normalization.
Streaming contexts (left / chunk / right, 80 ms encoder frames; latency = chunk + right):
| Bundle | Window | Latency |
|---|---|---|
70_13_13 |
7.68 s | 2.08 s |
70_7_7 |
6.72 s | 1.12 s |
70_2_5 |
6.16 s | 0.56 s |
Streaming is NVIDIA's buffered scheme: re-encode the [left | chunk | right] window each step, decode the chunk's
frames, hold back the right context; the RNNT decoder state carries across steps.
Parity
FLEURS, 5 clips per language (en, de, fr, it, pl, ru; 30 clips), M5 Pro. Small samples: treat WER as a sanity check.
- Offline fp16 vs NeMo (forward + greedy RNNT): 27/30 transcripts identical; WER equal in 5 of 6 languages (en 2.9% vs NeMo 3.8%).
- Streaming fp16 vs the same streaming loop with NeMo's PyTorch encoder/decoder/joint: 2.08 s 29/30, 1.12 s 30/30, 0.56 s 29/30 identical; WER identical in every language at every latency.
- int8: 1.12 s streaming 29/30 identical to PyTorch. WER equal to fp16 in 5 of 6 languages; French rose from 6.9% to 8.6% (streaming) and 7.8% to 9.5% (offline) on these 5 clips. fp16 is the safer default.
- Core AI offline encoder (GPU): cosine 1.0000 vs PyTorch, max |diff| < 0.01; 11/12 transcripts identical (one extra "and" in a long English sentence).
Speed (M5 Pro, median per call, coreml-cli)
| Encoder | GPU | Neural Engine | CPU only | ops on ANE |
|---|---|---|---|---|
| offline 15 s, fp16 | 15.6 ms | 26.0 ms | 85.5 ms | 99.9% |
| offline 15 s, int8 | 16.1 ms | 24.8 ms | 85.3 ms | 99.8% |
| streaming 1.12 s, fp16 | 11.6 ms | 11.8 ms | 43.6 ms | 99.7% |
| streaming 1.12 s, int8 | 12.2 ms | 11.4 ms | 43.4 ms | 99.6% |
A 0.56 s streaming step costs 11-12 ms of encoder time on the Neural Engine. The few CPU ops are int32 length/mask
math (0.03 ms).
How it was made
Traced from NeMo with the chunked attention mask baked into each streaming encoder (NVIDIA's buffered streaming,
att_context_size_as_chunk), the prompt adapter inside the encoder graphs, fp16 ML Programs via coremltools 9; int8
with per-channel linear symmetric weights; Core AI via coreai-torch. Same recipe as FluidInference's
parakeet-unified-en-0.6b port.
Use from Swift
FluidAudio (#1071): UnifiedAsrManager / StreamingUnifiedAsrManager
with UnifiedConfig(family: .nemotronMultilingual(.de)), or fluidaudiocli transcribe audio.wav --model-version nemotron-unified --unified-language de.
License
NVIDIA's model, released under the OpenMDW License 1.1 (LICENSE.md). This repository redistributes it converted to
Core ML and Core AI formats, including int8-quantized encoders.
- Downloads last month
- 228
Model tree for FluidInference/nemotron-labs-unified-asr-0.6b-coreml
Base model
nvidia/nemotron-labs-unified-asr-0.6b