Daniel Rasmussen commited on
Commit
4f3ec18
·
1 Parent(s): ada265b

Update model weights

Browse files
.eval_results/open_asr_leaderboard.yaml ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: hf-audio/open-asr-leaderboard
3
+ task_id: mean_wer
4
+ value: 8.91
5
+ date: '2026-04-14'
6
+ source:
7
+ url: https://huggingface.co/hf-audio
8
+ name: open-asr-leaderboard
9
+ user: hf-audio
10
+
11
+ - dataset:
12
+ id: hf-audio/open-asr-leaderboard
13
+ task_id: ami_wer
14
+ value: 15.87
15
+ date: '2026-04-14'
16
+ source:
17
+ url: https://huggingface.co/hf-audio
18
+ name: open-asr-leaderboard
19
+ user: hf-audio
20
+
21
+ - dataset:
22
+ id: hf-audio/open-asr-leaderboard
23
+ task_id: earnings22_wer
24
+ value: 12.09
25
+ date: '2026-04-14'
26
+ source:
27
+ url: https://huggingface.co/hf-audio
28
+ name: open-asr-leaderboard
29
+ user: hf-audio
30
+
31
+ - dataset:
32
+ id: hf-audio/open-asr-leaderboard
33
+ task_id: gigaspeech_wer
34
+ value: 12.56
35
+ date: '2026-04-14'
36
+ source:
37
+ url: https://huggingface.co/hf-audio
38
+ name: open-asr-leaderboard
39
+ user: hf-audio
40
+
41
+ - dataset:
42
+ id: hf-audio/open-asr-leaderboard
43
+ task_id: librispeech_clean_wer
44
+ value: 3.84
45
+ date: '2026-04-14'
46
+ source:
47
+ url: https://huggingface.co/hf-audio
48
+ name: open-asr-leaderboard
49
+ user: hf-audio
50
+
51
+ - dataset:
52
+ id: hf-audio/open-asr-leaderboard
53
+ task_id: librispeech_other_wer
54
+ value: 9.35
55
+ date: '2026-04-14'
56
+ source:
57
+ url: https://huggingface.co/hf-audio
58
+ name: open-asr-leaderboard
59
+ user: hf-audio
60
+
61
+ - dataset:
62
+ id: hf-audio/open-asr-leaderboard
63
+ task_id: spgispeech_wer
64
+ value: 3.10
65
+ date: '2026-04-14'
66
+ source:
67
+ url: https://huggingface.co/hf-audio
68
+ name: open-asr-leaderboard
69
+ user: hf-audio
70
+
71
+ - dataset:
72
+ id: hf-audio/open-asr-leaderboard
73
+ task_id: tedlium_wer
74
+ value: 5.71
75
+ date: '2026-04-14'
76
+ source:
77
+ url: https://huggingface.co/hf-audio
78
+ name: open-asr-leaderboard
79
+ user: hf-audio
80
+
81
+ - dataset:
82
+ id: hf-audio/open-asr-leaderboard
83
+ task_id: voxpopuli_wer
84
+ value: 8.73
85
+ date: '2026-04-14'
86
+ source:
87
+ url: https://huggingface.co/hf-audio
88
+ name: open-asr-leaderboard
89
+ user: hf-audio
.gitattributes CHANGED
@@ -19,6 +19,7 @@
19
  *.pb filter=lfs diff=lfs merge=lfs -text
20
  *.pickle filter=lfs diff=lfs merge=lfs -text
21
  *.pkl filter=lfs diff=lfs merge=lfs -text
 
22
  *.pt filter=lfs diff=lfs merge=lfs -text
23
  *.pth filter=lfs diff=lfs merge=lfs -text
24
  *.rar filter=lfs diff=lfs merge=lfs -text
 
19
  *.pb filter=lfs diff=lfs merge=lfs -text
20
  *.pickle filter=lfs diff=lfs merge=lfs -text
21
  *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.png filter=lfs diff=lfs merge=lfs -text
23
  *.pt filter=lfs diff=lfs merge=lfs -text
24
  *.pth filter=lfs diff=lfs merge=lfs -text
25
  *.rar filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -13,7 +13,7 @@ library_name: transformers
13
 
14
  # ABR's niagara-38m-batch.en SSM
15
 
16
- The niagara-38m-batch.en model is a [State Space Model](https://huggingface.co/blog/lbourdois/get-on-the-ssm-train) (SSM) with attention that performs automatic speech recognition (ASR), trained and released by [Applied Brain Research](http://www.appliedbrainresearch.com/) (ABR). This model contains \~38m parameters, transcribes speech in English, and was trained on 15k hours of speech data (competitors use about 200k hours). This is a batch model variant designed for direct comparison with other models on the Open ASR Leaderboard. SSMs are an ideal solution for streaming contexts, but to provide a more direct comparison with other models in the leaderboard this model is not streaming. A live streaming variant of this english ASR model with sub-120ms latency and features such as punctuation, capitalization and custom vocabulary along with other similar models for other languages is available under commercial license from [ABR](https://www.appliedbrainresearch.com/state-space-models).
17
 
18
  ## Why “Niagara”?
19
 
@@ -86,7 +86,7 @@ This model provides transcribed speech as a string for a given audio sample.
86
 
87
  The SSM ASR model is trained for English speech recognition and transcribes audio into text. ABR developed the model to demonstrate small, efficient, real-time, accurate speech recognition can be performed with SSMs and run on low cost third party hardware. The model uses 38m parameters. The version posted here is a non-causal model (like most on the leaderboard), to give fair performance comparisons. It is also available as a cascaded model, meaning it produces extremely low latency (\<120ms from first audio to token) causal outputs as well as 1s latency non-causal outputs. These two streams can be merged to have a quick response that updates after 1s with the final result.
88
 
89
- Release Date: February 27, 2026
90
 
91
  ### Model Type
92
 
@@ -102,7 +102,7 @@ The model was trained on datasets partially listed below. It uses MFCC preproces
102
 
103
  ### Datasets
104
 
105
- The training datasets contain \~15k hours of English speech, including:
106
 
107
  - LibriSpeech (clean)
108
  - VoxPopuli
@@ -119,43 +119,43 @@ The training datasets contain \~15k hours of English speech, including:
119
  Our evaluations show that the SSM ASR demonstrates better performance on benchmark datasets compared to other similarly sized or often larger ASR models. The posted model transcribes audio to lower case english with no punctuation. Performance is reported in terms of Word Error Rate (WER%) for the non-causal model.
120
 
121
  ```
122
- Average WER = 8.84 %
123
  ```
124
 
125
- | Dataset | WER |
126
- | :---- | :---- |
127
- | AMI-IHM | 16.81% |
128
- | Earnings-22 | 10.99% |
129
- | GigaSpeech | 13.73% |
130
- | LibriSpeech (clean) | 3.35% |
131
- | LibriSpeech (other) | 8.80% |
132
- | SPGISpeech | 2.97% |
133
- | TED-LIUM | 5.83% |
134
- | VoxPopuli | 8.24% |
135
 
136
  ## Related models
137
 
138
  | Model | Parameters | Average WER |
139
- |:----------------------------------------------------------------------------------| :---- | :---- |
140
- | [abr-ai/niagara-19m-batch.en](https://huggingface.co/abr-ai/niagara-19m-batch.en) | 19M | TBD |
141
- | [abr-ai/niagara-38m-batch.en](https://huggingface.co/abr-ai/niagara-38m-batch.en) | 38M | TBD |
142
 
143
  ## Competitive Comparisons
144
 
145
- ![][image1]
146
  Figure 1: ASR Leaderboard Data accuracy vs parameter count. Transformer-based models marked in grey. SSMs improve the pareto front of transformers.
147
 
148
  Models with fewer than 100M parameters, sorted by accuracy.
149
 
150
  | Model Name | Parameters (M) | Accuracy (%) |
151
- |:----------------------------| ----- | ----- |
152
- | abr-ai/niagara-38m-batch.en | 38 | TBD |
153
- | moonshine-base | 61 | 90.00 |
154
- | whisper-base | 74 | 89.70 |
155
- | abr-ai/niagara-19m-batch.en | 19 | TBD |
156
- | moonshine-streaming-tiny | 34 | 88.00 |
157
- | moonshine-tiny | 27 | 87.30 |
158
- | whisper-tiny | 39 | 87.30 |
159
 
160
  ## Limitations
161
 
 
13
 
14
  # ABR's niagara-38m-batch.en SSM
15
 
16
+ The niagara-38m-batch.en model is a [State Space Model](https://huggingface.co/blog/lbourdois/get-on-the-ssm-train) (SSM) with attention that performs automatic speech recognition (ASR), trained and released by [Applied Brain Research](http://www.appliedbrainresearch.com/) (ABR). This model contains \~38m parameters, transcribes speech in English, and was trained on \~50k hours of speech data (competitors use about 200k hours). This is a batch model variant designed for direct comparison with other models on the Open ASR Leaderboard. SSMs are an ideal solution for streaming contexts, but to provide a more direct comparison with other models in the leaderboard this model is not streaming. A live streaming variant of this english ASR model with sub-120ms latency and features such as punctuation, capitalization and custom vocabulary along with other similar models for other languages is available under commercial license from [ABR](https://www.appliedbrainresearch.com/state-space-models).
17
 
18
  ## Why “Niagara”?
19
 
 
86
 
87
  The SSM ASR model is trained for English speech recognition and transcribes audio into text. ABR developed the model to demonstrate small, efficient, real-time, accurate speech recognition can be performed with SSMs and run on low cost third party hardware. The model uses 38m parameters. The version posted here is a non-causal model (like most on the leaderboard), to give fair performance comparisons. It is also available as a cascaded model, meaning it produces extremely low latency (\<120ms from first audio to token) causal outputs as well as 1s latency non-causal outputs. These two streams can be merged to have a quick response that updates after 1s with the final result.
88
 
89
+ Release Date: April 15, 2026
90
 
91
  ### Model Type
92
 
 
102
 
103
  ### Datasets
104
 
105
+ The training datasets contain \~50k hours of English speech, including:
106
 
107
  - LibriSpeech (clean)
108
  - VoxPopuli
 
119
  Our evaluations show that the SSM ASR demonstrates better performance on benchmark datasets compared to other similarly sized or often larger ASR models. The posted model transcribes audio to lower case english with no punctuation. Performance is reported in terms of Word Error Rate (WER%) for the non-causal model.
120
 
121
  ```
122
+ Average WER = 8.91 %
123
  ```
124
 
125
+ | Dataset | WER |
126
+ | :---- |:-------|
127
+ | AMI-IHM | 15.87% |
128
+ | Earnings-22 | 12.09% |
129
+ | GigaSpeech | 12.56% |
130
+ | LibriSpeech (clean) | 3.84% |
131
+ | LibriSpeech (other) | 9.35% |
132
+ | SPGISpeech | 3.10% |
133
+ | TED-LIUM | 5.71% |
134
+ | VoxPopuli | 8.73% |
135
 
136
  ## Related models
137
 
138
  | Model | Parameters | Average WER |
139
+ |:----------------------------------------------------------------------------------| :---- |:------------|
140
+ | [abr-ai/niagara-19m-batch.en](https://huggingface.co/abr-ai/niagara-19m-batch.en) | 19M | 10.47 |
141
+ | [abr-ai/niagara-38m-batch.en](https://huggingface.co/abr-ai/niagara-38m-batch.en) | 38M | 8.91 |
142
 
143
  ## Competitive Comparisons
144
 
145
+ ![ASR Leaderboard Data accuracy vs parameter count](plot.png)
146
  Figure 1: ASR Leaderboard Data accuracy vs parameter count. Transformer-based models marked in grey. SSMs improve the pareto front of transformers.
147
 
148
  Models with fewer than 100M parameters, sorted by accuracy.
149
 
150
  | Model Name | Parameters (M) | Accuracy (%) |
151
+ |:----------------------------| ----- |--------------|
152
+ | abr-ai/niagara-38m-batch.en | 38 | 91.09 |
153
+ | moonshine-base | 61 | 90.00 |
154
+ | whisper-base | 74 | 89.70 |
155
+ | abr-ai/niagara-19m-batch.en | 19 | 89.53 |
156
+ | moonshine-streaming-tiny | 34 | 88.00 |
157
+ | moonshine-tiny | 27 | 87.30 |
158
+ | whisper-tiny | 39 | 87.30 |
159
 
160
  ## Limitations
161
 
niagara-38m-batch.en-cpu.pt CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e8c0e30ff39eaa6384c30a72e5fb31995cebefec3cd83bd8bea8290904f9e438
3
- size 155975431
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ce29f582f057f4fd6d8375b4f093f71f616fe8b7e2b95efcd6480cc87cf8b0a7
3
+ size 155999747
niagara-38m-batch.en-cuda.pt CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ffd05a0776d95a1719183a921c509953ff7bc51f31341f7392cf9665ca02d71d
3
- size 155978382
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ca7e45f5acac5bf0ae9b9a4737999ab037a474444bbcdb70368deb038ba80dc6
3
+ size 156002698
plot.png ADDED

Git LFS Details

  • SHA256: 2d2e2976e74a1c6353d26c6c3defc43100022a29757a6af15c8e547b550eb390
  • Pointer size: 131 Bytes
  • Size of remote file: 280 kB
sentencepiece_1024.model CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c8be70290b623a76004f077f56c90d14f9eb098b813b3997a6a460dcaa2e5ce8
3
- size 254559
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:80bdd5b93f5688ea8e5a659d165f087877c5868497a0b818dfe11a919770010b
3
+ size 254572