wav2vec 2.0 Base 960h CTC for Core ML (forced alignment)
torchaudio's WAV2VEC2_ASR_BASE_960H (wav2vec 2.0 Base fine-tuned on 960 hours of LibriSpeech,
from fairseq) converted to Core ML for word-level forced alignment in
SwiftHeardLyrics (CTCForcedAligner): it times the words of sung lyrics to a median
start error of 38-41 ms on JamendoLyrics, against 86 ms with Apple Speech's timings refined by the
vocal envelope.
- Input
audio: 16 kHz mono, [1, 1600...480000] samples (0.1-30 s). - Output
log_probs: [1, frames, 29], one frame per 20 ms, over torchaudio's labels (blank-, word separator|, A-Z and the apostrophe). - float16, except the attention, which computes in float32: in float16 on the CPU it overflows on sung vocals. SwiftHeardLyrics runs it on the CPU (180 MB, up to 0.75 GB of memory).
Converted with coremltools by scripts/export_wav2vec2_ctc.py in SwiftHeardLyrics; the files and
their SHA-256 are in manifest.json.
License
MIT, as the original weights (LICENSE), which torchaudio redistributes from fairseq.