What Audio Models Learn Before Fine-Tuning

Speech encoders arrive at fine-tuning with tens of thousands of hours of audio structure already baked into their weights. This examination of internal representations contrasts self-supervised models like wav2vec 2.0, HuBERT, and w2v-BERT against weakly supervised systems like Whisper. It explores layer-by-layer probing of tongue position and word boundaries, the resurgence of speaker identity in deep layers, the loss of lexical tone during quantization, and the trade-off between flexible latent representations and out-of-the-box robustness. Key sources: • Baevski, Zhou, Mohamed et al., wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations, arXiv v3 (submitted 2020-06-20, revised 2020-10-22). Retrieved 2026-11-08. — https://arxiv.org/abs/2006.11477v3 • Hsu, Bolte, Tsai et al., HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units, arXiv v1 (submitted 2021-06-14). Retrieved 2026-11-08. — https://arxiv.org/abs/2106.07447v1 • Radford et al., Robust Speech Recognition via Large-Scale Weak Supervision (Whisper), arXiv, 28pp. Retrieved 2026-11-08. — https://arxiv.org/abs/2212.04356 Episode 6492 — myweirdprompts.com/n/6492 (permanent short link: myweirdprompts.com/507949) Listen on Android: MWP FM Player — myweirdprompts.com/apk