Speech

Two-stage Textual Knowledge Distillation for End-to-End Spoken Language Understanding

Two-stage textual knowledge distillation from text BERT to speech model via utterance-level representation matching in pretraining and predicted logit matching in finetuning - ___[ICASSP 2021](https://2021.ieeeicassp.org)___

ST-BERT: Cross-modal Language Model Pre-training For End-to-end Spoken Language Understanding

Cross-modal pretraining of speech and text for end-to-end spoken language understanding via masked and conditioned cross-modal language modeling - ___[ICASSP 2021](https://2021.ieeeicassp.org)___