Distillation

Two-stage textual knowledge distillation from text BERT to speech model via utterance-level representation matching in pretraining and predicted logit matching in finetuning - ___[ICASSP 2021](https://2021.ieeeicassp.org)___