Paper-Knowledge_Distillation-Adversarial_Training-NLP

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

Paper-Knowledge_Distillation-Adversarial_Training-NLP

★ 10 Open GitHub ↗

some paper of Knowledge Distillation and Adversarial Training about NLP

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Knowledge Distillation 和 Adversarial Training 与 NLP 相关的论文收集
分享一下自己看论文收集总结的一些paper. 持续更新~   欢迎大家来一起更新呀

### Adversarial Training (Adversarial Examples)
- [Explaining and harnessing adversarial examples](https://arxiv.org/pdf/1412.6572.pdf) 2014 开山之作
- [Distributional smoothing with virtual adversarial training](https://arxiv.org/pdf/1507.00677.pdf) 2015
- [Adversarial training methods for semi-supervised text classification](https://arxiv.org/pdf/1605.07725.pdf) (FGM) 2016 [code1](https://github.com/tensorflow/models/tree/master/research/adversarial_text)  [code2](https://github.com/TobiasLee/Text-Classification/blob/master/models/adversarial_abblstm.py)
- [Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence with Adversarial Examples](https://arxiv.org/pdf/1803.01128.pdf)  2018
- [Interpretable adversarial perturbation in input embedding space for text](https://arxiv.org/pdf/1805.02917)  2018
- [Generating Natural Language Adversarial Examples](https://arxiv.org/pdf/1804.07998.pdf)  2018 
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://arxiv.org/pdf/1706.06083) （PGD） ICLR 2018
- [Freelb: enhanced adversarial training for language understanding](https://arxiv.org/pdf/1909.11764.pdf) 2019
- [A Survey: Towards a Robust Deep Neural Network in Text](https://arxiv.org/pdf/1902.07285/) 2019
-  [Adversarial attacks on deep learning models in natural language: A survey](http://web.science.mq.edu.au/~qsheng/papers/TIST-revisedversion-2019.pdf) 2019
-  [Technical report on Conversational Question Answering](https://arxiv.org/pdf/1909.10772) 2019 问答任务中基于RoBERTa融合对抗样本以及知识蒸馏技术
-  [SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization](https://arxiv.org/pdf/1911.03437.pdf)  2019

### Knowledge Distillation（知识蒸馏）
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://arxiv.org/pdf/1910.01108.pdf) 2019

- [Tinybert: distilling bert for natural language understanding](https://arxiv.org/pdf/1909.10351.pdf) 2019

- [Technical report on Conversational Question Answering](https://arxiv.org/pdf/1909.10772) 2019

- [Distilling Task-Specific Knowledge from BERT into Simple Neural Networks](https://arxiv.org/pdf/1903.12136.pdf) 2019

- [Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding](https://arxiv.org/pdf/1904.09482v1.pdf) 2019   [code](https://github.com/namisan/mt-dnn)

#### 其他资源

- [awesome-knowledge-distillation](https://github.com/dkozlov/awesome-knowledge-distillation )
- [Must-read Papers on Textual Adversarial Attack and Defense (TAAD)](https://github.com/thunlp/TAADpapers)
- [Adversarial_Learning_Paper](https://github.com/Guo-Yunzhe/Adversarial_Learning_Paper)
- [Adversarial-Machine-Learning](https://github.com/tanjuntao/Adversarial-Machine-Learning)
- [PLMpapers](https://github.com/thunlp/PLMpapers)
- [A PyTorch-based knowledge distillation toolkit for natural language processing](https://github.com/airaria/TextBrewer)
- [一文搞懂NLP中的对抗训练](https://zhuanlan.zhihu.com/p/103593948)
- [【炼丹技巧】功守道：NLP中的对抗训练 + PyTorch实现](https://zhuanlan.zhihu.com/p/91269728)

Paper-Knowledge_Distillation-Adversarial_Training-NLP

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README