NLP-papers

KO | EN

gitlite — search

#python #java #python3 #arduino #golang #machine-learning #rust #html #flask #javascript #seismology #nodejs

★ 17 Open GitHub ↗

Collecting interesting and Must-Read NLP Papers ( -2020)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h1 align="center">Must-Read NLP Papers ( -2020)</h1>

This repository contains important NLP papers (most), well-explained materials that everyone working in the field should know about and read.

I also implements several State-of-the-art NLP models. You can find that on my repo.

Check [Neural Network Language Model (NNLM)](https://github.com/pjlintw/NNLM)  
      [Attention Is All You Need (Transformer)](https://github.com/pjlintw/tf-transformer)

### Highlight of this repo:
* NLP: Pretrained Language Model, Machine Translation, Text Summarization
* CV: Image-to-image Translation
* Learning Algorithm: Meta Learning

### Index

* Deep Learning
  * [Overview](#Overview)

* NLP
  * [Clustering-&-Word-Embeddings](#Clustering-&-Word-Embeddings)
  * [Event Recognition](#Event-Recognition)
  * [Gated Recurrent Unit](#Gated-Recurrent-Unit)
  * [Language Modeling](#Language-Modeling)
  * [Image Captioning](#Image-Captioning)
  * [Image Recognition](#Image-Recognition)
  * [Image-to-Image Translation](#Image-to-Image-Translation)
  * [Machine Translation](#Machine-Translation)
  * [Meta-Learning](#Meta-Learning)
  * [Named Entity Recognition](#Named-Entity-Recognition)
  * [Probabilistic Graphical Models](#Probabilistic-Graphical-Models)
  * [Reinforcement Learning](#Reinforcement-Learning)
  * [Sentence Compression](#Sentence-Compression)
  * [Sequence Models](#Seqeunce-Models)

## Overview

* Yongjun Hong, et al. How Generative Adversarial Networks and Their Variants Work: An Overview. ACM 2019. [[ACM]](https://dl.acm.org/citation.cfm?id=3301282)

* Samuel L. Smith, et al. Don't Decay the Learning Rate, Increase the Batch Size. ICLR 2018. [[ICLR]](https://openreview.net/forum?id=B1Yy1BxCZ)

## Clustering & Word Embeddings

* Peter F Brown, et al. Class-Based n-gram Models of Natural Language. 1992. [[ACL Anthology]](https://www.aclweb.org/anthology/J92-4003/)

* Tomas Mikolov, et al. Efficient Estimation of Word Representations in Vector Space. 2013. [[ArXiv]](https://arxiv.org/abs/1301.3781)

* Tomas Mikolov, et al. Distributed Representations of Words and Phrases and their Compositionality. NIPS 2013. [[ArXiv]](https://arxiv.org/abs/1310.4546)

* Quoc V. Le and Tomas Mikolov. Distributed Representations of Sentences and Documents. 2014. [[ArXiv]](https://arxiv.org/abs/1405.4053)

* Jeffrey Pennington, et al. GloVe: Global Vectors for Word Representation. 2014. [[ACL Anthology]](https://www.aclweb.org/anthology/D14-1162/)

* Piotr Bojanowski, et al. Enriching Word Vectors with Subword Information. 2017. [[ACL Anthology]](https://www.aclweb.org/anthology/Q17-1010/)

## Cross-lingual Learning

* Junjie Hu, et al. XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization. 2020. [[ArXiv]](https://arxiv.org/abs/2003.11080)

## Evaluation Metric

* Kishore Papineni, et al. BLEU: a Method for Automatic Evaluation of Machine Translation. 2002 [[CiteSeer]](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.19.9416)

* Chin-Yew Lin. ROUGE: A Package for Automatic Evaluation of Summaries. ACL 2004. [[ACL Anthology](https://www.aclweb.org/anthology/W04-1013/)

## Event Recognition

* Amosse Edouard. Event Detection and Analysis On Short Text Messages. 2018. [[ResearchGate]](https://www.researchgate.net/publication/322486860_Event_detection_and_analysis_on_short_text_messages)

* Deepayan Chakrabarti and Kunal Punera. Event Summarization Using Tweets. ICWSM 2011. [[ResearchGate]](https://www.researchgate.net/publication/221298063_Event_Summarization_Using_Tweets)

* Maria Vargas-Vera and David Celjuska. Event Recognition on News Stories and Semi-Automatic Population of an Ontology. Web Intelligence 2004. [[ResearchGate]](https://www.researchgate.net/publication/42790057_Event_recognition_on_news_stories_and_semi-automatic_population_of_an_ontology)

## Gated Recurrent Unit

* Junyoung Chung, et al. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. CoRR 2014. [[ArXiv]](https://arxiv.org/abs/1412.3555)

## Image Captioning

* Steven J. Rennie, et al. Self-critical Sequence Training for Image Captioning. CVPR 2017. [[ArXiv]](https://arxiv.org/abs/1612.00563)

## Image Recognition

* An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. [[ICLR]](https://openreview.net/forum?id=YicbFdNTTy)

## Image-to-Image Translation

* Jun-Yan Zhu, et al. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. ICCV 2017. [[ArXiv]](https://arxiv.org/abs/1703.10593)

* Yunjey Choi et al. StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation. CVPR 2018. [[ArXiv]](https://arxiv.org/abs/1711.09020)

* Taesung Park, et al. Contrastive Learning for Unpaired Image-to-Image Translation. ECCV 2020. [[ArXiv]](https://arxiv.org/abs/2007.15651)

## Language Modeling

* Yoshua Bengio, et al. A Neural Probabilistic Language Model, J. of Machine Learning Research. 2003. [[ACM DL]](https://dl.acm.org/citation.cfm?id=944966)

* Rafal Jozefowicz, et al. Exploring the Limits of Language Modeling. 2016. [[ArXiv]](https://arxiv.org/abs/1602.02410)

* Matthew Peters, et al. Semi-supervised sequence tagging with bidirectional language models. ACL 2017. [[ArXiv]](https://arxiv.org/abs/1705.00108)

* Matthew Peters, et al. Deep contextualized word representations. NAACL 2018. [[ArXiv]](https://arxiv.org/abs/1802.05365)

* Jacob Devlin, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. 2018. [[ArXiv]](https://arxiv.org/abs/1810.04805)

* Jeremy Howard and Sebastian Ruder. Universal Language Model Fine-tuning for Text Classification. ACL 2018. [[ArXiv]](https://arxiv.org/abs/1801.06146)

* Alec Radford, et al. Improving Language Understanding by Generative Pre-Training. 2018. [[OpenAI](https://openai.com/blog/language-unsupervised/)]

* Alec Radford, et al. Language Models are Unsupervised Multitask Learners. 2019. [[OpenAI]](https://openai.com/blog/better-language-models/)]

* Zhenzhong Lan, et al. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. ICLR 2019. [[OpenReview]](https://openreview.net/forum?id=H1eA7AEtvS)

* Zihang Dai, et al. Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context. ACL 2019. [[ArXiv]](https://arxiv.org/abs/1901.02860)

* Zhilin Yang, et al. XLNet: Generalized Autoregressive Pretraining for Language Understanding. NIPS 2019. [[ArXiv]](https://arxiv.org/abs/1906.08237)

* Colin Raffel, et al. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. [[ArXiv]](https://arxiv.org/abs/1910.10683)

* Nikita Kitaev, et al. Reformer: The Efficient Transformer. [[ArXiv]](https://arxiv.org/abs/2001.04451)

* Kevin Clark, er al. ELECTRA_Pre-training Text Encoders as Discriminators Rather Than Generators. ICLR 2020. [[ArXiv]](https://arxiv.org/abs/2003.10555)

* Tom B. Brown, et al. Language Models are Few-Shot Learners. 2020. [[ArXiv]](https://arxiv.org/abs/2005.14165)

* Louis Martin, et al. CamemBERT: a Tasty French Language Model. ACL 2020. [[ArXiv]]()

## Machine Translation

* Dzmitry Bahdanau, et al. Neural Machine Translation by Jointly Learning to Align and Translate. ICLR 2015. [[ArXiv]](https://arxiv.org/abs/1409.0473)

* Minh-Thang Luong, et al. Effective Approaches to Attention-based Neural Machine Translation. EMNLP 2015. [[ArXiv]](https://arxiv.org/abs/1508.04025)

* Massive Exploration of Neural Machine Translation Architectures. ACL 2017 [[ArXiv]](https://arxiv.org/abs/1703.03906)

* Yun Chen, et al. A Teacher-Student Framework for Zero-Resource Neural Machine Translation. ACL 2017. [[ArXiv]](https://arxiv.org/abs/1705.00753)

* Ashish Vaswani, et al. Attention Is All You Need. 2017. [[ArXiv]](https://arxiv.org/abs/1706.03762)

* Guillaume Lample and Alexis Conneau. Cross-lingual Language Model Pretraining. 2019. [[ArXiv]](https://arxiv.org/abs/1901.07291)

* Alexis Conneau et al. Unsupervised Cross-lingual Representation Learning at Scale. ACL 2020. [[ArXiv]](https://arxiv.org/pdf/1911.02116.pdf)

* Christos Baziotis et al. Language Model Prior for Low-Resource Neural Machine Translation. EMNLP 2020. [[ArXiv]](https://arxiv.org/abs/2004.14928)

## Meta Learning

* Chelsea Finn, et al. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. ICML 2017. [[ArXiv]](https://arxiv.org/abs/1703.03400)

* Sachin Ravi and Hugo Larochelle. Optimization as a Model for Few-Shot Learning. ICLR 2017. [[OpenReview]](https://openreview.net/forum?id=rJY0-Kcll)

* Andrei A. Rusu, et al. Meta-Learning with Latent Embedding Optimization. ICLR 2019. [[ArXiv]](https://arxiv.org/abs/1807.05960)

* Aravind Rajeswaran et al. Meta-Learning with Implicit Gradients, et al.: Meta-Learning with Implicit Gradients. NIPS 2019. [[ArXiv]](https://arxiv.org/abs/1909.04630)

## Multi-Task Learning

* Victor Sanh, et al. A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks. AAAI 2019 [[ArXiv]](https://arxiv.org/abs/1811.06031)

## Named Entity Recognition

* Guillaume Lample, et al. Neural Architectures for Named Entity Recognition. ACL 2016. [[ArXiv]](https://arxiv.org/abs/1603.01360)

* Xuezhe Ma, Eduard Hovy. End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF. ACL 2016. [[ArXiv]](https://arxiv.org/abs/1603.01354)

* Matthew Peters, et al. Semi-Supervised Sequence Tagging With Bidirectional Language Models. ACL 2017. [[ArXiv]](https://arxiv.org/abs/1705.00108)

* Kevin Clark, et al. Semi-Supervised Sequence Modeling with Cross-View Training. EMNLP 2018. [[ArXiv]](https://arxiv.org/abs/1809.08370)

* Matthew Peters, et al. Deep Contextualized Word Representations. NAACL 2018. [[ArXiv]](https://arxiv.org/abs/1802.05365)

* Abbas Ghaddar and Philippe Lannglais. Robust Lexical Features for Improved Neural Network Named-Entity Recognition. COLING 2018. [[ACL Anthology]](https://www.aclweb.org/anthology/C18-1161/)

* Alan Akbik, et al. Contextual String Embeddings for Sequence Labeling. ACL 2018. [[ResearchGate]](https://www.researchgate.net/publication/328562439_Contextual_String_Embeddings_for_Sequence_Labeling)

* Alexei Baevski, et al. Cloze-driven Pretraining of Self-attention Networks. 2019. [[ArXiv]](https://arxiv.org/abs/1903.07785)

## Probabilistic Graphical Models

* John Lafferty, Andrew McCallum, Fernando C.N. Pereira: Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. ICML 2001. [[ACM DL]](https://dl.acm.org/citation.cfm?id=655813)

## Reinforcement Learning

* Kristopher D. Asis, et al. Multi_Step Reinforcement Learning_A Unifying Algorithm. AAAI 2018. [[ArXiv]](https://arxiv.org/abs/1703.01327)

## Sentence Compression

* Thibault Fevry and Jason Phang. Unsupervised Sentence Compression using Denoising Auto-Encoders. CoNLL 2018. [[ACL Anthology]](https://www.aclweb.org/anthology/K18-1040/)

## Sequence Models

* Ilya Sutskever, et al. Sequence to Sequence Learning with Neural Networks. 2014. [[ArXiv]](https://arxiv.org/abs/1409.3215)

## Text Classification

* Yoon Kim, et al. Convolutional Neural Networks for Sentence Classification. EMNLP 2014. [[ArXiv]](https://www.aclweb.org/anthology/D14-1181/)

* Xiang Zhang, et al. Character-Level Convolutional Networks For Text Classification. NIPS 2015. [[ArXiv]](https://arxiv.org/abs/1509.01626)

* Yoon Kim, et al. Character-Aware Neural Language Models. AAAI 2016. [[ArXiv]](https://arxiv.org/abs/1508.06615)

* Zichao Yang, et al. Hierarchical Attention Networks for Document Classification. NAACL 2016. [[ACL Anthology]](https://www.aclweb.org/anthology/N16-1174/)

* Alon Jacovi, et al. Understanding Convolutional Neural Networks for Text Classification. EMNLP 2018. [[ACL Anthology]](https://www.aclweb.org/anthology/W18-5408/)

## Text Generation

* Lantao Yu, et al. SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient. AAAI 2017. [[ArXiv]](https://arxiv.org/abs/1609.05473)

* William Fedus, et al. MaskGAN: Better Text Generation via Filling in the______. ICLR 2018. [[ArXiv]](https://arxiv.org/abs/1801.07736)

* Weili Nie, et al. RELGAN: RELATIONAL GENERATIVE ADVERSARIAL NETWORKS FOR TEXT GENERATION. ICLR 2019. [[ICLR]](https://openreview.net/forum?id=rJedV3R5tm)

* Kaitao Song, et al. MASS: Masked Sequence to Sequence Pre-Training for Langauge Generation. ICML 2019. [[ArXiv]](https://arxiv.org/abs/1905.02450)

* Mike Lewis, et al. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation. Translation, and Comprehension, ACL 2020. [[AxXiv]](https://arxiv.org/abs/1910.13461)

## Text Style Transfer

* Zichao Yang, et al. Unsupervised Text Style Transfer using Language Models as Discriminators. NIPS 2018. [[ArXiv]](https://arxiv.org/abs/1805.11749)

* Sandeep Subramanian, et al. Multiple-Attribute Text Style Transfer. ICLR 2019. [[ArXiv]](https://arxiv.org/abs/1811.00552)

## Text Summarization

* Romain Paulus, et al. A Deep Reinforced Model for Abstractive Summarization. ICLR 2018. [[ArXiv]](https://arxiv.org/abs/1705.04304)

* Angela Fan, et al. Controllable Abstractive Summarization. ACL 2018. [[ArXiv]](https://arxiv.org/abs/1711.05217)

* Yaushian Wang and Hung-Yi Lee. Learning to Encode Text as Human-Readable Summaries using Generative Adversarial Networks. EMNLP 2018. [[ACL Anthology]](https://www.aclweb.org/anthology/D18-1451/)

* Peter J. Liu, et al. SummAE: Zero-Shot Abstractive Text Summarization using Length-Agnostic Auto-Encoders. 2019 [[ArXiv]](https://arxiv.org/abs/1910.00998)

* Christos Baziotis, et al. SEQ^3: Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive Sentence Compression. NAACL 2019. [[ArXiv]](https://arxiv.org/abs/1904.03651)

* Jingqing Zhang, et al. PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization. ICML 2020. [[ArXiv]](https://arxiv.org/abs/1912.08777)

NLP-papers

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README