mongolian-text-classification

KO | EN

gitlite — search

#react #golang #android #rust #deep-learning #linux #docker #macos #java #kubernetes #pytorch #security #openai #open-source

mongolian-text-classification

(★ 32)

Cyrillic Mongolian text classification with tensorflow 2, and also some fine-tuning on TugsTugi's Mongolian BERT model and other NLP experiments are included.

mongolian-text-classification Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# mongolian-text-classification
Mongolian cyrillic text classification with modern tensorflow and some fine tuning on TugsTugi's BERT model.

# Load Mongolian BERT in Tensorflow 2
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1ReDLH2DDiCt_Y800vGub8OuYJlR-TsZw)

# Generate text using Mongolian BERT

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1jJA-YSAsbq5gbpyGYE-8p-rCzgqSU9eX)

# Visualize Mongolian BERT using bertviz and pytorch model

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1UEDNlfEmXxZy1jRrE7pCTZNu8DplWVQv)

![Alt text](https://raw.githubusercontent.com/sharavsambuu/mongolian-text-classification/master/images/bert/mongolian-bert-attend-visualization.png)

# Fine tuning TugsTugi's Mongolian BERT model
On TPU mode, loading checkpoints from the file system doesn't supported by the bert and bucket should be used.

Fine tuning mongolian BERT on TPU, You need own bucket in order to finetune on TPU [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1CnGd2OnNDlxe6ZUjmOa7zg__CcKk5X85)

Fine tune a mongolian BERT on GPU, a lot of computation needed, a low batch size matters due to memory limit [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1u9mVeWRh7GWLONAzZ3XpJciPfv38vHaZ)

# Classifiers using simple neural networks

No 01, Simplest classifier [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1Ulv6tUAjOsp-jN4sTdef3lTuJb0yX4qy)

No 02, Pretrained Word2Vec initialization from Facebook's fasttext, kind of transfer learningish. Embedding layer is not trainable in this case [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1SfwdhIoRMi4kXeAN8eUjYXKuT5zig9WV) and with trainable embedding layer [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1WQvCa6KDOxQ2YjDdb48g4zsN60_Svbhg)

No 03, 1D Convolution [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1JgJN74E1w1x8RSjm9qi06uw6y0I_9k1J) and multiple 1D convnets [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1lTh2dG64L4aJsCip714sCA_xQgMttxOb)

No 04, LSTM [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1j0MN3UTGz-990bl61n5B1mrtjnq8hSdh)

Visualize RNN neuron firing in text generation [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1ndM1G-0qZx4wi6E9kPL1D9IjaM0pq3r9)

No 05, LSTM with Attention, visualization of attention scores in text classification [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/10nPgRmbZsjad46CdVJKRHklestXcEpZ5)

No 06, Classification with Mongolian BERT and Tensorflow 2.0, with frozen bert layers [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1JQ87pFlGkDMbpHQp9ZSiyrQwdiRUAm-N)

No 07, Classification with Mongolian BERT large and HuggingFace and Tensorflow 2 [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1e8t6MZMvpoINkXtv7o1h3ESZq04pXPUv?usp=sharing)

# Mongolian sentence interpolation experiments

Sequence loss in keras and tf2 [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1jlyB2fOi_JBAi4WPMVDJ_e8-_WHtQK_9)

Variational Auto Encoder for Mongolian text [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1tBTudj9M5CGih3p8Uxj0R1SA6f3BJj-Z)

# Other experiments
Predict next word, greedy text generation [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1urjsJUuNTnTAAAqu_eXpIkwRWUi72xp_)

# Series included(or will) followings
word2vec initialization, 1D Convolution, RNN variants, Attention, Some weights visualization for reasoning, Transformer, Techniques to handle longer texts and so on...

# useful references and resources
  - Mongolian BERT models
    https://github.com/tugstugi/mongolian-bert
  - Mongolian NLP
    https://github.com/tugstugi/mongolian-nlp
  - Eduge classification baseline using SVM
  	https://colab.research.google.com/github/tugstugi/mongolian-nlp/blob/master/misc/Eduge_SVM.ipynb
  - News crawler
    https://github.com/codelucas/newspaper
  
# Images and screenshots

![Alt text](https://raw.githubusercontent.com/sharavsambuu/mongolian-text-classification/master/images/cnn-weights/1.png)
![Alt text](https://raw.githubusercontent.com/sharavsambuu/mongolian-text-classification/master/images/cnn-weights/2.png)
![Alt text](https://raw.githubusercontent.com/sharavsambuu/mongolian-text-classification/master/images/cnn-weights/3.png)
![Alt text](https://raw.githubusercontent.com/sharavsambuu/mongolian-text-classification/master/images/cnn-weights/4.png)

mongolian-text-classification

mongolian-text-classification Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

mongolian-text-classification

mongolian-text-classification Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README