Cloze-Test_Pytorch_BERT

KO | EN

gitlite — search

#python #java #python3 #arduino #golang #machine-learning #rust #html #flask #javascript #seismology #nodejs

Cloze-Test_Pytorch_BERT

★ 11 Open GitHub ↗

This is about the English test - cloze test which use the Google BERT model to predict the probable word.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Cloze-Test_Pytorch_BERT

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/matteosoo/Cloze-Test_Pytorch_BERT/blob/master/demo_tutorial.ipynb)

## How to use?

* Download PyTorch:
  [Official PyTorch Website](http://pytorch.org/ "pytorch")
  
  or using command line:
  ```
  conda install pytorch torchvision -c pytorch
  # MacOS Binaries dont support CUDA, install from source if CUDA is needed
  ```

* With pip:
  ```
  pip install pytorch-pretrained-bert
  ```
  ```
  pip3 install --upgrade gensim
  ```

## Overview
Ten answers choices are given. Select the best answer to complete the sentence.
  
  * Example
  
    Create a text file and save in folder as follows:
    ```
    1 Longman , the third , looked at the others over his shoulder .
    2 Goldband , the fourth , had a gold sash round his waist ; and little Playman did nothing at all , and was the more proud .
    3 There was too much ostentation , and so I came away . '
    4 ` And now we are sitting and shining here ! '
    5 said the bit of bottle-glass .
    6 At that moment more water came into the gutter ; it streamed over the edges and washed the bit of bottle-glass away .
    7 ` Ah !
    8 now he has been promoted ! '
    9 said the Darning-needle . '
    10 I remain here ; I am too fine .
    11 But that is my pride , which is a sign of respectability ! '
    12 And she sat there very proudly , thinking lofty thoughts . '
    13 I really believe I must have been born a sunbeam , I am so fine !
    14 It seems to me as if the sunbeams were always looking under the water for me .
    15 Ah , I am so fine that my own mother can not find me !
    16 If I had my old eye which broke off , I believe I could weep ; but I ca n't -- it is not fine to weep ! '
    17 One day two street-urchins were playing and wading in the gutter , picking up old nails , pennies , and such things .
    18 It was rather dirty work , but it was a great delight to them .
    19 ` Oh , oh ! '
    20 cried out one , as he pricked himself with the Darning-needle ; ` he is a fine fellow though ! ' '
    21 I XXXXX not a fellow ; I am a young lady ! '	am		am|born|find|picking|pricked|said|sat|seems|streamed|thinking
    ```
    * The XXXXX in the 21th sentence is the target we want to predict.
    * Also, the example in 21th sentence '**am**' is the answer in this question. You could change it whether the token you want.
    * The end of the 21th sentence are given 10 probable selections.
    
  * Results
  
    ![demo_result](https://github.com/matteosoo/Cloze-Test_Pytorch_BERT/blob/master/img/demo_result.png)

## Reference
  * BERT model: https://github.com/huggingface/pytorch-pretrained-BERT
  * gensim data: https://github.com/RaRe-Technologies/gensim-data

Cloze-Test_Pytorch_BERT

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README