ABCNN

KO | EN

gitlite — search

#typescript #llm #golang #android #machine-learning #deep-learning #linux #macos #java #cli #vue #pytorch #ios #openai

(★ 280)

Implementation of ABCNN(Attention-Based Convolutional Neural Network) on Tensorflow

파일 탐색기

최종 버전 다운로드 (.zip)

- WikiQA-ABCNN1-1-LR.txt
- WikiQA-ABCNN1-1-model.txt
- WikiQA-ABCNN1-1-SVM.txt
- WikiQA-ABCNN1-2-LR.txt
- WikiQA-ABCNN1-2-SVM.txt
- WikiQA-ABCNN2-1-LR.txt
- WikiQA-ABCNN3-1-LR.txt
- WikiQA-ABCNN3-1-model.txt
- WikiQA-ABCNN3-1-SVM.txt
- WikiQA-ABCNN3-2-LR.txt
- WikiQA-BCNN-1-LR.txt
- WikiQA-BCNN-1-model.txt
- WikiQA-BCNN-1-SVM.txt
- WikiQA-BCNN-2-LR.txt
- WikiQA-BCNN-2-model.txt
- WikiQA-BCNN-2-SVM.txt
- checkpoint
- WikiQA-ABCNN2-1-1-LR.pkl
- WikiQA-ABCNN2-1-1-SVM.pkl
- WikiQA-ABCNN2-1-1.data-00000-of-00001
- WikiQA-ABCNN2-1-10.meta
- WikiQA-ABCNN2-1-11-LR.pkl
- WikiQA-ABCNN2-1-12-LR.pkl
- WikiQA-ABCNN2-1-12-SVM.pkl
- WikiQA-ABCNN2-1-12.meta
- WikiQA-ABCNN2-1-13-LR.pkl
- WikiQA-ABCNN2-1-13.index
- WikiQA-ABCNN2-1-13.meta
- WikiQA-ABCNN2-1-14-LR.pkl
- WikiQA-ABCNN2-1-14.data-00000-of-00001
- WikiQA-ABCNN2-1-15-LR.pkl
- WikiQA-ABCNN2-1-15-SVM.pkl
- WikiQA-ABCNN2-1-15.data-00000-of-00001
- WikiQA-ABCNN2-1-15.index
- WikiQA-ABCNN2-1-16-LR.pkl
- WikiQA-ABCNN2-1-16-SVM.pkl
- WikiQA-ABCNN2-1-16.data-00000-of-00001
- WikiQA-ABCNN2-1-17-SVM.pkl
- WikiQA-ABCNN2-1-17.data-00000-of-00001
- WikiQA-ABCNN2-1-19-LR.pkl
- WikiQA-ABCNN2-1-2.data-00000-of-00001
- WikiQA-ABCNN2-1-2.index
- WikiQA-ABCNN2-1-20.index
- WikiQA-ABCNN2-1-20.meta
- WikiQA-ABCNN2-1-21-LR.pkl
- WikiQA-ABCNN2-1-21-SVM.pkl
- WikiQA-ABCNN2-1-21.meta
- WikiQA-ABCNN2-1-22-LR.pkl
- WikiQA-ABCNN2-1-22-SVM.pkl
- WikiQA-ABCNN2-1-22.data-00000-of-00001
- WikiQA-ABCNN2-1-22.index
- WikiQA-ABCNN2-1-22.meta
- WikiQA-ABCNN2-1-23.meta
- WikiQA-ABCNN2-1-24-LR.pkl
- WikiQA-ABCNN2-1-24-SVM.pkl
- WikiQA-ABCNN2-1-24.index
- WikiQA-ABCNN2-1-24.meta
- WikiQA-ABCNN2-1-25-LR.pkl
- WikiQA-ABCNN2-1-25-SVM.pkl
- WikiQA-ABCNN2-1-25.index
- WikiQA-ABCNN2-1-26-LR.pkl
- WikiQA-ABCNN2-1-26.index
- WikiQA-ABCNN2-1-26.meta
- WikiQA-ABCNN2-1-27-LR.pkl
- WikiQA-ABCNN2-1-27-SVM.pkl
- WikiQA-ABCNN2-1-28.data-00000-of-00001
- WikiQA-ABCNN2-1-29.data-00000-of-00001
- WikiQA-ABCNN2-1-29.index
- WikiQA-ABCNN2-1-29.meta
- WikiQA-ABCNN2-1-3.data-00000-of-00001
- WikiQA-ABCNN2-1-3.index
- WikiQA-ABCNN2-1-30-SVM.pkl
- WikiQA-ABCNN2-1-30.data-00000-of-00001
- WikiQA-ABCNN2-1-30.index
- WikiQA-ABCNN2-1-31.index
- WikiQA-ABCNN2-1-31.meta
- WikiQA-ABCNN2-1-32.index
- WikiQA-ABCNN2-1-33-LR.pkl
- WikiQA-ABCNN2-1-33-SVM.pkl
- WikiQA-ABCNN2-1-34-SVM.pkl
- WikiQA-ABCNN2-1-34.index
- WikiQA-ABCNN2-1-34.meta
- WikiQA-ABCNN2-1-35-LR.pkl
- WikiQA-ABCNN2-1-35.meta
- WikiQA-ABCNN2-1-36-SVM.pkl
- WikiQA-ABCNN2-1-36.data-00000-of-00001
- WikiQA-ABCNN2-1-36.meta
- WikiQA-ABCNN2-1-37-LR.pkl
- WikiQA-ABCNN2-1-37-SVM.pkl
- WikiQA-ABCNN2-1-37.meta
- WikiQA-ABCNN2-1-38-LR.pkl
- WikiQA-ABCNN2-1-38.data-00000-of-00001
- WikiQA-ABCNN2-1-38.meta
- WikiQA-ABCNN2-1-39-SVM.pkl
- WikiQA-ABCNN2-1-4-LR.pkl
- WikiQA-ABCNN2-1-4.index
- WikiQA-ABCNN2-1-4.meta
- WikiQA-ABCNN2-1-40-LR.pkl
- WikiQA-ABCNN2-1-40-SVM.pkl
- WikiQA-ABCNN2-1-41-SVM.pkl
- WikiQA-ABCNN2-1-41.data-00000-of-00001
- WikiQA-ABCNN2-1-41.index
- WikiQA-ABCNN2-1-41.meta
- WikiQA-ABCNN2-1-42-LR.pkl
- WikiQA-ABCNN2-1-42.data-00000-of-00001
- WikiQA-ABCNN2-1-42.meta
- WikiQA-ABCNN2-1-43-LR.pkl
- WikiQA-ABCNN2-1-43.index
- WikiQA-ABCNN2-1-43.meta
- WikiQA-ABCNN2-1-44-LR.pkl
- WikiQA-ABCNN2-1-45-SVM.pkl
- WikiQA-ABCNN2-1-45.index
- WikiQA-ABCNN2-1-46-LR.pkl
- WikiQA-ABCNN2-1-46-SVM.pkl
- WikiQA-ABCNN2-1-46.meta
- WikiQA-ABCNN2-1-47.data-00000-of-00001
- WikiQA-ABCNN2-1-47.index
- WikiQA-ABCNN2-1-48-SVM.pkl
- WikiQA-ABCNN2-1-48.data-00000-of-00001
- WikiQA-ABCNN2-1-48.index
- WikiQA-ABCNN2-1-49-SVM.pkl
- WikiQA-ABCNN2-1-49.data-00000-of-00001
- WikiQA-ABCNN2-1-49.index
- WikiQA-ABCNN2-1-49.meta
- WikiQA-ABCNN2-1-5.index
- WikiQA-ABCNN2-1-50.data-00000-of-00001
- WikiQA-ABCNN2-1-50.index
- WikiQA-ABCNN2-1-6-SVM.pkl
- WikiQA-ABCNN2-1-6.index
- WikiQA-ABCNN2-1-7-LR.pkl
- WikiQA-ABCNN2-1-7-SVM.pkl
- WikiQA-ABCNN2-1-7.index
- WikiQA-ABCNN2-1-7.meta
- WikiQA-ABCNN2-1-8.data-00000-of-00001
- WikiQA-ABCNN2-1-8.index
- WikiQA-ABCNN3-2-1-LR.pkl
- WikiQA-ABCNN3-2-1-SVM.pkl
- WikiQA-ABCNN3-2-1.data-00000-of-00001
- WikiQA-ABCNN3-2-1.index
- WikiQA-ABCNN3-2-1.meta
- WikiQA-ABCNN3-2-10-LR.pkl
- WikiQA-ABCNN3-2-10-SVM.pkl
- WikiQA-ABCNN3-2-10.data-00000-of-00001
- WikiQA-ABCNN3-2-10.meta
- WikiQA-ABCNN3-2-11.data-00000-of-00001
- WikiQA-ABCNN3-2-11.index
- WikiQA-ABCNN3-2-11.meta
- WikiQA-ABCNN3-2-12-LR.pkl
- WikiQA-ABCNN3-2-12-SVM.pkl
- WikiQA-ABCNN3-2-12.data-00000-of-00001
- WikiQA-ABCNN3-2-13-LR.pkl
- WikiQA-ABCNN3-2-14-LR.pkl
- WikiQA-ABCNN3-2-14-SVM.pkl
- WikiQA-ABCNN3-2-14.data-00000-of-00001
- WikiQA-ABCNN3-2-14.index
- WikiQA-ABCNN3-2-14.meta
- WikiQA-ABCNN3-2-15.data-00000-of-00001
- WikiQA-ABCNN3-2-15.meta
- WikiQA-ABCNN3-2-16-SVM.pkl
- WikiQA-ABCNN3-2-17-LR.pkl
- WikiQA-ABCNN3-2-17.index
- WikiQA-ABCNN3-2-17.meta
- WikiQA-ABCNN3-2-2-LR.pkl
- WikiQA-ABCNN3-2-2-SVM.pkl
- WikiQA-ABCNN3-2-2.data-00000-of-00001
- WikiQA-ABCNN3-2-2.index
- WikiQA-ABCNN3-2-2.meta
- WikiQA-ABCNN3-2-3-LR.pkl
- WikiQA-ABCNN3-2-3-SVM.pkl
- WikiQA-ABCNN3-2-3.data-00000-of-00001
- WikiQA-ABCNN3-2-3.index
- WikiQA-ABCNN3-2-3.meta
- WikiQA-ABCNN3-2-34.data-00000-of-00001
- WikiQA-ABCNN3-2-4-LR.pkl
- WikiQA-ABCNN3-2-4-SVM.pkl
- WikiQA-ABCNN3-2-4.data-00000-of-00001
- WikiQA-ABCNN3-2-4.index
- WikiQA-ABCNN3-2-4.meta
- WikiQA-ABCNN3-2-5-LR.pkl
- WikiQA-ABCNN3-2-5-SVM.pkl
- WikiQA-ABCNN3-2-5.data-00000-of-00001
- WikiQA-ABCNN3-2-5.index
- WikiQA-ABCNN3-2-5.meta
- WikiQA-ABCNN3-2-6-LR.pkl
- WikiQA-ABCNN3-2-6-SVM.pkl
- WikiQA-ABCNN3-2-6.data-00000-of-00001
- WikiQA-ABCNN3-2-6.index
- WikiQA-ABCNN3-2-6.meta
- WikiQA-ABCNN3-2-7-LR.pkl
- WikiQA-ABCNN3-2-7-SVM.pkl
- WikiQA-ABCNN3-2-7.data-00000-of-00001
- WikiQA-ABCNN3-2-7.index
- WikiQA-ABCNN3-2-7.meta
- WikiQA-ABCNN3-2-8-LR.pkl
- WikiQA-ABCNN3-2-8-SVM.pkl
- WikiQA-ABCNN3-2-8.data-00000-of-00001
- WikiQA-ABCNN3-2-8.index
- WikiQA-ABCNN3-2-8.meta
- WikiQA-ABCNN3-2-9-SVM.pkl
- WikiQA-ABCNN3-2-9.meta
- make_dev.py
- msr_paraphrase_data.txt
- msr_paraphrase_dev.txt
- msr_paraphrase_test.txt
- msr_paraphrase_train.txt
- msr_paraphrase_train_backup.txt
- msr_paraphrase_train_without_dev.txt
- result.txt
- test.py
- extract.py
- WikiQA-test.txt
- WikiQA-train.txt
- WikiQA-train_bak.txt
ABCNN.py
preprocess.py
README.md
show.py
test.py
train.py
utils.py

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs

#### [Update]: Someone has reported me that the problem of a loss being 'nan' can be attributed to tf.sqrt function which outpus 'nan' when its input is very small or negative. Therefore, I recommend you modify tf.sqrt functions adequately if you have in the trouble.

#### \[Warning\]: Some people have reported that there are some bugs that losses go to NaN in case of ABCNN-2 and 3. (I don't know the exact condition where the bugs appear.) Unfortunately, I have no plan to revise the code around the corner. Please be careful when using the code or please send me the pull requests when your revised version of the code works properly. Thanks.

This is the implementation of **ABCNN**, which is proposed by [Wenpeng Yin et al.](https://arxiv.org/pdf/1512.05193.pdf), on **Tensorflow**.  
It includes all 4 models below:
- BCNN

|               |          |   MAP   |   MRR   |
    |:-------------:|:--------:|:-------:|:-------:|
    | BCNN(1 layer) |  Results |  0.6660 |  0.6813 |
    |               | Baseline |  0.6629 |  0.6813 |
    | BCNN(2 layer) |  Results |  0.6762 |  0.6871 |
    |               | Baseline |  0.6593 |  0.6738 |

- ABCNN-1

|                  |          |   MAP   |   MRR   |
    |:----------------:|:--------:|:-------:|:-------:|
    | ABCNN-1(1 layer) |  Results |  0.6652 |  0.6755 |
    |                  | Baseline |  0.6810 |  0.6979 |
    | ABCNN-1(2 layer) |  Results |  0.6702 |  0.6838 |
    |                  | Baseline |  0.6855 |  0.7023 |

- ABCNN-2

|                  |          |   MAP   |   MRR   |
    |:----------------:|:--------:|:-------:|:-------:|
    | ABCNN-2(1 layer) |  Results |  0.6660 |  0.6813 |
    |                  | Baseline |  0.6885 |  0.7023 |
    | ABCNN-2(2 layer) |  Results |  ------ |  ------ |
    |                  | Baseline |  0.6879 |  0.7068 |

- ABCNN-3

|                  |          |   MAP   |   MRR   |
    |:----------------:|:--------:|:-------:|:-------:|
    | ABCNN-3(1 layer) |  Results |  0.6612 |  0.6682 |
    |                  | Baseline |  0.6914 |  0.7127 |
    | ABCNN-3(2 layer) |  Results |  0.6571 |  0.6722 |
    |                  | Baseline |  0.6921 |  0.7105 |

### Note:
- Implementation is now only focusing on AS task with [WikiQA](https://www.microsoft.com/en-us/research/publication/wikiqa-a-challenge-dataset-for-open-domain-question-answering/) corpus.
(I originally tried to deal with PI task with [MSRP(Microsoft Research Paraphrase)](https://www.microsoft.com/en-us/download/details.aspx?id=52398) corpus
but it seems that model doesn't work without external features classifier requires.)
- My code has verified that **BCNN works fine as the authors proposed.** (watched even better results than the paper's.)
-  In the case of ABCNNs, results are inferior to ones in the paper but somewhat competitive. Careful hyperparameter configuration and detailed re-examination may help to achieve optimized results.
- I doubt that there are some bugs on ABCNNs(especially ABCNN-2 which has 2 conv layers) and will keep watching codes. Please be careful when using the results.

## Specification
- **preprocess.py**: preprocess (training, test) data and import word2vec to use.
- **train.py**: train a model with configs.
- **test.py**: test the trained model.
- **ABCNN.py**: Implementation of ABCNN models.
- show.py: pyplot codes for test results.
- utils.py: common util functions.
- MSRP_Corpus: MSRP corpus for PI.
- WikiQA_Corpus: WikiQA corpus for AS.
- models: saved models available on Tensorflow.
- experiments: test results on AS tasks.

## Development Environment
- OS: Windows 10 (64 bit)
- Language: Python 3.5.3
- CPU: Intel Xeon CPU E3-1231 v3 3.4 GHz
- RAM: 16GB
- GPU support: GTX 970
- Libraries:
    - **tensorflow 1.2.1**
    - numpy 1.12.1
    - gensim 1.0.1
    - NLTK 3.2.2
    - scikit-learn 0.18.1
    - matplotlib 2.0.0

## Requirements

This model is based on pre-trained Word2vec([GoogleNews-vectors-negative300.bin](https://drive.google.com/uc?id=0B7XkCwpI5KDYNlNUTTlSS21pQmM&export=download)) by T.Mikolov et al.  
You should download this file and place it in the root folder.

## Execution
> (training): python train.py --lr=0.08 --ws=4 --l2_reg=0.0004 --epoch=20 --batch_size=64 --model_type=BCNN --num_layers=2 --data_type=WikiQA

Paramters
    --lr: learning rate
    --ws: window_size
    --l2_reg: l2_reg modifier
    --epoch: epoch
    --batch_size: batch size
    --model_type: model type
    --num_layers: number of convolution layers
    --data_type: MSRP or WikiQA data

> (test): python test.py --ws=4 --l2_reg=0.0004 --epoch=20 --max_len=40 --model_type=BCNN --num_layers=2 --data_type=WikiQA --classifier=LR

Paramters
    --ws: window_size
    --l2_reg: l2_reg modifier
    --epoch: epoch
    --max_len: max sentence length
    --model_type: model type
    --num_layers: number of convolution layers
    --data_type: MSRP or WikiQA data
    --classifier: Final layout classifier(model, LR, SVM)

## MISC.
- [Original code by the author.](https://github.com/yinwenpeng/Answer_Selection/tree/master/src)

ABCNN

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

ABCNN

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README