dali

KO | EN

gitlite — search

#hacktoberfest #ai #golang #go #machine-learning #docker #macos #windows #awesome #vue #mcp #openai #ios #open-source

(★ 20)

Domain Adaptation of Neural Machine Translation by Lexicon Induction

dali Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

Domain Adaptation of Neural Machine Translation by Lexicon Induction
===
Implemented by [Junjie Hu](http://www.cs.cmu.edu/~junjieh/)

Contact: junjieh@cs.cmu.edu

If you use the codes in this repo, please cite our [ACL2019 paper](https://www.aclweb.org/anthology/P19-1286).

@inproceedings{hu-etal-2019-domain,
	    title = "Domain Adaptation of Neural Machine Translation by Lexicon Induction",
	    author = "Hu, Junjie and Xia, Mengzhou and Neubig, Graham and Carbonell, Jaime",
	    booktitle = "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics",
	    month = jul,
	    year = "2019",
	    address = "Florence, Italy",
	    publisher = "Association for Computational Linguistics",
	    url = "https://www.aclweb.org/anthology/P19-1286",
	    doi = "10.18653/v1/P19-1286",
	    pages = "2989--3001",
	}

Installation
==
- Anaconda environment
```
conda env create --file conda-dali-env.txt
```

- Install fairseq
```
cd fairseq && pip install --editable . && cd ..
```

- Install fastText
```
cd fastText && mkdir build && cd build && cmake .. && make && cd ../..
```

- Download MUSE's dictionary
```
cd MUSE/data/ && bash get_evaluation.sh && cd ../..
```

Downloads
==
The preprocessed data and pre-trained models can be found [here](https://drive.google.com/drive/folders/1PmlmLg8ZgR4MVLb0svP2j5oE0jQ1G3Z4?usp=sharing). Extract ***dataset.tar.gz*** under the ***dali*** directory. Extract ***{data-bin, it-de-en-epoch40, it2emea-de-en}.tar.gz*** under the ***dali/outputs*** directory.

- ***dataset.tar.gz***: train/dev/test data in five domains: it, emea, acquis, koran, subtitles.
- ***data-bin.tar.gz***: fairseq's binarized data.
- ***it-de-en-epoch40.tar.gz***: fairseq's transformer model pre-trained on data in the it domain.
- ***it2emea-de-en.tar.gz***: fairseq's transformer model adapted from it domain to emea domain using DALI-U.
- ***S2T+T2S-de-en.lex***: the lexicon induced by DALI-U. 
- ***embed-vec.tar.gz***: {it,emea,acquis,koran,subtitles}.{de,en}.vec embeddings trained in five domains respectively, and {de,en}.vec embeddings trained in the combination of five domains.

The pre-trained model in the it domain can obtain the BLEU scores in the five domains as follows. After adaptation, the BLEU in the emea test set can be raised to *18.25* from *8.23*. The BLEU scores are slightly different from those in the paper since we used different NMT toolkits (fairseq v.s. OpenNMT), but we observed similar improvements as we found in the paper.

<table>
  <tr>
    <th rowspan="2">Out-of-domain </th>
    <th colspan="5"> In-domain </th>
  </tr>
    <td>it</td>
    <td>emea</td>
    <td>koran</td>
    <td>subtitles</td>
    <td>acquis</td>
  </tr>
  <tr>
    <td>it</td>
    <td>58.94</td>
    <td>8.23</td>
    <td>2.50</td>
    <td>6.26</td>
    <td>4.34</td>
  </tr>
</table>

Demo
==
- Preprocess the data in the source (it) domain
```
bash scripts/preprocess.sh
```

- Train the transformer model in the source (it) domain
```
bash scripts/train.sh [GPU id]
```

- Perform DALI's data augmentation
	1.1 (Unupervised Lexicon Induction) Train the word embeddings
	```
	bash scripts/train-embed.sh
	```
	1.2 (Unupervised Lexicon Induction) Train the crosslingual embeddings by supervised lexicon induction
	```
	bash scripts/train-muse.sh [path to supervised seed lexicon]
	```
	1.3 (Unupervised Lexicon Induction) Train the crosslingual embeddings by unsupervised lexicon induction
	``` 
	bash scripts/train-muse.sh
	```
	1.4 (Unupervised Lexicon Induction) Obtain the word translation by nearest neighbor search
	```
	python3 extract_lexicon.py \
	  --src_emb $PWD/outputs/unsupervised-muse/debug/v1/vectors-de.txt \
	  --tgt_emb $PWD/outputs/unsupervised-muse/debug/v1/vectors-en.txt \
	  --output $PWD/outputs/unsupervised-muse/debug/v1/S2T+T2S-de-en.lex \
	  --dico_build "S2T&T2S"
	```
    2.1 (Supervised Lexicon Induction) Obtain the word translation by GIZA++
    ```
    bash scripts/extract_lex_giza.sh
    ```
	3. Perform word-for-word back-translation
	```
	bash scripts/wfw_backtranslation.sh 
	```

- Preprocess the data in the target (emea) domain 
```
bash scripts/preprocess-da.sh
```

- Adapt the pre-train model to the target (emea) domain
```
bash scripts/train-da-opt.sh
```

- Translate the test1 set in the emea domain
```
bash scripts/translate.sh \
  outputs/it-de-en-epoch40/checkpoint_best.pt \
  outputs/it-de-en-epoch40/decode-test1-best.txt \
  outputs/data-bin-join/it \
  test1
```

dali

dali Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

dali

dali Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README