kosp2e

KO | EN

gitlite — search

#ai #typescript #go #android #machine-learning #linux #macos #java #cli #awesome-list #vue #pytorch #mcp #openai

(★ 45)

Korean Speech to English Translation Corpus

kosp2e 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# kosp2e
Korean Speech to English Translation Corpus

## Dataset
### Freely available
* Speech files
* Train/Dev/Test filenames' list their English translation
### Provided under request (in [this link](https://docs.google.com/forms/d/1UTpOrKIWK9uzngh7eIm-3oAp5b5vTdBj4ZapHj8cyBI/edit))
* Korean scripts
* Other metadata (for StyleKQC and Covid-ED)

## Howto
```
git clone https://github.com/warnikchow/kosp2e
cd kosp2e
cd data
wget https://www.dropbox.com/s/y74ew1c1evdoxs1/data.zip
unzip data.zip
```
Then you get the folder with speech files (*data* and subfolders) and split files' list (*split* and .xlsx files).

- The total number of *.wav* files in *Zeroth* dataset does not match with the total number of translation pairs that are provided, since some of the examples were excluded in the corpus construction to guarantee the data quality. However, to maintain files of the original *Zeroth* dataset, we did not delete them from the *.wav* files folder. The preprocessing and data loading is not affected by the difference of file list.

## Baseline
| Model | BLEU | WER (ASR) | BLEU (MT/ST) |
|:--------------------------:|:----:|:------------:|:---------------:|
| ASR-MT (Pororo) | 16.6 | 34.0 | 18.5 (MT) |
| ASR-MT (PAPAGO) | 21.3 | 34.0 | 25.0 (MT) |
| Transformer (Vanilla) | 2.6 | - | - |
| ASR pretraining | 5.9 | 24.0* | - |
| Transformer + Warm-up | *8.7* | - | 35.7 (ST)* |
| + Fine-tuning | *18.3* | - | - |

- *Some of the numerics* differ from the [paper](https://arxiv.org/abs/2107.02875)  (after fixing some errors), but may not influence the results much.

### Recipe
- [Fairseq](https://github.com/pytorch/fairseq) is required for the basic recipe. You may install [specific fairseq version](https://github.com/pytorch/fairseq/tree/148327d8c1e3a5f9d17a11bbb1973a7cf3f955d3) for replication.
```
wget https://github.com/pytorch/fairseq/archive/148327d8c1e3a5f9d17a11bbb1973a7cf3f955d3.zip
unzip 148327d8c1e3a5f9d17a11bbb1973a7cf3f955d3.zip
pip install -e ./fairseq-148327d8c1e3a5f9d17a11bbb1973a7cf3f955d3/

pip install -r requirements.txt
```

- First, you preprocess the data, and then prepare them in a format that fit with transformer. Other part follows [fairseq S2T translation recipe with MuST-C](https://github.com/pytorch/fairseq/tree/148327d8c1e3a5f9d17a11bbb1973a7cf3f955d3/examples/speech_to_text). 
- This recipe leads you to the *Vanilla* model (the most basic end-to-end version). For the advanced training, refer to the [paper](https://arxiv.org/abs/2107.02875) below.
```
python preprocessing.py

python prep_data.py --data-root dataset/ --task st --vocab-type unigram --vocab-size 8000

fairseq-train dataset/kr-en  --config-yaml config_st.yaml \
--train-subset train_st --valid-subset dev_st --save-dir result --num-workers 4 \
--max-tokens 40000 --max-update 50000 --task speech_to_text \
--criterion label_smoothed_cross_entropy --report-accuracy \
--arch s2t_transformer_s --optimizer adam --lr 2e-3 --lr-scheduler inverse_sqrt \
--warmup-updates 10000 --clip-norm 10.0 --seed 1 --update-freq 8 --fp16 
```

## Acknowledgement
This work was supported by **PAPAGO, NAVER Corp.** The authors appreciate Hyoung-Gyu Lee, ‪Eunjeong Lucy Park, Jihyung Moon, and Doosun Yoo for discussions and support.‬  Also, the authors thank Taeyoung Jo, Kyubyong Park, and Yoon Kyung Lee for sharing the resources.

### License
License of each subcorpus (including metadata and Korean script) follows the original license of the raw corpus. For *KSS* and *Covid-ED*, only academic usage is permitted.

### Citation
```
@inproceedings{cho21b_interspeech,
  author={Won Ik Cho and Seok Min Kim and Hyunchang Cho and Nam Soo Kim},
  title={{kosp2e: Korean Speech to English Translation Corpus}},
  year=2021,
  booktitle={Proc. Interspeech 2021},
  pages={3705--3709},
  doi={10.21437/Interspeech.2021-1040}
}
```
arXiv version is [here](https://arxiv.org/abs/2107.02875).

## Contact
Contact Won Ik Cho tsatsuki@snu.ac.kr for further question.

kosp2e

kosp2e 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

kosp2e

kosp2e 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README