Robust_e2e_gan

KO | EN

gitlite — search

#python #hacktoberfest #llm #go #machine-learning #linux #nodejs #macos #java #windows #kubernetes #mcp #openai #open-source

(★ 19)

PyTorch implementation of "Jointly Adversarial Enhancement Training for Robust End-to-End Speech Recognition"

Robust_e2e_gan 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Jointly Adversarial Enhancement Training for Robust End-to-End Speech Recognition

This is an implementation of our InterSpeech2019 paper "Jointly Adversarial Enhancement Training for Robust End-to-End Speech Recognition" on Python 3, PyTorch. We propose a jointly adversarial enhancement training to boost robustness of end-to-end systems. Specifically, we use a jointly compositional scheme of mask-based enhancement network, attention-based encoder-decoder network and discriminant network during training.

# Requirements
Python 3.5, PyTorch 0.4.0.

# Data
### AISHELL
You can download [AISHELL](http://www.aishelltech.com/kysjcp) to run the code.
You can run ```sh run.sh ``` for AISHELL, but it's recommended that you run the commands one by one.

### Your Own Dataset
You need build train, dev and test directory. Each has ```clean_feats.scp``` ```noisy_feats.scp``` and ```text```. You can run ```python3 data/prepare_feats.py data_dir feat_dir noise_repeat_num``` to generate the noisy data ```noisy_feats.scp```.

# Model

The system consists of an enhancement network, an end-to-end ASR network, a fbank feature extraction network and a discriminant network. The enhancement network transforms the noisy STFT features to the enhanced STFT features. The fbank feature extraction network is used to extract the normalized log fbank features. The end-to-end ASR model estimates the posteriori probabilities for output labels. The discriminant network is used to distinguish between the enhanced features and clean ones.

# Training

### E2E ASR training
You can train the E2E ASR network using the clean speech data and multi-condition training strategy i.e., optimization with both the clean and noisy speech.

```
python3 asr_train.py --dataroot Your data directory(including train, dev and test dataset) 
```

### Enhancement Training
You can train the enhancement network by the mask loss function.

```
python3 enhance_base_train.py --dataroot Your data directory
```
or the mask fbank loss function.

```
python3 enhance_fbank_train.py --dataroot Your data directory
```
or the gan loss function.

```
python3 enhance_gan_train.py --dataroot Your data directory
```

### Joint Training
You can jointly train the enhancement network and end-to-end ASR network by the ASR loss.
```
python3 joint_base_train.py --dataroot Your data directory
```
You can also jointly train the enhancement, end-to-end ASR network and a discriminant network by the adversarial loss.
```
python3 joint_train.py --dataroot Your data directory
```

# Decoding
We use the beam search for decoding in all the experiments.
```
python3 asr_recog.py 
```

Robust_e2e_gan

Robust_e2e_gan 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Robust_e2e_gan

Robust_e2e_gan 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README