lipnet-replication

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

lipnet-replication

(★ 28)

A replication of Google DeepMind's paper:LipNet: End-to-End Sentence-level Lipreading

lipnet-replication Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

## lipnet-replication
A replication of Google DeepMind's paper:LipNet: End-to-End Sentence-level Lipreading

using keras framework and tensorflow backend.

###1. preprocess

preprocess the videos to frames and crop it to lips.(preprocessing codes haven't uploaded yet)
The dir structure looks like:

```
./data/GRID/
    lip/
        s1/
            bbaf2n/
                0.jpg
                1.jpg
                2.jpg
                ...  
                74.jpg
            ...
        s2/
            ...
        ...
    
    alignments/
        s1/align/
                bbaf2n.align
                ...
        s2/align/
                ...
        ...
```

Dir `./data/GRID/lip/s{i}/{name}` saves the lip sequence pictures of the video with {name} of person s{i}, and dir `./data/GRID/alignments/s{i}/align/{name}.align` is the alignment file correspondingly.

**data augmention and normalization havent implemented yet. I may finish this part later.**

###2. train

```
python train_multiuser.py
```

the dataset splition and the network is the same as the Deepmind's paper.
file `gridDataset.py` is the GRID data generator and `model/lipnet.py` is the model.

lipnet-replication

lipnet-replication Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

lipnet-replication

lipnet-replication Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README