lipnet-replication

KO | EN

gitlite — search

#typescript #ai-agents #ai #dsh-plugin #deepseek-harness #open-source #cli #claude-code #codex #developer-tools #react #windows

lipnet-replication

★ 28 Open GitHub ↗

A replication of Google DeepMind's paper:LipNet: End-to-End Sentence-level Lipreading

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

## lipnet-replication
A replication of Google DeepMind's paper:LipNet: End-to-End Sentence-level Lipreading

using keras framework and tensorflow backend.

###1. preprocess

preprocess the videos to frames and crop it to lips.(preprocessing codes haven't uploaded yet)
The dir structure looks like:

```
./data/GRID/
    lip/
        s1/
            bbaf2n/
                0.jpg
                1.jpg
                2.jpg
                ...  
                74.jpg
            ...
        s2/
            ...
        ...
    
    alignments/
        s1/align/
                bbaf2n.align
                ...
        s2/align/
                ...
        ...
```

Dir `./data/GRID/lip/s{i}/{name}` saves the lip sequence pictures of the video with {name} of person s{i}, and dir `./data/GRID/alignments/s{i}/align/{name}.align` is the alignment file correspondingly.

**data augmention and normalization havent implemented yet. I may finish this part later.**

###2. train

```
python train_multiuser.py
```

the dataset splition and the network is the same as the Deepmind's paper.
file `gridDataset.py` is the GRID data generator and `model/lipnet.py` is the model.

lipnet-replication

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README