gesture_jester

KO | EN

gitlite — search

#hacktoberfest #ai #android #machine-learning #rust #docker #windows #awesome #vue #pytorch #ios #mcp #openai #open-source

(★ 12)

Use your hands to control a Music Player

gesture_jester 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Gesture Jester
## Description
This repository contains the Python code to develop your own hand gesture recognition system.

The app consists of 3 different modes:
1. __**Data Collection Mode:**__ Allows the user to collect train, test, or validation data on a variety of hand gestures
2. __**Model Testing Mode:**__ Test the model's ability to discern between different gestures through real-time visualizations
3. __**Music-Player/Gesture Mode:**__ Use gestures to play music, pause music, and change the volume of music

I presented this app as my final project for a coding bootcamp I attended. The slide deck for my presentation has been included in the repository.

## Packages Used
- OpenCV 
- Keras
- PyGame
- NumPy

## Files
Here's a list of files in the directory:
- `src/demo.py`: Contains all the functions to start and run the app
- `src/music`: Contains the song that will be played during 'gesture mode'
- `Presentation (Gesture Jester)`: Contains the slide deck for my project presentation

## Usage
In order to start the application, do the following:
1) Clone the repo
```
git clone https://github.com/mdylan2/gesture_jester.git
```
2) Navigate into the folder, set up a virtual environment and activate it
3) Once you've activated the virtual environment, install the requirements
```
pip install -r requirements.txt
```
4) Download `my_model_weights.h5` from [this Kaggle link](https://www.kaggle.com/dylanmendonca/training-hand-gesture-model) and store the file in the src folder. The model was trained on data 
that I collected through the 'data collection mode' of the app. If you find that my trained model doesn't work well for you or you feel like you need more gestures,
you can train the model using images of your own hand. Please refer to the Kaggle link for more information on building or training the model.
5) Navigate into the src folder and run the following command:
```
python demo.py
```
6) Some of the features have been included in the snippets in the Application Interface section

## Gesture Recognition Model
The model was built to recognize 7 different hand gestures and the absence of a hand. The encoding of the gestures is included in the demo file under the variable `GESTURE_ENCODINGS`. 
List of gestures recognized (in the order of the encodings):
1. `Fist`
![Fist](https://raw.githubusercontent.com/mdylan2/gesture_jester/master/imagesandgifs/fist.png)
2. `Five`
![Five](https://raw.githubusercontent.com/mdylan2/gesture_jester/master/imagesandgifs/five.png)
3. `None`
![None](https://raw.githubusercontent.com/mdylan2/gesture_jester/master/imagesandgifs/none.png)
4. `Okay`
![Okay](https://raw.githubusercontent.com/mdylan2/gesture_jester/master/imagesandgifs/okay.png)
5. `Peace`
![Peace](https://raw.githubusercontent.com/mdylan2/gesture_jester/master/imagesandgifs/peace.png)
6. `Rad`
![Rad](https://raw.githubusercontent.com/mdylan2/gesture_jester/master/imagesandgifs/rad.png)
7. `Straight`
![Straight](https://raw.githubusercontent.com/mdylan2/gesture_jester/master/imagesandgifs/straight.png)
8. `Thumbs`
![Thumbs](https://raw.githubusercontent.com/mdylan2/gesture_jester/master/imagesandgifs/thumbs.png)

## Music Player Model (Gesture Mode)
Currently the system is set up to recognize 6 gestures:
1. `Rad`: Loads the song (I Saw Her Standing There)
2. `Fist`: Plays/unpauses the song
3. `Five`: Pauses the song
4. `Okay`: Increases the volume. Hold the pose to continue increasing the volume
5. `Peace`: Decreases the volume. Hold the pose to continue decreasing the volume
6. `Straight`: Stops the song
7. `None`: Does nothing

## Application Interface
### `Capture Background`
![Capture Background](https://raw.githubusercontent.com/mdylan2/gesture_jester/master/imagesandgifs/capturingback.gif)

### `Data Collection Mode`
![Collect Data](https://raw.githubusercontent.com/mdylan2/gesture_jester/master/imagesandgifs/datamode.png)
![Collect Data 2](https://raw.githubusercontent.com/mdylan2/gesture_jester/master/imagesandgifs/collecting_data.gif)

### `Testing Model Mode`
![Test Model](https://raw.githubusercontent.com/mdylan2/gesture_jester/master/imagesandgifs/testing_model.gif)

### `Gesture Mode (Music Player)`[Click the Image below!]
[![Youtube Link](https://raw.githubusercontent.com/mdylan2/gesture_jester/master/imagesandgifs/gesture_mode.PNG)](https://vimeo.com/351121225)
>  **Note: Video is actually on Vimeo. Blame YouTube and its copyright issues**

## Questions/Contributions/Future Work
- I have tried to include as much instruction for use on the app GUI. Please let me know if you have any further questions.
- I didn't put much focus into the Music Player (was more of a proof of concept) so there might be a lot of bugs in there. I'm hoping to continue building GUI features around the music player so the user can see the volume, current song, etc.
- Last but not least, I'm very open to any recommendations or contributions! Please reach out to me on Github if you would like to chat.

gesture_jester

gesture_jester 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

gesture_jester

gesture_jester 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README