Exercise_Recognition_AI - GitLite

KO | EN

gitlite — search

#ai #golang #go #android #machine-learning #rust #deep-learning #linux #docker #java #windows #pytorch #ios #open-source

Exercise_Recognition_AI

(★ 121)

Developing a virtual personal fitness tracker and exercise activity recognition A.I. using computer vision and deep learning.

File Explorer

Download Latest Version (.zip)

- - CI.yml
- LSTM.h5
- LSTM_Attention.h5
- (2021) Human action recognition using attention based LSTM network with dilated CNN features.pdf
- (2022) A Novel CNN-based Bi-LSTM parallel model with attention mechanism for human activity recognition with noisy data.pdf
- (tutorial) Real-Time 3D Pose Detection & Pose Classification with Mediapipe and Python _ Bleed AI.pdf
- 2022_An_Attention-based_Hybrid_2D_3D_CNN-LSTM_for_Human_Action_Recognition.pdf
- feature_engineering.ipynb
.dockerignore
.gitignore
app.py
Dockerfile
environment.yml
ExerciseDecoder.ipynb
LICENSE
pose_tracking_full_body_landmarks.png
README.md
requirements.txt

# Installation Guide

1. Get the code

git clone https://github.com/chrisprasanna/Exercise_Recognition_AI

Downloads the entire project code from GitHub to your computer.

cd Exercise_Recognition_AI

Moves into the project folder you just downloaded.

2. Docker

Easy Recommended

Prerequisites

Git Needed to download the project code from GitHub.
Docker Desktop Needed to build and run containers. Install it and keep it running in the background.

- [x] Build a Docker Image

Type this command into your terminal and run it.

✅ Run docker compose ps to check the containers are Up. If the README mentions a port, open http://localhost:PORT in your browser.

Pulled directly from this repo's README.

3. Python

Easy

Prerequisites

Git Needed to download the project code from GitHub.
Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.

pip install -r requirements.txt

Installs the Python libraries listed in requirements.txt (or similar).

python app.py

Runs the Python script (or module).

✅ If it runs without errors and prints output in the terminal, it worked.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# :robot::video_camera: Building an AI for Real-Time Exercise Recognition using Computer Vision & Deep Learning :weight_lifting_man::muscle:

## Description
In this project, I designed an AI that uses webcam footage to accurately detect exercises in real time and counts reps. OpenCV is used to access the webcam on your machine, a pretrained CNN is implemented for real-time pose estimation, and custom deep learning models are built using TensorFlow/Keras to recognize what exercise is being performed. In addition, this project includes a guided data collection pipeline which is used to generate training data for the custom deep learning models. Using my data, the LSTM model achieved an accuracy score of 97.78% and a categorical cross-entropy loss of 1.51e-3 on the validation dataset. The attention-based LSTM achieved an accuracy score of 100% and a categorical cross-entropy loss of 2.08e-5 on the validation dataset. From that point, joint angles are extracted from the pose estimation coordinates and heuristics are used to track the exercise and count reps. Finally, visualization tools are included that display joint angles, rep counts, and probability distributions.

https://user-images.githubusercontent.com/88418264/176807706-960e19dd-4261-46f6-bdc0-cf8a6077cc82.mp4

## Web App
I made two web interfaces to use and interact with the Personal Fitness Trainer AI. The links and descriptions are provided below.

### Streamlit + Docker + Heroku + GitHub Actions (CI/CD)

[Web App Link (Heroku)](https://ai-personal-fitness-trainer.herokuapp.com/)

I used Streamlit, a Python library designed for people who are not expert web developers, to design an application to use the AI. Streamlit allows you to build data science applications without worrying too much about the UI design, which is all handled by the Streamlit API. I then constructed a Dockerfile that provides instructions to build a Docker image with the running application. The application was then deployed on the web using Heroku and their Docker Container Registry. Finally, I automated the deployment pipeline using GitHub Actions. I did this by designing a workflow to build the Docker image and push to Heroku's registry whenever I pushed changes to the main branch of this GitHub repository. Essentially, the workflow file automatically performs the same commands that I ran on my local machine: login to the Heroku container registry, build the Docker image, and deploy it to the web.

### Streamlit Cloud

[Web App Link (Streamlit Cloud)](https://chrisprasanna-exercise-recognition-ai-app-app-enjv7a.streamlitapp.com/)

I also deployed the AI directly from Streamlit to their cloud. This was quick and easy, however, the biggest downside of Streamlit cloud deployment is its speed issues. The entire Python script is re-run in the browser every time the user interacts with the application. I included the link to this application for documentation purposes but I would recommend you use the link from the previous section.

## Installation
- Download this repository and move it to your desired working directory
- Download Anaconda if you haven't already
- Open the Anaconda Prompt
- Navigate to your working directory using the cd command
- Run the following command in the Anaconda prompt:
	```
  	conda env create --name NAME --file environment.yml
  	```
	where NAME needs to be changed to the name of the conda virtual environment for this project. This environment contains all the package installations and dependencies for this project.
  
- Run the following command in the Anaconda prompt:
  	```
  	conda activate NAME
  	```
	This activates the conda environment containing all the required packages and their versions. 
  
- Open Anaconda Navigator
- Under the "Applications On" dropdown menu, select the newly created conda environment
- Install and open Jupyter Notebook. NOTE: once you complete this step and if you're on a Windows device, you can call the installed version of Jupyter Notebook within the conda environment directly from the start menu.  
- Navigate to the ExerciseDecoder.ipynb file within the repository

## Features

- Implementation of Google MediaPipe's BlazePose model for real-time human pose estimation
- Computer vision tools (i.e., OpenCV) for color conversion, detecting cameras, detecting camera properties, displaying images, and custom graphics/visualization 
- Inferred 3D joint angle computation according to relative coordinates of surrounding body landmarks
- Guided training data generation
- Data preprocessing and callback methods for efficient deep neural network training
- Customizable LSTM and Attention-Based LSTM models
- Real-time visualization of joint angles, rep counters, and probability distribution of exercise classification predictions

## To-Do

* Higher Priority
	- [x] Add precision-recall analysis
	- [x] Deploy the AI and build a web app
	- [x] Build a Docker Image
	- [x] Build a CI/CD workflow
	- [ ] Train networks using angular joint kinematics rather than xyz coordinates
	- [ ] Translate AI to a portable embedded system that you can take outdoors or at a commercial gym. Components may include a microcontroller (e.g., Raspberry Pi), external USB camera, LED screen, battery, and 3D-printed case
* Back-burner
	- [ ] Add AI features that can detect poor form (e.g., leaning, fast eccentric motion, knees caving in, poor squat depth, etc.) and offer real-time advice/feedback for 
	- [ ] Optimize hyperparameters based on minimizing training time and cross-entropy loss on the validation dataset
	- [ ] Add more exercise classes
	- [ ] Add additional models. For instance, even though BlazePose is a type of CNN, there may be benefits to including convolutional layers within the custom deep learning models

## Credits

- [MediaPipe Pose](https://google.github.io/mediapipe/solutions/pose.html) for the pretrained human pose estimation model
- [Nicholas Renotte](https://github.com/nicknochnack) for tutorials on real-time action detection and pose estimation
- [Philippe Rémy](https://github.com/philipperemy/keras-attention-mechanism) for the attention mechanism implementation for Keras

## License
[MIT](https://github.com/chrisprasanna/Exercise_Recognition_AI/blob/main/LICENSE)