Gesture-Controller

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

Gesture-Controller

★ 9 Open GitHub ↗

Developed python exe setup using tkinter to fully control mouse & keyboard movements by human hand and eye gestures. It works with the help of different libraries and packages like OpenCV, Mediapipe, Cvzone, Dlib, Haar Cascade, etc.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Gesture-Controller

## Team Member:-
* **Meet Variya** - https://github.com/meetvariya

## Problems:-
* Many of the Disabled People find it difficult to write using keyboard ,Also in Covid Situation many people don’t want to touch stuff that’s related to Covid.
* Sometimes when Mouse or Keyboard doesnt Work Properly Incase of a laptop is a Difficulty.

## Solution (Overview of project):-
* We make a Gesture Based Keyboard Interface using Webcam so that one can type things in a new Interactive way.
The Way it Works is using your Hand gesture and Eye tracking to use the Gui keyboard and mouse to Interact With it.

## Key Features:-
* Developed python exe file setup using tkinter to fully control mouse & keyboard movements by human hand & eye gestures.
* It works with the help of different libraries and packages like OpenCV, Mediapipe, Cvzone, Dlib, Haar Cascade, etc.
* Also included Voice & basic storage features. Helpful in such a circumstances like COVID-19.

## Working:-

Model Detects Face and Hand points using following modules:

### dlib's 68 points facial detection:

* The mouth can be accessed through points [48, 68].
* The right eyebrow through points [17, 22].
* The left eyebrow through points [22, 27].
* The right eye using [36, 42].
* The left eye with [42, 48].
* The nose using [27, 35].
* And the jaw via [0, 17].

### HandtrackingModule's 21 points in hand:

* 4,8,12,16,20 - top of all five fingers, thumb top to pinky fingure top respectively.

Different packages:

cvzone's HandTrackingModule to detect hands.
Dlib's shape predictor(shape_predictor_68_face_landmarks.dat) and haarcascade classifier(frontal_face_detcetor.xml) to detects face 
imutils to fetch eyes landmarks from dlib's 68 points facial detection.
Eye-aspect-ratio(EAR) for detect blinking.
pynput for controlling keyboard events.
pyautogui for mouse events

## Dependencies:-

* CvZone
* Pynput
* HaarCascade
* PyAutoGui
* Tkinter
* Mediapipe
* auto-py-to-exe
* Dlib

## References:-
* > ###  [Demo Link](https://drive.google.com/file/d/1swmmA05yG83uVJiyZJT6lMpWhf7o_nkY/view?usp=sharing)
* > ###  [Reference link for .dat file](https://github.com/italojs/facial-landmarks-recognition/blob/master/shape_predictor_68_face_landmarks.dat)

https://user-images.githubusercontent.com/49865067/159437765-fd7fbaf9-934e-4f2a-9d2b-ac7c19b0cdfa.mp4

Gesture-Controller

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README