head-pulse-track

KO | EN

gitlite — search

#javascript #hacktoberfest #react #ai #golang #go #rust #deep-learning #linux #windows #cli #kubernetes #ios #mcp

head-pulse-track

(★ 21)

Detecting Pulse from Head Motions in Video

head-pulse-track 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Detecting Pulse from Head Motions in Video
This is the python implementation of this [paper](https://people.csail.mit.edu/mrub/vidmag/papers/Balakrishnan_Detecting_Pulse_from_2013_CVPR_paper.pdf)

## Explanation
This is abstract directly quated from paper
> We extract heart rate and beat lengths from videos by
measuring subtle head motion caused by the Newtonian
reaction to the influx of blood at each beat. Our method
tracks features on the head and performs principal compo-
nent analysis (PCA) to decompose their trajectories into a
set of component motions. It then chooses the component
that best corresponds to heartbeats based on its temporal
frequency spectrum. Finally, we analyze the motion pro-
jected to this component and identify peaks of the trajecto-
ries, which correspond to heartbeats. When evaluated on
18 subjects, our approach reported heart rates nearly iden-
tical to an electrocardiogram device. Additionally we were
able to capture clinically relevant information about heart
rate variability.

![project overview](https://raw.githubusercontent.com/irfan798/head-pulse-track/master/data/overview.png)

## Warning
Base of this implementation is made in 3 days for a hackathon, so there are still plenty of bugs and ugly hacks.

__Sampling Frequency__ or __FPS__ for video is important for getting accurate results from FFT and Bandpass filter.

All push requests and questions are welcome.

# Usage

For using algorithm from paper:
```bash
python main.py --graph 
```
same as
```bash
python main.py -c 0 --face_detector haar --graph 
```

If you want use __improved version__ with face landmarks:
```
python main.py --face_detector face_shape
```

If you want to use video file instead:
```
python main.py -f path
```

If you want to test modules:
```
python modules/face.py
python modules/tracking.py
```

# Project Structure

Project consists of 3 modules for now:

* Face - FacePoints
* tracking - TrackPoints
* signal_processing - SignalProcess

```bash
├── main.py
├── modules
│   ├── face.py
│   ├── signal_processing.py
│   └── tracking.py
├── papers
│   ├── Balakrishnan_Detecting_Pulse_from_2013_CVPR_paper.pdf
│   ├── electronics-08-00663.pdf
├── README.md
├── requirements.txt
├── tests
│   ├── common.py
│   ├── dlib_dedect.py
│   ├── fft.ipynb
│   ├── lk_track.py
│   ├── signal_pross.ipynb
│   └── vid_test.py
├── data
│   ├── haarcascade
│   │   ├── ***
│   └── shape_predictor_68_face_landmarks.dat

```

## __FacePoints__
This module is for detecting face and getting trackable points on face. You can select detector with *dedector_type* argument.

1.  __haar__ 
   * OpenCV's Haar face detection, gets face rectangle and rescales, then removes eyes from area
   * Finds optically trackable points in that filtered area
   * Fastest one
  2.  __dlib__
   * Same as Haar, but this time uses Dlib's face detection for face rectangle
   * More accurate
  3.  __face_shape__
   * Finds 68 face landmarks on face. Then removes eyes and mount landmark points for getting more stabilized data
   * Slow but more accurate

## __TrackPoints__
This module is responsible for tracking points throughout video.

Takes
`face_detector, max_trace_num=150, max_trace_history=60` as input.

If `Haar` or `Dlib` is used on face detection then tracker checks every point with backtracking. If a point is not consistent with forward and backtracking, then trace is marked as faulty and and gets deleted.

On every update non-faulty traces are counted. 
If their number is not equal to `max_trace_num` then new trackable points are requested from `face_detector`.
Higher the trace number, higher the CPU power needed.

If `face_shape` is used on face detection, then initial landmark points are tracked again with opticalflow. 
If a point on face landmarks marked as faulty then face landmarks are requested from `face_detector` but only faulty points are replaced.

Because face landmarks only gives out integer coordinates, not using opticalflow will result losing details.
Opticalflow tracks points on subpixel level and returns float coordinates.

## __SignalProcess__
This module is responsible for detecting heart beat from tracked signals.

Initially, only Y components are selected from tracked traces and then filtered with 5th order butterworth bandpass filter to get only frequencies between [0.75Hz, 3Hz] -> (45bpm, 180bpm)

If `Haar` or `dlib` used as getting initial points, then PCA is used to change dimension of traces down to 5.

Dominant Frequency is found for each 5 signals with FFT. Then one of those dominant frequencies is evaluated as most periodic by checking total spectral power accounted for by the dominant frequency

### __IF face_shape is used__
Instead of selecting individual traces with PCA, mean of traces are calculated with assumption of each landmark point has equal movement.
Then dominant frequency is found by appliying FFT to the mean signal.

## TODO:
- [x] Make main with source selection and argument parsing
- [x] Make signal_processing a module
- [ ] Output BPM into a file
- [ ] Make plotting faster
- [ ] For much better signal processing, add timestamp to every points tracked location in __TrackPoints__
- [ ]  For live feed, first get real fps with calculating time between every frame, then use it as __sampling rate__

head-pulse-track

head-pulse-track 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

head-pulse-track

head-pulse-track 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README