AutomEditor

KO | EN

gitlite — search

#javascript #hacktoberfest #react #ai #android #machine-learning #docker #java #nodejs #windows #cli #vue #awesome-list #pytorch

(★ 51)

AutomEditor is an AI based video editor that helps video bloggers to remove bloopers automatically. It uses multimodal spatio-temporal blooper recognition and localization approaches. The models were trained in keras and integrate feature fusion techniques from face, body gestures (skelethon), emotions progression, and audio features

AutomEditor Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

![Alt text](https://raw.githubusercontent.com/toxtli/AutomEditor/master/frontend/img/logo.png "AutomEditor logo")

# AutomEditor
AutomEditor is an AI based automatic video editing tool that helps video bloggers to remove bloopers automatically. It uses multimodal spatio-temporal blooper recognition and localization approaches. The models were trained in keras and integrate feature fusion techniques from face, body gestures (skelethon), emotions progression, and audio features.

![Alt text](https://raw.githubusercontent.com/toxtli/AutomEditor/master/frontend/img/img_diagram.png "AutomEditor diagram")

## Demo

http://www.carlostoxtli.com/AutomEditor/frontend/
(You need to link your own backend)

![Alt text](https://raw.githubusercontent.com/toxtli/AutomEditor/master/frontend/img/screenshot.png "AutomEditor screenshot")

## Instructions

### Requirements

Install the following software
 - ffmpeg
 - OpenSMILE
 - OpenFace
 - python 2 or 3
Place the OpenSMILE and OpenFace directories at the side of this cloned directory.
 
### Data preprocessing

Run the scripts in the following order
- ./backend/feature_extraction/extract_audio_files.py
- ./backend/feature_extraction/generate_audio_feature.py
- ./backend/feature_extraction/generate_face_feature.py
- ./backend/feature_extraction/generate_face_visual.py
- ./backend/feature_extraction/generate_emotion_feature.py
- ./backend/feature_extraction/generate_body_feature.py
- ./backend/feature_extraction/generate_body_visual.py
Copy all the .pkl files to the ./backend/data/ folder

### Training

For training individual models just give the specific model
* ./backend/experiment/train.py --model emotion_feature
For training fusion models just give the comma separated list of models and add the fusion parameter
* ./backend/experiment/train.py --fusion --model body_fusion,face_fusion,audio_feature,emotion_feature

### User interface

Run the web server
* ./backend/feature_extraction/server.py
Expose the frontend directory to the web, for instance:
* cd ./frontend/
* sudo python -m SimpleSTTPServer 80
Navigate the URL, for instance: http://localhost

## Architecture

![Alt text](https://raw.githubusercontent.com/toxtli/AutomEditor/master/frontend/img/img_model.png "AutomEditor Architecture")

## References

This work is based on this paper: https://arxiv.org/pdf/1805.00625.pdf

Slides: https://docs.google.com/presentation/d/1ds8nGurfxUgnizWwI61eLh8OTJ4l4K486LwPe38I0Eo

Publication in progress ...

AutomEditor

AutomEditor Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

AutomEditor

AutomEditor Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README