clmtrackr

KO | EN

gitlite — search

#python #hacktoberfest #ai #typescript #llm #golang #machine-learning #nodejs #awesome #cli #kubernetes #vue #ios #mcp

(★ 6,500)

Javascript library for precise tracking of facial features via Constrained Local Models

File Explorer

Download Latest Version (.zip)

- clmtrackr.js
- clmtrackr.min.js
- clmtrackr.module.js
- - scale.fix.js
  - slimbox2.js
- - anim1.gif
  - annotated2_small.jpg
  - elements.png
  - elements2.png
  - emotions.jpg
  - emotions_neg.jpg
  - facemodel.png
  - facemodel_numbering_new.png
  - facemodel_numbering_new_small.png
  - facemodel_w_filters_crop1.jpg
  - header3.jpg
  - header_image3.jpg
  - packing.png
  - poisson_example.jpg
  - poisson_face.jpg
  - poisson_face2.jpg
  - post_deform2.jpg
  - post_deform3.jpg
  - post_facedeformation2.jpg
  - post_gender2.jpg
  - response_fig2b.jpg
  - training_patches.png
- - clm_pca.html
  - dat.gui.min.js
  - main.js
  - model_pca_20_svm.js
- - closelabel.gif
  - coderay.css
  - loading.gif
  - nextlabel.gif
  - prevlabel.gif
  - slimbox2.css
  - styles.css
- reference.html
- - emotion_classifier.js
  - emotionmodel.js
  - face_deformer.js
  - gendermodel.js
- - audrey.jpg
  - audrey_crop.jpg
  - average.png
  - average2_crop.jpg
  - bieber_small.jpg
  - bill-murray-snl_crop.jpg
  - cage.jpg
  - cage2_crop.jpg
  - cap12_edit.mp4
  - cap12_edit.webm
  - cap13_edit2.mp4
  - cap13_edit2.webm
  - chuck_crop.jpg
  - clmtrackr01.jpg
  - clmtrackr02.jpg
  - clmtrackr_03.jpg
  - facemodel_numbering_new.png
  - facemodel_numbering_new_small.png
  - fragrance-George-Clooney-main_crop.jpg
  - franck.mp4
  - franck.webm
  - franck_01829.jpg
  - franck_02045.jpg
  - franck_02159.jpg
  - franck_02221.jpg
  - franck_02683.jpg
  - franck_02888.jpg
  - franck_03909.jpg
  - franck_03979.jpg
  - franck_04000.jpg
  - franck_04035.jpg
  - franck_04117.jpg
  - icon_angry.png
  - icon_happy.png
  - icon_sad.png
  - icon_surprised.png
  - ironman.jpg
  - joconde.jpg
  - joconde_crop.jpg
  - Justin-Bieber2_crop.jpg
  - kim1_crop.jpg
  - obama4_crop.jpg
  - picasso_drawing_crop.jpg
  - queen20_crop.jpg
  - ri_1_crop.jpg
  - scream_crop.jpg
  - sean_guru2_crop.jpg
  - SeanConnery.jpg
  - skullmask.jpg
  - terminator_crop.jpg
  - vendetta.jpg
  - walter2_crop.jpg
- - bootstrap.min.css
  - border-h.gif
  - border-v.gif
  - imgareaselect-default.css
- caricature.html
- classviewer.html
- clm_emotiondetection.html
- clm_genderdetection.html
- clm_image.html
- clm_video.html
- clm_video_responses.html
- example.html
- face_deformation_video.html
- face_mask.html
- facedeform.html
- facesubstitution.html
- modelviewer_pca.html
- modelviewer_spca.html
- model_pca_10_mosse.js
- model_pca_10_svm.js
- model_pca_20_mosse.js
- model_pca_20_svm.js
- model_spca_10_svm.js
- model_spca_20_svm.js
- - faceDetection.js
  - faceDetection_worker.js
- - mosseFilterResponses.js
- - svmfilter_fft.js
  - svmfilter_webgl.js
- - debugging.js
  - events.js
  - fft.js
  - webgl_tests.js
- clm.js
.gitignore
LICENSE.txt
package-lock.json
package.json
README.md
rollup.config.js

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

clmtrackr
======

[![npm version](https://img.shields.io/npm/v/clmtrackr.svg)](https://www.npmjs.com/package/clmtrackr)

![tracked face](https://auduno.github.io/clmtrackr/examples/media/clmtrackr_03.jpg)

**clmtrackr** is a javascript library for fitting facial models to faces in videos or images. It currently is an implementation of *constrained local models* fitted by *regularized landmark mean-shift*, as described in [Jason M. Saragih's paper](http://dl.acm.org/citation.cfm?id=1938021). **clmtrackr** tracks a face and outputs the coordinate positions of the face model as an array, following the numbering of the model below:

[![facemodel_numbering](https://auduno.github.io/clmtrackr/examples/media/facemodel_numbering_new_small.png)](https://auduno.github.io/clmtrackr/examples/media/facemodel_numbering_new.png)

[Reference](http://auduno.github.io/clmtrackr/docs/reference.html) - [Overview](https://www.auduno.com/2014/01/05/fitting-faces/)

The library provides some generic face models that were trained on [the MUCT database](http://www.milbo.org/muct/) and some additional self-annotated images. Check out [clmtools](https://github.com/auduno/clmtools) for building your own models.

For tracking in video, it is recommended to use a browser with WebGL support, though the library should work on any modern browser.

For some more information about Constrained Local Models, take a look at Xiaoguang Yan's [excellent tutorial](https://sites.google.com/site/xgyanhome/home/projects/clm-implementation/ConstrainedLocalModel-tutorial%2Cv0.7.pdf?attredirects=0), which was of great help in implementing this library.

### Examples ###

* [Tracking in image](https://auduno.github.io/clmtrackr/examples/clm_image.html)
* [Tracking in video](https://auduno.github.io/clmtrackr/examples/clm_video.html)
* [Face substitution](https://auduno.github.io/clmtrackr/examples/facesubstitution.html)
* [Face masking](https://auduno.github.io/clmtrackr/examples/face_mask.html)
* [Realtime face deformation](https://auduno.github.io/clmtrackr/examples/facedeform.html)
* [Emotion detection](https://auduno.github.io/clmtrackr/examples/clm_emotiondetection.html)
* [Caricature](https://auduno.github.io/clmtrackr/examples/caricature.html)

### Usage ###

Download the minified library [clmtrackr.js](https://github.com/auduno/clmtrackr/raw/dev/build/clmtrackr.js), and include it in your webpage.

```html
/* clmtrackr libraries */
<script src="js/clmtrackr.js"></script>
```

The following code initiates the clmtrackr with the default model (see the [reference](http://auduno.github.io/clmtrackr/docs/reference.html) for some alternative models), and starts the tracker running on a video element.

```html
<video id="inputVideo" width="400" height="300" autoplay loop>
  <source src="./media/somevideo.ogv" type="video/ogg"/>
</video>
<script type="text/javascript">
  var videoInput = document.getElementById('inputVideo');
  
  var ctracker = new clm.tracker();
  ctracker.init();
  ctracker.start(videoInput);
</script>
```

You can now get the positions of the tracked facial features as an array via ```getCurrentPosition()```:

```html
<script type="text/javascript">
  function positionLoop() {
    requestAnimationFrame(positionLoop);
    var positions = ctracker.getCurrentPosition();
    // positions = [[x_0, y_0], [x_1,y_1], ... ]
    // do something with the positions ...
  }
  positionLoop();
</script>
```

You can also use the built in function ```draw()``` to draw the tracked facial model on a canvas :

```html
<canvas id="drawCanvas" width="400" height="300"></canvas>
<script type="text/javascript">
  var canvasInput = document.getElementById('drawCanvas');
  var cc = canvasInput.getContext('2d');
  function drawLoop() {
    requestAnimationFrame(drawLoop);
    cc.clearRect(0, 0, canvasInput.width, canvasInput.height);
    ctracker.draw(canvasInput);
  }
  drawLoop();
</script>
```

See the complete example [here](https://auduno.github.io/clmtrackr/examples/example.html).

### Development ###

First, install [node.js](http://nodejs.org/) with npm.

In the root directory of clmtrackr, run `npm install` then run `npm run build`. This will create `clmtrackr.js` and `clmtrackr.module.js` in `build` folder.

To test the examples locally, you need to run a local server. One easy way to do this is to install `http-server`, a small node.js utility: `npm install -g http-server`. Then run `http-server` in the root of clmtrackr and go to `https://localhost:8080/examples` in your browser.

### License ###

**clmtrackr** is distributed under the [MIT License](http://www.opensource.org/licenses/MIT)

clmtrackr

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

clmtrackr

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README