NMF

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

★ 8 Open GitHub ↗

Implementation of Projected gradient NMF

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Projected Gradient for Non-Negative Matrix Factorization

Reimplementing NMF algorithm proposed in C. Lin, <b>"Projected Gradient Methods for Nonnegative Matrix Factorization,"</b> in Neural Computation, vol. 19, no. 10, pp. 2756-2779, Oct. 2007.
doi: 10.1162/neco.2007.19.10.2756

## Introduction

Non-Negative Matrix Factorization (NMF) is a group of unsupervised machine learning techniques which factors a multivariate dataset as the approximate product of two lower dimension matrices. NMF is widely applied in various fields such as astronomy, computer and recommender systems. In this project we are interested in implementing NMF algorithm and analyze the robustness of the algorithm when the dataset is contaminated by different types of noise.

Our objective is to ﬁnd two non-negative matrix factors _W_ and _H_ which multiply to give as close as possible a representation of our original image _V_.

Where _W_ contains a basis optimized for the linear approximation of data in _V_, weighted by the components in _H_, i.e. Each column of _W_ is a basis element. If _V_ is our proverbial cake, then _W_ and _H_ are our list of core ingredients and their respective amounts.

More in depth information on NMF can be found at [Wikipedia](https://en.wikipedia.org/wiki/Non-negative_matrix_factorization).

With NMF, we are hoping that we can learn representation/latent factor of the images that we can reconstruct faces from any situation, in this case it will be reconstructing noisy face or even reconstruct face image in dark lighting condition.

## Projected Gradient methods for NMF

By altering the existing alternating non-negative least squares algorithm where it will alternatively fixes one matrix and improves the other.

Applying projected gradient method into alternating non-negative least squares algorithm we could get a faster convergence rate. Proposed projected gradient method can be seen in the pseudo code below:

![Figure1](https://github.com/MingSheng92/NMF/blob/master/pg_method.PNG)

For more information on the proofs and intuition of the algorithm, please kindly refer to the [research paper](https://www.csie.ntu.edu.tw/~cjlin/papers/pgradnmf.pdf).

## Results

As shown in the figure below, we manage to get a decent reconstuction(denoise) from the noisy images.

![Figure2](https://github.com/MingSheng92/NMF/blob/master/ProjectedNMF.jpg)

From the figure above, though we have managed to reconstruct the image but it seems to over-smooth the image. YaleB dataset has better reconstruction result, as I believe this is due to the dataset have perfectly centered face compared to ORL dataset. This suggests that we need some pre-processing on ORL to obatain a better result.

Part of the _W_(learned dictionary) for YaleB dataset is shown below.
![Figure3](https://github.com/MingSheng92/NMF/blob/master/learned_dict.jpg)

## How to use

Example on how to use program can be found at [PG-NMF_example.ipynb](https://github.com/MingSheng92/NMF/blob/master/PG-NMF_example.ipynb).

## Future work

Implement other different algorithms so that we can compare the robustness of different proposed approach.

## Additional information

https://angms.science/doc/NMF/nmf_pgd.pdf

latex generated with https://www.codecogs.com/latex/eqneditor.php

NMF

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README