LSTM-AE-TS-anomaly-detection

KO | EN

gitlite — search

#python #hacktoberfest #react #ai #llm #machine-learning #deep-learning #linux #docker #macos #nodejs #awesome #pytorch #openai

LSTM-AE-TS-anomaly-detection

(★ 13)

An LSTM auto-encoder for Multivariate Cyclic Time Series (MCTS) anomaly detection. The code can be adapted to any MCTS but was applied on Wafer dataset.

LSTM-AE-TS-anomaly-detection Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Anomaly Detection using LSTM-based Auto-Encoder for Multivariate Time-Series
This project is aimed at implementing an LSTM-based auto-encoder for anomaly detection in multivariate and potentially cyclic time-series data. The dataset used in this project is from the field of semiconductor manufacturing and contains online process control measurements recorded by various sensors during the processing of silicon wafers. The dataset has been preprocessed and contains both normal and abnormal process samples. However, the focus of this project is not on the specific domain or physical process being monitored, but rather on developing a reproducible code for anomaly detection in multivariate and cyclic time-series.

## Auto-Encoders
Auto-encoders are neural networks that aim to reconstruct their input data, which are often images or sequences. They consist of two main parts: an encoder, which maps the input data to a lower-dimensional latent space representation, and a decoder, which maps the latent space representation back to the input data. During training, the network is trained to minimize the reconstruction error between the input data and the reconstructed data, often using mean squared error (MSE) as the loss function. Once the network is trained, it can be used to reconstruct new data, and the reconstruction error can be used as a measure of anomaly score.

In this project, I have utilized three different types of auto-encoder architectures, each with their own unique approach to data analysis:
- **LSTM-AE with overlapped windowing**: Used to analyze time-series data with overlapping windows, providing a more accurate representation of the underlying patterns.
- **LSTM-AE directly on the multivariate cycles**: Provides a more comprehensive understanding of the data in terms of cycles.
- **Deep convolutional auto-encoder**: Used to learn features directly from raw data using convolutional layers, providing a more efficient and accurate way of encoding the data.  
  
Each of these architecture have been applied on an independent jupyter notebook with good results for LSTM applied directly on multivariate cycles while having almost perfect results with convolutions. (See metrics below)

## Dataset
The "Wafer" dataset used in this project was formatted by Robert Olszewski as part of his doctoral thesis. It contains online process control measurements recorded by various sensors during the processing of silicon wafers for semiconductor manufacturing. The dataset contains both normal and abnormal process samples. There is a significant class imbalance, with abnormal samples representing 10.7% of the training set and 12.1% of the test set. The variables recorded by the sensors are not explicitly labeled and are likely a combination of multiple variables.

## Normal cycle vs anomaly

<img src="images/normal_vs_anomaly.png">

## Cycle windowing with overlap
In order to boost the encoding of temporal dependency, a good practice in LSTM data preprocessing is to segment the sequences into windows with overlap. We applied this method by using a subsequence length of 12 and a gap of 4 for each cycle of length 152 for the wafer dataset usecase.

<img src="images/cycle_vs_windowed.png">

## Training and validation loss functions (MAE)
**To remind you: The auto-encoder was trained solely on the normal cycles available in the training set. None of the anomlies were introduced during training!**

<img src="images/loss_function.png">

## Reconstructed cycle
By training an LSTM auto-encoder with windowing on the training set, we obtain a windowed reconstruction with overlaps. In order to get rid of the overlap by keeping maximum information, I have applied an average over every overlap and obtained the below reconstruction.

<img src="images/reconstruction.png">

## Anomaly detection
To identifies anomalies, we will calculate the MAE per cycle using the reconstructed cycles. In principles, the model will struggle at reconstructing anomalies at least slightly more than with normal cycles since he is less familiar with them.

<img src="images/reconstruction_error.png">

# Evaluation metrics on test set

<img src="images/evaluation_on_test.PNG">

## Getting Started
To use this repository, you'll need to have the following prerequisites installed:

- Python 3  
- NumPy  
- TensorFlow

You can install the dependencies using pip:

pip install -r requirements.txt
        
## References
- Olszewski, R. T. "Generalized feature extraction for structural pattern recognition in time-series data". No. CMU-CS-01-108. Carnegie Mellon University, School of Computer Science, 2001. [URL](https://dl.acm.org/citation.cfm?id=935627)  
- Malhotra, P., Vig, L., Shroff, G., & Agarwal, P. (2016). LSTM-based encoder-decoder for multi-sensor anomaly detection. arXiv preprint arXiv:1607.00148.

LSTM-AE-TS-anomaly-detection

LSTM-AE-TS-anomaly-detection Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

LSTM-AE-TS-anomaly-detection

LSTM-AE-TS-anomaly-detection Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README