hoidla

KO | EN

gitlite — search

#php #javascript #java #shell #python #linux #go #c-plus-plus #dotfiles #macos #game #mysql

★ 74 Open GitHub ↗

Set of real time stream processing algorithms that can be used by big data streaming platform

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

## Introduction
Set of reusable big data real time streaming algorithms. Can be used by  Spark Streaming, Storm 
or any other stream computation framework

## Philosophy
* Plain java API that can be used from any stream computation framework

## Blogs
The following blogs of mine are good source of details. These are the only source
of detail documentation
* http://pkghosh.wordpress.com/2014/09/10/realtime-trending-analysis-with-approximate-algorithms/
* http://pkghosh.wordpress.com/2014/10/05/tracking-web-site-bounce-rate-in-real-time/
* https://pkghosh.wordpress.com/2015/02/19/real-time-detection-of-outliers-in-sensor-data-using-spark-streaming/
* https://pkghosh.wordpress.com/2016/09/19/alarm-flooding-control-with-event-clustering-using-spark-streaming/

## Solution
* Probabilstic frequent count with sketches and count based algorithms
* Probabilstic cardinality or unique item count
* Probabilstic set inclusion 
* Different sampling methods
* Windowing including simple stats
* Pattern detection
* Event cluster detection

## Getting started
Project's resource directory has various tutorial documents for the use cases described in the blogs.

## Help
Please feel free to email me at pkghosh99@gmail.com

hoidla

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README