storm-streaming

KO | EN

gitlite — search

#python #java #python3 #arduino #golang #machine-learning #rust #html #flask #javascript #seismology #nodejs

storm-streaming

★ 17 Open GitHub ↗

Sample Storm Topology for stream processing sensor data

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Storm-Streaming #

This is a sample storm topology for stream processing sensor data.  In this case, we're applying simple rules to events being generated by a trucking fleet.  The goal is to send email alerts to a dispatch center when a threshold of "Un-Safe" events have been generated by a specific truck/driver.

## Building ##

There are three dependencies to building this example:

1. Java 1.6
2. Maven
3. ActiveMQ
4. storm-jms

Dependencies 1, 2, and 3 can be downloaded and installed respectively. Once they've been installed and you can do a `mvn -version` from a command prompt, I'll cover how to build and install storm-jms, and then this project.

### Building Storm-JMS ######

I've forked @ptgoetz's storm-jms project, as one addition was needed to build the project.  A pull request has been issued and once it has been added, you can build directly from his repo.  For now, we'll go ahead and get it from my repo using the commands below:

git clone https://github.com/pcodding/storm-jms.git
	cd storm-jms
	mvn clean install
	
### Building Storm-Streaming ###

Now that the storm-jms dependency has been installed, we can build this project.  The process is very similar and is as follows:

git clone https://github.com/pcodding/storm-streaming.git
	cd storm-jms
	mvn clean install
	
## Project Explanation ##

This project uses the following classes to define the topology `com.hortonworks.streaming.impl.topologies.TruckEventProcessorTopology`.  It has one spout, `JmsSpout`, and one bolt, `TruckEventRuleBolt`.  The `TruckEventRuleBolt` class receives each `Tuple`, and processes it using the `TruckEventRuleEngine` class.  The `TruckEventRuleEngine` class stores each "Un-Safe" event for a driver and sends an email if 5 events have been captured.

## Project Configuration ##

The `src/main/resources/config.properties` has the following properties:

notification.email=you@yourdomain.com
	notification.subject=Unsafe Driving Alerts
	jms.host=sandbox
	jms.port=61616
	jms.notificationQueue=sensor_data
	jms.notificationTopic=sensor_events
	
The `notification.*` are related to the emails sent after the event threshold has been reached, and the `jms.*` configuration properties control the JMS connection and system details.  Make sure you edit the `notification.email` before running the project.

## Running the Project ##

To run the project, just use the `run.sh` shell script to compile and execute the project on a stand-alone machine.

## Simulating Data ##

What's a Storm topology without events?  Not too exciting.  I've created a data simulator that helps with simulating unsafe driving events among other things.  You can find it here, and this is the process to get started:

git clone https://github.com/pcodding/stream-simulator.git

Edit the `src/main/resources/config.properties` to point to your ActiveMQ installation, then build and run the project:

mvn clean package
    ./run.sh 6 -1 com.hortonworks.streaming.impl.domain.transport.Truck com.hortonworks.streaming.impl.collectors.JmsEventCollector

storm-streaming

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README