StreamingLogisticRegression

KO | EN

gitlite — search

#typescript #llm #docker #nodejs #macos #java #windows #awesome #kubernetes #vue #awesome-list #pytorch #ios #open-source

StreamingLogisticRegression

(★ 20)

Logistic Regression in Spark Streaming with Online Updating

StreamingLogisticRegression 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# README or not

![Image of Pipeline](https://raw.githubusercontent.com/keiraqz/StreamingLogisticRegression/master/flask/app/static/img/pipeline.png)
![model](https://raw.githubusercontent.com/keiraqz/StreamingLogisticRegression/master/flask/app/static/img/model_accuracy.png)

## Train offline model in Spark

```
# save model to file
$SPARK_HOME/bin/spark-submit spark_training.py
```

- shortcut for streaming demo

```
bash run_streaming.sh
python flask/run.py
```

## Run the streaming example

-  Start zookeeper & Kafka

```
sudo /usr/local/zookeeper/bin/zkServer.sh start 
sudo /usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties &
```

- Start Redis

```
redis-server
```

- Start Spark Streaming

```
$SPARK_HOME/bin/spark-submit stream_predict.py
```

- Start Kafka Producer

```
python auto_producer.py
```

- Start flask app

```
cd flask
python run.py

# in the browser
localhost:5000
```

-  Stop zookeeper & Kafka

```
sudo /usr/local/kafka/bin/kafka-server-stop.sh
sudo /usr/local/zookeeper/bin/zkServer.sh stop
```

## Tools and Dependncies
- Scala 2.11
- Spark 2.0.1
- Zookeeper 3.4.8
- Kafka 0.9
- Redis 2.6.9

- for Spark Streaming
	- spark-streaming-kafka-0-8_2.11-2.0.1.jar
		- kafka_2.11-0.8.2.2.jar
			- metrics-core-2.2.0.jar

- for Python (pip install)
	- kafka-python==1.0.2
	- redis==2.10.5

### Install Tools

- Spark 2.0.1

Download: http://spark.apache.org/downloads.html

- Others

For installing and configuring Zookeeper, Kafka and Redis, please refer to
[install_tools.sh](https://github.com/keiraqz/SparkModeling/blob/master/install_tools.sh)

## Dataset

Mohammad, Rami, McCluskey, T.L. and Thabtah, Fadi Abdeljaber. (2015). Phishing Websites Data Set. UCI Machine Learning Repository [https://archive.ics.uci.edu/ml/datasets/Phishing+Websites#].

StreamingLogisticRegression

StreamingLogisticRegression 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

StreamingLogisticRegression

StreamingLogisticRegression 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README