firehose

(β˜… 343)

Firehose is an extensible, no-code, and cloud-native service to load real-time streaming data from Kafka to data stores, data lakes, and analytical storage systems.

File Explorer

Download Latest Version (.zip)
  • .gitignore
  • build.gradle
  • Dockerfile
  • gradle.properties
  • gradlew
  • gradlew.bat
  • LICENSE
  • lombok.config
  • README.md
  • settings.gradle

# Installation Guide

1. Get the code
git clone https://github.com/raystack/firehose

Downloads the entire project code from GitHub to your computer.

cd firehose

Moves into the project folder you just downloaded.

2. Docker

Easy Recommended
Prerequisites
  • Git Needed to download the project code from GitHub.
  • Docker Desktop Needed to build and run containers. Install it and keep it running in the background.
$ docker pull raystack/firehose

Type this command into your terminal and run it.

$ docker run -e SOURCE_KAFKA_BROKERS=127.0.0.1:6667 -e SOURCE_KAFKA_CONSUMER_GROUP_ID=kafka-consumer-group-id -e SOURCE_KAFKA_TOPIC=sample-topic -e SINK_TYPE=log -e SOURCE_KAFKA_CONSUMER_CONFIG_AUTO_OFFSET_RESET=latest -e INPUT_SCHEMA_PROTO_CLASS=com.github.firehose.sampleLogProto.SampleLogMessage -e SCHEMA_REGISTRY_STENCIL_ENABLE=true -e SCHEMA_REGISTRY_STENCIL_URLS=http://localhost:9000/artifactory/proto-descriptors/latest/raystack/firehose:latest

Runs the built image as an actual container.

βœ… Run docker compose ps to check the containers are Up. If the README mentions a port, open http://localhost:PORT in your browser.

Pulled directly from this repo's README.

3. Gradle (Java/Kotlin)

Medium
Prerequisites
  • Git Needed to download the project code from GitHub.
  • JDK (Java) Required to build and run Java/Kotlin projects.
  • Gradle If the repo includes gradlew (Gradle Wrapper), you don't need to install Gradle separately.
$ ./gradlew clean build

Builds the project using Gradle.

$ ./gradlew runConsumer

Builds the project using Gradle.

$ ./gradlew test

Builds the project using Gradle.

$ ./gradlew checkstyleMain checkstyleTest

Builds the project using Gradle.

$ ./gradlew clean

Builds the project using Gradle.

βœ… A BUILD SUCCESSFUL message means it worked. The output is created under build/.

Pulled directly from this repo's README.

Today's picks

superset β˜… 74,352

Apache Superset is a Data Visualization and Data Exploration Platform

Python
openai-cookbook β˜… 75,446

Examples and guides for using the OpenAI API

Jupyter Notebook
ui β˜… 121,773

A set of beautifully-designed, accessible components and a code distribution platform. Wor

TypeScript
browser-use β˜… 109,987

🌐 Make websites accessible for AI agents. Automate tasks online with ease.

Python
tech-interview-handbook β˜… 142,052

Curated coding interview preparation materials for busy software engineers

TypeScript
llm-course β˜… 81,890

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.
// repository documentation