KO
|
EN
gitlite โ search
Search
#python
#android
#nodejs
#php
#mongodb
#docker
#hacktoberfest
#ruby
#rust
#kotlin
#buildkite
#example
clickhouse-kafka-connect
โ 209
Open GitHub โ
ClickHouse Kafka Connector
Download README (.md)
Explore Similar Repositories
cubo
:
On-Demand Earth System Data Cubes (ESDCs) in Python
tbparse
:
Load tensorboard event logs as pandas DataFrames for scientific plotting; Supports both PyTorch and TensorFlow
helyim
:
seaweedfs implemented in pure Rust
ClassGPT
:
ChatGPT for lecture slides
What-to-eat-today
:
๐ฝ๏ธๅบไบๅพRAGๆๆฏ็AI็พ้ฃๆจ่ๅฉๆ - Datawhale all-in-ragๆ็จๅฎๆๆกไพ๏ผ้ๆNeo4jๅพๆฐๆฎๅบใMilvusๅ้ๆฃ็ดขไธๆบ่ฝๅฏน่ฏ็ณป็ป
// repository documentation
Was this content helpful?
โ 0
(0 ratings)
Select Rating:
โ
โ
โ
โ
โ
Submit Feedback
Recent Feedback
×
Download README
Do you want to download the
README.md
file for
clickhouse-kafka-connect
?
Download (.md)
# ClickHouse Kafka Connect Sink ## About clickhouse-kafka-connect is the official Kafka Connect sink connector for [ClickHouse](https://clickhouse.com/). The Kafka connector delivers data from a Kafka topic to a ClickHouse table. ## Documentation See the [ClickHouse website](https://clickhouse.com/docs/en/integrations/kafka/clickhouse-kafka-connect-sink) for the full documentation entry. ## Design For a full overview of the design and how exactly-once delivery semantics are achieved, see the [design document](./docs/DESIGN.md). ## Help For additional help, please [file an issue in the repository](https://github.com/ClickHouse/clickhouse-kafka-connect/issues) or raise a question in [ClickHouse public Slack](https://clickhouse.com/slack). ## KeyToValue Transformation We've created a transformation that allows you to convert a Kafka message key into a value. This is useful when you want to store the key in a separate column in ClickHouse - by default, the column is `_key` and the type is String. ```sql CREATE TABLE your_table_name ( `your_column_name` String, ... ... ... `_key` String ) ENGINE = MergeTree() ``` Simply add the transformation to your connector configuration: ```properties transforms=keyToValue transforms.keyToValue.type=com.clickhouse.kafka.connect.transforms.KeyToValue transforms.keyToValue.field=_key ``` ## Performance Testing There is a dedicated gradle project in this repo - `benchmark` for performance testing. Please see its [README](./benchmark/README.md) for more information and how to run.
