ClickHouse Kafka 引擎与物化视图摄入实践

---

title: ClickHouse Kafka 引擎与物化视图摄入实践

keywords: Kafka engine, materialized view, JSONEachRow, group_name, topics

description: 使用 Kafka 引擎与物化视图从 Kafka 主题摄入数据到 MergeTree，保证高吞吐与可查询性。

tags:

• ClickHouse
• JSONEachRow
• Kafka engine
• group_name
• materialized view
• topics
• 数据库
• 数据摄入

categories:

• 文章资讯
• 技术教程

---

Kafka 源表与目标表：

CREATE TABLE kafka_orders (
  id UInt64,
  sku String,
  qty UInt32
) ENGINE = Kafka SETTINGS 
  kafka_broker_list = 'kafka:9092',
  kafka_topic_list = 'orders',
  kafka_group_name = 'ck-consumer',
  kafka_format = 'JSONEachRow',
  kafka_num_consumers = 1;

CREATE TABLE orders (
  id UInt64,
  sku LowCardinality(String),
  qty UInt32,
  ts DateTime DEFAULT now()
) ENGINE = MergeTree()
PARTITION BY toYYYYMM(ts)
ORDER BY (ts, id);

物化视图落盘：

CREATE MATERIALIZED VIEW mv_orders TO orders AS
SELECT id, sku, qty, now() AS ts FROM kafka_orders;