ClickHouse Incremental Materialized Views: ClickHouseCalc `AggregatingMergeTree` Sizing
In our deterministic ClickHouseCalc model, ClickHouse Materialized Views operate as insert-time stream triggers—transforming incoming tick batches into compact `AggregatingMergeTree` state blocks before raw rows even merge on disk.
ClickHouseCalc Compression, RAM & TCO Schedule for materialized-views-realtime-bi
| Ingestion Scale | Raw Volume | MergeTree NVMe Disk | Node RAM & vCPU | Modeled TCO Delta (2026-11-12) |
|---|---|---|---|---|
| 100M Ticks/Day + 4 MVs (365d · 38 GB/d) | 13.87 TB Raw Ticks | 1.35 TB Gen4 NVMe (LZ4 + DoubleDelta) | 32 GB RAM / 8 vCPU | ~$1,950/mo Cloud Warehouse vs $37.50–$58/mo UltaHost VDS |
| 1B Ticks/Day + 8 MVs (365d · 380 GB/d) | 138.7 TB Raw Ticks | 10.6 TB NVMe (ZSTD-3 + 22% Buffer) | 64–128 GB RAM / 24 vCPU | ~$11,400/mo Cloud Warehouse vs $109.00/mo InterServer Ryzen |
ClickHouseCalc 1-Minute OHLCV `AggregatingMergeTree` & Trigger View SQL
CREATE TABLE market.ohlcv_1m_target
(
symbol LowCardinality(String),
bucket DateTime CODEC(DoubleDelta, LZ4),
open_price AggregateFunction(argMin, Float64, DateTime64(9)),
high_price SimpleAggregateFunction(max, Float64),
low_price SimpleAggregateFunction(min, Float64),
close_price AggregateFunction(argMax, Float64, DateTime64(9)),
volume SimpleAggregateFunction(sum, Float64)
)
ENGINE = AggregatingMergeTree
PARTITION BY toYYYYMM(bucket)
ORDER BY (symbol, bucket);
CREATE MATERIALIZED VIEW market.ohlcv_1m_mv TO market.ohlcv_1m_target AS
SELECT
symbol,
toStartOfMinute(exchange_ts) AS bucket,
argMinState(price, exchange_ts) AS open_price,
max(price) AS high_price,
min(price) AS low_price,
argMaxState(price, exchange_ts) AS close_price,
sum(qty) AS volume
FROM market.raw_ticks
GROUP BY symbol, bucket;
ClickHouseCalc MergeTree Production Checklist for materialized-views-realtime-bi
- Batch source inserts (`>= 10,000` rows per block) because every insert block into `raw_ticks` synchronously emits a corresponding aggregation part into each attached Materialized View.
- Enable `parallel_view_processing = 1` on 8+ vCPU servers so chained Materialized Views execute across cores in parallel.
- Pair a shorter 14-to-30-day TTL on raw nanosecond ticks with multi-year retention on the compact `ohlcv_1m_target` rollup table.
ClickHouseCalc Engineering FAQ: materialized-views-realtime-bi
- Why does ClickHouseCalc require `-Merge` combinators when querying `AggregatingMergeTree` rollup tables?
-
Because background part compactions run asynchronously, multiple unmerged parts may hold partial states for the same `(symbol, bucket)` key; wrapping queries with `GROUP BY symbol, bucket` and `argMinMerge()` finalizes exact results at read time.
- When does ClickHouseCalc favor `LZ4` over `ZSTD(3)` on hot financial tick ingestion partitions?
-
For ultra-hot 7-day partitions ingesting monotonic nanosecond timestamps and prices, `DoubleDelta` already strips numeric redundancy while `LZ4` minimizes insert CPU overhead.