Cloudflare HTTP Logpush to ClickHouse: ClickHouseCalc WAF & Edge Telemetry Sizing
In our deterministic ClickHouseCalc model, piping gzipped Cloudflare HTTP Logpush NDJSON streams through Vector into MergeTree packs structured status codes, client ASNs, RayIDs, and WAF actions at >11.5x ZSTD(3) compression without per-GB SIEM ingestion fees.
ClickHouseCalc Compression, RAM & TCO Schedule for cloudflare-http-logpush-sink
| Ingestion Scale | Raw Volume | MergeTree NVMe Disk | Node RAM & vCPU | Modeled TCO Delta (2026-11-12) |
|---|---|---|---|---|
| 100M Edge Hits/Day (30d · 92 GB/d) | 2.76 TB Raw NDJSON | 265 GB NVMe (ZSTD-3 + 22% Buffer) | 16–32 GB RAM / 8 vCPU | ~$1,480/mo Cloud SIEM vs $19.99/mo Hostinger KVM 8 |
| 500M Edge Hits/Day (90d · 460 GB/d) | 41.4 TB Raw NDJSON | 3.95 TB Enterprise Disk | 64 GB RAM / 16 vCPU | ~$8,900/mo Cloud SIEM vs $79.00/mo InterServer Ryzen |
ClickHouseCalc Cloudflare HTTP Logpush MergeTree DDL & `IPv6` Schema
CREATE TABLE edge_logs.cloudflare_http
(
EdgeStartTimestamp DateTime64(3) CODEC(DoubleDelta, LZ4),
ClientIP IPv6 CODEC(ZSTD(3)),
ClientASN UInt32 CODEC(T64, LZ4),
ClientCountry LowCardinality(FixedString(2)),
ClientRequestHost LowCardinality(String),
ClientRequestMethod LowCardinality(String),
ClientRequestURI String CODEC(ZSTD(3)),
EdgeResponseStatus UInt16 CODEC(T64, LZ4),
EdgeResponseBytes UInt64 CODEC(T64, ZSTD(1)),
CacheCacheStatus LowCardinality(String),
WAFAction LowCardinality(String),
RayID FixedString(16) CODEC(ZSTD(1))
)
ENGINE = MergeTree
PARTITION BY toDate(EdgeStartTimestamp)
ORDER BY (ClientRequestHost, CacheCacheStatus, EdgeResponseStatus, toStartOfMinute(EdgeStartTimestamp));
ClickHouseCalc MergeTree Production Checklist for cloudflare-http-logpush-sink
- Normalize both IPv4 and IPv6 client addresses into a single 16-byte `IPv6` column (`toIPv6(ClientIP)`) for binary CIDR range filtering without regex overhead.
- Enable `async_insert = 1` and `wait_for_async_insert = 0` on a scoped `logpush_ingest` user when accepting direct HTTPS micro-batches.
- Attach an incremental Materialized View summarizing per-minute 5xx status counts and edge bandwidth so Grafana panels read pre-aggregated kilobytes.
ClickHouseCalc Engineering FAQ: cloudflare-http-logpush-sink
- When does ClickHouseCalc recommend lightweight Rust Vector disk buffers instead of Kafka for Cloudflare Logpush?
-
Below ~300M daily HTTP requests, running a Rust `Vector` sink with a 20 GB local NVMe disk buffer absorbs ClickHouse restart windows cleanly without operating a separate JVM/Kafka broker tier.
- How does ClickHouseCalc accelerate 90-day attacker ASN forensic lookups in Cloudflare MergeTree logs?
-
Adding a secondary data-skipping index `INDEX asn_idx ClientASN TYPE set(1000) GRANULARITY 4` lets ClickHouse skip granules where the target ASN never appeared.