Langfuse v3 Self-Hosted ClickHouseCalc Sizing: ZSTD(9) Prompt & Completion Traces
In our deterministic ClickHouseCalc model, Langfuse v3 stores verbose LLM system prompts, RAG context windows, and agent spans in ClickHouse where ZSTD(9) sliding windows deduplicate repeated prompt preambles across adjacent 8,192-row granules.
ClickHouseCalc Compression, RAM & TCO Schedule for langfuse-llm-trace-storage
| Ingestion Scale | Raw Volume | MergeTree NVMe Disk | Node RAM & vCPU | Modeled TCO Delta (2026-11-12) |
|---|---|---|---|---|
| 10M LLM Spans/Day (90d · 14.8 GB/d) | 1.33 TB Raw Prompts | 125 GB NVMe (ZSTD-9 + 22% Buffer) | 16 GB RAM / 4 vCPU | ~$590/mo Managed SaaS vs $10.49/mo Hostinger KVM 4 |
| 50M LLM Spans/Day (365d · 74 GB/d) | 27.0 TB Raw Prompts | 2.45 TB Enterprise Disk | 32–64 GB RAM / 12–16 vCPU | ~$3,900/mo Managed SaaS vs $58–$79/mo UltaHost / InterServer |
ClickHouseCalc Langfuse v3 `observations` ZSTD(9) MergeTree Schema
CREATE TABLE langfuse.observations_compressed
(
id String,
trace_id String CODEC(ZSTD(3)),
project_id LowCardinality(String),
type LowCardinality(String),
model LowCardinality(String),
start_time DateTime64(3) CODEC(DoubleDelta, LZ4),
input String CODEC(ZSTD(9)),
output String CODEC(ZSTD(9)),
prompt_tokens UInt32 CODEC(T64, LZ4),
completion_tokens UInt32 CODEC(T64, LZ4),
total_cost Decimal64(8) CODEC(ZSTD(3))
)
ENGINE = ReplacingMergeTree(start_time)
PARTITION BY toYYYYMM(start_time)
ORDER BY (project_id, type, toDate(start_time), id);
ClickHouseCalc MergeTree Production Checklist for langfuse-llm-trace-storage
- Set `LANGFUSE_S3_EVENT_UPLOAD_ENABLED=true` against a local MinIO or S3 bucket so multi-megabyte base64 vision payloads bypass synchronous MergeTree column blocks.
- Apply `CODEC(ZSTD(9))` specifically on `input` and `output` string columns to maximize dictionary compression on identical RAG prompt templates.
- Place `(project_id, type, toDate(start_time))` at the front of `ORDER BY` so token-spend billing aggregations scan narrow date ranges.
ClickHouseCalc Engineering FAQ: langfuse-llm-trace-storage
- Why does ClickHouseCalc model Langfuse v3 on ClickHouse MergeTree rather than PostgreSQL TOAST tables?
-
PostgreSQL TOAST storage incurs VACUUM bloat and row-level I/O amplification on 8 KB prompt JSON, whereas ClickHouse columnar storage aggregates `total_cost` and `model` columns without reading `input` or `output` `.bin` files.
- How does ClickHouseCalc address data-residency requirements for private enterprise Langfuse v3 deployments?
-
Deploying Langfuse v3 and ClickHouse on a dedicated single-tenant VDS or bare-metal server keeps prompt histories within your selected regional datacenter (Singapore, Frankfurt, or US) under your own encryption keys.