This guide shows you how to deploy ClickHouse on Kubernetes using the Altinity ClickHouse Operator and the VCS-managed Kustomize overlay shipped in the repository. The deployment includes ClickHouse server, ClickHouse Keeper, and a Prometheus-compatible metrics exporter, all driven from a single environment file.
Single source of truth
Everything that varies between deployments lives in clickhouse/env/config.env: topology, image references, resource limits, storage class, and storage size. Avoid editing the raw manifests. Update config.env, then re-apply.
Prerequisites
Before you begin, confirm that:
- You have a Kubernetes cluster on a recent supported version with
kubectlconfigured against it. - You have permissions to create namespaces, custom resource definitions, and cluster-scoped operator resources.
- The cluster does not already have an Altinity ClickHouse Operator installed. The bundle in this repository will conflict with an existing operator.
- Your storage class supports the disk size set in
SERVER_STORAGE_SIZE(default 400 Gi per replica) and is bound to a provisioner that produces persistent volumes.
Reviewing config.env Before Deploying
Open clickhouse/env/config.env and review the values that drive the deployment. The most important sections are:
| Group | Variables | Default |
|---|---|---|
| Topology | SHARDS, REPLICAS, KEEPER_REPLICAS | 1 / 1 / 1 |
| Images | CLICKHOUSE_SERVER_IMAGE, CLICKHOUSE_KEEPER_IMAGE, OPERATOR_IMAGE, METRICS_EXPORTER_IMAGE | Pinned in config.env |
| Server resources | SERVER_CPU_REQUEST, SERVER_CPU_LIMIT, SERVER_MEM_REQUEST, SERVER_MEM_LIMIT | 1 / 2 / 4Gi / 8Gi |
| Keeper resources | KEEPER_CPU_REQUEST, KEEPER_CPU_LIMIT, KEEPER_MEM_REQUEST, KEEPER_MEM_LIMIT | 500m / 1 / 1Gi / 2Gi |
| Storage | SERVER_STORAGE_SIZE, SERVER_STORAGECLASS, KEEPER_STORAGE_SIZE, KEEPER_STORAGECLASS | 400Gi / standard / 25Gi / standard |
| Pull secret | PULL_SECRET_NAME | Empty (public registries only) |
The defaults target development
The default topology is one shard, one replica, and one Keeper. This is appropriate for development and smoke testing only. For production, plan replicas and Keeper count up front. Keeper must run with an odd number of replicas so the cluster can hold a quorum.
Reviewing User Credentials
The default credentials are stored as SHA-256 hashes in clickhouse/secrets/chi-users.env:
- default user:
default/default123 - admin user:
admin/admin456
Replace these hashes with values you have generated yourself before deploying to anything that is not a local evaluation cluster. Both passwords appear in connection strings used by Countly and Kafka Connect, so rotating them later requires coordinated updates.
Deploying the Stack
From the repository root, apply the ClickHouse overlay. This installs the Altinity operator, Keeper, and ClickHouse server in the clickhouse namespace, and is fully declarative:
kubectl apply -k clickhouse/
The overlay applies in this order:
- The
clickhousenamespace. - The Altinity ClickHouse Operator bundle.
- A ClickHouse Keeper custom resource (
chk.yaml) with inline configuration for console logging and metrics. - A ClickHouse installation custom resource (
chi.yaml) with the Altinity metrics-exporter sidecar.
Verifying the Initial Deployment
Watch pods come online and confirm Ready status:
kubectl -n clickhouse get pods
Expected outcome:
- ClickHouse pods report
2/2 Ready(server + metrics-exporter sidecar). - Keeper pods report
1/1 Ready. - All pods are in
Runningstate.
Quick functional check using the default user:
kubectl -n clickhouse port-forward svc/ch 8123:8123 & curl -s "http://default:default123@localhost:8123/" -d "SELECT version()" kill %1
A version string in the response confirms the cluster is up and accepting queries.
Deployment is up
With pods Ready and the version query returning a result, the cluster is ready for the rest of the Countly stack. Run the full validation checklist documented separately to confirm replication, metrics, and storage are healthy before depending on the cluster in production.
Connecting Countly to This Cluster
Countly reads its ClickHouse password from the secret managed by the Kafka and Countly overlays. By default, the same password is used for three consumers:
- Countly application services
- The ClickHouse default user
- Kafka Connect's ClickHouse sink
Keep these in sync when you rotate the ClickHouse default user password. The recommended Secret Manager naming convention is <customer>-clickhouse-password.
Common Issues and Gotchas
Read the operator logs first:
kubectl -n clickhouse logs deployment/clickhouse-operator -c clickhouse-operator
Confirm the Altinity CRDs were installed:
kubectl get crd | grep clickhouse
Then confirm the storage class named in SERVER_STORAGECLASS exists in the cluster.
A single-Keeper deployment works without quorum negotiation. A three-Keeper cluster needs time to stabilize and may briefly report no leader. Check Keeper logs for both pods:
kubectl -n clickhouse logs chk-clickhouse-keeper-keeper-0-0-0 kubectl -n clickhouse logs chk-clickhouse-keeper-keeper-0-1-0
Verify network policies allow inter-pod communication on Keeper's coordination port.
Confirm the metrics-exporter sidecar is up and listening on its port:
kubectl -n clickhouse logs chi-ch-prod-main-0-0-0 -c metrics-exporter --tail=20
Then confirm the service exposes the metrics port and that ServiceMonitor labels match the labels on the service. Test the endpoint directly with port-forward before debugging Prometheus.
standard storage class is not present on every Kubernetes distribution. Check what is available with kubectl get storageclass and update SERVER_STORAGECLASS and KEEPER_STORAGECLASS in config.env accordingly, then re-apply.Related Resources
This guide shows you how to deploy ClickHouse on Kubernetes using the Altinity ClickHouse Operator and the VCS-managed Kustomize overlay shipped in the repository. The deployment includes ClickHouse server, ClickHouse Keeper, and a Prometheus-compatible metrics exporter, all driven from a single environment file.
Single source of truth
Everything that varies between deployments lives in clickhouse/env/config.env: topology, image references, resource limits, storage class, and storage size. Avoid editing the raw manifests. Update config.env, then re-apply.
Prerequisites
Before you begin, confirm that:
- You have a Kubernetes cluster on a recent supported version with
kubectlconfigured against it. - You have permissions to create namespaces, custom resource definitions, and cluster-scoped operator resources.
- The cluster does not already have an Altinity ClickHouse Operator installed. The bundle in this repository will conflict with an existing operator.
- Your storage class supports the disk size set in
SERVER_STORAGE_SIZE(default 400 Gi per replica) and is bound to a provisioner that produces persistent volumes.
Reviewing config.env Before Deploying
Open clickhouse/env/config.env and review the values that drive the deployment. The most important sections are:
| Group | Variables | Default |
|---|---|---|
| Topology | SHARDS, REPLICAS, KEEPER_REPLICAS | 1 / 1 / 1 |
| Images | CLICKHOUSE_SERVER_IMAGE, CLICKHOUSE_KEEPER_IMAGE, OPERATOR_IMAGE, METRICS_EXPORTER_IMAGE | Pinned in config.env |
| Server resources | SERVER_CPU_REQUEST, SERVER_CPU_LIMIT, SERVER_MEM_REQUEST, SERVER_MEM_LIMIT | 1 / 2 / 4Gi / 8Gi |
| Keeper resources | KEEPER_CPU_REQUEST, KEEPER_CPU_LIMIT, KEEPER_MEM_REQUEST, KEEPER_MEM_LIMIT | 500m / 1 / 1Gi / 2Gi |
| Storage | SERVER_STORAGE_SIZE, SERVER_STORAGECLASS, KEEPER_STORAGE_SIZE, KEEPER_STORAGECLASS | 400Gi / standard / 25Gi / standard |
| Pull secret | PULL_SECRET_NAME | Empty (public registries only) |
The defaults target development
The default topology is one shard, one replica, and one Keeper. This is appropriate for development and smoke testing only. For production, plan replicas and Keeper count up front. Keeper must run with an odd number of replicas so the cluster can hold a quorum.
Reviewing User Credentials
The default credentials are stored as SHA-256 hashes in clickhouse/secrets/chi-users.env:
- default user:
default/default123 - admin user:
admin/admin456
Replace these hashes with values you have generated yourself before deploying to anything that is not a local evaluation cluster. Both passwords appear in connection strings used by Countly and Kafka Connect, so rotating them later requires coordinated updates.
Deploying the Stack
From the repository root, apply the ClickHouse overlay. This installs the Altinity operator, Keeper, and ClickHouse server in the clickhouse namespace, and is fully declarative:
kubectl apply -k clickhouse/
The overlay applies in this order:
- The
clickhousenamespace. - The Altinity ClickHouse Operator bundle.
- A ClickHouse Keeper custom resource (
chk.yaml) with inline configuration for console logging and metrics. - A ClickHouse installation custom resource (
chi.yaml) with the Altinity metrics-exporter sidecar.
Verifying the Initial Deployment
Watch pods come online and confirm Ready status:
kubectl -n clickhouse get pods
Expected outcome:
- ClickHouse pods report
2/2 Ready(server + metrics-exporter sidecar). - Keeper pods report
1/1 Ready. - All pods are in
Runningstate.
Quick functional check using the default user:
kubectl -n clickhouse port-forward svc/ch 8123:8123 & curl -s "http://default:default123@localhost:8123/" -d "SELECT version()" kill %1
A version string in the response confirms the cluster is up and accepting queries.
Deployment is up
With pods Ready and the version query returning a result, the cluster is ready for the rest of the Countly stack. Run the full validation checklist documented separately to confirm replication, metrics, and storage are healthy before depending on the cluster in production.
Connecting Countly to This Cluster
Countly reads its ClickHouse password from the secret managed by the Kafka and Countly overlays. By default, the same password is used for three consumers:
- Countly application services
- The ClickHouse default user
- Kafka Connect's ClickHouse sink
Keep these in sync when you rotate the ClickHouse default user password. The recommended Secret Manager naming convention is <customer>-clickhouse-password.
Common Issues and Gotchas
Read the operator logs first:
kubectl -n clickhouse logs deployment/clickhouse-operator -c clickhouse-operator
Confirm the Altinity CRDs were installed:
kubectl get crd | grep clickhouse
Then confirm the storage class named in SERVER_STORAGECLASS exists in the cluster.
A single-Keeper deployment works without quorum negotiation. A three-Keeper cluster needs time to stabilize and may briefly report no leader. Check Keeper logs for both pods:
kubectl -n clickhouse logs chk-clickhouse-keeper-keeper-0-0-0 kubectl -n clickhouse logs chk-clickhouse-keeper-keeper-0-1-0
Verify network policies allow inter-pod communication on Keeper's coordination port.
Confirm the metrics-exporter sidecar is up and listening on its port:
kubectl -n clickhouse logs chi-ch-prod-main-0-0-0 -c metrics-exporter --tail=20
Then confirm the service exposes the metrics port and that ServiceMonitor labels match the labels on the service. Test the endpoint directly with port-forward before debugging Prometheus.
standard storage class is not present on every Kubernetes distribution. Check what is available with kubectl get storageclass and update SERVER_STORAGECLASS and KEEPER_STORAGECLASS in config.env accordingly, then re-apply.