This guide shows you how to point a single-host Docker Compose deployment at a MongoDB, ClickHouse, or Kafka cluster you already run, instead of the ones the stack starts for itself. Use it when you have a managed database, a shared cluster, or a datastore another team owns. If you want Countly to run everything on one host, follow Deploy Countly with Single-Host Docker Compose instead.

Prerequisites

Before you begin, make sure you have:

• A working single-host deployment, or a host prepared for one. This guide only changes where the data lives.
• Network reachability from the Countly host to each external endpoint. For ClickHouse that means the native TCP port as well as the HTTP port, not just HTTP.
• Credentials with the privileges listed for each datastore below. Countly creates schemas and topics on first boot, so read-only credentials are not enough.

Choosing What Runs Externally

Each datastore is controlled by its own variable in .env. Leave a variable empty and the bundled container runs as usual; set it and that container is never created.

The three are fully independent. A managed MongoDB alongside a local ClickHouse and Kafka is as valid a combination as moving all three, and every mix in between works the same way.

Connecting to an External MongoDB

Set the full connection string in .env:

MONGODB_URI=mongodb://countly:secret@mongo-a:27017,mongo-b:27017/countly?replicaSet=rs0&authSource=admin

mongodb+srv:// works too. Credentials, extra hosts, TLS, and maxPoolSize all travel inside the URI.

The cluster must meet two requirements:

• It must be a replica set. Countly uses change streams and transactions, neither of which a standalone mongod provides. A standalone will connect and then fail at runtime.
• The user needs readWrite on four databases: countly, countly_drill, countly_out, and countly_fs.

Once MONGODB_URI is set, MONGODB_REPLICA_SET, MONGODB_DATABASE, and MONGODB_MAX_POOL_SIZE no longer apply. Put replicaSet=, the database name, and maxPoolSize= in the URI instead.

Connecting to an External ClickHouse

Set the HTTP endpoint, scheme included:

CLICKHOUSE_URL=https://clickhouse.internal:8443
CLICKHOUSE_USER=countly
CLICKHOUSE_PASSWORD=your-password
CLICKHOUSE_DATABASE=countly_drill

The scheme is required. Startup fails without it, because the ClickHouse sink connector derives its host, port, and TLS setting by splitting this URL.

Required Grants

The bundled user definition at clickhouse/users.d/countly.xml is a working template for the external one.

Native Port and TLS

ClickHouse dictionaries connect over the native TCP protocol rather than HTTP, so they need their own port and TLS flag. Managed ClickHouse usually wants:

CLICKHOUSE_NATIVE_PORT=9440
CLICKHOUSE_SECURE=true

The defaults, 9000 and false, match a plain self-hosted server.

CLICKHOUSE_PASSWORD may be left empty for an external server that uses no password. The bundled server still requires one, because it templates the password into its own configuration at boot.

Connecting to an External Kafka

Set the bootstrap servers, comma-separated:

KAFKA_BROKERS=broker-1.example.com:9093,broker-2.example.com:9093

Raise the broker's message ceiling before the first event arrives. Countly publishes batches up to 50 MB, and the Kafka default of roughly 1 MB rejects them:

message.max.bytes=52428800
replica.fetch.max.bytes=52428800

Countly creates its own topics at boot, so the account needs permission to create them — or you pre-create them yourself. Every create the stack issues uses --if-not-exists, so on a pre-provisioned cluster the topic step reduces to a reachability check. See the kafka-init service in docker-compose.yml for the exact names, partition counts, and retention settings.

Authentication

For a cluster that requires credentials:

KAFKA_SECURITY_PROTOCOL=SASL_SSL      # PLAINTEXT | SSL | SASL_PLAINTEXT | SASL_SSL
KAFKA_SASL_MECHANISM=SCRAM-SHA-512    # PLAIN | SCRAM-SHA-256 | SCRAM-SHA-512
KAFKA_SASL_USERNAME=countly
KAFKA_SASL_PASSWORD=your-password

These reach the Countly services, Kafka Connect, the topic bootstrapper, and the push scheduler alike. Leave all four empty for a plaintext cluster.

KAFKA_BROKERS=broker-1:9093,broker-2:9093
KAFKA_BROKERS_JSON=["broker-1:9093","broker-2:9093"]

Starting the Stack

Nothing about the command changes. Compose works out which containers to create from the variables themselves:

make up

There is no extra flag, profile, or Compose file to remember.

Three short-lived containers run before anything else. Each proves its datastore answers and the credentials work, creating the database or topics if they are missing. Nothing else starts until all three pass, so a wrong connection string produces one clear error instead of a service that starts healthy and fails quietly later.

Verifying the Connection

After make up returns, work through these checks:

1. Read the three checks. Each exits 0 when it is satisfied:

   docker logs countly-mongodb-init
   docker logs countly-clickhouse-init
   docker logs countly-kafka-init

   You are looking for MongoDB ready (writable primary), ClickHouse ready., and Kafka topics ready. With an external MongoDB you will also see External MongoDB — leaving replica set configuration alone., which confirms the URI was recognised.

2. Confirm no bundled datastore container exists for anything you moved off:

   make ps

   There should be no countly-mongodb, countly-clickhouse, or countly-kafka.

3. Run the health checks. This prints a line for each external datastore and checks only what still runs locally:

   make verify

4. Send one event and confirm it lands in ClickHouse. Replace the app key with your own:

   curl -s "http://localhost/i?app_key=YOUR_APP_KEY&device_id=smoke&events=%5B%7B%22key%22%3A%22smoke_test%22%2C%22count%22%3A1%7D%5D"

   Then query the drill table. Note that n holds the custom event name and e holds the event type:

   SELECT n, ts FROM countly_drill.drill_events WHERE n = 'smoke_test' ORDER BY ts DESC;

Hosts with Dedicated Data Disks

Production deployments usually keep MongoDB, ClickHouse, and Kafka on dedicated disks, and provision-docker-host.sh installs a boot-time guard that stops Docker from starting if one of those disks is missing. The guard reads the same .env file, so it has to be regenerated before it knows that a datastore now lives somewhere else:

sudo bash scripts/provision-docker-host.sh fine-tune

The fine-tune mode reapplies the host tuning and rewrites the guard without reinstalling any packages.

Hosts that keep their data under the default ./data path are not affected, because there is no dedicated disk to check.

Common Issues and Gotchas

What Countly No Longer Manages

Moving a datastore off the host means this deployment stops managing it. It does not size it, back it up, or monitor it. make verify skips containers that no longer exist, and the MongoDB and Kafka exporters under make up OBS=1 only cover bundled datastores. The mount guard skips the matching data directory, since nothing is written locally for it.

Point your own monitoring and backups at the external servers. The one thing this host still tracks is the boot-time mount guard described above, which needs regenerating whenever the set of local datastores changes.