Sometimes you need your raw events outside Countly: to archive older data, or to put Countly data into a data lake next to data from your other platforms. In Countly 26.01 and later, raw events are stored in ClickHouse, not in MongoDB. This article explains how to export them, which method fits which job, and how to build a recurring feed that does not miss late events.

Understanding Where Raw Events Are Stored

Every event your SDKs send is stored as one row in the drill_events table of the countly_drill database in ClickHouse. These rows are often called "drill events". MongoDB still holds the rest of your Countly data, such as apps, users of Countly, and settings.

This change matters if you used a tool that reads MongoDB to copy events, such as a MongoDB replication tool (for example, AWS DMS) or an old export script. After the upgrade, the MongoDB drill collections receive no new events. Queries against them return zero rows without an error, so the feed seems to work while it delivers nothing.

Choosing an Export Method

Countly gives you three ways to export raw events. All three read from ClickHouse.

For a data lake that receives millions of events each month, use the server scripts or the background mode of the export API.

Prerequisites

• For the server scripts: shell access to a machine or container that runs the Countly Server, and the ID of each app you want to export.
• For the export API: the API key of a Countly user who has Read access to DB Viewer, Drill, or User Profiles for the app. The export contains only the apps that user can read.
• For background downloads: the user who downloads the file needs Admin access to the app.

Exporting with the Server Scripts

The export scripts are part of every Countly Server. They are in the bin/scripts/export-data/ folder of the Countly directory. On a bare-metal server, run countly dir to find that directory. In the Countly container images, it is /opt/countly.

Both scripts read the same settings from environment variables:

Exporting a Date Range

Use export_drill_events.js to export the events that happened in a date range. It writes one .jsonl.gz file for each app, named drill_events_<app_id>_<start>_<end>.jsonl.gz.

cd /opt/countly/bin/scripts/export-data
EXPORT_APPS='{"<app_id>":[]}' EXPORT_DIR=/data/exports node export_drill_events.js 2026-10-01 2026-11-01

The start date is included and the end date is not. If you leave out both dates, the script exports the last 30 days.

This script selects events by the time they happened on the device. That is right for a one-time export of history. For a recurring feed, use one of the methods in the next sections, which select events by the time they reached Countly.

Running Scheduled Exports

Use export_specific_events.js for a feed that runs on a schedule, for example from cron. It takes one period: yesterday, 7days, 30days, 60days, or all.

cd /opt/countly/bin/scripts/export-data
EXPORT_APPS='{"<app_id>":["Purchase","Login"]}' EXPORT_DIR=/data/exports node export_specific_events.js yesterday

The script selects events by the time they reached Countly, in whole days that end at midnight of the server's local time. For example, yesterday run at any time today exports exactly the events that arrived yesterday. If you run it every day with yesterday, each event is exported once.

The output is one JSON array file per app and event, in the format of Countly 25.03 and earlier. The file names contain the app name, the event key, the period, and the time of the run. If no events arrived in the period, the script writes no files.

With the all period, list the event keys in EXPORT_APPS. An empty list does not work with all: the script stops with a "Cannot parse DateTime" error.

Running the Scripts in a Container

If Countly runs in containers, run the scripts inside the Countly API container. It already has the connection settings for ClickHouse.

docker compose exec -e EXPORT_APPS='{"<app_id>":[]}' countly-api \
  node /opt/countly/bin/scripts/export-data/export_drill_events.js 2026-10-01 2026-11-01

docker compose cp countly-api:/opt/countly/export ./countly-export

kubectl exec -n <namespace> deploy/<release>-api -c api -- \
  env EXPORT_APPS='{"<app_id>":[]}' \
  node /opt/countly/bin/scripts/export-data/export_drill_events.js 2026-10-01 2026-11-01

kubectl exec -n <namespace> deploy/<release>-api -c api -- \
  cat /opt/countly/export/<file_name> > <file_name>

Files written inside a container are lost when the container restarts. Copy them out when the export finishes, or set EXPORT_DIR to a folder on a mounted volume.

Exporting with the Export API

The export API has two modes. The direct mode returns a file in the response. The background mode, called fast, creates a background task and gives you a file to download when the task is finished.

Exporting Large Ranges in the Background

The fast mode exports a whole time range of one app to a .jsonl.gz file. It accepts these parameters:

Dates accept ISO 8601 text, such as 2026-10-01T00:00:00Z, or a timestamp in milliseconds.

A background export takes three steps:

1. Start the export. The response contains a task ID.
2. Check the task until its status is completed. If the status shows an error, read errormsg.
3. Download the file.

# 1. Start the export of the events that reached Countly in October 2026
curl -X POST "$HOST/v2/export/db?api_key=$KEY&app_id=$APP" \
  -H 'Content-Type: application/json' \
  -d '{"db":"countly_drill","collection":"drill_events","mode":"fast",
       "app_id":"'$APP'","from":"2016-11-02T00:00:00Z","to":"2026-11-01T00:00:00Z",
       "cd_from":"2026-10-01T00:00:00Z","cd_to":"2026-11-01T00:00:00Z"}'
# → {"data":{"task_id":"a1b2c3…"}}

# 2. Check the task
curl "$HOST/v2/tasks/a1b2c3…?api_key=$KEY&app_id=$APP"
# → {"data":{"status":"completed","errormsg":null,…}}

# 3. Download the file
curl -o october.jsonl.gz "$HOST/v2/tasks/a1b2c3…/download?api_key=$KEY&app_id=$APP"

Both time ranges apply together. In the example, from and to cover the last ten years, so a late event that happened long ago is still included. Only the cd_from and cd_to range limits the export to October.

Each user can run two background exports at the same time. If you start an export that is already running, you get the ID of the running task.

Exporting Filtered Rows

The direct mode returns CSV, XLSX, or JSON in the response. It accepts a MongoDB-style query, a projection of the fields you want, a sort on one field, and limit and skip. Countly translates the query for ClickHouse.

curl -X POST "$HOST/v2/export/db?api_key=$KEY" \
  -H 'Content-Type: application/json' -o purchases.csv \
  -d '{"db":"countly_drill","collection":"drill_events","type":"csv",
       "query":{"a":"APP_ID","e":"Purchase"},"projection":{"ts":1,"e":1},"limit":1000}'

The direct mode is for small exports. The limit is capped by the export limit setting of your server, which is 10,000 rows by default. An XLSX file holds at most 1,048,575 rows. For more rows, use the background mode or the server scripts.

You can also export rows in Countly. In DB Viewer, open clickhouse_countly_drill > drill_events and use Export.

Building a Recurring Feed to a Data Lake

Selecting Events by Arrival Time

Every event row has two times:

• ts: the time the event happened on the device.
• cd: the time the event reached Countly.

For a recurring feed, always select by cd. Devices go offline and SDKs send events again later, so an event can happen on Tuesday and reach Countly on Friday. A ts window for Tuesday ran before Friday and skipped that event. Nothing shows an error, so you find the gap only when totals do not match months later. A cd window always catches the event.

Use windows that are closed and start and end at midnight, for example from October 1 at 00:00 to November 1 at 00:00. A rolling window such as "the last 30 days from now" moves while the export runs, so consecutive runs overlap at one end and miss rows at the other.

The yesterday period of the scheduled script and the cd_from and cd_to parameters of the API both follow these rules. The since_last=true option of the API also does: it starts where the last export of the same events ended.

Planning the First Exports After an Upgrade

When your data moved to ClickHouse during the upgrade, every imported row received the time of the import as its cd. So for the first weeks after the upgrade:

• A cd window that includes the upgrade day selects almost all of your history. Plan for a very large first export.
• Do not add a ts filter to make a cd export smaller. After the upgrade, the two times are not related: one week of cd can hold rows whose ts covers several years.

Making Large Exports Faster

• Export fewer fields. Most of the time goes into turning rows into JSON, not into reading them. Event rows carry wide fields, such as user properties, segmentation, and custom data. If you do not need a field, leave it out.
• Plan by data size, not by row count. Rows with many properties export about three times slower than small rows.
• Give ClickHouse enough CPU. JSON conversion uses CPU cores. A ClickHouse container limited to two CPUs exports slowly, even with fast disks.

Exports run at a lower priority than dashboard queries and use at most a quarter of the memory and CPU of ClickHouse. A large export does not block Countly, but it can take hours.

Understanding the Output Formats

Most data lakes load .jsonl.gz files directly. If you need CSV, you can convert a file with a tool such as Miller:

gunzip -c export.jsonl.gz | mlr --ijson --ocsv cat > export.csv

A .json array file can be several gigabytes. Most JSON parsers cannot load a file of that size and fail with an error about string length. Read the file as a stream, for example with jq --stream, or export shorter periods.

To check a .jsonl.gz file, test the archive and count its rows:

gunzip -tv export.jsonl.gz && gunzip -c export.jsonl.gz | wc -l

Applying Data Masking

If you use data masking in Countly, the export API and DB Viewer apply your masking rules automatically.

FAQ and Troubleshooting

Can I keep using AWS DMS or another MongoDB replication tool?

Not for events. In Countly 26.01 and later, events are not stored in MongoDB, so a MongoDB replication tool copies no new events. Use the server scripts or the export API instead.

Can I read events from Kafka?

Kafka is part of Countly's internal pipeline that moves events into ClickHouse, and it is not an export interface. ClickHouse holds the final copy of your events, including later changes such as merged or deleted users. Use the export methods in this article for a complete and correct copy.

Why does my old export script return no rows?

Scripts written for Countly 25.03 and earlier read the MongoDB drill collections, which no longer receive events. Replace them with the scripts in this article. If your data lake expects the old row format, set EXPORT_LEGACY=1.

Why are some events missing from my daily export?

Your export probably selects events by ts, the time the event happened. Events that arrive late fall outside a window that has already run. Select by cd instead, as described in Selecting Events by Arrival Time.

Why does the API answer 503?

Countly cannot reach ClickHouse. The export does not return an empty file in this case. Check that ClickHouse is running, then start the export again.