In Countly, jobs (also referred to as internal jobs or scheduled jobs) are background tasks that run periodically to maintain data, process analytics, and perform cleanup.

You can see them in:

countly_jobs collection (in the DB_Viewer)

Dashboard ->  Management -> Jobs

Background jobs run automatically at scheduled intervals, ensuring that several features remain accurate and up to date. For example, Cohorts, Flows, Retention, alerts, crash symbolication, and cleanup all depend on background processing to refresh or compute results.

By offloading heavy operations to scheduled jobs, Countly prevents performance bottlenecks, reduces system load, and maintains a smooth user experience across all features.

Common Jobs in Countly

1. alerts:monitor

This job is responsible for sending out alerts configured using the alerts plugin. Each alert has its own corresponding job. If this job is suspended, the system will not check or send out any email notifications.

 2. api:clear

This is a system cleanup job. On Countly system restart, existing jobs are canceled, and new ones are created with the state 'scheduled'. This job deletes old canceled jobs. If this job is suspended, the collection will not be cleared, potentially leading to a buildup of canceled jobs. The running time and resource impact of this job are minimal.

3. api:clearAutoTasks:

This is another system cleanup job that removes old, automatically generated reports, which can be viewed in the Report Manager. Whenever you receive a message like, "This request is running for too long.. We have switched it to report manager and will notify you when it is finished," it indicates that an auto report has been created. This job deletes reports older than 90 days. Similar to api:clear, the impact on the system is low.

4. api:clearTokens:

This is a system cleanup job. It deletes expired tokens. Tokens can be created to give access to parts of the system (or to the whole system) without sharing the API key. For more info about tokens: Token Manager

Also, tokens are created to keep the dashboard user session. If user A creates a token that is never used, this job will delete the token once it expires. Usually, tokens are also deleted if someone tries to use them after the expiration date. The impact is low, as in the other system cleanup cases.

It is important to note that switching off api:clearTokens, api:clearAutoTasks, and api:clear would not be recommended for a long time, because it would stop cleaning up for each job.

5. api:ping:

It confirms that the API is reachable and sends basic info about the enabled tracking or consent settings.

6. api:task:

It recalculates any manually created reports. Those are visible in the report manager. Every time a widget is created, there is one or multiple reports created for this widget. Each time a job runs, it checks if it is time for the report to be regenerated. The regeneration interval can be set under Settings > API > Regeneration interval for reports. Suspending this job would pause the report recalculation and ensure that those queries are not run while the job is suspended.

 Suspending this job would significantly prevent long queries from running, as in many cases, those queries come from created dashboard widgets.

7. api:topEvents:

This job recalculates the information displayed in the Events Overview in Countly.

Suspending this job would mean that those numbers would not get updated. As this calculation happens on aggregated data and each event is rechecked serially, it should not have as serious an impact on the system as drill queries. But as you also have many events, suspending this job could be useful to make it easier on the database.

8. api:userMerge:

This job is intended to finish unfinished user merges. If there is a server issue (e.g., the merge is interrupted), then there is a record in countly.app_user_merges collection, and the server will finish this merge once the job runs. 

If the SDK sends merge A > B and then, before it is fully finished B > C, merge B > C will not be processed right away, but only after the job sees as A > B finished, it will process B > C.

All started merges are stored in countly.app_user_merges collection. It is normal to have a low amount of merges there.

Suspending this job for a few hours or up to a few days would be fine if the incoming merge amount is low. We would recommend that you monitor the document count in this collection if the job is suspended.

9. cohorts:fix_cohorts:

This job does 2 things:

1)Every cohort consists of rules. To reduce calculations, rules are shared between cohorts. This job ensures there are no rules left in the system, which none of the cohorts use. The rule is either user segmentation or some event the user did or did not.

2)Kicks out users from cohorts with time expiry.

In the real-time cohorts model, we update states for each user on incoming data.

For example, if the rule is 'Ever did any session', then the moment incoming data is received, it is stored, and that user is included in this cohort. And there is no reason for the user to fall out of it.

If the rule for the cohort is 'Did event Buy once in last 7 days', then the moment the user does this event, the user goes into cohort. 

If 7 days pass and the user is active all the time but does different events, then on incoming data user will fall out of cohort.

But if there are no events sent for this user, then the user will not be updated automatically.

In this case, this job will recheck this user and take it out of the cohort.

• It is low impact.
• It could be a lot of calculations if there are many cohorts with expiring rules, and if there are a lot of users who do events and then stop doing any events.

Suspending this job could help ensure there is no database activity. However, it should not be off for long, as it could worsen data for cohorts with expiring rules.

 10. cohorts:generate_cohorts:

1)If real-time cohorts are enabled, this job runs a query on the app_users collection for each cohort to make sure the total number of users is correct. Every time we do increments and subtractions on data in the database on incoming data, this helps ensure the correct number.

2) If real-time cohorts are disabled, this would trigger full cohort recalculation for each cohort. This would be a lot of calculations on the database. Any “all-time” cohort would create a serious strain on the database.

Refer to this article to understand the jobs collection schema.

11. api:ttlCleanup

This job (TTLCleanup) is responsible for removing expired records from database collections that are set up to have a "TTL" (Time-To-Live) property, helping you save storage, keep data fresh, and prevent buildup of obsolete records.

12. dashboards:refreshDashboards:

This job fetches all the dashboards available in the system and reviews them to identify which ones have a refresh rate configured. Once these dashboards are identified, we trigger the regeneration process specifically for them, ensuring their data stays up to date and accurately reflects the latest activity.

13. drill: fixMeta

This job is a periodic background process in the Countly Drill plugin, designed to maintain and update metadata for analytical event data.

Similarly, Countly includes several user-related jobs, such as users: daily, users: post_session, and users: yearly. These jobs handle tasks such as closing inactive user sessions and expiring user properties when no session end event is received within a specified timeframe. If any of these jobs are disabled, session duration will not be flushed to Drill, because it is only processed when a session ends and User property updates will not occur, including the expiration of outdated properties.

Countly also includes maintenance jobs such as views:cleanupMeta, which prevents view metadata from overflowing, and report: send, which delivers scheduled email reports.