A Countly SDK is a library that, when imported into your project and initialized, provides an interface for sending data to your Countly server. While some SDKs can automatically collect data if the platform permits, all SDKs offer methods for you to manually control when and what data to record. In this documentation, most of this data will be referred to as an "Event."

Essentially, an SDK acts as a request generator. Instead of manually formatting requests for the server to digest, SDKs create them for you based on the data you wish to send.

For detailed configuration options and methods, please refer to the specific SDK's documentation. This document serves as a general, elementary guide to how SDKs function, contrasting with our SDK Development Guide, which delves into specific actions, triggers, and request parameters.

We strongly recommend you to read Core Mechanics to get yourself familiar with how SDKs work. Features section is more selective and you may read it only if you are interested with how a specific feature works.

Core Mechanics

Recording Data

Countly SDKs primarily collect data for analysis on your Countly server. Most SDKs are designed as singletons, limiting data submission to a single Countly App on your server. The sole exception is the Web SDK, which supports multiple independent instances, each capable of sending data to a different Countly app.

Data collection by SDKs falls into two categories: manually recorded data and automatically recorded data.

Automatic data collection

Callout: Automatic data collection is not enabled by default. SDKs can use platform signals to record data automatically

Automatic data collection consists of SDKs making decisions about what and when to collect data. All platforms SDKs run on have their unique signals that indicate a process is happening. If SDKs can listen to these signals, and these signals overlap with a feature SDK wants to track then SDK simply uses these signals to record internal Events.

An example of this is automatic session tracking. Sessions are defined as time periods when an app is visible to the user in Countly. By listening to app foreground and app background going signals in mobile platforms SDKs can record begin and end session Events automatically. However in a platform that does not have these kinds of signals SDK becomes simply unable to track sessions automatically and would need the user to define and track what a session is according to signals they see fit.

Manual data collection

To simply put you can feed any data you can capture by yourself to SDKs. SDKs expose methods for you to record different kinds of Events. These events can be:

• Session events
• View events
• Crash events
• Feedback events
• User property events
• Custom events

Sending and Storing Data

SDK Queues

To prevent data loss and mitigate overwhelming your server, SDKs are designed with a robust queuing mechanism. When you record an Event using an SDK method, the data is not sent directly to your server (with some exceptions). This approach avoids potential data loss due to:

• Loss of internet connection or offline user status
• Busy or down servers

Instead, SDKs implement the following logic:

• Events are batched into single requests.
• Requests are queued instead of being sent immediately.
• The queue is persistently saved.
• Requests in queue is sent at regular intervals.

Common Questions about this Mechanism:

• Queue Size: The default queue size is 1000 requests, but this value is configurable.
• Sending Interval: For most SDKs, the regular sending interval is 60 seconds, while for the Web SDK, it's 500 ms.
• Full Queue Behavior: If a new request is added when the queue is full, the oldest request will be dropped.
• Failed Requests: If a request fails, the SDK will attempt to resend it 60 seconds later.
• Event Batching: Events are batched when they reach a count of 100, or when the request sending interval is met, whichever occurs first.

SDK Requests

Countly SDKs communicate with your server by sending requests to endpoints such as `/i` or `/o/sdk`. While most of these are GET requests, longer requests are automatically sent as POST requests. You also have the option to configure SDKs to exclusively use POST requests.

Currently, SDKs are in the process of transitioning to use POST requests by default.

Each SDK request includes information vital for data recording, such as:

• app_key: Specifies the Countly application where the data should be recorded.
• device_id: Identifies the user associated with the recorded data.
• sdk_name: The internal name of the SDK.
• sdk_version: The version of the SDK.

SDKs generate various types of requests, differing in their content (parameters). These request types include:

• Event requests (for both custom and internal event data)
• Session requests (for begin, update, and end session data)
• Token requests (for Push Notification token data)
• Remote config requests
• Server Config requests
• Health Check requests
• Feedback Widget requests (for survey, NPS, and rating widget data)
• Crash requests
• Location requests
• ID change requests
• User Profile requests
• Content zone requests

Device ID

To send data (requests) to your server, Countly SDKs require two pieces of information: the `app_key` (identifying the Countly app) and the `device_id` (identifying the user). The `device_id` determines which user the data belongs to.

By default, upon first initialization, all SDKs generate a random, persistent UUID as the `device_id`. This effectively tracks each distinct device (e.g., browser, phone, tablet) as a unique user.

However, the SDK allows you to change the `device_id` to any value you prefer. This is useful for tracking individual users instead of devices, especially if you have a unique user identifier (like an email address).

Additionally, the SDK offers an "offline mode" (or "temporary ID mode"). In this mode, the `device_id` is set to a temporary value that prevents requests from being sent to the server. You can then change the `device_id` to your desired value, overriding the temporary one and allowing data to be sent to your server again.

For a more systematic overview of this logic, have a look at this table.

Direct requests

Certain time-critical requests are exempt from the standard queueing mechanism to prevent detrimental delays. These essential requests include:

• SDK behavior settings requests (Server Config)
• SDK Health Check requests
• Remote config requests
• Feedback widget requests
• Content zone requests

Should the server be busy or the user be offline, these requests will fail. Unlike queued requests, these are not persistently saved and therefore will not be retried by the SDK.

Only a limited number of SDKs support sending events as direct requests. All other direct requests are either automatically generated by the SDK or linked to a specific feature. Some SDKs also provide a method to trigger the immediate sending of stored queues.

Storage of data

SDKs store queues and other data, such as device ID and ID type, in platform-permitted locations. These typically include:

• Browser local storage
• Shared preferences (or equivalents)
• Files or SQLite

The size of an SDK request varies based on the data transmitted, with a common request size being approximately 600 characters. A full request queue with such requests would be around 0.6 MB. This size will increase with larger events. The upper limit is determined by internal SDK settings, configurable during initialization or via the SDK Behavior Settings panel.

Backoff mechanism

If your server experiences a peak load that exceeds its capacity, it can quickly become unresponsive. To mitigate this, SDKs are designed with an internal logic to determine if the server is under load and, if so, to back off temporarily, helping to flatten the peak curve. This backoff mechanism is triggered when the following three conditions are met:

• The server response time is slow (over 10 seconds).
• The request queue is not full (below 50%).
• The request being sent is not old (younger than 24 hours).

When all these conditions are satisfied, the SDK will pause for 60 seconds before attempting to resend the requests. All these values are configurable through the SDK Behavior Settings.

It's important to note that direct requests are considered more time-critical than queued requests. Therefore, SDKs will continue to send direct requests regardless of the server's current condition.

Features

Logging

You can print the internal logs of all SDKs to your IDE's console (or equivalent). These logs provide crucial information for testing and development, allowing you to verify correct functionality. The logs will display details such as:

• SDK initialization
• Method calls and associated data
• Requests created and sent to the server
• Responses received from the server
• Occurrences of incorrect data or errors

Crashes

Countly SDKs provide functionality for recording and reporting errors to your server. Errors are categorized as either handled or unhandled.

Handled errors are those you explicitly capture within Try/Catch blocks in your code, and these can be directly passed to the SDK.

Unhandled errors, if supported by the platform, are automatically captured by the SDK when you enable this feature, as it subscribes to the platform's crash handler. However, to prevent conflicts, it is recommended to use only one third party SDK for capturing unhandled crashes.

SDK Behavior Settings

The SDK Manager plugin on your Countly server allows you to enable and adjust most SDK features. The SDK behavior setting panel provides options to:

• Modify internal SDK limits and default values.
• Toggle features on or off.
• Enable or disable all tracking or networking.

Initially, the SDK and server share the same default values for all settings. During SDK initialization, the SDK first checks for a stored Behavior Settings configuration object. If one exists, its rules are applied. Afterward, the SDK sends a request to the server to fetch and apply the latest configuration, which is then saved. For long-running applications, new values are re-fetched every 4 hours.

The only instance the SDK won't have a stored Behavior Settings configuration object is during its very first initialization. To address this, you can provide a configuration during the initial SDK setup. This configuration will only be used for the first initialization. You can download a suitable configuration directly from the SDK behavior setting panel.

It's important to note that the SDK server configuration overrides any developer-set SDK configuration. However, only enforced configuration options from your panel will take precedence.

Health Checks

 Upon initialization, the SDKs will generate and transmit a concise report to your server detailing the previous runtime. This report will encompass:

• The total number of error logs.
• The total count of warning logs.
• Information regarding the last network issue encountered.
• The HTTP status code from the last network response associated with that issue.

Subsequently, you can access and review this data within your SDK Manager plugin.

Journeys and Content

Journeys is a feature that enables you to display specific content to users based on predefined triggers, such as a session start or a user recording a particular event. The decision of whether content should be shown is made on the server side, utilizing data sent from the SDK.

Here's a breakdown of the logic, using a journey designed to show content when a session starts as an example:

1. Journey Setup: You configure the journey on the server.
2. User Interaction: The user interacts with your app, and the SDK sends a 'begin session request'.
3. Trigger Calculation: The server receives the request and determines if a journey trigger has been met.
4. Content Availability: Based on the calculation, the server flags content as available for the next time the SDK queries for it.
5. SDK Request: The SDK regularly sends a request to check for any available content for the user.
6. Server Response: The server responds with the available content information.
7. Content Display: The SDK receives the content information and then displays it.

From this process, it's important to note:

• Trigger information must be received from the SDK for content to become available.
• There may be a delay between a journey being triggered and the content being displayed.

Remote Config

Remote Configuration enables the sharing and control of data within your projects via the SDK. You can define the data you wish to share with the SDKs directly from your Countly Server. The SDK offers the flexibility to fetch this data either automatically through predefined triggers or manually when deemed appropriate.

The fundamental communication process is as follows:

1. You establish remote configuration values within your dashboard.
2. The SDK transmits a request to determine if remote configuration is available for the current user.
3. The Server evaluates if any remote configuration values are available for the user and responds accordingly.
4. The SDK receives and stores this value, making it accessible for developer use.
5. Developers can then leverage these values through SDK methods to implement their desired functionalities.

Remote configuration is highly user-specific. If your application supports multiple users, the SDK will indicate if the saved remote configuration values originated from a previous user. In such scenarios, if you are manually handling remote configuration fetch requests, you can re-request the values. Additionally, you will be alerted if network issues prevented the SDK from fetching new values.

Session flow

In Countly, sessions are defined as periods when an app is visible to the user. Automatic session tracking, when enabled, uses app foreground/background actions on mobile SDKs and SDK initialization and browser closing on web platforms to automatically record session start and end events. However, on platforms without these signals, the SDK cannot track sessions automatically, requiring users to define and track sessions based on their own suitable signals.

Session tracking process involves three distinct request types:

• Begin Session Request: Used solely as a starting timestamp.
• Update Session Request: Provides the time elapsed since the session began or the last update.
• End Session Request: Also includes the time elapsed since the session started or the last update.

The triggers for these methods are typically:

• Begin Session: SDK initialization, app foreground signal, or an equivalent event.
• Session Updates: An internal timer (60 seconds default).
• End Session: App background signal, a `beforeunload` event, or an equivalent.

On the web, an internal caching mechanism also marks a user as inactive if no action (e.g., clicking or scrolling) is taken for 20 minutes while the website is in focus.

View tracking

 A view represents the page currently visible to the user and Countly SDKs provide a manual view tracking option. Some SDKs support multiple open views simultaneously, while others limit it to one view at a time.

For each view, the SDK sends two requests to the server:

• View Start Request: Acts as a timestamp.
• View End Request: Reports the view's duration.

View duration can be affected by user actions. On mobile platforms, for instance, minimizing an application automatically ends a view, and maximizing it restarts the view. In web environments, if the focus is lost, the SDK pauses its internal timer and resumes it when focus is regained.

Every view has an internal ID, which is included with each event sent to the server. This allows you to trace which events were sent under specific views from your server.

Feedback widgets

Countly Server lets you create Survey, NPS, or Rating widgets, with the ability to set conditions determining widget availability for specific users. 

The typical communication flow for displaying widgets with SDKs is as follows:

1. SDK requests a list of available feedback widgets.
2. Server calculates and sends the widgets available to the user.
3. The appropriate widget is selected from the list and displayed.

SDKs provide a simplified method that combines these three steps. However, for custom client-side logic, SDK offers manual methods that can be employed to manage each step independently.

Conditions set for a widget’s availability must be met before it can be displayed by the SDK. For instance, a rule can be set to make widgets exclusive to iOS devices. However for the server to apply such rules, it must first receive platform information from the SDK. Only then can the server make the relevant widget available. You should check the SDK Queue mechanism explained above to understand why a delay could happen until a widget can become available.

This feature will be integrated into Journeys, allowing users to select triggers for displaying feedback widgets without requiring code.

User Properties

User properties capture information about the user's general state and are handled uniquely by the SDK and server. They are never batched with other data types and are sent as individual requests. This information is then appended to all subsequent events on the server.

User properties have their own cache where they can be cached into a single request at the end. The difference of this cache from a queue is that cached values can override each other if it is for the same key. So recording ‘theme:light’ after ‘theme:dark’ would override the ‘dark’ with ‘light’. This internal cache is converted into a request when an event is recorded, a session update occurs, or the device ID changes, which helps reduce the number of requests if user properties are added consecutively.

The key aspect of user properties is its relationship with events. Recording user properties act as a ‘process event queue’ signal to event queue and likewise recording an event acts as a ‘process user property cache’ for the user property cache. This ensures the order in which server gets the events and user properties are correct.

Consider this example:

• Initial User Property: You set a user property, `theme:light`, which is stored in a user property cache.
• Property Sent: When an event is generated or a session updates, the cached user property is sent to the request queue.
• Event Generation: The user performs actions, generating 5 new events that enter the event queue.
• New User Property: The user changes the theme, so you record a new property, `theme:dark`.
• Event Bundling: The SDK bundles the 5 pending events and places them into the request queue.
• Property Added: After the events are in the request queue, the SDK adds the `theme:dark` user property to the user property cache.

Consequently, the 5 events will be associated with the "Light" theme on your server, while any events occurring after the "Dark" theme property is recorded will be associated with the "Dark" theme.

Consent

Upon initialization without specific configurations, the SDK generates only two requests:

• SDK behavior settings request: Checks for server-defined rules.
• Health Check request: Reports any internal errors.

Beyond these, no features will be active, and no data will be collected until you manually invoke methods or enable automatic tracking for features.

However, if you wish to enable automatic tracking and utilize manual methods, but only after a user grants consent for data tracking, you must enable a special SDK mode. This mode, when activated during initialization, requires consent for each feature before it can be used. This includes consent for:

• Sessions: Tracks visible usage time.
• Events: Allows your events to be sent to the server.
• Views: Enables tracking of views accessed by a user.
• Scrolls: (Web only) Allows tracking of user scrolls on the heatmap.
• Clicks: (Web only) Allows tracking of user clicks and link clicks on the heatmap.
• Forms: (Web only) Allows tracking of user form submissions.
• Crashes: Allows tracking of errors.
• Attribution: Enables direct attribution tracking.
• Users: Allows collection/provision of user information, including custom properties.
• Star-rating: (Mobile only) Allows user rating and feedback tracking via the star rating dialog.
• Feedback: Allows usage and reporting for survey, NPS, and rating widgets.
• APM: Allows performance tracking of applications by recording traces.
• Location: Allows recording of a user's location (country, city area).
• Remote-config: Allows users to download remote configuration from the server.

The SDKs do not store these consent values. Therefore, you are responsible for saving them and providing them again after each initialization, or for prompting the user for consent every time they use your application.