The Data Manager feature can be used to plan, manage, edit, delete, and modify Events and Event Segmentations, including performing actions such as transformations on Events or User Properties and managing their data types. The Manage Events View is now part of Data Manager.

Getting Started

First of all, make sure Data Manager is enabled. To do so, go to the Sidebar > Management > Feature Management and enable the Data Manager toggle.

Then, to access Data Manager at any time, on the Main Menu, go to Utilities > Data Manager.

Data Manager Overview

Understanding Data Manager Actions

Before we jump into a detailed, step-by-step guide on how to use Data Manager, let us first delve into some of the aspects that the feature touches upon. These include:

• Event -An Event is any interaction between your users and the app that you have selected to track. To know more about Events, click here.
• Event key -It is the unique key used for defining each event, and it is case-sensitive.
• Segmentation - Segmentation involves the application of segments - key qualifiers that help in filtering data - to Events. To know more about Segmentation, click here.
• Segmentation Type - Segmentation type can be any / list / boolean / string / integer (number).
• Event Groups - Event Groups allow you to create and manage groups of existing Events. This helps you visualize Events that are highly related when compared to other Events, while also making it easier to work with events when using other features.
• Validations -This allows users to keep their data clean by ensuring that no Event or Segmentation is set and sent via the SDK or API by mistake. Through Validations, Countly checks Events before using it for aggregated or granular data against set Event Segment keys.
• Segmentation value regex validation -This is used to detect PII data (like phones, social security numbers). If a user expects some fields may contain PII information, they can set up a regex to detect it, like email address or PII data (social security number which has pattern, etc.) and Countly would ignore or validate it.
• Transformations -Transformations is available only in Countly Enterprise and can be undertaken by any user with update permissions. A transformation modifies the target data as defined by a created rule. Two types of transformations can be applied to Events, Event Segmentation, or custom User Properties: Rename and Merge. This process is retroactive so when the transformation is applied, historical data and new data will be updated. It provides the ability to correct any mistakes on implementation or easy re-structuring of data on UI without the need for any effort on the code side or running a script. Details of transformations for each tab of Data Manager are discussed in the relevant sections below.

• Status -This can be created, approved, live, blocked, or unplanned, and you can update the status of events or segmentation from the bottom bar that appears when you select any event or segmentation from the tables.

  • Created -Not approved but planned events are shown with "Created" status. It means that the event is not live, not visible to users (e.g., on Drill or All events etc.), not in use in charts, tables, drop-downs, and Countly hasn't started to track it yet.
  • Approved -Controlled by users and is has recently begun to track, but not visible to users on the event or segmentation list, not be in use on charts, tables etc.
  • Live - Events that are tracked and counted, visible to users (on Events page, drop-downs etc.), is in use on charts, tables etc.
  • Blocked -Event data is no longer collected, visible to users for historical data (on Events page etc.), but is in use on charts, tables, drop-downs, etc., due to historical data.
  • Unplanned -Event data that are sent via SDK or API which are not on the Event Scheme. If developers or different departments create an event without planning it on UI, it is assumed as unplanned.

Using Data Manager

Data Manager consists of two tabs: Events and User Properties.

• In the Events tab, you can manage Events, Segmentations, Event Groups, Validations, and Transformations, including their data types.
• In the User Properties tab, you can manage actions related to specific user properties.

To get started, go to Main Menu > Utilities > Data Manager. We will go through each section in detail below.

Events

You can see a list of your Events in Data Manager under the Events tab, along with some key information:

1. Event name: It is the display name of an event, and is editable. An event name input field is limited to the SDK limits.
2. Description: It is used to describe the event and segmentation. A description input field is limited to the SDK limits.
3. Category: It is used to categorize and manage events. If any category is not defined, events are shown as “uncategorized”.
4. Count: This shows the number of times the event has occurred.
5. Last modified: The date and time this event was most recently modified by a user. The format is DD/MM/YY hh:mm.
6. Last Triggered: The date this event was most recently received by Countly. The date format is DD/MM/YY.
7. Status: It is a predefined field that shows the status of an event and can be Created, Approved, Live, Blocked, Unplanned, or Deleted.
8. Visibility: It shows the visibility of an event and can be Visible or Hidden. When the visibility is hidden, it will not show up on the dashboard. However the data will be retroactive.

Some of the columns are shown on the table by default. The remaining are customizable, and can be added to the table by any user with the permission to do so. Click on the 3-line button to select and deselect the columns as per your preference.

Note: Last modified by and last modified date are fetched from the Audit log feature. If Audit logs is disabled, these data aren’t shown. A notification is displayed on the page to inform users to enable the Audit Logs feature to see this data.

On the Events tab in Data Manager, you can:

• View event details with their segmentations
• View the event key
• Edit visibility
• Edit status
• Edit the event name
• Edit the event description
• Edit the event properties name (count, sum, and duration, but not segmentations)
• Categorize event(s)
• Delete event(s)
• Search for an event
• Filter events by Status, Category, Visibility (or Search bar may be used for Visibility and category)
• Export event list with all data shown on the table
• Customize columns to add different columns to the table
• Transform
• Create an event
• Undertake a Bulk Edit via The Event Schema file

Note:

• Blocking any events can be applied by using the Filtering Rules feature. On the Data Manager feature, users can only see which events/segmentations are blocked.
• The Event tab is accessible to users of the Community Edition, except for the Transformation feature.

Creating an Event

You can create an event on the Events tab in Data Manager, but SDK implementation requirements will still be valid. To create an event click the Create New button and then select the Create New Event option.

1. Give your (display) name to the event, which is a mandatory field.
2. Enter a unique event key, which is a mandatory field. Event_key of the events sent via SDK will be matched over event_key on UI. event_key is case sensitive. Note that if event_key doesn’t exactly match with event key sent via SDK, all requests for this event will be shown as “Unplanned” on Validation and a warning message will be shown.
3. Enter a description by considering the applicable character limitations. Description is an optional field.
4. Select a category (this is not mandatory; if it is not selected, the event is automatically listed as “uncategorized”)
5. Add segmentation (this is not mandatory) to your events if it needed and desired, by clicking on Add more.
6. Enter the Segmentation key.
7. Set the type of each segmentation.
8. Check the “Required” option if segmentation is mandatory.
9. Enter segmentation description. Make this field mandatory if the “Required” option is checked.
10. Click Save Changes.

Created events are listed with “Created” status on the Event table.

Editing and Managing Event Category

You can edit the Event category in two ways:

Edit Event from drawer:

You can edit the category of any Event from the drawer on the Event Detail page by following these steps:

• Open event details page by clicking on the 3-dot ellipsis menu next to any event and clicking the Edit button.
• Select any category from the Category drop-down
• Click the Save button to complete the process.

Shortcut: Event overview table

• Select the event by checking the box next to the event name from the event table
• Click the Change Category button on the action bar that will appear at the bottom of the page
• Select any category from the Category drop-down
• Click the save button to complete the process.

Editing and Managing Events

Some Event details can be edited directly from the Events tab or the Event Details page. These include:

• display name
• description
• category
• status
• event properties name
• visibility

Note that event_key cannot be updated from the drawer. Editable display event name is used on Countly for better UX and taxonomy.

To update any Event field, follow the below steps:

1. Click on the 3-dot ellipsis menu at the right on any Event
2. Click on Edit
3. Click the Event Display name and enter the new display name for the event.
4. Click the description field to update the description
5. Select any category from the drop-down list to update the category of the event (not available for Community Edition)
6. Click on “Add event segmentation” to add more segmentation if you need.
7. Select Status from the drop-down list. Updated event status will be directly applied to segmentation status. Please use Segmentation tab to edit each segmentation status if you need to keep it different from the event status.

You can also update the category and status by checking the box next to the Event name, followed by selecting any action (update category/status/visibility) you need from the action bar.

Deleting an Event

Available in both Countly Lite and Countly Enterprise, deleting an event will remove the historical data and remove the event from the Events feature and other query builder drop-downs, such as Drill. This means that you will no longer be able to query on the event. Keep in mind that when you delete an event, it will remove existing data from your historical data. If new event data is sent via SDK after it was deleted, Countly will count it from zero right after the first data is received via SDK. Deleted events and segmentations will no longer be available in drop-downs of query builders or filters unless it is blocked from filtering rules or it will show up again as an unplanned event (for Enterprise Edition users).

Follow the below steps to delete any event

1. Click the checkbox next to the event name on Events tab.
2. Click Delete from the action bar at the bottom of the page.
3. A verification modal will appear for the deletion process. Confirm your deletion request.

OR

1. Click the 3-dot ellipsis menu on the right of the relevant Event row.
2. Select the Delete option and confirm the warning message

Blocking Events

You can stop Countly from collecting event data by blocking it. When you block an event, Countly will immediately stop processing data for it until you decide to unblock it. Countly provides a Filtering Rules plugin feature for this purpose. This feature is used for blocking or unblocking any event instead of doing it with Data Manager.

Transformation

Transformation on Events can be applied to merge events. Event transformation allows users to merge multiple events or merge into one if an event already exists. It is simply renaming one event as another, as in, changing the event key to another event key. If these events have different segmentations from the events that will be merged, then these events’ segmentations simply add as a new segmentation to the event. But if the segmentations are the same, then the first one’s description is used for listing and reporting.

Important Note: Transformation is permanent, so it cannot be reversed for existing data. Transformation can be disabled from the UI but historical raw data cannot be reversed. Only new incoming data will be updated when the transformation is disabled.

You can merge events in either of the following ways:

Merging by selecting events (for existing events)

This is useful for merging a few events into an existing event or a new event name. You can select the event from the dropdown and merge the selected events to this existing event. To use this method, follow the below steps:

1. Click the Create New button and then click on Create New Transformation

   

2. Select the Merge by selecting events option
3. Select the events
4. Select the event which will be used to rename and merge selected events
5. Click the Save button to complete the transformation.

Merging by Regex

If you need to merge multiple events to one event, like renaming 1200 event keys and merging them to a new event or existing event, it would be easier to do so by regex, rather than handling each case separately.

This option allows you to rename multiple event keys by Regular expressions instead of selecting them manually. Events are selected by regex for event keys and changed with the provided value. As a result, multiple events would be merged. To use this method, follow the below steps:

1. Click the Create New button and then click on Create New Transformation

   

2. Select the Merge by Regex option
3. Enter regular expression in the input field to select event_keys
4. Then enter the new value you want to use for event_keys and Save.

Use Case

Event	Segmentation 1	Segmentation 2
Event1: add_card_item	item_name	purchase_count
Event2: add_card_category	category_name	purchase_count
Event 3: Add_card	item_name	page_title

If a user needs to merge Event1 and Event2 to Event3 with the purpose of no longer tracking the same user behavior with different events and seeing the correlation between these data under a single event, the user should use Transformation. After transformation, the user sees Event 3 on both aggregated data (e.g., Events charts) and granular data (e.g., Drill queries). Note that it will be valid for both existing and incoming data.

The new structure would still show separate event documents (e.g., Drill has a separate document for each occurred event):

Event	Segmentation 1	Segmentation 2
Event1: Add_card	item_name	purchase_count
Event2: Add_card	category_name	purchase_count
Event 3: Add_card	item_name	page_title

But for meta information it would show all these segments. For example, in the Drill query dropdown for Add_card event it would show all the segments to be able to select.

Event 3: Add_card	Segmentation
Segmentation 1	item_name
Segmentation 2	page_title
Segmentation 3	purchase_count
Segmentation 4	category_name

• Event 1 had a segmentation (item_name) that matched that of Event 3 (item_name), so the segmentation with the same and unique segmentation name was directly merged to the existing segmentation of Event 3.
• Event 1 and Event 2 had the same segmentation with the same segmentation (purchase_count) but Event 3 didn’t have this segmentation before the transformation. So, purchase_count data of both Event 1 and Event 2 was merged to the new segmentation of Event 3 named Purchase Count with purchase_count segmentation.
• categoryName segmentation of Event 2 was added to Event 3 as a new segmentation.

Users can disable/enable/edit or delete transformations from the UI. When a transformation is deleted, transformation on existing data cannot be reverted. But transformation on incoming data will stop.

Event Groups

Event Groups allow you to combine multiple events into a single unit for consolidated data analysis. For example, if you have similar events that you track, grouping them together can provide a clearer overview. These grouped events will also be available in the drill and other features, which lets you access more detailed data.

When you navigate to the Event Groups tab, you will see the following information:

1. Name of the Event Group
2. Description of the Event Group
3. An option to change the visibility of the selected group.
4. An option to delete the event group.

Creating an Event Group

To create a new event group, click on + Create New and select Create Event Group.

1. Enter a name for the group.
2. Add a description for the event group.
3. Select the events you wish to merge.
4. Use the toggle button to adjust the visibility settings for the group.
5. Enter any properties related to the event group. To learn more about these properties, click here.
6. Click Save to finalize the creation of the event group.

Editing an Event Group

To edit an existing event group, you can either:

• Click on the three-dot ellipsis in the event group drawer and select Edit.
• Or click on the specific event group name and then select the Edit Group button to make changes.

Segmentation

Event segmentations can be found on the Segmentation tab in the Data Manager. The list of segmentations contains some key information.

• Segmentation name: The name of the segmentation
• Event name: Event name of the segmentation
• Status: It is a predefined field and can be Created, Approved, Live, Unplanned, or Deleted

• Last modified: The date and time this event was most recently modified in the format of DD/MM/YY hh:mm
• Last modified by: The name of the user who last modified the segmentation
• Visibility: Visibility status, as inherited from the parent event.

  

Some of these columns are shown on the table by default. The rest are customizable and can be added to the table as per requirements.

On this tab, you can:

• View all segmentations with their events
• Edit status
• Edit segmentation description
• Omit segmentations
• Delete segmentations
• Search for an event or segmentation
• Filter segmentations by status, name, and event
• Export segmentation list with all data shown on the table
• Transform

Note:

• The Segmentation tab is accessible by Countly Lite users with only the Omit option on the action button. Transformation will not be accessible by Countly Lite users.

Editing and Managing Segmentations

Users can update the below fields in segmentations:

• description
• Status
• Type
• Required (When the validation checks an incoming event with a segment key defined as required then the segment key should be in the segmentation object of this event. If the required segmentation key is not present, then a validation error should be triggered, flagging the missing segmentation field and its parent event)

To update any fields of the segmentation, follow the below steps:

1. Click the segmentation you want to edit
2. Click the Edit Segmentation button on the Segmentation detail panel
3. Update the status from the drop-down list
4. Update the segmentation type from the drop-down list
5. Type in description field to update the description
6. Uncheck or check the Required button to update this option

Note: The segmentation detail page is not available for Countly Lite users.

You also have the option to update status by selecting the event from the checkbox, and then selecting the action (update status or delete) you need from the action bar.

Deleting Segmentation

This is available only for Countly Enterprise users. Deleting a segmentation will remove the segmentation from the Events feature and other query builder drop-downs like Drill, which means that you will no longer be able to query on the segmentation or see data for the deleted segmentation. The same logic described on the Delete Event topic is valid for the Segmentation delete process.

To delete segmentations, you need to follow the below steps:

1. Click the checkbox next to the event name in the table on the Segmentations tab.
2. Click Delete from the action bar at the bottom of the page.
3. A verification modal will appear for the deletion process. Confirm your deletion request.

Alternatively, you can delete a segmentation from the segmentation detail page by clicking on the 3-dot ellipsis button to the right of the relevant row, and clicking Delete.

Transformation

You can apply transformations on segmentations to:

• Merging Segmentation
• Renaming Segmentation
• Modifying Segmentation Value
• Copy to Custom User Property

Merging segmentation

This allows you to merge multiple segmentations into a new segmentation. The values for segmentations will be replaced (overwritten) by the segmentation being merged. See the example below:

Before merge:

Event: contactButtonclick
Segmentation 1= page		Segmentation 2= trigger
Value	Count	Value	Count
/	256	ab-testing-plugin-link	1
/about	8	academic-program-contact	1
/academic-program	1	apply-for-partnership	4
/academy	3	automated-push-notifications-plugin-link	1

The new structure will be the same on DB, but still have separate segmentation documents (like Drill has a separate document for each occurred segmentation)

Event: contactButtonclick
Segmentation 1= page		Segmentation 2= trigger
Value	Count	Value	Count
/	256	ab-testing-plugin-link	1
/about	8	academic-program-contact	1
/academic-program	1	apply-for-partnership	4
/academy	3	automated-push-notifications-plugin-link	1

For meta information, however, it would show only the new segmentation For example, in the All Events drop-down for the linkClick event, it would show one segmentation (new one) to be able to select.

Event: contactButtonclick
New Segmentation = pageTriggers
Value	Count
/	256
/about	8
/academic-program	1
/academy	3
ab-testing-plugin-link	1
academic-program-contact	1
apply-for-partnership	4
automated-push-notifications-plugin-link	1

You can merge segmentations by following the below steps. Click the +Create new button and then select Create new Transformation

1. Choose the Merge option by checking the radio button
2. Select the event from the drop-down
3. Select the segmentations to merge
4. Enter the new segmentation name
5. Select a transformation method that will apply to the data (existing data, incoming data, or both)
6. Click the Save button to complete the transformation.

A transformation is permanent, so modifying all the data cannot be reversed. Transformation can be disabled from the UI but historical raw data cannot be reversed. Only new incoming data will be updated when the transformation is disabled.

Renaming segmentation

This is used to rename segmentation for incoming data. Note that if you rename a segmentation from the Schema, it will only cover the existing data, not incoming data. But with Transformations, you can choose the transformation option for existing or incoming data, or both. In case both existing and incoming data are renamed, then both Rename options should be performed to rename both existing and incoming data.

To rename segmentations, follow the below steps:

1. Click the Create New button and then select Create Transformation
2. Choose the Rename option by checking the radio button

   

3. Choose “Rename with Specific Value” or “Rename with Regex” with Specific Value”
4. If the “Rename with Specific Value” option is chosen:
   • Select the event from drop-down
   • Select the segmentation from drop-down
   • Enter the new segmentation name (if it is renamed to segmentation that already exists, then it would overwrite the value in existing segmentation. Countly display a warning message after checking for conflict)
5. If the “Rename with Regex” is chosen:
   • Enter regular expression to select segmentations for the transformation
   • Enter the new segmentation name (if it is renamed to segmentation that already exists, then it would overwrite the value in existing segmentation. Countly display a warning message after checking for conflict)
6. Click the Save button to complete the transformation.

After the process is completed, the Schema is automatically regenerated in the background.

Modifying Segmentation Value

This allows the modification of event segmentation values (only values). To modify the segmentation value, follow the below steps:

1. Click the Create New button and then select Create Transformation

   

2. Select the Modify segmentation value option from Transformation types.
3. Select the segmentation from drop-down
4. Enter regular expressions to find values, like (title) for the given example above
5. Enter new value
6. Click the Save button to complete the transformation.

Use Case:

Keep consistency for taxonomy on segmentations:

value 1= pageTitlevalue 2=page_titlevalue 3=title

Select and modify them as PageTitles.

Copy to Custom User Property

This option allows you to copy a segmentation to a custom user property. Unlike other transformations, copying to a custom user property automatically applies to both existing and incoming data without requiring you to choose a specific transformation method. To copy segmentation to a custom user property, follow these steps:

1. Click the Create New button and then select Create Transformation

   

2. Choose the Copy to Custom User Property option by selecting the radio button.
3. Choose whether to use an existing custom user property or create a new custom user property
4. Select the Event and the segmentation you want to copy.
5. Once the options are configured, click Save Transform.

Omitting Segmentation

This allows you to delete segmentations from aggregated data only, and keep recording granular data as string. Omitted segments will not be saved in the future, and past data for these segments will be purged immediately after you save these settings. Data for omitted segments will still be stored in Drill.

To omit segmentation value, you need to follow the below steps:

1. Select segmentation by clicking the checkbox next to the segmentation name on the Segmentations tab.
2. Click the Omit button from the action bar at the bottom.
3. A verification modal will appear, instructing you to type in a confirmation phrase to continue with the omit process. Confirm your omit request.

Bulk Edit

Bulk edit is a great feature for when you have hundreds of events and need to update the event-based data structure. Using Event Schema, you can update event and segmentation fields over a CSV file.

The fields that can be planned/updated via Event Schema are:

• Category
• Event Key
• Event Display Name
• Event Description
• Input Field (if required)
• Status
• The number of Segmentation Event includes
• Event Segmentation
• Segmentation Description
• Segmentation Type
• Segmentation Status
• Segmentation Is Required
• Segmentation Is Array
• Segmentation List Options
• Segmentation Regex

You can export and import the Event Schema via CSV to plan or update events and event segmentations. Please check the table below to see the structure.

Updating Status

To update the status, follow the below steps:

• Export Event Scheme as CSV file
• Type one of the available statuses – Created, Approved, Live, Deleted, or Unplanned. For blocked events, you have to use the Filtering Rules feature. Even if you change the status of blocked events (e.g., blocked to created) and see the updated status on UI, Countly will keep blocking it until you disable Filtering Rules.
• If the unplanned status is changed to created or Created, Approved, or Live, it means that these events or segmentations with unexpected status are approved. For live status, Countly uses these events/segmentations data for all charts and tables.
• Avoid breaking the sheet structure to cause import problems.
• Upload updated schema as CSV file and preview changes before publishing them. Click the Publish button to approve all changes.

Updating Description, Category, and Input Fields

These fields can be updated via a CSV file by following the below steps:

• Type in description or category or input field user-defined to update them.
• All input fields are limited to the SDK limits. Don’t use special characters.
• If the “Segmentation Required” cell is set to “True” for any segmentation, the description field of that segmentation will be a mandatory field. Therefore, you cannot leave description fields empty for required segmentations.
• Empty cells in the Category column will be shown as “uncategorized” on event overview table on the UI.

Adding New Input Field

Input fields are used to classify events or add description/tag to groups, and for easy identification of them. To add a new input field, you will need to do so by adding a new column. Add a new column for extra definitions or notes, by following the below steps:

• Add a new column in the CSV file and type in its title (business goal), limited to the SDK limits. All input fields are assumed as input text fields by Countly.
• Enter text for each cell to the added input field.
• A maximum of 2 new columns can be added. If you add more, only the first 2 of them are imported to the Countly UI. The order of the columns is considered while selecting the input fields from more than 2 columns.

Updating Segmentation List Options

The Segmentation list option is used for “list” type segmentation, to provide list items. If you update any item from the list, these options will be updated automatically on the Events chart and drill. This update is retroactive and affects historical data.

Segmentation Regex Validation

Segmentation Regex on the Event Schema is used to validate segmentation values. You can set a regular expression to define a certain pattern for segmentation value.

Examples:

• User can set regex for segmentation value that included only data from a provided list like Sign up with email or Signup with Google or Signup with Twitter.
• PII Data Validation, where all segment values should be validated for PII data. If any PII is found, the validation warning message should indicate this.

Bulk Edit Draft

Event Key**

Event Display Name

Event Description

category

Input Field (e.g., Business Goal)

Status

#of S Event includes

Event S Name

S. Description

S Type

S Status

S Is Required

S List Options

S Regx

S Regx condition

Example set

login

Log in

When login form is sent.

User Account

Will be triggered on /login page

live

3

type

which type of login option is used

list

live

TRUE

email, Facebook, Twitter

email|Facebook|Twitter

match

errorID

which Error code is shown for fields

list

live

FALSE

200 400 404 513

errorMessage

list

created

FALSE

Responded successfully. Request is malformed. Not found. Server error.

*S = Segmentation. Due to narrow cell size, Segmentation was shown as S but it will be given as Segmentation on the CSV file.

Validations

An event is any type of action that you can send to a Countly instance. All data passed to the Countly server via the SDK or API should be in UTF-8. So, it is not possible to collect data not sent by SDK if:

• event_key doesn’t match exactly (with the event_key set on Event Schema); response warning message and store data as “unplanned”, or
• segmentation type doesn’t match exactly (with the segmentation type on Event Schema); response warning message and store data as “unplanned”, or
• segmentation name doesn’t match exactly (with the segmentation name on Event Schema); response warning message and store data as “unplanned”, or
• event name doesn’t match exactly (with the event name on Event Schema); response success message and count it for the event that matches the event_key of the sent data, or
• PII data is detected; drop it and show the error message for the source and destination, or

When the validation checks an incoming event with a segment key defined as required then the segment key should be in the segmentation object of this event. If the required segmentation key is not present, then a validation error should be triggered, flagging the missing segmentation field and its parent event.

Use Case

User A plans an event on Countly Data Manager and makes it status as live:

event_key = submit_buttonEvent (display) name = Click Submit ButtonSegmentation name= button_IDSegmentation type = string

But User B implements them to SDK and data is sent via SDK as given below:

event_key = submitButtonEvent (display) name = Click Submit ButtonSegmentation name= buttonIDSegmentation type = string

Countly checks all sent event and segmentation data before using it on your data, and validates them with the Event Schema. Checklist of this validation process is:

• Event_key
• Segmentation name
• Segmentation type
• Status (for both segmentation and event)
• PII data

For this example:

❌ event_keys of planned event on UI and sent via SDK don’t match

➖ Event display names are same but doesn’t matter for validation, i.e., out of the checklist

❌. Segmentation type of planned event on UI and sent via SDK don’t match

✅ Segmentation types of both are same

✅ Status of the event created on UI is live

So, Countly triggers a warning message on UI and doesn’t inject sent data to user data. But Countly still keeps counting sent data till you block it. If you approve the unplanned data, it is injected to user data and this event is listed on Event Schema; it will be visible on Events, Drill, and other drop-downs using events.

If Segmentation Regex is used to detect PII data or manage segmentation value in a defined pattern, a warning message is triggered for any error. Countly doesn’t save or collect data, and instantly drop it.

List of Validation Errors

Error message	Explanation
Unexpected event	Event status is NOT live Or Unplanned event
Unexpected segmentation	Segmentation status is NOT live Or Unplanned event segmentation
PII data	PII data was detected and dropped
Missing data	Required segmentation is missing
Mismatch	Event segmentation does not match data type

Countly provides 2 options as validation rules:

• Allow unexpected events and automatically use it in the Event Schema -If you select this option as a validation rule, Countly doesn’t show any warning on UI and assumes it as a new event or segmentation. So it is automatically triggered, without any warning, and added to the Event Schema.
• Trigger a warning and hide the data -If you select this option as a validation rule, Countly shows a warning on UI, marks it as unplanned, and keeps it on the Validation page until you approve it or block it from Filtering Rules. If you approve it from the Validation page, then it is added to Event Schema as approved but it still won’t be used with other data until it you move it to live status. During this stage, Countly counts and saves it but doesn’t inject this data to used data on UI (tables, charts etc.)

You need to send these from Settings.

Note: You can set up alerts for warnings by using the button on the Validation tab. When you click this button, you will be directed to the Alerts feature. For this feature, the Alert feature must be enabled. If it is disabled, the Alert button will be disabled.

Expected Behavior for Existing Events and Default Settings for v22.x

For Countly Enterprise

All events and segmentations created before this release (existing events) will be assumed as status=live for once, and you won’t get any notification (validation error) on the product. Any events sent after upgrade time, will be flagged as unexpected. Also, Validation on the Management/Settings view is set as Allow unexpected events and automatically use it in the Event Schema by default. So, you won’t get any notification for existing events and if you send a new event data, you can see new event’s data on their dataset but Countly flags them as unexpected on Event Schema (Data Manager/Events and Segmentation).

For Countly Lite

All events and segmentations created before this release (existing events) will be assumed as status=live and you won’t get any notification (validation error) on the product. Any events sent after upgrade time will be flagged as live. Validation configuration isn’t available in Countly Lite.

Data Type

Available only for Countly Enterprise, this allows you to change the data type of both event segmentations and user properties, including custom properties. In this way, data types are updated without any need for custom script coding to fix a mistake made at implementation.

This feature was part of the 20.11 release. However, with Countly 22.03, it has been added to the Segmentation tab and Manage User Properties detail page.

• Edit Event Segmentation Type - Event segmentation type can be changed from segmentation detail page. Default data type is also shown. You can select a new data type from the drop-down.
• Edit User Property Data Type - On the User Property tab, you can update data type of user properties and custom properties. Default data type is also shown. You can select a new data type from the drop-down.

Example:

A user can change the string data type of “location” property to list type. Then, it is shown as drop-down on query builders and user selects an item from the list.

Data Regeneration

Data regeneration is generally needed after any changes are applied to metadata or aggregated data. For example, if a user has some missing fields in Drill, or if the user has switched data type to a big list and they now want to populate it with data, they can regenerate metadata to populate those inputs. Or if you deleted a lot of granular data, modified granular data, or over reported users or new users, you can regenerate aggregated data to get new fresh stats directly from the current state of granular data.

Two data regeneration options are provided:

• Regenerate Metadata
• Regenerate Aggregated Data

When regenerating aggregated data one of three data types from Sessions, Events, and Views are selected. If Events or Views are selected a further selection must be made regarding which Event or View is to be regenerated as only a single one can be regenerated at a time.

To regenerate data, follow the below steps:

1. Click the 3-dot ellipsis button on the top right side of the page
2. Click on Regenerate Data
3. Select one of the choices for the regeneration process:
   • Regenerate Meta Data
   • Regenerate Aggregated Data
4. If Regenerate Aggregated Data is selected, choose the Data Type to be either Session, Event, or View
   • If Event is selected, pick an event from the drop-down list.
   • If View is selected, pick a page from the drop-down list.
5. Select the time period for which the data regeneration should happen using the date-picker
6. Click the Save button at the bottom of the page

Interaction with Other Features

Event, segmentation, and user properties tables are automatically created on Data Manager for existing users after the Countly 22.03 upgrade. For Enterprise Edition users, Event Schema is initialized when the first validation rule is set or during the first export of the Event Schema. All visible events with live status should be available on the Event feature (e.g., drop-down on All Events) and query builders that use Events. If any event is hidden on Data Manager, the event will be hidden on the Events feature.

User Roles and Permissions on Data Manager

The Data Manager feature will be available for both Countly Enterprise and Cuntly Lite users; however, the Countly Lite version will be limited to features of Manage Events as in Countly 20.11.

The following table show which "Manage Events" actions (vertical) will be possible with the given user permissions (horizontal):

	Create	Read	Update(Edit/Write)	Delete
View	-	●	-	-
Plan Event	●	-	-	-
Download Schema Template	-	●	-	-
Export Schema	-	●	-	-
Import Schema	-	-	●	-
Export Event Table	-	●	-	-
Edit Event	-	-	●	-
Delete Event	-	-	-	●
Transformation	-	-	●	-
Change Category	-	-	●	-
Change Status	-	-	●	-
Change Visibility	-	-	●	-
Omit (for segmentation)	-	-	-	●

 

The following table show which "Validation" actions (vertical) will be possible with the given user permissions (horizontal):

	Create	Read	Update(Edit/Write)	Delete
View	-	●	-	-
Approve Warning	-	-	●	-
Delete Warning	-	-	-	●
Set Validation Rule	-	-	●	-
Update Validation Rule	-	-	●	-

 

Managing User Properties

In the User Properties Tab, you can manage actions related to user properties.

Understanding User Properties Action

• Data Redaction- Data Redaction allows you to mask Personal identifiable information (PII) for compliance purposes, such as GDPR. When applied, redacted values are hidden in the UI, database queries, and API responses. However, the recorded data remains stored in the database.
• Deletion- If you are certain that a user property is no longer needed, it is best practice to perform a data cleanup. You can delete the user property through the Data Manager. However, if devices continue to send data for this property, it will reappear in the UI. To prevent this, ensure that the property is also removed from your SDK implementation.

1. Property Name- The name of the user property.
2. Data Redaction- Indicates whether the data has been masked or not.
3. Expiration- Displays the expiration date of the user property.
4. Property Type- Shows the type of the user property.
5. Data Type- Displays the specific data type for the user property.

Transformation

You can create new transformations to merge multiple user properties, modify custom property values, or rename custom property values. 

Merge Custom Property

This is useful for merging two or more custom user properties. This is especially useful when you have multiple properties tracking similar data and you want to consolidate them into one for easier management and reporting. To utilize this, follow the below steps:

1. Click on +Create New and click Create new transformation

   

2. Select the properties from the dropdown.
3. Give a new name to the property.
4. After selecting the Transformation method click on Save.

Modify Custom Property Value

Modify Custom Property allows you to change the values of existing custom properties using regular expressions. To modify a custom property value, follow these steps:

1. Click the +Create New and select Create New Transformation.

   

2. Choose the property you wish to modify from the dropdown menu.
3. Use the Regular Expression field to define patterns or input a specific value to replace.
4. Enter the Value to Replace with your desired new value.
5. Select the appropriate Transformation Method (Incoming, Existing, or Both), then click Save.

Rename Custom Property

Rename Custom Property lets you rename an existing custom property either with a specific value or using regular expressions. Follow these steps to rename a custom property:

1. Click +Create New and select Create New Transformation.

   

2. Select the property you want to rename from the dropdown menu.
3. Choose whether to rename using a Specific Value or a Regular Expression.
4. Enter the New Property Name for the selected property.
5. Select the appropriate Transformation Method (Incoming, Existing, or Both), then click Save.

Editing User Properties

To edit a user property, select a specific user property and click on edit in the three-dot ellipsis menu.

Here, you can change the data type or enable Data Redaction for the property.

Expiration of User Properties

There are two ways to set expiration dates for user properties:

1. At the End of the Session- This option clears the user property’s value at the end of the user’s session, useful for temporary data such as language preferences.
2. After a Given Number of Days- This option allows setting an expiration date based on a specific number of days. For example, you may want to track a 7-day trial subscription, after which the user property should expire.

You can also enable Data Redaction when setting an expiration.