Analyzing Your Data

Built-in Reports

We have started this document by stating Countly is designed to be helpful not only for teams who interact with data daily but also for everyone in an organization that touches the digital customer journey. For that reason, Countly UI doesn’t assume or promote a query-centric usage pattern where the dashboard users are expected to build queries to report on and visualize data. Instead, Countly comes with built-in reports and overview screens that are optimized at a data storage and processing level for fast and continuously refreshing reporting next to its Data Deep Dive and Dashboards capabilities.

Essentially, reporting on various data points are available out of the box without the need to build queries to get the results. For instance, to visualize the most popular devices, platforms or platform versions your customers are using your apps from, you can take advantage of the built-in reports to see these metrics per number of sessions, total and new users. Similarly, the Events section provides you with an overview of the count, sum and duration properties, and you can choose to view the data grouped by a selected segmentation. As the last example out of the many built-in reports in Countly, User Profiles provides you with a list of your customers in a customizable tabular format for easy filtering & exports, whereas the Detail View of a User Profile gives you the metadata and the activity of an individual customer.

We aim to make the product analytics data more accessible and consumable to a larger audience within a company to enable organizations like yours to make more data-driven decisions for the benefit of their customers.

Data Deep Dive

Built-in reports are crucial to onboard a larger audience to a platform like Countly; however, we don’t deny that having the right toolset by your side for deeper data exploration is essential to discover insights. Our approach at Countly is making this toolset as accessible as possible so that it is not reserved just for teams with data expertise. Below you can read more about the main tools in Countly to help you with your data deep dive and how they work together seamlessly to amplify the insights.

Drill

Drill is an ad-hoc visual query builder to explore all data points in Countly. After selecting a data point, you can filter it using multiple criteria via combining them using AND and OR operators, and you can group the results using the BY operator. 

Drill lists all User Properties as filters for all data points since Countly automatically associates the User Properties to all granular data. Based on the data point you select, you also have access to additional filters, such as Events having Segments available as filters.

Drill is your tool of choice for questions such as below, but combinations of filters and grouping capabilities enable endless possibilities.

1. What’s the distribution of users who have performed event X with segment Y based on their custom property Z? Where X can be a purchase, Y can be a purchase category and Z can be the type of account the user holds to understand if there is a correlation with user account type and purchase from a specific category.
2. Is there any change in the number of sessions from users who are in cohort X or Y in the last 60 days? Where cohort X can be users who performed a certain transaction within the app and cohort Y can be users who performed another similar transaction to understand whether the app usage of users from these cohorts are increasing or not.
3. Which platforms are the users using our application on who has given an NPS lower than 7? To understand whether a lower NPS correlates with any metric such as the platform of the user thus indicating a necessity to dig deeper such as looking at onboarding, funnel completion rates and crashes in these specific platforms that might lead to unsatisfied users.

Drill has the following actions that you can take advantage of after executing your query:

1. View a list of User Profiles and explore individual ones that fit into your query criteria
2. Send a Push Notification to users that fit into your query criteria
3. Save your query into the Report Manager to use it while Building Dashboards
4. Bookmark your query to later use it again

Since Drill is a tool that works on a granular, large data set, the prefered way of visualizing frequent queries should be through adding them to Dashboards instead of running the same queries repeatedly. Also, some Drill queries, depending on your server specifications and sizing and the total size of the data set, might take a few minutes to complete. In these cases, your queries will automatically be switched to a background running mode via the Report Manager and you’ll be notified once the results are ready.

Cohorts

Cohorts is a tool to dynamically group users based on their User Properties and/or their actions and lack of actions. Cohorts work on top of the Drill data thus all filters available via Drill are also usable while creating a cohort.

Cohorts has visualization capabilities of its own to display the number of users in a cohort, the change over time and most notably the distribution of configurable User Properties among the users in a given cohort to discover commonalities. You can also visualize your Cohorts via Dashboards.

Apart from the available visualization options, other powerful aspects of Cohorts include:

• Cohorts are available as a data filter in plugins such as Drill, Formulas, Funnels and User Profiles. You can filter the reports, visualizations and list of users based on the cohorts they are in.
• Cohorts are available as a conditional filter in plugins such as Remote Configuration, A/B Testing, NPS and Surveys. You can conditionally set a Remote Configuration variable based on the cohorts the user belongs to, or trigger a Survey only if a user is in a particular cohort.
• Cohorts are available as a trigger in plugins such as Hooks and Push Notifications. You can trigger a webhook via the Hooks plugin if a user enters a cohort and likewise you can trigger a push notification if a user enters or exits a cohort.

Formulas

Formulas is a visual formula builder to construct mathematical formulas using the data points and their properties as variables and filters letting you construct new KPIs and metrics without having to track additional data points or needing to do the calculations outside of Countly. You can visualize your formulas within the Formulas view itself or via Dashboards.

Some select examples of formulas you can build include:

• Percentage of users in one or more cohorts, compared to overall users or other cohorts. For a telehealth application, there can be a cohort for each of the main medical specialities and you can create a formula for a speciality like paediatrics to see how the number of users getting help about the topic in the last 30 days compares to the overall users who had an appointment within the same period.
• Per-user average, such as finding the per-user average of the sum property of an Event X. For a navigation application, event Journey might include “kilometres driven” stored as the sum property of the event, and using formulas you can easily find out what is the average kilometres driven by users who completed a journey.
• Combined count of multiple events or combined filtered occurrences of events. For a banking application with an event to capture credit card applications and another one to capture loan applications, you can report total applications from “Premier” customers for both loans and credit cards.
• Conversation rate using the start and end events (or views) of a conversion funnel. For a SaaS application, you might be taking advantage of automatic view tracking and then you might have an event for when the user completes an account upgrade. Using formulas you can calculate the percentage of these users over users who have viewed the pricing page.

Funnels

Funnels is a user path analysis tool that you construct step by step paths using any data point to visualize users passing through and getting stuck at each step. You can use all your data points to construct your funnel steps and you can apply filters to each of the steps individually. You can visualize Funnels within its dedicated view and also via Dashboards.

Some highlights about the Funnels include:

• Funnels work retroactively. Meaning, you don’t have to create all your funnels prior to being able to explore the data, but you can create them on-demand and explore data for any past period.
• Funnels offer you the option to report the data based on the same session or session independent occurrence of the steps you select.
• Funnels automatically report time in-between average for each step, for you to visualize how much time it takes for users to move between steps.
• For each funnel step, the number of users passing through and getting stuck are presented as links so that you can navigate to the individual list of User Profiles from each step.

Zoom-in / Zoom-out

Countly allows you to explore the data in the individual user detail and at a very high level with the same ease. The User Profiles plugin is at the forefront of the zoom-in capabilities of Countly and the connection of the User Profiles to other plugins such as Drill, Cohorts and Funnels makes zooming in and out to explore the data in the detail you need exceptionally intuitive. 

User Profiles plugin groups and lists all profiles the data points originate from based on your selected user identification method. From this general list, which itself is filterable based on user properties and cohorts, you can go into the user profile detail view to see all of the below on an individual level:

• Default, reserved and custom user properties
• Cohorts the user entered into and exited from
• Datapoint timeline, including sessions, events, views and crashes
• Funnel completion progress 
• Crashes/errors the user experienced
• Performance monitoring traces
• Compliance Hub data (opt-ins and opt-outs)
• A/B tests the users have been part of together with variant information

Some examples where you can zoom into a list of user profiles include:

• Users who are a part of a Drill query result.
• Users who completed and dropped off at any step of a funnel.
• Users in a cohort.
• Users not in a cohort.
• Users at a particular retention bucket.
• Users who have experienced a particular crash/error.
• Users who have actioned upon a particular push notification.

• Create a cohort of users with wanted behaviour. From there you go into the list of user profiles that belong to the cohort, and you look into a few of the profiles to explore the activity timelines, thus their behaviour inside the app.
• Execute a Drill query with a BY operator. From there you discover a top segment of an event that you didn’t necessarily foresee. You further filter the Drill query and add in a filter to just look at the data for that particular segment value. From the query result, you go into user profiles to look into the cohort history of a few individual user profiles.
• From a funnel’s last step, you go to the list of all user profiles who have completed the funnel. You explore a few individual user profiles to look into their funnel completion rates of other funnels.