Surveys is used to run custom surveys to collect, store, search, and track user feedback from web and mobile applications. This feature is available in Countly Enterprise. The Countly Surveys plugin allows you to create both user property-based and behavior-based surveys, which means you can customize your surveys depending on your user segmentation.

What are Surveys used for?

User research and customer feedback collection are key inputs for any successful product or service. To better fulfill your customers’ needs, you must listen to your target market and get first-hand insights about its needs, requirements, and pain points. Understanding this is crucial to launching a product that has a strong value proposition and delivers on the essential needs and expectations of your target market, leading to enhanced customer satisfaction.

Surveys provide first-hand, immediate feedback from both high engagement users, who may give you specific opinions about your product, and low engagement users, who can give you critical insights. With the Countly Surveys plugin, you can customize your surveys and target specific segments of your app or website users based on various conditions, such as app version, performed event, or other custom user properties.

The main benefits of using the Countly Surveys plugin are: 

• Receive feedback from a segmented user base

• Get a high response rate

• Collect and test user response to specific or new features

• Analyze your users’ product experience

• Identify pain points in the user experience

• Enhance user retention and lower churn rates

• Powerful market research

• Give your users a voice and analyze verbatims

Surveys Types

There are 2 types of Surveys based on user/customer targeting:

Attributional: This is used to collect responses from users based on user properties like city, app version, device, etc.

Behavioral: This is used to collect responses from users based on performed or not performed events (e.g. clicked on “add to cart”).

How to run your Surveys and analyze them on Countly

1. Enable the Surveys plugin

The Surveys plugin is enabled by default in Countly Enterprise and you can find it in the left main navigation menu: Reach>Feedback>Surveys. If you still don’t see it, please enable it from Settings/Plugin on the top right corner. If you don’t have access to these settings, please contact your account admin.

2. Creating a Survey

On the left main navigation menu, in the Reach section, click Feedback link to access the sub-menu and choose the Surveys plugin.

You will see a blank Surveys dashboard, so start by creating your first survey. Click the green button “New Survey” on the top right corner to start.

Creating a Surveys widget on Countly has 3 simple and easy steps:

1. Settings & Questions
2. Customize Appearance
3. Targeting

2.1 Settings & Questions

This first step contains the inputs necessary for creating the survey and includes:

1. Survey Name - for internal purposes. It will not be displayed publicly.
2. Questions

There are 5 types of questions in Countly Surveys:

Open-ended questions: These questions require respondents to type in their answers. Such responses help companies to better understand individual users/customers. However, it’s quite difficult to analyze collected data with open-ended questions because responses are not standardized and are hard to categorize.

Drop-down lists: Drop-down lists are an easy way of showing a long list of potential answers. They allow respondents to choose a single answer quickly.

Rating scales: These questions provide a scale of your choice – from 0 to 10, for example. A respondent selects the number that best represents their answer.

Multiple choice: These questions allow respondents to select more than one answer from a list of options.

Radio button: These buttons allow respondents to choose only one answer from the listed options. Choosing a second option automatically deselects the first one. 

❗ Tip: You can add a maximum of 5 questions to each survey.

Once you complete the “Survey Name” field and add at least one question, you will be able to move on to the next step. If you leave the “Survey Name” field empty, the Next Step button will not be active until you enter a name for your survey.

On the left side of the screen, you will see a widget preview with its questions and thank-you message. All questions can be customized by question types and by editing the possible choices. You can also mark the questions as “Required” and randomize the order of choices using the provided checkboxes. 

❗ Tip: Make your survey name identifiable, short, and simple. Avoid using special characters or long names for the best appearance of the survey widget.

2.2 Customize Appearance

Countly allows you to choose how and when exactly you would like a survey widget to be shown to your targeted audience, and whether or not the same user should see the survey more than once. There are two types of settings for these options depending on what user behavior triggers the appearance of the survey widget and what widget persistence is set:

• Behavior that triggers the widget:

After the page loads: The survey widget will be shown once your website or app is fully loaded.

After a delay of 10 seconds: For web applications, the user needs to remain on the same URL for 10 seconds for the widget to be triggered. For mobile apps, the user needs to remain on the defined page for 10 seconds. In both cases, the survey widget will be shown to the user after 10 seconds.

When the user abandons the page: Available only for desktop devices, the survey is triggered when the mouse moves towards the top part of the browser.

When the user scrolls halfway down the page: Once the user reaches 50% down your page, the widget will be triggered. 

• Widget persistence:

Until they submit a response: This is the default option. The survey widget will be shown until the user responds on every new session based on targeting parameters.

Until they close the widget: The survey widget will be shown until the user closes the widget based on targeting parameters. If the user closes it, the same survey widget will not be shown again and it counts for the Total Shown metric but will not be considered in the Responses calculation (see 3. Overview Surveys below).

You can customize the main widget color and style, which point to both main widget colors and submit button appearance. You can also add a logo. Your choices will be reflected and shown on preview on the left side.

2.3 Targeting

The Survey plugin of Countly provides you with enhanced options for both page-based, behavioral (i.e. based on user cohort) and event/action-based targeting. It is mainly a 2-part segmentation mechanism: user property segmentation and user behavior segmentation. The slight but important difference between them is that user property segmentation is not dependent on an event.

User property segmentation: it helps you narrow your target group of users down using user properties. For instance, you may want to only show iOS users AND individual customers defined by a custom user property named accountType. To do that, you can use the query builder in the "USER PROPERTY SEGMENTATION" panel as shown in the image below. You can always extend your query to make your target set even more specific. If you don't want to specify any user properties, you can skip this stage. 

User behavior segmentation: They can be added once you are done with User Properties. First, you must select the users who have performed/not performed a certain event or session. Then, set the frequency and time range. When you complete entering these criteria, add property will become enabled. 

You can add filters such as country of a user, last entry time, input method, etc. and you can group similar users. The more filters you use, the more granular and deeper your data will be.

You can keep adding conditions by using the + Add Condition button.

Use case example: You can get to know your users’ first shopping experience on your e-commerce app by routinely running a survey to gather contextual feedback from users with a few questions. You can do that by following the next steps on Countly: 

1. Create a new Survey.
2. Define your questions, such as “How do you rate your shopping experience on our app?” or “What did you like most on our app?”
3. Customize the survey appearance.
4. Define your target audience using the following conditions for User Behavior Segmentations:

• users who performed clickPayment event is 1 time during all time
• which paymentResult is successful

You can also create a Cohort for your target audience on Countly Cohort plugin and just set a rule by using this Cohort.

By default, survey widgets are enabled. If you want to disable them, just uncheck “Set widget active” checkbox. You may also change the survey widget status, appearance, and settings after you create it.

Now your survey is ready to go public.

3. Overview Surveys

After you have created at least one survey, you will see a list of surveys - active or inactive - and a general report showing the overall results for the running surveys. 

Running Surveys: Total number of active surveys.

Total Responses: Total number of responses given by each individual user who responded to each survey. If a user responds to more than one survey (i.e. gives responses 2 times for 2 active surveys) this is counted as 2 instances on Total Responses.

Response Rate: Percentage of the Number of Responses over Total Shown

Below the Overview box, you will see your list of Surveys: there is a toggle button to quickly activate or stop any particular survey.

4. Individual Survey Detailed View

Further details about each particular survey can be accessed by clicking on the survey name or by clicking on the “Detail” button on the right hand side.

Upon accessing the details of an individual survey you will see:

• A top box with its general information, such as status and chosen target (see Section 2.3).
• A “SUMMARY” box covering the same details found in the general Surveys Dashboard:
• “Times Shown” expresses how many times your survey was shown to the target audience. Some users may not be willing to respond to your survey, so it is important to know this metric. For example, if you set your survey widget persistence for until they submit a response (see Section 2.2), the Times Shown number will be probably high.
• “Responses” shows the total number of responses you have received for that particular survey.
• “Response Rate” is calculated by Times Shown divided by total Responses.
• You can track changes through time by selecting a particular time frame with the calendar menu on the top-right corner. The filter will affect both the Summary results and the table with Results/Questions underneath.
• Two tabs labeled “Results” (selected by default) and “Responses” on the table. 

4.1 Results tab

This tab shows the results of each individual question.

Note that each question type (see Section 2.1) will display responses differently.

For open-ended questions (text based answers), you see the last 10 answers. If you want to see all the answers, click on the “View All Responses” link below these questions. You can apply a date filter or platform and app version filter to responses. You can also export the results as a .csv, .xlsx, or json file.

4.2 Responses tab

This tab shows the responses of each individual survey, displaying the responses by each user. You can filter responses by questions.

❗ Tip: Difference between the Surveys, NPS, and Ratings plugins: they are all Feedback plugins, but they address different customer experience aspects: 

• • • Surveys run based on behavioral or property segmentation, providing flexibility to customize up to 5 questions and to customize target segmentation.
    • NPS is triggered when users/customers perform an event and has a very standardized calculation method.
    • Ratings can be added to any page and incorporate a scale of emojis to answer a very straightforward customer satisfaction question. Their segmentation is based on the user’s device and the page they view.

FAQs and Troubleshooting

Can I run Surveys based on behavior of users, like a performed event?

Yes. Countly provides you the ability to target users based on performed events such as a clicked button, the submission of a form, or a successful money transaction. Also you can target users based on user properties like country, membership type, language, etc. 

Why can’t I visualize a survey?

When Surveys are triggered by user properties, these should be determined by Cohorts. And because each cohort has a determined regeneration time, a cohort needs to be regenerated for the user to see the widget. 

The logic behind the visualization settings of Surveys widgets depends on the type of SDK, mobile or web.

In general, when determining these settings, note that survey persistence options (“Customers are going to see the Survey widget” in the image above) are valid for both mobile and web SDKs.

In web SDKs, the survey behavior option (“Users will see this survey”) needs to be chosen as well, because the web SDK fetches the response data from the server, as shown here: the web reporting logic needs an API to fetch the survey from the server as well as to display them to the end user. You can find more information here.

In mobile app SDKs, feedback widgets are not automatically shown, so app developers need to display them manually.