Etuma Blog

Lists by Topic

see all

Subscribe to Email Updates

Customer feedback text analysis: what works and what doesn't

[fa icon="calendar"] Jun 4, 2015 1:50:00 PM / by Matti Airas

With survey processes like NPS and CES an ever higher share of customer’s voice is open-ended text comments. Social media discussions and incoming emails and web-forms further complicate the picture. Meanwhile, the feedback volume is increasing. 

We are often asked when is the right time to put a customer feedback analysis system in place. Our rule of thumb is to do it once you average more than one thousand open-ended text fields per month. 

There are a number of ways to analyze open-ended customer comments. Some provide statistically relevant information, others don't.

What doesn't work

These methods fail because they are inaccurate, don't provide statistical information, and/or the information is too general or too granular.

  1. Extracting brands is useful information when it comes to social media and media discussions but fails to provide actionable information when it comes to your own feedback data analysis: it doesn’t tell WHAT people are talking about.
  2. Extracting whole comment sentiment based on the number of good and bad words in the whole comment and how far are they from the brand mention is quite easy but it fails to provide information about the topic of discussion: WHAT people are talking about. It also often fails to get the sentiment right because the sentiment analysis is based on a list of good and bad words and not the structure and rules of the language.
  3. Extracting keywords doesn’t work because keywords cannot be turned into statistical information: there is almost an infinite number of keywords in any language. The 'topic' of discussion is statistically distributed “too wide”.


What works

Mapping relevant brands, words and phrases into "contextual baskets" works.  We call these baskets 'topics'. Detecting the sentiment of each 'topic' mention increases the usability of the analysis results.

Analysis must be industry specific. The nature  of language dictates this: words and phrases can have a different meaning depending on the industry. Also the necessary analysis granularity level depends widely on the industry.

  • If you are a customer experience professional working for a hotel chain, you want to be able to track all the key experiences in a hotel stay: CHECK-IN, CLEANLINESS, TOWELS, BED, SAFETY, MINI BAR, NOISE, WLAN etc.) If you track the airline customer journey, a 'topic' called HOTELS might be sufficient
  • If you are a customer experience professional working for an insurance company, you want to know how your different products (CAR INSURANCE, TRAVEL INSURANCE, HOME INSURANCE etc.) are performing. If you are tracking the telecom customer experience, a 'topic' called INSURANCE might be sufficient.
Sentiment analysis must be based on language structure and rules. It is also nice to know how people feel about different aspects of your operations. Each 'topic' mention should be scored. We think that three-level sentiment scoring is sufficient on 'topic' level (NEG-NEU-POS) and five level for the whole comment (VERY NEG to VERY POS) .



Getting a lot of unstructured feedback from customers or employees? Looking for a way to analyze them? No worries, we wrote a "Guide for selecting the right method and tool for feedback analysis". Read it and you will be able to select the correct solution for your needs.

Topics: Feedback Analysis

Matti Airas

Written by Matti Airas

My passion is to figure out how to turn open-text feedback into well structured usable information.