by Christopher Rozum

GUEST COLUMN In order to thrive in today’s competitive market, companies need to listen to their customers with the objective to obtain an understanding of their specific requirements. Many companies that are recognized as leaders in their market, have made investments to implement a comprehensive Customer Listening System (CLS). A component of a CLS system is reaching out to customers through a Customer Satisfaction (CSAT) survey process to obtain insight into their specific requirements and their perception of the company’s current performance against these requirements.

(Source: Merging Elements Quarterly, 2008 Issue 1)

Customer Satisfaction is one of the most complex metrics in the contact center industry to both measure and improve performance. It takes a delicate balance of science and art to deliver a exceptional experience that delivers and even exceeds customer expectations.This article is meant to provide the answers to the 8 most common questions we have faced over resent months during engagements with clients.

To answer these 8 questions, we tapped into our benchmark database and analyzed the CSAT survey process from 152 companies across various industries including: Financial Services, Insurance, Technology, Entertainment, Telecommunications, Travel, and Manufacturing.

QUESTION 1: WHY HAVE A CSAT SURVEY PROCESS? As pressures continue to rise for operating costs to be decreased, the investment dollars spent surveying customers is under great scrutiny. One of the reasons companies are evaluating their CSAT survey process is in large part due to the lack of value they receive from the output. With many companies in this situation, we have found the root cause to be unclear objectives for the CSAT survey process. Too often companies make the decision to begin surveying their customers and dive into the implementation without first understanding the output that they want to obtain from the data.

The first step to build a world-class CSAT survey process is to clearly define its strategic objectives. Taking this approach reduces significant rework and unnecessary expense. The companies in our benchmark database that delivered the highest CSAT performance clearly defined the details for each of the following objectives for their CSAT survey process:

1. Understand customer requirements
2. Measure and improve satisfaction performance
3. Measure and incent vendor performance
4. Benchmark their performance relative to direct competitors and other industries

Sophisticated analytics (translate customer requirements into operational initiatives, set performance targets) Invest time to understand and define the details of each of these strategic objectives. The output of this step will establish a foundation from which to build a world-class CSAT survey process that delivers significant value.

QUESTION 2: WHAT IS THE RIGHT METRIC TO MEASURE CSAT PERFORMANCE? We have found that many companies really struggle to answer this question because there is huge variation in opinions from industry experts. Each market research firm, statistician, and data analyst can have varying opinions on the correct metric. The three primary elements under constant debate are: 1) what question(s) on the survey to use to calculate the overall metric, 2) what scale to use for the response to each question(s), and 3) metric calculation.

Of the 152 companies analyzed, there were three primary questions used on the surveys to calculate overall CSAT performance. The following table illustrates the specific benefits of utilizing each question.

The next element is the scale for the response to each question(s). Of the companies analyzed, we identified the following scales used: 2pt, 4pt, 5pt, 7pt, 9pt, 10pt, and 11pt. The following chart illustrates the usage of each scale. There are numerous reasons to choose one scale over another. For example, one reason to consider using a larger scale (9pt or 10pt) is that it can produce more accurate multivariate analysis results. Companies who have prioritized benchmarking their performance may consider using a 5pt scale because of the high adoption rate resulting in data availability.

The final element is the metric calculation for overall performance. Again, there is significant variation in the industry on the actual calculation. We isolated the following four calculations as the primary methods for this metric. Of the companies analyzed, 68% were using Box Performance Score with the majority utilizing top 2-box, although, top box is gaining in acceptance.

1. Box Performance Score (top 2-box, top box) – the total number of surveys with the selected responses divided by the total number of surveys
2. Average Score – the average of all the survey responses
3. Calculated score – the score is a calculated by combining the results of several questions in the survey
4. Net Promoter Score – the % promoters - % detractors based on the “how likely to recommend” question from the survey

When analyzing the companies with the top quartile performance, 78% of them were utilizing multiple metrics to manage and improve performance. At the end of the day, the right answer is the one that delivers against the defined objectives of the CSAT survey process.

QUESTION 3: WHAT IS THE BEST CHANNEL TO CONDUCT SURVEYS? Over the last few years, we have observed the contact center industry migrate to primarily use three channels to conduct CSAT surveys: outbound phone, outbound email/internet, and IVR. Choosing the right channel is a delicate balance of obtaining the data that adds value, data integrity, and cost. When choosing the best survey channel, evaluate the following considerations. Several industry leaders have implemented multiple channels to deliver on independent objectives.

QUESTION 4: HOW MANY SURVEYS SHOULD WE COLLECT? The first step to answer this question is understanding what level to measure and manage performance. For example, some companies desire to measure the performance of the overall center while others want to measure the performance at a site, program, and even the agent level. When using the services of an outsource provider(s), collecting a statistically significant sample per vendor facilitates accountability.

Once the level to measure and manage performance is understood, the specific inputs can be entered into a sample size calculator to obtain the number of surveys to collect. Sample size calculators utilize the following variables to calculate sample size: number of customer contacts, historical performance, confidence interval/precision. The industry standard practice is set survey quotas to deliver a confidence interval (precision) of +/- 5%.

Experimenting with the levels and precision, companies can adjust the number of survey samples required to balance the business objectives with cost.

QUESTION 5: WHAT ARE THE BIGGEST DRIVERS OF SATISFACTION? While the relative impact of each company’s drivers is unique, many of the companies analyzed had Problem Resolution and Agent Authority ranked as the biggest drivers. Multivariate statistics helps companies quantify the impact specific drivers have on overall satisfaction. The following chart is an example from an Insurance company. For this organization, Resolution was the biggest driver of satisfaction relative to all other attributes.

QUESTION 6: HOW CAN I LINK CSAT DATA TO CONTACT CENTER OPERATIONS? In the absence of data analysis, center leaders use their experience and instinct to form opinions and guide operational business decisions. We define these unproven opinions as “industry myths” and this is where data analysis provides a center leader significant value. Proving or disproving myths empowers a center leader to make better and more consistent business decisions.

One of the Technology companies that we analyzed had the myth that there was a strong relationship between Service Level and CSAT performance. Utilizing this line of thinking, the operations team managed to a very fast Service Level target (90% within 20 seconds). The following chart provides an illustration of the actual relationship between these two metrics for this company. Upon analyzing the chart, the team discovered that there is no impact on how quickly they answered the phone on CSAT. After the team was equipped with the knowledge from the data analysis, they modified their target to a slower Service Level (80% within 60 seconds). The result of the target change had no negative impact on CSAT and provided a decrease in overall costs by a reduction agent FTE.

Another company in the Entertainment industry had the myth that there was a strong relationship between an agent AHT and CSAT performance. As a result, the management team was very reluctant to focus on reducing AHT to decrease costs in event that it would have a negative impact on CSAT. The following chart provides an illustration of the relationship between these two metrics for this company. The chart clearly quantified that there is no relationship between the two metrics. After empowering the team with the knowledge from the data analysis, they changed their behavior and made a 17% reduction in AHT without negatively impacting CSAT performance.

Another problem many companies have is wading through all the pages of text from their market research reports. One of the companies in our database receives a monthly report from their market research firm that fills a 3-inch binder. The operations team at this company is often overwhelmed by the data and is unable to pull out the details that will make a real difference to the customer experience.

QUESTION 7: CAN WE LINK OUR QUALITY MONITORING TO CSAT PERFORMANCE? Many contact centers use quality monitoring to measure process compliance and identify opportunities to improve the customer experience. On the process compliance side of quality monitoring, companies evaluate whether or not an agents adheres to a defined process. In many situations, the process adherence items do not touch the customer; therefore, they may not correlate to CSAT outcomes.

When it comes to improving the customer experience, many companies that we analyzed used their experience and instincts to define the items on the monitoring form. In most cases, the companies could articulate sound logic for why each item is on the form and how they thought it would correlate to CSAT. After running the quality monitoring and CSAT data through our multivariate models, we determined that none of the quality monitoring forms analyzed have a positive influence on CSAT performance.

Over the last ten years, we have perfected sophisticated models to quantify the relationship between quality monitoring activities and CSAT performance. The following tables illustrate an example for a Financial company with a baseline analysis followed by a post performance improvement project analysis. After performance improvement, the company utilized the monitoring activities to produce a 28% improvement in CSAT performance. They were also able to predict their CSAT outcomes within +/-2% facilitating the ability to make real-time corrections.

Baseline

After Performance Improvement

QUESTION 8: CAN WE LINK CSAT IMPROVEMENTS TO THE BOTTOM LINE? Tying improvements in CSAT to the bottom line is a difficult analytical problem to solve. One method is to calculate a return base on how likely customers will repurchase and/or recommend a company’s products or services. Our studies have shown this method to be only 65% accurate. From our perspective, this is not accurate enough to tie to hard financial dollars.

The best approach is to analyze CRM data with CSAT data to understand customer behavior after interacting with the contact center. For example, does a customer continue using a service or do they cancel within a short period. While this type of analysis is a much more accurate, it often takes a lot more effort and data. In some cases, it can be very difficult to obtain access to the data required to do the analysis.

NEXT STEPS The next step is to take the information learned from these 8 answers to evaluate your company’s CSAT survey process and identify opportunities to enhance it to add more value. As you complete the evaluation, please feel free to contact us with any questions and feedback.

GUEST PROFILE

CHRISTOPHER ROZUM (Founder & CEO, Insite Managed Solutions)

Christopher Rozum has more than 17 years of hands-on strategic and operational experience in the contact center industry. Utilizing his depth of experience in Captive Centers, Outsourcing Firms, and Partner Management Organizations, he has a reputation for delivering impressive results and pioneering leading edge analytics.