Case Study: The Ideal Web Chat Concurrency
Find out how to identify the balancing point between efficiency and customer experience.
November 17th, 2016
How many concurrent chats can one customer service operator handle? We decided to take a close look at a few key metrics to see where the scales balance between output and customer experience.
One of web chat’s greatest assets is how easy it is to minimise the frustration of a calling queue. It can mean instant connection to an operator with little to no waiting time, if implemented correctly. No more call queues bubbling over, and overall higher customer satisfaction.
Web chat best practice means we should be taking advantage of concurrency, by giving operators multiple chats to work through at once. We can fairly assume that when operators are overloaded, the margin for human error rises, response times slow, and the customer experience will take a hit. But it’s hard to know how many chats are too many, and to identify where the true peak of operational efficiency lies.
In this operational efficiency analysis, we looked at four key live chat metrics; contacts-per-hour, response times, customer satisfaction (CSat) and internal quality ratings. We mapped them against maximum chat capacities to determine the ideal chat concurrency formula. Our client is a major UK logistics provider. Here’s what we found.
Our goal was to identify the point of diminishing returns. When does increasing web chat capacity no longer create a sizeable improvement in productivity? When does it eventually take a turn downward? Check out the graph below.
For our client we found that the highest CPH was achieved on a chat limit of 5. Those operators with 6 chats experienced a significant drop in productivity, but those with 4 or less spent a good portion of their shifts waiting for customer responses.
This is where response times come in. Response times should drop in correlation as an operator begins to reach that point of diminishing returns – the more tasks they have to complete, the slower their messages will be. Here’s what we saw:
It’s more or less what we’d expected to see. More chats means more to do, and more time in-between messages. The most significant rise in response time occurred after a cap of 5, which also lined up neatly with the large drop in productivity.
All in all, it’s fairly easy to spot an upward trend as chat concurrency rises, which curbs off around 5 and begins to drop at 6. This early in the investigation, we’d suggest a chat limit of 5.
We’re going to use two quality metrics to balance the scales against productivity – customer satisfaction, and internal quality ratings. Customer satisfaction in particular was a point of concern. Our original hypothesis was that customers would notice a slower service and feel less satisfied with their experience. As it turns out, we saw no significant fluctuation:
We can speculate that as the solutions offered by the operators didn’t change, even if they were slower, customers felt no differently about the service they received. Further, it could be argued that the change in response time wasn’t significant enough to impact on the customer experience.
Our last avenue for investigation was internal quality assessment. Could such radical changes really have no impact on the service quality? We took a much more critical view of service quality than the customer, factoring in processing, brand tone of voice, empathy and efficiency to rate interactions with a score between 1 (low) and 5 (high). Here’s what we saw:
We identified that the quality difference up to 3 concurrent chats was minimal – more fluctuation than any noticeable trend. But from 3 onward we began to see the strain on the operator taking effect, as key process points became more difficult to juggle and remember in a real-time setting. At a cap of 5, we felt quality was getting too low for our service standards – although it did vary by the operator.
By looking at all four live chat performance benchmarks together, we were able to draw our conclusion. Four or five chats was the ideal balance between productivity and quality, dependent on the operator.
However, we wanted to cement our findings. So we carried out a more focused study on another client, this time a rapidly growing online fashion retailer.
The results were very different.
At a web chat cap of 3, we saw relatively stable contacts-per-hour numbers, and operators and customers alike seemed happy. This was just like we saw in our first study; but when we shifted that cap to 4, the operation faced almost immediate strain. Operators quickly became overwhelmed, and productivity dropped.
We reverted the change as soon as the client agreed, and took a long hard look at what had happened.
Whilst we don’t have enough data to share anything statistically significant in that short period, we can say with great confidence that 4 concurrent chats were too much for the operators to comfortably juggle.
We theorised that complex handling processes had been a large part of the problem.
With that theory in mind, we turned to the advanced workflow possibilities provided by our sister company and software provider, Gnatta. We wanted to give operators interactions from live and non-live channels, and allow them to more effectively manage their time.
Using workflows, we assigned operators a cap of 3 web chats, and an additional 2 social media interactions to handle in between chats. This time, we saw contacts-per-hour lift from 14.3 to an astonishing 16.2.
With these new results, we were able to draw a more educated conclusion. Chat capacity does of course influence productivity, but can’t outweigh complex handling processes.
Client A had relatively simple, if-this-then-that solutions to most problems. Client B, however, had longer processes involving the warehouse, suppliers and couriers. These interactions required more time and attention to resolve. But whilst waiting for answers to questions, operators were quite capable of progressing the occasional social interaction.
Where the ideal for Client A was 4 or 5 concurrent chats, Client B required a blended channel approach to achieve the ideal concurrency formula.
The most important lesson for us was that every operation is different. Processes and query types are major contributing factors in finding the ideal web chat concurrency. Optimising operational efficiency is crucial in achieving that ideal.
How do we know? We spent years working it out the hard way. If you’re struggling to manage customer service costs, check out our cost reduction calculator right here to see what we think you could save by addressing customer service inefficiencies.