- Designed the vision and roadmap for the product: Its use cases, behavior, core requirements
- Defined success metrics and reporting
- Set up a process to derive customer insights from large amounts of unstructured language data
- Worked through all technical design aspects of the problem by collaborating with team members from architecture, engineering, data science
- Worked through user interface design with eBay’s UX team
This project was part of our holistic approach to deliver a better eBay help platform. Our goal was to provide quicker, easier, and more delightful service by offering machine learning-backed tools to improve the experience for members at different stages of the help journey (discovery, self-service options, contact and routing, resolution). This routing product addressed key issues in the contact stage.
Users & The problem
User need: Get me to the right person to solve my problem, don’t make me talk to multiple agents
When users need to contact eBay, eBay has to know who is best skilled to help them. When users are routed to the wrong agent, the agent has to transfer them to a different department. This is a poor experience for the member because it takes more time and they often have to repeat their problem again to the new agent. This also usually happens when they’re already quite frustrated because something has gone wrong with their eBay purchase or sale.
Business need: Reduce transfers for less wasted agent handle time, better staffing efficiency
It’s expensive for the business to have agents fielding the wrong contact. The more that agents are devoting time to transferring contacts away, the more redundancy and lost opportunities to field more contacts. This phenomenon means that the business has to staff more people to cover the same amount of contact volume or else customer wait times increase which means a poor service experience.
Solving the routing problem
Routing is fundamentally a disambiguation problem. In order to get a contact to the right person eBay has to definitively know the nature of the problem. The old routing flow just had two layers of disambiguation to try to segment an issue, but there is so much complexity to eBay that this often wasn’t enough and transfer rates remained high.
I had already been working on the HelpBot in which many of the problems were the same. The premise of the HelpBot was to converse with users to figure out what sort of help they needed and then recommend a solution. So, using the same models and disambiguation questions, we applied this to the eBay Help Platform in the UK in the form of a natural language routing flow (pictured above). Users would type a phrase into a text box and, depending on the level of confidence in the results by the machine learning model, the application with either ask more follow-up questions to determine the problem with more certainty, or, if the model had a high confidence in the nature of the problem, route the user directly to a certain eBay staffgroup.
Impact and project status
- Transfer rate reduction
- Increase user satisfaction (tracked by automated surveys, periodic user interviews)
I can’t go into too much detail here, but we decided to pause the experiment in order to focus our resources on expanding the HelpBot and our machine learning-based search products to other countries.