How to access
- Hit the button below
- Sign into eBay
- Click “Chat with us” on the bottom of the page
Check it out
- I hacked out the first prototype of the HelpBot using a simple python application and the Facebook Messenger Send API. I then used this sample app to pitch the idea to executive leadership and secure investment to start the initiative
- Designed the vision and roadmap for the product: Its use cases, behavior, core requirements
- Defined success metrics and reporting
- Implemented an agile workflow
- Set up a process to derive customer insights from large amounts of unstructured language data (user transcripts with the bot)
- Worked through all technical design aspects of the problem by collaborating with team members from architecture, engineering, data science
- Worked through user interface design with eBay’s UX team
- Secured cross-team alignment on features, timelines, goals and deep integrations with eBay’s existing data systems to take actions on users’ behalf
- Worked with engineering and design to implement improvements in a regular and rapid fashion
- Collaborated with customer service agents to drive improvements back into the HelpBot
The renewed vision for eBay’s global customer experience organization, starting in late 2016, is to provide help to eBay members in the simplest, easiest and quickest way possible while making them feel great about eBay along the way.
This laid the foundation of our design direction for the bot, which was: get to the root of the problem and give an answer as quickly as possible. We also needed to ensure that, for anything more complicated, the user gets to the right eBay representative to take care of the issue.
Our Users & The Problem
- Users don’t want to hunt for a solution to their problem. They want to explain their issue in natural language and receive a recommendation for what to do in their specific case
- Users want reassurance that a certain solution is the right way to go
- Users don’t want to spend time to contact eBay if they don’t have to
When users arrived on eBay’s old help site, they confronted a search bar and series of “popular solutions” (FAQ-like links to help articles such as “How to return an item”). This layout worked well enough for experienced eBay users who knew which solutions applied to their issue, but not nearly as well for first-time or occasional eBay buyers and sellers who were forced to hunt for the help article or webform matching their problem. For example, it’s not obvious to many people that, in order to resolve an issue where an item arrives missing parts, eBay’s policy is that you have to open a return with the seller to start the process toward resolution.
Focus groups with these new and occasional users, along with survey data, revealed that many people were easily frustrated by having to hunt and peck for an answer. Users would rather take valuable time out of their day to contact eBay live help than to try to self-serve with mixed success. Of the users who did not want to contact, a portion spent significant time attempting to help themselves but left without a resolution (which engendered a bad feeling about eBay as a whole).
Business needs: Reduce as many simple contacts to eBay as possible to free up agent time to work on more complex problems for members
eBay’s customer insights team samples and classifies the types of contacts that eBay receives on a regular basis. Of these, it was apparent that eBay received an outsize number of calls and chats regarding how to open returns and what to do about unreceived purchases or sold items. These were usually instances where the member could have figured out what to do on the eBay site, but either couldn’t find help easily or they chose to contact anyway for reassurance.
Agent time is expensive. So, to save unnecessary spend, the business is always interested in reducing the number of contacts to eBay that concern basic issues when there are existing processes designed to handle them.
Why a bot?
The most natural way for two parties to disambiguate a common problem is to have a conversation. Applied to a digital surface like the eBay help platform, conversation manifests as a chat window (an experience that users are very familiar with at this point). Staffing the chat window with humans ran against the need of the business to reduce the number of contacts coming into the system and it also wouldn’t solve the customer pain point of having to spend a chunk of their day queuing up and waiting to speak with someone. A “conversational AI” was the best the solution that tackled all of the issues.
Which use cases? Where?
We chose to roll out the bot in the UK first as it is eBay’s largest market with widespread chat support. At the time of this writing, the bot is live in a variety of places, primarily the eBay consumer-to-consumer selling flow and the help platform, where it intermediates chat contacts. Users click “Chat with us” on the bottom of common help article pages and the bot is there to either assist the user or hand the contact to the right customer service agent. When the contact comes through to the agent, the bot passes any information it was able to gather so that the agent can pick up where it left off.
The data showed that a disproportionate number of the queries coming into eBay’s contact centers boiled down to a few main areas. For buyers: How to get a refund (specifically for a return or an unreceived item) and questions around next steps on existing cases or refund requests (“My case has been open for a few days now, what’s the latest? How do I get a resolution?”). For sellers: How to handle returns and refunds, how to manage disputes with buyers, and questions around next steps on existing cases or refund requests.
We built out many different use cases covering the areas above, so that users could take action or get an answer without having to contact. Below, I’ll highlight some of the major ones.
Quick answers to frequently asked questions
- Straightforward questions and answers about common tasks or needs, such as “How do I reset my password?”
Open Returns (buyers)
- One of our most-used features
- The HelpBot converses with the member to determine whether the problem necessitates a return and, if it does, guides them through submitting a claim
Open item not received request (buyers)
- Another of our high-traffic use cases
- Very similar to the return scenario where the bot facilitates the opening of a claim on an unreceived item to initiate a refund or to have the seller send a replacement item
- Below also shows the power of natural language entity extraction in action: We can filter the user’s purchase history by information they provide up front
Check on a case status (buyers)
- One of the highest-trafficked use cases in the HelpBot. This use case lets users get updates and take actions on open claims or cases (refund disputes, returns issues).
- We built deep integrations with the same decision engine that powers the agent desktop used in eBay’s contact centers so that the HelpBot can adjudicate disputes automatically. This is a very delightful experience for members because it often saves so much time. Many rave reviews here.
Impact so far, project status
Over time, we updated our bot design pattern to make the experience as frictionless as possible.
For example, letting users type in natural language plays a huge role in narrowing and understanding their help intent, but it’s not always convenient or effective, especially when inputs are confined to a few possible options (i.e. “Would you like a replacement of the item or a refund?”). Gradually, we reduced the amount that users had to type in key places in order to keep the conversation flowing quickly and efficiently. We rolled out more buttons, quick reply components, and quick start introduction cards into the user interface. Additionally, we baked in personalization features so users wouldn’t have to start a conversation from scratch. For instance, if we know a user has a claim open, it’s likely they want to see the status of their claim and whether or not they received a refund. So, the bot would open the dialog by showing an option to quickly check on a case status.
Aligning with the needs that we were trying to solve, our main metrics were:
- User satisfaction: the metric is determined by regular surveys, but supplemented with transcript review, and conventional user interviews.
- Contact deflection: we track how many people begin to initiate a chat contact and then choose not to after speaking with the HelpBot.
I can’t comment with the specifics around these numbers because it is internal information :/ What I can say is that we achieved enough success in our first market (UK) that we began expanding the bot to other countries to help intermediate chat support in the same way, along with bringing the bot to other channels (voice, social, SMS).