AI Chatbots at Drift with Jeff Orkin

John Prial: It's been a long time since my social life has been normal. I mean, it is easy to become a bit of a hermit with all the restrictions, but if you can get out and about, there are opportunities to have a friendly chat. And I am making a number of new friends, and sometimes these friends are pretty helpful, too. It's just a matter of questions as I navigate around. Now, what am I talking about? Well, navigating around websites of course, and chatbots. But can they do more? Can they understand me better and get ahead of my questions? Seriously, the evolution of natural language processing and the exploding opportunity for positive customer interactions at scale means that chatbots are here to stay. And the efforts of some really innovative companies have shown that chatbots can really provide the type of help that many people need. Today I'm talking with Jeff Orkin, the head of the AI Lab at Drift. Drift delivers solutions for go- to- market teams that are focused on personalized customer experiences in real time. And in my conversation with Jeff, you'll see how. I'd like to also add the Drift partnered with the Marketing AI Institute to publish the 2021 State of Marketing AI Report. We're going to have a podcast on that to follow shortly. I think I see a theme. I'm John Prial, and I don't need a chatbot to be my friend yet. And welcome to Georgians Impact Podcast. Tell me just a bit about your gaming background and I'm really curious, because I did some reading about it. And what you learned in terms of the restaurant game that you produced. Tell me about that.

Jeff Orkin: Oh, right. Right. My PhD research project was the restaurant game where we paired people online anonymously in a first- person restaurant simulation. We had data from all of these players. And so we could learn how to react, not just in the expected situations, but when people break the rules.

John Prial: And in terms of gathering data, it's already been contextualized. You have the context. You're in a restaurant. And if you have your bell curve, most of the data in the middle are people ordering and commenting on their food. And yet at the same time, you also managed to find some funky edge cases. And it's important to train your models around edge cases. As I listened, this is about gathering data, as opposed to say, training the person, training a wait bot, for example. Unlike being Eliza where all you care about is what can Eliza say? This is what you're hearing. This is really gathering. You're right. Gathering data was step one. Once I had that data, the rest of my PhD was exploring, how do you learn from that data? So I was trying to learn models unsupervised of language and behavior. And I could get things that if you watched for a few seconds, the characters seem to be doing the right thing. But if you watch for a longer period of time, you would realize the characters were totally confused, repeating themselves, saying things out of context. And so I started to think about, how can we create tools that would let us just hire random people on the internet to do a little work, explaining what was happening in conversation transcripts? Not just conversation, they were mixes of action and words. And use that to build a model that could power, ultimately, a waitress bot, who you could interact with either... It was a 3D environment, but you could speak to her with either type text or a microphone. So these crowdsource people are basically labeling the data.

Jeff Orkin: Right.

John Prial: Did you give them any context for it? I know it's supervised learning, so to speak, but it was unsupervised labeling where they... Did they have any context or you told them to just have at it?

Jeff Orkin: They didn't have a lot of context. They had never seen the game, but they could see an entire transcript. So they could tell basically what was happening. The transcripts were pretty easily readable, describing the actions the characters took and all the text of what they said to each other. And so we developed tools that would let us hire people anywhere in the world to break up these conversations into what we call events, which are sequences of utterances that could overlap in time. And they could be a mix of utterances and actions. All of this eventually led to my startup GiantOtter. GiantOtter was commercializing this same approach. And we rewrote the technology three times before arriving at an implementation that was production ready. And then we started exploring a pretty wide variety of applications for it, ranging from social emotional learning, to applications for businesses to assess job candidates, and then we started getting some requests to use it for training people for sales and support. And by this time Alexa had been introduced and everyone was interested in voice. We added voice and we started creating these applications where you could practice a sales call. Like if you were a new salesperson, you could practice overcoming objections with a virtual customer that you're trying to sell to.

John Prial: The key, as I listened to this. So you amass a corpus of data, whether it's through a restaurant game... I really like labeling as a generic term. You extract events and actions that can then get mapped. So you're drilling down further and further into interactions, which really is a perfect setup because the reason why I really wanted to chat with you is we're going to talk about chatbots. So what's interesting is I think about the chatbots I don't like on my banking app or whatever. And what they really are they attempt to be a crappy Q& A. What do you want? And here's an answer. And they don't do a good job of parsing the question and we're sort of stuck already. You brought forward, I think I see now it's historically tied even to this restaurant game, that this is a long running, I'll use the word transaction. This is a long running conversation. So you're building events and actions. So talk to me about the goods and bads of chatbots, and feel free to take a swipe at my bank or whatever. But I'd love to hear what you see makes it effective and ineffective.

Jeff Orkin: Yeah, that's a good question. So you're right. The work we were doing at GiantOtter, that's now transitioned into Drift AI is really focused on understanding entire conversations, and using that context to understand what someone says. Even if what they say might be vague or grammatically incorrect, we can make sense of it if we know when they said it in a conversation. And that's because we're always recognizing events. Everything someone says either starts or continues an event. This is different than the other bots on the market. The bots out there right now kind of fall into two buckets, either there's decision trees where you just by hand, make a tree of questions and answers that branch. And these tend to be very brittle. Users don't want to stick to your tree. And as soon as they try to diverge, the bot kind of falls apart. And then there's, the big tech companies all have their intent- based bots like Siri, Alexa, dialogue flow. These are all intent- based architectures where they do use machine learning to classify your input. But they're very passive. They wait for you to ask a question and then they say," Oh, you're trying to order a pizza. Okay." And then that activates a skill and then you're locked into that skill, and the skill either at that point walks through a tree, or it has a set of slots that need to be filled. Like, what size is your pizza? What are the toppings?

John Prial: In that case? It's very much a short- term transaction. Once you order your pizza, they're done with you. The skill is over. That's back to zero. The decision tree in theory could work better. It just doesn't

Jeff Orkin: Yeah. You'll never have enough branches in your tree to cover what real human conversation looks like.

John Prial: So you look at this very differently. So I think it's a good opportunity. I liked the way you described kind of what works and doesn't work with chatbots. So let's talk a little bit about Drift now. I want to later get into the piece parts of sales versus marketing, but I really want to kind of get it how'd you get started. I know how the restaurant game got started. Here you are now, you've got sales automation, marketing automation solutions. Let's get into data sources. How did all that begin?

Jeff Orkin: That was one of the great things about coming to Drift, is that they've got this goldmine of chat data. Drift's original product is a chatbot for booking meetings with sales reps. It's a bot that qualifies leads and either connects you in real time to a rep or lets you book a meeting on a calendar. And they've been doing this for maybe five years and have amassed a dataset of like 3 billion of these sales conversations. They could be sales or marketing focused conversations. And some support as well, but sales and marketing is really the focus. Prior to Drift at GiantOtter, we were this company building data- driven bots, which was kind of a new twist on bots. But we were a tiny company. At our biggest we were seven people. And so it was a real challenge to convince any... We were going after big enterprise companies, financial companies and things, or even healthcare companies and saying, We'd talk to them about our new approach and they'd be like," Yeah, this sounds really different." We'd showed them a demo. They'd be impressed. And we'd say," All right, let's get started. Just send over your data." And they'd say," Oh, we can't do that. Forget it." So at Drift, it was kind of this perfect marriage where we had this interesting technology that benefits from data and they just have so much data. And the way natural language processing has progressed in recent years, that data is even more valuable, because the approaches people are taking are based on these new models called transformer models. They're neural networks that are language models that have been open- sourced by big tech companies like Google or organizations like OpenAI. And so you can take one of these models, that's been trained on billions of documents and then fine tune it on your own data to make it uniquely good at understanding, in our case sales and marketing conversations.

John Prial: So you show up, your eyes are wide open, because this is exciting. So now you've gone to start bringing this logic of events and actions and annotations to this data. So this is probably the step one for you, is to now take this data, convert it into this different mindset. And I'd never heard the term and I really liked the term data- driven bots. So you began to work on the data, so you could create a data- driven bot. Is that right?

Jeff Orkin: Yeah. And actually at GiantOtter, we were trying to coin the term bottom- up bots. Instead of starting with a whiteboard and thinking top down, like what should my boss talk about? You just start with the real transcripts of how do real people converse? And start explaining what's happening there, and build a model out of that. So originally we brought our same tool set to Drift. Actually, to close the deal with Drift, the final piece of the decision the company was making about doing the acquisition was the tech due diligence. Where the CTO said," All right, you guys build bots from data. So why don't you come here? We'll give you our data and show us what kind of bot you can make in a week." And we had never made a bot quite that fast, but with a bunch of late nights, we got there Monday and by Thursday night, we had a bot that was working pretty well trained from their data. And so that became the demo for a while. Eventually that became a real product that first was launched on Drift's site. And then we got an early first customer and learned how to do it in a more repeatable streamlined way. When we got to Drift and we built a bot from their chat data, we did some analysis and we took, initially just 100 chat conversations, and did our data annotation to find the events in those conversations. And you can think of the events as being kind of the topics that people are hitting in these conversations. And what was interesting is that we graphed how each conversation flowed through these topics. And we found that in a hundred conversations, there were 88 unique paths through topics, because every conversation between a couple of people approaches maybe the same topics, but in a totally different order. Or they start a topic and then they diverged to something else and returned to it. And we often show people visually a graph of this and it's a mess. And it just gets across how complicated human conversation is and why there's just no way you can just build a little tree that's going to cover the way people are really going to converse.

John Prial: I love it. So let's talk about learning, then. You've got all this data, you've got kind of this mindset, this approach to looking at the data. What does the term then human in the loop mean to you in this case?

Jeff Orkin: So we use this human in the loop process where we're taking real data, organic data. And we get humans to explain what's happening. Is this chunk a question answer pair? Is this part of the qualification process, part of the flow? And kind of stitch this together. But humans benefit us in another way as well. Just having a human look at the data that's making it into an AI system. You see there have been some horror stories over the years, like recently, some from experiments with the GPT- 3 model, where a model trained on a lot of conversations from Reddit learns how to converse, and could say some pretty surprising and sometimes hurtful things. And years before that, Microsoft-

John Prial: Tay.

Jeff Orkin: Yeah, exactly. Tay. Boldly put out this bot that learned from the internet. And you can, you can imagine how that goes. So that's something important to us at Drift, is there's a human in the loop in part for that purpose. Everything that your bot says has been vetted by a human. It's never going to say something that it just learned and might've gotten out of context, or should not have learned.

John Prial: That is a critical selling point, that we will not be Tay. How did you begin to evolve how you defined events and actions and begin to really structure what the piece parts are of a conversation?

Jeff Orkin: What happened at Drift is that the tool set has really evolved. In fact, a team at Drift has totally rewritten the original GiantOtter tools, replaced them with something called BotUniversity. It's kind of a play on words, because it's a tool that teaches the bots. You know, the bots go to school to learn how to converse. BotUniversity is really nice in that it restructures the tasks around what we're trying to do, rather than focusing so much on just looking at the data itself. So at Drift, when you become a Drift AI customer, you're paired with a human employee at Drift called an AI Conversation Designer, who's an expert at using BotUniversity to build bots. And their work is kind of bucketed into maybe four buckets. On the one hand, they're adding questions and answers. So they're reviewing real human transcripts and they're finding out what people are asking frequently, and they're grabbing these fragments and bringing them into BotU to create question and answer pairs. And then we have a tool called the paraphrasing tool. Really, it focuses on the question. It digs into even more data, because there's only so much data a human can review, but then the machine can take the cue from the human, dive into much more data, and say here are 65 other ways we think might be the same way to say what you just said. And then a human reviews that to decide which to import.

John Prial: This is a Drift employee, this is not the customer. This is a Drift employee that's helping to build the bot.

Jeff Orkin: This is a Drift employee.

John Prial: This is not a coder. This is a no- code wold we've built for these, for these people, right?

Jeff Orkin: Yeah. I mean, the vision for GiantOtter, that's been more fully realized at Drift is to have a platform that allows non- experts to build complex bots. Bots that are more complex than what you typically see on platforms like Alexa. So yeah, these conversation designers, they have a wide variety of backgrounds. Sometimes they've been building bots on other platforms, but often they haven't. Sometimes they're writers, or people from theater, or video gaming backgrounds, or things like customer support, things where they're very customer focused. And yeah, we teach them to use this tool set to do things like organizing question and answer pairs, but also setting up different conversation flows for lead qualification, where there's often different criteria and protocols for qualifying a lead for each company that's a customer of Drift's. So that they can figure that. And what's interesting about our system is that we can combine these qualification flows with the question and answer, to seamlessly bounce back and forth. When we were talking about intent- based bots, they're very reactive. They wait for you to ask a question, they answer it. They might have a follow- up and then they're done, like you were saying. They're kind of a very short experience. Our bot is both proactive and reactive. So it's asking the user questions to try to qualify them and keep them engaged. But at any point, the user can say," I'm not really interested in answering your question. I've got my own questions." And they can interrupt and ask their own questions. The bot can answer that. And then depending on where that went, they may diverge into a totally new flow, or they might return to the qualification flow. But the idea is to put the buyer in the driver's seat and really let the buyer, or the prospect, take the lead in the conversation the way hopefully a real salesperson does.

John Prial: So you do use the term for Drift AI, conversational marketing solutions and conversational sales solutions. From my perspective, it's really just where they are on the funnel. Are they a prospect that's just kicking the tires or there's someone that's come back multiple times to look at your website. Versus maybe someone who's already using their product. How do you differentiate marketing vs. sales, and how do you differentiate who the person in the audience might be?

Jeff Orkin: I don't know if there's a big difference. When you first visit the website, conversational marketing is trying to learn about who the website visitor is. But then you get into conversational sales once you've asked enough questions to realize they're a qualified lead, and you want to connect them with the right person to talk to, to move them to the next stage. The way I think about it is marketing is where you're thinking about, across all of the people coming to our website, what are people asking a lot? And then sales is where you're like, all right, this specific person seems qualified. How can we move them along so that they have a good experience and can quickly make the purchase they're trying to make?

John Prial: They're just moving along the funnel. So I'd like to talk just a bit about the data. There is part of the Drift solution is showing a snapshot analytics about conversation. So what do you show?

Jeff Orkin: Right now we have a part of the product called conversation analysis that lets you do a search to see, how often have certain words been coming up over time? You can get back a list of the conversations that mention that word. What's I think more exciting is we're starting to adapt the powerful AI technology from the bot to be able to understand human- human conversations. So the bot often connects you to a human sales rep to continue chatting. Sometimes this continues over the phone. We're digging into that data and showing that our models, with using the events and the classifier, can understand what people are saying to each other. And that leads to being able to power semantic search, where maybe you want to find every conversation where people were talking about pricing. But off the top of your head, you don't know every word someone might've used to refer to pricing. But using semantic search, using the AI that can understand how different words relate to each other, we can find conversations that used other wordings like cost, or subscription, or enterprise plan, or whatever. And give kind of more satisfying results to find what you were really looking for when you were looking for the word price.

John Prial: Interesting. So help me understand a little bit of what you consider measures of success. There could be one measure that says, I talked to the bot for one nanosecond and I was passed over to a real sales rep or whatever the chat version of dwell time might be that actually had a productive dialogue. You shared lots of information with the customer. What are the different types of measures that exist in what matters?

Jeff Orkin: It depends a little bit on the customer, actually. For some customers, it's purely the amount of pipeline generated. How many leads did you generate? For others, it's more about being more selective with who they talk to. Any company only has so many salespeople, and if you can make the conversations they have higher quality, because they're talking to the right people that are really their ideal customer profile, that's valuable. The other thing we offer with our bot is deflection. Salespeople can waste a lot of time talking to people who actually just have a support question, or just want a free demo, or whatever. And our bots can carry on enough of a conversation to get to a point where they could understand what is your intention on this site? And based on that, they might point you toward a video or some resource on the web, rather than connecting you to a person. In some cases to do that even better, we can reach out to external APIs. One of our customers is a shipping company and they were having a problem where their salespeople were spending all their time answering questions about like," How much is it going to cost me to send this box from Boston to San Francisco?" And all of that information is available on their website, but people don't take the time to look for it. So we built an integration with their shipping price API, so that the bot can answer those questions, rather than the people. The people that actually get connected are the people that are like, we want to sign a contract for business shipping, where we're going to ship this much stuff every month.

John Prial: So I was thinking about, you said you came to Drift and you had this treasure trove of data that was there for you, which made it easier. If someone's thinking about implementing a chatbot, and let's just say, all they had was a series of emails to the help information. Help me with this, help me with that. And I was able to feed a thousand different emails of 10 years of history of support questions. And it's not really support, but Q& As. Is that a similar acceptable corpus of data that I could send to Drift that they could build some bots around emails?

Jeff Orkin: Yeah. We can start with whatever data a customer has. There's kind of a range of possibilities and we're happy to consume any natural language data, including transcribing phone calls or Zoom calls. It's certainly easiest when someone's already a Drift customer and we just take their whole backlog of old chat of Drift bots. But we've taken data from other chat companies that can be exported. We have done some things with some email. So really, as long as it's people talking to people about the right topics, it can work.

John Prial: Do you think we'll ever see a website that has no more words on it? And just as a bot on it saying" Hi, can I help you?" What do you think?

Jeff Orkin: I think the answer is no, because the website gives so much context for why you're there. And our bots leverage that. Depending on what webpage you start a conversation with the bot, we customize what the conversation looks like. Were you on the pricing page, or were you on a product feature page or a page for an event that's coming up?

John Prial: We started, we were talking about the restaurant game being context. That's context. crosstalk I love it. I think that's right. I'm not sure I'd be happy going to a website when I know what I want to do and I know where to go get it, to have somebody ask me the question. But having that bot available to me obviously can also make my life as an end- user that much better. So just to wrap it up, a question then. This is so cool. The people that are working, building these bots, you talked about theater people and everything. So talk to me about how you see chatbots affecting people's career choices. Obviously people want careers in tech. You don't need to necessarily have a tech degree to have a career in tech anymore. So what's your view of makeup of tech teams and bot teams and everything. I'd love to get your vision of careers for people.

Jeff Orkin: Yeah. I think the best teams are multidisciplinary. Not just multidisciplinary, but also people with cultural differences as well. I think it's great when you can make the process of building a bot less technical, because it's another interactive medium that we want more people to be able to succeed in. And then, the direction things are going in the world, COVID totally shook everyone's world up in ways that are not going to return to the way they were before. For one example, Drift announced that they've gone digital first, meaning we all work from home. We still have offices that they call conversation spaces, where we can meet for conferences and whatnot, or get a team together to work through something. But for the most part, we're all working from home. So bot building is yet another career that you can have from home. You don't have to be someone answering phones for a company or writing code. You can do it from home.

John Prial: So Jeff, thank you for that fantastic discussion. Good luck. I can't wait to see the next conversation I'll have with a Drift bot.

Speaker 4: Thank you. This was a great conversation.