Practical Uses of AI in Customer Support with Craig Stoss

In this episode, Priscilla is joined by Craig Stoss, VP of Solutions at Kodif, to break down what AI actually looks like in a real support workflow and how teams can use it in practical, low-risk ways.

They dig into why AI is more than just chatbots, where it actually saves time (and where it doesn’t), and how tools like summarization, tone refinement, translations, and reporting can make day-to-day support work more efficient without sacrificing quality.

Craig also explains the biggest misconceptions about AI, including why it can’t make human judgments, what it’s genuinely bad at, and why measuring success with traditional CSAT can lead teams in the wrong direction.

Plus, they discuss:

• Common mistakes when implementing AI 
• How to think about a “frustration index” instead of satisfaction scores
• The skills support professionals should start building now

This conversation is for support teams who want to use AI without losing trust with their customers, and for anyone trying to figure out where AI fits into their workflow without overcomplicating things!

Where to find Craig:

• LinkedIn.com/in/cstoss
• Kodif.io

We want to hear from you! Share your support stories and questions with us at happytohelp@buzzsprout.com!
 
To learn more about Buzzsprout visit Buzzsprout.com. 

Thanks for listening!

Transcript

Welcome to Happy to Help

Priscilla 0:00

Welcome to Happy to Help, a podcast about customer support from the people of Buzzsprout. I'm your host, Priscilla Brooke. Today we're getting practical about AI and customer support. We'll talk about common misconceptions about AI, actionable tips for incorporating AI into your daily workflow, and what you need to know to get the most out of your AI strategies. Thanks for joining us. Let's get into it. All right, so today is one of those episodes where I'm really excited. I know that at the end of this episode, everyone listening is going to have some really good tangible things to take and put into their workflow tomorrow. And so those kind of episodes always get me really excited. I feel like we've all listened to podcasts where they give you these big overarching ideas and concepts to think about, which can be super helpful. But sometimes you just want someone to tell you what to do and how to handle something. And so I'm really excited that we're going to talk about those practical uses of AI, especially for the people listening to this who don't use AI much or are kind of scared of it or have a smaller team and don't, you know, have the funds or the budget to buy a big AI platform, like how they can use AI strategies in their day-to-day. So joining us today to talk about AI is Craig Stoss. Craig is the VP of Solutions at Kodif, which is an AI-driven customer experience and support automation platform for e-commerce customers. He has over 20 years of experience in customer-facing roles and is passionate about the impact that customer experience can have on your business. Today, he's going to join us and bring us his insights into AI and give us those practical tips about using AI and customer support. So thanks for being here, Craig.

Craig 1:47

Thank you for having me. I'm so I'm so excited for this conversation. I think it's so relevant right now to so many different types of businesses.

Who Made Your Day Recently

Priscilla 1:53

Oh, yeah, for sure. And I feel like it's one of those conversations that gets a lot of people, whether they're super excited about AI or they're super hesitant. Like it's just there's a lot of feelings and emotions that come along with AI. And so it's always a good conversation to have to like break it down and look at it in a way that can kind of take that emotion out of it. We always kick off our episodes with the same question. So, Craig, who has made your day recently?

Craig 2:19

So my son and I went to get our haircut. And the barber we have has a kind of electronic waiting board where you check in, you put you put your your name in. But the way it works is when there's multiple people in one group, you're supposed to select them all at once and hit go. So you're kind of grouped together. And I just forgot. And then, of course, before I could get my son checked in, another person had checked in. So now there was this big gap between me and my seven-year-old son. And I thought, oh great. I'm gonna, you know, get my he's gonna get his care haircut, and then I'm going to have to wait for 20 minutes and again get my hair cut. And he's gonna be sitting in the, you know, getting bored. And it was already about a 30, 40 minute wait to get uh seen, anyways. Wow, yeah, it was a very busy Sunday at that the barber shop. And I was I was really regretting this because when my son gets bored, it's chaos. And oh, and just to end the boot, they had a TV in the lounge that the uh the TV was not working that particular day on Sunday. So that even made it worse. He couldn't even be distracted by the TV.

Jordan 3:18

That's unbearable. Yeah.

Craig 3:20

Yeah, yeah. So I chatted with the woman who was doing my hair and she was very nice. And and without a without me even asking, she accommodated and said, listen, we'll just reassign you. And I thought, you know what? That's a recognition of the the situation that that you're in, right? And and the other person didn't even notice, right? You know, because my son's hair takes two minutes to cut because it's he's small. So that made my day in a sense that it was a really small thing that, you know, bending the rules a bit, but uh, but it really it really helped probably everyone in the situation so that my son didn't have to run around the the barbershop going crazy for 20 minutes.

Priscilla 3:57

Yeah, it had it benefits and impacts that people didn't even realize, right?

Craig 4:01

Exactly.

Craig's Background in Support & AI

Priscilla 4:02

It's funny, I was talking about a similar situation just this morning. We were talking about refunds, and I was thinking about how sometimes working in a situation in customer support, and it's clear that a refund is the right thing to do, whether they've asked for it or not. In the situation I was talking about this morning, someone had upgraded to a paid plan, but then immediately decided that it wasn't gonna work for them and this downgrade. But they didn't ask for a refund and the specialist offered one anyway. It was one of those things where the impact of the offer, because it hadn't been requested, was even more than it would have been had the person asked for it. So if you had said, hey, can you switch this around for us, they probably would have done it and it would have been fine. But the impact of the person noticing that you needed that without you having to ask for it, that's such a bigger impact. And so I think it's one of those, like, as a customer support specialist, being able to identify when the right thing to do something, even if it's not requested, can be a cool way to handle a situation. And it shows like you care about the situation more than just reacting to the request. I think that's a great story. And shout out to them for making that easier for everyone involved. Before we kind of get into AI, can you tell us ju"st a little bit about how you got into customer support, but then also a little bit about the work you're doing now, kind of at that intersection of AI and customer support?

Craig 5:26

You know, I grew up in the 80s and and you know, computers and things were just not really a thing. The internet hadn't been invented yet. And so what I finally did get to touch my first computer, you know, it at the time it wasn't seen as it is today, where it's like, you know, the internet and games and flashiness, right? It was it was a tool. It was, it was, you know, just a fancy calculator. And so, you know, I spent the first year coding and being so fascinated that I could talk to an inanimate object and have it do things that I asked it to do. And as I kind of developed and developed and developed that kind of skill, and I got into um, you know, there was a program in Canada and rural communities to bring when the internet didn't go mainstream in the 90s, there was a program in in rural communities Canada where they would hire students to teach senior citizens about what the internet was and how to get an email address and how to get government forums. And and I was selected to run this kind of program. And that was right there that I really saw this intersection between technology and the human parts. And and from there I really did back down. I did take a computer science degree. Uh, it was mainly in like you know, the full life cycle of software development. It wasn't just development. I learned how to write requirements documents, I learned how to QA, uh, I learned how to present ideas and gather information from a customer. All my internships during that, the during my university years were in support departments or at least in customer-facing departments. And I just really felt like that was where I wanted to be. I didn't want to develop full-time, I wanted to kind of be that intersection between technology and people. And over the years, I did I've done lots of different roles. I was consultant for Fortune 500 companies for software development best practices. I got into support leadership later after that. Most recently running, uh, I was at a BPO and then uh running some of their customer programs uh for their teams. And then now in the AI world. And the thing that's always fascinated me about software in general is support departments have all have this kind of like cost center mentality. And so software and software solutions for support have always been this kind of commodity in some ways, right? It was one of these things that there's tons of cool solutions, there's tons of systems. 10 years ago, when I first got into leadership, I was pitched on tagging solutions and categorization solutions and reporting solutions and chatbots, early versions of chatbots. Yeah. That stuff was just stuff that you would have a hard time affording. Yeah. And what I where I think AI is different, and what's really cool about the AI industry and where we're going is that these things are now more affordable. The ROI, given the scale and scope of most AI solutions, is much more clearer than say a legacy knowledge management system. You know, it it's very easy to see that. And so, yeah, that's how I got into it. And and now that's how it intersects into my current role, where I'm still in that technology and I'm still, you know, as a solutions person, I'm still talking to customers every day about how they can use technology. And that's ultimately my passion is about making these experiences high quality for them.

Priscilla 8:34

Yeah, I love that. Can you tell us a little bit about Kodif specifically and what Kodif does and what it offers for its customers?

Craig 8:40

Yeah, I've known of CODA for about five years. I met the founders at a conference and we got talking about the future of AI, and it was just on that cusp of Chat GPT and open AI solutions coming out.

Priscilla 8:53

Yeah.

Craig 8:53

And one of the things that I've always believed in, and this has nothing to do with AI at all, is that support needs to be personalized and contextual, always, right? And CODA, when I was talking to the founders, was one of the first AI solution companies I talked to, and I've talked to dozens at this point, that really was how it was built from the ground up was everything they did, all their prompts, all their back-end work was about making support contextual and personalized. So if you say something like, My grandfather lives in Germany and I want to ship them your product, it's not just gonna say, Oh, I'm really sorry we don't shift to Germany. It's gonna say something like, Oh, what a great idea that you want to ship something to your grandfather. I hope he has a wonderful birthday, but I'm really sorry we don't shift to Germany. Wow. You know, that's a much more human-sounding message. It takes in the context your question, it personalizes it, those types of things. And so every feature we build has like really strong abilities to categorize, to understand, to empathize. Like we have a lot of pet food companies, and sometimes when you're a pet food subscription company, people have to cancel their subscription because a pet has died.

Priscilla 9:59

Right.

Craig 10:00

And you want to opt that empathy and say, Oh, I'm really like that's not the time where you go in and be like, oh, well, we'll offer you 25% discount if we if you were staying with us. It's like, yeah, no, the person's just lost a family member, right? So but that's what really drew me to Kodif initially, and and that's something that goes into every feature we we produce.

Initial AI Skepticism

Priscilla 10:18

Wow. Obviously, you're about to get into all of this as far as you know, how to use AI in a way that is personal, is human, and includes that empathy and stuff. So let's jump into kind of like the early days of AI and customer support. I remember when AI first was kind of coming on that it was really scary, right? And it was like, okay, is this just gonna like take away all customer support roles? And I had just really started to solidify my existence in customer support and just loving it as work. And then, oh no, AI is coming in. Is this just gonna take it all out? We've been working so hard to build this up. And now are we gonna just give it over to robots? So I'm curious, you know, looking at how it's evolved over the last three to four years, when you started working with customer support teams, Craig, to implement these AI strategies, what were some of the early reactions you were hearing from customers and how has that kind of changed over the last couple of years?

Craig 11:19

First and foremost, everyone's associated AI with chatbot. Those two terms were synonymous in many people's heads. And I think the first thing we always had to dispel was that's not true. I think that the most common interface for a AI tends to be a chatbot, but that's not, they're not the same. And the GPTs and the claws and the uh and the Geminis of the world are not anything related to the historic chatbots that we're used to. And and so that was the very first thing was kind of getting into this mode of why is this different or why is this better? Because initial chatbots required you to train it on kind of questions and then answers. So here's a list of questions, here's a list of answers, right? Here's how they pair together. And for the most part, there was no kind of context of synonyms, there was no context of of different ways of asking questions. It was it was really, you know, match this keyword and then this thing might be this question, and here's some answers. And and you you remember this from like early Zendesk AI days, where like it would send you a response, and it was like these three articles might help you. And they were sometimes just way off base. They just happen to mention the keyword you used, right? And that's not a shot at Zen Desk, right? And I mean, that's the technology that we had. Yeah. But when when you talk to customers and that's what they're associating with AI, you have this upward battle of saying, no, that's not what we're talking about. We're talking about, you know, large language models, we're talking about contextual language models that understand synonyms, they understand uh language, they understand colloquialisms, they understand these things. And I use the word understand a bit loosely. I think there's there's probably someone screaming at their phone right now, they don't understand anything. And that's true. Understand is a loose term here, but they are trained on this and they they under they understand the connections, I guess is the better way of phrasing it.

Priscilla 13:12

They can navigate it, yeah.

AI Misunderstandings

Craig 13:13

Yeah. So that's the first of the early reactions. I think how this has evolved is now that in user-focused industries, people have seen a little bit about how AI is improved. They see these little personalizations, they see that the answers are not guesses anymore. They're they're fairly accurate. And so there's there's a little bit more of that acceptance. And when it's done well, and there, you know, being able to say something like, I would like to speak to a human is a seamless process to get to a person or get to a human for a certain type of ticket, or do so without you even, you know, going back to the hairdresser, doing so without even asking, just recognizing, oh, this person's frustrated. Let's get them to a human. Right. That technology enables that. And I think that that people are seeing that this is now a benefit uh overall. So there's definitely been acceptance very recently.

Priscilla 14:02

Do you see any current misunderstandings about AI? Is there like this far into it? Are there things that you see kind of trends where people misunderstand things about AI and how it comes into customer support?

Craig 14:15

Yeah, there are two big things that I would I would argue uh I encounter on a regular basis. The first is AI cannot make judgment. I think that's a really important distinction. So, what we just talked about with rules, AI has to have the rules in place. It can make judgments to a point where you can teach it how to make a judgment, but at that point, it's not so much a judgment as a different type of rule, right? And and so you know, I hear complaints all the time. I was actually, we're going on a trip with some other couples in a in a couple weeks, and we were talking about because we all have children, and they were saying, well, every airline seemed to have a different algorithm to assign seats with adults next to children, and which is true. It's interesting because I would argue that a lot of that's done using some version of AI. There's some algorithm in there. It may not be traditional LLM AI as we're talking about, but there's some algorithm in there. And it's like, well, they're clearly just not putting, they're either purposely not putting the rules in correctly to make you buy a seat and force you to, or the rules are wrong, right? Because that's something I think you can define. So going back to the judgment piece, so if I say to it, here's how to judge if someone deserves a refund, and then say things like, you know, if the order is greater than 30 days old, they cannot get a refund. Well, that that's a hard rule. Cannot get a refund. Right. Let's use the dead pet example, right? Like if the customer expressed their pet passed away, well, maybe we'll make an exception. You can you're allowed to make an exception for that or something, but you have to teach it how to make that judgment. You can't make a human-based empathetic judgment. And that's a big, big one that customers ask for all the time, especially when it comes to things like warranties, where it's like, well, you know, you have to make a judgment. Was this a manufacturer's defect or was this a uh was this human error? I mean, I don't think humans do a great job at that, let alone trying to teach an AI how to do something like that. So that's the first one. And then the second one is that there are certain things, and this will change, and it probably will change by the time you release this episode, but there are certain things that AI is just not good at. I saw a LinkedIn post where someone asked an AI to draw a map of Europe. Uh and by AI I mean a large language model, and it did a horrible job. It lay it was two different Frances, uh, it labeled Berlin instead of Germany. Like it was just, it was awful. It was a horrible map, right? And he was like, checkmate AI people, you know, can't even draw a map of Europe. And I was like, but that's you know, that phrase, I think it was an uh Einstein phrase where he said, if you judge a fish by its ability to climb a tree, it's gonna fail. That's that's what this is doing, right? You're judging an AI based on something it wasn't intended to do. And so when we get customers or when I have conversations with people about AI, that's something that I always am very specific of. This is a large language model, and like the key is in the term language, there, right? It it's very good at guessing the next word based on the previous 10 words. It's not good at certain other things and it will improve. And you can see some of the problems with the improved AI uh image generation. You know, there's other problems because of that. But I think people need to understand what it cannot do because it's it's not magic.

Most Beneficial Uses of AI in Support

Priscilla 17:29

Yeah. Where do you see AI saving the most time for support teams? And do you have any examples of tasks that you've seen where AI just does a better job than a human and can make things more efficient for the human to work on other things?

Craig 17:45

So specific to support, right? I think the AI solutions that are in the market, you know, have two at a high level functions, right? FAQ, question and answer stuff that's completely repeatable. Everybody gets the same answer. AI is great at that and and can basically abolish that kind of level of question. Yeah. That is sometimes high ROI. Usually that's more medium, low ROI, because those questions already were quick. You had a macro. I think the one advantage AI has is that it's not macro based, right? So it's more generative and can sound a bit more human than a clearly pre-written solution. The second one is the automation of activities. And this is where you can integrate in your tools and automate things that would take a human agent five, 10 minutes, copying and pasting, using APIs, using integration. The AI can call out these things, recognize when they need to call out these things, and and even checking things like formatting, you wouldn't believe how every different system uses a different date time format, uses a different address format. You know, if you're international, you have you have dozens of different phone formats that you would have to accommodate. And then things like language, most AI solutions are multilingual out of the box. So instead of hiring one person that sits uh you know, sits in Italy and can speak Italian, Spanish, French, right? You can now an AI agent that does most of that for you, translates tickets to English so they can be reviewed by your Englishy if that's what you need to do. So that's kind of the first thing that I would say to look at when it comes to to support specifically. But to benefit the team as a whole, right, that's where you know you start needing to recognize what AI is good at, right? And AI is great at pattern recognition, understanding anomalous data. And this is where it's really cool for things like voice of customer, or very cool for like CSAT interactions or effort interactions, right? Like the customer effort score stuff, where you could take a huge data set, a month's worth of tickets, a quarter's worth of chat, you can correlate that to you know your customer success export tool, your is your customer in red, yellow, green status. You can probably nowadays where we have all of our record uh calls are recorded, you could probably inject a bunch of that data. You get all this data into a big bucket and say, for every customer that's in a red status, tell me the top 10 things that they're concerned about. Yeah, you know, and and AHI is really good at that. Like, really, really, really good at that. That's something I think support can use to heighten their visibility within the company, right? By able to say, like, go to the product team, go to the service team, go to the onboarding team, go to whatever, and say, Hey, I have hard evidence that this is the the case, right? And so those are those are really kind of the places I always tell people to start. But I mean, there's tons of these things, right? I I don't reformat anything anymore. Like if I have if I have data in format X and I want it to format Y, I just upload it to an AI and say, here's an example of the format, the old format, here's an example of the new format, go. And you know, a minute later, I don't have to worry about, you know, changing things from CSV to JSON or or whatever it might be, like whatever you have to do within your support department. Right. There's tons of these little things.

Daily AI Strategies

Priscilla 20:54

Yeah. Well, and I think you hit on one that for me feels really huge, which is the reporting aspect of it and the help that that can bring. I mean, there's so much to learn in your support inbox, right? And when you're working with customers, there's so much data there and insight there. And when you have a support team that's struggling to get through tickets, they aren't able to do all the analyzing. And so having an AI solution that can come in and help do that analyzing and turn that into actionable results where you can go and take that to the marketing team or the developers and say, look, everyone's confused about the same thing. We've got to do something to change this. That can be a really huge, really beneficial way to use AI, especially if you are a little bit more anxious about having the AI interact directly with your customers. Like that's a way to kind of like build your trust with AI abilities, is in that kind of like non customer facing way. Also you mentioned translations, which is a huge one in the way that we use AI at Buzzsprout, is that we have people all over the world that are reaching out and we don't have anyone right now who is multilingual on our team. And so we use the translation solutions a lot with AI. And it's a huge help to us to be able to communicate with people on their terms in their language without, you know, forcing them to conform to ours just because we're speaking English. So I think that's another really good tool, a really good way to use AI. What about in the regular day-to-day, you're working in the inbox? What little ways do you use AI? Maybe not even directly with customers, but just like to make your life easier.

Craig 22:43

I mean, anywhere where you want to sound less robotic, like instead of using a macro, AI is great at that. So you might have a macro, but you could run that macro. So run the question, you know, in in the email and the macro and say, customize this macro to answer the question in a better way, for example. I think that summarizing things is really good.

Priscilla 23:06

Yes. Right.

Craig 23:07

I feel like when you're doing a handoff of a ticket, like, oh, I have to go on vacation, so I'm gonna hand this ticket off to someone that's been going on for a while, maybe. AI can summarize and say, like, here's here's what set seems to be outstanding. Here's the types of things that we're already trying, so you're not repeating yourself. The tone, you know, like some people struggle with tone. And and I mean, that's a that's a skill, right? To to add empathy and so run it through and say, Hey, like, I really need this to sound empathetic. We made a mistake. I don't want to place blame, I don't want to assume fault, but at the same time, this message has to be clear. And this is the type of stuff that's AI is really good at. Yeah. You know, you could write small little scripts. I I have a few uh that I use for troubleshooting. This is a great example, actually. So we we have some logs that store the entire interactions between our customers and our AI. And some of the details are not exposed to the user because they're usually too technical, or there's some backing stuff that of course you don't want to expose to any any end user. Right. But we need access to it because if the customer can't figure it out, we have to do some troubleshooting from our supports perspective. The problem is that to access it, you have to, you know, take some piece of information, search a database to get that information, get the export, then look through every single line of the database to find where the error occurred. And that was a time consuming process. I wrote a small tool. And by wrote, I mean I vibe coded using, you know, using cursor, because I'm not strong at Python at all, but I wrote a small script that I would give it uh the identifier, whether that be a keyword, what we call a conversation ID, which is a unique identifier for every conversation. It goes to the database, gets everything related to that conversation. And then not only does it give you the export to a Google Sheet of all the raw data so you can review it. I wrote a prompt that would take all that data and spit out a report on that data. So, what did the AI find? You know, talk about anomalies and pattern recognition. I would say, like, did you see any failures in the integrations? You know, that's a big cause of problems, right? Um, did you see any repetition? Did you see all these things, like a list of 10 things to identify? And it spits out a report. And now you could start the troubleshooting instantly. It takes about uh 30 seconds to run all that through the script, and you have a report that in theory you could just send to the customer and say, Well, here's the root cause of the problem. That is super powerful, right? If you're dealing with technical support. And AI is awesome at that.

Priscilla 25:33

Yeah.

Craig 25:33

Because it it's all pattern recognition, and that's what AIs are good at.

Priscilla 25:37

Yeah. Another area that we use AI is help us with research. There's a lot of support that we get questions about adjacent software or platforms that people are using, where typically you would expect the customer support team to say, hey, sorry, we actually don't do support for X over there. We do support for Bud Sprout. And so we can help you with this, but I'm not an expert on X. And I'm just saying X as a random example, just realized that X is also a platform. So didn't do that on purpose. Um, but it's it's one of those things where you kind of feel like, oh, I'm kind of trapped here because I don't know how to answer the question about this other platform. And that's what they're asking me about. And a way that you can use AI to level up that response is to do a little bit of research through AI and say, okay, here's the question they're asking. Can you lead me to the right help article or can you explain something to me? And then it allows you as the support specialist to get a higher level of knowledge and to deliver that to your customer, which is so big when you're not working for that company. So you're now giving them a complete experience with the help of AI.

Craig 26:51

I love that. In fact, I actually just did that for one of our customers. They had a CSS problem embedding our widget on a specific platform that they wanted to embed it on. And I was like, I've never seen this platform before. I don't know anything about it. And I did exactly that. Yeah. Exactly that. It just went and said, okay, well, I can't answer the question, but I can tell you this is where we need to look somewhere in this area. And right. Yeah. Great example.

Priscilla 27:12

Yeah. And kind of on that same lane is brainstorming, which is something that I really love. I think it's probably the way I use AI the most in my day to day is to brainstorm things. I'm a very verbal processor. So what I would love to do is sit down with another person in a room and talk through something. That's not always going to be an efficient way to solve a problem. And so being able to sit with AI and throw out ideas and get feedback and kind of brainstorm things. What am I missing and help fill in those gaps? I think that's another way to use AI, especially when you're in the support inbox and you're troubleshooting an issue. Say, okay, I'm here where we are, here's where we've tried. What am I missing? Like, what am I not seeing? Those kinds of things can be a really helpful way to just use AI in your regular, you know, daily process.

Craig 27:59

I completely agree. If I start so many of my AI prompts with, I'll have my headset in and I'll be walking somewhere and I'll be like, I'm in a rant for a while, and then you're gonna summarize what I say into something clean and you know, pull out what you feel the most coherent thoughts are and most relevant. When I'm alone in the car, I'll I'll use to write emails. I'll just plug it in and say, like, I need to write three emails. One is about this topic, and here's what I want to generally say. Next, here's, you know, and I often don't send them verbatim because I find that even tone of voice still is a bit off, even if you've tried to train it. Yeah. But at least it gives me the framework and I can just copy paste and then change the few words, which I still think is is faster. And I can be productive, you know, in in the car sometimes. But you know, formatting documents, you know, like a root cause analysis is sometimes a big one in support, depending on your sport team, right? You might have to deliver a root cause analysis doc. Don't make up a format. Don't try to figure out the best way to highlight things or what's the what should go in the executive summary. Upload all the data about the problem, whatever it was that you're root cause analyzing, and say, build me a root cause analysis report, and it will spit you out something that is in a format. Again, you're not relying on knowledge there. You're relying on the fact that it's betrayed on thousands and millions of the root cause analysis reports that it can give you the formatting that makes sense for your use case. And yeah, all of those use cases are ways that you can just save minutes or hours. And you know, going back to what you said, people are you know fearful of AI and someone's job is not going to disappear because you are reusing the best practices on how to build a report, right?

Common AI Implementation Mistakes

Priscilla 29:35

Right. Well, and I think that that's why conversations like this are so important because the people who are fearful of losing their jobs, the people who are worried about the stereotype of, okay, this AI chatbot is going to replace me, and that is the extent of AI and customer support. Hearing these conversations where we're breaking it down in a way that's like, hey, actually, this is something beneficial to you. It's important to hear these kind of conversations where it takes it out of this like scary unknown to, oh, okay, you're talking about using AI as a way to brainstorm, or you're talking about using AI as a way to help you analyze your tickets. Like those kind of things, I think, really help and take away that fear factor of this chatbot is gonna do my job better than I can, and now I'm no longer gonna be necessary. Okay. So when I think of my earliest conversations about AI here at Buzzsprout, one of the things that I was worried about was that we had built up for the last couple of decades this relationship with our customers and this expectation where they knew that if they emailed us, they were gonna get a response from a human really quickly. And so I think it's really important for us to talk about some of the mistakes that people make when they implement AI because there is a lot of risk there because you are working with your customers who are, at the end of the day, the most important part of your business. So, you know, what have you seen kind of as examples of what not to do when people are implementing AI in maybe not the best way?

Craig 31:06

Sometimes the tendency is to go back to uh decision tree-based logic. That's the first mistake that I try to talk to our customers about is the benefit of AI is you don't have to do decision trees anymore. For example, if you're a clothing company and you want to, you know, help people pick the right size of the clothing you sell, there is a set of things that you need to know about the person's purchase. If they're a man or a woman or a child. If it's for a shirt for a man, you probably need chest size. If it's for a uh blouse for a woman, you might need uh bust, waist, and hips. You have a set of things you need to know. And the historic way of solving that was to have a decision tree. Are you a man or a woman? Man, oh okay. Are you buying pants, shirt, you know, pants? AI, you could just teach it that. Here is the things you need to know. Here's where that information is. Whatever you don't know, figure out with the customer or infer based on the customer's profile that you have access to or the context that you have access to, and then only ask the questions that require. Don't repeat them, stone have them repeat themselves. You have context of who that person is, put all that together and only ask the questions that you might not have an answer to. Using macros is a really interesting one. We have customers who have this beautiful generative model that can generate really human-sounding, personalized messages, and they want to reuse the macro text from their help desk and just set use our AI to send the same text as the the help center would have. And the last one, and probably one of the biggest mistakes I see is people, and this is a this is one that gets to me more of a personal level, and this is maybe my hot take for the the podcast. I love it. CSAT for AI is different than CSAT for a human. And so many people are so locked in, dude. We have to have CSAT. We have to have CSAT. And they forget that CSAT is is not really what you need to measure in AI. Data shows, you know, if you if you were to look at all of our customers and export the CSAT data, it's always slightly below average. And that's what you would expect for a neutral, like people use AI differently and report on it differently than they would with a human. Because if I'm asking, do you ship to Germany? And the answer is no, that's an A and a pretty neutral question. And if the answer, it's either gonna be, yeah, oh, you know, I'm sad because, you know, it's negative because you said no, or it's positive because you said yes. Like there's no, there's no nuance to CSAT like there would be with a human, where it's like, oh, their tone was they just treated me poorly, and you know, I'm gonna give them a lower score, right? What you really want to measure with AI is the things that frustrate people about AI. Did they go into a loop? Did they ask the customer to repeat themselves? Did the AI misunderstand in the customer and say, no, that's not what I meant? Right. Those are things you should be measuring. I call that frustration index, and that's something that our product measures because CSAT to me is not as relevant with AI as people think it is, because it's just a totally different concept when it comes to how people aren't frustrated by humans differently than they are by bots because they treat bots differently than they treat humans. Like it's it's not an apples to apples comparison. So that's one that I always looked at. And coincidentally to your question, is that everything that I put in that list of frustrated index, I would label as the mistakes you do when you implement AI, right? If you allow your AI to get into loops, if you don't train it on synonyms well enough so that it understands the context and and and misunderstands your users, you know, if you don't have the right guardrails in your product that prevent hallucinations and sell you uh, you know, a one-dollar car, like that Chevy example or the I don't know if the Air Canada example where they gave away a bereavement flight that didn't exist, you know, made it. But that's that's the problem with AI. That's bad AI. Yeah, right. It's that those are the mistakes you make, is that you don't put those guardrails in place. So those are kind of the maybe the top three or four examples that I have of people like, you know, implementing AI poorly. Yeah. And that you need to avoid.

Priscilla 35:05

And I'll add my own to that, which is not giving people a way to get to a human.

Craig 35:11

100%.

Funny AI Stories

Priscilla 35:12

I think that a lot of times I will hear these stories of people like jumping into AI. It's so exciting, it's so great, but they don't give you a route out to a human. And I think that's a huge misstep to not have that, especially if your users are used to working with humans. And then now you put this AI in front of that, you're gonna lose trust if they don't have a way to get to you. And I think you said it earlier, this idea that, you know, it's not that they don't want to work with AI, they just don't want to work with bad AI. They don't want to work with a bad chatbot that's gonna throw them in the loop. And so you're gonna run into those situations. And I think that frustration index is a really good way to measure the success of your AI is okay, how much are my customers getting stuck in these frustration cycles? Because that's what you want to avoid when you're implementing AI. You wanna make sure you're giving a smooth, top-notch experience because I think customers are gonna have a shorter fuse when it comes to working with AI than with a human when you're like, give them a bad response. Immediately it's gonna be like, nope, I knew I knew this was gonna be bad and now I want out. You know, so you have to be really careful with that. Do you have any funny stories where AI just got it wrong?

Training Skills Prompting Personalization

Craig 36:23

I would say it's wrong, but I have a couple of funny ones where we had a particular clothing brand and it was very flowery, very summary, like that was their brand. And they speak with the poetic, you know, nature of Maya Angelou was something again, very kind of like poetic and summary and happy. And so when we were building the tone of voice, um, we put that in there, you know, speak with the poetic, you know, angles of of Maya Angelou. The bot was hilarious. There's some of my favorite ones that came out of this was, you know, it was things like, you know, what's your shipping policy? It's like to traverse the world with through the winds of change, as the, you know, and it was just it went on and on and on. It was like, we will take it to Europe, we will take it to Asia, and we will even cross the Pacific over into the Americas. And like it was just that was their shipping policy as interpreted by that. And then we had another customer, they were uh a company owned by a famous uh rapper, and so they wanted to have that kind of built into the same thing in the tone of voice, and and the same thing. The the the questions came back with this like real beat, like you know, kind of nonsensical, lots of slang, lots of so those are a couple of funny things, but I can't think of an example where an AI has gotten it completely wrong. I think there's some really interesting things that you don't think about when you're training AI. So, for example, we this was actually a Kodif example. So we use Kodif on our website as you might expect, and someone was asking for a phone number to contact us, and we don't really have a receptionist, like we don't have a phone in our office. We're a small startup, but the AI, in one of our privacy policy documents, found a phone number for an old office when the company was first incorporated. So the document was out of date, but but it found this phone number, and it's like, oh, the only number I have access to is this phone number, which the person then called, and of course it was disconnected. And so it was one of those things where it's like, well, where did it find the number? But it was just so funny because someone was like, Well, how did it, where would it have picked up a number? And it did it hallucinate this phone number, but it didn't hallucinate it. It was it was in the training that, oh, it had said if you looked at the document to contact Kodif for uh a privacy or security concern. Here's this phone number. Yeah, just stuff like that, where we get these customers who who ask us, well, how did it come up with this response? And you look and it's like, yeah, you know what? It actually it makes sense it came up with this response because of these uh the connections it's making here, even though those connections wouldn't wouldn't have been made by a human anyway. I love the I love the Total Voice things because I think that those are really interesting interpretations.

Priscilla 38:56

Well, and I think it speaks to the importance of, and you were talking about this a little bit ago, but the idea that like our skills are going to change now for customer support specialists, because you're gonna need to lean into these other skills in order to correctly train the AI and those strategies. And so it kind of leads well into my next question, which is about those skills, because you need to be able to have people on your team who can train the AI in the right tone to navigate those kinds of um examples that you just gave, where someone asked for a certain kind of tone and it didn't react in the way that is gonna ultimately be the way you go moving forward. So, how do you train people on training the AI? So, what other kind of skills do you see needing to be developed? Say you're in customer support and you're looking at the future of your career and you're like, I want to invest in some of those skills now. What are those skills that people are gonna need to have sharpened going forward in customer support?

Craig 39:56

So, I mean, that the number one is just understanding how AI prompts work.

Priscilla 40:01

Yeah.

Craig 40:01

Right? AI prompting is a is going to become a skill. And in fact, I would argue it's something that they should start teaching in in the high school areas. Yeah. It's gonna become that fundamental. And again, in five years, probably I'll look back on that statement and say how stupid I was, because now AI can interpret anything. But yeah, right now, AI prompting is so vital. And what I mean by that is that let's go back to the root cause analysis report example. In a world where you're producing that report, you know, the skill is to logically take factual data and analysis and put it into some logical order to read, right? That's the skill. In a world where you're not producing the report, you are sharing the information within with an LLM and asking it to produce the report based on the information. The skill is to describe the information, what are the key factors it has to highlight, what are the things to ignore, what to point out, what not to point out. And there's a specificity to that, right? There's a skill to being able to be very precise in your language to tell something how to do something. And uh the better you're at at doing those things, the more the report will sound like you, the more the report will look like how you want it, the less editing you'll have to do after the fact. But that bit is going to become the skill. And I think the number two skill is we have to get into a mindset of this personalization more as humans, too. I had a really cool experience with Air Canada recently where I had to change something on a flight and I called, and the first thing it said to me before it even started ringing was based on your phone number, uh, we think you're calling about uh your flight to Denver next week. If this is correct, press one. Wow. Right? Like, wow, seriously, because like now, how many times have you tried to say those six-digit alpha numeric codes in a broken NATO alphabet? And N, you mean N as in Nancy? No, M, M, N, you know, yeah. That was a beautiful use of not really even AI, it's just use of technology to get that and personalization. Personalization, but because that's what people are going to get used to with AI, humans have to adopt that mentality now and be like, oh, if I have that information available, if I if I see a phone number with a name, or if I see uh the person has you know purchased this one product recently, again, don't ask them to repeat themselves. Don't ask them to spell everything out for you. Use the context to help them. Start the conversation with uh, well, how did you like that lipstick that you bought last week? You know, like there's a way to adapt that into your style and tone of voice. And I think again, that's a skill you're gonna have to teach uh your team. And I don't think it's unreasonable to say that that will become an expectation because if a bot can do it, a human should be able to do it.

AI Predictions & What Should Stay Human

Priscilla 42:41

Yeah. Well, I think it's one of the coolest things about AI and customer support, is it's going to give customer support specialists the ability to get even more personal and to personalize things and to focus on building relationships instead of just closing tickets. And I think that's really where if you're incorporating AI into your workflow so that you can focus on creating stronger relationships with your customers, you're doing that in the right way. Right. You're you're looking at it from the right perspective. If you're incorporating AI to eliminate a support team, that's probably not the right route to take. In five, 10 years from now, when we're using AI still, what aspects of this customer support relationship environment, like what are the things that need to stay human?

Craig 43:30

Things like empathy and situational re uh judgments and reactions are always gonna have to be in a human form, right? Yeah. It's so hard when you say five, 10 years, uh, because I just this this technology is moving so fast. So fast. It's one of these things that I struggle to know what my seven year old is gonna do in the workspace, right? Not because I don't think he's gonna have a job, by the way, just as we already talked about. But so when it comes to what should stay home, then I think it's a stuff that is uniquely human. Situational reasoning, I think, absolutely is there. There's there's no, you know, this artificial general intelligence, the AGI that the people talk about. I'm my God is it's not five, 10 years away. Like we're it it just doesn't seem possible to me. I don't think it's a hundred years away, but I, you know, creativity, like solving a problem. You you mentioned, you know, you do research with with AI, which again, I do too. I think it's a smart idea, but that's not ever going to generate a new idea. It's gonna validate other ideas, it's gonna say, hey, other people have done it this way. It's gonna help you absolutely. Yeah, no criticism, but it's not gonna generate a net new idea. It can't, because that's not how AI currently works. Right. AGI would have to exist for that. And so that's the stuff. Anything, you know, being able to, again, from a support perspective, build a creative solution, like, oh, I'm sorry, we're not gonna be able to reship this thing because it's out of stock, but here's what I can do because you know, I you're a valued customer, right? Like that type of creativity, uh, of problem solving. And I think anything to do with really deep product or service or whatever industry knowledge, like that deep level where it's like, I understand exactly what you're trying to achieve with our products and services, and I, as an expert in that area, can help guide you down a path to success for this specific use case. Again, that that's just something that if you, you know, you might be able to train an AI on on some of that, but I don't think it's gonna have that deep, full understanding that a human conversation can have. And so, yeah, those are areas that I think are uniquely human that that probably won't go away until AGI exists. And even then, AGI is not magic either. Yeah. It still has to have this basis of how to operate. And I do picture a world where I call it the big white box. That's my my name for it. Everyone is gonna have this big white box, whether it be a website or or maybe it's a physical box. It doesn't really matter. And that thing, that box knows everything about you. It knows your age, your height, your clothes size, it knows what type of dishwasher you have, it knows where you live, it knows that your floors are carpet, it knows whatever. It knows everything. And when you have an issue, you're gonna go to that big white box and describe your issue, and it's just gonna figure out how to do it, whether that be contact a human or a sport department, whether it be to call a service person to come and fix it, whether it be to buy something net new, whatever. The box will know everything about you, will know your spending habits, budgets, it will know everything. And you just say, you know, hey, my upstairs tap is leaking, and it's gonna be like, oh, well, that tap is still under warranty from Mowen. I'll submit the warranty request, and here's the place to do it, and I know your address. So I'm you don't even have to fill it in. It'll be there tomorrow. That's coming, right? Yeah, so that's a different type of support, right? That is someone, you know, from every all these companies plugging into the big white box to say, you know, if I want to be a plumber and I want a service call to these houses, that big white box needs to know about my plumbing services and my service radius and my prices and my and so there's another skill. You know what I mean? Like it you're gonna have to teach this big white, and that's but that's where we're going, right? And and so the you know, going back to what the human element is of that, it's the teaching of the big white box, how to get a hold, how to start a warranty request for a dishwasher, how to hire a carpet cleaning service because you spilled wine on it. Like that's the thing. That's the next you know, another layer of how support's gonna get there.

One Truth About AI

Priscilla 47:25

Yeah. I feel like we've touched on so many things in the last you know, hour. I'm gonna ask you one last question to wrap this up. What is one truth about AI that you want people to take away from this?

Craig 47:37

It would be disingenuous of me, especially someone who's in the industry, that it's not scary, right? There's a lot of unknowns, there's a lot of things that could go wrong with it. You know, there were people back in the 90s. I remember people saying this internet's gonna destroy civilizations, and and to some degree, maybe it has. Maybe it has. We can we can have that argument as well. But the thing is, is that it's it's all about adaptation. We should be trepidatious. I just don't think we should be uh uh A, completely ignoring it, and B worried that it's gonna cause mass global unemployment. Uh that's just not so the truth to me is that there's a middle ground where we accept that this is a technology that now exists, it will never not exist. And so, how do you want to use it in your daily life? And everyone can choose how to enable it or how not to enable it. And that's that's really got to be up to up to you. And then the rest is up to our leaders to to help society thrive with it. So, yeah, that's probably my my opinion on the truth of AI.

Learn More About Craig and Kodif

Priscilla 48:39

Well said. Thank you so much for being here, for sharing with us, Craig. This was really informative. I hope that our listeners are able to take some of this into their daily work. If they want to learn more about you, and if they want to learn more about Kodif, how can they learn more about you and Kodif and how they could use it?

Craig 48:57

Well, I'm LinkedIn. That's the main place I am. So look, I'm the only Craig Stoss on LinkedIn. I'm almost guaranteeing you of that. Nice. Um and uh and then Kodif is is k-odif.io. But yeah, I mean, please reach out. I love having these conversations. So if you have questions, if if you want to discuss something I said or you know, if you disagree, I I love hearing disagreements too, because that's how we we build better, uh, better content for people. So yeah, that's uh that's how you get a hold of me.

Priscilla 49:26

Yeah, and it's good to have these conversations about AI and to learn more about it. And that's the way things become less scary, is when you understand them more. So if you are listening to this and you're scared about AI, reach out to Craig, have the conversation with him, and you'll learn more and it'll take some of that fear away a little bit. If you have a question or a topic that you would like us to cover on a future episode, you can shoot us a text using the Send Us a Text link in this episode description. As always, if you like the episode, please share it with someone who works in customer support or leave us a review on Apple Podcasts. Thank you again to Craig Stoss for joining us on today's episode. Thank you for listening. Now go and make someone's day.