148. How to Drive Innovation & Democratize AI as a CS Leader ft. David Karp (Disqo)

And what they did actually wound up being turned into some enhanced capability that we are able to do and monetize with some customers now.

Josh Schachter [Host]:
Unchurrent is proudly presented by the Gainsight Podcast Network. About a year ago, a 300 person company called Disco faced a familiar challenge. Everyone was talking about AI, but many employees didn’t know where to start with it. So they tried something different. They protected two days from their calendars and ran a company wide AI hackathon. 70 people, 15 teams, engineers partnering with customer success managers, marketing teaming up with data scientists, all building prototypes to solve real business problems. Several of those prototypes are actually still being used today. But here’s what really caught my attention.

Josh Schachter [Host]:
After the hackathon ended, the CS team got so inspired they ran their own mini hackathon. People started holding AI office hours. The momentum didn’t die. Today. David Karpen, Disco’s chief customer officer and incidentally, one of the strongest CS leaders I know, is going to walk us through exactly how they pulled off the hackathon. He’s giving you his tactical playbook because running a hackathon is one thing. Creating lasting change, well, that’s what we’re all really after. I’m Josh Schachter.

Josh Schachter [Host]:
This is Unchurned. Hey everybody and welcome to this special episode of Unturned. I’m Josh Schachter and I’m here with David Karp. David is the CCO of Disco, which is helping brands grow by measuring the impact of their advertising. David, this is the second or third time you’re on the show. I don’t know, but welcome back.

David Karp:
Thank you. It’s always great to be with you, always.

Josh Schachter [Host]:
Yeah, absolutely. Okay, so we’re going to go like straight into it here. I want to get very specific with the topic at hand for this episode. This episode has become more and more about AI as everybody’s thinking about their transformations. And you and I had had coffee this week and we were discussing something really cool that you did. Actually, it was about a year ago, truth be told, a little bit less, but I think it’s still very relevant and I would love for our listeners to come out of it and say, oh, wow, I want to do that too. And thank you, David, because here is the playbook, here’s how I can take this to my org. So you, as a senior leader at Disco, you helped organize, execute the Hackinator project.

Josh Schachter [Host]:
Tell us a little bit about Hackinator, which is your AI hackathon, but tell us a little bit like, where did that idea come from to do that hackathon?

David Karp:
So we were trying to think. There’s two things we were trying to solve for Josh. One was, how do we help the whole company, all of Disco, start to think differently, get excited about AI, get comfortable with AI and start to embrace AI at the same time. We’ve always been passionate about innovation. So how do we get the company, the entire company involved and sort of think about leveraging AI to drive innovation in the company, specifically in ways that would help our company grow and benefit our customers. And so that’s the. That’s the problem we were trying to solve. And so we worked across our executive team, partnered closely with our chief technology officer to drive this hackathon.

Josh Schachter [Host]:
And so somebody raised their hand. You’re in a, in a, in a board table type boardroom meeting. Whatever somebody says, oh, yeah, let’s do a hack, a hack, a hackathon. And then there’s momentum around that, I assume. Yeah. What were the steps? I want to walk through the logistics of it, how you got there, how much work it was, the outcomes, all those sorts of things. What were your first steps in moving towards this? Once you said, okay, we like this idea?

David Karp:
Yeah. One thing I do want to note, Josh, is the foundation of it was as a leadership team. We had to all agree it was important and that we were going to go first. So we had been ourselves starting to use AI. So it gave us a little bit of authenticity when we could go back to the rest of the company and say, let’s go. And so we did have just, you know, we had a program manager helping run the whole thing because we wanted to coordinate a lot of. But the first thing we needed to do was invite people to have an opportunity to start to nominate and formulate ideas of problems they wanted to help us solve for the business using AI. And we didn’t assume that people were just going to know what that was.

David Karp:
So we spent some time as a leadership team saying, what are some areas we think are kind of starter ideas? We seeded the ideas for people and we gave them a number of potential business problems they could solve to help them know what we were thinking about, and then let people come back with their ideas on what they wanted to submit and say, here’s a business problem. I think we can solve using AI.

Josh Schachter [Host]:
Do you remember what some of those starter ideas are, the challenges that you wanted people to focus on?

David Karp:
Yeah. So we had a, we had a number of different things. There were things out there and again, it was very specific to each department. So are there things we can do with AI to rethink how we put our data together to create more valuable insights for clients? Are there some of the process flow that a customer would need to work through from onboarding to kind of initial value realization? Could we use AI for that? When we do a handover from sales to cs, can we use AI for that? There was a number of different places. We have a really rich data set and so part of it was can we also maybe let. Let AI discover trends in our data aspects, our data that we’re not seeing, that we could create new solutions for customers out of that as well. So we gave a number of different ideas specific to kind of a customer journey, specific to what a department is trying to do related to customer impact. And that’s how we started the ideas.

Josh Schachter [Host]:
Okay, so first off, you all agreed as senior leaders that you wanted to do something to create more of a movement around AI in the organization and have it become more effective for the org. You’d already been doing this, but you all yourselves were active in using AI and could lead by example.

David Karp:
Correct.

Josh Schachter [Host]:
You came up with that punch list of those key challenges so that you could create some guardrails and some focus areas that you believed would be most effective for the org. You. You sourced a program manager to help get it going. Yep. And where’d you go from there?

David Karp:
Yeah. So we created, we created some rules. So things like whatever you come up with needs to be something that is going to create impact within a certain time period. So six to 12 months. So we didn’t want to do, we didn’t want people to come up with ideas that were so cool, but they weren’t going to benefit the business in the near term. If we’re going to spend time on this, let’s create some near term benefit. We really wanted people to leverage AI in some capacity. They needed to have some kind of working prototype example that we could go deploy.

David Karp:
Those were some of the parameters that we put around it that we, that were, that helped focus people on the types of things they could go solve, either our idea or what they came up with theirselves. And then we gave them a kind of, we came up with a scoring mechanism that was everything from how quickly could we go deploy this, how big of an impact did we think it would have on the business? How creative was the presentation of the idea? We did all some of those normal things you would do. What was powerful about it is by, by putting those parameters around it. What typically we found what happened with hackathons is the constraint was an engineering constraint. If you could build a team with engineers, guess what, you could be successful in a hackathon in this case, if you can imagine it and you could think about AI and you could be part of a hackathon. So we let people come up with those ideas, we put those parameters around it. We created opportunities for people to put forth an idea and say, hey, I think I need some help. I need, I need somebody who understands this part of the business.

David Karp:
I really need a marketing person. So we let people kind of say I need, I need help, I need help on my team or hey, I’d love to join a team. We were very transparent about those ideas. So that’s how we kind of started to get teams formulated, put parameters around it. We gave people, here’s a set of AI tools that we support as a business that we can give you access to. And then the last thing we did is we put some, some skin in the game for people is we put some real decent prizes like the winning team got a couple thousand dollars and certainly we’re going to get a lot of notoriety in the company. So we really wanted it to be worth people’s while. We wanted to give them support, we wanted to help them think of the ideas and create teams.

David Karp:
But we also wanted to minimize the normal constraint that keeps a lot of people from participating. Our hope was we would get more participants in this hackathon than any more more traditional hackathon. We done that we’d done. We wound up with 15 teams that had valid ideas that they were going to submit. And each team was anywhere from three to six people. We’re only 300 people. So you’re talking, you know, 60, 70 people participating actively and coming up with innovative solutions that are going to benefit our customers and benefit the company. Game changer.

Josh Schachter [Host]:
Yeah, that’s a large cultural impact. So it sounds like the teams were self organizing. How did that work? Did, did, did teams did like somebody, you know, say I want to captain a team. I have an idea that you had a community forum. People raised their hand like how, what was the formation process?

David Karp:
Yeah, we kept it, we kept it pretty simple. We used, you know, good, good old fashioned. We didn’t quite go, you know, tablets and stone, stone tablets. But we Used, you know, some basic Google sheets. But whoever had the idea could say, hey, I, this is my team. They came up with their own fun names for their teams and then crowdsourced participants. Some people just went off and talked to teammates. But you had teams led by certainly by some engineers, by some product managers, by CSMs, by marketing people.

David Karp:
So we had different leaders that said, hey, I’ve got an idea, I’m excited about it, I want to put it forward, I want to go solution it and I want to get some, some friends and colleagues to be part of it. So that was, it was we, we created some mechanisms where it made it easy for people to join the team. But it’s whoever had the idea that wanted to lead it and they came up with their own fun names. We themed the entire thing. We called it the Hackinator. So we did a lot of to make it fun, but we were really getting after it because we wanted people to come up with something that would help the business and help them learn and get excited about AI.

Josh Schachter [Host]:
Did you encourage cross functional participation within teams or was that something that naturally spawned?

David Karp:
It was a little bit of, I think it was a little bit of both. So part of what we did when. And it was a lot of natural fallout that was cross functional. Again, you know, people, we tend to work cross functionally anyway and most of what we do requires cross functional partnership. But as people submitted ideas, we did go through and put our own lens on do we think they might, you know, do we have questions to clarify what problem they’re trying to solve? Because maybe, maybe in some situations we asked teams to combine because their problems were so similar. We thought it would be better if they work together. Sometimes we saw situations where we think that team could benefit. Let’s see if we can help them get somebody added that’s going to give them that perspective.

David Karp:
So, for example, we had some teams that were engineering teams, product teams, really, really smart, but they were trying to solve a problem that was going to directly touch a customer’s interaction with us. Boy, that team could really benefit from someone who actually talks to those customers. And every day, if it’s going to really help them with their solutions, let’s suggest they add in someone from sales or CS or someone else who can help make sure that what they’re going to create is going to resonate with customers.

Josh Schachter [Host]:
Yeah, hearing you talk about it in hindsight makes so much sense that you were there behind the scenes helping to shepherd the process for them and helping to curate and guide the team formation. And the ideation formation, even before the hackathon, I wouldn’t have thought of that. When you first told me about this, I’m like, oh, great. So there’s a weekend, everybody shows up and then the 24 hour, 48 hour clock starts ticking and then they go and form teams. But it was, it was, it was rehearsed a little bit or prepared before so that you could make the most out of that time.

David Karp:
Look, what’s so great is people did most of all of that themselves. Yeah, I think starting from scratch though, especially where a lot of people are with AI, when you ask them to come up with some solutions where I can help, it’s really hard for them to imagine it. So it’s giving them some starter ideas that we’ve thought of because we’ve been using it for a while. That just sparks their thinking and then they go from there. And people aren’t always sure. They don’t realize that if they add in a different perspective it can help. So I mean really, once you, once we gave people a little bit of vision and a little bit of inspiration, I would credit all the people on all the teams for driving it all. We were just trying to make sure behind the scenes we were helping people be as successful as possible because we really wanted to take that time and turn in a benefit for the company and customers that was really, really critical for us.

David Karp:
But the really underlying heart of it was how do we change culturally and get people excited and embracing AI and we thought this was a great way to build some momentum.

Josh Schachter [Host]:
When did it take place? What was the start and stop?

David Karp:
Oh gosh, you’re tapping into my old brain.

Josh Schachter [Host]:
I don’t mean like, was it October, November? I mean, was this a weekend type of activity? Was this over the course of. Was it.

David Karp:
We, we did it like it was a couple, like two days in, during a, during, during the week. We carved out space for people and said like. So we, we allowed people to work on it really whenever they wanted. So the official days to prove everything out and do your work was over. Was over. Like a two day, two day period, I believe is what we wound up doing. But ahead of time, if people wanted to do some things, they could, but it could not impact any of their ongoing work. Yeah, right.

David Karp:
So we didn’t want to slow people down if they wanted to start to do some things with AI, but we really put the primacy on you. Like it was not, I can’t come to that meeting because I’m doing something for our hackathon. Once we got into the. Into that window, we still had to have people cover for each other because there’s a lot of customer responsibilities. But that was the timeframe when we told people, go ahead and go, go, go focus on the hackathon. But we didn’t do it on a weekend. We actually did it. We did it during a couple days, a couple days of the week.

David Karp:
And it culminated with people. Then we had a phenomenal session where people came together, presented their ideas.

Josh Schachter [Host]:
And that was immediately. At the end of that, 48 hours.

David Karp:
At the end of that, now you’re gonna come and. Yeah. And show your idea. So part of that time was putting together your presentation of what you’ve done.

Josh Schachter [Host]:
Yeah, yeah. So everybody wanted to have the product. The product marketer on their team as well, or just use Gamma app, I suppose. So what was. When you get to those final presentations, what was the scoring and evaluation criteria for the ideas?

David Karp:
Yeah, and a lot of it was, is this a. Is this a valid. Is this a workable idea? Do we think we can actually take this idea and put it into practice with some reasonable hardening of the idea? Is it going to have benefit for a customer or disco? Is it going to help our business? Is it going to help us dramatic efficiency? Is it going to help us out in the market? Are we going to grow? Is there revenue opportunity? So those were really the core criteria. But then we did look at creativity, things like that. How compelling was the idea and the way people presented it. So we had a couple of scoring mechanisms around that, and that really helped us because you can imagine, like anything, some of these ideas were nuanced, and so the judges had some different perspectives on how important things were. So we had to make sure we had a scoring mechanism that worked and that we could describe it back to the participants so they would understand what they were shooting for. And then what we did is we actually.

David Karp:
We had 15 teams and we came out of it and we actually not. Not intentionally cut it in half, but we cut the finalists in half. There were so many really great ideas and the scoring was so close that we actually went back to some teams and said, hey, can you clarify this piece? Hey, you stopped here. What would you do? What would we need to do to take this idea to the next level? Because we really thought there was so many great ideas that were workable, that we could deploy. We wanted to go back and give people a chance to sharpen their thinking. And that’s how we ultimately came up with our final scoring and our top Three teams all got prizes and it was very competitive.

Josh Schachter [Host]:
Sounds like, it sounds like there was a lot of motivation. So at the end of the hackathon period they presented to the judges which was composed of senior leadership members. You had the judging criteria there, a large part of it which was, you know, what’s the business impact of course. Cause that’s at the end of the day you’re trying to build culture and impact. You had those, those 15 teams pitch, eight of them was a close call. So you needed some clarification I suppose. Then you took a few days and then came back and they, they, they took a little time, got you some follow up responses and then you came back and you declared the three winners. What did those winning teams win?

David Karp:
So the winning team won a couple thousand dollars. I think it was about $1,000 for second place, 500 for third. It was, it was real. What we also did with the winning team was we gave them a chance to present their solution to the entire company at one of our all hands meetings. I will say it was a CS led team but one of the most amazing things that happened because I think we gave people a little bit of vision, we gave them the chance to self organize. They saw the impact they could have. One of the things that that CS team did, there was one blocker in their solution that they couldn’t figure out in the couple of days. So they ran their own little mini hackathon to say how do we create little teams to see if we can solve this problem.

David Karp:
And what they did actually wound up being turned into some enhanced capability that we are able to do and monetize with some customers now.

Josh Schachter [Host]:
That’s awesome.

David Karp:
So they, they presented, that team presented at a kind of our final end of year hackathon. So they got a lot of notoriety, they got some additional benefits in there in their bank accounts. But I think it really, what do.

Josh Schachter [Host]:
You think was the main driver for them? What do you think was their greatest motivation in going all.

David Karp:
I think which this is again and I think, I think most teams are this way. There was an op. They kept this team, kept hearing from customers. There was something that customers wanted us to do for them. And it wasn’t something that was so prevalent that we would prioritize it on our product roadmap. But it was a close in kind of advanced need that clients had. But it really took some sophistication with our data, some nuanced things you’d have to do. And the team said, I wonder if, if we just apply AI against our data sets.

David Karp:
Can we create some aspect of this solution without having to take engineering time and all those things and wait and go do something for a client? Now that was really the driver behind it because they could link it to benefit. These are people that were obviously incredibly passionate about delivering value and impact for customers. And that was their driver. And they got to, they got to make it happen. That’s really what was behind it.

Josh Schachter [Host]:
Yeah. So you had 15 teams participate. They ended up with 15 working prototypes. Through the hackathon, you had eight that were really, sounded like really more compelling. The three winning teams. And then the ultimate victors were, were, were the CS team. Go. Cs.

Josh Schachter [Host]:
What, what do you do? What did you do to make sure that the momentum didn’t die the next week or so?

David Karp:
Yeah, right. Yeah, it’s, it’s important. So we did a couple things because we really thought about this was just one step on a journey. We thought it was important that we make sure that everybody in the company who wants it gets access to a couple tools that we would sanction, we would get enterprise licenses, we would create some training, some policies on how do you protect data and use it responsibly. We also gave people some, some training courses that we thought were good, foundational courses, general concepts, prompting and other things. So those are some just poor things. Hey, everybody’s excited now. Everybody’s aware.

David Karp:
Let’s go equip you. The next thing we did is we started running AI office hours every couple of weeks. And that was just a chance for people to come together, bring it, do a little show and tell, come and ask some questions. And we would get, I think typically 50, 60 people, which is again a large swath of the company, from every part of the company. Hey, can I show you something I did. I’m not sure if this is okay. Hey, I need help from somebody. We would pair people up through that.

David Karp:
Those were big things that we did to keep building in the momentum. And then the last thing we did is we built a very dedicated cross functional team to try to drive a bit more, you know, AI workflow, AI agent type work each team came up with. Here’s the business processes that we’re working on that we think we can apply it. That team meets once a month. They do a lot of show and tell. They hold each other accountable. So we were just continually moving people forward to get more comfortable, to start thinking differently, to start to think that I, I have agency to use that word in how AI impacts what I do every day. I can actually Turn it into a benefit for how my work gets done and how I, how I translate that into value for disco and for our customers.

Josh Schachter [Host]:
Yeah. Do you know how many of those hackathon prototypes are now in usage, in production usage at disc?

David Karp:
So I think they have some of them evolved, but in some respects you could argue probably three or four have come to life in different ways, like in really, really cool, compelling ways. And some of them evolved, but the idea and the initial prototype was seeded from that and then other people picked it up and said, well, let’s add to it, let’s turn it. So pretty, pretty powerful stuff.

Josh Schachter [Host]:
So looking back at the experience as folks are listening and I’m sure you’ve got a lot of SaaS leaders that are saying, you know, this is amazing, sounds like it was very successful, congratulations. But I feel overwhelmed and can I really do a hackathon? What would be your recommendation to them? Is that where you would start or would there be a different starting point or would you have other recommendations of what they could do?

David Karp:
Yeah, look, I, I think for any of us, Josh, the first thing is am I myself getting comfortable using AI? What really drove, I think a compelling approach was we each, each of our leaders had to decide ourselves do we want to do or do we think this is right and let’s try it ourselves. And we spent time, we’d get together and we would use, apply AI differently. I had a number of incredible sessions with my CEO, we were trying to analyze customer data and we would start using AI ourselves. Just basic stuff. So I think number one, if you’re not doing that, I think it’s a good opportunity to say how do I change? Go go grab a colleague in the industry, go find a way and get comfortable yourself and then start working with other leaders and where, you know, you have problems that are cross functional, start dreaming about how do we, how can we start to solve it ourselves? I think if you don’t, if you don’t believe it and you don’t have some level of comfort, it’s really hard to ask other people to do it too. So essentially what we did is we had a little bit of a top down approach. It started at the top, but our real dream was how do you make it bottom up as well. And so that it was vision, vision from the top, equipping from the top.

David Karp:
But once you’ve equipped people, let them start to have enough success that they start to drive it. You need top down so you’re, you’re effective, you’re efficient with it. You’re not. You don’t have 17 different tools. But bottoms up, let us go really fast. And I think for any, any leader, that’s what you can do. You don’t have to go do some big hackathon. You can start yourself, maybe get a couple people on your team that are dreaming of doing something, get them started again.

David Karp:
We, we had a company wide hackathon, Josh, but the CS team decided to do their own. I didn’t even know they were doing it until they had been doing. They were a couple weeks in and somebody told me, oh, oh, by the way, we, we’re doing our own hackathon. Amazing. But that’s only because.

Josh Schachter [Host]:
So CS leaders listening in that are not on the elt, the, you know, the slt, they can run this themselves.

David Karp:
They can do it themselves. Yeah, a hundred percent, I’d say, because. But because the team had done it once and they were using it and said, we can do this, they were comfortable doing it themselves, it had to start with them trying it first. You don’t, this does not, this does not have to be some big coordinated effort. You can do it right where you are with a couple of people. I think any of us can think of business problems we’re challenged with right now that if we applied a little bit of AI, it can drive some efficiency, some insights, some benefit for customers. Just start there.

Josh Schachter [Host]:
Yeah. Well, listen, David, congratulations. You’re always leading the charge. It sounds like it was a lot of work, but a big payoff at the end. And honestly, I asked you a lot of questions there, 20 questions. But thank you for going through it all with me because I wanted to make sure this episode was very tactical and tangible for folks that might want to dabble with it on their own. And I’m sure you’ve inspired many folks that are listening today. So thank you very much.

David Karp:
I appreciate it and always appreciate our partnership. And you’re, you’re constantly raising the bar yourself, Josh. You give us a lot to aspire to, and I really appreciate it. And if hopefully this is helpful for, for somebody, and if somebody wants to dig in and go deep and have a conversation about it, I’m always happy to do that, do that too, because I’d love to learn what other people are doing.

Josh Schachter [Host]:
Thanks.