154. The Ex-Google PM Who Spent 15 Years Building AI Agents ft. Jacob Bank

An agent does work on your behalf and so you can think of them less like a traditional productivity tool and more like a teammate that you’d hire to do work on your behalf. I think the biggest thing holding people back is, is to figure out a, what do they want help with at all, and then B, when is an agent appropriate to be the tool that helps you? And so let me try to give you a couple guidelines of when an agent is the right tool for the job and when it’s not the right tool for the job.

Josh Schachter [Host]:
You’re listening to Unchurned, brought to you by the Gainsight podcast network. It’s 2011 at Stanford University. Jacob bank is researching AI agents in the Multi Agent Systems Lab. Two doors down, Andrej Karpathy is inventing deep learning. Jacob was in the right place, working on the wrong thing, at least for that moment. He drops out, starts Timeful, an AI calendar assistant, sells it to Google in 2015. And six years later he leaves his director role at Google because he sees what’s coming. Productivity software is about to completely transform from tools we click buttons in to AI agents that do the work for us.

Josh Schachter [Host]:
He founds Relay App letting non technical people build AI agents without coding. The first two years, they’re brutal. Then ChatGPT launched. Everything changed today. Jacob bank has spent 15 years on this vision and now finally the world is ready. I’m Josh Schachter and this is Unchurned. Hey everybody and welcome to this episode of Unturned. I’m Josh Schachter, very excited about this particular episode because I’m here with Jacob bank, who is the founder and CEO of Relay App.

Josh Schachter [Host]:
And Jacob is going to be telling us all about how to build agents, which is something that we’re all trying to figure out together as a community. Jacob, thank you so much for being on the program.

Jacob Bank:
Thanks, Josh. I’m excited.

Josh Schachter [Host]:
I’m very excited, Very excited. Okay, so let’s start a little bit with why don’t you introduce Relay App to us all. Know what you guys do as a platform, a little bit about the stage of the company, those sorts of basics to get us going.

Jacob Bank:
Great. Relay App is a platform for building your own workflows and AI agents. If you’ve used tools like Zapier or N8N or Gumloop or Lindy. We’re in the same category as those tools. So we enable non engineers and non coders to create AI powered workflows for a wide variety of operational tasks. And we’re going to dive into a bunch of the ones that are useful today for customer success. And the key thing to keep in mind about Relay app is that it is built for users who are less technical. So if APIs and HTTP requests and JSON objects are all terms that are unfamiliar to you, you’re going to likely have a better time with Relay than other tools in our space.

Josh Schachter [Host]:
I love that. I was just at a Brooklyn meetup in New York yesterday last night and I said it was on MCP building MCP servers and I thought to myself, you know what, let’s head over to Williamsburg and learn a little bit about how to build mcps. And I got there and it was all so over my head. I did not, I couldn’t understand. I, I nudged the guy next to me, my elbow was like, hey man, do you understand what’s going on here? So I love the fact you’re doing this for dummies like myself here, right. For basic folks that maybe have a little bit of technical background, but by no means are developers. That’s a beautiful thing.

Jacob Bank:
And maybe to add a little more detail that could be useful, our customers fall into two buckets. One bucket are less technical or semi technical roles within technology companies. Marketing, sales, customer success, customer support, hr, finance, recruiting, et cetera. And many of our customers are real world businesses that install slip resistant floors for factories, or paint warehouses, or make menu or make carbonated beverages. And so we get to see a wide spectrum of different use cases and users. But the general spirit is to enable small and medium businesses in particular and less technical folks to get the benefits of AI agents.

Josh Schachter [Host]:
Yeah, yeah. And we should talk about the differences between small, medium and then enterprise. Maybe why you’re not in the enterprise game. What differences are there? A lot of our audience are across all spans and all segments of different companies out there. You’re backed by some cool folks. You’ve got Khoslaventures, Andreessen Horowitz, you know, you’ve got the creme de la creme in there. You were a director of product management at Google before starting Relay app. And so I guess I want to know, I mean, that’s a pretty compelling track to be in as a director of PM at Google.

Josh Schachter [Host]:
So what was the driving force and what did you see Four and a half years ago when you jumped over and found this company.

Jacob Bank:
Maybe I’ll tell you my slightly broader career arc, which I think makes it make sense as opposed to just the last jump, which is I started my career as an AI researcher and I thought I was going to be professor. I was studying AI at Stanford and my research was always at the intersection of how can we build AI assistants that help people get work done. I ended up dropping out of my PhD to start my first company, which.

Josh Schachter [Host]:
Is, this is by the way. So, so you, you had your PhD, you were getting your PhD around 2011.

Jacob Bank:
That’s right.

Josh Schachter [Host]:
I started 14 years ago. You started in New York before then.

Jacob Bank:
And let, let me tell you, like the research group I was in at Stanford was called the Multi Agent Systems Research Group. And so AI agents have only burst onto the scene now in the last year and we can talk about some of the reasons why, why they’re so hot right now, but there’s actually been a long history of foundational AI work that has made this moment possible. And so you know the story of my first startup. I dropped out of my PhD to work on my first startup, which was called Timeful, which was an AI powered digital calendar assistant. And the way it worked is you would tell it, I want to spend this much time in meetings, I want to spend this much time on my personal priorities, I want to spend this much time on creative projects. And the agent, the Timeful agent would help you automatically organize your, your calendar. We then ended getting, ended up, ended up getting bought by goog Google in 2015. So that’s how I ended up at Google and I stayed there for six years where I led the product teams for Gmail, for Google Calendar and for several of our other productivity tools.

Jacob Bank:
And so I’ve worked, I’ve had this same problem that I’ve basically worked on for my entire 15 year career, which is how can we use AI to help people get stuff done in productivity tools. And for the first like 14 of those years it felt like we were too early. And now finally, now finally it’s time.

Josh Schachter [Host]:
You’ve been. So you’ve been researching agents for 15 years, building them for 10 years effectively.

Jacob Bank:
If I’m thinking, yeah, something, something like that. Like, something like that, 13, 14 years. Depends what you count as an agent. But. But yes.

Josh Schachter [Host]:
And how many hundreds of millions of dollars did Meta offer you to join them recently?

Jacob Bank:
I, I wish. Unfortunately. Unfortunately I don’t know how to chart train large language models, which is the thing that’s Actually important right now.

Josh Schachter [Host]:
Yeah. Okay. You should have taken that other class at Stanford. I suppose.

Jacob Bank:
Well, so this is actually funny. I don’t know if you’re interested in this backstory like, so I was in the multi agent systems group and literally two doors down from my office is where deep learning was being invented. Like Andre Karpathy was in the office and two doors down, Andrew Ang was the door, three doors beyond that. And so I was, I was there when it was all happening. I was just working on the wrong stuff.

Josh Schachter [Host]:
Yeah, but you were working under some pretty, you know, highfalutin people as well. I, I see.

Jacob Bank:
Yeah, I know.

Josh Schachter [Host]:
I’m under Dan Ariely. I mean he’s a leading behavioral economist.

Jacob Bank:
I’m being a little facetious. I was very, very lucky in what I got to work on at Stanford. My two co advisors were both amazing. One is Dan Ariely, the well known behavioral economist and the other is Yoav Shoham who is the pioneer in multi agent systems. He wrote like, he wrote the paper on how AI agents work like back in the early 90s. And so those two. And he’s now the co CEO of a really cool AI company called AI21. So I was joking that I was super lucky to have those advisors and they became my co founders at Timeful.

Josh Schachter [Host]:
Amazing. Okay, so you sold to Google, you sold Timeful to Google. You were there for six years. So you’d already released the golden handcuffs of the acquisition. And so there was something then that said to you, okay, great, now is the time to jump into Relay app. What was that impetus for you?

Jacob Bank:
I knew that the time was right for the long awaited rethinking of productivity software. Basically, for the last, whatever 40 years since the computer, personal computer was invented, productivity software has had the same shape. You type in, you click some buttons on your computer, you open up a screen and then you do your work in that screen. You know, you send emails in Gmail, you write documents in Microsoft Word. You have gone to visit a productivity tool and then you do work within that productivity tool.

Josh Schachter [Host]:
Yep.

Jacob Bank:
The refactor that I’m excited about is from thinking about assistive software not as a tool where I go to do work, but an active assistant, agent, teammate, use whatever word you want that helps me do the work. And I think now is the moment where we’re all making this huge transition in how we think about getting work done. From I go to an app and I click buttons to, I tell my AI assistant what I want it to do and then it helps me do it, or it does it on my.

Josh Schachter [Host]:
But you didn’t start this app, this, this, this platform, this company six months ago. You started this four and a half years ago.

Jacob Bank:
That’s right. We were a little early, I think so, yeah.

Josh Schachter [Host]:
I mean, and they say, like, I mean, listen, investors, you’ve got great investors, but investors, it’s just as, just as harmful to be too early as it is to be too late.

Jacob Bank:
The first two years of the company were very, very hard. Very hard. They were hard for a couple reasons, not so much because fundraising was hard. Luckily, I had relationships with investors from my previous company, but customer expectations. People were not really ready for AI to play a large role in their work. And so I remember the very first pitch we gave to the very first prospective customer was, we’re going to build this AI project management assistant for you and here’s all the things it’s going to do. It’s going to write your status reports, it’s going to update your bugs, it’s going to, you know, manage your backlog, et cetera. And the reaction to that was like, I don’t believe you and I don’t want that.

Jacob Bank:
And then it was only, you know, ChatGPT then came out in November of 2022, and then in 2023, people started to find AI actually useful in their work. And then in 2024, we started to see the first AI companies, you know, cursor, et cetera, really, really taking off. And so I think already, you know, when we started the company in like GPT 2 days, I think our AI was already good enough to be useful for some of these use cases, but people’s minds were not yet ready. And then over the last three years, we’ve seen not only a huge evolution in the capabilities that AI has, but we’ve seen an equally huge evolution in how excited people are about figuring out how to put that AI to work. And I think it’s actually required both of those evolutions, the model quality and the people’s readiness to make related app like the viable company that it is now.

Josh Schachter [Host]:
Yeah, I mean, in terms of that viability, I can imagine. I don’t know your numbers, but I can only imagine it looked like a hockey stick with a really long blade at the bottom. I’m a hockey player, so that, that translates exactly right.

Jacob Bank:
There’s like. I mean, it literally took us two and a half years to get our first paying customer. It was excruciating. And now we’ve like. It turned out that all that hard work put us in the right place at the right time, when AI agents took off, as they have in the last year or so.

Josh Schachter [Host]:
Yeah, okay, so educate us a bit on agents. I, I go around and I’ve been speaking at a lot of different conferences and in front of a lot of different groups. And recently I get a big like disparity of, of folks when I ask them if they know what agentic is or even if I ask them if they’re using agentic and then if they could actually explain it back to me. And some places like the fair amount of folk, but a lot of places like barely any hands go up. And it’s even in San Francisco, I was giving a presentation for a hundred CSMs NCS leaders and like less than 5% of folks raised their hands that they’re using agentic or even could explain what agentic is. So help us demystify that to start. What is agentic AI?

Jacob Bank:
So a large language model, which is the core technology that powers most of the AI experiences we all use. For example, ChatGPT takes in text and spits out text. That’s what a large language model does. You give it, it’s, it’s actually the mechanism behind it is incredibly complex. But the actual machinery of what it produces is very simple. You give it text, which is called the prompt, and it gives you back text, which is called the response. And all the response is is what the large language model thinks is the most likely word that would have followed your prompt. It’s like that’s all it is.

Jacob Bank:
It takes in a set of words and it predicts the set of tokens that are, that are going to come next. That is the core of a large language model. Now when you want to put these large language models into action, you often want to have them do more for you than just taking in text and outputting text. And so an addition, an additional capability was added to these large language models, which is called tool calling. You might have heard about tool calling and what tool calling means is that in addition to the large language model giving you a text based response to your prompt, it can perform actions in other tools. An example of an action might be looking up the weather@weather.com. another example of a tool might be calling a Google search. Another example of a tool might be sending an email.

Jacob Bank:
Another example of a tool might be looking up a record or modifying it in your CRM. And so tool calling enabled large language models to take action and not just produce text. There’s a specific way you can set up your AI Interaction that combines a prompt and tool calling to create an agent. And let me tell you how that works. An agent has a goal and a set of tools, and it is responsible for figuring out how to use those tools to achieve that goal. For example, I might say a prompt like, I’m about to be on a podcast with Josh. Can you help me research him, his background, his company, and then write me a briefing document with some bullets that I should mention? The set of tools you have available are you can look up profiles on LinkedIn, you can look up websites, you can run Google searches, and you can create a Google Doc. And then the agent would be responsible for figuring out how many times to call each of those tools and in what order to call each of those tools to achieve the goal I set out in my prompt.

Jacob Bank:
And so the way you should think about an agent, the technical definition of an agent is you give the AI a goal and a set of tools and it figures out how to achieve what you’ve asked to do using those tools. Now, if that definition feels a little bit abstract, there’s a practical definition that I’ve found to be more useful when I’m working with our customers. And the practical definition that I found to be more useful is think about the AI tools you interact with in three categories. Category number one is a chatbot. We’ve all used ChatGPT by now. You type in chatgpt.com, you write in your prompt, and it gives you back a text response. That’s a chatbot AI. Tool.

Jacob Bank:
Category number two is a copilot. You’ll know you’re in a copilot if like 80% of the screen is the thing you’re making. The slide deck, the website, the code, and then the 20% of the screen on the left or the right is the AI assistant that helps you modify that code or spreadsheet or whatever it is. So chatbots and copilots are still productivity tools in the traditional sense. You go to a destination and you click buttons and you type stuff on your keyboard and then a slide deck pops out. An agent is fundamentally different. An agent does work on your behalf. And so once you’ve told your agent what you want it to accomplish, which tools it needs to work with, how you want it to perform that task, repeatedly, over time, it will just do it.

Jacob Bank:
And so the reason people are so excited about agents is, is not because of the technical details of iterative goal oriented tool calling. I don’t think that technical detail really matters to most people. What people are excited about with agents is that they do work on your behalf behind the scenes while you’re sleeping. And so you can think of them less like a traditional productivity tool and more like a teammate that you’d hire to do work on your behalf.

Josh Schachter [Host]:
Yeah, yeah. And I didn’t hear you say reasoning, but I assume that’s kind of embedded in what you were talking about as far as the agent has the tools and so the reasoning is figuring out effectively how to use those tools in what sequence and that sort of thing.

Jacob Bank:
Yeah. Reasoning is a related concept that lets large language models have more complex chains of thought. To answer a question, reasoning models can be applied in either like a traditional text in text out situation or they can be applied in a tool calling situation. When people say reasoning, what they typically mean is that the model has an additional level of thinking that it’s capable of doing that breaks down the problem into sub steps and then tries to solve each of those steps individually and then reasons about the whole answer as opposed to just like text in, text out. And so reasoning is a related concept to tool calling. Often to make your agent perform well, you will want to use a model that has reasoning capabilities. For example, you know the OpenAI O series models, but it’s not a strict requirement. You can actually do agentic tool calling with a non reasoning reasoning model.

Jacob Bank:
That’s probably getting a little more in the weeds than is necessary for, for, for the audience. But you should think about like switching to a reasoning model when you want step by step analysis and then breaking down the problem into into subcomponents. That’s like, like what I think about it. If like if I was giving the instructions to someone on my team and I was going to give them like really detailed step one, step two, steps three, step four to create this analysis then I would choose a reasoning model if I wouldn’t feel the need to give those detailed step by step instructions for a task. Usually a regular non reasoning model is sufficient.

Josh Schachter [Host]:
But then where, if it doesn’t have the reasoning, where is it getting the logic and the understanding of how to use the tool, how to go fetch the tool?

Jacob Bank:
The tool specification is available to the large language model upfront like you tell it which tools it has available to reasoning. Like the reasoning or the chain of thought just governs the mechanism by which it breaks down the problem and tries to solve it. And so even if the model does not break down the problem and use chain of thought reasoning, it can still do a decent job at like calling a tool to send A couple emails.

Josh Schachter [Host]:
Okay, we’ll leave it at that. Good. We won’t get too technical of that piece. What about context? I don’t think I heard you use that explicitly, but you hear all about context, and context is all that matters.

Jacob Bank:
And I’m really, I’m really glad you mentioned that, because in my experience working with AI over the last two years, everyone makes such a big deal about prompt engineering and writing the right prompt and iterating on your prompt and making sure your prompt is following the perfect prompt framework. And I just haven’t found that to be necessary in practice. What I’ve found is that a very simple prompt with very good context is more than enough, like for most problems. And so I it, I think about context in two ways. Context, generally speaking, is the supporting data or background information that the AI needs to achieve a task on your behalf. So, for example, if you’re asking the AI to classify an email, it better have the email. If you’re asking the AI to write a metrics report based on a spreadsheet, you better give it the spreadsheet, et cetera. But context also includes other background information that is relevant to the task.

Jacob Bank:
For example, if you want to have your AI write a blog post, you probably want to give it some historical examples of blog posts that you’ve written and maybe like a tone and style guide or some other information about your brand. And so those elements together, both the immediate data the AI is operating on and then the additional background or reference information it uses, are both referred to as context. Now, again, to get a little bit technical, there are two ways you can give your large language model context. The simplest way you can do it is you can just write the context right into the prompt. So an example of that would be, you know, please write a blog post about AI agents and customer success. And here are four examples of blog posts I’ve written in the past. And here’s our tone and style guide. Like, you just dump all of that directly into the prompt.

Jacob Bank:
That’s the easiest way to give your large language model context. Now, every large language model has what’s called a context window. You might have heard the term context window, and that basically describes how much text you can stick into the prompt before the large language model will get overloaded. And so context windows keep getting bigger. Different models have different context windows. But if you have a very complex task, at some point you will want to give the AI more information than you can put into the context window. So, for example, like, you cannot put the entire corpus of Wikipedia into a context window or like the whole web into a context window. And so when you have more context for the AI than you can fit into a single context window of your prompt, there’s another technique that you can use called retrieval augmented generation.

Jacob Bank:
You might have heard of this referred to as rag or you might have heard to refer this referred to as vector search or vector database. Those are all related concepts. But the way RAG works is instead of passing the whole knowledge base into the prompt, like instead of passing all of Wikipedia into the prompt, you tell the AI you have the ability to search Wikipedia as needed to achieve your goal. And so the way that works is that’s why it’s called retrieval augmented generation. You first, based on what you’re trying to do, search your knowledge base, like look at Wikipedia to find which articles are relevant and then you pass only those into the context window. So you can think of retrieval augmented generation as just like a pre processing step to get just the right elements out of the knowledge base before passing it into the the prompt. Is that, is that useful? Is that too technical of an explanation?

Josh Schachter [Host]:
No, that is, I mean we, we. And, and so the other thing is that RAG was, was all the buzz, you know, in my world, six to nine months ago, let’s say at the start of 2025. Now you don’t really hear as much about RAG anymore. It’s all mcp. So talk us through that migration.

Jacob Bank:
Yeah, I think RAG has become less common for two reasons. One is context windows have gotten much larger and so things that before you would have need to use a RAG based approach for, you can now just like gam it all into the prompt and it works just fine.

Josh Schachter [Host]:
Yep.

Jacob Bank:
And the second is in many cases instead of do using a RAG based approach, you can use the tool calling based approach to give the large language model access to up to date information. So like, like in the use case we were talking about where let’s say I want to do a follow up email after a customer call and I want to reference their data from Salesforce. Instead of passing in the whole Salesforce corpus as a knowledge base, what I can do is I can just give my AI agent a tool and say hey, when you need to write the email, just look up that particular contact in Salesforce on the fly. And then the AI, instead of needing the full RAG style knowledge base, can just directly call into Salesforce and find exactly the information it needs as part of doing its job. So that’s why the combination of long context windows and tool calling has addressed some of the use cases that you might have needed rag for like a year ago. There are still a bunch of good use cases for knowledge bases and rag. Um, but, but there, there, the need has definitely lessened as context windows have gotten bigger for, for many use cases. Now, you mentioned mcp and I’m glad you mentioned mcp.

Jacob Bank:
Let me, I think there’s a lot of confusion about what, what, what MCP is. And even though I’m working carefully, like I’m in this space, I do this all day, every day for a living. It took me like six months to actually understand what MCP is. So let me, let me, let me try to, let me try to explain it.

Josh Schachter [Host]:
Then we’re all toast if that’s the case.

Jacob Bank:
No, but I, I, I think I finally, I finally, I finally figured it out and then I think I have an easy way to explain it. So we talked about how tool calling enables large language models to take actions in the real world. And to call a tool, you need to give the large language model very specific instructions about how to use that tool. Like, here’s where the tool is, here’s what you need to pass into it, et cetera, et cetera. The analogy I’ll use for this is, let’s say, Josh, you’re the large language model and you’re a hungry person who wants to eat at a restaurant. And the tools of the large language model are all the restaurants in the pre MCP world. Every restaurant would have written down their address in a different way. They would have written down their menu in a different language.

Jacob Bank:
Some of them would have had the menus on the window, some of them would have the menus inside, some of them would have the menus on the board. Some of them would have had table service, some would have had counter service. Some would have taken one currency, some would have taken another currency. And for you who just wants to get a bite to eat, it would have been very confusing for you to figure out which restaurant to go to and how to get your food and pay for it.

Josh Schachter [Host]:
Are these, these other menu, these different menus? Are, is this different APIs?

Jacob Bank:
Exactly. These are traditional APIs. And they’re all saying, like, I can do these things for you in this way, I can do these things for you in this way. But it was very, very hard because they all expressed their capabilities in different ways. They explained their data formats in different ways. They expressed their addresses in different ways. And mcp. MCP is not a restaurant.

Jacob Bank:
MCP does not serve food. MCP is Not a diner. MCP is not getting food. MCP is a protocol. It’s called Model Context Protocol. It’s a protocol that makes it easier for LLMs to talk to tools. And so to use the restaurant analogy, the MCP protocol would say, hello, every restaurant. You must write your address in this format.

Jacob Bank:
So it goes on Google Maps. You must post your menu at the window in this way. You must, you know, seat people according to this protocol. You must give them the check according to this protocol. And so MCP is not actually a tool that you call or a specific servers that. That exists. It’s just a protocol that makes it way easier for tools to make themselves available for large language models. And so now that many tools have adopted the model Context protocol, your large language model knows exactly what’s available and how to get it.

Jacob Bank:
Does that make sense?

Josh Schachter [Host]:
Yeah, it’s kind of like an API for all APIs in some way.

Jacob Bank:
Yeah. And now this is where I’m probably quibbling a little, where it’s not an API in the sense that it’s not actually like a direct endpoint that you hit. It’s a specification document of, like, how to make a good API that’s easy to call.

Josh Schachter [Host]:
Okay, okay, makes sense. Makes sense. And then that gives your Agentix server. I’m just, I’m winging it here. You tell me when I’m, when I, when I sound off. That gives it access to that tool. And that tool has formatted the information that the agent needs in order to be able to use that information.

Jacob Bank:
Yeah, exactly. MCP just makes it easier for agents to call tools because all the tools announce what they can do in a similar way, take instructions in a similar way, and give back their outputs in a similar way.

Josh Schachter [Host]:
Okay, great. All right. I feel like we’ve laid the foundation here really nicely. Honestly, man, that was. That was a great explanation. I do, I know you do a lot of these YouTube videos, and so you’re. And you’re big on helping to educate the overall community. So thank you.

Josh Schachter [Host]:
Thank you for just kind of giving us that lesson here. Let’s talk about Relay App. I want to understand exactly what folks are using Relay App for. Not only to help, you know, you spread the word about your business, but also just, it’s an agent builder. And I want people to understand what these agent builders are all about. Who should be using them, can be using them. You know, how, why, what, when, where type of thing.

Jacob Bank:
Maybe the best way to start is like, Let me just give you a few examples. A few examples. That we use our product for and that our customers use our product for. And I’ll, you know, keep it, keep it optimized for this audience and customer success. Broadly speaking, most of my work falls into two categories. One category of work is reactive work I have to do based on something that happens. For example, after I meet a customer, I need to send a follow up. When a new support request comes in, I need to look up in the CRM to see how high priority they are.

Jacob Bank:
When a customer has a question about a piece of documentation, I need to update the documentation. So these are, you can think of these as like traditional, like trigger and action types of automations. Like when this thing has happened or is happening, I need to take this action as a result. And so in the customer success context, here are some of the agents you might want to build that take that shape. When I’m about to meet a customer, automatically write a briefing report for me of who they are, what their role is, what their company is, what the status of their account is, what their usage data is. When I finish the meeting with that customer, automatically send them a nice follow up that references all of the things that we mentioned. When I have a QBR coming up, automatically make a deck for me with the right format. When a new customer signs up automatically, you know, bucket them by industry or job function and then send them the appropriate email cadence.

Jacob Bank:
Those are a few examples of AI workflows and agents you can build that will assist you in common. Touch points of the customer journey that are usually triggered by like the customer doing something. The second class of workflows that are really valuable are those that happen repeatedly on a schedule. So for example, here’s some that we have every week, look through all of my customers recompute their customer health score. And for customers that have a health score below a certain amount, pick one of the four interventions that we use. Whether it’s like sending them an email or setting up a meeting or documentation or whatever it is, or every day look at all the support tickets that have come in in the last day and summarize them so we know kind of the general pulse of the health of our system. Or every week look at all of the customer conversations we’ve had and pull out the best quotes and the worst quotes and write a report for our product team. And so what I recommend everyone do is think about your work in those two categories.

Jacob Bank:
What are the things that you do or you wish you had time to do every day, every week, every month, that are analyses, sort of across the customer base as a whole, or across all of the support issues that have come in, or across all the social posts you want to listen for, or across all of the customer conversations you’ve had. And then also think about these specific customer triggered user journeys. Like when you have a meeting with a customer, when a customer emails you, when a new customer signs up, when they activate a new feature, when a new feature launch happens, and anything that can be articulated as when blank happens, I want to do some other set of actions that’s like a very good candidate for an AI agent to take on part of that work for you.

Josh Schachter [Host]:
Okay, that’s great. So what’s the biggest hurdle for people like to get started? Why? Why? You know what’s holding people back in your experience when you talk to folks about using Relay app or any of the other platforms to build their own.

Jacob Bank:
Agents, I think the biggest thing holding people back is to figure out a, what do they want help with at all? And then B, when is an agent appropriate to be the tool that helps you? And so let me try to give you a couple guidelines of when an agent is the right tool for the job and when it’s not the right tool for the job. An agent is the right tool for the job. If you want proactive, integrated and repeated help, and let me break that down, by proactive, I mean the AI agent will automatically wake up on its own and do work for you, whether you’re awake or asleep or on vacation or not. Number two, it’s integrated deeply into your tools, meaning it can read data from your CRM, it can update things, it can write emails, it can create calendar events. And then number three, which is probably the most important one is it happens repeatedly over time and you want the agent to get better and better as time goes on. And so if you told me, Josh, I want to create a one off strategy to make my podcast more useful to customer success leaders. That is not a good tool. That’s not a good job for an agent.

Jacob Bank:
Go to ChatGPT, do some deep research, do like a one off chat bot based interaction to answer that question. And so a lot of people come to me with use cases that are actually not a good fit for an agent because they’re one off. They don’t need to be proactive and they don’t need to be integrated. On the other hand, if you told me one of the time consuming things in my podcast is that every week I need to research the guests in advance and come up with specific questions that are good for your background. And now that’s something an agent can help with, right? Because it can proactively wake up before the podcast. It can go look in all your tools to find the right information. It can get better and better over time as it learns from your podcast recordings. So that’s one big problem we see people face, which is like, they know they want AI to help them with something, but they don’t know if an agent is the right tool for the job or if they should find a different tool.

Jacob Bank:
The other thing that people struggle with is because all this AI technology is so new and it’s moving so fast, and there’s so much hype and so many things that work better than expected or not as well as expected that it’s hard for people to imagine, like, what is AI even capable of? And what should I ask it to do? And for that I recommend using the same mental model you use if you were going to hire a person. So, like, every time I, you know, I’ve worked in corporate jobs before and, and every single manager that I worked with was always asking for more headcount. Always. 100% of the time where they were asking for more headcount, they would always say, ah, our team is so busy, we’re so swamped, we’re so overwhelmed, we need more people, we need more people, we need more people. And as a manager, what you’d always say is, I, yeah, I understand you’re overwhelmed. Please write down the job description of what this new person would do. And then the, the, the person who’s asking for more headcount would come back and say, here’s the job description. And then you’d have the following conversation.

Jacob Bank:
A, can we reshuffle some of our existing priorities to do that thing better? Or B, should we hire someone new? And now you have an option C. The option C is could we have an AI agent do that work for you? And so what I would recommend is think about if you magically had the budget to hire another person or an intern or a contractor or an agency, what would you have them do? And then you break down those use cases into things that can’t possibly be done by AI, like take clients out to lunch, like, you’re not going to have any chatbot or agent doing that. And then of the things that could possibly be, you know, knowledge work that’s done by an AI assistant, which of them are one off chatbot kind of use cases that you just want to go to ChatGPT and do some deep research on and which are agents that you want to set up to work repeatedly on your behalf.

Josh Schachter [Host]:
Yeah, I love that framework. That’s, that’s very compelling. You mentioned up front that a lot of your business is selling to SMBs and mid market companies. What are some of the distinctions in those groups versus more enterprise level organizations and how they’re approaching AI and agentic?

Jacob Bank:
The reason that we sell mostly to small and medium businesses is, is not because of some, I think fundamental trait of our product or the fact that enterprises don’t need AI agents. It’s more just that we haven’t yet built a go to market that can scale to large enterprises because we’re totally self serve PLG at the moment. We don’t even really have a sales assisted, assisted motion. Now a couple of the practical things that I’ve seen come up that make SMBs faster at adopting these tools. Number one, SMBs typically have more control over their data and what tools they can use. So in large enterprises you typically need to go through like a very long vendor security review and approval process and work with procurement and it. Whereas you know, small businesses can immediately, immediately adopt a tool. The second reason which I think might be more relevant to the previous point that’s not generic to software but specific to this AI agent wave is I think in large corporations people have been so trained to always view hiring more people as the answer that it can be really tough to shake their mental model out of that.

Jacob Bank:
Whereas small and medium businesses are typically so budget constrained that they’ve never had the ability to hire people. And so they’ve always thought in sort of like scrappier ways about how they can either, you know, cut corners or creatively get work done. And we’ve seen that that instinct has made them very fast at adopting AI agents because they’re always on the lookout for new tools that can, that can help them, help them get stuff done. And they’ve never thought of hiring their way out of a problem as, as the way to solve it because they’ve just never had the budget to be able to do so. And so yeah, I’m trying to think about what that means for, for, for customer success professionals working with these different audiences. I think with like if you’re a customer, if you’re a company that sells to small and mid market, I think you have an opportunity to give them a very, very fast path to value and then they can expand their usage from there. If you’re selling to enterprises, you’re probably going to need to put a lot more effort into Reframing the way the future of companies is going to look like. Companies are just going to look different, look different in the future.

Jacob Bank:
They’re going to think about headcount differently and the enterprises that can adapt quickly to that environment are going to succeed and the enterprises that are trapped in the old way of thinking I think are going to, are going to really struggle. But it’s tough. Right, because in an enterprise setting there’s often such a deeper entrenchment of scopes and hierarchies and my area of ownership and my territory and I need these 50 people because that’s how, you know, my status in the organization is, is created. And I, oof. I haven’t touched any of those problems yet at Relay App, but I, I can imagine they’re, they’re sticky.

Josh Schachter [Host]:
Yeah, yeah. I mean it’s. At gainsight, it’s something that we’re, we’re obviously tackling and talking to a lot of larger custom customers and enterprises. It’s fun stuff.

Jacob Bank:
Yeah, yeah.

Josh Schachter [Host]:
There’s lots of potential, but there’s some love to go through as well.

Jacob Bank:
Yeah. And I’m, I’m a huge optimist. Like I really believe that we have this amazing new technology that can do a huge amount of work on our behalves and therefore can enable us to spend our time on just the things where we add unique value and we’re creative and they, they’re often the things that we enjoy most. But I think anyone who is trying to minimize the disruption that is likely to happen, I think they’re doing us all a disservice. This is good. Like companies with knowledge workers are going to be totally different. Totally, totally different five years ago from, from what they are now. And I don’t think the right answer is sticking our head in the sand and pretending that’s not going to happen.

Jacob Bank:
It’s like adapting as quickly as possible to make that a success for us and our employees.

Josh Schachter [Host]:
Yeah. Yeah. Well, so sticking with the theme of the future, what will agentic, what will agents look like in a couple of years? Do you have any sense of what’s on the frontier?

Jacob Bank:
I think right now agents are limited in two ways. Number one, people don’t trust them and number two, they can’t do everything they need to do. Let me talk about each of those in sequence. An autonomous system that’s working for you while you sleep, that can update your CRM and send emails on your behalf is pretty cool, but pretty scary scary, right? Like that’s, that’s seeding a Lot of control for very high stakes actions to someone that’s not you. And so we’ve put a lot of work in our product into helping people build trust in their AA agents in two ways. The first way is to make your agent more of a workflow. So there’s a spectrum between agent and workflow of like how much autonomy you give the agent to make decisions versus how much you give it, you know, strict instructions. And what I always recommend is like if you know what your sales qualification process is and it’s the same six steps every time, just tell the agent via the, via workflow to do that stuff.

Jacob Bank:
Exactly. Don’t make it reinvent the wheel every single time it’s qualifying customer. So that’s one way you could build trust. And the second way you can build trust is add a human in the loop for any high stakes action. You make sure a human approves it. And so I think as we get better at giving our agents instructions in the form of flowcharts and workflows and we get better at working with them in a human loop way, trust will increase. The other challenge which we talked about briefly when we were talking about MCP is that agents just can’t do everything yet not every software tool has an API. Not every API has all the capabilities.

Jacob Bank:
And so like for example, we work with a bunch of real estate firms and real estate’s, they often have these like specific proprietary data sets and data brokers and none of them have APIs. So you have to like go to a website and log in and you know, search to the right page. And agents are not yet good at doing those sort of web based interactions that humans are good at. I think that’s going to change like in the next year. Agents are going to get really good at that. And so my prediction is that pretty much any task that you do on your computer as part of your work, an agent will be good at doing within a year. And then the question will just be how do you teach your agent? How do you keep an eye on your agent? How do you correct your agent? How do you have that kind of that trust in the form of the flowchart and the human loop to make sure it’s doing the right stuff and not the wrong stuff.

Josh Schachter [Host]:
Yeah, yeah. And you’re already seeing parts of this. OpenAI recently released the Atlas web browser. So clearly they’re going to be implementing agentic skills to help you, you know, do things that eliminate what you’re doing in your browser. Shopping for, for items and things like that. So it’s already getting there. And then I think you’re right. I think part of thinking about the future of agents is thinking about how we’re going to manage agents.

Josh Schachter [Host]:
The human in the loop model, and thinking about agents as. I mean, in the professional world, we’re probably gonna end up allocating a fair amount of time to. Instead of doing the work itself, you know, working on calibrating the agent.

Jacob Bank:
That’s exactly right. We’re all. We’re all managers now. Even if you’re an ic, you’re a manager now because your job is to give your agent clear instructions to work with your agent over time, to teach it the skills it needs to make sure it’s doing a good job, giving it feedback. Like, that’s now like the most important part of everyone’s job. It’s not yet today, but I think in the next year or two, it will be.

Josh Schachter [Host]:
Yeah. Oh, wow. Sounds so fun sitting there tinkering with machines.

Jacob Bank:
Honestly, I love it. I love it because it’s like I get to think strategically about what should happen and what the opportunity is, and then it does all the legwork for me. It’s amazing.

Josh Schachter [Host]:
Awesome. All right, let’s leave on this note. What’s a good place or resource for people to learn about the latest news as it relates to AI and agentic or said otherwise? Where do you go to get your information on what’s going on in the world of AI and agentic?

Jacob Bank:
There are some pretty good curators of resources and communicators about AI. So, like, I follow the Ben’s Bites newsletter. There’s a couple of YouTube channels I follow. This guy, Jeff Su, posts really good videos when new, when new features come out. So in general, I recommend following like, a couple of the good people who are on the pulse about AI to get an overall sense of what’s happening. But there is no substitute for trying things out for yourself. Like, this is the thing I really want to reiterate, which is in this world where things are moving so fast and there’s so much hype and all of these tools have a free tier. Like if someone posts something on LinkedIn or Twitter, that’s like nano banana made all ad creative irrelevant.

Jacob Bank:
Just, like spend five minutes playing with Nano Banana to come up with your own opinion about that. And so I follow the leading voices just to know which models are coming out. But I make sure to allocate like an hour every single day to just, like play with AI tools. And it sounds crazy. It sounds like a lot but it’s, it’s been worth its weight in gold because if, if, if, if every week I can find one new use cases for AI or one new tool that’s going to save me hours every week going forward, like it’s totally worth it.

Josh Schachter [Host]:
And I spent a couple hours tinkering with Relay, that app last Sunday, last week on, you know, the workflow that you had helped me to build.

Jacob Bank:
And that’s why, and that was such a valuable experience, right, because you knew in the abstract what you wanted to do and you thought it was possible with AI, but only when you really dive in do you see all the nuances of like, well, this is the data LinkedIn lets us access and it doesn’t let other data. And here’s how to actually stick the stuff into your Google sheet in a format that the AI can read it. Like, you really hurt you, you, you see what all the practical hurdles are beyond like the 1 sentence Twitter headline.

Josh Schachter [Host]:
And then this past week, after having done that, it really did help me with my job. Now it’s a little bit unique because my job is thought leadership around AI and objective. And so. Right. So I, so it’s behooving on me to beholden on me to do this. But, but it was, it was great. I mean like without that firsthand experience, the conversations that I was in would not have had the same level of depth. So, so anyways, well, thank you for taking that time with me last week to walk me through it.

Josh Schachter [Host]:
It was a lot of fun and.

Jacob Bank:
I, and I love it because that’s, and so, so for everyone, we should all be practitioners of AI. But also, if you are building an AI tool, there is no substitute to actually sitting down and watching the customer use it. Now this was always important. User research was always important. Seeing how users use the tool was always important. But it’s even more important now because AI is non deterministic. It’s intelligent, like it’s not the same, but doing the same exact thing. It’s going to be so specific to that customer’s environment and their setup and their business goals.

Jacob Bank:
And so for me it’s just like a really fun new opportunity to learn the ins and outs of lots of other people’s works and then see where AI can be useful. So for all the customer success pros out there, like, it’s not as simple as just seeing like, oh, they used the feature four times in the last week, they’ve adopted the new feature. Like you have to go one level deeper to understand with what data? Were they using it? What goal were they trying to achieve with it? Did it actually work for them? Like, are they. Is it getting better for them over time? Like, AI products are more powerful than traditional productivity tools, but they also create, like, much more complexity and richness that you need to understand.

Josh Schachter [Host]:
Jacob bank, CEO, Founder of Relay App. We’re going to leave it there. Thank you so much for being on the program. I hope we can have you back soon. And I’m going to keep tinkering with the product.

Jacob Bank:
Awesome. Thanks, Josh. This was great.

Josh Schachter [Host]:
Have a great day.