Episode 109 - Pat Barry

Erik Martinez: When most people hear AI agents, they jump straight to the autonomous robot that runs your business while you sleep. I get it. I watch the Terminator too, but that's not where any of us are. Pat Berry and I sat down to talk about what an agent actually is, where you should start, and why the move from one custom GPT to a real Agentic workflow is less about the tech and more about how you organize the work.

Pat, welcome back to the show.

Pat Barry: Thank you.

Erik Martinez: Uh, Pat's back on, for kind of our regular segment to talk about something in AI that we find, interesting that people are talking [00:01:00] about, wanna learn a little bit more about, and that topic is agents. And no, I'm not talking about the guys who signed the multimillion dollar contracts for people like Patrick Mahomes or who's that quarterback at Chicago? I can't remember that guy's name.

Pat Barry: Oh, come on. You know who Caleb is. Everybody does .

Erik Martinez: Caleb Williams, Mr. Last minute win every game

So, Pat, here's the first question. When business owners hear the term agents, what do they usually get wrong and how do you define an agent in the real world?

What's your take on that?

Pat Barry: I've heard so many different interpretations at this point.I have a client, we started and, an agent to them was, full on, AI robot.

Erik Martinez: One of the Tesla robots that they're making right now?

Pat Barry: I was like, "Yeah, I mean, that is.

I'm like, We don't need to go quite that far. I don't have that, good of engineering skills, but thank you."

I think a lot of them just, you know, are confused. I don't think there's one, solid definition. So, it's,me, it's, you know, ChatGPT, just even the free version, sitting there, typing back and forth, that's an agent.

I mean, [00:02:00] anything that's, an LLM platform that you're typing information into and getting some sort of output from, whether that's my nerdy data science stuff and attribution modeling or, you know, using it to find the best deal on a, I don't know, on a bike or something. I don't know.

I was gonna pivot to you, because I know you've heard some interesting definitions of agents as well. I guess in your realm and your day-to-day, how do you define an agent? What's the craziest definition of an agent you've ever heard? That's what I'd like to hear.

Erik Martinez: Well, I don't know if there's a crazy definition because I can see where people are leaping to, the full-on autonomous robot who goes and makes decisions and then runs around the world and terrorizes it. I watched the Terminator as a kid too. Absolutely loved it. Schwarzenegger was great, but, joking aside, those things are starting to get built, but they're gonna be fairly simple at first, right?

We're seeing examples of these things, in the real world right now. I think the simplest form of an agent is just something that you can use to do something for you. [00:03:00] So that might be just a simple task. And I used the example with, the training group that you and I were talking to yesterday, just a custom GPT is a form of agent.

It is not autonomous. It needs your input, but once you kinda give it the instructions "Hey, go write me, some questions for the podcast. It goes and does that for you. You're not necessarily doing it. Now you may interact with it and stuff, but that's a form of agent.

And sometimes, I'm like, "These are good questions. I don't even edit them. Because I've trained it. So I think there's that. And then there's, a semi-automated agent, something that would trigger, like, off of Zapier.

Pat Barry: I got it. Yeah.

Erik Martinez: So the example is, you know, we'll record this call. Um, the grain recorder that's in here, when it's done, there's a trigger in Zapier that picks up the transcript and moves it over to Google Drive for me.

Pat Barry: Oh, nice.

Erik Martinez: Yeah. Super, simple, but super helpful for me to get that in a spot where it's searchable and easy to use. As opposed to me having to go in the grain and copy and paste everything out.

Pat Barry: Exactly.

Erik Martinez: I [00:04:00] think there's that example. And then, I think the next step is kind of a streamlined pipeline type thing.

So, think of a series of agents that do very specific tasks and hand something off to, an assembly line. For me, I've got this, system that I created using Claude that takes some input from me. The first agent takes input from me, turns it into a brief. That brief goes to a writing agent. The writing agent sends it off to an evaluation agent. The evaluation agent sends it back to the writing agent, who sends it back to the evaluation agent. That sends it to me for review. And it's all sequenced, right? And there's nothing really complicated about it other than it's actually passing data and I never see it until the whole process is done. I do the intake and I don't see it till it's done. Which is fantastic, and then I can do my editing, and sometimes it takes a little more editing than I want. Then there's, what I would say is the multi-agent, multi-system type thing.

So, envision that same pipeline, but now it's talking to Google Drive and a SharePoint [00:05:00] and, a government website to pull Bureau of Labor Statistics data or whatever it is, right? It's interacting with all those different systems, pulling all the information together and outputting something. Without you having to touch it.

So I think, if you're moving along the curve, you're gonna start with something really simple. Start automating some of those little simple things. Then build some pipelines, and then you're gonna build a multi-agent, multi-system pipeline that does all sorts of crazy things.

And then the final version is kind of that autonomous agent that sits out there and, it gets a set of instructions and it just goes and does a whole bunch of things.

Pat Barry: Yeah.

Erik Martinez: Every day, all on its own, and produces work. I know some of those things do exist, but I don't know that most of us are anywhere near that.

We're probably closer to step one and two than we are to three, four, and there's very few people at five. I think Claude Code and Chat GPT's Codex are kind of in that five range now, right? It's writing a lot of its own code, and it's [00:06:00] reversibly writing that code, which is kind of cool and scary all at the same time

So would you agree with those or do you think I'm full of crap?

Pat Barry: No, I would agree.

Erik Martinez: Oh, I was hoping you'd say I was full of crap so we could fight.

Pat Barry: Nah, I tend to agree with you, man. I think you covered a lot there. So, yeah.

Like, we'll see how things evolve. Especially with, the news around Claude Mythos, and things like that. I know ChatGPT came out with something today.

They're allowing, I don't know, anybody into their, cyber attack cohort, whatever, to test their model. That's all complicated stuff. I think it's still, you know, the simplicity of it, like, the single, tiny task, we talked about before we started.

It's still kind of like, one man shop. I have some automations, but nothing too crazy. I'm always thinking about, what's that one thing I could build for myself this month. Do you think about that type of stuff? You know, we're halfway through April, spring is coming.

I guess I have more automations for, my family type stuff, like calendar stuff with kids and sports and all that. I'm trying to think of, automations I need for work. They're all kinda done.There's no event coming up in, May or June that I'm like, oh my God, like, I'm [00:07:00] gonna have to automate something crazy.

But, I guess, do you sit around and think about stuff like that? Like, what's an automation that you could build this month?

Erik Martinez: I think about these things all day, every day and go, "Where am I gonna find the time? Which is why I've been working really hard with Claude Code and Claude Cowork, to do that. I'm actually very excited about testing, Chat GPT's Codex to do some of this stuff too. But, I think, if I were to pick a simple agent that would be useful at home. The one that keeps coming to mind is Hey, go through my bank statements and my credit card statements and help me reconcile that stuff.

Tell me where I'm spending. Tell me when something spikes and, you know, your bank to a certain extent provides some of that data, but it only provides it from its own context. It doesn't have the credit card statements the mortgage account statements your car payment statements.

Could, you bring all that together and have a financial picture of where everything's going? Is it reconciling? What's new and unique this month?" And help people budget a little bit [00:08:00] better for their own personal finances. I think about that one all the time because I've, just had a conversation with my youngest daughter.

It was a little less of a conversation, more of be growling at her, but, we were talking about, electric bills. I live out in a country and I have a house that's 100% electric.

Pat Barry: Oh, really?

Erik Martinez: Yeah, so there's no natural gas, right? Obviously we have water, but water doesn't run on our electricity, right? It's pumped in from the county.

Pat Barry: Yeah.

Erik Martinez: They're running the air conditioner. It was like 68 degrees outside and they're running the air conditioner at 68 degrees. And I'm like, " Why are you doing that? " And somebody in the house was complaining that they're cold and I'm like, " Well, that's because you've got the AC on at 68 degrees when it's 68 degrees And she got mad at me and she's like,that's awesome." I'm like, "Yeah, unless you're the one paying the electric bill. It could be a really great tool for people to be able to say, "Hey, you know what? I can grab all my personal financial information, dump it in system, do an analysis once a month, every quarter, have a picture of how we're actually spending.

Hey, you know what? We wanna go take that [00:09:00] vacation to Disney. How much do I really need to save in order to do that at our current spending rates and where our discretionary income's going from?" I think about that stuff, yeah, because I think it applies to all of us, right?

But you gotta take that same concept and now apply it to your business and do the same thing.

Yeah, that's what you have your financial people for, but if you're a small or medium sized business where you don't have in- house accounting people and you're just relying on your external CPA or your external tax accountant. Hey, this is something you can do. Grab all your statements. Grab your credit card statements and go, Hey, tell me what am I doing with all those subscriptions?

You and I have talked about this before.

Pat Barry: Yeah.

Erik Martinez: Why am I spending X amount of thousands of dollars on subscriptions? Are we really using those things?" And just little ways to use, existing AI tools to do some evaluation. So I think that's not a horribly complicated one. That's probably a little more complicated than most people wanna start with.

Start with just one bank statement, just do an analysis for 12 months [00:10:00] and see what you find. I think that would be a nice, simple, agentic use case, build a custom GPT and boom, pop that in there every single month and then have it update.

Pat Barry: Yeah, man. I like that idea. That might be a good use case for me. Right now thinking more about, some sort of a meal prep thing, that I can help my wife with while we're steeped at soccer, baseball, and softball season with three kids.

Erik Martinez: Oh yeah, for sure.

Pat Barry: More around the house type of stuff.

Erik Martinez: Yeah, here's the things we like to eat. Here's the things we don't like to eat. Here's the things we need to eat.

Pat Barry: All those things. Like, Each child needs a separate list because the things they like to eat are very different. I think it's more just timing of everything. Today is our worst day because my oldest will have soccer practice at 6:00 PM.

My middle will have tutoring at 6:00 PM. I have to go coach baseball at 6:00 PM with my son. So it's like all three kids are going three different places at once. They gotta eat because I have to eat too. I'm starving by the time we get home. So, even just finding meals, asking AI, all right, we can't eat out.

I mean, I do all right, but even that too, it's just [00:11:00] unhealthy.

Erik Martinez: But, that's a great use case. Like, hey, we've got a really complicated family schedule. I've got three kids. They're all in activities. My wife's got her job. I've got my job. What do we eat this week? Plan it out so you go to the grocery store, get it all figured out in advance. that's a fantastic use case.

Pat Barry: It's healthy.

Erik Martinez: Yeah, for sure.

So, you know,I was just talking about this pipeline concept and we were talking about chaining agents and, I'm kind of curious of what you think. At what point does a helpful tool become a real Agentic workflow? And what makes that jump worth it?

Where do you go from the one-off custom GPT to something that you need to automate in some fashion, whether that's using, Zapier or Make or any of the Google tools or now Copilot's coming out with some cool stuff? What would you do?

Pat Barry: I think a lot of it depends on what you wanna do, but, real life example, I've got client I'm working with in Copilot. They're in the financial sector and they do financial marketing for their clients. Some of [00:12:00] it, it's just, this is what they're creating for these big financial people.

We've had to sit with their lawyers and their concern is, like, "Hey, if we're creating marketing materials for these big banks and financial institutions, we kinda gotta make sure that,it's not, poached from something else that we did, you know, five years ago, two years ago, whatever it is.

And I'm like, "Oh, that makes sense." And so we started to talk about, creating an agent to help them do their jobs and just, whether it's, this piece of marketing for this financial thing or whatever. But then we had to think through, "Well, can this thing also, check the work?"

And I was like,it can, but it's got quite a bit of responsibility now having to search through all your client folders. You know,we have a couple different style guide knowledge documents." I'm like, "There's a lot in here." There's no way I can test this.

We're just using, the real basic, Copilot for business. This is not the fancy, Copilot Studio where you can actually, do a lot more. So, it was more of I'm looking at, you know, all right, what are the megabytes and kilobytes of the knowledge docs I have attached to this thing? And then, you know, you've got with Microsoft, I think if I remember right, it's 8,000 characters for [00:13:00] your system prompt.

So, that's quite a bit. I mean, that's a pretty long system prompt. I think our system prompt was, like, approaching about 7,000 characters, almost 8,000. I was like, you know, the less we give it, that it tends to produce better results. And we're relying on knowledge docs.

And it got to the point, I'm like, how big of a deal is it if I created just a QA agent so that when you're done, whatever draft you're on, you know, let it create the first draft, touch it up a little bit, 20, 30 minutes, whatever, but feed it into this QA agent. We'll name these things. You know, we'll give it fun names so they kind of treat it like an employee and get used to, just that type of workflow.

But that's what we ended up deciding on. And it was really more to me of the first agent or generation agent that's generating the materials. It's just got too much. I picture it like an Amnesiac intern, you know. That's how I look at AI, it forgets everything, I tell it.

And it's good, I can create an agent for it that, okay, it remembers the system instructions, but I still have to tell it certain things. There's only so much to think and process. Now, again, this is the out of the box, model that you get for the 30 bucks a month. It's pretty good deal.

[00:14:00] Now, if I went and built my own. Yes, of course, we could make it powerful as heck. But I think you just start to get to that point where you're, like, what is the task that I've set this thing up for? What's the complexity of it? Okay, we cap it here. This next task has to be done by another agent.

I think it's more just common sense. As the tools get better, I do think they will have better measurement for, like, " Hey, what you're asking this agent to do and what you set up is just too much. You need a second agent to do some of this. " So again, as technology improves, that's kind of how I think about it.

But I know you and I too, have different sets of tasks because we have very different clients. You just built out, a three, four, five stepper, agentic workflow system.

At what points were you sitting there going, like, all right, this is too much?

Erik Martinez: It'san eight stepper and it has at least eight more to go before it's fully baked.

Because I'm turning it into a system. But, honestly, the main reason I started the segment was, before I switched a good chunk of my workflow over to Claude, I was working in ChatGPT, and when I downgraded from the pro version back to the regular [00:15:00] business version, I started hitting limits again, which was the reason I went to the pro version in the first place.

Pat Barry: Yeah.

Erik Martinez: And What I learned in that experience was, " Hey, you know, the longer my prompt got in, let's say I was creating a custom GPT, and I've seen this in, Gemini's too. The closer I got to that 8,000 character context window, the slower it got, and the more likely I was going to run into a tool calling exception. So, I figured out pretty quickly that, oh, you know what?

The way to get around this is to split this into two separate agents that just talk to each other."At first I was doing that manually. I output the file. Copy and paste the file over to the new agent and have the agent do its thing.

So my thinking has evolved on this though because I started thinking about it in terms of how we set up org charts, at the macro level . And, I've been doing EOS, with an implementer and we've been talking about, different roles and different seats and the roles that go along with that.

And what I realized is if [00:16:00] you're gonna build an Agentic team that has responsibilities. These things are smart, but they're not as smart as you think they are. And what I mean by that is, on any given test, they are way smarter than I am. We do a math test and it's gonna whip my butt every single day, right?

It probably would whip my butt in, marketing, and I've been doing direct marketing for 30 years. But it doesn't have good long memory context.

Pat Barry: Right.That's the biggest thing.

Erik Martinez: So one of the things I like about Claude is that it's pretty upfront about what you're using, and when you hit your limits, it tells you. Where ChatGPT and Gemini do not necessarily.

Now, Gemini seems to give you a fair amount for, the dollar you're paying. You get a lot, and that's probably because Google's, ridiculously profitable.

Pat Barry: But, if you sit there and go, "Hey, when am I gonna run into that tool calling exception because I hit a limit?" ChatGPT doesn't tell you any of that.

No, it doesn't.

Erik Martinez: So,you start thinking about it and [00:17:00] saying, "Okay, well, if I have a job, and let's say that job has a task, and let's say that task has 10 steps, but three of those steps are kind of discovery, three of those steps are kind of assembling information, and three of those steps are output."

I think that's three different agents. Highly specialized and coordinated, and that might be a direct pipeline, depending on what it is you're doing, but that way, you can almost guarantee that your prompt, the role you're giving that agent is incredibly specific, and you're getting the best from that agent.

It's the same thing with us. The reality is we all have some capacity for what we can actually manage in our brains at any one given time.

And so even when we're multitasking, we're really focused on one small thing for a little bit of time, then we're switching our attention and context to something else, and then we go back, we do this very rapidly. Well, the agent has to do the same thing, and it's more likely to get confused than we are.

Pat Barry: Yeah.

Erik Martinez: So,that's a really long answer to the question, but if you say, "Hey, if I can break my [00:18:00] tasks down into phases, each phase probably is an agent.

Pat Barry: That's a good way to think about it.

Erik Martinez: That's just one way of thinking about it. It doesn't mean that I'm right. it's one way of maybe trying to get the best out of it.

Pat Barry: Yeah, man. Good stuff.

Erik Martinez: Okay, so we just talked about putting agents on the org chart. I knowthe last time we talked about this, you thought there was some value in this. How do you think companies are gonna react? Or how are people gonna react when you start naming teammates who are not human, they're synthetic?

Pat Barry: I mean, it's gonna depend on, the maturity of AI use within that particular company. It just depends. Everybody's different. Probably not well at first. Most people I know don't like change. At least in my career, I've been in that tech space for so long, like, private equity and venture capital, that's just a normal thing to me.

Overall, just depends on how quickly the change is introduced, I would say. And then having people build trust with that particular AI agent. You know, when I'm conversing, I give it the level of detail I would give to somebody working for me. Now, granted, it's a little bit different with data. It's pretty black and [00:19:00] white what we need to do.

I don't know, do you have a welcome party for the new agent? let's all talk to this and we'll turn our camera on and make sure it has OCR and congratulate it for joining the team.

I think it's hard. You wanna treat it like a regular coworker, but regular coworker is a human. You're not gonna sit down to lunch with the AI. It can't eat food.

Erik Martinez: Do you know that for sure, Pat?

Pat Barry: Well, maybe some of those robots, they got a Tesla. I mean, they gotta eat something.

Erik Martinez: So,I don't think a lot of companies are to that point yet where it's like, all right, I sit down and start our day just with this stuff.

Pat Barry: Even my companies I work with, that's why I'm there. They don't work like this. They've got people inside, you can pick them out, like, "Oh my God, I love this stuff. Look how quick my day is going." But to the level where it's sit down and we have an AI project manager. Go tell it what it needs to do, it's gonna do these things automatically for us, update schedules, send email, whatever.

I don't think that'll start to be a thing until maybe, like, a year from now. Or at least adopted, but it depends.

Erik Martinez: I think that adoption will take a little bit longer than that. I was talking with a guy on episode 106. His name is Joe [00:20:00] Newberry, and he's the founder of, Exec Clone. And basically, he's taken OpenClaw,

Mac Minis, and he's packaged a system, and he's got an intake process that's very detailed.

And he's basically creating the Pat and Erik clone. With the idea that, that is your COO.

So if you're the visionary in your group,here's your COO who's, like, going to hand out work assignments and, hold you to task for getting your crap done. You know, stuff like that. Now, I haven't played with it yet. We're working on getting that set up for me. But, I'm actually kind of excited about it because I'm sitting here going, you know, there's so many things in my day between emails coming in and text messages coming in and IMs coming in from all different sources and lots of different channels, like, keeping it all straight.

Well, if I had something like that, he's, paying attention to it and at least organize it for me and say, "Hey, you know what? You got these fifteen things to respond to. These are the three most important. Give me the answers to the rest and I'll take care of it." Oh, yeah. Sign me up. I'm in. But,[00:21:00] will it work that efficiently?

I don't know. I think it will. I think we're starting to see, when you train some of these agents that they're capable of doing a lot more than you think, but right now, it's so foreign to us to work that way.

Pat Barry: Yeah.

Erik Martinez: You know,you've turned me onto this podcast that I really enjoy. And Jordan says it all the times.You should be thinking about changing the way you work. So he doesn't like to use the term human in the loop, he likes to call it expert driven loops. Because it's a different way of thinking about the work. It's like, " Hey, I'm a programmer, but I no longer write any code. " The agents are writing the code. I'm the one orchestrating the activity.

Pat Barry: Everybody's a manager now.

Erik Martinez: And that right there is a fundamental shift in how we all work. We aren't necessarily built to think that way. So, I think that's, you know, kind of the thing I'm looking at.

You know, question for you. If a leader in a business wants to use AI agents today. Knowing what we know today. Without creating chaos in their [00:22:00] organization. What type of guardrails would you suggest putting in place to make sure that, they get speed and quality and trust, but also defend against some of the crazy things that could happen?

'Cause, you know, I think AI just accelerates whichever mess you've decided to pour yourself into.

Pat Barry: Yeah. I'm working with a client right now where, we have an agent that just, that sits in Copilot and we're gonna cut it off from the web. Because it just doesn't need web. It's there to search SharePoint for materialsthat my client has created for previous clients.

Some of it's just having to do with a lot of the, paperwork they get put in front of them and what they have to agree to with their use cases with AI, and what their clients demand of them. And so that was something I suggested. I'm like, "Listen, you know, I can build these really easy in Microsoft.

It's just a button I pushed just to shut it off from the web." I think that's like a good guardrail. You don't necessarily need it searching the web. I need to search all of SharePoint for these types of things. I think that was, kind of a guardrail I start to think about. It's such an obvious one. Another thing we're doing is giving it Access to just certain folders, within [00:23:00] SharePoint.

That's another good guardrail. We know for a fact, it doesn't need to be in any of the contract stuff. It just doesn't. And that's real sensitive type of stuff too. So, I think just segmenting off access to knowledge points. Those are probably the biggest things that I think about.

At the same time too, AI use policy, like any company should have one. That should just be a normal, thing at this point for most companies. Your HR department should definitely have one. Let the company tell you what you can do and what you can. So, you can't necessarily give that to an agent, but I guess those are types of guardrails I think about.

In terms of, who can use it, as a guardrail, I think about stuff like that too.

Erik Martinez: Yeah, I'm curious, I know you work with some, larger organizations that have strict security policies. So, in those scenarios where you're, segmenting off an agent or AI from a specific thing. Are you giving it its own set of access credentials? Is that's what happening at the IT level?

Pat Barry: No, at the IT level with, my larger clients, it's just cut off completely. So for example, I have one client, we use Snowflake, to store a mix. It's [00:24:00] hardcore PII data. As well as their marketing data. And, you can connect it to Claude directly.

Well, me and my client, she's really into AI too, we're just dying. But the IT policy is no. I completely understand. The guy's awesome. He's like, I can't do it. I'm sorry. If I turn it on for you two, I gotta turn it on for everybody."

I'm like, "Yeah, I get it. " He's like, "You know darn well, what would happen." And I'm like, "I know. " People would be asking questions without any context. They'd be drawing conclusions from data that's potentially not transformed. This is a data warehouse. Some of it's raw. So if they don't know where to go or what to ask or, what databases and schema to look at. It's just gonna create internal chaos.

Erik Martinez: It starts getting me thinking about some of these systems. Because I've worked on systems that were ancient, and everything's stored in clear text.

Some of these more modern systems where things are hashed, but you're not necessarily segmenting, the really key parts of the PII from the things that could be really useful for analysis.

And quite frankly, I think with today's tools, some of the things we've talked about in terms of being able to access SharePoint and Copilot. [00:25:00] Or, we were talking about how I have some stuff in SharePoint that I'm syncing over to Google Drive. Like, Google Drive and SharePoint in these cases are their own databases to a certain extent, right?

Folder structure and all that stuff. So, it is kind of interesting to think,there really have to be some good guardrails set up for this. And we're just at the beginning of this whole process of, how do we truly segment that data and protect it in the most effective fashion?

Kinda freaky, not to put everybody on a downer, but, you know, these are real things that we should all be thinking about a little bit.

Pat Barry: Yep.

Erik Martinez: Oh man, do you have any last thoughts for the, listening audience?

Pat Barry: No, it's good timing. I gotta go grab the kids from school. So, this is perfect.

Erik Martinez: All right, man. Hey, thanks again for coming on, Pat. Always a fun conversation.

Pat Barry: Always fun, man. Thanks for having me on. Good luck to all the people that are building agents right now. It's a wave of the future.

Erik Martinez: It's the wave of the future.

Here's what I'd leave you with. Pat and I both ended up in the same place. Most of us are at step one or step two on the [00:26:00] agent curve, and that's fine. It's a custom GPT or one zappier trigger takes a real task off your plate. Pick the one, build it, let the next one show up when it's ready.

We'll see you next time on the Digital Velocity Podcast. Thanks for listening and have a great day.

Narrator: