Welcome to the big cheese podcast. We've been talking for 20 minutes and we're gonna keep this rolling Brandon's Just totally thrown off because we're in our new studio. We're in our new studio and uh, what you Probably looks good from this perspective. Yeah, but from our perspective It looks like a bomb has went off on everything on this side Dylan just set up the cameras clapped four times and said i'm going to my office if it stops recording Too bad. It just left. I see some knobs and some buttons and some lights. It's got you. It's got you. I think we're working. So, why don't you introduce? So why don't you introduce? I will. So our audience to our guests. Yes. So we actually have a new guest. Well, first of all, you'll notice that Sean or Jacob is not here. Jacob had a spa day today and he's getting a Brazilian wax and his toes, something with his toes. There's something with his toes and his nails.

• Yeah, so he's gonna come back looking like a million bucks. Today we're joined by Max. Max and I have actually worked together at an insurance client, was the first time that we worked together probably about five or six years ago now.
• Now, yeah.
• And we were doing some AI early on AI work with them.
• Yeah.
• And now you have actually gone on and you've done, you set up your own agency and then I think you're also now with a new company which we haven't actually talked about yet. Why don't you just tell us a little bit about what you're doing, Max?
• Yeah, well hello everybody, my name is Max Wilkungfoe. I am-- - Sorry, real quick, I didn't want to butcher his last name. I literally tried like a button.
• Then don't try it.
• Then I won't try it, but again, the most wonderful last name that my white American ass struggles with.
• Yeah, I mean it rolls off of the tongue once you get it, but it's a struggle. It took me into like third or second grade to really get it out and like write it.
• And really?
• Yeah, I mean to really like get it and then be able to teach other people how to say it, right, 'cause that's the whole thing.
• Yeah, so like I have been super interested in machine learning since like the OG TensorFlow days. I remember when Google first released it and I was like, "Yo, what is this?" And the only reason that I was super interested in it was because I studied psychology in university at Texas Tech University. And I had the pleasure of working under a professor who was focused on human-centric psychology and human decision-making and basically the people who focus on a flight tower and all the buttons and making sure that there isn't fatigue in the human mind when they're having to deal with all of this different conditions and interfaces and so on. Same thing that happens in a jet plane or a cockpit. And so leaning into that world but loving the code, it leaned me heavily into like, oh shit, we're gonna need machine learning eventually because our brains literally can't process the amount of data going on. So that's what took me to TensorFlow and then I ended up at an insurance client. with Brandon and I started to see the world very differently because like research machine learning is very different than like application machine learning and without data like you can't do anything. That's exactly what we ran into. Yeah and I was. Well we keep talking about it. It keeps common theme from this podcast has been everyone's trying to do AI and the enterprise AI and then in the enterprise and the data pipelines, the data security, the compliance, all the constraints that the tech debt from there, all their existing infrastructure brings is like this like stopping point for them. And so yeah, without data, it's like, yeah. Well, so with that specific problem that we were trying to solve that, that was that was that subrogation? Was the was the objective there? Yeah, subrogation. But we also did the computer vision one where we were accepting damage. Yeah, that's right. Where we were kind of going through and identifying damage. But yeah, it was the same thing. We went through like, you know that you can build these models to be able to identify like who's at fault for this accident, right? And that's really all you're doing when you're doing the segregation in an insurance company. Is our client at fault or is their client at fault? And if their client's at fault, then we need to sue them. That's it. The craziest thing. Dude, for four years, I was thought that segregation was this magical thing, this highly complicated thing. Like, I couldn't really process it because my dumb, slow ass mind couldn't process all this stuff. I finally go to chat GBT and I'm just like, explain to me like I'm five years old what segregation is 'cause I've been building technology towards segregation and I'm still not exactly sure what it is. And it's like, yeah, who's at fault? And I'm like, are you kidding me? And I go back and I'm like, that's all it literally is. Anyway, so we go and we try to attack it. The problem was is none of the data that we had told us who was at fault. So we couldn't even train it. And that was before LLM's and we were only doing TensorFlow and more traditional machine learning models. But it was totally, it like, yeah, you just got stuck. You just couldn't do it without that data.
• So from a coding perspective, what was that stack like?
• Yeah, so the first thing we did was using like the space natural language processing. And so your bag of words techniques and trying to figure out, you know, counting words, basically building out like features on the amount of words inside of that data set for each claim. And that was kind of the problem that Brandon's getting at is that the structure of that data was just all like someone stream of conscious. like I talked to the client today and they said that this happened and then okay, I talked to the other insurance company and then they said that this happened. And now I'm talking to the client witness and they said that this happened. And then it's all short. It's like all short codes. It's all like THC said this and then how many, how many corporations have you worked for that all have their own coded language? Oh God. And it's just this way. It's like, it adds no value. It's just this way for people to protect their jobs, right? They're like, we need the CSV to go on the POP on the SOP. Okay. And then we'll be PVP.
• But honestly, somebody knows what you're saying when you say that, right? It's crazy that we build these dialects when we're in corporations so that we can speak faster to each other. But when you come now to structure and you want to structure beyond your organization, that becomes your biggest bottleneck 'cause you have to reprocess everything. Everything must be normalized. And that's the issues we're having with data kind of going back there.
• Well, yeah, so I mean, that kind of brings up this the chasm between the chat GPT models, which were trained on internet data, right? Which is highly, I would say normalized, right? 'Cause there's so much of it, A, and then B, you know, people are talking in a language that hopefully most people understand, especially, you know, English, which is its main language. But if you get to corporate data, corporate data's shit.
• Yeah, well, and so what was interesting is that I got pulled back into solving that same problem that you got pulled into five years ago.
• I was gonna say that you worked on that recently.
• Exactly, yeah, so they were like, hey, can we revisit this now that we have LLMs?
• Yeah.
• And so Nick was actually, we wanna say his last name, but Nick actually, we were kind of tasked, he was gonna go do a traditional ML model again, doing pretty much what you and the team were doing. And then I was gonna go use an LLM to try to, and we were gonna compete to see who could get to the closest thing. I think I won, but that's fine, whatever.
• Not stressed.
• But what I realized is with these LLMs, if you ask an LLM, here's our claim notes, is subrogation an option here. Subrogation is always an option. It just depends on which insurance client is asking it. And so there's an interesting thing and I've seen with these large language models and I've seen it both in just my day-to-day practice. I've seen it in me like trying to get freaky with my uncensored models. There's a three body problem. And it's very good at maintaining two concepts, two groups like one person talking to another person, having to kind of go back and forth. The moment you introduce a third party into the equation, it just starts to lose its marbles a little bit. It can't keep track of who are we talking about? Was it this person, who's doing this, who's doing that, whatever, it kind of like loses its thing. It can maintain two states. So it's a moment that I drop subrogation from this conversation at all. And I just said, who's that fault?
• Hmm.
• It was able to break it down. Every single time before that, when I was all about subrogation, is subrogation was always a hundred percent?
• Well, yeah, 'cause it's thinking about the words and the data that it has to go through when it goes through that permutation versus this very simple.
• Exactly.
• And that's what I think is, and I've seen this multiple times this week as I'm seeing internet content and things, and even you showing me image generation, there was this guy who posted on LinkedIn this week and he posted, "AIs come really far and a long way, "but I told it to tell me how many Rs "are in the word strawberry." And it said two, but there's three, right? And he was like, "That's why we need to take this stuff with a grain of salt. And I sit that and I step back and I said, "Abst, that's a good, that's an interesting little thing. "You gotcha, you got it." But like, you're not understanding the concept of the LLM, right? The LLM is not a, it's not a calculator.
• No. - No.
• It's not a calculator. It's, and you guys can explain this much better than I can, but an LLM is going through a very specific training process And then it's actually thinking about just what the next letter, the next word is. Right. And it's ability, it's a much more abstract way of solving problems. And it is conversational. Right? Because the prompts that it's been gendered, that you give it and its engines responses get sent back. Right? And it's a way to do things. But some people, you know, the first thing they do is they go to chat GPT and they Google their name or something. You know what I mean? And I think that anybody that's using LLMs, especially when they're coding or trying to solve problems in business, it's like, it's not always what you think it is and just because it doesn't give you the exact answer that you thought it might, or if it might give you inaccurate information, doesn't mean it's not any good. It might actually be your fault.
• Right, yeah, yeah. Well, math is a great example of that, right? It's just not good at math, that's fine. Now, functions and tooling and those types of things, those are all getting added in. Obviously, chat GBT has tooling and all that built into it. but we're starting to even see like the open source. My Ollama just rolled out feature functions within their tooling. So like those types of questions can get answered by a very specific tool that's not just a huge, large language model.
• It's like the difference for me is like ask it to draft out a program to use Rust to emulate, manipulate images in the browser.
• Right. - It's awesome.
• Right. - Versus, hey, take that code and run it and run these unit tests.
• Yeah, yeah. - And then give me the output. Like that's not what it's doing.
• It's not gonna do that.
• You know what I mean?
• So actually that leads to one of the interesting things that I had is, and--
• You might want to talk into the mic.
• That was one of the things that I, (mumbles)
• So Zed, oh it was a news news article just came out from Zed. If you don't know what Zed is, Zed is a rust based editor that's competing with VS Code. Right, so VS Code, if you've used VS Code, you know that thing weighs 50 gig of, it'll eat whatever ram you give it, And it's just, it's a fat boy. So Z has come out as a, as a rust based version of it, but they've incorporated AI just out of the gate. Now at first it was pretty just basic. You could go and put in your chat GBT API key and then like ask questions and do all that kind of stuff. This new version though, they've incorporated a cloud. So you can add cloud, you can do a llama, you can do chat GBT. I think they have Gemini in there where you can put in all your API keys. And then they have this thing where the context, When you start your chat, you can be like slash tabs. Now every tab that's available now becomes part of the context of the chat window or of our chat. So now when we're asking it, it has that. Or you could be like, you know, just this open file or just this. So it's starting to get better to where it was.
• Well that solves your artifacts issue when you're coding 'cause one of the things that I'll do is if I'm doing something and I am having this ongoing conversation, I'm usually trying to solve a problem that I haven't solved before. it keeps giving you, it'll give you pretty much the same code back or it'll give you something similar. But if you just didn't have to reprompt it and you might have made some edits to that code. So the in editor stuff is nice. If there was an option that had the same user experiences at chat GPT, from a UX perspective, I'd probably use it. But yeah, that context is important and that kind of brings up another thing, thing which is the concept of LLMs with specifically Chaggy PT like the mini for example. Now they're coming out with they've come out and they just announced it the fine tuning that you can do. And so like Brandon has he has he's like a heavy Chaggy PT user and he was an original GPT guy so he's got all these GPTs that he uses. I don't know have you built any GPT. Okay so you okay so I have some I don't typically use them I'm just like I I just threw stuff at the wall with TEDGPT and just see where it goes. But you've got like 20, 30 GPTs that are basically, you know, it's a layers of prompts that are built in. So then when you go ask this GPT, this, this GPT question, you know, you're probably going to get a what you want or what you thought you wanted. And you don't have to go redo a bunch of work. Right. Well, now you've got programs and products like cosign, which is, have you seen cosign? No, I haven't. So cosign is a startup that's kind of gone viral a little bit, but essentially they are they don't mention chat DPT at all on their website or their video. Yeah. So a couple weeks ago they came out of the video and it's their founder sitting there acting cool. And like basically it latches onto your code base and you create tickets in GitHub. You create tickets and it will put a poll request in for the ticket. Whoa. Try to fix the bug. Yep. It'll go through a series of calculated workflow steps. So it'll be like and it'll it'll it'll attempt to solve the problem and then make up its own tests essentially And then it'll iterate on the problem until it's done. Well The first of all in their example. It was like this really well written ticket. Yeah Step one right good ticket You know so so for all your product managers out there and you know engineers and developer have you like you have to get it had to have a great ticket.
• Yeah.
• Right.
• But what they did and what they're doing is interesting, which is they're not using the, they're not just, well, first of all, I thought they might be using their own model or whatever. They're using chat GPT for many, they fine tuned it. And I think that they're just constantly sending your code into their fine tuned forked model.
• That would be interesting. Yeah, if they're just continually rebuilding it over and over again.
• I'm sure that's in terms of service.
• Yeah. (laughing)
• You know what I mean? So it's, this bot essentially is iterating and coding and fine tuning and it's like, for me, it's like that the fine tuning concept is interesting if you're willing to accept the open AI as your--
• Overlords.
• Yeah.
• Right. I mean, that's how it's gonna go though, for the future of Enterprise AI, right? Like everyone's gonna have multiple fine tune models that handle specific tasks. And even if we get into the agents conversation, they're gonna have specific tools that those specific fine-tuned models can use that allow them to start achieving different things at scale. And so like the point of being able to make board requests, like you can do that for certain tasks to the point of you have to write a fine-tuned ticket and the ticket needs to have maybe even examples and the specific file you're trying to change because they'll get lost in it, but it makes it totally possible to scale that out across multiple developers, increasing productivity just based upon how many lines of code have changed or what outcomes are we seeing?
• You could, on the other side in the intake, you could take human language and try to turn it into a ticket that is trained on previously good tickets, you know what I mean?
• I think we're gonna get to a point where we're gonna be having debates on which QA developer AI agent is the best, right?
• Literally.
• Is it Steve? Steve.ai or Thomas.ai or Jessica, whatever, right? They're all basically gonna be able to have these agents They can go out, they can look at your bugs, they can look at your reports, they can look at your shit, they're gonna go through, they're gonna try to test them, they're gonna try to prove it, they're gonna try to solve it. And people will be like, yep, I paid that guy $60,000 a year and he just comes in and just fixes our bugs all day, everyday, non-stop, just as a robot. We're gonna get there and it's gonna be friggin wild.
• Yeah, I think the use cases that come to mind for me right away is security, refactoring.
• Oh yeah.
• Performance, but where it gets muddy for me is like database design, UI, right? Like the UI, sure, but now you're gonna have to feed it. Figma files, you're gonna have to, you're gonna have to fully build up. Design system, and sure. So it comes back to those like that data garbage in, garbage out stuff where it's like, if your company has a great design system that's in code, like an Airbnb kind of company.
• Yeah. then you could probably train models on it.
• Then sure, it's probably AI enabled in the future. It would know what button component to pull in. It would know that your cards have the button on the right and the secondary action and all that. You know what, it would know certain things, but out of the gate, and that's why. I think the rich just keep getting richer. You know what I mean? And these industries, and that's been one of our premises.
• Yeah, I think there's a moat to this though, right? 'Cause there's so many older enterprises and businesses which have sunken themselves into this tech debt hole to where this knowledge lives amongst 20 different people who've just been at the company their whole lives and like they are the only ones who could even give you the knowledge base that you would need to be able to make those changes. And so I think we're gonna see an evolution of new companies starts developed and they're gonna be able to say, hey, we're actually like AI base, like from the base to your point that you were talking about, I think you said cosign or the one that was. Yeah, so like being at that core, your base is AI and you're using AI tools to help you expand your scale As you begin to like list out all the things that make up your database, list out all the things that make out your APIs, and these models have access to that, you can iterate at lightning speeds by writing just a simple prompt. And like even your prompts can be generated by one of these agents, right? So like you can create the greatest prompt that has all of this context, and then that prompt now informs all the others.
• And just get smarter and smarter.
• Well, so to me that brings up two things. So it's almost like BC and AD.
• Literally.
• People are like, "Hey, I'm after a yacht." So if I'm a, I know a guy who was a CIO at a major company in the Indianapolis, he just took CIO on it, a new company in the Indianapolis. And I worked for both, or a consultant for both, and I'm sitting there thinking, "I think if I was him, I would probably just blow the whole thing up." Right, I would start over.
• Right.
• I would start the money.
• Yeah.
• I mean, and at the end of the day, half of our developers would probably love it, 'cause they'd be like, "Oh, I don't have to work on this. a Java thing anymore or something. You know what I mean? We can redesign the database, you know? And I would just fork off half and put everything in a maintenance mode for like two years.
• And if it's described, like, I mean, these business processes are outlined down to like the API, the data type, 'cause I'm a data engineer, so like in my world, you know, you need to know your data types, you need to know the possible values, you might be able to get the ranges. If that's defined to where these models have access to it, and then you start building from there, I mean, like anything's possible.
• Yeah, that's a good point. I think that a lot, and I think a lot of the companies do have a lot of that tech debt. Maybe you're, do have a lot of a little bit more of that document. Maybe, maybe, maybe you're not, but I've actually seen it where like, here's our data dictionary. There's a 7,000 columns on this table, right? And here's what they all mean.
• Yeah, yeah.
• There's a-
• Consolidate.
• I think you could get there. But yeah, I mean, you're coding and you're, you're in your volat- The other option is essentially, you're gonna get replaced. Like you said, a company's gonna get started. So if the other thing is I say, okay, I'm going to go do a startup and I'm going to go identify companies that probably have tech debt and I'm going to say, you know what, I'm going to start a company that does exactly what they do and I'll probably be able to compete with them within a year or two. Right. For sure. And that's why I antiquated industries, right, are going to be the first ones that get attacked. Like we're already seeing its supply chain. You're seeing these API companies popping up e-commerce, you know, these legacy businesses are going to be under fire because maybe not today, maybe not even three to four years from now, but in five, seven years, these companies just won't be able to keep up with the scale that these fine-tuned models will be able to move at just because like, if you can QA your code, like automatically, like--
• Well, yeah, in the front, you're saying the coal in all the fires is developer velocity encoding.
• Facts.
• At least from three dudes that. (laughing)
• Right, but the idea is that like, you have a product or you have something there's technologies is the deliver of the service. And if you can't iterate on that service and improve that service, you're not gonna be successful. And the crux of iterating on the service typically is writing some code or designing some change.
• Yeah, there's gonna be a, I think that we're gonna see this point happen where companies who really leverage AI like truly embrace AI as like a core thing, they're going to go hyperbolic. And everybody else who did and is going to sit there and be like, well, why the fuck are they a $10 trillion company now? It's like, well, because they actually-- they went full on. I mean, they went full on. They said, we got to start with our data. This is a conversation that we have with that insurance client years ago. Like, guys, you have to get this data right. If you want to be future thinking in a high-end technology company, you've got to get your data right. And if they would have started that back eight years ago, they would be there today and they're not, right? So like, again, if you truly wanna be bleeding edge and like go dominate the future, get your data right. You have to, you just absolutely have to.
• Unfortunately, there's probably a lot of boards that are wary or CIOs or product managers that are like,
• And all of them should be fired.
• Can we just skip to the end here? Can we just skip to the end here?
• Skip to the end here.
• Can we not do that hard work that you think that we need to do? Can I just get this vendor in here that's going to over promise and under deliver? I mean, that, you know, because they don't, they don't want to hear that. And they probably never have, because how many times have someone come into their office and been like, if we, if we, if we just refactor the whole thing, it's just going to be fine. Yeah. We're not even saying refactor. You don't have to refactor anything. You have to get your data right. Get it, get a true data leader in there and says, listen, we have to be able to query our data. We should be able to query it in a way that's very consistent and easy to understand. of the data should come back clean. And if we can get to that point, then we can start ingesting this with AI all over the place. And if you can't get us, you're fired. Right? Like again, if you're a data person at a large organization and your big mission isn't getting your data ready to be consumed by AI, you need to be kicked to the fucking curve in my opinion.
• Yeah. I mean, I think there's a lot of truth to it because we're at an age where like, we're kind of leaving the age of information behind and we're moving into the age of like datasets that are operable together, you know? Like you need to be able to pool your datasets together from multiple sources. And the only way that you can do that consistently at scale is if you're cleansing your data completely. Going back to the people who are in these organizations that are antiquated and old and have knowledge in their brains, those transformations are happening in four to five places. No one can keep track of them. And when they break someone's specialist has to go in and fix it just to get clean data. It shouldn't be that hard because you can't update that process, you can't maintain that process, you can't manage that process, it's hard to be agile with that. And so I feel that then. I think people are going to have to reframe how they think about their data in order to compete in the future. And that's always been like the mindset. People are like, yeah, we're data centric. We have the modern data stack. Like there's been all these terms that people have been able to use. We're in the cloud. Yeah, right. It's not just having this infrastructure available to you. It's being able to leverage your business knowledge and translate that into what your data is actually saying. So going back to the subrogation piece, the hardest part was like, okay, we have this natural language text, but we have no idea what aspect of this is screaming and subrogation and how do we define that? What does that mean? If that's a business, no one can tell me what that means. How is AI supposed to solve this problem for you?
• Exactly. - You know?
• It's so crazy.
• It's so crazy that when you guys were asking those questions, they were just like, well, what do you mean? And it was just like, what do you mean? What do we mean? This is nuts. You need to have, like, how can you guys not have gone through and identified things that should have been subrogated historically you haven't okay well now we need to start right now like today every single one needs to be flagged somewhere or another right and they didn't do it they didn't do it and now you're getting your five seven years later whatever it is and you don't have that information it's just it's it's kind of mind-blowing that that we're here so I I'm very hopeful that now we're getting to a point where everybody can just understand that literally data is the new oil and that you've got to have a good amount of data. If you've got a good amount of data and it's clean, you can change worlds with this.
• So, okay, so data, great, moving on. When it comes to code--
• This is Sean's board with data.
• Yeah, I'm bored. When it comes to, and I don't even know how to answer this question, when it comes to LLMs, like why is, why are they so good at writing code? How do you, how would you answer that?
• The corpus that they've been trained on, like the model has, there has to be specific curriculum for these models based on code. So like there's this website called PO and it basically gives you access to like all the different models that are out there under like subscription or whatever, you just basically go in, it's chatbot and there's specific ones that are like based on code. And it really makes me think that like, okay, these models or at least the data sets that these models are built on are probably in OpenAI, probably in anthropic as well.
• Well, just what, what, GitHub, right?
• Yeah, exactly.
• Right, GitHub, right there.
• Co-pilot.
• Oh yeah, there's been times I've gotten responses from Chad GPT and I can almost pinpoint the exact issue that they've resolved on. So what kind of data is on those little mini models that you were talking about? What were they training them on?
• Oh, you mean like the ones we were doing.
• Yeah, so opening, I--
• Oh yeah, that was more.
• It's Chad GPT-4's, oh, this overarching general purpose thing that happens if you really get a writing code. Like what were those little mini things?
• Yeah, so this is the old school traditional machine learning models that we're talking about.
• Yeah.
• And that was, was that all, was it all TensorFlow that we were using back then?
• I mean, four, so it was, got NLP, there's a library we use, but we were just using an NLP library, and then we put it in, we put the outputs of that library into a TensorFlow model. And then I think we did like, it was a decision tree at the end of the day of the data.
• Yeah.
• But I mean, I think the core, how they train those models and the curriculum they use, and they're probably scraping GitHub if not already have a license with them to gain access to it, is why those models have so much information on how to record code. - So basically, they're scraping the internet for all the issues.
• Yes. - And the way people are thinking through problems, but they also understand the language at some level, or they think they do.
• Well, again, I mean, they understand it in context of what each character--
• The means. - The next character is, right? So that's how it understands it.
• Well, sure. The thing that blows my mind is just the breadth, right? Like, I can go to the same prompt. >> Yeah. >> I can ask it to write me something in JavaScript Rust. I can ask it to write SQL. I can do all of these different things. And for me, it's this ability for me to step outside of my comfort zone, right? It's like, hey, I'm gonna try something that I've never done before. So typically, I'm a UI guy. So I write HTML, CSS, and JavaScript, and I can build React apps, and I can build Angular apps, and I can build view. And I can build node APIs and things like that. But now I'm doing it to do things that I would have never thought possible. Like image manipulation and rust or barcode scanning or stuff that I'm doing. And I'm doing it.
• In a couple of seconds. (laughing)
• I know. It's like this core backbone thing that I've never, it's like, hey, go build me this library. Does this thing that I've never, would never have the time to learn or understand? And then show me how to suck it into my app. And boom.
• Well, so, but there was an interesting thing that happened with that specific problem that you were solving that I think is an interesting, human psychology discussion around it. Is that, so you go and you end up building this thing and you end up, it ends up working. But then the client comes back and is saying that, hey, I need you to be able to process 34 K images at one time.
• I got a bad requirement. (laughing)
• But it's funny because like again, you can go through this process and you can build these tools that are very specific, but you don't necessarily have the mental model of how to tweak it once it's done.
• Yeah, so. - Right, and so yeah.
• Yeah, I mean, I think that you run into an issue where it can get you really far, but if you're not an expert in that specific area, like I'm still not an expert in rust, right? So like, like there might, like for example, today I was dealing with a memory leak and I, that's one of the hardest things.
• Oh yeah, exactly.
• And the memory was just shooting up and it was never getting shot off. Now I know how to debug memory leaks the browser really well, because back in the old days, some of the UI libraries before React had really bad memory like issues and Chrome had those tools. But you can get really far, but then you do kind of peek out. But I still think that even for most, like a senior, a senior, I hate using that word, an experienced developer that's using AI on a daily basis is like the superhuman right now. And it's to the point where it's like your productivity and your capabilities are just increasing the breadth and the depth through the roof.
• So what does the workflow look like, right? So like a developer who has like a chat GBT enterprise grade available, or they're even building agents specifically for that developer, like how does their workflow change? What does their productivity hours look like now?
• Well, for developers like for my developers, a lot of them are actually using the, a couple of different ways. Some of them are using the NVS code, code suggestion stuff and they're talking to their code more like you're doing with the context window thing. I'm doing more abstract problem solving in chat GPT and then bringing it in. So somebody's like, I need this new solution and you go and you have it abstract out the potential ways to do it, then actually write the code. I think that there's, I'd like to see there will be a little bit more of like what the the cosign is doing, which is like in the background, like hewing code, giving it tickets to solve on its own.
• And then reporting back and be like, "Hey, I fixed it, by the way." You just keep pulling the human into a loop of that kind of stuff.
• Yeah, and I think that the missing piece is that that may or may not be solved with prompt, layering or even fine tuning is like having something truly live and breathe as your code base lives on. as it grows. - It's almost like a fine-tuned piece.
• Yeah, I think we're gonna get there. And I think what Zedd's doing is kind of the first step into that, where it's gonna be intelligent enough to be able to say, okay, he just asked me a question. I know what my source tree is, which files should I pull? That would be relevant to this question. Pull those into the context, be able to answer it. I think we eventually get to this place where we're not having to like tell the AI, here's the files I wanna work on, here's what I wanna do. It just kind of knows. And that would be awesome when we get there. I mean, again, I use this like, again, I use chat GBT and Claude and Gemini, but Claude and chat GBT primarily, every day, 20, 30 times a day, guaranteed. We were talking about what was the story, a storybook for UI libraries, right? So I'm working with one of my clients. I was like, I want to sit up at this library so they can see all of our custom components and on React. Like, so I'm doing this all in React. Traditionally I do spell setting up spelt and storybook has always been a nightmare with react It was literally like one command and it just worked. I'm like this bullshit. That's why you stick to a ecosystem It's got actual users Nevertheless the the real the real fascinating part of it though is that That I was able to go and just basically be like here's my component go right in you a storybook Class for whatever they're not classes, but you know, they're whatever their implementation story. It's story That's right and sure enough and it does a great job It goes through it comes up with every permutation of it knows all the different variables No, all the inputs all the outputs and it just does it and and I was able to knock out, you know 40 different components and literally like an hour and and all I can think of like three major use cases that for AI that that I see like people like you or me using one is writing the Basically doing the chores so writing tests and we're doing all the cleanup stuff For you you give it a swagger give it an a spec and then just basically ask that API spec questions like hey Please write me a Function that goes and gets me all this stuff and it'll it'll be context aware of the swagger So it'll know about your API. Yeah, so I said I said went up for a Theraprove, which is one of my clients And I just got the Swagger document from the development team. And so I started working on some front end stuff. And I'm like, I just, and I did, I went and created a GPT, uploaded that file as one of my files there and just called it, you know, Theraproove API. And you just go in and be like, hey, how would I go and find all the patience for a specific caregiver? And it just outputs it all right there for you. And it's just like, this is so great.
• The other thing that I would argue that it's really good for is just like, I can, I've had a code base that was on next JS9 and I needed to get a 13. So that's four major versions. And I literally just took the, I took the main files that were problematic, pasted them in there. I was like upgrade this to a stateless component and remove this, you know, remove anything that does this, this, this, boom, boom, boom. I almost could have created like a little workflow to do it. So it was basically doing refactoring for me. But you know, it's like, get it to just like do stuff for you. The non-creative stuff that keeps you going, for me, I'm super ADHD. So like, I don't want to do like all this stuff. Right. Yeah. Sucks. Yeah. You know what? I'm in the yet total. Just the repetitive task. Right. It gets boring. I mean, developers feel the same way too. So like, I think prompt engineering and getting developers on that train of like how to properly prompt and what are the things you need to know about this model so you can prompt it correctly. what are prompting tools you can use are really gonna be like the productivity gains that are going to see like the game change for a lot of developers, which we're already seeing. Like that's what we've just been talking about. I'd put so much stuff. So I'm a data engineer right now for a pharmaceutical tech company. And essentially we're building off of Snowflake, DBT and then building into our application. And I'm constantly like, what is the best way to build this be this model out, right? And I'll give it like a spec of what the model is. They don't always put the the call names into it just to be secure. but like how is the best way to build this model? What's best way to join this model? And one thing that's missing for data people, I'd say, is like access to the actual database so that it can just go in and say, "Okay, this is what's going on. This is your join key. This is all of it lives already in DBT." So I can see like a tool like DBT starting to say, "Okay, here's your agent, just the way Snowflake is with Co-Pilot, and we're really being able to give developers tools and now start acting on your data."
• Does it matter if Chad GBT has access to some of your code?
• For sure.
• Yeah.
• For sure.
• I mean, 'cause like, yeah. Like some of it's just innocuous. Like it's so, the ecosystem of your app is so like decoupled and there's all these different things. Like what, where do you get to the point where--
• Samsung would disagree.
• Yeah.
• Samsung would say, "Hey, we've put all of our chip designs up there and those fuckers started turning around and answering questions on how you designed Samsung."
• But they gave them chip designs. They didn't give them their React front end. (laughing)
• Well, you know what I mean? At what point, it takes a lot of pieces to the puzzle to put it together to create something.
• To create a full thing. Like, yeah, I mean, I'd say the only thing that you really kind of have a risk of when you're using these tools is potentially leaking your environmental variables, leaking some of those secrets that are more, like you should never actually be up there. The one thing that I was building that I think three weeks or maybe four episodes ago we were talking about where I ended up just building a thing that literally just took my entire code base and just use that as the context. And fuck it, we're gonna use Gemini's huge context window and just put everything into that context. And it was great, except for then I'm like, oh yeah, I'm leaking all my secrets. I had to go and kill all my, as my, I had Gemini, I had Claude and I had OpenAI API keys in that code. And I'm like, oh, sorry, I have to go and kill it. But yeah, I think that that's where we're gonna be.
• And I mean, like we still, you know, like pen test happened and you have people come in and try to see what they can find, see what they can get access to. And that's always gonna be their biggest thing, is like you have keys and your GitHub commits. It's on the code anymore, but it's in the commit.
• Yeah, it has down there that one thing, like a year and a half ago that nobody even paid attention to.
• And it's crazy 'cause like co-pilot, you'll just type in like environment or whatever variable class you're using, and then you put the variable name equals, and then co-pilot kicks in, and it puts like some key in there, some UUID in there, and it's like wait, like where did that come from? You know, like what is that? That's not a random key that you're trying to predict. Like, you learn something and now you're trying to recreate it 'cause that's what you think I need. And I mean, like these security things, like they're concerns because eventually, and it's already happened, like chat, GBT, their biggest problem has been like setting prompts on before like anyone types anything in to stop people from gaining access into what chat DBT is doing or what OpenAI is doing.
• Yeah, they don't wanna, they don't wanna deal with that.
• Right.
• Because they're, that's really the way that they can get hacked is by accidentally exposing their customers information.
• Well, I think too, so like that, that Aaron--
• Don't commit your keys. That's an easy thing not to do, okay? If you're on GitHub, if you're on Bitbucket, any secret, like--
• Yeah, don't put your secrets in your code.
• The Git ignore is the first file you create.
• Exactly.
• The first file you create.
• It's so crazy 'cause like, again, you can just do like the SK- searches within GitHub. And you're gonna see all of the leaked--
• Do you know how many Google Analytics Google Maps keys are on GitHub.
• Oh, I can't see it.
• Oh, yeah, yeah.
• Google Analytics is pretty much just open, but.
• Yeah, but no, you can easily get screwed. And now I will say GitHub's done a much better job at letting you know. Like if you've committed something that has a key into it, it pretty much like immediately is like, yo dumbass, you just submitted something that has a key into it, you need to go address this.
• But most keys, don't you think go into the key thing? Most keys shouldn't just be open use. like there should be some second factor associated with the keys.
• But it's up to the company to implement that.
• I know, but there should be his standards. Like if you implement a key, like it shouldn't just be, like for example, I shouldn't be able to send Google Analytics data off my domain for that domain. You know what I mean?
• I mean, there are standards, are they followed? (all laughing)
• Well, but so like with now you can go in and you can create these. So they've updated, open AI at least, is updated to more of this project based API key. you can go through, you can be like, okay, they can read, they can write, they can do this, they can do this. So you can really kind of now narrow down your API keys. But yeah, the majority of the ones that are out there right now are those like OG API keys that are literally like root access to your shit. And those people once they get it, AWS, AWS would be the worst. I mean, like AWS, like open AI fine. Like if somebody gets your thing, they're gonna start racking up a bunch of things. AWS do-- I think they designed their security system with the in mind of, let's just get people to just use as much of our services as possible.
• Yo, buckets used to be public. You guys remember that?
• By default.
• By default.
• Are you serious? And so all those leaks happened, they were just like, oh, they should be private by default. Oh really? Who would have guessed?
• Like if you would get someone's, their access keys for AWS, and they haven't gone through that process of locking those things down, you could literally in one hour rack up a bill of $150,000. - Yeah.
• So ridiculous.
• Yeah, I mean, like there's so many people now, like hackers, their goal is to get access in and then create a key and then don't use it.
• Yeah, let it be a ghost key.
• Yeah.
• And you don't use it, but you have access to this entire organization. There are kids who--
• Did you call it a ghost key?
• A ghost key.
• Oh shit.
• There are kids who do this for Dropbox or Google Drive. You basically get access to the account, using a leak database, you go in, I'm kind of exposed. (laughing) You go in and you basically create a key and you just leave your, some email address as the access to that key, full access, and then you just never use it, you leave it there. Who goes in to check these permissions? Especially if you name it after somebody in the company, right? This is social engineering 101, but it's so easy now to just gain access. So to your point, you need to follow the standards of like expire all keys. all keys must expire and you must recreate them. And if you don't, then like you run the risk.
• And then there's certain services that are built for non-tech people to install. They're like, hey, Marketer, here's your Google tag manager. Dropping each other script you want. (laughing) It's like, whoa, who's out of that idea? They're fucking stupid.
• It's literally a JavaScript to the head of your website that can do anything.
• That can do anything and they're like, well, we wanna make it easy for marketers to never have to talk to a developer. Well, that was your first problem.
• These guys have no idea what JavaScript is.
• And that's what makes them targets, right? 'Cause you have tools like Zapier, Make, like all the NAN is another one. And all these tools that allow you to kind of drag and drop code, but they need an API key, especially if you have database access or if you're pulling out a service like, you know, chip monkey or whatever, and you're trying to get these tools to be used in this like automated environment, like those keys need to expire. They don't want them to expire 'cause they don't want to have to edit that workflow, especially for a key, right?
• Like how many times you have this man?
• Sometimes you have this random person for a vendor you've never met, for a comp, for a founder, or somebody that just, I get on, I've been on it, like this having this week, yeah I'm on a group text. I still haven't received access to the Shopify administration system. And I'm like, who the fuck are you? (laughing) And they're like, oh well we're the ads, we're gonna run these ads for these videos. Oh okay, so you're just gonna, you're just now, I don't know who you are, but sure, here's admin access. Go change the bank account that the money flows to. You know, and people just don't, but they're like, this is an IT problem. And like, I think as a business person that doesn't write code or isn't involved, like understand that like, you're just, if you don't get in the details on some of that stuff, you're gonna expose your customer data, you're gonna put yourself at risk.
• I literally had this with one of my other clients who's HIPAA related, and I'm working with it. They're working with the market.
• Yes, the market and company wanted, we just put that in there. Just go ahead and just install the Facebook ads and go do this. I'm like, guys, we're like, no, we can't do this. We do realize that we're a HIPAA, well, HIPAA is critical for our business.
• The last thing you wanna do is go, uh-huh.
• Like, you,
• The last thing you wanna do is drop a pixel from Facebook and Google in your web app.
• Exactly.
• Exactly.
• This is dangerous.
• It's just, it's just, and they just don't think about it. And I get it, like, again, they're in that marketing mindset. They're not really thinking about it from like the product limitation. They're just like, we just need to have ads. We need to have blah, blah, blah, blah.
• So one thing I get about the marketing thing is the conversion tracking and the event tracking. But here's a novel idea. Use your fucking database. Okay. - Exactly.
• Every time someone creates a post in your app, guess what's happening? A database injury. (laughing) Okay. And so we use platforms like Q.dev. I don't know if you've seen that. It's a platform that latches onto, it's a semantic layer for your database.
• Okay, cool.
• It sits on top of Postgres. and it provides you secure reporting. And we blink that into our dashboards and our databases and we say, I don't want your fake ass vanity metrics. Just give me access.
• You mean the actual data.
• If it's something that needs to be tracked and it's a transaction in the system that users are doing, if they're clicking a button, usually it's doing something. It's creating something. Do it against real data. And don't worry about some marketing guy that says, when this button gets, I want to know about the button.
• Yeah, yeah, yeah.
• So there's better ways to do that.
• Yeah.
• Yeah.
• So that's an interesting, what was the name of the product?
• Cube.dev. I've been using it for a couple of years now.
• That is a very interesting. I've never heard of it, but it absolutely is.
• I, in my mind, I call it transactional analytics, which is like just report like actual like cubed style data on your database. Like counts and like, you know, what does someone wanna do there? They wanna like, how many times does this happen?
• Yeah.
• Counts.
• How many users are we getting? How many are coming back? similar to like a segment or like a rudder stack or like yeah. I think similar to a segment. Okay, cool. I think the difference is it's just actually on top of the database. Yeah. And so if it's true, you know, so one thing you'll start doing is just say, well, maybe we should put this in the database now as a separate table. Yeah. Right. Which would make you a better product overall because again, having your analytics within your own data set versus having to be relying on a third party is always going to be better. Yeah.
• Well, at the end of the day, what is an analytics platform doing? It's creating data that you didn't have.
• Yeah.
• Right? And it's also putting it in.
• And then when you probably do have it, you're just not willing to talk to the development to figure out do you have it or not?
• Yeah, yeah, you're saying, well, it's just a pivot table, okay? It's just a sum.
• Right.
• And a lot of people are like, well, the database is this foreign object that I don't understand, but your database has actually got value beyond belief. I mean, you know it. (laughs)
• I mean, this is why I've been interested in the low-code, no-code space, because I think giving so many more people access to development tools is going to change the game. Just like we're talking about these marketers who don't know, but now we're starting to know and they can do things. I think air tables are a great example of building a database first and then building the application on top of that is immensely invaluable. People are now able to understand, okay, how can I capture my users? How can I connect my users to what they're doing? Their sessions, really breaking it down. Or like, this is how tables work and rows work, I think can begin to mitigate this issue of by getting separate from the database and not understanding it.
• I think that's a really empathetic way of thinking about it. And I think that I have seen a lot of people that are non-technical, learn more and get more attuned to it. So maybe it's less of, it may be less of the whole concept of not using those external tools. We may be educating or figuring out ways to have your team say, hey, we have this data or you or let's build this data or you know, let's build this platform, you know, like in getting them to and or just working or working with people that are that are empathetic to your side of it and are a little bit more savvy of those tools. Yeah, I mean, that's why self service, I think, is becoming more popular self service tools in these organizations, like how can I give somebody the ability to spin up a database really quickly and then populate that with data, and then we'll take that and pump it into the data warehouse. Because one, we want you to have access to anything you need so you can develop rapidly, but also we want to be able to take that data and bring it into everything else and then generate reports for you. Well, that is a, I feel like a separate but unbelievably great point, which is what happens when the business or even the guy like me, I'm not necessarily infrastructure IT. But if I can't get what I want, what am I going to do? You're gonna go ask somebody. I'm gonna go ask somebody. I'm gonna go create it on my own. I'm gonna go talk to a vendor. Right. So if I can't create a spin up a database quickly, like, well, we We shouldn't have people spinning up databases. That's just too, we have to have governance and we need this. Well, you know what they're gonna do? Is they're gonna go to airtable.com or they're gonna go to postgres.com, you know? Or they're gonna go to super base, you know? They're gonna figure out a way to do it without your ass.
• Internal team, internal IT teams are gonna have to absolutely deal with this, right? Now right now that I can go out and I can circumvent pretty much every technology that you're not giving me as an organization and I can find things that will put it out there Do I really care about their privacy policy? Did I accept their terms of service? Yes, I did. Am I represented over the company? Yes, I am. And are we bound to that now? Yes, we are. And are we gonna do this? Yes, we are. Does everybody gonna be pissed off? Yes, they are, right? Like it's fucking bananas. And we see it every day, these people going out, they're signing up for stuff, they're putting in data, they're doing all this crazy stuff, that IT people are gonna absolutely have to be able to start giving them more of this power in-house to allow them to do it. Otherwise, they're just gonna feel like they're literally handcuffed the entire time.
• Yeah, well, you know, it's definitely a world where there's just so many capabilities and there's so many possibilities. And you have to figure out like the best ways to skin it. I don't think you can sit there and go, I'm just fully against a guy, I would never use it. Or I'm fully against shadow IT, I don't think that anybody should use air table. You know, I don't wanna bring anything into our environment that's external and start of it. You have to be open-minded, but you have to understand where the risks are and the opportunities. And at the end of the day, from a pure productivity standpoint, the companies that adopt AI went into their coding and development workflows are gonna produce more because it is undeniable from three experienced guys, and the host of people behind us that are already using these tools every day. we talked about the survey from Gartner. 90% of just employees at companies are already using AI in their daily workflows. But it's the C-suites, the companies that have an implemented AI to their employees. They're already going out to chat to you to write the email. They already signed up for the Gramerily account in their own personal, because it saves the money, right? They're already using AI, so that the employees are bringing AI into your company if you like it or not.
• Right, yeah. It was, that's kind of like the, both the Dropbox and Yammerly. Or Yammerly? Was that the name of it? Yammer? Yammer. Yammer, yeah. So Yammer was kind of like the first Slack, right? And both the Dropbox and Yammer did a brilliant job. You came in, you started using Yammer. You're like, "Hey, invite three of your friends." And you threw your friends go and sign up. They were like, "Hey, invite three of your coworkers." And they go and invite. Next thing you know, I get an invitation down to the CEO of, it was Cuz Sigma Micro at the time, and it became fifth gear. And they're like, "Branon, you do realize we have now "300 people signed up for Yammer." And when we wanna go and actually pay for this, it's $5 a month per user, right? And I'm like, "Oh, sorry about that guys, "I didn't realize it would go on." It literally went viral within that company. And so those things, those companies, I don't feel like we've had anything like that recently. I mean, Chad GPT probably did, but it wasn't like inviting your friends. Everybody just sold the value of what Chad GPT brought.
• Wouldn't it be cool if Chad GPT was built into Slack or it had its own Slack where everybody's,
• Did you just break our mic?
• Maybe.
• Okay, those are things that we need to break.
• Like wouldn't it be cool if I could basically have it, and that's where you said the three body problems. That's why they probably haven't had it.
• Yeah, right.
• Would you introduce that other person?
• Well, you know, I think at the end of the day, it's the world's your oyster, the possibilities are endless. The risks are also endless. the barriers to entry can be severe if you're in corporate tech debt land. But if you're a developer out there and you're kind of hesitant and you're not using this stuff, use it to learn, use it to refactor, use it to write tests.
• Dude, every-- - Is it to concept solutions?
• Every component that you finish, right? I want you to copy that component, go paste it into chat GPT and just say, what problems exist? That's it. And it's gonna go and feel like, yeah, well, the way that you did your state management here And here is causing a double reaction. So let's cut it down to just a better way to learn in general because it's a real world, it's real world learning and it's basing it off of thousands.
• Instant feedback.
• Yeah, exactly.
• Instant feedback. So, no, I think that's good. I think that if you like this podcast, I wanna ask you to do something right now. I want you to go to YouTube, check out Big Cheese AI and subscribe to our channel. We wanna get to 500 subscribers by the end of the year. And that's our goal.
• Hit that button. Hit that button, baby.
• One of you, man.
• This has been the Big Cheese Podcast. Thank you so much for joining us, Max. We'll see you next week.
• Adios. - See you guys.