Goldman CIO Marco Argenti on the Warp-Speed Improvements in AI

Bloomberg audio studios. Podcasts, radio, news. (upbeat music) - Hello and welcome to another episode of The All Thoughts Podcast, I'm Tracy All-Away.

- And I'm Joe, why, isn't all? - Joe, the thing about AI, I feel like it's accelerated all of our timelines, right? Like it's phenomenal to me to think back that back to the days of chat, GPT,

and then when did that come out 2020? - late 2022. - 2022. - That's just crazy to think. - And then that's really unbelievable the gap

and I've been thinking about this, like just a the explosion of capabilities. - Yeah. - And the thing I've been thinking about is that, you know, after the first year or so when it came out,

we talked to executives. And we're like, how are you using AI and your workflow? - Everyone's experimenting, it's great, everyone's using AI. - Yeah.

“- And you should be too, it's always very vague.”

And now in 2026, the story is that AI is so powerful

that it's going to destroy all these legacy software companies. So what I would say is we must be past the age of experimentation. I think that-- - It really is cases. - Using it better have some example of like,

here is a workflow where we're using it. - Well, exactly. And to this point, now that we're past the age of experimentation, I'm very curious how executives and managers are actually evaluating the return on investment in AI and what they actually

want to see from it at this point. So you know, are you going to replace all your third party SaaS contractors with internal coders? And what does that look like from an actual head count perspective from a cost savings perspective?

We can actually get some concrete details on this now. So I'm very excited to say we do in fact have the perfect guest. Someone who we had on before to talk generally about AI and someone at a company that has been doing. You know, they got into it pretty fast.

The last time we spoke to this person was in 2024 and even since then. - Eight years ago. - Eight years ago. - I guess it just feels like light years in an AI time. - So just one last thing on the last few years of AI,

which is that when she had GBTA used-- When it came out, I played around with a lot. When I had it right poems and all this stuff. And then I bet if you actually looked at my AI usage, I looked through a trough.

Whereas I wasn't really getting any productivity. It was nothing really could do that I needed. It's still sort of a seem like a toy. So I had this, like a tense burst of use for the first several months. And then this trough.

And now these days with the expansion of capabilities, particularly cloud code of finding all kinds of new things. So there is like we're out coming out of the trough.

“I think a lot of people are actually finding things.”

At least if I can generalize for my own experience. - Yeah, absolutely. So we do in fact have the perfect guest. We've brought back Marco Argenti. It is of course the chief information officer over at Goldman Sachs.

Someone we had on the podcast back in August of 2024. So Marco, thank you so much for coming back on. - That is, thank you for having me. - How much have things changed for you? Does 2024 seem like 20 years ago now in AI time?

- Yeah, I barely remember even what happened back there. - That's a nice way of saying you forgot what we talked about on the podcast. But literally like things are really changing on a weekly basis almost right now. And if I look at the evolution not only since a year ago, but even since six months ago,

I think it has been nothing short than a revolutionary.

A year ago we barely talked about agents or the word almost didn't exist. We were using AI's like a chat companion there. - Yeah, there was such a function. - Yeah, it was telling you, oh I'm sorry. I don't know who's the president of the United States because my cut-off date

is like a year and a half before or thinks of that nature. Now you can say, hey, as a person, you can say, hey, my plane just got canceled. And it's going to redo all your plans. It's going to check for like available flights. It's going to do all these capabilities of personal assistance.

And that translates in corporations also in a lot of utility that you can see in everyday tasks. So I would say, when I like to say to my people about else in general, I say, this is not a drill, this is real. You know, it's not the age of experimentation anymore. There's a tool that now can do a lot for you.

And so we put it at work. And we put it at work starting from developers, but don't expand in many, many other areas. So I would say actually if I look at the increase of capabilities of this small stuff. Well, what we've seen in the last six months or so with really the evolution of this advanced reasoning capabilities they came out.

“I think that finally got us the confidence that you can use AI for everyday's work with the right supervision.”

And also in many cases for mission critical applications.

It's not a toy anymore. Essentially that you can expect results from. And I think that's the biggest change. So today I would say that there is almost nobody in Goldman.

We gave our GSAI or GSAS systems to 47,000 people. Most of them use it every day. Most of them use multiple times a day.

And what's interesting is it's like the first time you see a tool like I don't know Microsoft Excel.

You can almost not predict what people are going to do with that. Okay, maybe it's born for doing some form of accounting and then people write entire applications on top of that or use it for project managers. What management or things of that nature. And AI is kind of turning that way. If I look at what people do with that.

It's really things that surprises everyday because. Well, what do you give us some examples?

“So in production right now, what are some workflows or novel things that were not workflows before?”

The UC within Goldman that AI is doing for people today. So let's start from the GSAI system trip. That can answer really complex questions based on external and internal data. That generally before used to take some times hours or even days, sometimes weeks to answer. You can do very complex research for you in topics.

You know, for example, we can ask questions that come from clients such as hey, how does the recent geopolitical events on the hormone straight actually impacted sportfolia? What could be a potential rebalances strategy or you could ask intersection of like I don't know how does the a certain fed decision on interest rate actually impact the volatility of certain assets. So you ask this multi-dimensional questions. What GSAI system does calls out a model retrieves relevant information creates a plan to answer that question.

That's kind of the key because this AI is they really plan before responding rather than just giving you the first thing that comes to mind.

“And so that's what kind of at the very surface thinks that one of the most common use cases which is we really enhance the client experience but being able to answer questions internally and externally in a much much faster way.”

But really complex question, not simple questions. We had to wire up hundreds of data sources and also most importantly, which is something that I tell everybody has me, give me some advice on how to implement AI in a corporation. Data quality is really the determinant between good AI and not so good AI. And so we do a lot of work to not only take a bunch of data but also making it understandable to the AI. So for example, just a little bit deeper. We have a tool called Legend AI which is our lake house which allows you to go from query to MCP server, connected to GSA assistant, i.e. from data to answers, you can wire it up literally in two or three clicks and it does all of that for you.

So the quality of the data, the quantity of the data not only that it is not just the bitter lesson here but it's also the lesson of you need to curate your data. You get better answer disproportionately, that's something that has driven that. So that is kind of the knowledge aspect of AI which is, I would say the most widespread because every single one in the firm has that.

And it's the highest use we are like way above a million prompts per month and it's growing really, really fast.

And then of course, you know, you're asking me like really impacting in production, every developer in Goldman is enabled with agentic AI. Okay, so we were probably one of the first, if not the first to launch a dev in most a year ago, which is the fully agentic developer assistant. We have Claude Cod, we have many other tools, GitHub, Copilot, agent, etc. But on that, you really see the step change. There is no question that there is changing the way developers work. And by the way, it's not just about doing the exact same things more efficient.

It's changing the way developers actually do their work. And that is very, very easy to see how that kind of changes the paradigm of what a developer does. You're much more of a product manager, you're much more of a planner, you're much more of an idea generated.

The most important thing for the developer today is to be able to explain things rather than jumping on other things.

I don't know, you want to know that I'm just going to say that resonates because I've been like Bob coding, but I can't explain how any of it works.

“So someone is like, you know, I like build a little toy apps and stuff, but I get really anxious. I couldn't explain that's why I'm not a software developer.”

Yeah, well, just on this note, I mean, people tend to talk in generalities when it comes to AI boosting productivity or maybe AI changes the way we work or leads to some new ideas. From your seat at Goldman, you're a manager, you're looking at the bottom line of like all these businesses. What exactly is the outcome, the specific outcome that you would like to see from your developers using something like cloud code.

I was looking at some of the reports on some of the deliverables for our cloud migration, which is a very important thing for us. And I was looking at this really big project that was saying, it was not only green. It was like two months ahead of schedule. And I was saying, this is how we know when things are going to work. You're going to consistently start seeing projects that are actually finishing ahead of schedule. Which means, and then people are ambitious, they want to do more. And therefore you end up with output that is much higher than what you had before.

“And listen, with developers obviously, the biggest question that everybody asks is, okay, what are you going to do? Are you going to cut developers like that?”

So, first of all, with all the innovation that I've seen in the last two years or so, I can't have never seen a moment where really people were reducing the number of developers.

Because if I look at the things they were not doing in a certain year, because of budget reasons, because of complexity reasons, because of prioritization, the stuff that is below the cut of the backlog, it's a lot. And not a lot of that is really driving the growth of the business. So, it's good to have the optionality to do it. You have the optionality to say, "Now I have 120% of my capacity. I have 130% of my capacity. Do I want to do 130% more? Great. If I don't, I have the option to reduce." So, that's really how we measure it. It's really the impact on the timelines of delivery. It's output.

It's basically quality and timeline becoming quality gets actually better, and the timelines get short.

So, that's what we measure.

“Obviously, you wanted a big question for the market this year, is what is the impact of AI on legacy software provided?”

Yes, and there's various theories about how they could be disrupted. The reports I think about anthropic having, quote, "for deployed engineers inside Goldman Sachs." So, anthropic employees bailing out AI systems internally, maybe that could replace them like you see, software. Right now, can you say, like, there is a change in the balance of power when there is a given piece of software for negotiation?

I think generally there is. First of all, there has always been that tension because imagine, for example, imagine when the software that didn't run on the cloud,

and then all of the sudden a bunch of new vendors are coming to you and say, "Hey, wait a second. Why are you running that on your mainframe or your on-prem? Why don't you run it on the cloud?" Or remember when software needed to be installed, then everything became browser-based and so on, so there has always been a little bit of a cycle and a renewal. What I say is that today, that cycle of renewal is much faster. That's really what it is. And I would say, I generally resist making like really broad categorizations. AI is, by the way, that's possible.

“To me, it's like saying computers. But even software is very broad. And so within the software category, I think there are winners or losers. There are winners and long-term losers in the long-term,”

but it's really like people tend to make it a category and then maybe throw the baby with the bath water. So here's an example. To me, the question that I asked myself with regards to which vendors am I going to, which software am I going to have, like a few years down the road, is, software generally is attached to a process or a certain ways of working, okay? It does something for you and it puts it in the form of an application that you use. The real question is, is that process and/or ways of working going to be the same? What is it going to change in five years?

Then you can determine what is basically the likelihood that the software is going to be robust to that or not. Example. Is accounting or closing the books going to be very different from now? I don't think so. Really. It hasn't really changed. Everything changes, but it hasn't really changed much. So if you're operating in the general ledger type of category, I don't think all of the sudden you take a GPT or a cloud and it's going to close your books magically. You start to do the countering. You still need to do a lot of that and it's very regulated, importantly, right?

It's extremely regulated, jurisdiction by jurisdiction, country by country, product by product, industry by industry. So that part is kind of to me in kind of the safe mode. And then you go to the other end of the spectrum and you have sometimes software that kind of is aligned to the way people do things today, like software being one of them. Like if you look at the software developer lifecycle, a lot of that is changing. Developers are developing software by developing specs today. And so if you're too much in the weeds down there in that mechanics and you don't adapt for AI's or agents doing their work, it's a software development lifecycle, deployments, rollbacks, monitoring, observability and all that.

Or if you're adding a sort of a UX on things, that's another class. If you have a very simple process, I don't know, you're doing surveys or expense reports or whatever. And now people are going to start expecting, you know, their personal assistance or agents to kind of do all that mechanic for that.

“And so I think that part is probably something that has a bigger question mark on top.”

And so I always ask myself for the process question first, the process transformation section, the question first, and then consequently was the tool that is going to support that.

Just to press on this point though, have you replaced any third party software providers with something that's been developed internally through AI? We have terminated contracts already, yes, absolutely. Okay. No, I'm not going to ask you your follow-up question because I'm going to tell you many secrets from the software. But overall, yes, absolutely.

You know, the thing is like the whole virus is built, okay. The equation has changed quite a bit. Bivers is built. It was always like, okay, guys, how long does it take to do it to build this? And you get an answer, which is where we can do it in like, X amount of years and X amount of millions of dollars. Now I'm starting to see people coming to me and say, by the way, I had some time, you know, this weekend, then here is a perfectly working application. The cost or at least for simple applications, the cost of kind of build from a time perspective, from an actually cost perspective has gone down quite dramatically.

So right now, the little things are most likely going to be built, the very big large software that is to be deployed at scale across thousands of people and etc. And that big complexity, as you know, from we all do toy stuff with our clock codes, our home and whatever. You know, there's still some rough edges, you know, and so it's hard to think that all of the sudden the big applications are going to disappear. And so that's really what what I'm saying, that if I look at the applications that I buy today, there is a lot of small applications.

And so the build is kind of the pendulum is starting to swing back towards the build, at least for that category for sure. What is the forward deployed engineer? I know that's like one of the hot buzzwords of 2026 and I saw headlines that were anthropic forward deployed engineers at Goldman. I have no idea what they mean. Yeah, what is that term? I think what do they do when they got there?

Okay, so I think Lisa, that name has changed quite a bit too far. Yeah. No, no, no, no, no, that's the latest. So you are fully, you are in the real, real latest of that term.

“But remember, I mean, there was a time where you used to call them solution architects, right?”

And so the point is, right now, I think one trend that I see, which is also kind of true for us, is when things change so much and so rapidly, you kind of want to go to the origin of who produces this new thing. Okay, so the least intermediate is you have and probably the faster you can go. And so going and working directly with the model providers is generally good idea because if you're putting someone in the middle, this company is going to have to be trained. That's going to have to be trained, that's going to have to be, you know, there is a cycle, which at this point of very rapid change is going to slow you down.

And so the first thing that that term means is those are people that are actually normally building the product.

They're normally building the cloud or GPT or X or so that's the first differentiation. They're straight to the source of the eye production in a way. And second is that they're generally product people, so people that have actually built those tools rather than people that are more like support and deployment. And so this characterization is you take the classic cell support team or solution support team, which was mostly doing integration. And when things are so rapid, it's like imagine if there is like something like, I don't know, if the cloud style works changes so fast, instead of a fashion assistant, you want to talk to the tailor because they can actually make it.

Things are changing so fast, so those are the tailor's best. So on this note, one of the things we heard in support of SaaS was this idea that while integration is still going to be really important and that's really going to be like the major hurdle for a lot of the stuff.

Have you found that AI is making integration even faster at this point? Has that basically become irrelevant nowadays?

“No, I think integration is extremely important, especially for the industry cause like systems of record.”

So when you do something like, you know, when you do a process, then you have a source of data like your CRM systems could be a system of record or you have your client system or record your accounting system or record. And those, when they become the authoritative source of an answer, they need to integrate with the rest of the firm and the rest of the data, the rest of the application says, so I can see that those vendors that sit on top of those.

So I think those who will evolve so that you still get the same level of automatism and you still get the same benefit of speed.

“But it kind of comes from within, I think that part is probably something that would remain very valuable.”

So in general, I don't have anything against it. Again, I don't have anything against the SaaS category at all overall, but I have, as I said, different opinions of who actually is going to adopt the future and adapt to the future and those want to. You mentioned this idea of people have a few extra hours over the weekend and they come in the morning and they're like, well, I had some extra time and I decided to do this.

What's the coolest or most novel example of something that people basically vibe coded in a limited amount of time that wouldn't have happened, say, two years ago?

So I've seen people doing cloud migrations of legacy applications that were on premise, once they have been enabled with those tools, literally in a matter of hours. I've seen someone build a complete travel assistant for corporate travel assistant that looks at your calendar and looks at the flight delays and look at the book and stuff literally like doing a meeting where they were not paying attention. So those are some of the things that I'm doing right now. Yeah, well, we're doing this podcast. Well, that brings me to exactly what I wanted to go next, which is I'm curious, like, do a large corporations have a token budget the way they would have a dollar budget in the past.

So like, I would love to have unlimited access to coding models and whatever and actually just play around and try to work on that.

So my favorite questions, but I'm curious, like, how you think about token allocation within the firm and whether there's intro firm competition for compute.

Yeah, token allocation could be like included in your performance. Yeah, right. If you do well, you got more to different teams and stuff like that. Whether that part of what you think about for planning. Absolutely. So a few months ago, I did, like, I spoke about predictions for 26 and one thing that I said was it's going to be the birth of the personal assistant and that kind of happened with open claw and all that stuff.

Yeah, yeah, stuff kind of early on. And then the one was this going to be a token sticker shock for CFOs, right, that all of the sudden they're going to start seeing bills that they're absolutely did not expect. Jensen Wong said an interview today, sorry, not today in recent weeks something about like, if I'm paying an engineer $500,000, I hope that he spent at least $250,000 on tokens. Now again, as many people pointed out, that's like the barber saying, oh, you really need to get a haircut every week.

Less, we're talking about some pretty big numbers, a lot more than just like a cloud max plan for 200. Right now, let's talk about that.

“Right now, okay. So first of all, lesson number one is you need to centralize the access to models.”

Okay. So that you can monitor, meter it and then optimize it. Okay. So the wild west of everybody goes and calls an API and starts consuming tokens and then you find out later on is a big problem. And so that's why we built with this GSA platform, which is what's called a model gateway. And the model gateway intelligently routes requests to the combination, the Pareto frontier of quality and cost. Okay. So you've got to centralize that.

It's not the one size fits all because many cases if you're asking, what's the weather? You don't need to call out a cloud opus 4.6. You can ask it to leave an local model that you're on very cheaply on prem. And so there are ways to optimize that way before you start even having the conversation you're consuming too much. Yeah, this is very interesting to me is a big part of the problem that you're trying to solve. And we know like, Chad UBT, they intelligently route. They do some on there. You go to Chad UBT dot com and they'll try to route into the best model and there might even be some conflict of interest because they probably want to route it to the cheapest model, the user wants the most performant model.

But how much of the work of your senior engineers is essentially solving this problem of the right query going to the Pareto optimal model. It is a big part of the time of the spent by the AI central group.

“Okay, platform group. The platform group works a lot about where do I get the right data for example for this question and which model do I write route it to?”

That's a big, you know, because again, I spoke about Pareto frontier meaning the optimization between quality, which we don't want to compromise and the actual cost. And you can be ISO quality at very different price points because not all questions require the most expensive token. So that's point number one. So what I'm trying to say is my philosophy is to try to isolate the developer or the user from the token anxiety.

Is there a little bit like will electric cars, okay? At one point if you have 18 miles of range, you're always optimizing routes and maybe I'm not going to go there. I don't need this ice cream today.

We don't want people to go there yet, at least. Right now it's a time where people need to really find the best way to kind of do more and do better, the best possible work with AI.

That is meaning internally in sort of a central team, optimize it in a way that we're going to make it economic.

“And I think reducing the token anxiety is a big challenge, but I think it really frees up creativity and what you can do with AI.”

It's also like there are certain problems that you don't want to optimize too early, okay? So for example, Yeah, how much time do you want to optimize now for a, remember we used to kind of optimize the way to web pages because they were too slow to load. And then at one point the editor, so whatever say why can't I put the at another image and then people are starting to say, okay, why don't you do it? And then on the backhand, I'm going to work in optimizing your images rather than asking you, at the most you can put three images on the homepage, right?

So that's the approach people right now, I would rather have them err on the side of usage and let me worry about optimization.

And the other point is really at the end, human hours always tend to be the most expensive cost, okay?

And so as long as you're talking cost per hour is less than your wage per hour, that is a kind of a positive ROI. So at that point it's fine. What's your feeling about future costs of tokens and whether they're going up or down? Because you hear different things on this, one of the things you hear is that again, going back to the beginning of this conversation, AI hasn't proved so quickly in the course of months, if not weeks, that those costs are destined to come down.

But on the other hand, we know that the hyperscalers are still losing money hand over fist for power users such as yourself at Goldman. So where do you think those are going over time? My personal view is token cost is going to go down quite a bit, but token numbers are going to go up probably more. And so total token cost is going to actually, we're going to have to accept that it's going to be a major item of cost in any organization. And it's to be compared to the cost of people and not to be compared to the cost of the or TCP IP packets for computer or any of that.

If you look at just a number of tokens being used for the same use case, if you go the reason you're out or you don't go the reason you're out. If you go the agent, the crowd or not the agent, the crowd.

“If you go the open claw route where you know, it checks every, you know, start having these tasks that are firing one after the other and then you have to start to have very fires, et cetera, et cetera.”

So I think the trend will continue with regards to more and more of those, but the cost, the per unit cost of token, I pretty, I'm pretty sure that it's going to go down.

Also because as GPUs are becoming more powerful, they cost per what, hopefully it's going to go down.

And then also like to be fair, I mean, these hyperscalers are doing a lot of optimizations to try to run those stacks on their own hardware, right, which will potentially can also generate some economies of scale. [Music] Can Goldman employees like run open claw on their work computers? And I'm curious like about the degree to which you have people who like want to, I want to install this or this seems really cool. And then think about the security imperative and how you handle that aspect, not the token anxiety,

but the sort of, I want to install this, this is all of this, I mean, as you know, like as a bank or put it in lockdown in terms of what you can install, you cannot install stuff that is not, you know, in the corporate app store in a way. And so there's no way. But you feel it's definitely no way though, because you can't just ask cloud how to install itself. But it's not going to be able to execute, it's not going to be able to create the actual executable, it can actually even GSA assistant today can spit out a lot of code,

but it spits out source code. Now, source code doesn't run executable, so it needs to be built and it needs to be turned into an executable, it needs to be signed, otherwise the operating system is going to refuse to run it, and so it just doesn't run unless you have that. But do you feel an anxiety where startups, they're probably, and there's not a ton of startup investment banks, but they're various been tax and other things, I want chip away parts of your business, and they can run perhaps faster, and they can be a little bit more liberal about what their employees are allowed to do, etc.

“Do you feel like you have to keep a certain cadence of expanding the list of those executables that are able to be run?”

So I'll give you two answers. So first I want to make sure that I answer your first question, which is, we're not using OpenClaw, okay, but some of the properties of OpenClaw is actually having formed the way we are building our agentic platform, okay.

There are three characteristics that make OpenClaw where it is. One is, it's a constant loop, so it's basically what you know in information theory, you can call an observer pattern, it's something that continues to run an observer, so there is that. It's a constant observer, so you're answering constantly. The other one is, it can schedule events, every 7 a.m. do this or I, you know, like a personal life, we all have something like that, it's been the news in the morning and all that, so there is a schedule ability of tasks.

The third one is, you can instruct it to kind of change, it's on behavior, because it has these files dot md files sold dot md the way you, so you can say things like, hey, I would like you to never use this term or please change or change the way you filter news and so it kind of writes it's on software to do things for you without you even seeing what's behind the scenes.

“So instead of letting people install OpenClaw on their computers, what we do is we incorporate some of those characteristics into our agentic platforms so that it does things that are more similar to OpenClaw, so that's a lot of sense.”

The second question was interesting because basically if I read behind the questions, are you asking whether there is a sort of a velocity disadvantage with regards to as versus others. I often say that there is a difference between speed and velocity speed is almost like you have a certain sprint, okay, but then at some point you're going to hit the wall security wall scalability wall there's going to be a bug you don't know what you're doing and it's going to first sooner or later you're going to be hit by that.

Maybe like most airplanes that are in autopilot that can do everything so theoretically you and I could go on the cockpit and for a long time during the flight we will feel pretty good about that right we will drink some soft drinks or your tea.

We might maybe watch some videos we will be very happy.

That is one point one point in time in the flight there's going to be some storm and there's going to be an autopilot disconnect and you and I are going to look at each other. That I'm going to say oh right where's the pilot. Could I just say recently I was on a flight I tell you this that I was going to Newark. We circled three times we tried to land it was during a storm the kept not landing everyone was like starting a good pretty annoyed because we were up there for a while and then the flight attendant comes on and says unprompted by the way we have plenty of gas.

And everyone is like this is a question this is an answer that no one had been saying anyway I'm sorry I didn't ever get really nervous. By the way we have plenty of gas but you know what I was almost fearing that you would say is there a pilot. And then we did land in Washington DC. Oh you did. Yeah from Newark.

No that's not good. But that makes sense.

“So that's why I mean by velocity is really like is like the marathon is sustained speed for a long time in a certain direction.”

I don't think by randomizing that you actually gain velocity you gain an instant speed of some sort. And so I'm kind of optimizing for velocity. But related to Joe's point though you are a regulated bank. Absolutely right and so there are restrictions on what you can do in terms of technology. I'm very very curious what your discussions with regulators are right now because a lot of regulators this is still pretty new to them.

A lot of the models basically are black boxes.

How do you convince them that like they're running as they should that they're spitting out the correct output that you understand how they're actually functioning. So this is not the first time that banks use neural networks okay. These are just much larger neural networks but we've been using neural networks for like decade plus. And so every bank has already gone through the motions of explaining that neural networks don't have perfect explicability.

“Therefore you need to change the control system around them.”

You need to look at what actions can they actually do and then you limit the actions okay. There is functions called the like a model risk management which is a very standardized you know function within every bank. Therefore each of those neural networks you need to have an inventory you need to have a risk tiering and you need to put controls around them.

So it is not really that much of a new thing is more of an evolution where now you have things that are much faster and much more powerful.

But the basic pattern and the basic discussion with regulators is kind of the same which is. Are you classifying the risk tiering of the application right? And then which controls are you putting and are you putting human supervision and human in the loop. So for example for code. We don't allow AIs to auto approve the wrong code okay.

The same way as a developer would do.

“And we kind of have a sort of a zero trust model there because we don't assume that it may be a junior developer is going to be less more bug free than in AI right.”

And so we have several controls in place for example. There needs to be a human more senior than you that actually looks at the code and then certifies and approves. And then we after that before it goes to the production and goes to something that is called CICD or continuous integration continuous deployment pipeline where when it goes through the build phase etc.

There is a lot of checks that are injected into that there are security checks there are tech risk checks.

So I don't think at the end of the day you really lose too much velocity or at all you just need to invest more in those kind of things. And the regulators I think if you bring them back into sort of a familiar territory and you're also honest on things that you know and things that you don't know and for the things that you don't know you kind of put higher protections I think the conversation is generally very positive. We have an episode that we recorded several weeks ago that we still haven't released. I don't know exactly the timing of that one of this on we interviewed Scott Bock the former CEO of Green Hill the boutique investment bank and part of the reason we had that conversation was because we want to know like if AI is going to someday disrupt bank is we know like what was banking as we know it's we're talking about the history of investment banking.

But one of the things that he talked about was that a big advantage that the banks had was this sort of information asymmetry and that they would know a lot more about their industries and so forth than their clients and this was profitable. Now going back to your answer the very first question you're like okay a client might call goldman and they said what is the straight of our move closure mean for this portfolio shock etc. I was going to ask this exact question yeah. I kind of think I could do that.

“I think I could I mean no fence I'm sure your platform is a little bit better than what it like but I think I could like at 90% the way there and I bet I could like with a little bit of data.”

I could build a basket that says I want helium shortage basket which companies would I short if I think the helium shortage is going to get worse I could write build a basket the way a trading desk would.

I think that AI erodes a certain structural source of profit. I think you can get to the 90% but I think clients are really paying us for that extra 10%. So I think that's the answer.

So what is the extra 10% in that context is it the models are slightly better or is it also the basis to you know we buy a lot of data that you know is very expensive and it's a massive quantities and it's very up to date and very real time. So we have a slightly little bit of a data advantage. We operate across multiple asset classes so we see the trading side we see the asset management side. So we have a sort of a correlation between assets advantage that we see those because generally rates move interest rates can move yields can move you know there is a correlation between all those indicators so there is another advantage.

We have a global advantage where people on the ground in a hundred plus countries and these people have relationship and information travels through those channels. And also you know like we generally deal with very complex portfolio so this is not UNIME we have in three stocks or four or five this is like very complex multi assets with complex products like swaps or swaps or exotic products etc etc. And so that's really the 10% that the clients that we have really value and where we really need to get is like at the end of the day listen look a formula one okay.

The difference per time per lap between the Mercedes and take you know your favorite last team it's sometimes one second out of two minutes and that is the difference between you know getting a hundred million dollars a year sponsorship or a ten thousand dollars sponsorship. So for sophisticated clients that 10% is really worth the money is and that's really what people are paying as well. So actually you mentioned all the different businesses at Goldman and there are a bunch of them like asset management there's banking there's trading.

A lot of those businesses aren't supposed to talk to each other in various ways and so when it comes to the data is there like a data leakage issue where you might have a model that's in house like GSAI. That's pulling data from different sides of the company in ways that maybe it shouldn't be maybe it's really hard to tell given the complexity of the model.

“Is that something you have to pay attention to absolutely so we have the concept of info barriers okay and info barriers are are enforced throughout the entire system okay.”

And they're linked to your idea or your account okay so if I'm on the private side I can only see certain information if I am on a public side I can only see certain information and I cannot even.

And that badge is attached to the exact same info barriers as any application or any computers and so this I enforce the basically at the source so even if it is the same type of model.

That particular use of the model that particular session of the model then needs to get a ticket or a badge and that badge or those keys just take them to sort of place and so this is one of this been. But it took us almost two years to build a GSAI platform these are this back to the reason why you can't be casual about this thing this thing has not been built by some random vibe coders because you need to worry about cyber you need to worry about info barriers you need to worry about all that and so.

“But there are places where you can leverage and do correlations but there are others where you absolutely can and this kind of that is foundational to the fact that you need to be ready for you can't be casual about that.”

So I take your point that there's never been a technology that you've seen in your career that is actually reduced the need for software engineers and that the nature of the job of software engineers change them maybe gets more high level and whatever. Setting aside that volume question setting aside the pure head level of head count question is AI changing right now across anything technology or otherwise. The types of person you're looking for or changing something about the nature of the type of talent your person yeah absolutely great question so I think in this day and age almost nobody.

“Is an individual contributor really because when you're working with agents. You need to have at least three fundamental characteristics one is you need to be able to explain what you want to get that.”

The second one is you need to be able to delegate work guess what because you're going to have multiple agents one specialized for example in doing addon or DCF calculations and one is specialized in doing research so you need to be able to break down the work into chunks that can be executed in parallel in some way and then three you need to have the ability to supervise you need to actually look at the output and say okay I'm good with this or go back. It turns out that those three things I explain delegate and supervise are kind of the one of one of managers managers need to have those yeah why they can't manage a team.

And so AI is kind of turning everybody a little bit into a manager and those are kind of the skills that we are actually looking for people that they know that they're going to have agency on tools that at some point are going to be even more proficient and specific specialized than they are.

And so the most important thing is really the ability to ideate to explain to delegate and then to really know what good looks like.

And I think that is a big change and I don't think everybody is going to actually rapidly go through that and I think we're doing a combination of training there is a combination of exposing them to other people like one of the advantages of having forward deployment engineers is also that there is a little bit of clash of culture that is happening on the table and so people think really really differently. And that pushes people outside their comfort zone. That's why I'm saying that there is a little bit of a metamorphosis happening there is not just about the efficiency is really thinking about is my job going to stay the same now is actually changing quite a bit.

I'm thinking how to frame this question but what's work life balance like now for a developer at Goldman because you have this existential angst about jobs potentially changing at the same time you have AI tools that enable more productivity and you also have this thing happening where I feel like. Joe maybe you know more about this than I do but I feel like a lot of vibe coders like it's addictive right it's like you're pressing the button of a slot machine you're interacting with cloud and you're seeing what it spits back out over and over again until you get that big win.

And so I've heard people talk about burnout among developers who are just doing so much with this right now that they're just hitting that button over.

“There's a good discussion in the odd lot discord recently about exactly this some engineers and semiconductors feeling that the job has become less satisfying and I think it's sort of what you're getting at the sort of slot machine.”

Well it's like oh you're getting like hit the prompt okay this is the great output then it's like they feel the work is like let's satisfy and stuff like that and actually like writing code so yeah I mean listen again this work and the fact that I'm a little bit older than most here engineers I kind of I've seen that the first time people had like sell. I've said for the first time people had python oh my god I don't need to know Java and then the kids start to code and there is this old coding movement and then you get to you start creating your applications.

I've seen the first time people have mobile stuff and you know mobile apps and so I think a little bit of that is because it's new to be perfectly honest and I think yes there is a little bit of that.

Is that because maybe of that but also because of what you can get the reason of sort of in a way reward cycle that is pretty quick yeah. People are very excited actually there's there's some sort of a joy of the profession that is actually coming out as if engineers were feeling like this job is new again because a lot of engineers have seen the same patterns sometimes for four to three decades.

So there has been something that I have served there is also a lot of peer pressure there is a lot of fear of missing out and so people rather than is no longer.

Me trying to push the car up here is more people are actually looking at their peers and they're looking at oh my god how could you do that and so it's kind of spreading a result of quite a bit which is really nice to see. And so so far I have to say that has been positive positive change and also one other thing that we're talking about burnout. I see that a lot of people get fatigue I don't want to talk about burnout but they get fatigue when there are a lot of repetitive tasks especially for a developer here's an example.

“Let's say you go from a version of a Java library or spring boot to another version and then all of a sudden you compile and you get or you build and you get all this errors that says you need to upgrade.”

Lastly upgrading libraries is not the most fun draw and if you need to do it 100 times or it's like someone says by the way guys we have this new design new log or new colors implement it on like 200 websites it might be fun the first 10 and then it becomes a drag.

And so I think taking that away kind of they focus more and more on the plan for example.

And so right now let's do a migration plan to the cloud of a complex application they spend maybe 70% of their time going back and forth with the very powerful set of the eyes to really get the plan right. They feel a little bit more elevated and the mechanical part it's kind of left to to the machine the same way I mean listen I started developing when I was literally flipping switches okay and then pressing a button that we should move the register up one. And then came some languages they would like see oh my god now I don't have to flip switches anymore but the guess what I need to do memory management I need to do pointers I mean it is a lot of heavy lifting.

I have a memory leak I'm gonna spend a week before I actually finally identify that and then it comes Java or garbage collection I don't have to worry about memory leaks anymore I'm fantastic and then comes Python which is all that rigidity also much easier to be type free and so forth and so every time you can keep raising the bar and a lot of the kind of mechanics can go the way I think.

“This has been like a 10 years jump in a metro two years but I think overall nobody really likes to have that toy and that mechanical work and I'm actually quite happy that people are gonna spend.”

Maybe initially more time because they're excited but don't think that I'm enjoying rather than things that this is they dread. All right well market we'll have to have you back on the podcast in another year and a half I guess in just a few months on the reduced AI timeline thank you so much for coming back. Thanks for having me thank you so much thank you. So Joe that was great to catch up one thing I thought was really interesting was his point about the discussions with the regulators and framing it like very similar to previous technical advances where you're not necessarily explaining.

“Exactly how the models are coming to certain conclusions yeah but you're more focused on actually limiting the risks and making sure that they're in the right bucket for assessment.”

No I thought that was really interesting just that some of these technologies the black box yeah LLMs are not the first black box sure I mean we've actually been talking about black box trading for years in finance before.

So the idea of like okay though these things that are happening we can articulate them and whatever you're not the first rodeo for finance is really interesting.

I'm also you know I thought the whole conversation about I'm token budgets and allocations are interesting the idea of like okay part of the job ears you have a bunch of different models everyone in theory wants the most performant model. But how do you find that optimization where you get the best performance relative to price. I felt like a pretty interesting like engineering problem. Yeah I would actually love to do more on that question because it's such an interesting question of incentives right and like how does how do you actually like prioritize.

How do you know what constitutes a good output and how do you sacrifice a little bit of quality for like 10x less token budget or whatever like you said there should be it would be very interesting to talk about how that problem specifically gets solved in the type of an order.

I'm Jill Wiesenthal you can follow me at the stalwart follow our producers coming Rodriguez at current arm and national Bennett at dashbot and kill Brooks and kill Brooks and from more odd labs content good up Bloomberg dot com slash odd lots of a daily newsletter and all of our episodes.

And you can chat about all these topics 24/7 in our discord with fellow listeners discord gg slash odd lots.

And if you enjoy all thoughts if you like it when we talk to Goldman Sachs about how they're actually deploying AI across the company then please leave us a positive review on your favorite podcast platform.

“And remember if you are a Bloomberg subscriber you can listen to all of our episodes absolutely at free all you need to do is find the Bloomberg channel on apple podcasts and follow the instructions there.”

Thanks for listening. (gentle music)