Scaling Laws: Google's Kent Walker on Regulating Frontier AI

So, the new Bundesliga series, as well as the future, is a great mental and sport director.

“Because all games are then in the same way as the KickBass series.”

And if you're now watching KickBass, what's that? KickBass is the fourth fan of the Football Manager.

But that's the first thing I'd like to say.

You're a little bit more ignorant, you can even read the rules. Or you'll see the real Bundesliga profile in your card. So, this time, you'll see the difference between the two of us. The Countdown Loft, the Bundesliga season on the day, is an August. It's a great honor to have a group of friends and supporters.

KickBass, just download and leave. Good kick and fun. It's the law for podcast. I'm Alan Rosenstein, associate professor of law at the University of Minnesota. I'm Alan Rosenstein, associate professor of law at the University of Minnesota, and senior

editor and research director at Law Fair. Today, we're bringing you something a little different. An episode from our new podcast series, Scaling Laws. It's a creation of Law Fair and the University of Texas School of Law, where we're tackling

the most important AI and policy questions.

From new legislation on Capitol Hill to the latest breakthroughs that are happening in the labs. We cut through the hype to get you up to speed on the rules, standards, and ideas shaping the future of this pivotal technology. If you enjoy this episode, you can find and subscribe to Scaling Laws wherever you get

your podcasts and follow us on X and Blue Sky. Thanks for listening.

“When the AI overlords take over, what are you most excited about?”

It's not crazy, it's just smart. And just this year, in the first six months, there have been something like a thousand laws.

I'm actually building the scaffolding around how it's going to work, how every day folks

are going to use it. AI only works if society lets a work. There are so many questions have to be figured out, and nobody came to my bonus class. Let's enforce the rules of the road. Welcome back to Scaling Laws.

The podcast brought you by Law Fair and the University of Texas School of Law that explores the intersection of AI, policy, and of course the law. I'm Kevin Frazier, the Director of the AI Innovation and Law Program at UT and a Senior Editor at Law Fair. Today we're joined by Kent Walker, President of Global Affairs at Google and Alpha.

Google has proposed a new model for governing the most advanced AI systems, a frontier AI regulatory organization, or thorough. The organization would be privately funded, staffed by technical experts and overseen by the federal government. In many ways, it would look like finra.

And does that a good thing? Can a finra for AI meet the challenges posed by Frontier AI in line of recent events like the Hugging Face Open AI incident?

“And perhaps more pressing, would a Frontier AI regulatory organization even be constitutional?”

These are big questions that demand immediate answers, which is why I'm so glad Kent joins Scaling Laws. To get in touch with us, email [email protected], or follow us on x or blue sky. And with that, giddy up for a great show. Kent, welcome to ScalingLaws.

Kevin, it's a real pleasure to be with you. So from the outset, I should disclose that about a decade ago, you were my skip, skip, skip level boss when I was a legal assistant at Google. I don't think we ever cause paths. In fact, I think if I had seen you in the hallway, I probably would have dodged you out

of fear of, oh no, did I do something wrong? Did I disclose some data I wasn't supposed to do? So it's nice to connect a decade later. Well, I hope you had a good experience at Google. It's been delighting the life of the watcher career since now.

No, it's not. That's that's very kind of you and you know, those who can't do podcast. So I'll leave it to listeners to decide whether that's fully accurate. But in that decade, a lot has changed. For one, Google continues to spell out high level thoughts on AI governance.

You all were way ahead of the crowd in 2018 in spelling out some AI principles for how we should think about this new technology. And ever since then, you've been continuing to try to help policy makers grapple with this technology. Most recently, you all announced AC self-proclaimed pragmatic approach to AI governance in

And we're going to dive into that proposal soon. But I want to start at a high level to be blunt, why should tech policy observers treat this proposal any differently? As I noted, you've been spelling out principles and policy ideas for a decade.

“What's different about this moment and this proposal when it comes to taking this seriously?”

Yeah, it's the right question. We think it's time to break the stalemate. If there have been a lot of talk about no regulation, there's been talk about heavy regulation. We do think there's a middle path through there that at a time when we need to build trust

in confidence in these AI tools and the tools themselves are getting increasingly powerful.

There is a pragmatic approach to a light touch regulation that still pro-innovation, that still allows American technology to lead the world. Does it have an evidence-based way and creates standards and testing protocols that people can have confidence in? How much weight would you say Google is placing on this specific proposal?

Again, we're going to get into its two prongs. But we know that right now, there are wild ideas floating around the hill in light of this hugging face, open AI incident, where we've seen loss of control suddenly become at headline news, whereas for many folks it used to be a sort of hypothetical scenario. Now we know that models are breaking out of their harnesses, breaking out of these testing

systems, and perhaps being even capable of hacking fairly sophisticated actors.

So is this a real pivotal moment for Google because if you all don't lead with this sort of middle path approach, talks of kill switches, talks of government ownership of AI companies, these are now policy proposals that you can hear hanging out at a Starbucks in DC, which even a few months ago would have been ludicrous to think about. But the rate of development of these models, the increase in capability, certainly shows

how high the stakes are. At the same time, we really need to start in a grounded evidence-based place. We don't have consensus on what the standard should be, what does good look like for these tools.

“So one of the key elements in the proposal is to come together to have technically informed”

about just benchmarks, evaluation suites, etc. that would more if it would grow as the technology

evolves, and that would be reassuring to people, and across the industry would set a good housekeeping seal of approval standard for how to do this right. We think that would actually help us land that tremendous benefits in AI, the trillions of dollars in economic progress, the advances we're seeing in science, in biology, and energy, and so much more, but grounded in a way that actually well integrated into some of our existing

legal frameworks. So this call for a mental path starts with a frontier AI regulatory organization that you all say might look something like a Finra for AI, or you even mentioned state bar associations as a potential model of this kind of independent organization working with industry that has the government as a backstop.

But what does this mean in more detail, how would this Farrow work out as you all imagined? Sure, the proposal talks about two different types of regulatory structures. One is that the Farrow, the frontier model approach, and the other is one that I'm sure we'll get to in a moment with regard to widely deployed AI applications that people are already using every day.

At the high end at the frontier end, we've talked about a variety of different analogies that have been used in America for a long time, whether it's Finra or the PCAB, the public company accounting, oversight board, or less known groups like there's the North American Electric Reliability Corporation, which is supervised by FERC. In all of these settings, you have the private sector coming together to help set standards

and norms for an industry, still with federal oversight supervision.

“I think it's important to have private participation this because the tools are moving so quickly.”

That would allow you to have industry funding and allow you to actually hire computer scientists that market rate salaries, so you're bringing in the best in the brightest to help in this really important work. It also allows a certain nimbleness because it is moving so quickly, sometimes if you write something in the law, you're stuck with that for some number of years.

Here's your seeing models, advance every few months. So you need to have that ability, almost like an underrated laboratory where you can go in and assess and confirm that given models are meeting the standards that you set forth. So I want to attack this from two different perspectives. I'll give you the version that says, Kent, you're not dreaming big enough, right?

We just saw this instance of a loss of control.

sure that their models work as intended, and you're thinking, oh, let's just go for a

thinner-out for AI, we'll kind of do a copy and paste scenario. Don't we need something more immediate?

“Don't we need something more drastic, isn't this the time for a kill switch type intervention?”

Why not go that far? Yeah, I think the best approach is a complementary one. We're not suggesting that this would replace testing by classified government agencies, the national labs, et cetera. It would compliment that kind of work.

We're going to make sure that we're looking at the AI risk, the chemical biological, radiological nuclear cyber risk that have been talked about, and that are starting to emerge. But at the same time, maintaining the flexibility to be able to test in different ways for a whole variety of different areas. And that kind of later touch approach is what will allow America's models to continue to

be at the cutting edge, and that's going to be critical in this global competition that

we're in right now. And thinking from the other side of the spectrum, then, which is to say, hey, this is a great idea, but it still may have some issues from a institutional capacity perspective, and maybe this is just another instance of big tech trying to create a regulatory mode that doesn't have some degree of public oversight.

We know, for example, there have been concerns about finra and about other organizations that they've become too detached from the government. Over time, I've been nerdy now about finra and the SEC. Over time, finra has become more and more closely tied to the SEC. And the SEC has clamped down more and more on just the extent of independence and expertise

that finra is actually able to execute. So might this just eventually devolve to the government creating some new agency that slow and bureaucratic and sclerotic, how do we get around that? Yeah, I think this is the right set of questions to be asking, and we should learn the lessons from all these different public private partnerships or regulatory organizations

have been set up with regard to composition, the board, tools, incentives, the core element

“here, though, I think is important to keep in mind.”

And that's the notion that you really want to have evidence-based scientific standards that are evolving at the speed of the technology. And the best way to do that is to bring in people who are really at the cutting edge of this work. If we don't do that, we're going to be stuck with standards laws that don't evolve fast enough

to keep up with the new tools, or we're going to be so risk-preferring that we don't build trust or so risk-averse that we fall behind in this global race. So we think that there's a middle ground approach, figuring out exactly how we integrate that in with federal testing for some of the most extreme risks, and the existing infrastructure for widely deployed AI.

That's going to be something that Congress is working on, I know the administration is

working on, so it's a really critical area to make sure we get right.

So building off of this idea of the difference between a frontier AI regulatory organization and then widely deployed AI. Number one, how would you spell out the difference between what should qualify as frontier versus this notion of widely deployed AI? Yeah, so this is part of the standards setting.

We need to figure out a classification regime for capabilities of these models. Too often laws have just fallen into if you're larger than X number of flops in your training, you're subject to one regime and if you're smaller, you're subject to a different regime. Most people would agree that's not a very good way to slice and dice because we're seeing more and more smaller models that are increasingly capable.

We really need capabilities based assessment to get this right. And if it has certain capabilities, it moves into the frontier AI regime where you have the security of government oversight, the consensus of the standards body, and the speed of private sector expertise. If not, if it's working with more traditional classes of issues, where you're worried about

“important questions, like, how do we keep kids safe?”

How do we make sure information integrity is maintained? How do we think about the evolving future work? Well, there we have existing bodies of law and regulation that can actually be important foundations for additional work, and we need to update those, but we don't necessarily need to reinvent the wheel and create an entirely new regime to deal with those fairly

traditional questions and old line and new bottles. So you all break down this widely deployed AI kind of framework into four different categories. We have workforce preparedness, we have protecting kids and families in digital work. We have modern energy infrastructure and data center ecosystems and provenance and information security.

These are massive categories that are in many ways pretty distinct and novel from some

of the other challenges we've faced with respect to digital governance, but there's definitely a rhyming function going on here with respect to helping folks prepare for a new economy, with respect to protecting kids in particular. And so to what extent, just to again, put on a bit of a skeptics hat here, folks say, all right, it's Google again, saying trust us, we're going to figure out how to deploy this

“technology. Well, at the same time, I think I can go pick out a copy of the Atlantic from”

a year or two ago, and I'm sure I would find something about how Chromebooks were poorly diffused across the U.S. and there were educators who felt like they didn't know how to use those books. And that's led to as Jonathan Hight would say, a real suffering of attention and a real inability for kids to connect and grapple with some of these technologies. This is a long way of saying, why should we trust this approach now? Why, what is it that Google has learned

from prior iterations of trying to govern its latest and greatest tech that suggests that this widely deployed strategy is the one that makes the most sense? Yeah, we very much want to engage with a broad ecosystem of participants on each of these issues. And they are, as you know, very different. It's important to have horses for horses on some of these things when you're approaching the future work. It's a different question than how do you keep

kids safe online and different questions from how do you think about the energy grid of the United States? So we can work though with the existing learnings we have in each of those areas. We don't want to fight the last war. There are new challenges that are raised and no opportunities are created by these AI models. In many ways, we focused on AI models in the form of chat bots, but the biggest breakthroughs we're seeing are on the scientific side. We're seeing this AI is not just

a scientific breakthrough. It's a breakthrough in how we're making breakthroughs. So we're starting to see

AI design drugs coming to clinical trials in the course of the next year or so. We're seeing material science advances. We're seeing quantum accelerating. So we're going to have all of this

“very important, very fundamental progress that's going to be happening. But with any form of progress,”

there's disruption, there's change. We've learned over the years we do need to engage with the stakeholders to make sure that you get that right, that you bring as many people as long as you can. I'm happy to go into the individual areas where we can say, you know, here's some ideas we have for the next steps. Nobody's got a silver bullet in any of these areas, but we do believe that we have some great foundations to build on. And if we do it in a collaborative way, we can get to

a solid outcome, a outcome that helps build confidence and trust and optimism in these technologies at the same time that's continuing the rate of progress. And it is, I want to stress. It's important to have that spirit of optimism. Optimism is a strategic advantage when it comes to any new tools. Without optimism, you don't invest. Without optimism, you don't adopt the new saying. You had to lend to a defensive crouch. And sadly, in the first time in my career, I've seen this,

America is the least optimistic country in the world when it comes to AI. We recently surveyed more than 60 countries and China, Singapore, many other countries are quite optimistic, are applying these tools very broadly. The United States trails. And we need to find ways to turn that around. And we think we can do that by engaging in each piece of different fronts.

I share that concern because when I talk to my students, some of them will say, I've never used

an AI tool and I never want to. And to me, that's just really tough to grapple with because as you pointed out, while there are certain negative use cases as with every tool, the sheer number of possibilities that you can unlock when you properly harness AI are just huge. And that pessimism is really hard to tackle, especially because so much of it is cultural. And so I wonder how we can begin to frame some of these regulatory paradigms in a more cultural context such

that people get away from the big tech equals bad, AI equals bad. And instead, see this as a means

“for them to contribute. And I think that the idea of something like a faro is really interesting”

by virtue of having the opportunity to bring in new and novel stakeholders to the governance process whereas right now I do feel like a lot of folks feel like it's happening behind closed doors. Yeah, I mean, we think it's very important to make sure that AI is delivering value for everybody in the country. And I tell people you've been using AI for 15 years. If you've used Google search or translate or Gmail or maps, you weren't getting into traffic jams. You were finding information

all AI behind the scenes. And so now this next generation of gender of AI needs to also deliver.

It needs to lower drug costs. It needs to create batteries that have more capability. It needs to help us with our electrical dreads. And do a lot of that. And we need to make sure people realize that it's AI behind the scenes in the same way, making life better, helping with living standards, helping people with with real problems that they're concerning. If we can come up with it, you literally cure for cancer. And people recognize that that's a result of American ingenuity,

which is another way of thinking of AI. That I think will start to turn this around. And then you compliment that with a regulatory structure where they feel as though there are appropriate checks and balances. There are limitations on things going wrong. But when you get it right,

as you see with you know, waymo cars in the street, people love waymo cars. Once they have a

chance to ride in them and they realize that they're actually safer than a car driven by a human.

“So that kind of learning by doing for all of us, I think is going to be key.”

It's one of my favorite tourist activities in Austin. Anytime someone visits, we go get some breakfast tacos. We hop in a waymo. And it's instantly the best morning they've had in a long time. But I do want to stress that we need labs to really embrace that idea of showing the positive AI outcomes that are achievable with this investment in such a big technology. Because right now, when again, when I talked to my students or I talked to AI skeptics,

it's the idea of, "All we're getting is AI Slop." And for what, you know, we have these big data centers. We have all these potential privacy threats. We have all these new cyber security threats. And what are we getting from it? And so I think that storytelling notion is so important.

But to go back for a second to the faroe and the idea of needing something that allows for

a harmonized approach, the world over. Something that's unique about Google is more so than any other tech company. You all have multiple tools with billions of users around the world. And one thing that you all flag in your proposal is some degree of aspiration that the faroe could become a sort of standard setting organization that could result in reciprocity agreements with countries around the world. Now, to what extent given our geopolitical moment, do you think

that's feasible? And to be more concise, our folks ever going to trust a regulatory regime that's so grounded in the US. Do we need to be thinking about an international scheme first and then

“working backwards to the US? Well, I think you can pursue parallel tracks. It has been good to”

see the recent announcement that the US is going to be engaging with China around this secretary Besson who will be engaging with his counterparts and to run up to a summit that's going to be happening this fall. But from the faroe perspective, we do see that it has deep roots in international standard organizations, which are transnational, which do have the ability to bring along people from different countries. Now, most of the leading AI companies to date are American.

And I think there's a strong role for American leadership in a lot of helping set these standards in ways that are broadly accessible and available to models around the world. But if we can do this, if we can have a proof point of how can the leading labs come together, align on some of these areas, start to expand from that and build from that core, I think that's a winning model. And then if you can get other countries to also sign up and support them more broadly, and this becomes

sort of a standard of excellence around the world, that's that would be a great outcome. Now, Google has heralded for starting this whole transformer architecture, who knows if we would have ever gotten to this point without the innovation that you all made possible. Is there a fear that having something like a faroe that has clear regulations, clear standards, and yes, they may be evolving and iterating over time? Is there a concern about path dependence here, right? If we see that faroe

is generally regulated or including the major labs today and/or their former employees and folks who are adjacent to those communities might we unintentionally foreclose some of those innovations that we want to see just because we have a regulatory regime in which going down the middle path is the easiest thing to do and taking those risky bets is suddenly something that looks a little

“legally risky. I think it's very important a guiding principle here is that we should be looking at”

regulating outputs not inputs. You don't want to micromanage the science. There may be a lot of new and different ways of accomplishing something of getting great outcomes in a responsible way that minimizes the risks of cyber attacks or other sorts of harmful risky behavior. You don't want to prescribe exactly the way of achieving the goal. The objective would be to get alignment

to understand how much perfection are we going to demand of these tools. It says 99.999.999

while recognizing that they have remarkable benefits on the other side as well. So those kind of

“democratic value choices will be really important. But once you align on that in the same way”

the laws align on the core notions of you want to make sure that citizens feel safe and you establish property rights and you have all the others of core agreements of a society that make society work. You have free expression, United States, and the like, that's the starting point.

And then you free the innovators to figure out, okay, within that environment had one maximized

for the best outcome, the most productivity, the biggest scientific breakthroughs and the like. Well, I'll just say to you, Ken, and to the rest of the Google team, keep the data coming. You all recently released your Atlas report activity task landscape and adoption study. A wonderfully long acronym that actually comes out pretty well. And this is just a deep dive into how folks are

“using Gemini and giving key insights into some of these risks, but really into some of these incredible”

benefits, one benefit in particular, folks using Gemini to look into civic engagement opportunities. And folks finding novel use cases at home and at work. And so keep that data coming so we can have more informed policy making. But before I let you go, Ken, I need to know. Let's imagine we've got some folks on the hill right now listening and their boss just came and said, hey, this kill switch proposal looks really interesting. I'm scared out of my mind from all of this

hugging face open AI news. What's the reason to take a deep breath and to not result into panic right now? What's your final message to folks who are maybe considering some some hasty action? All right, I think the goal throughout the industry and our capital is to build safe models, not to have the break last moment or even to get to the break last moment. You want to build safety, responsibility by design into these tools, even as you allow the the innovators to

create new generations of tools, not to lock things up with a small group, but to have open innovation.

“And we think the best way to do that is a clear set of standards with the ability to test”

and validate and to base all of that as you elude to in the outless report on a solid base of evidence. So we understand what the trends are, what are people using these tools for, what are the bad use cases, and what are the many, many good use cases, and how do we optimize the letter? Well, Ken, we'll see if the folks on the hill hear this message. Thank you so much for joining scaling laws. We'll have to leave it there. Kevin has been a real pleasure. Thanks so much,

Richard. scaling laws is a joint production of law fair in the University of Texas School of Law. You can get an ad-free version of this and other law fair podcasts by becoming a material subscriber at our website, law fairmedia.org/support. You'll also get access to special events and other content available only to our supporters. Please rate and review us wherever you get your podcasts. Check out our written work at lawfairmedia.org. You can also follow us on x and blue sky.

This podcasts was edited by Nome Osband of Go-Rotio. Our music is from alibi. As always, thanks for listening.