[MUSIC]
>> Welcome to the AI Energy and Climate Podcast.
“A special series from the DSR network hosted by David Sandalot,”
inaugural fellow at Columbia University Center on Global Energy Policy. Join us as we talk with leading experts to explore the intersection
between these critical issues that will impact the future of each and
every one of us. [MUSIC] >> Hi, I'm David Sandalot. This is the AI Energy and Climate Podcast. Several months ago, I saw a new story that caught my eye.
It was about a partnership between three companies. In video, the Pulti Group, which is one of the nation's largest homebuilders, and a technology company called Spam. The partnership is begun installing many data centers at homes around the US. The idea is to use existing electric interconnections and
excess power capacity in homes to build distributed networks of computing capabilities and pay homeowners for this service. Now, there's a lot of innovative activity happening at the grid edge. These days, including with virtual power plans, speak, or the grid charging, and much more.
This struck me as one of the most innovative. One can imagine this partnership helping address three problems at once.
First, the long lead time status center developers are facing now to secure
electric interconnections. Second, the enormous community opposition to large centralized data centers. And third, rising electric bills for homeowners could be a triple win. At the same time, one can imagine a number of barriers to launching a program like this, at scale, including challenges with fiber connections and more.
So I was thrilled when Archrow, the CEO and founder of Spam, agreed to join me on the AI Energy and Climate Podcast for a conversation about his innovative partnership within video and the Pulti Group. Arch founded Spam in 2018, after Spending Time at Tesla and other companies, and graduating from Stanford.
He's a visionary and a number of respects building a company
that's transforming grid edge architecture.
And he may be on to a big idea when it comes to data centers. I hope you enjoy our conversation. Archrow, welcome to the AI Energy and Climate Podcast. Thanks for having me, David. Well, thanks for joining us.
I'm really looking forward to this conversation, because I read about your very interesting announcement within video and the Pulti Group, a major home builder month or two ago, and I've been looking forward to learning more about it ever since.
And as I understand that the idea is to create a network of small computing though, it's a kind of mini data centers that homes around the United States.
“So tell us, what's your product and how's this going to work?”
Yeah, absolutely. For a long time now, Spam has been solving the digitalization of the electricity grid problem. So at its core, we've been building a router for electrons if you will, that helps us increase the utilization of the existing grid.
So a couple of years ago, I was thinking about what is the biggest driver of load growth for us across the country. It's undeniably compute. And compute is more a thing from being compute for training, which is a cost center to build models, to compute for inference,
which is a revenue center, where these models are being applied to deliver inference value, agentic inferences, physical AI, et cetera. And that inference doesn't have to look the same in terms of how you structure the compute. Doesn't have to be large contiguous racks of GPUs, running proprietary algorithms in a large shell. Most of us run tasks that can be served by small clusters of GPUs.
So if you take the power side of the equation that we've been solving with Spam,
“and try to patch that with what I think is this burgeoning demand for inference compute,”
the answer is extra. Distributed compute that is purpose-built for inference that can leverage existing power infrastructure using our digital block control systems. And so you're putting GPUs, but it doesn't have that much idea to understand it. At home and maybe just some of the basics for people so they can help visualize this. What does your unit look like? Does it make noise of the generate heat?
Yeah, so really think about the device as being a slightly larger than an air conditioning unit size appliance. It's taking all of the key elements that you would see in a large scale data center, and miniaturizing it into a purpose-built device that has a condition shell, has power monitoring and power controls, has resiliency, has high bandwidth networking with fiber connected directly to it,
A system with a Barack of GPUs and CPUs that can do the inference work.
And we've also done quite a bit of systems integration to directly pair that with a liquid cooling system,
“a plate-based liquid cooling system that is also self-contained with a quiet heat bump.”
So in a traditional data center you would have, let's say a 42-u rack, which is like a bit of roughly six-foot tall rack, with air cooled servers running CPUs and GPUs in them, with an area of fans, high-speed fans that are trying to move air through it. Those tend to be very, very loud, and you didn't care about that noise when you were in a large server room, if you will, or in a large data center.
Obviously this is a problem that needed to be solved in our solution, because we're going to put this in the side of your home with a side of a commercial building, and we've done that by essentially taking that thermal mass away with conductive heat transfer over the liquid, and then having one large fan that spins rather quietly with a heat bump. So it's less than 60 dB in a meter, if you want.
And not surprisingly, computers also move into a model where we're having directly put cooling enhances both the life longevity of the GPUs and the performance of the GPUs. That's a natural sort of intersection of technology development in both those areas, that we've been tremendous in extra. Yeah. So it's fascinating. So to talk about the power system aspects
“of your image, how much power does it draw in more?”
Yeah. So we've probably talked about a couple of tiers of our product that we're building, so you can think about each unit as having a defined power on-lib for IT load, which is the GPUs doing the compute work, and then cooling load and the balance of system load if you will, that's managing the system. In a nominal residential setting, especially in a new home construction setting, the incoming power is around 200 amps of power. That's the conductor sizing
if you go. From our fleet of span homes, even fully electric homes, we've seen empirically that there's about 80 amps, roughly 19 kilowatts of unused capacity in the 200 amp conductor
capacity viewer, that's almost always available. Right. So the data that we have from our fleet
is virtually all of our homes, 200 amp homes, approximately 98 percent of the grader, have 80 amps of underutilized capacity, 100 percent of the time, across the fleet.
“Not surprisingly, this also caskases down to 100 amp homes that have around 40 amps of”
underutilized capacity. But 40 amps is not large enough to build sufficient density of GPUs for inference applications. But 80 amps, 19 kilowatts is actually pretty well sized. So within that, let's say 1920 kilowatt power envelope, we have around 5 to 6 kilowatts of cooling to take the ejected heater away from the GPUs, and we have around 12 to 13 kilowatts of compute. That corresponds to either 8 B300 GPUs or 16 RTX pros, or looking ahead. There are GPUs from AMD and
City Breast and other companies that also fit within the power envelope to DDW. That's the modular system size, if you were the rebuilding initially. And so almost all homes are rated for this much power you're saying. All new homes by design are rated to 200 amps if not greater. Yeah. So before extra, one of the big problems we were solving for new home construction partners like Pulti, for example, is how do you avoid overbuilding even beyond 200 amps? Like for the last 10 or 15 years,
single family homes across the U.S., it's about 800,000 of a million homes being built every year, where we were being built at 200 amps of copper coming in. That was the design standard, if you will, from the U3 coming into a single family home. With the advent of electrification, with the desire to have homes ready to add EV charging in the future or add electric water heating in the future or induction cooktop in the future, the load calculation was pushing the size of the
conductor up to 320 amps of 400 amps even. And for a long time now, the last 3 or 4 years, even helping home builders avoid that cost, which they have to pass through to the home buyers,
by using span, that'll ensure that you never exceed 200 amps, because quite frankly 200 amps is
48 kilowatts at 240 volts of continuous power, concurrent power draw. That almost never happens. Right. Now we're applying that same logic to say, what if I were to use that, you know, 20 kilowatts are the 50 kilowatts available to your home to be able to deduct a deploy an extra not. A preemptively answer the question that often comes up. Needless to say, if you go upstream from a single home with a transformer level, it's not a linear sum in terms of how they size the
transformers of the substation upstream. So we're not proposing to deploy an extra in order in every new home. In a community, we've done a tremendous amount of power flow map to say, a roughly 25 to 30 percent penetration of extra nodes into a new home community, does not in any way impact the upstream infrastructure. We can very, very comfortably operate that system within that power on level. Walk me through that, because presumably you're increasing the base load draw on the system,
So transformers are going to be running harder, and that's going to have some...
on the transformers, maybe other parts of the system, right? Yep, that's right. So let's talk about
“it from a nodal perspective, what we do from a compute and load control perspective and a battery”
perspective, and then I'll talk about it at a zonal perspective. Let's say a collection of homes if you want, right? A typical and new home construction power system modeling would have a single pattern on a transformer, often 100 KV a transformer, but sometimes a 167 KV a transformer powering a collection of 8 to 10 single family homes, right? And then you you cascade that upwards, you get to sort of make a lot of scale substations at power, a community of dozens of homes if you,
right, or hundreds of homes. What we are designing is every extra node will be paired with a span panel in a battery, and every sort of simplified way, every extra home will have a neighbor to the left and the right that also gets a free span panel and a battery. So at the home that has the extra node, we have the ability to throttle compute or move compute workloads around when needed
“to other extra nodes that might have capacity. We have the ability to throttle your home loads”
with span, which is what we've been doing for years now, and we have a on-site on-premise battery that can inject power into a bus bar as well. So our bus bar is ready to 2125. So think of it as an on-grid microgrid is doing some cleverly attempt optimization. Then beyond that, at a fleet level, or let's say it is on a level, we have two additional homes that also have load controls and batteries that can inject power into the AC grid that they share, downstream of that 100 KV a transformer.
So we're able to ensure that in some at a fleet level, even if you take hundreds of homes in a
new home community, a third of them have extra nodes and all of them have span panels and batteries,
we're able to ensure that the network is resilient to this increased demand. I'm sorry to interrupt, but I really have to tell you about our sub-stack. The DSR network sub-stack is the absolute best way to follow deep state radio. The DSR daily words matter need to know silicaciousness, AI energy and climate and the daily blast. Sign up to get notified every time a new episode drops and stay up to date on the latest news
and expert analysis. You can also support us by becoming a paid member, paid members, get an ad-free listing experience, select episodes two days early, access to live streamed episodes, and 50% off of David's need to know sub-stack. Please consider joining us at DSR network.substack.com. That's DSR network.substack.com. Thank you and back to the show. I've talked a little about the commercial aspects of this and let's hear how much you can
save up pricing, how much public right now, but what it's much as you can say, what at the cost of these items, what does this look like to a homeowner? If I wanted to, if I wanted to do this,
“what would I need to pay in order to get your service and what would I earn from it?”
Absolutely. So, without giving away too much, I think at a high level, the way to think about the cost of our system is there's a very clear speed to power advantage that we can deploy computer capacity today as opposed to waiting four or five years to build a hundred megawatted center, but there's also a very measurable capex advantage. So, if you think about the cost of the compute, that is, you know, carry faster, like it's the same to us as it is to a large scale data
center, if you buy a computer the same volume, that's on the order of $30 million from megawatt
is the going rate if you will for inference compute or training could be. The in a traditional data center, the rest of the system. So, the land, the shell, the cooling, the gen set, the transformers, the batteries, all of that combined, sighting permitting, is on the order of $15 to $20 million of megawatt today. That's for a hundred megawatt scale data center. Because we've taken away all of that complexity of having to land design development, the connection, you know, gen sets,
transformers, et cetera. By the way, all have exceedingly worse lead times, and instead transformers into a single node that has all those components pre-fabriculated of actually, our cost is $3x to $5x cheaper on that $15 million number. So, we're at a sub $5 million of megawatt today and we're not quite at the scale that we plan to be out in the next six to all months. So, there's that capex advantage that then translates into economic value for the
compute off-taker, which is very, very clear, along with the speed to power advantage. We do have an increased optics compared to a large scale data center, because we're buying energy at effectively detail energy prices from the utility. The average utility rate across the US is around 12 cents a kilowatt hour, as opposed to let's say a bespoke rate you can get four to five cents a kilowatt hour at a large scale data center. But even at that increased optics at the sight level,
Energy optics at the sight level, our capex is so low that, in fact, if payba...
than a traditional data center. So, that's at the, that's called a, like, acid economic
“level rate. That is the consumer economics. We have to be mindful of the fact that, essentially,”
you are the landlord, let's say you're an expert in node host, I'm borrowing a small amount of physical space, and I'm also utilizing some of your utilities, your power and ISP, your fiber connectivity, right? And we've tried to flip the script to say, let's make it even simpler, because the computer's so valuable, what if if I were to just be able to pay for your energy and internet use as you would use normally? So, each expert on node has the potential to essentially
offer the individual consumer between two hundred and four hundred dollars of economic value, each month, which in most parts of the country fully compensates you for the cost of your energy consumption and your, let's say, home internet connection with much higher bandwidth fiber that we can deliver to you home. So, your estimate is two hundred and four hundred dollars
a month of value being generated per unit per month, basically, and then you deliver that to the customer
through reduction and bills. Yeah, we're saying that's the value we will, you, that's not, that the value of the computer generates is far greater than that, because computer doesn't still significant and let's say fairly volatile. Each node generates, you know, close to over $10,000 of economic value per month at the current computerizing, right? And keep in mind,
“much of that goes towards being the cost of the capex, right? You have to pay for the server,”
you have to pay for the GPUs, you got to pay for the cooling system, you got to pay for the installation, cost, the operating costs etc. Not to all of that, we have the ability to compensate code and code, you as the host customer, host partner for us. On the things that I think Macamost, most Americans, right? Or anybody really, it's like, can you bring the cost of my, you know, utilities down? And by doing that, and maybe you're going to this later in the conversation,
were you able to shift the narrative from not in my backyard to yes, please, in my backyard? That's a great motto, I like that. Yeah, just on the, another part of the infrastructure issue here, just got to go back for the minute, talk, talk about the fiber that you need here. I mean, this is our homes connected with the right time of February, because I know in general, the downloads a lot faster than me uploaded in most home, connections in its sounds like if I understand what you're
“doing, you're going to be needing a lot of uploads to be to make this work. So what I'm looking”
to say to challenge? Yeah, a home home internet as we know it is very different than what you think about enterprise, you know, what I would symmetric connectivity is, you know, fiber. So on, on the one hand, there is, there has been quietly a massive amount of investment going into building on fiber infrastructure. In fact, all the new home partners who are working with typically are working with retail providers like AT&T, Oversize in or Spectrum or Comcast were already landing
fiber to the new home construction sites, and often have fiber coming all the way to the customer a lot, if you will, right? We have deployed nodes right now that are able, where we were
very easily subscribed to high quality 1 gig per second, symmetric, uplink and downlink fiber,
per site. And that is what the node expert node receives and you might get from the switch, something that is comparable to what you want to be able to buy from AT&T, AT&T, for example, like 304Mg per second type of service, concurrently, to having an extra node sitting on your price. Now, a lot of the IP that we developed also goes into what we call the secure orchestration there. If you think about it, your network traffic is not at its peak capacity all of the time
back and forth, just like you would think about cart traffic or energy traffic, if you will. So, by designing the systems, where we have enough on extra node compute and memory, where the model, which is the heavy, you know, weights and, you know, several gigabytes of data if you will, being stored locally, the traffic becomes the tokens going in and out. And the tokens going in and out, and practically what we are seeing with nodes that we have deployed,
is not at the gigabitscale, it is usually in the megabitscale of traffic flowing back and forth. So, once installed, once deployed, what we are building in the residential sites, one gig of symmetric fiber connectivity seems plenty for most applications. In some of the commercial sites, we are going into what they would do, source five gigs, attend gigs of symmetric fiber connectivity as well, that obviously for certain applications
becomes meaningful. Let me just keep probing at different aspects of such an interesting idea and it's so innovative and creative, another question that occurred to me involved, that's right. GPUs are improving quality, but, you know, going from Blackwall to Ruben to Feynman, and Nvidia GPUs, and the other innovations in GPUs that are happening. Do you envision physically swapping out GPUs from these units or some other,
but I think it would seem interesting if I thought what's your plan?
Yeah, plan-up solutions, right?
for life of these operating systems of these GPUs and CPUs, and not just because of their
“expected operating life, but also because of the, as you rightly mentioned, technology evolution,”
it's happening very, very rapidly. So, the servers are designed to be fieldy-placeable. So, the servers have purpose-built connections for power, networking and cooling, so directly with cooling that goes into it, there are all quick-fit connectors, and the servers sit on trays so we can pull out. So, obviously with authorized access to the site, to be a service request where let's say a server or GPU is not performing, we can do a very quick
field swap. We will not do any IT maintenance onsite, and if it's a planned swap, as in, we've reached five-year life for the asset, and now we want to upgrade it to something else. The core infrastructure is the real value. We've essentially now built a large network of distributed power and compute infrastructure capabilities, where we can upscale the compute
has needed. That power networking is cooling will always remain.
“To talk about how you see this scaling, what's the vision here where you'd be entering into”
deployment agreements with home builders, with utilities, with what are your channel partners and trying to make this work? Yeah, you know, we broadly think about the partnerships we're framing in three categories. These host partners, there are technology partners, and then there are off-take partners, right? In that order, we had to solve for host partners, the home builders, and eventually the homeowners. We have a number of commercial real estate partners that have sites
that are not suited for large-scale data center operations at all, like they might be standard power, they might be less than a megawatt, less than five megawatt, but that they often have physical space, either on the rooftop of a building or on the side yard of a building, if you will. So we've now crude partnership on the host site where we can deploy just over the next year over a gigawatt of inference compute without breaking a sweat. And that's existing sites.
If you look at new homes that are built every year, we can deploy an additional gigawatt of compute
“every year. If you attach our product at like 20, 25% attached in new homes being built. Right?”
So there's the existing site model where we can deploy several gigawatts and just be the existing pipeline of host partners we can do a gigawatt, and then we have this evergreen model we can deploy to. Then we have technology partnerships, we have partnerships with the like of Nvidia,
who are who've been incredible in helping us think through the roadmap, just as you talk
about several evolutions, liquid cooling evolutions, what is the evolution of models and what type of models need can be done on what type of compute. And ultimately what we're saying is once we built the infrastructure, you can choose as a off-taker what combination of compute and models you want to run across a distribution of sites. So if by end of next year let's say we have tens of thousands of sites that are up and running totally in gigawatt of compute, you have a tremendous amount of
flexibility in determining what combination of physical compute and or GPUs and what models you want to deploy. Now the host partnership site or the off-taker partnership site, we are we are working with the the large hyperscalers and the frontier labs that are desperately wanting to find more compute or more power to deploy compute because many of them actually have access to the servers but not really a place to put them in and we're able to solve that problem for them. And that
opportunity is in the hundreds of megawatts to gigawatt scale compute per year. You've got another category of inference aggregator so the folks that are building enterprise solutions that are
that are not the companies that are building the data centers, right? And the themselves are second
in stack from a capx perspective compared to the large hyperscalers or the ones that are primarily building data centers. And we're able to give them access to lower costs to compute across the country and then the third tier is just merchant compute off-taker. Which is where we are active today, where we have compute nodes and you can go rent today. If you go to like a vast day, I would like any idea you can go find an extra node. You want nodes an extra node this because to you it's just
cloud compute and you can choose how many servers you want how many GPUs you want for how many weeks a month and it gives you a price and you can just rent it, right? So you have a retail window right now you can people can just go to your website and to their own x-free unit. Not through our website so we're not trying to become the marketplace we put our compute node today in third-party market places which are which is where an academic institution or a small startup could just go and
rent GPUs, right? Or sensibly like you today when you go say write a query on cloud or chat GPT as a consumer or even enterprise you don't really know where that query is going to which compute in which part of the world is serving that that particular request and it's kind of the same idea. Our orchestration layer aggregates across all of our sites to give you essentially a
Seamless cloud of a large network or clusters of GPUs that if they meet the s...
service that you would find from a traditional data center is virtually the same value to you, right?
So if one of our listeners either at their home or their workplace wants to do this, wants to buy a rent a x-free unit or where do they go to? Yeah so we don't sell x-free units, we will build
“own and operate them if you want to be a host partner that's a great conversation for us to have.”
We will rent or we will be happy to partner with folks who want to off take meaningful trenches of compute capacity directly from us. So so it has to be at the point if you have an extra node sitting on the side of your home you're not necessarily using you're not the direct user necessarily of that node. You might be when personally I but because more relevant or physically I because more relevant new robots that want to share context in this compute right there that's
super low latency that's great but today the vast majority of compute is just going to be going back into this pool of aggregated compute that we have available and then we'll be building this optimization layer that helps you determine based on the type of inference that you're trying to do what is the right physical low latency node to send the query to what type of model is embedded in those devices how much contextual memory do you need and how much bandwidth do you need and we're
able to optimize that it's a traffic opportunity if you are right. Well it's fascinating product and a really fascinating idea that meets the moment you've been very generous with your time is there anything we haven't touched that you would like to say any messages like to get out.
“Yeah I think fundamentally what I've been really passionate about for the last couple of decades”
and what I think span is building is infrastructure right I think the the piece that you can you can choose to be a betting person on is you know how big is AI going to be which compute company is going to win which model company is going to be and I'm saying we're not placing bets on that or placing bet on is building infrastructure for maintaining our dominance or lead in AI and that comes down to investing much like we didn't railroads and much like we didn't
power systems we're doing that with now taking what is otherwise a very very analog electrical grid and making it digital much like we did with the telecom infrastructure 20 years ago right so and power and compute are two sides of the same coin as I mentioned earlier so a lot of what we're doing is you know we think of our sort of energy transition company and for me that energy transition is less about the the effect which is the gastroelectric transition it's the cause which is
the analog digital transition and that at its core is our machine yeah well I always close this show
with two questions for our guests and the first question is how are you using AI tools in your day-to-day life we use enterprise cloud here at its van I do not use AI to write my email response for me or do calibrate for me I often use AI either cloud or complexity and I often go back and forth with that in terms of doing research in in the last in the last I would say year and half two years have been obsessed with building extra I I've had to go learn a lot about compute
which is not my you know my academic work was all energy and park conversion so I've been using AI to become smarter on all of these things it's a pretty good tool yeah it's an amazing tool
I'm using it in the same way it's incredible for climbing learning curve on different topics
just unbelievable final question could you please recommend three books or articles to our listeners could be something older now that's a good question there are a couple of books that I often go back to there there's a book from anthropology book called Gunstiums and Steel this I was published I want to see almost 20 years ago a lot about kind of the industrial revolution and sort of the evolution of medicine and how that that essentially
“translated to how we think about economic growth across the globe right and I think there are some”
very interesting parallels there or you know as people often say history to pizza itself or has at least a rhythm or rhyme to it right that that's one that often comes to mind for me I'd say another book that I've enjoyed in the past that I tend to go back to is the diamond age by Neil Seamanson that's a really good fiction book in if you're trying to escape some of the let's call it dystopian use of AI that I've presented out there I think it's good to go
read some of these things from 20 to 30 years ago again some of these ideas are becoming more
Real now than they were before but I think that's a bit more to go back to an...
that I'd like to go back to just just a pleasure it is sake etch one draw collection of
“rock stories it's it's a fun read yeah well heartrou see your founder of spin you have a fascinating”
idea fascinating product thank you very much for joining us on the AI Energy and Climate podcast
thank you for having me David
this has been the AI Energy and Climate Podcast a special production of the DSR network
