\n
<\/p>\n
Nvidia has put its name behind a fledgling effort to put mini-data centers beside people\u2019s homes in boxes that look like HVAC units. It\u2019s a \u201cpower\u201d play, considering that the main bottleneck to building out more data center capacity is not money or chips, but rather retrofitting the electrical grid to supply the power.<\/p>\n
The idea<\/a>, put forward by a California smart utility box company called Span, is to put the GPUs where the power has already been allocated\u2014at the home. Span says the average household uses only about 42% of the electricity allotted to it, and rarely reaches peak usage. Span\u2019s smart utility boxes detect that, and steer the extra available power over to the GPUs, which live inside a \u201cnode\u201d that sits beside the house and looks something like an HVAC unit. The boxes contain 16 Nvidia GPUs, 4 AMD CPUs, 4 terabytes of memory, and a cooling system. When a large number of homes have these, the servers could be connected together in a network and work together on distributed computing jobs (workloads), Span says.\u00a0<\/p>\n In exchange for hosting a node, Span pays a big chunk of the homeowner\u2019s electricity and broadband internet bills.<\/p>\n And there may even be advantages for putting the compute power closer to the end users that are using the chatbots or AI services, Span says.\u00a0<\/p>\n It\u2019s a cool idea on paper, but it\u2019s almost completely unproven in real-world use. Span has been prototyping the units but has yet to install any of them beside real homes. I asked Span VP Chris Lander if his company has done technical studies showing that its brand of distributed computing will be fast and robust enough to handle real AI workloads. \u201cWe’ve done a bunch of technical studies internally [and] a bunch of modeling for different kinds of workloads, both from the business point of view [and] the product point of view and from the technical architecture point of view,\u201d he replies.\u00a0<\/p>\n The company is working with a homebuilder, Atlanta-based Pulte Homes, to build the nodes at new homes, but Pulte told CNBC<\/a> that it\u2019s so far put a Span unit next to exactly one home. \u201cI will say that we’ve been collaborating with Pulte amongst others to test the latest proof of concept design, the latest prototype that we have,\u201d Lander says. Span says it’ll have \u201cupwards of 100\u201d nodes of an advanced version of its prototype in a pilot project \u201clater this year,\u201d but isn\u2019t saying when or where the pilot will be built.\u00a0\u00a0<\/p>\n The main point of resistance to new data centers across the country is the risk that the facilities will result in higher electric bills for everyone in the area. Whether it’s a new central data center or a distributed data center, as Span proposes, that’s drawing more power from the grid, the risk of higher costs\u2014perhaps because\u00a0of transformers and other infrastructure running hotter and degrading more quickly\u2014could arguably be the same.\u00a0<\/p>\n Lander disagrees. \u201cWe believe it’s actually going to be the opposite\u2014that it’s actually going to give relief to customers, not just for those direct customers that are hosting, that we’re paying for their energy, but by allaying some of that additional CapEx spend that utilities would have to pay to build out [more] data centers,\u201d he says.\u00a0<\/p>\n The existence of Span\u2019s idea, which has gotten a good deal of attention in the press and on social media, is one of the first signs of the market finding ways to address the serious dearth of data center capacity needed to support the expected demand for artificial intelligence services. Data centers take time to build and often face political resistance, and the demand for AI compute is growing now<\/em>.\u00a0<\/p>\n On the political side, crowdsourcing already-provisioned power from households in a community may be easier than talking a city council into issuing a permit for a data center. As Reddit user unicynicist<\/a> put it: \u201cIt’s like Uber, but for turning your house into someone else’s unpermitted data center.\u201d<\/p>\n Still, it says something that Nvidia allowed Span to include its brand in the press release, but the GPU maker\u2019s involvement beyond that has been mostly consultation. \u201cNvidia’s been a thought partner, and they’ve been helping us from a business point of view to connect us with the right folks.\u201d But Nvidia is not an investor and has so far not donated any GPUs to Span\u2019s initiative.<\/p>\n And the chips are not cheap. In fact, the Span box contains some very expensive hardware. The chips and other technology could be worth $500,000 or more, based on available pricing information for the specific components. The risk of theft then becomes an issue.\u00a0<\/p>\n If the home-side server box concept proves out and produces a meaningful amount of compute resources, Nvidia could help Span find potential buyers. In fact, that effort is already underway. \u201cWe’ve had conversations with the breadth of potential compute offtakers, whether they’re hyperscalers, neoclouds, neoscalers, and AI service providers,\u201d Lander says.<\/p>\n