America's AI Action Plan Overlooks a Huge Problem: Building AI Servers Isn’t Easy

By Malathi Nayak

The White House unveiled Winning the AI Race: America’s AI Action Plan in July. It’s a blueprint to speed up data center permitting, expand skilled labor, and modernize the electric grid. But the plan sidesteps a critical issue: the difficulty of manufacturing AI servers at scale.

The deployment of fully utilized server racks in the U.S. must increase threefold in 2025, with additional growth required in subsequent years, as indicated by current projections. Amid a pressing AI infrastructure crisis, server makers are already struggling to meet the surging demand for fully loaded, hand-assembled server racks. There’s momentum in the U.S. tech industry and government to boost server production—yet significant technical challenges remain.

Complexity in Manufacturing

Product complexity is one issue: state-of-the-art AI racks typically contain nearly 30 individual server or switching trays—each with multiple circuit boards and hundreds of individually mated internal connections. The designs are getting denser, with more components per square inch, and as power requirements increase, complex liquid cooling systems are becoming more of the norm. After each tray is assembled, the functional testing for an individual tray can take 8-12 hours, before trays can be assembled into a rack and put through an additional battery of system-level tests.

A key bottleneck is low first-pass yield (FPY), says Marcy Alstott, a managing partner at OnTap Consulting, a firm that advises manufacturers on operations and supply chain issues.

FPY is a performance metric that measures the percentage of units that can be built without repair and retest. Server FPYs are typically 60–75% but can drop to 0–20% on very complicated systems or new models.

The more complicated a product is, the more likely it won’t work at the end of the assembly line,  Alstott explains. She believes improvements upstream in testing and quality will help drive speed and cost-efficiency.

Building in Quality

Instrumental, a manufacturing technology company whose software improves FPY on tray and server lines, has found that top-of-the-line AI servers require hundreds of visual inspections. Simpler models might require fewer. Based on the company’s work, the top five technical defects causing low FPY in AI server assembly are:

1. Unplugged or loose cables. Many servers have hundreds of individually mated cables that connect power and data. Loose connections can lead to intermittent overheating and communication failures – or eventually become disconnected after leaving the assembly line, resulting in failures out in the data center.
2. Connector pins can be damaged during cable installation. A line operator can inadvertently damage a socket by installing a cable backwards, misaligning it, or applying too much force while positioning it. 
3. Thermal management issues. AI servers consume lots of power and run hot – many of the top models use liquid cooling instead of air cooling to eke out more performance. To keep the server from overheating, it’s important that conductive pads are assembled correctly, liquid cooling connections are made, and any leak sensors are aligned properly. 
4. Missing or damaged board components. The printed circuit boards that go inside AI servers are large, heavy, and floppy. They have lots of small components that are easy to damage during handling and assembly.
5. Issues with pick-and-place caps. Plastic or metal pick-and-place caps on connectors must be manually removed. When an operator forgets to remove caps, they can be pushed down into the connector, causing damage or hindering the testing process.

    

These defects provide useful clues on areas where server makers must drive improvements, says Scott Jewler, a semiconductor industry executive who specializes in manufacturing business and operations. To meet the rising demand, server production should be “idiot-proof” leaving no room for technical defects, Jewler explains.

The challenge is that many of today’s top-of-the-line systems are competing on performance metrics like inference speeds and power draw – design for assembly is a distant secondary concern.

To gain an advantage in the race for dominance in the AI server market, Alstott says server makers must get quality right and improve FPY. “It's been proven that if you push your failures upstream and catch them early—so you’re sure all your solder joints are solid, your process is dialed in, and you've got Six Sigma control—then the chance that it’s going to fail later is much lower,” she said.

Ideally, “you're building in quality,” according to Alstott. “That’s the way forward.”