Why valves are a spacecraft engineer’s worst nightmare

Follow the space industry long enough and you’ll notice that an outsized number of the problems that cause catastrophic failures of satellites or launch vehicles can be traced to a physically small but ubiquitous part: valves.

Valves play a critical role in the spacecraft’s architecture, regulating the flow of pressurents like helium and propellants. They can also be found on launch vehicles, and by number they are one of the most common subcomponents in these systems. This reality came into sharp focus this week, when Astrobotic announced that its Peregrine lunar lander would not be able to attempt a soft landing on the moon due to a mission-ending propulsion leak — with likely origins in a valve that failed to reseal.

But Astrobotic is far from the only space company to have a mission cut short by valve issues on orbit. Boeing faced major mission delays for the second orbital test flight of its Starliner crewed capsule due to valve issues, and back in 2019, SpaceX’s Crew Dragon exploded during a ground test due to a leaky valve in the propulsion system.

“There’s a thousand different ways you can make a valve unhappy,” said Jake Teufert, CTO of Benchmark Space Systems, a Vermont-based startup developing propulsion systems for spacecraft.

Even a thousand might be an understatement. In general, valves are composed of a plunger that needs to reseat itself after being actuated, and that has to close into a seal. “If there are any issues with that, it can close wrong, it can cause leaks,” Grant Bonin, spacecraft designer and founder of gravityLab, said.

But this description is too simple, to the point of being misleading. Aerospace valves must be manufactured to ultra-high precision, be as lightweight as possible and be able to withstand a gauntlet of extremes: extreme temperatures, extreme fluids, extreme vibration environments and extreme pressures — sometimes up to thousands of pounds per square inch. Valves must also have ultra-low leakage requirements; Teufert said that some valves have allowable leakage rates equivalent to leaking only one gram of helium over the course of 200 years.

Complicating things even further are the underlying physical realities with which engineers and valve manufacturers must contend. For example, some fuels and oxidizers are incompatible with certain valve gasket polymers, and chemical incompatibility leads to nothing but problems, like corrosion or cracking. Engineers must also be vigilant against “foreign object debris,” or FOD, the tiniest particle of debris or impurity that can clog up a valve or prevent proper sealing. Even small leaks can cause runaway effects, because rapidly expanding gas makes things cold, which can take the valve out of its acceptable temperature range.

Engineers run spacecraft through a host of tests on the ground, but the flight environment can only be matched to an extent, Teufert explained.

“You can certainly throw something on the shaker table and do a [vibration] profile, but you may not be doing that while you’re also fully pressurized and exposed to oxidizer vapors, which is what’s happening in flight,” he said. “Most test houses, if you put a full tank of nitrogen tetroxide on a shaker table, are going to tell you not ‘no’ — but ‘hell no’.”

At the end of the day, engineers are up against an impossibly long list of failure modes and must at some point determine their confidence in the testing. Plus, it’s not uncommon for companies to eat up all of their schedule margin with design, procurement and build, and then the testing department is the one that receives the most schedule pressure.

“When you’ve got turnover internally, and when you’re working with a whole bunch of different vendors, it can be very easy to not test adequately, to miss some of these issues,” Bonin said.

It might be tempting to think, why not add an additional valve, so that if one fails to open, there’s a backup? But adding two valves (or any additional subcomponents) can create whole new failure modes that you would never have with one valve.

The other issue is supply chain. Despite the relatively high volumes of spacecraft coming out of SpaceX’s Starlink program, Amazon’s Kuiper, OneWeb and the whole swath of emerging space startups, spacecraft subcomponents are still very, very far from being mass production.

“At it’s core, the problem is that space just is not a mass market,” Bonin said. “Whenever anybody in aerospace talks about mass production, I chuckle, because we do things sometimes in intermediate volume, but we don’t do anything that’s truly mass production. So we’re not the high-priority clients for these companies.” Teufert echoed these thoughts, saying, “As an industry, we are still so much at the point of being artisanal, handcrafted hardware, if it’s anything aerospace specific, and that definitely extends to valves.”

Due to the relatively small volumes of product, manufacturing is still very much bespoke, with many valves made in a very small series for specific propulsion systems or spacecraft. But the competency of suppliers is not necessarily stable over time, because the process is so boutique and relies so much on tribal knowledge.

“If it’s something that they’re making a large, true mass production run of per year, they have great processes and well distributed knowledge as to how to make that valve or other component reliably over time,” Teufert said. “Whereas the weird little aerospace valve that they’re making 10 of per year for this niche market, that is [made by] some guy named Bob, who is in his early 60s, and has one foot in retirement. He’s making these every couple of years and then Bob leaves, and all that tribal knowledge goes out the door because there was no one to come up under Bob. I’ve seen that on a ton of components.”

No doubt this is not true for every program; for example, last summer valve designer and manufacturer Marotta announced it had delivered its 30,000th solenoid CoRe valve to SpaceX. But in other cases, smaller space companies must contend with longer manufacturing timelines at smaller volumes, buy commercial off the shelf or try to finagle a solution in-house.

SpaceX President and COO Gwynne Shotwell with Marotta reps. Image Credits: Marotta

“I can buy the same part twice, but if Jim made Part A and Joe made Part B, even though they’re the same part number, they’re dramatically different quality,” Bonin said. Or if your lead technician had a shitty Monday, they might have skipped a step. There’s just human error everywhere in this stuff.”