What are the elements of a good hardware development process? Back in 2000, Joel Spolsky published a self-described “sloppy” list of the twelve things to rate the quality of a software team. I first encountered it while listing software jobs on a site popular with software engineers—I was asked to report my “Joel Score” so that candidates would know what kind of software team we are (we give ourselves a 10, by the way).
As a mechanical engineer who cofounded a software company, I often find myself trying to find translations between the two worlds of hardware and software. I’ve taken inspiration from The Joel Test to posit my own criteria for rating the quality of a hardware development process, based on my experiences working closely with many hardware companies.
The Shedletsky Test
The Shedletsky Test is meant to be pretty simple: 12 yes or no questions. For each question where the answer is yes, your process gets one point. Here’s the test:
- Does your team formally review designs and drawings before tools are kicked off?
- Does every critical stack or fit in your product have a tolerance analysis?
Add one bonus point if they all live in a centralized location where each team member can access them. - Do your builds have explicit entry and exit criteria — and if a build does not meet exit criteria, do you repeat it?
Minus one point if you have ever done an XVT. - Do you send your own engineers and your own operations team members to builds?
- Do you individually serialize all units and track data by serial number?
- Do you destructively test 25-50% of your units at each build?
- Do you complete failure analysis on every unit that fails?
- Do you measure all critical-to-function and critical-to-quality dimensions at each build and calculate Cpk?
- Do you track issues, assign specific individuals to close them out, and follow up on the progress on a regular basis?
- Do engineers have easy and immediate access to build data from anywhere?
- Do you invest in building rapport with and training your factory team?
- Does your company invest in backup designs and engineering tools?
The Scores
If you totaled >10, your hardware development process is a tight ship and likely benefits from fewer development fire drills, higher quality builds, and a higher likelihood of on-time product releases.
If you totaled 10, you’re in good company. We have worked with several well-known Fortune 500 companies and admired startups who are at or “almost” at 10 points. A 10 point process still has holes through which issues can slip and fester, but this level of rigor in the hardware development process has been shown to be serviceable.
If you totaled 9 or below, your hardware development process has serious holes which could have consequences. While it’s possible to “get lucky” and not have serious issues as you move from development to production with a less-than-10 point process, our experience suggests companies with processes like these tend to struggle to release high quality products on-time.
The Shedletsky Test is company-size agnostic. Most of the items are more about process and rigor, and don’t directly cost money to implement, just expertise. While some items do require financial investment, these amounts are not out-of-reach of smaller companies if made into a priority.
Instrumental technology helps with Shedletsky Test numbers 5, 7, 8, 10, and 12. Contact us to learn more about how we can help to tighten up your hardware development process.
Related Topics
