What are the requirements for an excellent new product introduction (NPI) process?
In 2000, Joel Spolsky published a self-described "sloppy" list of the twelve things to rate the quality of a software team. I first encountered it while posting software job listings, where I was asked to report my "Joel Score" so that candidates would know what kind of software team we are (we give ourselves a 10, by the way).
As a mechanical engineer who co-founded a software company, I often wonder what hardware can learn from software and vice versa. I've taken inspiration from The Joel Test to posit my own "sloppy" criteria for rating the quality of a hardware development process based on my experiences working closely with many hardware companies across multiple manufacturing verticals.
The Shedletsky Test is company-size agnostic. Most items are more about process and rigor and don't directly cost money to implement, just expertise.
The Shedletsky Test
Twelve yes or no questions. Your process gets one point for each question where the answer is yes.
Here's the test:
- Does your team formally review designs and drawings before tool release? A formal process may include a checklist, multiple people spinning CAD, and redlining drawings.
- Are you confident that each critical stack or fit in your product has a tolerance analysis? Add one bonus point if they all live in a centralized location where each team member can access them.
- Do your builds have explicit entry and exit criteria — and if a build does not meet exit criteria, do you repeat it? Minus one point if you have ever done an XVT.
- Do you send product design engineers and operations team members to each build? If you are leveraging a JDM model, replace the product design engineer with the system engineer -- the person who owns the design on the brand side. If you build locally, do team members spend significant time on the floor watching and learning from the process?
- Do you individually serialize all units and track data by serial number?
- Quality process, based on production volume:
- For high-volume: Do you destructively test 25-50% of your units at each build?
- For low-volume: Do you have a documented strategy and operational plan to achieve Zero Defect or other contractually-obligated quality requirements?
- Do you complete and review failure analysis on each unit that fails?
- Do you measure and review all critical-to-function and critical-to-quality dimensions at each build and calculate Cpk?
- Do you have a main issues list (MIL) process? This includes tracking issues, assigning specific individuals to close them out, and regularly following up on the progress.
- Do engineers have easy and immediate access to build data without asking the factory?
- Do you invest in your factory team? This includes rapport building, training, and dedicated time to level each other up based on the expertise that each brings to the table.
- Do you invest in backup designs (Plan B, C, etc) and engineering tools for your team?
The Scores
If you totaled more than 10, your hardware development process is a tight ship and likely benefits from fewer development fire drills, higher quality builds, and a higher likelihood of on-time product releases.
If you totaled 10, you're in good company. We have worked with several well-known Fortune 500 companies and admired startups at or "almost" at 10 points. A 10-point process still has holes through which issues can slip and fester, but this level of rigor in the hardware development process is serviceable.
If you totaled nine or below, your hardware development process has serious holes which could have consequences. While it's possible to "get lucky" and not have serious issues as you move from development to production with a less-than-10-point process, our experience suggests companies with processes like these tend to struggle to release high-quality products on time.