Three ways reliability tests can come up short

Reliability testing trips up even the best engineers and companies.

A good reliability test unearths issues that could arise from common uses of your product — say, discovering that lint in a pants pocket can reduce the performance of a smartphone’s speaker and microphone.

But reliability testing can fail to surface issues when not executed properly. Common mistakes in reliability testing start with planning. These are the key reasons why a reliability plan can come up short:

Failure to anticipate a use. One hardware company discovered, to their surprise, that customers liked the sensation of flipping the switch on the device, meaning the switches were being flipped frequently (absent-mindedly, etc.), at significantly higher rates than anticipated. These situations are hard because you don’t know exactly how your product will be used in the field until it’s in the field. Instead of guessing, build some units and get them into the hands of real users. Don’t just hand devices out to your engineers (though you should do this too), but look for users who are particularly extreme in ways that will stress your devices — such as someone who does a lot of physical activity, or traveling, or who has cracked the coverglass on every phone they’ve ever owned. Do in-depth interviews after a period of time to understand the corner cases so you can create tests for any scenarios which become considered “reasonable use”.

Undertesting. In the early builds, it’s tempting to skimp on reliability test quantities because you know the design isn’t finalized anyway, and building 50 additional units just to destroy them seems like a waste of money. But the earlier you test, the sooner you can work out issues that can wind up costing even more time and money once you’re further in the development process. Testing needs to be baked into the budget and the schedule. While some specialized testing might require external laboratories, make every attempt to work with factories who have basic thermal chambers and drop robots — that equipment will cover most of the testing needs.

Overtesting. This can happen at larger companies with bigger budgets, where engineers will put hundreds of units through reliability testing to test A and B and C and many more. Overtesting requires more units to be built, which keeps the team in China longer, and can also slow down getting the results due to a finite amount of testing resources.

For a better idea of how to avoid these problems, download Instrumental’s Reliability Test Kit. You’ll find detailed test definition and setup instructions for a wide variety of reliability tests. In this kit, we also demonstrate the best practices for some of the most commonly tested scenarios, and offer efficiency tips for minimizing units tested and maximizing data.