Visual inspection is critical to quality control for any manufacturing process, especially in electronics manufacturing. There are several different options for defect detection with visual inspection available:
- Manual inspection uses trained human inspectors.
- Machine vision inspection uses industrial cameras running conventional computer vision algorithms. Keyence, Cognex, and automated optical inspection (AOI) fall into this category: they are good at basic measurements, basic comparison, and presence/absence.
- AI inspection, a subset of machine vision, uses industrial cameras running machine-learning algorithms. These algorithms improve with additional data and can both discover novel defects and identify known ones. Additionally, AI visual inspection supports paradigms ill-suited for conventional machine vision, such as wire routing, glue and coating keep-in/keep-out, and some cosmetic inspections.
This article is a primer comparing manual and AI inspections.
Key takeaways: manual inspection vs AI inspection
Manual visual inspection has many shortcomings. Despite them, manufacturers choose it because it’s simple and fast to start. The cost of manual inspection depends on both the cost of labor in the market and the cost of escapes from the manual inspection process. As a result, it’s often way more expensive than it seems.
Manual inspection shortcomings
Common manual inspection use cases
Manual inspection costs
AI inspection combines the benefits of learning over time with superior repeatability and precision over humans. Modern visual inspection AI systems can drop into your process and deliver reliable and consistent results within days. These artificial intelligence systems are often SaaS offerings because they constantly learn and improve.
Benefits & Performance of AI Inspection
AI inspection costs
Manual inspection: high escape rates, but easy to implement
Human visual inspectors inspect an electronic device in triplicate as a strategy to reduce escapes.
Ten years ago, when manufacturers were still doing manual inspections, it felt more like a delayed upgrade -- oh, we'll get to automating this part at some point. A decade later, I'm surprised by the number of prospective customers and manufacturers that have actively chosen manual inspection. They have evaluated and trialed automated alternatives and have decided on manual inspection due to ease of implementation and well-understood limitations.
Everyone acknowledges the shortcomings of manual inspections
- At best, you will catch 80% of defects, which means 20% of real defects will escape (Sandia National Labs, 2012). Even in duplicate, two best-in-class 80% inspectors will get you to only 96%.
- Inspections are incredibly operator-dependent -- some will be worse than others.
- Even the best operators will fatigue within minutes -- humans are terrible at repetitive tasks.
- There's no record of what the inspector saw, just their judgment.
Common use cases for manual inspection
- Final visual check of a sub-assembly or final assembly. In the first few months of a new product ramp, you might go from two to thirty lines, and then back down to five lines for sustaining production. Manual inspection is easy to ramp up and down. The capital expense and effort to purchase and ramp automated inspection across thirty lines is assumed to be too costly (but might not be).
- Cosmetic inspection (anywhere). While there are automated ways to measure gaps and offsets or identify visual anomalies, no product on the market truly delivers universal cosmetics judgment in a way that could completely replace a human manual inspector. I call this problem the "golden fleece of manufacturing inspection" and Instrumental is working on building this technology.
- Safety-critical or regulated inspections. Aerospace, defense, industrial, and medical devices often have regulatory or safety requirements for inspection. Manufacturers will brute force accuracy limitations (but not fatigue) by setting up inspectors in duplicate (96% containment), triplicate (99.2% containment), or even quadruplicate (99.84%). These firms choose manual inspectors because they understand and expect their shortcomings versus an automated solution, which can feel like a black box.
Automated optical inspection must perform significantly better and cost less to displace manual inspectors.
An electronic device with defects highlighted using AI inspection techniques.
Automated inspection also opens up new possibilities for quality control and continuous improvement that are not possible today because human inspection is ill-suited for them:
- In-process steps for yield protection -- identify issues earlier when they are cheaper to fix.
- Inspection steps with 20+ inspection points -- manual inspectors often work from a list of 10 or fewer. If you could inspect 200 different points in no additional time, wouldn't you? (Our customers do.)
How much does manual inspection cost? More than you thought.
In low-cost labor countries, manual inspection is "cheap." A contract manufacturing partner might charge $5-6 / hour for inspection operator time – though we’ve seen as much as $15 / hour passed through to our customers. You can consider two ten-hour shifts, six days each week (common in China) -- for $37K / year to cover one inspection location.
No one building in a high-cost labor country believes manual inspection is “cheap.” Inspection in the United States or Europe might cost $30-45 / hour depending on fully loaded costs and whether the labor is unionized. Assuming two eight-hour shifts, five days each week, that’s $187K / year, plus overtime to extend to the same duration as above. If you add inspection in duplicate or triplicate, it could be $375K or even $561K per year.
Manual inspectors will miss defects, causing escapes, which have a cost. Assuming duplicate inspection at the top of industry benchmarks, that’s still only 96% containment – meaning 4% of true defects will escape. Escapes are incredibly difficult to measure by definition (after all, they escaped), and many companies struggle to attribute field issues and returns to escapes.
If you’d like to estimate the cost of escapes due to duplicate manual inspection, I recommend looking at your First Pass Yield (FPY). Let’s assume your FPY is 80% – which isn’t great, but also isn’t terrible – and that visual inspection fallout accounts for 5% points of the fallout. You make 10,000 units:
80% FPY * 5% visual fallout * (1 - 96% duplicate inspector capture rate) * 10,000 units = 16 units per 10,000 made.
For those 16 units – what’s the cost of a field failure? If you’re doing high-volume manufacturing, it’s probably the cost of customer service calls, unit replacements, lost revenue on those replacements (you cannot sell them to someone else), and reputation damage (each ½ star on Amazon causes at least a 3% reduction in sales). If you’re working on industrial, aerospace, or defense parts, the costs might include field engineering costs to fix and the risk of future contract reduction or cancellation.
When you add it up, manual inspection is incredibly costly.
Sales conversion rates vs Amazon star rating, image from Pattern Data Science.
AI Inspection: superhuman performance with an auditable record
Instrumental's AI visual inspection system on an assembly line in an electronics manufacturing facility.
Even though there are still a lot of human visual inspectors on the line, inspection is still one of the most automated processes in electronics assembly. Over $3B in industrial cameras are sold every year. Conventional machine vision has been around for decades and can work for some applications – but hasn’t worked well enough to eliminate manual inspectors. Over the last five years, AI inspection – sometimes referred to as “smart AOI” – has provided a new option. These systems provide indefatigable, superhuman performance.
Instrumental provides real-time AI inspection functionality as part of its Manufacturing AI and Data platform. Engineers and operations teams leverage this technology from NPI through production to support the discovery and containment of defects.
What are the benefits of AI inspection compared to manual inspection?
- Fewer escapes. Note that at best human inspectors capture 80% of defects (data science calls this “recall”) – and there are lots of scenarios where performance is worse when they are fatigued, still training, or just inherently less observant. AI inspection can benefit from multiple algorithms running in concert – and they don’t fatigue. Recall rates will be more stable and can continue to improve over time, exceeding human benchmarks into the 80s and 90s.
- Detect subtle defects. Electronics are complex devices, and it’s easy to miss subtle defects or for operators to judge the same part differently. AI inspection is repeatable.
- Discover new issue types. While humans generally look at a narrow list of 10-15 items in their Standard Inspection Protocol (SIP), Instrumental’s AI inspection can look everywhere within the field of view for anomalies. Even in stable production, Instrumental customers regularly identify new defect types and can immediately start inspecting for them without impacting the inspection cycle time.
- Train like a human with infinite memory. AI inspection can take advantage of an unlimited (and auditable) memory as part of its training set – and can be easily replicated from line to line.
Additional benefits of Instrumental’s platform – which extends beyond AI inspection:
- Get started without golden or defective examples. Instrumental’s algorithms have been designed to work in real-world, few-shot (low-volume or low-defect rate) environments. Our Discover AI will identify anomalies with as few as five normal units.
- Remote administration and auditable data record. With 100% traceability, Instrumental users can access images, monitor factory production, and set up or adjust live AI inspections remotely.
- ROI in development and production. During development, engineers discover and fix more issues with higher efficiency. In production, quality and operations teams use Instrumental for quality control to prevent field failures and for continuous improvement to improve FPY, throughput, and product margins.
Head-to-head: Instrumental proves superhuman capabilities
4.6% of units that had passed three operators had real defects intercepted by Instrumental.
How much does AI inspection with Instrumental cost?
When considering the cost of any technology, it’s important to put it in the context of the value it provides – or ROI. As part of the Instrumental sales process, we work with customers to build a business case to ensure we can deliver a minimum of 3X ROI, with the first proof of value within 90 days. You can read more about the ROI our customers have experienced in our case studies.
Instrumental is software as a service – where pricing scales with data consumption and the AI licenses needed for your specific use cases. Our most basic offering is Trace, which provides a 100% data record of both visual and functional test data for the cost of the integration and a data plan. Customers can then layer on licenses as needed for their use cases:
- Discover, which leverages Discover AI to find new issues.
- Operate, which provides tools for monitoring ongoing production.
- Solve, which leverages Correlations AI to identify relationships that point to the root cause.
Instrumental is the best in the world at what we do: discovering and intercepting novel and known issues and providing a powerful toolset to actually solve them.
Learn more about Instrumental’s product offerings here.