BMW implementation of Deep Learning Machine Vision is a good example of AI usage in manufacturing.
The Business Problem.
Like all car makers BMW uses robots in its manufacturing process. Real time quality control is necessary at every step of the process as robots can do the wrong thing as efficiently and in volume as the right thing. Detecting a faulty part in a process that handles 30 million parts per day is a daunting task.
For example, each car body part is stamped out of a sheet of metal. It is chemically treated and painted or plated. Any scratch, dust, dirt, oil drop on the metal surface can result in the final piece being faulty or having a latent fault which is not discovered until after the car is assembled or worse, sold.
BMW used machine vision systems to inspect each part every step of the way in order to detect and reject parts before they end up too far in the process. Machine vision works on a pixel comparison basis. A camera takes a picture of a part being inspected and compares it to a reference picture of a good part stored in memory. It basically compares each pixel of the image with the corresponding pixel of the reference picture.
As a result machine vision has very strict camera positioning and lighting requirements to assure that a picture of the inspected part matches exactly the frame of the reference picture stored in memory. This makes the system prone to errors caused by camera vibrations, lighting variations or slight speed changes of the conveyor belt carrying the inspected part. These variations produce pictures that are slightly different than the reference and complex rules need to be programmed to help the system decide what is good and what should be rejected. False positives and false negatives can and do happen.
AI based vision works better.
A human inspector could easily ignore many of the variables that can fool a machine vision system and be able to correctly accept/reject parts as they travel by. The problem is that in addition to being more costly, human inspectors are slower, would get bored, tired, distracted and over time make more mistakes than the machine vision system.
AI vision system can be more accurate than a human, can make decisions in milliseconds and not suffer from any of these problems. This is exactly what BMW deployed.
Just like the AI dog recognition system described elsewhere that can classify an animal as a dog regardless if it is small, large, long hair, short hair, solid or spotty colored, laying down, rolling around, in the water or jumping to catch a ball, an AI inspection system can detect flaws on the assembly line. It does more than a picture comparison it interprets each picture like a human observer.
BMW uses a Convolution Neural Network to analyze each picture and decide if the part is OK or not. The system requires training photographs which show good and bad parts. Between 100 and 1,000 such photographs are taken depending on the complexity of the part or a difficulty of detecting a fault. Technique called “Transfer Learning” is used to make the training more efficient. The technique builds new knowledge on top of what it learned prior. This means that when new training pictures are added
the system uses some of the knowledge of cars, BMW models, colors, styles, similar parts, etc… it learned before to help learn the new feature faster.
BMW associates take the photographs, label them as good/bad and add several other parameters and submit them to the system. The AI system uses the sample photographs to learn. No software coding is required, the system learns to tell the difference once the training photographs are submitted.
AI achieves 99.9976% Confidence Level
In one example BMW uses the AI system to verify that the right model labels are attached to each car. Such labels as: GT 330i (2 labels) or 650i (single label) are attached at the back of the car. Some customers do not want any labels. An AI system inspects the labels and verifies their correctness. The photograph below shows that even in bad lighting conditions (top right) the AI system is able to identify the correct model with 99.9976% confidence level.
- 55 Car models produced
- 70% of cars are custom configured
- 30M parts handled/day
- 9,000 cars made/day
- >1B possible parts configurations
- Matthias Schindler the head of BMW AI project explains its details
- BMW shares its AI algorithm
- BMW shared AI code location on Github
- Quick overview video of how this works in the BMW factory.
- BMW applies AI to paint inspection