Industrial Visual Inspection – Defect and Anomaly Detection in Manufacturing

Industrial Visual Inspection – Defect and Anomaly Detection in Manufacturing

Dorota Owczarek - May 8, 2022

In industrial production, the detection of defects and anomalies is crucial for maintaining quality standards. In many cases, human inspectors are used to examining products as they come off the production line. However, with the advent of machine learning and artificial intelligence, it is now possible to use custom models to detect defects and anomalies in images. In this blog post, we will explore the use of machine learning in visual inspection and discuss some of the benefits it offers for manufacturers. We will look at real-world use cases from different industries and see why AI-based visual anomaly detection is becoming increasingly popular in modern manufacturing facilities.

Traditional Inspection on Production Lines

Since the beginning of the industrial age, the manufacturers have been using different techniques to monitor the quality of the processes and products on the assembly line. These techniques - until relatively recently - would have to involve humans. But with gradually increasing manufacturing volume and growing demand, it naturally has been becoming harder and harder to monitor the quality and detect issues on the assembly line. The inspectors would find it hard to deal with such a great deal of products, which would affect the results. The monotonousness of such a task doesn’t help, causing fatigue and increasing the probability of errors.

The Introduction of Automation for Anomaly Detection

Automation was a breakthrough for the manufacturers, allowing them to increase their production volume massively without compromising quality standards. Suddenly, they were able to automate a big part of the manufacturing process, including the most error-prone tasks, like defect and anomaly detection. But artificial intelligence was a real game-changer, replacing programmed, poorly adaptive with flexible, self-learning, and self-improving ones.

Machine Learning and Computer Vision to Aid Anomaly Detection

Visual monitoring in its most traditional form has many limitations – with the bigger one being a relatively slow reaction. Once the machine detects an anomaly or defect, it can trigger an automatic reaction, whereas, without AI, these actions have to be carried out manually. In manufacturing, where every second counts, it’s counterproductive. Particularly in Pharma, one relatively small issue can affect the whole batch, causing massive losses.

Another thing is the consistency of quality assurance. With the automated tools, all data regarding the defects and anomalies stays in the system.The machine can draw conclusions from it, continuously improving its detection capabilities. Whereas in the case of the traditional defect and anomaly detection method, the effectiveness of quality monitoring may drastically fall with any personnel change – and that means costs.

AI-based Computer Vision solves these problems, improving quality control and quality assurance. How exactly? Let’s get into that.

Can AI-Based Computer Vision Detect Defects and Identify Anomalies?

Artificial intelligence imitates human behavior – and as its branch, computer vision reproduces human capability to interpret the image. The technology draws from the complexity of the human vision system, imitating its way of processing visual information. While the mystery of capturing images was solved hundreds of years ago (and sealed with the invention of the camera), interpreting it has always been the tough part. With artificial intelligence, it finally became possible.

Human vision process

Human vision process

In the case of computer vision, an eye is replaced with the sensing device and the visual cortex - with an interpreting device powered by the machine learning algorithm. Trained with large image datasets, the deep learning model can interpret the content of a given image. That mechanism is used in manufacturing for defect and anomaly detection.

AI-based computer vision that can be used for anomaly and defect detection

AI-based computer vision that can be used for anomaly and defect detection

Fed with image samples of the impeccable products that fulfill all the quality requirements, the deep learning model constitutes its own logic, learning to identify different types of flaws. The image captured by the device is evaluated on its basis, and if it doesn’t match, the defect is immediately flagged. Developers can draw from different methods, creating generative anomaly detection algorithms like Bayesian networks, Recurrent Neural Networks (RNN), Convolutional Neural Networks (CNN), GAN, or Variational Autoencoder.

In the most common scenario, algorithms classify the products as defective or not defective. Models can also be trained with supervised learning to classify defects based on their severity, type, or status. That enables creating more sophisticated mechanisms of automatically triggered reactions and gathering detailed data.

Applying Machine Learning to Quality Assurance – Industry Use Cases

The increasingly competitive entrepreneurial landscape makes companies constantly improve the quality of their products, and AI-based automated tools can be an invaluable support in this process. With computer vision, they can evaluate the quality of their products more efficiently without engaging an additional workforce or losing consistency.

The use cases below prove the versatility of this technology. Let’s take a closer look at defect and anomaly detection applications across industries.

Visual Inspection in Pharma and Life Sciences

Cross-contamination of the cell lines is one of the most common risks the pharmaceutic and scientific labs face on a daily basis. Sometimes it takes just one dropper to be accidentally reused for the contamination to occur. The outcomes may be costly since they can have a negative impact on lab research or even make the pharmaceutical product unusable.

Keeping the quality standards and training, the pharmaceutical companies can reduce the probability of cross-contamination, but it still can happen - and image processing with deep learning can help identify it at the early stage. The deep neural network can detect small morphology changes in cells, protecting pharma companies and labs from having their results adulterated. For instance, this case study from ATM (Annals of Translational Medicine) proves that neural networks (specifically, a BCNN) can achieve close-to-100% accuracy in identifying cell lines.

For biotech companies, visual inspection is applied to monitor cell cultures, identify cross-contamination, and track morphological changes. Deep learning models can detect such changes at the stage when they are not yet visible to the human eye (through microscopes), making it possible to take preventive measures on time and avoid costly consequences. Dedicated ML software analyzes images of cells and provides reports with valuable insights to help clients improve the efficiency of their research.

Visual inspections help the pharmaceutical companies control the packaging, too. Pharma companies (particularly those in European Union) are subject to strict norms and regulations regarding the containers for their pharmaceutical products. Most products require airtight closure and the usage of specific materials. At the same time, any defect in packaging can affect the efficiency of the medication and its shelf life. With computer vision, such issues can be detected at an early stage.

Detecting Defects in Semiconductor Manufacturing Processes

Visual inspection can help companies from various industries keep an eye on the quality of the products themselves, but also on their state. They can use image processing for monitoring semiconductor manufacturing processes such as cleaning, film deposition, etching, exposure, implantation, resist coating, implantation of impurities, or heat processing. The models can detect anomalies in wafer levels and automate such processes as pre-place inspection.

The rule is the same – the visual inspection system based on a deep learning algorithm gets trained with images of good quality products after each semiconductor process and learns to identify defects. A good image processing device is the key in this case since the defects may be hardly detectable.

defect detection in semiconductors and chips manufacturing

Results of defect detection in semiconductors and chips manufacturing (from Nature)

Quality Assurance for Electronics Manufacturers

The capabilities of visual inspection are particularly impressive when it comes to electronics manufacturing. Well-trained models can detect even the tiniest defects that are barely visible to the human eye. Companies use deep learning techniques to find potential anomalies in microchips, transformers, monitors, CPUs, and other manufactured parts - many of them really tiny.

To keep the imaging non-destructive, the electronic manufacturers tend to draw information regarding defects from CT scans. Using X-ray tomography, they can generate radiographic projections of the manufactured elements. The generated image is later translated by the algorithm to its 3-d representation.

Due to the complexity of the elements, it’s essential for the manufacturers to narrow the scope of analysis to the parts susceptible to defects or damage. Otherwise, it will be challenging to appropriately train the deep learning network for defect/anomaly detection.

Visual Quality Inspection in Chemicals Production

Even though at first glance, visual inspection methods may not seem the most suitable for evaluating the quality of chemical manufacturing, they are actually proven to work. In typical chemical manufacture, the samples would get sent into the lab to be tested manually. This process is neither efficient nor accurate, as a few samples may not represent the quality of the whole batch. Meanwhile, testing every product on the chemical production line is far from cost-effective.

The manufacturers can evaluate their quality in a faster, cheaper way, using computer vision. The mechanism of visual quality inspection is similar to the case of solid objects – the model gets trained with the images of chemical products that fulfill quality standards and classifies those on the assembly line accordingly. Computer vision can detect anomalies based on the substance’s color, stratification, or physical state. If the product is clumped, oxidized, or layered, it can be automatically separated from the good ones and redirected from the production line.

Obviously, this method has some limitations since some defects in chemical substances are undetectable. However, a visual inspection can be an added value to the quality assurance system applied by the chemical manufacturer.

Off-The-Shelf Visual Inspection Systems vs. Custom Machine Learning Models

While there are quite some off-the-shelf visual inspection systems available on the market, many companies still decide to stick to custom machine learning models. The reason is simple – since off-the-shelf solutions are made to work for different companies, they enforce some compromises.

Even if they are flexible to a certain extent, there are limits that can sabotage the company’s business growth. For instance, the users may not be able to train the system using their own training data set, being forced to use the inbuilt databases. If their product and its features are unique, such a solution may not be enough to perform anomaly detection in a throughway.

Choosing to develop a custom machine learning model, the companies can adjust its type to the specifics of the task (take the cell line inspection described above, for which the BCNN has proven to deliver the most accurate results). With their own model, they can also extract regions of interest for increased accuracy, while off-the-shelf systems usually don’t offer such a possibility. As a result, those who pick custom anomaly detection models over ready-made solutions often enjoy more satisfactory outcomes.

The Benefits of AI-Based Visual Anomaly Detection in Manufacturing

AI is revolutionizing the manufacturing sector in many ways bringing numerous benefits. With AI-based visual inspection, the manufacturers can reduce their operational costs by:

  • preventing downtimes with predictive maintenance
  • reducing the workforce demand
  • offloading the quality inspectors and delegating them to more demanding tasks
  • reducing the number of returns and complaints

At the same time, they can improve customer satisfaction and enhance the company’s reputation. Less defective products delivered to the market obviously equals better satisfaction rates.

Would you like to take advantage of these benefits? Reach out to us to tell us more about the specifics of your industry and products; as an AI software development company, we might be able to help!

The Future of Efficient Manufacturing and Advanced Deep Learning for Anomaly Detection Model

Widespread adoption of anomaly detection methods based on deep learning in quality assurance is inevitable. The ever-growing competition on the market, as well as the need to meet the consumers’ expectations, will force the manufacturers to look for new ways to optimize their production lines. Applying machine learning algorithms to visual inspection tasks is one of those – it’s a move that can save millions of Euros or Dollars for big companies and make the production process more efficient.

Anomaly detection will undoubtedly shape the future of many industries, making our lives easier and healthier. After all, who wouldn’t want to receive a product that underwent a thorough quality check?

If you’re looking for a dedicated partner to help you develop custom models for computer vision and monitoring production lines, look no further and contact us today. We will be more than happy to help you take your business to the next level!

References

Huang, H., Tang, X., Wen, F. et al. Small object detection method with shallow feature fusion network for chip surface defect detection. Sci Rep 12, 3914 (2022). Nature

About the author

Dorota Owczarek

Dorota Owczarek

AI Product Lead & Design Thinking Facilitator

Linkedin profile Twitter

With over ten years of professional experience in designing and developing software, Dorota is quick to recognize the best ways to serve users and stakeholders by shaping strategies and ensuring their execution by working closely with engineering and design teams.
She acts as a Product Leader, covering the ongoing AI agile development processes and operationalizing AI throughout the business.

Would you like to discuss AI opportunities in your business?

Let us know and Dorota will arrange a call with our experts.

Dorota Owczarek
Dorota Owczarek
AI Product Lead

Thanks for the message!

We'll do our best to get back to you
as soon as possible.

Becoming AI Driven

Insights on practical AI applications just one click away

Sign up for our newsletter and don't miss out on the latest insights, trends and innovations from this sector.

Done!

Thanks for joining the newsletter

Check your inbox for the confirmation email & enjoy the read!

This site uses cookies for analytical purposes.

Accept Privacy Policy

In the interests of your safety and to implement the principle of lawful, reliable and transparent processing of your personal data when using our services, we developed this document called the Privacy Policy. This document regulates the processing and protection of Users’ personal data in connection with their use of the Website and has been prepared by Nexocode.

To ensure the protection of Users' personal data, Nexocode applies appropriate organizational and technical solutions to prevent privacy breaches. Nexocode implements measures to ensure security at the level which ensures compliance with applicable Polish and European laws such as:

  1. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (published in the Official Journal of the European Union L 119, p 1); Act of 10 May 2018 on personal data protection (published in the Journal of Laws of 2018, item 1000);
  2. Act of 18 July 2002 on providing services by electronic means;
  3. Telecommunications Law of 16 July 2004.

The Website is secured by the SSL protocol, which provides secure data transmission on the Internet.

1. Definitions

  1. User – a person that uses the Website, i.e. a natural person with full legal capacity, a legal person, or an organizational unit which is not a legal person to which specific provisions grant legal capacity.
  2. Nexocode – NEXOCODE sp. z o.o. with its registered office in Kraków, ul. Wadowicka 7, 30-347 Kraków, entered into the Register of Entrepreneurs of the National Court Register kept by the District Court for Kraków-Śródmieście in Kraków, 11th Commercial Department of the National Court Register, under the KRS number: 0000686992, NIP: 6762533324.
  3. Website – website run by Nexocode, at the URL: nexocode.com whose content is available to authorized persons.
  4. Cookies – small files saved by the server on the User's computer, which the server can read when when the website is accessed from the computer.
  5. SSL protocol – a special standard for transmitting data on the Internet which unlike ordinary methods of data transmission encrypts data transmission.
  6. System log – the information that the User's computer transmits to the server which may contain various data (e.g. the user’s IP number), allowing to determine the approximate location where the connection came from.
  7. IP address – individual number which is usually assigned to every computer connected to the Internet. The IP number can be permanently associated with the computer (static) or assigned to a given connection (dynamic).
  8. GDPR – Regulation 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of individuals regarding the processing of personal data and onthe free transmission of such data, repealing Directive 95/46 / EC (General Data Protection Regulation).
  9. Personal data – information about an identified or identifiable natural person ("data subject"). An identifiable natural person is a person who can be directly or indirectly identified, in particular on the basis of identifiers such as name, identification number, location data, online identifiers or one or more specific factors determining the physical, physiological, genetic, mental, economic, cultural or social identity of a natural person.
  10. Processing – any operations performed on personal data, such as collecting, recording, storing, developing, modifying, sharing, and deleting, especially when performed in IT systems.

2. Cookies

The Website is secured by the SSL protocol, which provides secure data transmission on the Internet. The Website, in accordance with art. 173 of the Telecommunications Act of 16 July 2004 of the Republic of Poland, uses Cookies, i.e. data, in particular text files, stored on the User's end device.
Cookies are used to:

  1. improve user experience and facilitate navigation on the site;
  2. help to identify returning Users who access the website using the device on which Cookies were saved;
  3. creating statistics which help to understand how the Users use websites, which allows to improve their structure and content;
  4. adjusting the content of the Website pages to specific User’s preferences and optimizing the websites website experience to the each User's individual needs.

Cookies usually contain the name of the website from which they originate, their storage time on the end device and a unique number. On our Website, we use the following types of Cookies:

  • "Session" – cookie files stored on the User's end device until the Uses logs out, leaves the website or turns off the web browser;
  • "Persistent" – cookie files stored on the User's end device for the time specified in the Cookie file parameters or until they are deleted by the User;
  • "Performance" – cookies used specifically for gathering data on how visitors use a website to measure the performance of a website;
  • "Strictly necessary" – essential for browsing the website and using its features, such as accessing secure areas of the site;
  • "Functional" – cookies enabling remembering the settings selected by the User and personalizing the User interface;
  • "First-party" – cookies stored by the Website;
  • "Third-party" – cookies derived from a website other than the Website;
  • "Facebook cookies" – You should read Facebook cookies policy: www.facebook.com
  • "Other Google cookies" – Refer to Google cookie policy: google.com

3. How System Logs work on the Website

User's activity on the Website, including the User’s Personal Data, is recorded in System Logs. The information collected in the Logs is processed primarily for purposes related to the provision of services, i.e. for the purposes of:

  • analytics – to improve the quality of services provided by us as part of the Website and adapt its functionalities to the needs of the Users. The legal basis for processing in this case is the legitimate interest of Nexocode consisting in analyzing Users' activities and their preferences;
  • fraud detection, identification and countering threats to stability and correct operation of the Website.

4. Cookie mechanism on the Website

Our site uses basic cookies that facilitate the use of its resources. Cookies contain useful information and are stored on the User's computer – our server can read them when connecting to this computer again. Most web browsers allow cookies to be stored on the User's end device by default. Each User can change their Cookie settings in the web browser settings menu: Google ChromeOpen the menu (click the three-dot icon in the upper right corner), Settings > Advanced. In the "Privacy and security" section, click the Content Settings button. In the "Cookies and site date" section you can change the following Cookie settings:

  • Deleting cookies,
  • Blocking cookies by default,
  • Default permission for cookies,
  • Saving Cookies and website data by default and clearing them when the browser is closed,
  • Specifying exceptions for Cookies for specific websites or domains

Internet Explorer 6.0 and 7.0
From the browser menu (upper right corner): Tools > Internet Options > Privacy, click the Sites button. Use the slider to set the desired level, confirm the change with the OK button.

Mozilla Firefox
browser menu: Tools > Options > Privacy and security. Activate the “Custom” field. From there, you can check a relevant field to decide whether or not to accept cookies.

Opera
Open the browser’s settings menu: Go to the Advanced section > Site Settings > Cookies and site data. From there, adjust the setting: Allow sites to save and read cookie data

Safari
In the Safari drop-down menu, select Preferences and click the Security icon.From there, select the desired security level in the "Accept cookies" area.

Disabling Cookies in your browser does not deprive you of access to the resources of the Website. Web browsers, by default, allow storing Cookies on the User's end device. Website Users can freely adjust cookie settings. The web browser allows you to delete cookies. It is also possible to automatically block cookies. Detailed information on this subject is provided in the help or documentation of the specific web browser used by the User. The User can decide not to receive Cookies by changing browser settings. However, disabling Cookies necessary for authentication, security or remembering User preferences may impact user experience, or even make the Website unusable.

5. Additional information

External links may be placed on the Website enabling Users to directly reach other website. Also, while using the Website, cookies may also be placed on the User’s device from other entities, in particular from third parties such as Google, in order to enable the use the functionalities of the Website integrated with these third parties. Each of such providers sets out the rules for the use of cookies in their privacy policy, so for security reasons we recommend that you read the privacy policy document before using these pages. We reserve the right to change this privacy policy at any time by publishing an updated version on our Website. After making the change, the privacy policy will be published on the page with a new date. For more information on the conditions of providing services, in particular the rules of using the Website, contracting, as well as the conditions of accessing content and using the Website, please refer to the the Website’s Terms and Conditions.

Nexocode Team