Automated Visual Inspection - Part 1: Introduction and Review of Object Types

by E. R. Davies , TechOnline India - January 06, 2009

This excerpt from "Machine Vision: Theory, Algorithms, Practicalities" looks at the process of inspection and objects that visual inspection systems have to cope with.

Receive a 15% discount on "Machine Vision: Theory, Algorithms, Practicalities" by ordering from and using offer code 92839.

22.1 Introduction
Over the past two decades, machine vision has consolidated its early promise and has become a vital component in the design of advanced manufacturing systems. On the one hand, it provides a means of maintaining control of quality during manufacture, and on the other, it is able to feed the assembly robot with the right sort of information to construct complex products from sets of basic components. Automated assembly helps to make flexible manufacture a reality, so that costs due to underuse of expensive production lines are virtually eliminated.

These two major applications of vision - automated visual inspection and automated assembly - have many commonalities and can on the whole be performed by similar hardware vision systems employing closely related algorithms. Perhaps the most obvious use of visual inspection is to check products for quality so that defective ones may be rejected. This is easy to visualize in the case of a line making biscuits or washers at rates of the order of one million per day.

Another area of application of inspection is to measure a specific parameter for each product and to feed its value back to an earlier stage in the plant in order to "close the loop" of the manufacturing process. A typical application is to adjust the temperature of jam or chocolate in a biscuit factory when the coating is found not to be spreading correctly. A third use of inspection is simply to gather statistics on the efficiency of the manufacturing process, finding, for example, how product diameters vary, in order to provide information that will help management with advance planning.

Another aspect of inspection is what can be learned via other modalities such as X-rays, even if the acquired images often ressemble visible light images. Similarly, it is relevant to ask what additional information color can provide that is useful for inspection. Sections 22.10 and 22.11 aim to give answers to these questions.

In automated assembly, vision can provide feedback to control the robot arm and wrist. For this purpose it needs to provide detailed information on the positions and orientations of objects within the field of view. It also needs to be able to distinguish individual components within the field of view. In addition, a well-designed vision system will be able to check components before assembly, for example, so as to prevent the robot from trying to fit a screw into a nonexistent hole.

Inspection and assembly require virtually identical vision systems, the most notable difference often being that a linescan camera is used for inspecting components on a conveyor, whereas an area (whole picture) camera is required for assembly operations on a worktable. The following discussion centers on inspection, although, because of the similarity of the two types of application, many of the concepts that are developed are also useful for automated assembly tasks.

{pagebreak}22.2 The Process of Inspection
Inspection is the process of comparing individual manufactured items against some preestablished standard with a view to maintenance of quality. Before proceeding to study inspection tasks in detail, it is useful to note that the process of inspection commonly takes place in three definable stages:

  1. Image acquisition
  2. Object location
  3. Object scrutiny and measurement

We defer detailed discussion of image acquisition until Chapter 27 and comment here on the relevance of separating the processes of location and scrutiny. This is important because (either on a worktable or on a conveyor) large numbers of pixels usually have to be examined before a particular product is found, whereas once it has been located, its image frequently contains relatively few pixels, and so rather little computational effort need be expended in scrutinizing and measuring it.

For example, on a biscuit line, products may be separated by several times the product diameter in two dimensions, so that some 100,000 pixels may need to be examined to locate products occupying say 5000 pixels. Product location is therefore likely to be a much more computationally intensive problem than product scrutiny. Although this is generally true, sampling techniques may permit object location to be performed with much increased efficiency (Chapter 10). Under these circumstances, location may be faster than scrutiny, since the latter process, though straightforward, tends to permit no shortcuts and requires all pixels to be examined.

22.3 Review of the Types of Objects to Be Inspected
Before studying methods of visual inspection (including those required for assembly applications), we should consider the types of objects with which such systems may have to cope. As an example, we can take two rather opposing categories: (1) goods such as food products that are subject to wide variation during manufacture but for which physical appearance is an important factor, and (2) those products such as precision metal parts which are needed in the electronics and automotive industries. The problems specific to each category are discussed first; then size measurement and the problem of 3-D inspection are considered briefly.

22.3.1 Food Products
Food products are a particularly wide category, ranging from chocolate cream biscuits to pizzas, and from frozen food packs to complete set meals (as provided by airlines). In the food industry, the trend is toward products of high added value. Logically, such products should be inspected at every stage of manufacture, so that further value is not added to products that are already deficient.

However, inspection systems are still quite expensive, and the tendency is to inspect only at the end of the product line. This procedure at least ensures (1) that the final appearance is acceptable and (2) that the size of the product is within the range required by the packaging machine.1 This strategy is reasonable for many products—as for some types of biscuit where a layer of jam is clearly discernible underneath a layer of chocolate. Pizzas exemplify another category wherein many additives appear on top of the final product, all of which are in principle detectable by a vision system at the end of the product line.

When checking the shapes of chocolate products (chocolates, chocolate bars, chocolate biscuits, etc.), a particular complication that arises is the "footing" around the base of the product. This footing is often quite jagged and makes it difficult to recognize the product or to determine its orientation. However, the eye generally has little difficulty with this task, and hence if a robot is to place a chocolate in its proper place in a box it will have to emulate the eye and employ a full gray-scale image; shortcuts with silhouettes in binary images are unlikely to work well. In this context, it should be observed that chocolate is one of the more expensive ingredients of biscuits and cakes. A frequently recurring inspection problem is to check that chocolate cover is sufficient to please the consumer while low enough to maintain adequate profit margins.

Returning to packaged meals, we find that these present both an inspection and an assembly problem. A robot or other mechanism has to place individual items on the plastic tray, and it is clearly preferable that every item should be checked to ensure, for example, that each salad contains an olive or that each cake has the requisite blob of cream.

22.3.2 Precision Components
Many other parts of industry have also progressed to the automatic manufacture and assembly of complex products. It is necessary for items such as washers and O-rings to be tested for size and roundness, for screws to be checked for the presence of a thread, and for mains plugs to be examined for the appropriate pins, fuses, and screws. Engines and brake assemblies also have to be checked for numerous possible faults.

Perhaps the worst problems arise when items such as flanges or slots are missing, so that further components cannot be fitted properly. It cannot be emphasized enough that what is missing is at least as important as what is present. Missing holes and threads can effectively prevent proper assembly. It is sometimes stated that checking the pitch of a screw thread is unnecessary—if a thread is present, it is bound to be correct. However, there are many industrial applications where this is not true.

Table 22.1 summarizes some of the common features that need to be checked when dealing with individual precision components. Note that measurement of the extent of any defect, together with knowledge of its inherent seriousness, should permit components to be graded according to quality, thereby saving money for the manufacturer. (Rejecting all defective items is a very crude option.)

1. On many food lines, jamming of packaging machines due to oversize products is one of the major problems.

{pagebreak}22.3.3 Differing Requirements for Size Measurement
Size measurement is important both in the food industry and in the automotive and small-parts industry. However, the problems in the two cases are often rather

Table 22.1 Features to be checked on precision components

Dimensions within specified tolerances
Correct positioning, orientation, and alignment
Correct shape, especially roundness, of objects and holes
Whether corners are misshapen, blunted, or chipped
Presence of holes, slots, screws, rivets, and so on
Presence of a thread on screws
Presence of burr and swarf
Pits, scratches, cracks, wear, and other surface marks
Quality of surface finish and texture
Continuity of seams, folds, laps, and other joins

different. For example, the diameter of a biscuit can vary within quite wide limits (~5%) without giving rise to undue problems, but when it gets outside this range there is a serious risk of jamming the packing machine, and the situation must be monitored carefully. In contrast, for mechanical parts, the required precision can vary from 1% for objects such as O-rings to 0.01% for piston heads. This variation makes it difficult to design a truly general-purpose inspection system. However, the manufacturing process often permits little variation in size from one item to the next. Hence, it may be adequate to have a system that is capable of measuring to an accuracy of rather better than 1%, as long as it is capable of checking all the characteristics mentioned in Table 22.1.

When high precision is vital, accuracy of measurement should be proportional to the resolution of the input image. Currently, images of up to 512 × 512 pixels are common, so accuracy of measurement is basically of the order of 0.2%. Fortunately, gray-scale images provide a means of obtaining significantly greater accuracy than indicated by the above arguments, since the exact transition from dark to light at the boundary of an object can be estimated more closely. In addition, averaging techniques (e.g., along the side of a rectangular block of metal) permit accuracies to be increased even further - by a factor √N if N pixel measurements are made. These factors permit measurements to be made to subpixel resolution, sometimes even down to 0.1 pixel.

22.3.4 Three-dimensional Objects
All real objects are 3-D, although the cost of setting up an inspection station frequently demands that they be examined from one viewpoint in a single 2-D image. This requirement is highly restrictive and in many cases overrestrictive. Nevertheless, generally an enormous amount of useful checking and measurement can be done from one such image. The clue that this is possible lies in the prodigious capability of the human eye—for example, to detect at a glance from the play of light on a surface whether or not it is flat.

Furthermore, in many cases products are essentially flat, and the information that we are trying to find out about them is simply expressible via their shape or via the presence of some other feature that is detectable in a 2-D image. In cases where 3-D information is required, methods exist for obtaining it from one or more images, for example, via binocular vision or structured lighting, as has already been seen in Chapter 16. More is said about this in the following sections.

22.3.5 Other Products and Materials for Inspection
This subsection briefly mentions a few types of products and materials that are not fully covered in the foregoing discussion. First, electronic components are increasingly having to be inspected during manufacture, and, of these, printed circuit boards (PCBs) and integrated circuits are subject to their own special problems, which are currently receiving considerable attention. Second, steel strip and wood inspection are also very important. Third, bottle and glass inspection has its own particular intricacies because of the nature of the material. Glints are a relevant factor—as it also is in the case of inspection of cellophane-covered foodpacks. In this chapter, space permits only a short discussion of some of these topics (see Sections 22.7 and 22.8).

Coming up in Part 2: Template matching and the computation and application of the radial histogram.

Printed with permission from Morgan Kaufmann, a division of Elsevier. Copyright 2004. "Machine Vision : Theory, Algorithms, Practicalities" by E. R. Davies. For more information about this title and other similar books, please visit

Related links:
Camera design for machine vision
Using Computer Vision in Real Applications: Two Success Stories
Introduction to video analytics
Video analytics architecture and intelligent IP cameras at the edge
Smart camera basics: Matching image sensor, analytics and processor performance
Machine vision 'bot' rocks out on Guitar Hero

About Author


blog comments powered by Disqus