Fusing Data Processing in the Construction of Machine Vision Systems in Robotic Complexes

The development of machine vision systems is based on the analysis of visual information recorded by sensitive matrices. This information is most often distorted by the presence of interfering factors represented by a noise component. The common causes of the noise include imperfect sensors, dust and aerosols, used ADCs, electromagnetic interference, and others. The presence of these noise components reduces the quality of the subsequent analysis. To implement systems that allow operating in the presence of a noise, a new approach, which allows parallel processing of data obtained in various electromagnetic ranges, has been proposed. The primary area of application of the approach are machine vision systems used in complex robotic cells. The use of additional data obtained by a group of sensors allows the formation of arrays of usefull information that provide successfull optimization of operations. The set of test data shows the applicability of the proposed approach to combined images in machine vision systems..


Introduction
The transition to new digital technologies made it possible to introduce automation into the processes performed by robotic centers and allowed the creation of complex systems allowing with the redistribution of functions. Data on the performance of operations, the current position of objects in space, as well as the entire technological process, can come in various forms. It can be: one-dimensional signals from sensors and state machines; two-dimensional signals from the machine vision system, including the visible and invisible by man wavelength spectrum; multidimensional signals, including data on the current position of the object in space. The use of combined information can improve the accuracy of operations performed, as well as introduce additional criteria to reduce the degree of inaccurate operations. The data obtained in the form of onedimensional signals are most often informative and allow the implementation of simple operations. The use of multidimensional signals allows the introduction of fuzzy conditions, such as: the exact location of the workpiece parts on the robot desktop, the organization of zero injury systems, the implementation of INDUSTRY 5.0 systems [1][2][3]. The use of multi-range images in these tasks allows us to eliminate the invisibility of part of the process due to difficult conditions, as well as to form optimal procedures for performing actions by an automated complex. The use of the visible spectrum in conjunction with the near-infrared data allows operations to be performed in the presence of a small water suspension and dust. The combination of visible and far-infrared data allows you to make a decision about overheating of working tools, quality and speed of operations [4].
The construction of machine vision systems is based on the analysis of visual information recorded by sensitive matrices. Most often, this information is distorted by the presence of interfering factors represented by the noise component. The presence of this component reduces the quality of the subsequent analysis. To implement systems that allow operation in difficult conditions, an approach is proposed in the work that allows parallel processing of data obtained in various electromagnetic ranges.
The area of the use is the approach proposed in this work may be machine vision systems used in complex robotic cells. The use of additional data obtained by a group of sensors allows the formation of advanced information and the subsequent optimization of operations. The set of test data shows the applicability of the proposed approach for combined images in machine vision systems.

The Fusing data processing
Combining data into a single information field is an essential task for systems that improve the accuracy of decision making. Combinations can be used as primary data: contour of objects and their temperature gradients; images of objects in grayscale and temperature fields arising around them; building 3D volumetric objects with information about its color or temperature gradient on its surface, etc. For data fusion, a common condition is one size and a fixation condition for the same process. Figure 1 shows a block diagram of a data fusion algorithm. The algorithm shown in Figure 2 consists of the following main steps: 1. The load images. 2. Convert images to a single format (change size). This operation can be performed using super-resolution or data compression algorithms [5]. The choice of operation depends on the priority of the data range.
3. Search for base elements (reference points). For the search, it is necessary to use algorithms for searching saliency maps and the boundaries of objects.
4. Algorithm for searching correspondence between objects located on images. The analysis can be carried out by searching for distances between the referents points and finding the optimal graph or using correlation analysis of the mutual arrangement of data structures [6]. 5. Select data priority. On the secondary data of searching for the boundaries of objects.
6. The images fusion and saving the result.

The algorithm for the search for objects of significance (detailed objects)
The algorithm of the based on the analysis of image areas in a sliding window. As the first step, a edges detector based on the Sobel and Prewitt detector is used. This detector is showing good results for both noisy and blurry images.
When searching for specific areas, the method "Density" is used, it is based on the search for objects in a particular area [6]. At the first stage of the method, the overall coefficient of detail in the whole image is considered, which is determined by the formula: when ) , ( y x I -pixel value with x and y coordinates; i -is the number of rows; j -is the number of columns; P -is the coefficient of detail. The coefficient of detail P is determined automatically for each image and its selection does not require empirical intervention. Then, in a similar manner, the density in each sliding window is calculated (2): The coefficient 0,1 is averaging associated with automatic selection of window size equal to 10% of the total image.
The "Density" all P and win P are compared, and completion of a decision is made on the about detail in this window. The threshold is set according to the given conditions on the number of boundaries found in the study area ( Figure 2).  On the based on the experimental results, it was found that the number of borders should not be less than two in one window, which corresponds to j i P P win all ⋅ ⋅ ≥ 2 , 0 . Next, the window is shifted by a pixel. On the based on the experimental results in paper [6], it was found that the best window size suitable for this task is a window of 5% of the whole image. In order to accelerate the operation of the method, an algorithm is proposed that skips "empty" windows (with missing contours) from subsequent analysis. The above description of the algorithm is implemented as follows ( Figure 2). The algorithm shown in Figure 2 consists of the following main steps: 1. The load image. 2. Search for the boundaries of objects in the image and the calculation of the total density of objects throughout the image.
3. Allocation of the local analysis window. 4. Calculation of the density of points in the local region P.
5. Comparison of the total density and calculated coefficient in the local window. If the local coefficient exceeds, then a decision is made to select a region since it has a large number of transitions and a complex structure.
6. Combining selected areas. Figure 2 shows an example of the selection of such areas in the test image.

Algorithm for searching the center of mass of objects and an equidistant curve for closed areas with objects inside
To search for local features in the images obtained in the on different diapason, a step-by-step analysis of binaryclosed sections determined at part 2 is used. The algorithm for determining the center of mass of objects is part of the algorithm for finding the equidistant curve [9], which is shown in Figure 3.  The algorithm presented in Figure 3 is implemented as follows: 1. The loading of local areas and the boundaries of the objects obtained in the previous stages of analysis (part 2).
2. Determination of the number of closed curves 3. Checking the conditions for the inclusion of areas in each other and the localization of empty areas (without objects inside).
4. For local areas without objects inside, the center of mass is determined by stepwise pulling the boundary inward, with the absorption of coinciding values.
5. For areas with the presence of internal objects, the calculation of the equidistant curve, by contracting the outer border and the inner borders of the located objects in the direction to each other.
6. Determination of points of change of direction by 90 degrees by analysis of 3 neighboring pixels of the equidistant curve.

Experimental results of the fusing data
As test data, use images of robotic complexes. Frames were acquired in the infrared range using a flir c2 thermal imaging camera and a visible spectrum on the Samsung camera. For fusion images is used algorithm of the present in figure 1. The camera shift was more than 0,5 meters. The distance from the chamber to the facility was 0,7 meters, the ambient temperature was + 20С (+ 68F).