Fringe Pattern Denoising using U-Net based neural network

Fringe visibility and noise removal, are key success factors in interferometric techniques, where novel deep learning techniques can be applied. We test the use U-Net deep convolutional network applied to the obtained interference images, trained with an ad-hoc generated image dataset with complex fringe patterns, computed using high order Zernike polynomials.


Introduction
Interferometric techniques have been used along years as one of the most important methods for non-contact measurements and quality assessment of wavefronts coming from objects surfaces, but one of the most relevant caveats for accurate measurements is the noise level presented in the obtained interferograms. To deal with this caveat image filtering techniques have been applied to remove the noise (specially speckle or salt & pepper noise) from the interferograms, using more or less complex filters based on Fourier transforms [1], or other filters for high frequency noise in the spatial domain [2]. More specifically for the speckle pattern interferometry, the issue coming from the background noise and the need to increase fringe visibility are key factors to the success of the techniques, making highly complex the increase of fringe visibility to facilitate the image interpretation in real time.
During recent years, Machine Learning (ML) techniques have been significantly improved in the hardware and software fields, current GPU hardware is able to accelerate usual tensor operations present in ML techniques. In addition, software frameworks like Keras, TensorFlow and Pytorch are able to accelerate the implementation of new Neural Network (NN) models and remove the implementation complexity to make affordable the use of these technologies, reducing its time to market. In this study, we will use these techniques to increase the visibility of interference fringes in speckle pattern interferometry, designing and training a deep convolutional network based in U-NET [3] architecture to denoise speckle interference fringes. We will train the model with a synthetic and heavily noised collection of images, performing all required training activities using public services in the cloud (Google Research Colaboratory) given the current confinement status across countries.

The U-Net Architecture
The proposed layout for the denoiser NN is based on the U-NET architecture, well know model broadly used to denoise images [4], and recently used to denoise wrapped phase maps for phase quantitative evaluation and SAR images [5].
The U-Net architecture is a convolutional encoderdecoder with internal connections between encoder and decoder paths (skip connections) with a well-proven performance when it is applied to denoise problems. Is based in one encoder path where 2D-Convolutional (Conv2d) and maxpooling layers are combined to reduce the spatial resolution of the input image capturing the image detail (features) with spatial filters in the Conv2D layers generating feature channels. Once the image has been fully encoded, there is a decoder path reversing the encoding operations.
The significant increase of performance in U-Net architecture over the traditional Convolutional Neural Networks (CNN) is the use (in the decoder path) of skip connections to improve the performance and minimise the training time. The skip connections are connections joining the outputs of the encoding steps with the equivalent decoding layer, introducing part of the encoding information to minimise the information loss in the decoding layers.

Model Design
The U-Net layout implemented has the following blocks,

Training
The most relevant aspect designing the applications of NN models to physical problems is the design of the training set. Using Zernike polynomials we followed the method designed in [6] to generate a dataset of 10.000 train images with size 128x128 pixels, based in randomly generate the coefficients for the first 14 Zernike polynomials. This set was the ground true for the inputs and for the noisy images in the training dataset, we included an additive and random noise component based on a normal distribution to align the noisy images to the typical images obtained in speckle pattern interference.
The training procedure for the model was implemented using ADAM optimisation [7] and MEAN_SQUARED_ERROR as the loss function; the training set was divided into training and validation (80% and 20%). The training process automatically stopped due to the early stop condition after 28 epochs, reaching a loss value of 0.0099 on the validation set.

Results
The trained model was applied to a new generated set of noisy images, not included in the training nor validation dataset. A sample of the obtained clean images were compared with the results of a denoise procedure using a median filter with a 3x3 kernel (Figure 1). The model improved the visibility of the fringes, removing the generated noise and increasing the visibility of the interference fringes. Structural Similarity Index (SSMI) [8] is the selected metric for the quantitative assessment of the perceived increase of quality across the reconstructed image.

Conclusions and Next Steps
The designed NN, based in U-Net architecture significantly increases the perceived quality of interference fringes, improving fringe visibility under varying noise conditions. The overall process can be implemented in cheap hardware GPU architectures, contributing to decrease the costs of interferometric equipment related to image processing.
Application to real ESPI experimental images will be done after the COVID-19 confinement.