Spatial noise and its reduction methods

Spatial noise in imaging is the variations or irregularities in the pixel values of an image that are unrelated to the actual features of the scene being captured. This type of noise occurs at the pixel level and can introduce unwanted randomness or fluctuations in the visual information.

Spatial noise majorly occurs due to sensor imperfections. The individual pixels in a sensor vary from each other in terms of the sensitivity of minor defects. This leads to spatial noise in the image captured.

There are also various other sources of spatial noise. External environmental factors such as temperature difference, electromagnetic interference, and vibration can also introduce spatial noise into the imaging system. Spatial noise can also occur during the digitisation process when the continuous range of analog signal values is converted into discrete digital values. This leads to quantisation errors and, in turn, gives rise to quantisation noise.

In this blog, we’ll be covering the different types of spatial noise reduction methods.

What is convolution operation?

Convolution is a formal mathematical operation used in image processing applications. The different types of noise reduction methods in the following section are done using convolution operation.

Convolution is done using a small matrix called a kernel or filter comprised of integers. Differently sized kernels containing different patterns of numbers produce different results under convolution. The size of the kernel is arbitrary, but 3×3 is used more often. The anchor point in the kernel (See Figure 1) is used to know the position of the kernel with respect to the image.

Figure 1: Kernel or filter

The kernel is slid or convolved across the entire image. The positioning of the kernel on top of the image pixel starts from the top left corner. And it is moved sequentially till the end. At each position, the element-wise multiplication of the kernel and the underlying image patch is computed. This is followed by summing up these products to obtain the value for the corresponding pixel in the output image.

(I*K)(x,y)=∑i∑jI(x+i,y+j)⋅K(i,j)

Mathematically, the convolution operation for a 2D image (I) and a 2D kernel (K) is given by:

The elements at the centre, i.e., in [2,2] of the image matrix under the convolution kernel, will be given the weighted value resulting from the above formula (the centre position [2,2] can vary depending on the convolution kernel window size). This process is repeated for the entirety of the image pixels, and the pixel’s integer values are replaced with the resulting values.

Let us take a look at different filtering techniques by which noise reduction is done in images.

Types of spatial de-noising with convolution

Average smoothing

In average smoothing, the intensity difference between pixel-to-pixel (i.e., the noise in the image) is reduced by an averaging technique. The kernel/matrix used in average smoothing is shown below.

Figure 2: Kernel used in average smoothing

In average smoothing, the kernel window used is a square matrix with equal weights. It is usually a square matrix of size (n x n), where ‘n’ is an odd number. The ‘n’ being odd is to have a well-defined, symmetric neighbourhood around each data point. The above given (Figure 2) matrix is a 3x3 kernel, with the weights evenly distributed.

The kernel window will be placed above the image pixel being processed (i.e., slide the kernel across each data point in the image). The average of the data points in the image matrix under the kernel window is calculated. The current data point in the image pixel is replaced with the calculated average. This process is repeated for every data point in the image.

Figure 3: Average smoothing example

In the above example, the kernel window is placed over the green highlighted area in the image. The centre pixel of the image ‘37’ is replaced with the weighted average of all the pixel data points under the kernel window. Here, the weighted average ‘39’ is calculated as follows,

Weighted Average=((30×1)+(35×1)+(37×1)+(40×1)+(37×1)+(43×1)+(40×1)+(43×1)+(42×1))/9

In the case of image pixels in the boundary, the convolution kernel extends beyond the image boundaries. There are several common approaches to solving this boundary issue. One such method commonly used to deal with the boundary pixels during convolution is called zero-padding (zero-fill).

In the zero-padding Technique, pixels outside the image boundary are assumed to have a value of zero. This is the simplest method, where pixels beyond the image boundary are treated as if they have zero intensity. There are also other methods used to deal with the boundary problem, such as mirror padding, wrap-padding, clamp padding, edge padding, etc.

The following figure shows the before and after of an image that has undergone average smoothing denoising.

Figure 4: Before and after average smoothing

The average smoothing process effectively blurs or smoothens the signal, reducing the impact of random noise. However, it’s essential to strike a balance with the window size since a window that is too small might not effectively smooth the noise, and a window that is too large might blur the signal too much, losing important details.

Keep in mind that average smoothing is a basic technique and more sophisticated methods like Gaussian smoothing or median filtering are often used for better noise reduction in specific applications.

Gaussian filter

Gaussian filter is a spatial noise reduction method used for reducing high-frequency noise. This filtering technique uses the Gaussian function to perform convolution.

The difference between the Gaussian filtering technique and average smoothing is that, in Gaussian filtering, more weight values are assigned to the points closer to the centre pixels, and lesser weight values are assigned to points far away from the centre pixel. This results in a smoother transition. This varying weight distribution allows the Gaussian filter to reduce noise effectively while preserving important details in the image.

In the case of a Gaussian filter, the kernel is based on the discrete approximation of the Gaussian function.

Figure 5: Example of gaussian filter kernel with standard deviation of 1

In the above example, the standard deviation is assumed to be 1. The values in the matrix are the weights assigned to the neighbouring pixels during the convolution operation. The central pixel has the highest weight, and the weights decrease as we move away from the centre.

To apply the Gaussian filter, you convolve the image with this kernel (Figure 4) kernel using standard convolution techniques. The resulting pixel value at each position is a weighted average of the surrounding pixels, with higher weights assigned to closer neighbours according to the Gaussian distribution.

Figure 6: Before and after gaussian filtering

Median filter

The Median filtering technique is commonly used for removing salt-and-pepper noise. It is widely used in the medical field as this kind of noise is prevalent in medical imaging.

Just as in other filtering techniques discussed above, in a Median filter, a kernel matrix is slid over the entire image. Move the selected window over each pixel in the image, covering one pixel at a time. For each window placement, collect the pixel values within the window and sort them in ascending order. The median is the middle value in the sorted list. Assign the median value as the new value for the pixel at the centre of the window. This process is repeated for every pixel in the image.

Figure 7: Median filtering example

Figure 8: Before and after median filtering

The key advantage of median filtering is its ability to effectively remove salt-and-pepper noise (random bright and dark pixels) in an image without significantly blurring edges. Since the median is less sensitive to extreme values than the mean, it tends to preserve edges and details in the image.

Bilateral filter

Bilateral filtering is a non-linear, edge-preserving noise filtering technique. It is particularly effective in smoothing images while preserving edges. The filter takes into account both the spatial distance between pixels and the intensity differences, allowing it to retain important image features while reducing noise.

The steps involved in the Bilateral filtering technique are as follows:

• Select a Window: Similar to other filtering techniques, bilateral filtering also uses a window or a kernel of a specific size (commonly 3×3, 5×5, or 7×7). This window will slide over the entire image.
• Calculate Gaussian Weights for Spatial Distance: For each pixel in the window, calculate a spatial weight based on the spatial distance between the centre pixel and the surrounding pixels. The weight is usually calculated using a Gaussian function, giving higher weights to nearby pixels and lower weights to distant pixels.
• Calculate Gaussian Weights for Intensity Differences: For each pixel in the window, calculate an intensity weight based on the difference in intensity values between the centre pixel and the surrounding pixels.
• Compute the Weighted Average: Compute the weighted average of the intensity values within the window using the bilateral filter weights.
• Repeat for Each Pixel: Slide the window over each pixel in the image and repeat the process, computing the weighted average for each pixel.

Bilateral filtering does not blur the images too much because it considers both how far pixels are from each other and how different their colours are. This helps preserve the important details, like edges in the image.

Figure 9: Before and after bilateral filtering

The key difference in bilateral filtering is that the weights in the convolution kernel are not fixed but depend on the spatial and intensity relationships between pixels.

Nonlocal means

Non-local Means (NLM) is a technique used in image denoising, aiming to reduce noise while preserving image details. Unlike traditional denoising methods, which operate locally on each pixel or small neighbourhood, non-local means consider similarities across the entire image for better noise reduction. The NLM does not use the convolution technique to do the filtering but uses the NLM algorithm.

The steps involved in the NLM technique are as follows:

• Similarity Measurement: For each pixel in the image, the non-local means algorithm measures the similarity between the pixel and all other pixels in the image. This is done by comparing patches (small regions around each pixel) rather than individual pixel values.
• Patch Matching: The algorithm looks for patches in the image that are similar to the patch around the target pixel. The similarity is often measured using a weighted Euclidean distance, considering both spatial and intensity differences.
• Weighted Averaging: Once similar patches are identified, their pixel values are combined to calculate a weighted average. The weights are determined by the similarity between the patches. Patches that are more similar contribute more to the average.
• Noise Reduction: The calculated weighted average becomes the denoised value for the target pixel. This process is repeated for every pixel in the image, resulting in a denoised version of the original image.

Figure 10: Scheme of NL-means strategy.

It can be observed from the above figure that, similar pixel neighbourhoods give a large weight, w(p, q1) and w(p,q2), while much different neighbourhoods give a small weight w(p,q3).

The non-local means technique leverages the idea that similar structures in an image should have similar noise characteristics. By final output image’s non-local similarities, the algorithm can effectively distinguish between signal and noise, leading to improved denoising performance.

Weiner filter

The Wiener filter assumes that the given image is a combination of a true underlying image (signal) and additive noise. The Wiener filter calculates the Power Spectral Density (PSD) of both the signal (the true image) and the noise. The PSD provides information about how the energy is distributed across different frequencies in the signal and the noise. It considers the Signal-to-Noise Ratio (SNR), which is a measure of how strong the signal is compared to the noise. A high SNR means a strong signal and less noticeable noise.

Using the information from the PSD and SNR, the Wiener filter creates a filter function. This function is designed to attenuate (reduce) frequencies dominated by noise while preserving frequencies dominated by the signal. The filter adapts its behavior based on the characteristics of the signal and noise.

In simpler terms, the Wiener filter is like a smart cleaner for images that understands which parts of the image are important (signal) and which parts are unwanted noise. It then applies a special filter to keep the important stuff while reducing the noise, resulting in a clearer and cleaner image. The key is adapting the filter based on the statistical properties of the signal and noise, making it a powerful tool for image denoising.

Figure 11: Before and after images of weiner filter

Low-noise cameras offered by e-con Systems

e-con Systems has over 20+ years of experience in designing, developing, and manufacturing OEM cameras. Over the years, they have developed a wide range of low-noise cameras with efficient noise reduction algorithms, in-built ISP systems, etc.

Some of its popular low-noise cameras are listed below:

• See3CAM_CU55 – 1/2.5″ 5.0 MP Low Noise USB Camera (Colour)
• See3CAM_CU81 – AR0821 4K HDR USB Camera with 1/1.7″ sensor
• See3CAM_CU22 – Full HD AR0233 HDR USB Camera with LFM
• See3CAM_CU20 – Full HD WDR HDR USB 3.1 Camera