Noise in images, friend or foe? Engineers have been asking themselves this question over and over since the invention of electronic imaging devices.
Clearly, from an end user experience, a noisy image is not a good image. This is an issue made more difficult because of the high sensitivity of the eye for the smallest of deviations. Also, for algorithms, noise leads to more efforts in analysis and less accuracy. For a camera manufacturer, the noise contains a plethora of information about the sensor and the electronic driving it. Thus, the noise is a developer’s best friend and if carefully analyzed will lead to the best possible camera design and implicitly to the best possible image for a given sensor and electronic components.
There are many sources which can introduce noise into the image. Some of them are created by the sensor design, some are generated by the additional electronic components which are contained in a camera. Some sources can be avoided, some not but luckily many can be corrected. This article shows an enumeration of the most common noise sources and information about possible methods to avoid them. One can separate the various contributions to the total noise in fixed pattern noise which is spatially fixed and random or temporal noise coming from contributions from sources which change with every frame.
1. Fixed pattern noise
The fixed pattern noise is spatially fixed, meaning that from image to image it does not change position. This type of noise is from a user point of view very disturbing as the human brain is very sensitive to visual patterns. If the image is analyzed by an algorithm, this type of noise can be easily ignored.
1.1 Dark current noise
Charge generation is an intrinsic property of the semiconductor materials. Due to slight differences in manufacturing the dark current generated in each pixel will vary slightly, thus generating a certain spatial pattern. This pattern can be corrected by subtracting a dark reference frame from the acquired frames. One has to update the dark reference frame for various temperatures, as the charge generation is temperature dependent. Thus the DSNU will be corrected but its noise is not. As the charge generation follows a Poisson distribution, the noise associated with it equal the square root of the dark current value.
1.2 Column FPN
In CMOS sensors the pixels in each column share the Analog-to-Digital Converter (ADC). The differences between these circuits, mainly consisting in different offsets, will lead to a slight difference between the values of adjacent columns of pixels. Even a one per cent variation is perceived a very strong. Various methods are used to determine the offset of each column and compensate for it, in order to get a homogenous image. Usually this involves using the value of the so called optically black pixels which are located at the top of the column and are shielded from light.
1.3 Defects
As already mentioned the manufacturing processes are not perfect. Defects in the form of contaminant atoms or modifications of the crystalline structure of the semiconductor material can be unwillingly introduced. These defects will generate charges in excess or drain them, thus contributing to the spatial noise. Typically these defects are corrected by an algorithm which replaces the values of the affected pixels with the mean value of neighboring pixels.
1.4 Photo-response non-uniformity
This type of noise has its origin in the structure of the pixels. Due to manufacturing uncertainties, the pixels are almost identical, the signal output will also be almost the same. As each pixel has a linear response with the illumination intensity, the differences between the signal of the pixels will also stay constant. Considering this, it is possible to devise an algorithm to correct for the variations. Close to the saturation capacity, the pixels usually deviate from linearity and such an algorithm will fail but the sensor manufacturers usually configure the sensor in such a way as to prevent the pixels from reaching their nominal saturation capacity.
2. Random noise
As the name says it, the random, also known as temporal, noise occurs unpredictably. From a user experience point of view this type of noise if not perceived as strongly as the fixed pattern noise as our brain interpretation of images is more robust against it. For an algorithm, this type of noise can be problematic as it is pretty difficult to distinguish it from fine textures. From an engineering point of view, the unpredictability it is definitely an issue, requiring active compensation mechanisms. One of the simplest methods is averaging a number of images.
2.1 Photon-shot noise
The photons arrive at the sensor not all at the same time but according to a Poisson distribution over time. The photon shot noise, or shot noise, is equal to the square root of the number of photons captured by the pixel. This kind of noise cannot be corrected for but knowing the illumination intensity and quantum efficiency of the pixel, it can be quite well estimated.
2.2 Reset noise
Charges accumulated in a photodiode have to be converted into a voltage. This is done with the help of a capacitance. After each conversion cycle this capacitance has to be reset to a certain reference voltage. This process happens with a certain uncertainty, dependent on temperature and capacitance value. This is why this noise is also known as kTC noise. Typically, this is corrected via the so called Correlated Double Sampling method, meaning that the voltage value of the capacitance is read immediately after reset and this value is seen as an offset of the final value. This is either implemented in the circuitry for CMOS sensors or a specialized IC takes care of it for CCD sensors.
2.3 Quantization noise
An Analog-to-Digital Converter, or in short ADC, will convert an analog into a digital signal. During this conversion some information gets lost, as a certain range of analog values will correspond to a digital step. The noise introduced by this is equal with the analog value divided by √bitdepth ADC. This type of noise cannot be corrected.
The Impact of Noise on Sensor Performance
The noise generated by a sensor greatly influences its sensitivity and its image quality. A sensor’s performance is proportional to its quantum efficiency; that is, the amount of signal generated by a photon, and is inversely proportional to its noise level. In addition, the maximum dynamic range of the achievable signal values is always reduced by noise to a certain degree. For example, dark areas can be made visible by simply manipulating the signal with increased gain to a fixed value. However, this method of gain increases noise at the same time, and often diminishes image quality.
Typically, signal-to-noise ratio (SNR) is the parameter employed for determining the quality of a sensor; i.e., SNR is the ratio of signal value to noise value (db). For image sensors, the SNR is the digital gray value minus the dark value (signal), divided by the standard deviation of the digital gray value (noise). Stated algebraically, SNR= msig/ssig.
For an ideal sensor with a quantum yield of one and without dark noise, SNR is the square root of the exposure which is expressed as the number of photons per pixel. In practice, an actual sensor cannot exceed this definition. In good lighting conditions, a real sensor can come very close to this value, but in low light (i.e., weak signals) dark noise greatly reduces the SNR.
Therefore, SNR is the practical value for expressing the dynamic range of the remaining signal after analog-to-digital conversion. Even though the human eye can only distinguish a maximum of 60 gray levels, image analysis algorithms can use a higher SNR to improve detection accuracy. Consequently, low noise and intelligent image processing can lead to more precise data acquisition.
In addition to selecting the optimum sensor, noise can be reduced through better lighting and lower heat output in the vision system. Experienced image processing experts, like those from the FRAMOS global team, can advise how best to avoid and correct noise. We at FRAMOS provide detailed advice about system components; their specifications and interdependencies; and, measurements and test setups to ensure that the image result is as noise-free as possible. During evaluation, we help customers set up, implement, and configure the best possible application architecture. Therefore, while noise is never going to help you, it does not have to hinder you.