7. Denoising

Painting detail by Louis Peyré (1923–2012). About the paintings.
Denoising estimates a clean signal from measurements corrupted by noise. Its central difficulty is to suppress random fluctuations without erasing edges, textures, or other features of the underlying signal. We model common acquisition noise, compare linear filters with nonlinear thresholding, analyze how regularity and sparsity govern reconstruction error, and adapt the methods to Poisson and multiplicative noise.
7.1 Noise Modeling¶
7.1.1 Noise in Images¶
Figure 7.1 illustrates the variety of imaging noise. Different acquisition devices produce different noise statistics, while the images themselves vary in smoothness and geometry.
Figure 7.1. Gaussian, Poisson, and multiplicative Gamma noise simulated on the same flower image.
The model should reflect both the acquisition process and the noise statistics. The regularity and geometry of the clean signal then guide the choice of a basis for its representation.
Since noise perturbs discrete measurements from acquisition devices, we consider only finite-dimensional real signals .
7.1.2 Image Formation¶
Figure 7.2 summarizes the model: a clean image combines with noise to produce the observation . The operation may be addition or multiplication. The observed image is modeled as a random vector.
Figure 7.2. Image acquisition, noise, and reconstruction by denoising.
In the statistical model, is a random vector with a specified distribution. A numerical experiment uses one realization, also written .
Additive Noise.
The simplest image formation model adds noise to a clean signal
where is the noise residual. Both and are random; an acquired image is one realization of , denoted by or below. Figures 7.3 and 7.4 illustrate additive noise on a signal and an image.
Figure 7.3. One-dimensional additive Gaussian noise at low, medium, and high levels, with a common clean signal and normalized noise realization.
Figure 7.4. Two-dimensional additive noise: the clean image and the observation , with .
The simplest model assumes independent centered Gaussian entries with variance : . This is Gaussian white noise.
Depending on the image acquisition device, other noise distributions may be appropriate, including uniform noise or heavy-tailed generalized Gaussian noise with density
In many situations, the noise is not additive; its intensity may, for instance, depend on the signal intensity. This is the case with the Poisson and multiplicative noise models considered in Section 7.4.
7.1.3 Denoiser¶
A denoiser produces an estimate of from the observation alone. The map is deterministic, but its output is random because it depends on the noise . With centered additive noise, has mean , so denoising estimates that mean from a single realization. Figure 7.5 illustrates the result.
The quality of a denoiser is measured using the mean squared risk , where averages over the noise . This risk is a theoretical quantity because is unknown in practice. In experiments with known clean data, we instead evaluate one realization using the signal-to-noise ratio , defined by
The SNR is measured in decibels (dB). Computing it requires the clean signal , so it serves as a benchmark in controlled experiments. Because an error does not fully describe perceptual quality, visual inspection should accompany the numerical comparison.
Figure 7.5. Left: clean image, center: noisy image, right: denoised image.
7.2 Linear Denoising by Filtering¶
7.2.1 Translation Invariant Estimators¶
A linear estimator of preserves addition, , and scalar multiplication. A translation-invariant estimator commutes with translations: , where . We assume periodic boundary conditions, in either one or two dimensions.
An estimator with both properties is a convolution filter:
where is a filter. Denoising filters are usually low-pass. To preserve constant signals, one imposes
where is the discrete Fourier transform.
Figure 7.6 illustrates denoising with a low-pass filter.
Figure 7.6. Gaussian filtering: noisy signal, kernel, and filtered signal (top), with their Fourier magnitudes below.
The filtering strength is usually controlled by the width of . A typical example is the (discretized) Gaussian filter
where normalizes the filter so that , preserving constant signals. Figure 7.6 shows the effect of Gaussian filtering in the spatial and Fourier domains.
Figure 7.7 shows the effect of increasing the filter width . Linear filtering works well on smooth signals, but removing more noise also blurs edges and other singularities.
Figure 7.7. Gaussian filtering with increasing width: three signals (top) and five images (bottom).
7.2.2 Optimal Filter Selection and Bias-Variance Tradeoff¶
The best filter depends on both the unknown signal and the noise level . Rather than optimize over all filters, we can tune a width parameter within a family such as the Gaussian filters in (7.1).
The triangle inequality separates the smoothing error from the remaining noise:
The filter width balances noise removal against excessive smoothing of singularities. A wider filter typically reduces while increasing .
Figure 7.8 shows the SNR as a function of filter width, evaluated using the known clean signal.
Figure 7.8. SNR as a function of filter width for a signal (left) and an image (right).
Figure 7.9 uses the width that maximizes SNR for the displayed observation. This choice uses the known clean signal to assess the best achievable result within the filter family.
Figure 7.9. Gaussian filtering at the SNR-optimal widths. From left to right: noisy signal, filtered signal, noisy image, and filtered image.
Some noise remains at the SNR-optimal width. Increasing the filter width introduces strong blur and reduces the SNR, although the result may look more pleasant.
7.2.3 Oracle Denoiser¶
To study the best possible choice of , first suppose that an oracle knows the clean signal .
Recall the model , where is Gaussian white noise: and the are independent and identically distributed. The clean signal is fixed. Choose an orthonormal basis , typically the Fourier basis, and restrict attention to linear estimators that are diagonal in it:
For the Fourier basis, this means where . Note that is a random vector.
The oracle may use to select the gains , but the estimator must still be linear in and have the displayed diagonal form. It cannot simply return .
Denoting (for the normalized discrete Fourier basis, ), the expected risk of this denoiser is
For a real orthonormal basis, the transformed noise is again independent Gaussian noise with variance . For a complex unitary Fourier basis applied to real noise, each coefficient still has second moment , which suffices here, although conjugate-frequency coefficients are dependent. By expanding the square and using , we obtain
Knowing , the oracle minimizes over . The minimization separates over the indices :
The solution thus satisfies
i.e.,
If , choosing gives the identity map. When decays, increasing suppresses the weaker coefficients more strongly, concentrating the gains on the dominant signal components.
7.2.4 Wiener Filter¶
We now replace access to the clean signal by a known probability model for it.
Suppose both the noise and the clean signal have known distributions. We seek a filter minimizing the error averaged over both sources of randomness, assuming and are independent.
We assume has the form (7.2).
The optimal thus minimizes
Independence of and allows the same risk calculation:
The optimal filter thus has coefficients
Relative to the oracle filter, the individual energy is replaced by the expected coefficient energy , the signal power spectrum in the Fourier basis.
This filter is known as the Wiener filter.
For the normalized discrete Fourier basis, . Define the averaged autocorrelation
Here indices are periodic and uses the unnormalized DFT of Chapter 2. This normalization is needed because the Fourier atoms used in the risk calculation have unit norm.
So far, has been restricted to the basis through the diagonal form (7.2). If is stationary, its law is unchanged by shifts: has the same law as . In that case, the Fourier Wiener filter (7.3) is optimal among all linear denoisers, even without imposing a diagonal form in advance.
An empirical estimate based on one noisy realization is . Its expectation is , and its variance can be large. Empirical Wiener methods therefore need noise subtraction, nonnegativity constraints, and usually additional smoothing or a signal model.
7.2.5 Denoising and Linear Approximation¶
Approximation theory, developed in Chapter 5, provides a different route to denoising bounds. It requires a class of clean signals rather than a probability distribution on that class.
Fix an orthonormal basis of . A simple estimator retains only the first coefficients of the noisy observation in :
This is a linear projection onto the space spanned by .
This denoising scheme is parameterized by the number of retained coefficients. Increasing retains more signal detail but also more noise.
For the discrete Fourier basis ordered by increasing frequency, this is ideal low-pass filtering by convolution with a discrete Dirichlet kernel.
This is the special case of (7.2) in which the gains are binary, . Although restrictive, this choice yields useful rates under the approximation assumptions of Theorem 7.1. It requires a regularity model for , but does not assume a random signal distribution .
The proof is finite-dimensional, but its bound has no explicit dependence on . Extending it to a continuum model requires interpreting white noise as a generalized random process.
Several observations help interpret this result:
The approximation assumption bounds the denoising error independently of the sample count , even though the total input noise energy grows with .
A larger approximation exponent gives faster decay of the error as tends to zero. The exponent approaches that of the parametric rate as increases.
The proof uses a finite sample count . White noise with constant positive variance in every coordinate has infinite energy in an infinite-dimensional Hilbert space, although other random vectors may have finite energy. The balancing choice applies when ; otherwise the estimator is the identity. A continuum interpretation as also requires increasing the sampling resolution.
Section 5.3.1 bounds approximation errors for continuous signal and image models. To transfer these bounds to sampled data, choose large enough that discretization error is smaller than denoising error.
The Sobolev model in Section 5.2.1 provides a useful example. For discrete signals with an increasing sample count , especially as decreases, a compatible weighted coefficient bound gives the same approximation argument as Proposition 5.3.
For normalized Fourier atoms ordered by increasing physical frequency, an appropriately scaled discrete weighted-energy bound approximates the periodic Sobolev model. The sample normalization, frequency weights, and boundary convention must match those of the continuous signal. Under this compatibility, low-frequency projection efficiently denoises smooth signals and images.
7.3 Nonlinear Denoising by Thresholding¶
7.3.1 Hard Thresholding¶
We consider a real orthonormal basis of , for instance a discrete wavelet basis.
The noisy coefficients satisfy
Orthogonal transforms preserve Gaussian white noise, so the coefficients remain independent centered Gaussians with variance . If the basis represents sparsely, most clean coefficients are small, leaving many noisy coefficients dominated by noise; see Figure 7.10. Donoho and Johnstone systematically developed thresholding estimators based on this observation [17].
Figure 7.10. Denoising using thresholding of wavelet coefficients.
Hard thresholding retains only coefficients whose magnitude exceeds the threshold :
where is defined in (5.4). Retaining the largest noisy coefficients gives the approximation of . Figure 7.10 illustrates how a suitable removes much of the noise while preserving sharp features.
7.3.2 Soft Thresholding¶
We recall that the hard thresholding operator is defined as
Hard thresholding makes a discontinuous keep-or-discard decision, which can create artifacts. Soft thresholding is a continuous alternative:
Set by continuity. Figure 7.11 compares the two nonlinear maps.
Figure 7.11. Hard and soft thresholding functions.
For and , these thresholding rules define two different estimators
Figure 7.12. SNR as a function of for hard and soft thresholding.
Preserving coarse-scale coefficients.
Soft thresholding biases retained coefficients by reducing their magnitudes. Shrinking coarse wavelet coefficients can introduce unwanted low-frequency artifacts. At a coarsest scale , it is therefore common to retain the approximation coefficients unchanged. In one dimension, this gives
Empirical choice of the threshold.
Figure 7.12 plots SNR against the threshold for a fixed natural image . In this experiment, hard thresholding performs best near , and soft thresholding near . The best soft-thresholded estimate has a slightly higher SNR.
Figure 7.13. Comparison of hard (left) and soft (right) thresholding.
These values describe the displayed experiments; the best threshold depends on the image, basis, noise level, and loss function.
7.3.3 Minimax Optimality of Thresholding¶
Estimating sparse coefficients.
To analyze the estimator and guide the choice of , we first assume that the coefficients
are sparse: most entries vanish, so the count
is small. As shown in (7.7), noisy coefficients
are perturbed by additive Gaussian white noise of variance . Figure 7.14 shows an example of such a noisy sparse signal.
Figure 7.14. Left: clean sparse coefficients ; right: noisy coefficients .
Universal threshold value.
When all nonzero clean coefficients are well separated from the noise level, choosing a threshold comparable to the largest pure-noise coefficient suppresses false detections:
This motivates a deterministic threshold based on the typical size of ; it is not an exact minimization formula for the realized denoising error. The random maximum depends on . For fixed , its mean is asymptotic to as . Its variance tends to zero with , so concentrates increasingly near its mean. Figure 7.15 illustrates these trends numerically.
Figure 7.15. Empirical mean (top) and standard deviation (bottom) of the largest absolute Gaussian noise coefficient.
Asymptotic optimality.
Donoho and Johnstone [17] showed that the universal threshold gives a risk bound for signals that admit accurate nonlinear approximations in . The risk is within a logarithmic factor of an ideal coefficient-selection risk.
The oracle inequality is a standard Gaussian thresholding bound; see [17]. The rate follows by balancing and , with integer rounding and truncation at as in Theorem 7.1. The added term accounts for residual noise even at the zero signal.
The universal threshold is conservative: it suppresses nearly all pure-noise coefficients. The smaller thresholds illustrated in Figure 7.13 often give better SNR in these experiments.
7.3.4 Translation Invariant Thresholding Estimators¶
Translation invariance.
Let be a denoising method, and be a translated signal or image for ( or ). The denoiser is translation invariant on the shift lattice if
where is a lattice of . A denser shift lattice imposes equivariance at finer spatial resolution. This corresponds to the fact that commutes with the translation operator.
Illustration. Unnumbered illustration: cycle spin principle
Translation invariance on a sufficiently fine set prevents the denoiser from favoring particular feature locations. Without it, the same feature may be reconstructed differently depending on where it appears, producing visible artifacts.
For denoising by thresholding
a sufficient condition is invariance of the basis under shifts in , up to unit-modulus factors:
where .
On the periodic domain, the Fourier basis is invariant under every shift of . The discrete Fourier basis is invariant under all integer shifts , where counts samples in one dimension and counts pixels in two dimensions.
Unfortunately, an orthogonal wavelet basis
is generally not invariant under arbitrary shifts in either the continuous or discrete setting. For instance, in 1-D,
Cycle spinning.
For a finite group of periodic shifts, average the shifted, denoised, and realigned outputs of to obtain a translation-invariant estimator:
One checks that
To obtain translation invariance at pixel precision for data with samples, one should use a set of translation vectors. For wavelets, this requires operations.
Figure 7.16 applies cycle spinning to hard thresholding in an orthogonal wavelet basis, using the following shifts: on an image, corresponding to shifts by pixels. This requires 16 forward and inverse wavelet-transform pairs. Averaging this subset of shifts does not in general guarantee exact pixel-shift equivariance. In this example, partial averaging reduces sensitivity to shifts, substantially improves the SNR, and suppresses oscillatory artifacts. These artifacts move as changes, so averaging reduces them.
In Figure 7.17, translation-invariant hard thresholding slightly outperforms translation-invariant soft thresholding, reversing their ranking in the orthogonal-basis experiment.
Figure 7.16. Soft thresholding in an orthogonal wavelet basis (left) and translation-invariant hard thresholding (right).
Figure 7.17. SNR as a function of for translation-invariant thresholding.
Translation invariant wavelet frame.
Cycle spinning also has a frame interpretation: replace the orthogonal basis by its translated copies,
Although is no longer an orthogonal basis, orthonormality of each translated copy gives the energy and reconstruction identities
Such redundant families are called tight frames.
One can then define a translation invariant thresholding denoiser
This estimator equals the cycle spinning estimator defined in (7.11).
Counted with multiplicity, the translated frame has elements. For a basis of signals with samples and a lattice of shifts, this gives up to vectors in . In a hierarchical construction such as a wavelet basis, however, different shifts can produce the same atom:
The number of distinct vectors in can therefore be much smaller than . For instance, for an orthogonal wavelet basis, one has
so many translated atoms coincide. For a signal of length , an undecimated transform has detail arrays of length , plus a coarse array. For an image (), it has detail arrays of entries, plus a coarse array. Appropriate multiplicity weights must be retained when replacing the full translated family by distinct atoms.
The fast undecimated wavelet transform, often called the “à trous” transform, computes these coefficients in operations. Each detail array is indexed directly by every pixel location:
where the wavelets and shifts are discrete and appropriately normalized. Each array is a band-pass filtered image, as illustrated in Figure 7.18.
Figure 7.18. Translation invariant wavelet coefficients.
Figure 7.19 illustrates hard thresholding of these translated coefficients.
Figure 7.19. Left: translation invariant wavelet coefficients, for , right: thresholded coefficients.
7.3.5 Other Shrinkage Rules¶
Other shrinkage rules offer different compromises between retaining large coefficients and suppressing noise. We describe two examples whose performance depends on the coefficient distribution of in the chosen basis.
Semi-soft thresholding.
Semi-soft thresholding interpolates between hard and soft thresholding through a parameter :
For , set . The limits and give hard and soft thresholding, respectively, away from the threshold boundary. Figure 7.20 shows an example of this nonlinearity.
Figure 7.20. Left: semi-soft thresholder, right: Stein thresholder.
Figure 7.21 shows an SNR improvement for a suitable choice of , although the visual difference from hard and soft thresholding is small.
Figure 7.21. Left: SNR as a function of and . Right: SNR versus , with optimized separately for each .
Stein thresholding.
Stein thresholding is defined using a quadratic attenuation of large coefficients
The value at is defined to be zero. This should be compared with the linear attenuation of soft thresholding
For large coefficients, Stein thresholding has the following shrinkage behavior:
as . Its deterministic shrinkage tends to zero, whereas soft thresholding retains a fixed shrinkage. This limiting property does not imply unbiased estimation at finite amplitudes.
Figure 7.22. SNR as a function of for Stein thresholding.
Stein and hard thresholding give similar results in the translation-invariant natural-image experiments shown here.
7.3.6 Block Thresholding¶
The nonlinear thresholding methods above are diagonal estimators, since they attenuate each coefficient separately
where
Define every attenuation factor to be zero at when . Block thresholding shares an attenuation factor across a group of coefficients, exploiting their statistical dependence. This is useful for natural images, where edges create clusters of large wavelet coefficients. Grouping also helps suppress isolated noise coefficients in otherwise smooth regions.
Partition the coefficients into disjoint blocks; for example, in a 2-D wavelet transform,
where each contains neighboring coefficients at a fixed scale and orientation. The block size is a parameter of the method. Figure 7.23 shows an example of such a block.
The mean squared coefficient magnitude in a block is
The block estimator
applies the same attenuation to every coefficient in a block,
for . Figure 7.23 shows the attenuated blocks and the reconstructed image.
Figure 7.23. Left: wavelet coefficients, center: block thresholded coefficients, right: denoised image.
Figure 7.24, left, compares the rules for . Stein block thresholding gives the highest SNR in this comparison. Figure 7.24, right, compares block sizes for Stein thresholding. The choice performs well in the displayed experiment.
Figure 7.24. SNR as a function of (left) and comparison of different block sizes (right).
Figure 7.25 compares the reconstructions. Stein block thresholding in an orthogonal wavelet basis approaches the quality of translation-invariant hard thresholding here, at lower computational cost. The block thresholding strategy can also be applied to wavelet coefficients in a translation invariant tight frame, which gives the best result among the methods compared in this experiment.
The same block attenuation can be implemented directly after the wavelet transform.
Figure 7.25. Left: translation invariant wavelet hard thresholding, center: block orthogonal Stein thresholding, right: block translation invariant Stein thresholding.
More advanced denoisers use richer statistical models to improve on the methods presented here; see, for instance, [27].
7.4 Signal-Dependent Noise¶
In many imaging systems, the variance of the noise affecting depends on the intensity itself. We now study Poisson counts and multiplicative noise. Both can be expressed as additive residuals, but their residual distributions depend on the signal.
7.4.1 Poisson Noise¶
Many imaging devices measure intensity by counting photons. Examples include digital cameras, confocal microscopy, PET and SPECT tomography.
Poisson model.
For an unknown mean intensity , a Poisson model describes the observed photon count:
The Poisson distribution has probability mass function
and its parameter varies across pixels with the underlying intensity. Figure 7.26 shows several Poisson distributions.
Figure 7.26. Poisson distributions for various .
One has
Thus denoising again estimates a mean from one observation, but the variance now varies with intensity. The absolute noise level increases as more photons reach the sensor. For positive intensity, the relative standard deviation instead decreases.
Figure 7.27 shows Poisson-corrupted versions of a clean image at different maximum mean intensities .
Figure 7.27. Poisson-corrupted flower images at increasing photon-count scales .
Variance stabilization.
Applying a thresholding estimator
directly to can perform poorly because a single threshold cannot adapt to spatially varying noise. A variance-stabilizing transform , applied pixelwise, maps the counts to . An additive Gaussian white-noise model can then be a useful approximation:
where are independent centered Gaussians with constant variance .
The model (7.14) is approximate, and its accuracy depends on the intensity range of . Two common variance-stabilizing transforms for Poisson noise are the Anscombe transform
and the Freeman–Tukey transform
Figure 7.28 compares the variance of after these transformations.
Figure 7.28. Exact Poisson variances of the Anscombe, Freeman–Tukey, and transforms as functions of the mean intensity.
A variance-stabilized denoiser is defined as
where is applied after projecting the denoised values onto its domain. The direct inverse generally introduces bias, especially at low counts; bias-corrected inverse transforms can improve the result.
At the moderate count levels in Figure 7.29, variance stabilization improves the denoising result.
Figure 7.29. Left: noisy image, center: denoising without variance stabilization, right: denoising after variance stabilization.
7.4.2 Multiplicative Noise¶
Multiplicative image formation.
A multiplicative noise model assumes that
where is a random multiplier, observed through one realization, with . Once again, the noise level depends on the pixel value
In SAR imaging, a common model forms by averaging independent observations, called looks:
where is additive Gaussian white noise and the multiplier has an exponential distribution:
Averaging independent looks divides the additive-noise variance by . When the remaining additive component is negligible, the average is approximately
where the multiplicative factor is distributed according to a Gamma distribution
Its mean is one and its variance is , so increasing reduces the relative noise level.
Figure 7.30. Multiplicative Gamma noise for increasing numbers of averaged looks, with standard deviation .
Figure 7.30 illustrates how the noise changes with the number of averaged looks.
A simple variance stabilization transform is
where
Here is the Gamma function and is its logarithmic derivative, the digamma function. The transformed observation satisfies
for strictly positive intensities, where is centered additive noise with variance ; is the derivative of the digamma function. Thus the transformed target is , not . Exponentiating an estimate of introduces a further statistical bias and may require correction.
Figure 7.31. Histogram of multiplicative noise before (left) and after (right) stabilization.
Figure 7.31 shows the effect of this variance stabilization on the distributions of and .
Figure 7.32 compares the two approaches at moderate noise levels , where variance stabilization improves the displayed result.
Figure 7.32. Left: noisy image, center: denoising after variance stabilization, right: denoising without variance stabilization.