Rémi Flamary

Professional website

Total variation in images

In this demo, we illustrate the effect of Total Variation regularization on images and how it can be used for image denoising.

Short introduction to Total Variation (TV)

Definition

Total variation is a measure of the complexity of an image with respect to its spatial variation. It has several variations in the image processing litterature. In this demo we will discuss its extension to color images. In color images, one can consider each pixel 3aac93d3673a2bc8ef6d622ebb2ae13c as a 3D vector.

The total variation of a color image b0c381a8151a7725c9d715f37f8aa2d2 can be expressed as

e147ccf9ea0299e33468e361b5a8458e
where 4bd01f4a6739d95ec7743e82072870d7 defines the pixel neighborhood (usually the horizontal and vertical adjacent pixels) and 368d971fd67db0ddff6d2bd1e46b960a is the b15a3315a69ce382beea50cbaa779f80 norm to the power of 7694f4a66316e53c8cdd9d9954bd611d.

We will focus in the following to 3 special cases

  • Classical 8524eb1789cf2093cfccc4c297138c7f TV computed independently on each color component.
    ac35eeff5b1cc8d86e715d3a3fdddcee
  • f2d02eaf32cb7a351989198531c0d12a TV computes the euclidean norm of the vector.
    1b60a81a59693e5f8dfd97a6f14fa6cf
  • Squared f2d02eaf32cb7a351989198531c0d12a TV computes the squared euclidean norm of the vector.
    802e930fc305b1d4e16d2e81b8b26780

Those variants of TV measure different spatial variation. 8524eb1789cf2093cfccc4c297138c7f wil sum the absolute values of the finite differences for each color component independently. f2d02eaf32cb7a351989198531c0d12a and squared f2d02eaf32cb7a351989198531c0d12a will sum respectively the norm and squared norm of the vector of difference treating all the color components simultaneously.

TV denoising

TV can be also be used for image denoising. This can be done by computing the following optimization problem

fc1582af8f0e8addc073019b30f39258
wher c6a6eb61fd9c6c913da73b3642ca147d is a positive regularization term and deada7802bc6f260e7c5bd2a7b5b92a7 is the noisy image. This optimization find the image b0c381a8151a7725c9d715f37f8aa2d2 thet both fit the noisy image but also has a limited TV. An example of the result of this optimization is given in the following image.

graph

In this image, we can clearly see that the squared f2d02eaf32cb7a351989198531c0d12a TV leads to a smooth image at the cost of loosing sharp edges and details in the image. 8524eb1789cf2093cfccc4c297138c7f and f2d02eaf32cb7a351989198531c0d12a TV both keep sharp edges. in the case of 8524eb1789cf2093cfccc4c297138c7f TV, all color component are treated independently leading to color artifacts around the edges.

Regularization demo

In order to see the effect of TV regularization on a real-life image, we will use the widly known Lena picture. We will apply the regularization on both the original image (up) and an image that contains noise (down).

Note that due to the size of the images, the update might take a while when changing the parameters, depending on your internet provider.

denoising result

Reg. type

Reg. parameter lambda

Performance

l1 norm
l2 norm
l2 norm squared

Current value: 0.07

Signal to noise ratio

SNR = 18.99 dB

Both the TV variant and the regularization parameter c6a6eb61fd9c6c913da73b3642ca147d can be changed in this demo. The Signal to noise ratio of the reconstructed image is also provided.

Note that an early stopping is used which might return an approximate solution for some values of the regularization parameter.

References

Total Variation have been around in the image processsing comunity for a while now since the pioneer works of [1]. A nice and simple paper about the extension to color image cen be found in [2]. Finally the algorithm used for solving the optimization probleme is a variant of ADMM as discussed in [3].

[1] Nonlinear total variation based noise removal algorithms, Rudin, L. I.; Osher, S.; Fatemi, E. (1992). Physica D 60: 259–268.

[2] A generalized vector-valued total variation algorithm., Rodriguez Paul, and Brendt Wohlberg. Image Processing (ICIP), 2009 16th IEEE International Conference on. IEEE, 2009.

[3] An ADMM algorithm for a class of total variation regularized estimation problems, Wahlberg, B., Boyd, S., Annergren, M., & Wang, Y. (2012).