Noise2Sim: New Solution to Image Denoising


In Audio conversations, noise is considered as the background sound that is not required but is present. It makes the overall audio a bit unclear. Similarly, noise in images is defined as the unwanted blurring that causes a lack of clarity. Therefore, denoising means removing this unwanted noise from the images.

Image editing. Image credit: alexx-ego via Pixabay, free licence

Applications of Image Denoising

Given its wide application, such as image restoration, visual tracking, image classification etc., much research has been done on image denoising in the last decade. Some widely used techniques to denoise images have their limitations.

Noise2Sim technique is presented as a solution to limitations of other widely used techniques to denoise images in the research paper presented by Chuang Niu and Ge Wang, that forms the basis of this text.

Objective of Research

The objectives of the research, as explained by Chuang Niu and Ge Wang are presented below:

  • We propose an NLM-inspired self-supervised learning method for image denoising that learns to map between central pixels in similar image patches and only requires single noisy images for training;
  • We develop an two-step procedure to manage the computational burden associated with globally searching of similar image patches and prepare training data efficiently for Noise2Sim denoising;
  • We design a refined training strategy to use Noise2Sim results for further Noise2Sim denoising, giving improved image quality;
  • We perform extensive experiments and statistical analysis, and demonstrate that our Noise2Sim method outperform the state-of-the-art Noise2Void method on common benchmark datasets;
  • We make our Noise2Sim software package publicly available

Common denoising Techniques

Let us try & understand underlying principles of some common denoising techniques:

  1. Local denoising methods: This method assumes that a pixel can be denoised using the mean value of its surrounding pixels.
  2. Non-local mean methods: This technique takes a weighted mean of all pixels in the image to denoise a pixel. The weight of each pixel is based on the distance of that pixel from the pixel we are denoising. Despite their superior performance, the non-local mean methods demand longer searching time, which is a practical issue in many applications such as real-time video image processing.
  3. Deep Denoising Methods
    1. Fully Supervised: Convolutional Neural Networks (OR CNN) is trained based on many paired noise-clean images in advance. A very deep CNN architecture makes it very costly to prepare or impractical to collect.
    2. Weakly Supervised: Denoising in Weakly supervised deep denoising model is a 3 step process:
      1. Self-learning methods are leveraged to train a denoising & noising model.
      2. These models are applied to noisy & clean images to generate paired datasets.
      3. Generated datasets are used to train the final denoising model.
    3. Unsupervised: Least restrictive & most desirable in practice since they use a single noisy image to denoise. Noise2Void/Noise2Self uses a single noisy image to predict masked pixels from its surrounding. The value of a pixel in the Noise2Void technique is predicted based on the value of its neighbor.

Noise2Void does not use self-similarity in an image to denoise. This limitation of Noise2Void brings us to Noise2Sim that uses a single noisy image for training and also leverages the similarity in the image to yield much effective denoising.

Noise2Sim Technique: Chuang Niu and Ge Wang define Noise2Sim as

an NLM-inspired self-learning method for image denoising. Specifically, Noise2Sim leverages self-similarities of image patches and learns to map between the center pixels of similar patches for self-consistent image denoising.


The research text discussed commonly used techniques & discussed their limitations.

  • Noise2clean technique required many paired noise-clean samples for network training.
  • Noise2Noise: Easier to collect noise2noise image pair, but could be impractical in some cases
  • Noise2Void: Given the limitation for Noise2Clean & Noise2Noise techniques, Noise2Void was developed as an effort to be able to denoise an image from a single image.

Further, Noise2Sim is presented as a useful alternative to the above techniques. The paper also presents evidence that Noise2Sim denoising is superior to Noise2Void; and can be equivalent to Noise2Noise & Noise2Clean techniques under mild practical conditions.

The research also proposes that the Noise2Sim model can be scaled to adjust accuracy & performance based on the task required that makes it even more desirable.

Source: Chuang Niu, Ge Wang “Noise2Sim — Similarity-based Self-Learning for Image Denoising

More to track …
Bulletin Observer Science and Technology

[Advertisement Plan]

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

As you found this post useful...

Follow us on social media!

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

Pin It on Pinterest

Translate »