ESRGAN

ESRGAN (Enhanced Super-Resolution Generative Adversarial Networks) is an advanced generative adversarial network designed for single image super-resolution. First introduced in September 2018 by researchers including Xintao Wang (who later joined Tencent ARC Lab), the model was designed to address the over-smoothing and unpleasant artifact issues commonly associated with previous upscaling methods like SRGAN. ESRGAN won first place in the PIRM2018-SR Challenge, establishing itself as a foundational architecture in the field of perceptual super-resolution.

Model Architecture

The model's generator is built on the Residual-in-Residual Dense Block (RRDB) as its core unit. Unlike previous designs, ESRGAN completely removes Batch Normalization (BN) layers, which were found to introduce unpleasant artifacts and limit generalization. Instead, RRDB combines multi-level residual networks and dense connections, facilitating deeper architectures that improve performance.

The adversarial training incorporates a Relativistic Discriminator (RaGAN). Rather than predicting whether an image is absolutely real or fake, the relativistic discriminator estimates the probability that a real image is more realistic than a generated one. This formulation helps the generator synthesize sharper edges and more realistic textures.

Furthermore, ESRGAN improves the perceptual loss by evaluating features extracted from deep convolutional networks (specifically VGG19) before activation, rather than after activation. This modification provides stronger supervision for brightness consistency and texture recovery, resulting in more natural boundaries and details.

Real-ESRGAN and Practical Extensions

In 2021, Tencent ARC Lab released Real-ESRGAN, an updated and more practical extension of the original model. Real-ESRGAN targets real-world blind super-resolution by simulating complex, real-world image degradation through a high-order degradation modeling process (addressing blur, noise, and JPEG compression artifacts). It also introduces a U-Net discriminator with spectral normalization to improve training stability and handle a wider range of distortions.