GitHub Page About

GDCorner

A blog on various development topics

Texture Upscaling Pipeline

At the beginning of the year I wrote about my initial experiments with image super resolution. This week I focused on some of the improvements I discussed and implemented a pipeline.

I’ve written an image processing framework which I’m releasing under the MIT license. The goal is to allow rapid iteration of image processing until good results are achieved in the majority of cases reducing or eliminating the need for any manual editing of images.

Denoised before ESRGAN and then downsampled

The pipeline can be easily modified to add, remove and tweak processing passes which makes it incredibly quick to iterate on how the images get processed. The ability to move passes around in particular has been nice to experiment with. An example is testing whether performing alpha channel recovery before or after downsampling produces better results.

Current Status

To get this working nicely I’ve forked ESRGAN so I could add a few improvements. I’ve added CUDA detection so it can automatically fallback to CPU processing, as well as an improved command line interface, allowing execution of ESRGAN with an optional model specified.

The pipeline itself currently has the following features

Future Work

Now that this pipeline is in place I plan to iterate much more quickly over this process. Some of the improvements I’m planning are:

Results

Here are some examples of the results from this pipeline. The passes are exagerated here to show off denoising. This pipeline allows these passes to be tweaked rapidly until a nice result is acheived in the majority of circumstances.

Desk & Office

This scene shows off a lot of detailed textures which demonstrate the results of the different passes.

Original Original Unfiltered ESRGAN Unfiltered ESRGAN Denoised before ESRGAN Denoised before ESRGAN ESRGAN Downsampled ESRGAN Downsampled Denoised before ESRGAN and then downsampled Denoised before ESRGAN and then downsampled

Translucency

Rainbow Six doesn’t have too many translucent textures, mainly masked alpha textures, but here is an example of some glass to see the alpha channel recovery. You can also see the metal beams which are a masked transparent material.

Original Original Denoised before ESRGAN and then downsampled Denoised before ESRGAN and then downsampled

Where to download

This is all available on my github here: Texture Upscaling Pipeline