A gentle introduction to Super Resolution techniques using Deep Learning – Part II

Image enhancement, picture quality improvement and increasing resolution of images without a significant drop in quality have been one of the major application areas of Deep Learning based Artificial Intelligence techniques in recent years. These technologies are collectively called Super Resolution (SR). Several researchers have developed successful techniques that have pushed State-Of-The-Art and applied them to several different applications and use cases. Recently, these techniques have also been successfully extended to video processing. In last post, we introduced some basics and this is about some additional use cases.

Noise removal

This is an important application where we remove noise artifacts from images. Most common cases of such artifacts occur in satellite and/or drone images, and also in compression noise introduced by converting to standard formats like JPEG etc. A well-trained model can remove different noise levels from an image and make it usable for further processing.

Source: Nvidia

Artifacts removal

Sometimes, old images have artifacts like fold marks, image warping effect like distorted objects and faces, like looking into a bad mirror for example. Deep Learning can help remove such artifacts if a model is shown enough examples of warped or distorted images and pristine images.

Image Coloring

This is a fascinating application where black and white (aka gray scale) images are colored automatically and assigning appropriate color to each object. This could be used to color old photos that may be either black&white or whose colors have become dull with time. We all have old family photos of our parents, grand-parents and family photos that are not so rich in color as modern cameras give us, and it could give us a wonderful feeling to see them in full color painted by a well-trained AI model.

Source: Pyimagesearch

Image In-painting and Repair

Sometimes, images have missing spots such as parts of objects and persons that have been blurred, edited out or redacted. Similarly there are folds and lines in old photos and we would want to reconstruct them by properly repainting the missing or damaged part of the object. Examples could be parts of the face or hand of a person, a leg of a table or chair, a spot missing on the carpet in a room and so on. This application could repaint those parts automatically by figuring out the missing pieces and painting them such that the newly added spots merge and mix well with the other parts, giving a visually smooth perception i.e. the viewer is not easily able to detect the patch work from naked eye.

Souce: Nvidia

Source: wikipedia

Watermark removal

Many times, we have images with embedded water-marks or text artifacts like numbers or letters written in the background or on the face of a subject or anywhere else that we may want to remove. In such cases, a Deep Learning model can be trained to remove such artifacts and leave the rest of the image as it is.

Changing image environment

This is also a very interesting application where we change the environment of an image such as ambiance, weather, daytime to evening or night etc. For example, an outdoor picture taken in winter or fall season may be transformed to a spring season image with leaves and vegetation changing color from yellow to green. Such type of application maybe relevant in the media and film industry where such special effects are commonplace. Automating such effect generation in bulk on large set of images or videos could be highly desirable for such applications.

Source: digitaltrends

In the rest of this article, we present a high level, simplified description of the basic Image resolution enhancement application using SR technology.

In next post I will elaborate some additional use cases.

About Author

Farhan Zaidi has over 25 years of experience in Software Architecture, Data Engineering and software development in a variety of languages and technologies. He is skilled in designing data-driven, enterprise-grade software systems, as well as Embedded Systems and Systems Programming. His recently co-founded DreamAI with a vision to apply Deep Learning technology in a variety of domains, with a special focus on Computer Vision for Medical Diagnosis, Natural Language Processing and general Predictive Analytics for business problems.