A gentle introduction to Super Resolution techniques using Deep Learning – Part I

Introduction and overview

Image enhancement, picture quality improvement and increasing resolution of images without a significant drop in quality have been one of the major application areas of Deep Learning based Artificial Intelligence techniques in recent years. These technologies are collectively called Super Resolution (SR). Several researchers have developed successful techniques that have pushed State-Of-The-Art and applied them to several different applications and use cases. Recently, these techniques have also been successfully extended to video processing.

This has opened major opportunities for automated and intelligent processing of media content spanning a wide variety of domains and industries ranging from photography, law enforcement and surveillance, to image and video processing for news media and entertainment industries.

This is the first of a series of posts on this very exciting topic. It will be followed by more articles that cover several aspects of this technology to give a quick, simplified and yet comprehensive view to a wide range of audience.

In this post, we present a gentle and simplified introduction to what SR is, how it works at a very basic level, and how this basic form of SR is adapted to different applications and use cases.

This article assumes a basic understanding of Machine Learning in general and Neural Networks in particular. Specifically, the reader should be familiar with the concepts and terms like model training, objective or loss functions, pixels and (ideally) Convolutional Neural Networks (CNN). However, if you are not familiar with CNNs, don’t worry too much, as we shall provide links to simple and easy to follow tutorials as we go along.

Some popular applications of SR Technology

Before we dive deeper into how SR works, let’s look at some of the mainstream applications of SR in real world today. The following are just some of the applications of this technology that is changing the world of photography and image processing in general.

Image Resolution Enhancement

This application comes in many forms and use cases. In the most simple case, an image is enlarged to a higher resolution while maintaining its quality and avoiding a phenomenon called pixelation.

However, image enhancement has many other forms as well. For example, one could enhance the sharpness, lighting, hue or contrast and apply additional filters that affect the overall visual perception of an image.

Similarly, there are special features added to images for portraits and group photos by modern expensive cameras and photographers, such as blurring the background while enhancing the face and figure outline of the subjects. Furthermore, high end cameras have several different modes such as night time photography mode, where special treatment and post-processing is performed on outdoor snapshots taken at night time etc.

All of this can be automated to a large extent by clever training of Deep Learning models and applying their prediction at either snapshot time or afterwards as a post-processing step, thus applying the effects to thousands of images in bulk.

About Author

Farhan Zaidi has over 25 years of experience in Software Architecture, Data Engineering and software development in a variety of languages and technologies. He is skilled in designing data-driven, enterprise-grade software systems, as well as Embedded Systems and Systems Programming. His recently co-founded DreamAI with a vision to apply Deep Learning technology in a variety of domains, with a special focus on Computer Vision for Medical Diagnosis, Natural Language Processing and general Predictive Analytics for business problems.