Aviation Operational Safety using Real Time Video Feed

Introduction

Aircraft safety is critical in ensuring smooth and hazard free operations in Airport logistics and ground operations. This project is aimed at providing safety of equipment, airport personnel and the aircraft itself when moving on the ground to dock at the airport terminal. The project uses Computer Vision techniques to process video feeds coming from cameras installed at docking tunnels and bridges, moving vehicles, stairs, luggage belts and other places to identify scenarios that may lead to situations compromising the safety of operations in any way and generate timely alerts on detection of such incidents.

Customer Story

Our customer is a solution provider in the AeroTech space for Ground Support Equipment, Gate Equipment and other related services.
They wanted to automate multiple operations regarding the movement of equipment and logistics to and from Aircrafts as they dock on the terminal. These operations are carried out manually by trained staff. However, since cameras are installed in abundance in such places, the customer decided to leverage those cameras and build an AI/ML based solution that could detect different scenarios critical to the safety, and generate timely alerts to avoid damage of equipment, aircraft and personnel.
The customer approached us via one of our partner companies which work closely with us for AI/ML based consultancy projects on Google Cloud Platform.

Our Solution

We have built several Computer Vision based AI/ML models based on hundreds of hours of camera feeds from the airport given to us by the customer. We implemented multiple use cases as follows:

  • Aircraft Door Safety Shoe Level
    • This scenario occurs when the aircraft is docking on the air bridge and the door is opening.
    • We Detect objects with bounded boxes: Aircraft Door, Airbridge Floor, Safety Shoe
    • We also measure the distances of different objects from the camera in each frame
    • Detect whether aircraft door has touched the airbridge shoe
    • Intent is to detect the optical proximity of the door and floor. The contact between door and floor should not occur. A timely alert needs to be generated
  • Collision avoidance of Airbridge with Aircraft Engine
    • This scenario occurs when the aircraft is docking on the air bridge and the door is opening.
    • We detect aircraft engine, and airbridge
    • Tracking of movement of Air bridge across each frame
    • Tracking of movement of Aircraft engine across each frame
    • Measure the distances of engine from the camera for each frame
    • Alert when proximity threshold crossed
  • Detection of objects and People on Stairs
    • This scenario occurs when stairs are moving for luggage and other purposes to carry objects and human personnel to and from the aircraft
    • We detect stairs and objects around the stairs
    • Track movement of objects around the stairs e.g. vehicles, people, boxes, carts
    • Track movement of stairs across each frame
    • Measure the distances of each object from the camera for each frame

      Key Technologies Used

    • Object Detection (Bounded Boxes around objects)
    • Predicting Segmentation masks (each pixel assigned to one class)
    • Depth estimation
      • Monocular (single camera based)
      • StereoVision based (requires Stereoscopic cameras or two cameras placement at a fixed distance from each other capturing the same scene)
    • Optical Flow maps
      • Predicting each pixel’s direction and magnitude of movement across a sequence of frames
    • Kalman Filters: Velocity estimation of moving objects across a sequence of frames

      Key Frameworks

    • Pytorch-Lightning
    • OpenCV
    • Jupyter Notebooks
    • FastAPI
    • Optuna
    • Kubeflow
    • Google Cloud Platform

Customer Benefits

This solution is important for the customer in terms of avoiding hazards and significantly reducing the damage to the aircraft body parts and the loss of ground equipment. Avoiding even one such incident may save several hundred thousand dollars and it has been observed by the customer that such incidents do occur a few times a year and result in heavy financial losses, delays in repairs and upgrades and other problems.