Project SkyWatch (a.k.a. Wescam at Home)

Summary

Professional aviation surveillance relies on a specific piece of hardware: the EO/IR (Electro-Optical/Infra-Red) gimbal. These are the gyro-stabilized turrets you see on the nose of police helicopters or military drones, capable of keeping a rock-solid lock on a target regardless of how the aircraft maneuvers. VIDEO I wanted to replicate this capability to help track aircraft from the ground—building a tool that allows a consumer camera to lock onto and follow a target with similar stability, but without the defense-contractor budget. The Hardware Constraint The core of this build is a generic PTZ (Pan-Tilt-Zoom) camera, the kind typically used for streaming church services or campus lectures. The camera in question: an AVKANS LV20N, a knockoff of a 20x zoom PTZOptics unit While cost-effective, these cameras present a major engineering challenge for tracking any object, let alone aircraft. Their motors are designed for slow, dampened pans across a stage, not for tracking a jet moving at 300 knots. The mechanical and electronics latency is significant; if you simply tell the camera to “follow that plane,” by the time the motors react, the target has often moved out of the frame. To make this hardware viable, the heavy lifting has to move from mechanics to mathematics. The Software Stack I built a custom control loop to bridge the gap between the camera’s sluggish motors and the dynamic speed of the targets. The stack fuses three main concepts to help the system maintain a visual lock: Visual Processing (OpenCV & CSRT)Once the camera is pointed at a target, the backend initializes a CSRT (Discriminative Correlation Filter with Channel and Spatial Reliability) tracker. Unlike simple contrast-based detection which can be easily fooled by clouds or changing lighting, CSRT tracks the specific visual features and texture of the aircraft. It calculates the error—how far off-center the target is in pixels—frame by frame to drive the control loop. Prediction (Kalman Filter)Ra...