Swin transformer based siamese network for thermal and optical image registration

Abstract

The process of multi-modal image registration is fundamental in remote sensing and visual navigation applications. However, existing image registration methods that are designed for single modality images do not provide satisfactory results when applied to multi-modal image registration. In this research, our objective is to achieve highly accurate alignment of both infrared and optical (visible range) images. To accomplish this goal, we explore the effectiveness of the Swin Transformer encoder and cosine loss in enhancing the keypoint-based image registration process. Simulation results show the improvement achieved in multi-modal registration by using a transformer based Siamese network.