Multimodal Sentinel-1/2 Data Fusion with a Dual-Path ViT-Kolmogorov-Arnold Network Pipeline for AI-driven Flood Event Classification

Timely identification of flood events is a challenging problem, as they are devastating natural disasters and cause severe damage to lives and livelihoods. Satellite-based flood identification is widely adopted because it offers extended spatio-temporal coverage compared to traditional ground and Unmanned Aerial Vehicle (UAV)-based methods. However, satellite-based flood identification is heavily hindered by adverse weather and cloud conditions. Utilizing Synthetic Aperture Radar (SAR) data helps address sensing challenges under adverse weather conditions; however, SAR provides limited spectral information. Therefore, fusing SAR with optical satellite data becomes an ideal choice, although multimodal learning with SAR–optical fusion remains a challenging task. This research proposes FloodSense, a dual-path ViT–KAN pipeline for AI-driven flood event classification via Sentinel-1/2 multimodal fusion, introducing a dual-path learning strategy that jointly captures global and local characteristics and enhances nonlinear mapping. The FloodSense pipeline comprises a Vision Transformer Flood Feature Extractor (ViT-FloodX) and a Spatial Spectral Feature Extractor (3D2D-Extract), jointly capturing cross-domain global contextual semantics and fine-grained spatial-spectral patterns. Moreover, a Kolmogorov–Arnold Network Feature Refinement (Refine-KAN) module enhances nonlinear feature representation by combining KANs with Multi-Layer Perceptron (MLP) layers, improving multimodal feature learning. The empirical evaluations of FloodSense are conducted on the popular SEN12-FLOOD multimodal flood event classification dataset and demonstrate competitive performance against the state-of-the-art (SOTA) by achieving an accuracy, precision, recall, and F1 score of 94.87%, 94.74%, 96.48%, and 95.61%, respectively. FloodSense, with its multimodal fusion capability, achieves efficient large-scale flood event classification under adverse weather conditions, requiring only 19.5 GFLOPs and an inference time of 1.34s per image.

Text 127707.pdf - Published Version Available under License Creative Commons Attribution. Download (10MB) Official URL or Download Paper: https://ieeexplore.ieee.org/document/11613203/

Actions (login required)

View Item