Multi-Modal Data Fusion Using Deep Neural Network for Condition Monitoring of High Voltage Insulator

A novel Fusion Convolutional Network (FCN) is proposed in this research for potential real-time monitoring of insulators using unmanned aerial vehicle (UAV) edge devices. Precise airborne imaging of outdoor objects, such as high voltage insulators, suffers from varied object resolution, cluttered backgrounds, unclear or contaminated surfaces, and illumination conditions. Accurate information about the insulator surface condition is essential and is of a high priority since insulator breakdown is a leading cause of electrical failure. A multi-modal information fusion (MMIF) system is developed during this research to analyze and classify possible contaminations present on the electrical insulators. A novel system, referred to as FCN, consists of a Convolutional Neural Network (CNN) and a binary Multilayer Neural Network (MNN) sub-classifier. While constructing the MMIF dataset for training and testing the novel FCN, the image classification output of the CNN is combined with the leakage current values (LCV) obtained as the classification output of MNN. Each sample of the MMIF dataset is, therefore, represented as a series of fusions. Later, sub-classifiers, of the FCN, are trained to identify the contamination types in the fusion series by implementing a voting system of sub-classifiers which is trained to identify a given class. As a result of the implementation of the proposed FCN, the classification accuracy increased by 8.4%, i.e., from 92% to 99.76%. To compare and benchmark the performance of proposed FCN, conventional classification algorithms are also implemented on the fusion of features that are extracted employing the wavelet transform and PCA methods. State-of-the-art CNN architectures are also discussed on account of their time consumption and memory usage. The conceptualization of a potential hardware implementation of the proposed FCN, on emerging edge devices, is also provided for completeness of the discussion. Pertinent outcomes of this research can be further extended to other potential applications of airborne imaging.