Representation-based ECG signal prediction for neural networks pre-training

Abstract

A limited amount of training data is a well-known challenge in the application of deep learning methods. This issue is particularly relevant in biomedical signal processing, such as the analysis of electrocardiograms, due to the labor-intensive nature of data preparation, which requires the involvement of qualified specialists. Self-supervised learning methods, originally developed in such domains as natural language processing and computer vision, have emerged as a potential approach to addressing this challenge and are increasingly being explored in biomedical signal processing. However, direct adaptation of self-supervised learning techniques from other domains does not fully account for ECG-specific characteristics, such as quasi-periodicity, localized morphological features, and susceptibility to noise. This highlights the relevance of developing ECG-specific self-supervised learning methods. This study presents a novel self-supervised learning approach for pretraining neural networks on unlabeled ECG data. The proposed method is based on predicting the short consecutive signal segment using a preceding one and a learned representation vector. The representation extraction and prediction models are trained jointly on the MIMIC-ECG-IV dataset using backpropagation to minimize the mean squared error between the predicted and original signal segments. As an example of a downstream task, a linear binary classifier was trained on the PTB-XL dataset to diagnose pathological conditions using Lead I. The number of training examples for each diagnosis was limited to thirty-four samples. Firstly, the representation model was pre-trained on the unlabeled MIMIC-ECG-IV dataset, and then linear classifiers were trained on the learned representations for each selected diagnosis in PTB-XL. A comparison was also conducted with a randomly initialized representation model trained jointly with the classifier in a fully supervised manner. The proposed method was evaluated against adaptations of Contrastive Learning, Contrastive Predictive Coding, and Masked Autoencoders method. To ensure a controlled experimental setup, implementations of all considered methods were developed using a unified codebase and shared architectural components. Experimental results demonstrated a significant advantage of all self-supervised learning approaches over joint training of feature extraction and classification models. The proposed SSL method outperformed other tested approaches, particularly for diagnoses with subtle short-term morphological features, such as atrial fibrillation and flutter. These findings suggest the potential for further research in developing ECG-specific self-supervised learning methods as a promising approach to improving neural network performance in scenarios with limited labeled data.