Critical Analysis of Validation Methods for Machine Learning Models Used in E-health Applications

Different types of data sets are established using various sensor-based networks and wearable devices for performing human activity recognition that can be used effectively in the e-health domain. Various machine learning models and relevant algorithms are used to perform detection with high accuracy. Depending on the characteristics of the data set, validation methods used to compute the accuracy may vary. The correct validation of machine learning algorithms is essential to correctly assess the performance of the models especially when the data analysed are related to the health domain. Activity recognition algorithms based on sliding windows with different overlapping ratios are popularly used for validation together with popular cross-validation methods such as k-fold, leave-one-out and leave-one-subject-out. In this study, validation methods commonly used for windowing-based activity recognition systems using wearable devices are analyzed. The advantages and disadvantages of each method are discussed taking into account various parameters. A case study, using the well-known MHEALTH data set, is presented with state-of-the art machine learning approaches. Experimental testing with a second window size using 10-fold cross-validation, a five-second window size using leave one out cross-validation, and a second window size using 5 fold cross-validation gave the highest accuracy, 96.71%, 95.65% and 95% respectively while the window overlap is 50%.