Human Organ Real-time Localization using HTC Vive Tracking System and Machine Learning Models

Abstract

Virtual reality and machine learning technologies have become focal points for research and development for medical studies in recent years. However, previous studies do not typically use virtual reality and machine learning in tandem. In this study, we propose a framework utilizing both virtual reality and machine learning to predict the localization of human organs in real-time. The HTC Vive Pro virtual reality system, while used originally for entertainment, is a viable, low-cost option for studies requiring precise measurements. Data collected by the virtual reality system is used as inputs for machine learning models for predictions of human organ localization in real-time. Further, data enhancement methods, such as data normalization and extreme event split, are leveraged to improve machine learning model performance. According to our experimental results, the gradient boosting regressor model performs accurately for almost every direction for either of the two tracker configurations, i.e., linear and triangular. The extreme event split can also improve machine learning performance, especially with rotational data. Overall, this framework is promising to be used as the localization basis for other surgical procedures, as well as other human organs.