3DeepVOG: An Open-Source Framework for Real-Time, Accurate 3D Gaze Tracking with Deep Learning

Zhao, Jingkang; Ahmadi, Seyed-Ahmad; Decker, Julian; Möhwald, Ken; Eulenburg, Peter zu; Zwergal, Andreas; Flanagin, Virginia L.; Wuehr, Max (2025): 3DeepVOG: An Open-Source Framework for Real-Time, Accurate 3D Gaze Tracking with Deep Learning. Digital Biomarkers, 10 (1). pp. 21-31. ISSN 2571-579X

Abstract

ntroduction: Eye movements are key biomarkers for diagnosing and monitoring neuro-otological, neuro-ophthalmological and neurodegenerative disorders. Video-oculography (VOG) systems enable detection of small, rapid eye movements and subtle oculomotor pathologies that may be missed during clinical exams. However, they rely on high-quality input, struggle with torsional movements, and are often limited by high costs in clinical and research settings.

Methods: To overcome these limitations, we developed 3DeepVOG, a deep learning-based framework for three-dimensional monocular gaze tracking (horizontal, vertical, and torsional rotation) that operates robustly across varied imaging conditions, including low-light and noisy environments. The method combines automated pupil and iris segmentation with geometrically interpretable estimation using a two-sphere anatomical eyeball model with corneal refraction correction. Torsion is tracked in real time using a novel mini-patch template matching approach. The system was trained on over 24,000 annotated samples obtained across multiple devices and clinical scenarios. Application was tested against a gold-standard VOG system in healthy controls.

Results: 3DeepVOG operates in real time (>300 fps) and achieves gaze errors of ~0.1° in all three dimensions. Oculomotor measures – saccadic peak velocity, smooth pursuit gain, and optokinetic nystagmus slow-phase velocity – show good-to-excellent agreement with a clinical gold-standard system. As proof of concept, we present a case of acute unilateral vestibular failure where 3DeepVOG reliably captures 3D nystagmus.

Conclusions: 3DeepVOG enables accurate, quantitative eye movement tracking across three dimensions under diverse conditions. As an open-source framework, it provides an accessible and scalable tool for advancing research and clinical assessment in neurological oculomotor disorders.

• Dokumententyp: Artikel (Klinikum der LMU)
• Organisationseinheit (Fakultäten): 07 Medizin
• DFG-Fachsystematik der Wissenschaftsbereiche: Lebenswissenschaften