Continuous monitoring of neonatal care activities remains challenging because routine caregiving procedures are rarely documented in electronic medical records, despite their potential impact on neonatal physiological stability. This study introduces the NEO TINY System (NTS), an integrated AI-enabled monitoring platform capable of synchronizing bedside video with physiological data collected from NICU medical devices.
The proposed framework combines an Inception-v3 Convolutional Neural Network (CNN) with a Long Short-Term Memory (LSTM) network to automatically recognize neonatal caregiving activities from video sequences while simultaneously evaluating associated changes in heart rate and oxygen saturation. The study analyzed video and physiological recordings collected from 10 neonates across two Level III NICUs, capturing 167 patting events, 64 diaper changes, and 108 tube feeding procedures.
The deep learning system achieved 95% training accuracy and 85% validation accuracy while automatically generating structured documentation of neonatal care activities that are typically absent from electronic medical records. The study also demonstrated statistically significant physiological responses associated with routine caregiving activities, highlighting the potential of AI-assisted monitoring to improve neonatal documentation, quantify stress associated with caregiving, and support future intelligent clinical decision systems in NICUs.
Premature infants admitted to Neonatal Intensive Care Units (NICUs) experience numerous routine caregiving procedures every day, many of which contribute to cumulative physiological stress that may adversely affect neurodevelopment. Existing approaches for measuring neonatal stress rely on manual documentation using the Neonatal Infant Stressor Scale (NISS), making continuous monitoring impractical in busy clinical environments.
This study developed a multimodal deep learning system that analyzes bedside video recordings together with synchronized physiological signals to automatically detect neonatal care manipulation activities, quantify their duration and frequency, estimate immediate physiological responses, and calculate stress exposure. The framework was trained and evaluated using 289 hours of synchronized video and physiological recordings collected during 330 clinical sessions from 27 neonates across two NICUs. Using advanced computer vision models and temporal activity localization techniques, the system achieved activity quantification statistically equivalent to expert human annotation within clinically acceptable error margins while accurately identifying activities associated with significant physiological changes.
The proposed framework represents an important step toward continuous, objective, AI-powered neonatal stress monitoring capable of supporting clinicians with standardized documentation, automated Neonatal Infant Stressor Scale (NISS) estimation, and real-time decision support in intensive care environments.
