Feed

Wearable and phone sensor data hold great potential for monitoring depression, yet effective integration of these diverse data sources remains challenging. Transforming these complex data into a learned embedding space provides a lower-dimensional representation that preserves essential temporal patterns while capturing the intricate inter-modal relationships. In this study, we evaluate how different fusion strategies for generating multimodal embeddings impact the effectiveness of clustering in identifying depression symptoms. We used a longitudinal dataset integrating physiological and social data such as electrocardiogram, accelerometer, respiration rate, and mobility/Bluetooth interaction data, collected over 35 days. An embedding-based approach using long short-term memory (LSTM) autoencoders was employed to learn latent space representations, followed by the application of K-Means and Gaussian Mixture Models (GMM) clustering algorithms to identify patterns within this learned space. Weekly Beck Depression Inventory-II (BDI-II) scores, held-out during training, served as the ground truth for performance evaluation. A custom metric, the BDI-Variance Ratio Clustering Score (BDI-VRCS), was developed to quantitatively assess clustering efficacy across different embedding spaces. Early fusion implementation with LSTM and GMM achieved the highest BDI-VRCS of 0.3309, outperforming both mid and late fusion strategies (0.112 and 0.132, respectively). This highlights the value of early integration of multimodal data, with social features playing a key role in capturing depressive symptoms.Clinical relevance- This study highlights the potential of integrating physiological and social data using multimodal fusion strategies to enhance depression monitoring and support the development of holistic, data-driven tools for early detection and personalized mental health interventions.