Outlier-Mined Anomaly Attention for Real-Time Multimodal Fusion of Wearable PPG/IMU and Facial Video in Cuffless Continuous Blood Pressure Estimation
Keywords:
Multimodal Fusion, Cuffless Blood Pressure Estimation, Photoplethysmography (PPG), Out-Of-Distribution Detection, Transformer-Based AttentionAbstract
Continuous cuffless blood pressure estimation under free-living conditions remains challenging due to frequent distribution shifts caused by motion artifacts, sensor detachments, and lighting fluctuations. We propose a real-time multimodal fusion system that integrates wearable photoplethysmography (PPG), inertial measurement unit (IMU) data, and facial video for robust blood pressure monitoring. The core innovation is an Outlier-Mined Anomaly Attention (OMAA) mechanism that embeds anomaly detection theory directly into the cross-modal attention computation. During training, we construct hard-negative out-of-distribution examples by applying controlled perturbations to each modality—synthetic motion artifacts for PPG, simulated sensor detachment for IMU, and lighting changes for facial video. An energy-based anomaly score is computed for each modality token, and tokens with the highest energy values are mined as hard outliers. These outliers calibrate a learnable gating function that attenuates attention weights involving anomalous tokens, effectively suppressing unreliable modality contributions during inference without requiring external detectors or full model retraining. A lightweight reconstruction head further refines the gating by providing residual anomaly cues from signal reconstruction errors. The fused representation is then passed to a regression head that outputs systolic and diastolic blood pressure estimates. The system operates on overlapping five-second windows from all modalities, with temporal convolutional networks extracting token embeddings of dimension 128. The OMAA mechanism replaces standard scaled-dot-product attention in a transformer encoder, enabling seamless integration into existing fusion architectures. Experimental results demonstrate that the proposed method maintains accurate blood pressure estimation even under severe motion and environmental disturbances. This work represents a significant step toward practical, continuous blood pressure monitoring in real-world settings by making multimodal fusion inherently robust to distribution shifts.
References
D Buxi, JM Redouté, and MR Yuce. Blood pressure estimation using pulse transit time from bioimpedance and continuous wave radar. IEEE Transactions on Biomedical Engineering, 2016.
YH Li, LN Harfiya, K Purwandari, and YD Lin. Real-time cuffless continuous blood pressure estimation using deep learning model. Sensors, 2020.
H Han, MJ Kim, and J Kim. Development of real-time motion artifact reduction algorithm for a wearable photoplethysmography. In 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2007.
D Novita, FW Adikusuma, N Rohadi, BM Wibawa, et al. Development of contactless human vital signs monitoring device with remote-photoplethysmography using adaptive region-of-interest and hybrid processing .... Intelligence-Based Medicine, 2024.
A Vaswani, N Shazeer, N Parmar, et al. Attention is all you need. In Advances in Neural Information Processing Systems, 2017.
YHH Tsai, S Bai, PP Liang, JZ Kolter, et al. Multimodal transformer for unaligned multimodal language sequences. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.
DP Kingma and M Welling. Auto-encoding variational bayes. Technical report, arXiv preprint arXiv:1312.6114, 2013.
Y LeCun, S Chopra, R Hadsell, M Ranzato, et al. A tutorial on energy-based learning. Predicting Structured Data, 2006.
Y Suh, B Han, W Kim, and KM Lee. Stochastic class-based hard example mining for deep metric learning. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
J Heo, HB Lee, S Kim, J Lee, KJ Kim, et al. Uncertainty-aware attention for reliable interpretation and prediction. In Advances in Neural Information Processing Systems, 2018.
J Chen, Y Li, X Wu, Y Liang, and S Jha. Robust out-of-distribution detection for neural networks. Technical report, arXiv preprint arXiv:2003.09711, 2020.
S Sagawa, PW Koh, TB Hashimoto, and P Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. Technical report, arXiv preprint arXiv:1911.08731, 2019.
H Feng, Q Shen, R Song, L Shi, and H Xu. Atfa: Adversarial time-frequency attention network for sensor-based multimodal human activity recognition. Expert Systems with Applications, 2024.
J Chen, Y Li, X Wu, Y Liang, and S Jha. Atom: Robustifying out-of-distribution detection using outlier mining. Lecture Notes in Computer Science, 2021.
R Kamal, A Ali, I Cleland, C Nugent, et al. Trustworthy stress detection in consumer wearables: Poisoning-resilient learning with feature-gated dualnet and generative restoration. IEEE Transactions on Consumer Electronics, 2026.
AEW Johnson, TJ Pollard, L Shen, LH Lehman, M Feng, et al. Mimic-iii, a freely accessible critical care database. Scientific data, 2016.
S Sakib, Z Hasan, and N Roy. A state-of-the-art survey of remote photoplethysmography for contactless health parameters sensing. WIREs Data Mining and Knowledge Discovery, 2025.
P Schmidt, A Reiss, R Duerichen, C Marberger, et al. Introducing wesad, a multimodal dataset for wearable stress and affect detection. In Proceedings of the 20th ACM International Conference on Multimodal Interaction, 2018.
Association for the Advancement of Medical et al. American national standard. electronic or automated sphygmomanometers. Ansi/Aami Sp10, 1987.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 The Author(s)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
