TITLE:
Analysis of Emotions Using Multimodal Data: A Case Study
AUTHORS:
Toshiya Akiyama, Kyoko Osaka, Hirokazu Ito, Ryuichi Tanioka, Allan Paulo Blaquera, Leah Anne Christine Bollos, Tetsuya Tanioka
KEYWORDS:
Humanoid Robots, Multimodal Data, Emotion Analysis
JOURNAL NAME:
Journal of Biosciences and Medicines,
Vol.11 No.12,
December
8,
2023
ABSTRACT: In this case study, we hypothesized that sympathetic nerve activity would be higher during conversation with PALRO robot, and that conversation would result in an increase in cerebral blood flow near the Broca’s area. The facial expressions of a human subject were recorded, and cerebral blood flow and heart rate variability were measured during interactions with the humanoid robot. These multimodal data were time-synchronized to quantitatively verify the change from the resting baseline by testing facial expression analysis, cerebral blood flow, and heart rate variability. In conclusion, this subject indicated that sympathetic nervous activity was dominant, suggesting that the subject may have enjoyed and been excited while talking to the robot (normalized High Frequency