The construction of a natural interactive human-computer interaction has become an integral component of intelligent system development, constituting a core subject within the field of human-computer interaction. Grounded in the utilization of human perception theories, this study proposes a robust method for speech emotion recognition and a model for inferring user emotional state changes, thereby achieving a human-computer interaction experience characterized by both listening and articulating, and conveying sentiments effectively. Simultaneously, focusing on the audio-visual modalities within human-computer interaction, this research explores methods for constructing interactive feedback in auditory and visual modalities within real human-computer dialog scenarios. This aims to synchronize textual, verbal, and visual expressions in human-computer interaction, enhancing its naturalness, improving user experience, and augmenting satisfaction and pleasure during interaction. The research contributions are as follows:

Introducing a method that robustly identifies user emotional states in real human-computer speech dialogue scenarios through the utilization of local and global attention mechanisms. This method generates robust and distinguishable representations of speech emotion, thereby enhancing the robustness and accuracy of speech emotion recognition systems in authentic dialogue scenarios.