Two graduate student research teams from the Departments of Artificial Intelligence and Software Convergence have proposed novel multimodal convergence learning methods and lightweight multimodal methodologies for emotion recognition, respectively

Two teams from the Artificial Intelligence and Software Convergence Departments at Kyung Hee Graduate School separately entered the ^2nd ETRI Human Understanding AI Paper Competition, organized by the Electronics and Telecommunications Research Institute (ETRI). They were honored with the Minister of Science and ICT Award. This competition ran from February to June, with a total of sixty-two teams participating.

Students Juhyuk Han, Hyejeong Jo, and Taeseong Kim from the Software Convergence Department (mentored by Professor Won Hee Lee) penned a paper on LMR-CL: A Multimodal Convergence Learning Approach based on Contrastive Loss for Emotion Recognition. Meanwhile, Students Hyeon Ki Jo, Yuri Seo, and Seol Roh from the Computer Engineering Department (mentored by Professor Eui-Nam Huh) crafted a paper titled Multi-Still: A Lightweight Multimodal Cross-Attention Knowledge Distillation Method for Real-Time Emotion Recognition. The students jointly expressed their gratitude and surprise, stating "We ventured into this challenge to hone our skills and are thrilled to have earned such recognition. Our heartfelt thanks go to our professors and team members for their unwavering guidance and hard work."

Professor Won Hee Lee's research team proposed a technique to predict a variety of emotions without bias, even in a world rife with data imbalances. They introduced a novel method to redress known issues, aiming to curate an AI model capable of harmoniously balancing a spectrum of emotions. Student Hyejeong Jo elucidated, "We explored how to employ multimodal functionalities for emotion recognition and its subsequent industrial applications. Our primary goal was crafting a versatile model that can freely harness diverse dataset."

It is noteworthy that all these students hail from an undergraduate research background, underscoring the profound value of early academic involvement. During their undergraduate research stints, they gleaned invaluable insights and perspectives. Student Taeseong Kim recalled, "Being exposed to a multitude of research methods and topics that I had never considered before was enlightening." Student Juhyuk Han added, "Given that the Software Convergence Department is a recent addition to the institution, there were few graduate students and not much information about graduate life. My time as an undergraduate researcher equipped me with a wealth of experiences that were instrumental in navigating graduate school.”

Professor Huh's team employed text and voice data from a Korean language-centric multimodal emotion dataset to craft a strategy that facilitates knowledge transfer from a teacher model to a student model. They devised an innovative method to optimize multimodal structures for real-time emotion recognition, leveraging knowledge distillation technology, pivoting on human-computer interactions. Their endeavors culminated in a lightweight model, retaining the integrity of intricate network functionalities.

In choosing their research themes, they meticulously scrutinized emerging research paradigms. Student Yuri Seo explained, "The latest trend revolved around creating AI services with tangible real-life applications. We ensured our research resonated with this prevailing theme. Student Hyeon Ki Jo mentioned, "We dedicated over six hours daily, deliberating on our research theme. The extensive research and exploration across multifarious topics enriched our paper writing.”