Zhedong Zheng

Zhedong Zheng

Asst. Prof. @ UMacau
IEEE Senior Member

When Words Smile: Generating Diverse Emotional Facial Expressions from Text

Authors: Haidong Xu, Meishan Zhang, Hao Ju, Zhedong Zheng, Erik Cambria, Min Zhang, hao-feiHao Fei

Published in Conference on Empirical Methods in Natural Language Processing (EMNLP), 2025

Recommended citation: Haidong Xu, Meishan Zhang, Hao Ju, Zhedong Zheng, Erik Cambria, Min Zhang, Hao Fei, "When Words Smile: Generating Diverse Emotional Facial Expressions from Text." Conference on Empirical Methods in Natural Language Processing (EMNLP), 2025.
Download PDF: https://zdzheng.xyz/files/2025/Haidong_When.pdf

Abstract: Enabling digital humans to express rich emotions has significant applications in dialogue systems, gaming, and other interactive scenarios. While recent advances in talking head synthesis have achieved impressive results in lip synchronization, they tend to overlook the rich and dynamic nature of facial expressions. To fill this critical gap, we introduce an end-to-end text-to-expression model that explicitly focuses on emotional dynamics. Our model learns expressive facial variations in a continuous latent space and generates expressions that are diverse, fluid, and emotionally coherent. To support this task, we introduce EmoAva, a large-scale and high-quality dataset containing 15,000 text-3D expression pairs. Extensive experiments on both existing datasets and EmoAva demonstrate that our method significantly outperforms baselines across multiple evaluation metrics, marking a significant advancement in the field.

Keywords:

words smile generating smile generating diverse facial expressions text 
@inproceedings{xu2025words,
author = "Xu, Haidong and Zhang, Meishan and Ju, Hao and Zheng, Zhedong and Cambria, Erik and Zhang, Min and Fei, Hao",
title = "When Words Smile: Generating Diverse Emotional Facial Expressions from Text",
abstract = "Enabling digital humans to express rich emotions has significant applications in dialogue systems, gaming, and other interactive scenarios. While recent advances in talking head synthesis have achieved impressive results in lip synchronization, they tend to overlook the rich and dynamic nature of facial expressions. To fill this critical gap, we introduce an end-to-end text-to-expression model that explicitly focuses on emotional dynamics. Our model learns expressive facial variations in a continuous latent space and generates expressions that are diverse, fluid, and emotionally coherent. To support this task, we introduce EmoAva, a large-scale and high-quality dataset containing 15,000 text-3D expression pairs. Extensive experiments on both existing datasets and EmoAva demonstrate that our method significantly outperforms baselines across multiple evaluation metrics, marking a significant advancement in the field.",
booktitle = "Conference on Empirical Methods in Natural Language Processing (EMNLP)",
pages = "27016--27034",
url = "https://zdzheng.xyz/files/2025/Haidong\_When.pdf",
funding = "2025A1515012281, 202401035, MYRG-GRG2024-00077-FST-UMDF",
year = "2025" }