Real-World Automatic Makeup via Identity Preservation Makeup Net

Authors: Zhikun Huang, Zhedong Zheng, Chenggang Yan, Hongtao Xie, Yaoqi Sun, Jianzhong Wang, Jiyong Zhang

Published in International Joint Conference on Artificial Intelligence (IJCAI), 2020

Recommended citation: Zhikun Huang, Zhedong Zheng, Chenggang Yan, Hongtao Xie, Yaoqi Sun, Jianzhong Wang, Jiyong Zhang, "Real-World Automatic Makeup via Identity Preservation Makeup Net." IJCAI, 2020. DOI: 10.5555/3491440.3491531
Download PDF: https://zdzheng.xyz/files/2020/Huang_ijcai20.pdf
中文解读: https://zhuanlan.zhihu.com/p/150116945

Code is available at: https://github.com/huangzhikun1995/IPM-Net

Abstract: This paper focuses on the real-world automatic makeup problem. Given one non-makeup target image and one reference image, the automatic makeup is to generate one face image, which maintains the original identity with the makeup style in the reference image. In the real-world scenario, face makeup task demands a robust system against the environmental variants. The two main challenges in real-world face makeup could be summarized as follow: first, the background in real-world images is complicated. The previous methods are prone to change the style of background as well; second, the foreground faces are also easy to be affected. For instance, the heavy makeup may lose the discriminative information of the original identity. To address these two challenges, we introduce a new makeup model, called Identity Preservation Makeup Net (IPM-Net), which preserves not only the background but the critical patterns of the original identity. Specifically, we disentangle the face images to two different information codes, i.e., identity content code and makeup style code. When inference, we only need to change the makeup style code to generate various makeup images of the target person. In the experiment, we show the proposed method achieves not only better accuracy in both realism (FID) and diversity (LPIPS) in the test set, but also works well on the real-world images collected from the Internet.

@inproceedings{huangreal,
author = "Huang, Zhikun and Zheng, Zhedong and Yan, Chenggang and Xie, Hongtao and Sun, Yaoqi and Wang, Jianzhong and Zhang, Jiyong",
title = "Real-World Automatic Makeup via Identity Preservation Makeup Net",
abstract = "This paper focuses on the real-world automatic makeup problem. Given one non-makeup target image and one reference image, the automatic makeup is to generate one face image, which maintains the original identity with the makeup style in the reference image. In the real-world scenario, face makeup task demands a robust system against the environmental variants. The two main challenges in real-world face makeup could be summarized as follow: first, the background in real-world images is complicated. The previous methods are prone to change the style of background as well; second, the foreground faces are also easy to be affected. For instance, the heavy makeup may lose the discriminative information of the original identity. To address these two challenges, we introduce a new makeup model, called Identity Preservation Makeup Net (IPM-Net), which preserves not only the background but the critical patterns of the original identity. Specifically, we disentangle the face images to two different information codes, i.e., identity content code and makeup style code. When inference, we only need to change the makeup style code to generate various makeup images of the target person. In the experiment, we show the proposed method achieves not only better accuracy in both realism (FID) and diversity (LPIPS) in the test set, but also works well on the real-world images collected from the Internet.",
booktitle = "IJCAI",
code = "https://github.com/huangzhikun1995/IPM-Net",
url = "https://zdzheng.xyz/files/2020/Huang\_ijcai20.pdf",
blog = "https://zhuanlan.zhihu.com/p/150116945",
doi = "10.5555/3491440.3491531",
year = "2020" }