U-turn: Crafting Adversarial Queries with Opposite-direction Features

Authors: Zhedong Zheng, Liang Zheng, Yi Yang, Fei Wu

Published in International Journal of Computer Vision (IJCV), 2022

Recommended citation: Zhedong Zheng, Liang Zheng, Yi Yang, Fei Wu, "U-turn: Crafting Adversarial Queries with Opposite-direction Features." International Journal of Computer Vision (IJCV), 2022. DOI: 10.1007/s11263-022-01737-y
Download PDF: https://zdzheng.xyz/files/2022/IJCV_Retrieval_Robustness_CameraReady.pdf
中文解读: https://zhuanlan.zhihu.com/p/593319732

Code is available at: https://github.com/layumi/U_turn

Abstract: This paper aims to craft adversarial queries for image retrieval, which uses image features for similarity measurement. Many commonly used methods are developed in the context of image classification. However, these methods, which attack prediction probabilities, only exert an indirect influence on the image features and are thus found less effective when being applied to the retrieval problem. In designing an attack method specifically for image retrieval, we introduce opposite-direction feature attack (ODFA), a white-box attack approach that directly attacks query image features to generate adversarial queries. As the name implies, the main idea underpinning ODFA is to impel the original image feature to the opposite direction, similar to a U-turn. This simple idea is experimentally evaluated on five retrieval datasets. We show that the adversarial queries generated by ODFA cause true matches no longer to be seen at the top ranks, and the attack success rate is consistently higher than classifier attack methods. In addition, our method of creating adversarial queries can be extended for multi-scale query inputs and is generalizable to other retrieval models without foreknowing their weights, i.e., the black-box setting.

@article{zheng2022query,
author = "Zheng, Zhedong and Zheng, Liang and Yang, Yi and Wu, Fei",
title = "U-turn: Crafting Adversarial Queries with Opposite-direction Features",
abstract = "This paper aims to craft adversarial queries for image retrieval, which uses image features for similarity measurement. Many commonly used methods are developed in the context of image classification. However, these methods, which attack prediction probabilities, only exert an indirect influence on the image features and are thus found less effective when being applied to the retrieval problem. In designing an attack method specifically for image retrieval, we introduce opposite-direction feature attack (ODFA), a white-box attack approach that directly attacks query image features to generate adversarial queries. As the name implies, the main idea underpinning ODFA is to impel the original image feature to the opposite direction, similar to a U-turn. This simple idea is experimentally evaluated on five retrieval datasets. We show that the adversarial queries generated by ODFA cause true matches no longer to be seen at the top ranks, and the attack success rate is consistently higher than classifier attack methods. In addition, our method of creating adversarial queries can be extended for multi-scale query inputs and is generalizable to other retrieval models without foreknowing their weights, i.e., the black-box setting.",
journal = "International Journal of Computer Vision (IJCV)",
url = "https://zdzheng.xyz/files/2022/IJCV\_Retrieval\_Robustness\_CameraReady.pdf",
code = "https://github.com/layumi/U\_turn",
blog = "https://zhuanlan.zhihu.com/p/593319732",
doi = "10.1007/s11263-022-01737-y",
year = "2022" }