Economics

Set (abstract data type)

Computer science

Image retrieval

Automatic image annotation

Closed captioning

Code (set theory)

Natural language processing

Task (project management)

Management

Image (mathematics)

Artificial intelligence

Information retrieval

Programming language

Coco

Annotation

IEEE Transactions on Multimedia

This paper contributes to cross-lingual image annotation and retrieval in terms of data and baseline methods. We propose COCO-CN, a novel dataset enriching MS-COCO with manually written Chinese sentences and tags. For effective annotation acquisition, we develop a recommendation-assisted collective annotation system, automatically providing an annotator with several tags and sentences deemed to be relevant with respect to the pictorial content. Having 20 342 images annotated with 27 218 Chinese sentences and 70 993 tags, COCO-CN is currently the largest Chinese-English dataset that provides a unified and challenging platform for cross-lingual image tagging, captioning, and retrieval. We develop conceptually simple yet effective methods per task for learning from cross-lingual resources. Extensive experiments on the three tasks justify the viability of the proposed dataset and methods. Data and code are publicly available at https://github.com/li-xirong/coco-cn.

COCO-CN for Cross-Lingual Image Tagging, Captioning, and Retrieval