Image editing

Affine transformation

Pure mathematics

Statistics

Computer security

Image processing

Generative adversarial network

Economics

Set (abstract data type)

Computer science

Semantics (computer science)

Code (set theory)

Pattern recognition (psychology)

Mathematics

Image (mathematics)

Operations management

Artificial intelligence

Matching (statistics)

Information retrieval

Programming language

Metric (unit)

Image manipulation

Image texture

Computer vision

Key (lock)

The goal of our paper is to semantically edit parts of an image matching a given text that describes desired attributes (e.g., texture, colour, and background), while preserving other contents that are irrelevant to the text. To achieve this, we propose a novel generative adversarial network (ManiGAN), which contains two key components: text-image affine combination module (ACM) and detail correction module (DCM). The ACM selects image regions relevant to the given text and then correlates the regions with corresponding semantic words for effective manipulation. Meanwhile, it encodes original image features to help reconstruct text-irrelevant contents. The DCM rectifies mismatched attributes and completes missing contents of the synthetic image. Finally, we suggest a new metric for evaluating image manipulation results, in terms of both the generation of new attributes and the reconstruction of text-irrelevant contents. Extensive experiments on the CUB and COCO datasets demonstrate the superior performance of the proposed method.

ManiGAN: Text-Guided Image Manipulation