Pattern recognition (psychology)

Operating system

Image (mathematics)

Pixel

Mathematics

Computer science

Gene

Quantum mechanics

Encoder

Power (physics)

Set (abstract data type)

Generator (circuit theory)

Computer vision

Biochemistry

Domain (mathematical analysis)

Artificial intelligence

Code (set theory)

Encoding (memory)

Mathematical analysis

Chemistry

Programming language

Image translation

Messenger RNA

Physics

Translation (biology)

We present a generic image-to-image translation framework, pixel2style2pixel (pSp). Our pSp framework is based on a novel encoder network that directly generates a series of style vectors which are fed into a pretrained StyleGAN generator, forming the extended $\mathcal{W} + $ latent space. We first show that our encoder can directly embed real images into $\mathcal{W} + $, with no additional optimization. Next, we propose utilizing our encoder to directly solve image-to-image translation tasks, defining them as encoding problems from some input domain into the latent domain. By deviating from the standard "invert first, edit later" methodology used with previous StyleGAN encoders, our approach can handle a variety of tasks even when the input image is not represented in the StyleGAN domain. We show that solving translation tasks through StyleGAN significantly simplifies the training process, as no adversary is required, has better support for solving tasks without pixel-to-pixel correspondence, and inherentl...

Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation