By Charalambos (Charis) Poullis in GAN — Aug 25, 2022

Unsupervised Structure-Consistent Image-to-Image Translation

Our method learns structure-consistent image-to-image translation without requiring a semantic mask. We learn to disentangle structure and texture for applications such as style transfer and image editing tasks. The first(left) image shows the first input image, and the other images show the generated images in which the structure is retained from the first input image and the texture from the second, third, and fourth input images, respectively, shown in the inset images. Note that the tree’s structure is preserved, and its texture -in this case, the foliage’s colour and density- changes according to the texture of the second input image in the inset. Our model was not trained on any season transfer dataset.

The preprint of our paper Unsupervised Structure-Consistent Image-to-Image Translation is available on arxiv.org. The work is co-authored by Shima Shahfar and Charalambos Poullis.

TL;DR: We present a structure-consistent image-to-image translation technique based on the swapping autoencoder. We introduce an auxiliary module which forces the generator to learn to reconstruct an image with an all-zero texture code, encouraging better disentanglement between the structure and texture information and significantly reducing training time. Our method works on complex domains such as satellite images where state-of-the-art are known to fail.