Weakly Supervised Semantic Segmentation Based on Web Image Co-segmentation

Abstract

Training a Fully Convolutional Network (FCN) for semantic segmentation requires a large number of masks with pixel level labelling, which involves a large amount of human labour and time for annotation. In contrast, web images and their image-level labels are much easier and cheaper to obtain. In this work, we propose a novel method for weakly supervised semantic segmentation with only image-level labels. The method utilizes the internet to retrieve a large number of images and uses a large scale co-segmentation framework to generate masks for the retrieved images. We first retrieve images from search engines, e.g. Flickr and Google, using semantic class names as queries, e.g. class names in the dataset PASCAL VOC 2012. We then use high quality masks produced by co-segmentation on the retrieved images as well as the target dataset images with image level labels to train segmentation networks. We obtain an IoU score of 56.9 on test set of PASCAL VOC 2012, which reaches the state-of-the-art performance.

Paper

Paper link: Arxiv

Bibtex:

@inproceedings{Shen:2018:wss,
          author    = {Tong Shen and
                       Guosheng Lin and
                       Lingqiao Liu and
                       Chunhua Shen and
                       Ian Reid},
          title     = {Weakly Supervised Semantic Segmentation Based on Web Image Co-segmentation},
          booktitle = {BMVC},
          year      = {2017}
        }

Code

The code can be found here.

Data

There are 20 folders in the dataset corresponding to the 20 object classes in PASCAL VOC 2012. Each folder has images containing the same semantic class, which are retrived from Google and Flikr. Dataset can be downloaded Here.

Acknowledgements