Any region can be perceived equally and effectively on rotation pretext task using full rotation and weighted-region mixture.

Journal: Neural networks : the official journal of the International Neural Network Society
PMID:

Abstract

In recent years, self-supervised learning has emerged as a powerful approach to learning visual representations without requiring extensive manual annotation. One popular technique involves using rotation transformations of images, which provide a clear visual signal for learning semantic representation. However, in this work, we revisit the pretext task of predicting image rotation in self-supervised learning and discover that it tends to marginalise the perception of features located near the centre of an image. To address this limitation, we propose a new self-supervised learning method, namely FullRot, which spotlights underrated regions by resizing the randomly selected and cropped regions of images. Moreover, FullRot increases the complexity of the rotation pretext task by applying the degree-free rotation to the region cropped into a circle. To encourage models to learn from different general parts of an image, we introduce a new data mixture technique called WRMix, which merges two random intra-image patches. By combining these innovative crop and rotation methods with the data mixture scheme, our approach, FullRot + WRMix, surpasses the state-of-the-art self-supervision methods in classification, segmentation, and object detection tasks on ten benchmark datasets with an improvement of up to +13.98% accuracy on STL-10, +8.56% accuracy on CIFAR-10, +10.20% accuracy on Sports-100, +15.86% accuracy on Mammals-45, +15.15% accuracy on PAD-UFES-20, +32.44% mIoU on VOC 2012, +7.62% mIoU on ISIC 2018, +9.70% mIoU on FloodArea, +25.16% AP on VOC 2007, and +58.69% AP on UTDAC 2020. The code is available at https://github.com/anthonyweidai/FullRot_WRMix.

Authors

  • Wei Dai
    Department of Intensive Care Unit, The First Affiliated Hospital of Jiangxi Medical College, Shangrao, Jiangxi, China.
  • Tianyi Wu
    Department of Mechanical Engineering, City University of Hong Kong, Hong Kong, China.
  • Rui Liu
    School of Education, China West Normal University, Nanchong, Sichuan, China.
  • Min Wang
    National and Local Joint Engineering Research Center of Ecological Treatment Technology for Urban Water Pollution, Wenzhou University, Wenzhou 325035, China.
  • Jianqin Yin
    School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China. Electronic address: jqyin@bupt.edu.cn.
  • Jun Liu
    Department of Radiology, Second Xiangya Hospital, Changsha, Hunan, China.