Object-Shadow-Generation-Dataset-DESOBA

Object Shadow Generation is to deal with the shadow inconsistency between the foreground object and the background in a composite image, that is, generating shadow for the foreground object according to background information, to make the composite image more realistic.

Our dataset DESOBA is a synthesized dataset for Object Shadow Generation. We build our dataset on the basis of Shadow-OBject Association dataset SOBA, which collects real-world images in complex scenes and provides annotated masks for object-shadow pairs. Based on SOBA dataset, we remove all the shadows to construct our DEshadowed Shadow-OBject Association(DESOBA) dataset, which can be used for shadow generation task and other shadow-related tasks as well. We illustrate the process of our DESOBA dataset construction based on SOBA dataset in the figure below.

Illustration of DESOBA dataset construction: The green arrows illustrate the process of acquiring paired data for training and evaluation. Given a ground-truth target image I_g, we manually remove all shadows to produce a deshadowed image I_d. Then, we randomly select a foreground object in I_g, and replace its shadow area with the counterpart in I_d to synthesize a composite image I_c without foreground shadow. I_c and I_g form a pair of input composite image and ground-truth target image. The red arrow illustrates our shadow generation task. Given I_c and its foreground mask M_fo, we aim to generate the target image I_g with foreground shadow.

Our DESOBA dataset contains 840 training images with totally 2,999 object-shadow pairs and 160 test images with totally 624 object-shadow pairs. The DESOBA dataset is provided in Baidu Cloud (access code: sipx), or Google Drive.

Prerequisites

• Python
• Pytorch
• PIL

Getting Started

Installation

• Clone this repo:

git clone https://github.com/bcmi/Object-Shadow-Generation-Dataset-DESOBA.git
cd Object-Shadow-Generation-Dataset-DESOBA

• Download the DESOBA dataset.

• We provide the code of obtaining training/testing tuples, each tuple contains foreground object mask, foreground shadow mask, background object mask, background shadow mask, shadow image, and synthetic composite image without foreground shadow mask. The dataloader is available in /data_processing/data/DesobaSyntheticImageGeneration_dataset.py, which can be used as dataloader in training phase or testing phase.

• We also provide the code of visualization of training/testing tuple, run:

python Vis_Desoba_Dataset.py

Vis_Desoba_Dataset.py is available in /data_processing/.

• We show some examples of training/testing tuples in below:

Bibtex

If you find this work is useful for your research, please cite our paper using the following BibTeX [arxiv]:

@article{hong2021shadow,
  title={Shadow Generation for Composite Image in Real-world Scenes},
  author={Hong, Yan and Niu, Li and Zhang, Jianfu and Zhang, Liqing},
  journal={arXiv preprint arXiv:2104.10338},
  year={2021}
}