Source code of our work: "Benchmarking Deep Models for Salient Object Detection"

Related tags

Deep LearningSALOD
Overview

SALOD

Source code of our work: "Benchmarking Deep Models for Salient Object Detection".
In this works, we propose a new benchmark for SALient Object Detection (SALOD) methods.

We re-implement 14 methods using same settings, including input size, data loader and evaluation metrics (thanks to Metrics). Hyperparameters of optimizer are different because of various network structures and objective functions. We try our best to tune the optimizer for these models to achieve the best performance one-by-one. Some other networks are debugging now, it is welcome for your contributions on these networks to obtain better performance.

Properties

  1. A unify interface for new models. To develop a new network, you only need to 1) set configs; 2) define network; 3) define loss function. See methods/template.
  2. We build a new dataset by collecting several prevalent datasets in SOD task.
  3. Easy to adopt different backbones (Available backbones: ResNet-50, VGG-16, MobileNet-v2, EfficientNet-B0, GhostNet, Res2Net)
  4. Testing all networks on your own device. By input the name of network, you can test all available methods in our benchmark. Comparisons includes FPS, GFLOPs, model size and multiple effectiveness metrics.
  5. We implement a loss factory that you can change the loss functions using command line parameters.

Available Methods:

Methods Publish. Input Weight Optim. LR Epoch Paper Src Code
DHSNet CVPR2016 320^2 95M Adam 2e-5 30 openaccess Pytorch
NLDF CVPR2017 320^2 161M Adam 1e-5 30 openaccess Pytorch/TF
Amulet ICCV2017 320^2 312M Adam 1e-5 30 openaccess Pytorch
SRM ICCV2017 320^2 240M Adam 5e-5 30 openaccess Pytorch
PicaNet CVPR2018 320^2 464M SGD 1e-2 30 openaccess Pytorch
DSS TPAMI2019 320^2 525M Adam 2e-5 30 IEEE/ArXiv Pytorch
BASNet CVPR2019 320^2 374M Adam 1e-5 30 openaccess Pytorch
CPD CVPR2019 320^2 188M Adam 1e-5 30 openaccess Pytorch
PoolNet CVPR2019 320^2 267M Adam 5e-5 30 openaccess Pytorch
EGNet ICCV2019 320^2 437M Adam 5e-5 30 openaccess Pytorch
SCRN ICCV2019 320^2 100M SGD 1e-2 30 openaccess Pytorch
GCPA AAAI2020 320^2 263M SGD 1e-2 30 aaai.org Pytorch
ITSD CVPR2020 320^2 101M SGD 5e-3 30 openaccess Pytorch
MINet CVPR2020 320^2 635M SGD 1e-3 30 openaccess Pytorch
Tuning ----- ----- ------ ------ ----- ----- ----- -----
*PAGE CVPR2019 320^2 ------ ------ ----- ----- openaccess TF
*PFA CVPR2019 320^2 ------ ------ ----- ----- openaccess Pytorch
*F3Net AAAI2020 320^2 ------ ------ ----- ----- aaai.org Pytorch
*PFPN AAAI2020 320^2 ------ ------ ----- ----- aaai.org Pytorch
*LDF CVPR2020 320^2 ------ ------ ----- ----- openaccess Pytorch

Usage

# model_name: lower-cased method name. E.g. poolnet, egnet, gcpa, dhsnet or minet.
python3 train.py model_name --gpus=0

python3 test.py model_name --gpus=0 --weight=path_to_weight 

python3 test_fps.py model_name --gpus=0

# To evaluate generated maps:
python3 eval.py --pre_path=path_to_maps

Results

We report benchmark results here.
More results please refer to Reproduction, Few-shot and Generalization.

Notice: please contact us if you get better results.

VGG16-based:

Methods #Param. GFLOPs Tr. Time FPS max-F ave-F Fbw MAE SM EM Weight
DHSNet 15.4 52.5 7.5 69.8 .884 .815 .812 .049 .880 .893
Amulet 33.2 1362 12.5 35.1 .855 .790 .772 .061 .854 .876
NLDF 24.6 136 9.7 46.3 .886 .824 .828 .045 .881 .898
SRM 37.9 73.1 7.9 63.1 .857 .779 .769 .060 .859 .874
PicaNet 26.3 74.2 40.5* 8.8 .889 .819 .823 .046 .884 .899
DSS 62.2 99.4 11.3 30.3 .891 .827 .826 .046 .888 .899
BASNet 80.5 114.3 16.9 32.6 .906 .853 .869 .036 .899 .915
CPD 29.2 85.9 10.5 36.3 .886 .815 .792 .052 .885 .888
PoolNet 52.5 236.2 26.4 23.1 .902 .850 .852 .039 .898 .913
EGNet 101 178.8 19.2 16.3 .909 .853 .859 .037 .904 .914
SCRN 16.3 47.2 9.3 24.8 .896 .820 .822 .046 .891 .894
GCPA 42.8 197.1 17.5 29.3 .903 .836 .845 .041 .898 .907
ITSD 16.9 76.3 15.2* 30.6 .905 .820 .834 .045 .901 .896
MINet 47.8 162 21.8 23.4 .900 .839 .852 .039 .895 .909

ResNet50-based:

Methods #Param. GFLOPs Tr. Time FPS max-F ave-F Fbw MAE SM EM Weight
DHSNet 24.2 13.8 3.9 49.2 .909 .830 .848 .039 .905 .905
Amulet 79.8 1093.8 6.3 35.1 .895 .822 .835 .042 .894 .900
NLDF 41.1 115.1 9.2 30.5 .903 .837 .855 .038 .898 .910
SRM 61.2 20.2 5.5 34.3 .882 .803 .812 .047 .885 .891
PicaNet 106.1 36.9 18.5* 14.8 .904 .823 .843 .041 .902 .902
DSS 134.3 35.3 6.6 27.3 .894 .821 .826 .045 .893 .898
BASNet 95.5 47.2 12.2 32.8 .917 .861 .884 .032 .909 .921
CPD 47.9 14.7 7.7 22.7 .906 .842 .836 .040 .904 .908
PoolNet 68.3 66.9 10.2 33.9 .912 .843 .861 .036 .907 .912
EGNet 111.7 222.8 25.7 10.2 .917 .851 .867 .036 .912 .914
SCRN 25.2 12.5 5.5 19.3 .910 .838 .845 .040 .906 .905
GCPA 67.1 54.3 6.8 37.8 .916 .841 .866 .035 .912 .912
ITSD 25.7 19.6 5.7 29.4 .913 .825 .842 .042 .907 .899
MINet 162.4 87 11.7 23.5 .913 .851 .871 .034 .906 .917

Create New Model

To create a new model, you can copy the template folder and modify it as you want.

cp -r ./methods/template ./methods/new_name

More details please refer to python files in template floder.

Loss Factory

We supply a Loss Factory for an easier way to tune the loss functions. You can set --loss and --lw parameters to use it.

Here are some examples:

loss_dict = {'b': BCE, 's': SSIM, 'i': IOU, 'd': DICE, 'e': Edge, 'c': CTLoss}

python train.py ... --loss=bd
# loss = 1 * bce_loss + 1 * dice_loss

python train.py ... --loss=bs --lw=0.3,0.7
# loss = 0.3 * bce_loss + 0.7 * ssim_loss

python train.py ... --loss=bsid --lw=0.3,0.1,0.5,0.2
# loss = 0.3 * bce_loss + 0.1 * ssim_loss + 0.5 * iou_loss + 0.2 * dice_loss
Real-time object detection on Android using the YOLO network with TensorFlow

TensorFlow YOLO object detection on Android Source project android-yolo is the first implementation of YOLO for TensorFlow on an Android device. It is

Nataniel Ruiz 624 Jan 03, 2023
Doubly Robust Off-Policy Evaluation for Ranking Policies under the Cascade Behavior Model

Doubly Robust Off-Policy Evaluation for Ranking Policies under the Cascade Behavior Model About This repository contains the code to replicate the syn

Haruka Kiyohara 12 Dec 07, 2022
New approach to benchmark VQA models

VQA Benchmarking This repository contains the web application & the python interface to evaluate VQA models. Documentation Please see the documentatio

4 Jul 25, 2022
Image Captioning on google cloud platform based on iot

Image-Captioning-on-google-cloud-platform-based-on-iot - Image Captioning on google cloud platform based on iot

Shweta_kumawat 1 Jan 20, 2022
CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation

CPT This repository contains code and checkpoints for CPT. CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Gener

fastNLP 341 Dec 29, 2022
TransCD: Scene Change Detection via Transformer-based Architecture

TransCD: Scene Change Detection via Transformer-based Architecture

wangzhixue 29 Dec 11, 2022
Unofficial Implementation of MLP-Mixer, Image Classification Model

MLP-Mixer Unoffical Implementation of MLP-Mixer, easy to use with terminal. Train and test easly. https://arxiv.org/abs/2105.01601 MLP-Mixer is an arc

Oğuzhan Ercan 6 Dec 05, 2022
PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World [ACL 2021]

piglet PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World [ACL 2021] This repo contains code and data for PIGLeT. If you like

Rowan Zellers 51 Oct 08, 2022
Fashion Landmark Estimation with HRNet

HRNet for Fashion Landmark Estimation (Modified from deep-high-resolution-net.pytorch) Introduction This code applies the HRNet (Deep High-Resolution

SVIP Lab 91 Dec 26, 2022
Definition of a business problem according to Wilson Lower Bound Score and Time Based Average Rating

Wilson Lower Bound Score, Time Based Rating Average In this study I tried to calculate the product rating and sorting reviews more accurately. I have

3 Sep 30, 2021
A particular navigation route using satellite feed and can help in toll operations & traffic managemen

How about adding some info that can quanitfy the stress on a particular navigation route using satellite feed and can help in toll operations & traffic management The current analysis is on the satel

Ashish Pandey 1 Feb 14, 2022
Code for the AI lab course 2021/2022 of the University of Verona

AI-Lab Code for the AI lab course 2021/2022 of the University of Verona Set-Up the environment for the curse Download Anaconda for your System. Instal

Davide Corsi 5 Oct 19, 2022
AutoPentest-DRL: Automated Penetration Testing Using Deep Reinforcement Learning

AutoPentest-DRL: Automated Penetration Testing Using Deep Reinforcement Learning AutoPentest-DRL is an automated penetration testing framework based o

Cyber Range Organization and Design Chair 217 Jan 01, 2023
The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation

PointNav-VO The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation Project Page | Paper Table of Contents Setup

Xiaoming Zhao 41 Dec 15, 2022
This project provides a stock market environment using OpenGym with Deep Q-learning and Policy Gradient.

Stock Trading Market OpenAI Gym Environment with Deep Reinforcement Learning using Keras Overview This project provides a general environment for stoc

Kim, Ki Hyun 769 Dec 25, 2022
Multilingual Image Captioning

Multilingual Image Captioning Authors: Bhavitvya Malik, Gunjan Chhablani Demo Link: https://huggingface.co/spaces/flax-community/multilingual-image-ca

Gunjan Chhablani 32 Nov 25, 2022
DFFNet: An IoT-perceptive Dual Feature Fusion Network for General Real-time Semantic Segmentation

DFFNet Paper DFFNet: An IoT-perceptive Dual Feature Fusion Network for General Real-time Semantic Segmentation. Xiangyan Tang, Wenxuan Tu, Keqiu Li, J

4 Sep 23, 2022
Data and Code for ACL 2021 Paper "Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning"

Introduction Code and data for ACL 2021 Paper "Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning". We cons

Pan Lu 81 Dec 27, 2022
Transformers based fully on MLPs

Awesome MLP-based Transformers papers An up-to-date list of Transformers based fully on MLPs without attention! Why this repo? After transformers and

Fawaz Sammani 35 Dec 30, 2022
利用Tensorflow实现基于CNN的中文短文本分类

Text Classification with CNN 使用卷积神经网络进行中文文本分类 CNN做句子分类的论文可以参看: Convolutional Neural Networks for Sentence Classification 还可以去读dennybritz大牛的博客:Implemen

Jeremiah 4 Nov 08, 2022