code for Multi-scale Matching Networks for Semantic Correspondence, ICCV

Related tags

Deep LearningMMNet
Overview

MMNet

This repo is the official implementation of ICCV 2021 paper "Multi-scale Matching Networks for Semantic Correspondence.".

Pre-requisite

conda create -n mmnet python==3.8.0
conda activate mmnet
conda install torch==1.8.1 torchvision==0.9.1
pip install matplotlib scikit-image pandas

for installation of gluoncvth (fcn-resnet101):

git clone https://github.com/StacyYang/gluoncv-torch.git
cd gluoncv-torch
python setup.py install

Reproduction

for test

Trained models are available on [google drive].

pascal with fcn-resnet101 backbone([email protected]:81.6%):

python test.py --alpha 0.05 --backbone fcn-resnet101 --ckp_name path\to\ckp_pascal_fcnres101.pth --resize 224,320

spair with fcn-resnet101 backbone([email protected]:46.6%):

python test.py --alpha 0.05 --benchmark spair --backbone fcn-resnet101 --ckp_name path\to\ckp_spair_fcnres101.pth --resize 224,320

Bibtex

If you use this code for your research, please consider citing:

@article{zhao2021multi,
  title={Multi-scale Matching Networks for Semantic Correspondence},
  author={Zhao, Dongyang and Song, Ziyang and Ji, Zhenghao and Zhao, Gangming and Ge, Weifeng and Yu, Yizhou},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2021}
}
You might also like...
A Pytorch implementation of
A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generative Modeling" (ICCV 2021)

Manifold Matching via Deep Metric Learning for Generative Modeling A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generat

null 69 Dec 10, 2022
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)

Hierarchical Memory Matching Network for Video Object Segmentation Hongje Seong, Seoung Wug Oh, Joon-Young Lee, Seongwon Lee, Suhyeon Lee, Euntai Kim

Hongje Seong 72 Dec 14, 2022
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)

Hierarchical Memory Matching Network for Video Object Segmentation Hongje Seong, Seoung Wug Oh, Joon-Young Lee, Seongwon Lee, Suhyeon Lee, Euntai Kim

Hongje Seong 26 Sep 26, 2021
Multi-Scale Geometric Consistency Guided Multi-View Stereo

ACMM [News] The code for ACMH is released!!! [News] The code for ACMP is released!!! About ACMM is a multi-scale geometric consistency guided multi-vi

Qingshan Xu 118 Jan 4, 2023
Siamese-nn-semantic-text-similarity - A repository containing comprehensive Neural Networks based PyTorch implementations for the semantic text similarity task
Siamese-nn-semantic-text-similarity - A repository containing comprehensive Neural Networks based PyTorch implementations for the semantic text similarity task

Siamese Deep Neural Networks for Semantic Text Similarity PyTorch A repository c

Shahrukh Khan 32 Dec 15, 2022
A PyTorch implementation of
A PyTorch implementation of "Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning", IJCAI-21

MERIT A PyTorch implementation of our IJCAI-21 paper Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning. Depen

Graph Analysis & Deep Learning Laboratory, GRAND 32 Jan 2, 2023
Implementation of Memory-Efficient Neural Networks with Multi-Level Generation, ICCV 2021
Implementation of Memory-Efficient Neural Networks with Multi-Level Generation, ICCV 2021

Memory-Efficient Multi-Level In-Situ Generation (MLG) By Jiaqi Gu, Hanqing Zhu, Chenghao Feng, Mingjie Liu, Zixuan Jiang, Ray T. Chen and David Z. Pan

Jiaqi Gu 2 Jan 4, 2022
Official Pytorch implementation of 'GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network' (NeurIPS 2020)
Official Pytorch implementation of 'GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network' (NeurIPS 2020)

Official implementation of GOCor This is the official implementation of our paper : GOCor: Bringing Globally Optimized Correspondence Volumes into You

Prune Truong 71 Nov 18, 2022
《Dual-Resolution Correspondence Network》(NeurIPS 2020)
《Dual-Resolution Correspondence Network》(NeurIPS 2020)

Dual-Resolution Correspondence Network Dual-Resolution Correspondence Network, NeurIPS 2020 Dependency All dependencies are included in asset/dualrcne

Active Vision Laboratory 45 Nov 21, 2022
Comments
  • NaN during training

    NaN during training

    Hi, congrats on your paper! I was trying to run your training code (with resnet 101 on pf-pascal) but directly after a couple of iterations, nan appear in the input. Have you ever seen this issue? Thanks

    opened by PruneTruong 2
  • In def calLayer1,i do not know where are self.conv1_1_down,self.conv1_2_down,self.conv1_3_down,self.wa_1

    In def calLayer1,i do not know where are self.conv1_1_down,self.conv1_2_down,self.conv1_3_down,self.wa_1

    Hello,this paper is very nice,i am very love it. I read your code,in Model.py, def calLayer1(self, feats): sum1 = self.conv1_1_down(self.msblock1_1(feats[1])) +
    self.conv1_2_down(self.msblock1_2(feats[2])) +
    self.conv1_3_down(self.msblock1_3(feats[3])) sum1 = self.wa_1(sum1) return sum1 I do not find where are these operation,self.conv1_1_down,self.conv1_2_down,self.conv1_3_down,self.wa_1,so where are these ,in which document.Thank you,looking forward to your reply.

    opened by liang532 1
  • How to prepare the PF-Pascal dataset?

    How to prepare the PF-Pascal dataset?

    I downloaded the PF-dataset-Pascal.zip from the Proposal Flow paper's web page, extracted it, and run the next line of command, but get errors about missing data files.

    Input:

    python test.py --alpha 0.05 --backbone fcn-resnet101 --ckp_name assets/model/mmnet_fcnresnet101_pascal.pth --resize 224,320

    Expected output: some results about the benchmark results.

    Actual output:

    currently executing test.py file.
2021-11-19 02:01:59,172 - INFO - Options listed below:----------------
2021-11-19 02:01:59,172 - INFO - name: framework_train
2021-11-19 02:01:59,172 - INFO - benchmark: pfpascal
2021-11-19 02:01:59,172 - INFO - thresh_type: auto
2021-11-19 02:01:59,172 - INFO - backbone_name: fcn-resnet101
2021-11-19 02:01:59,172 - INFO - ms_rate: 4
2021-11-19 02:01:59,173 - INFO - feature_channel: 21
2021-11-19 02:01:59,173 - INFO - batch: 5
2021-11-19 02:01:59,173 - INFO - gpu: 0
2021-11-19 02:01:59,173 - INFO - data_path: /data/SC_Dataset
2021-11-19 02:01:59,173 - INFO - ckp_path: ./checkpoints_debug
2021-11-19 02:01:59,173 - INFO - visualization_path: visualization
2021-11-19 02:01:59,173 - INFO - model_type: MMNet
2021-11-19 02:01:59,173 - INFO - ckp_name: assets/model/mmnet_fcnresnet101_pascal.pth
2021-11-19 02:01:59,173 - INFO - log_path: ./logs/
2021-11-19 02:01:59,173 - INFO - resize: 224,320
2021-11-19 02:01:59,173 - INFO - max_kps_num: 50
2021-11-19 02:01:59,173 - INFO - split_type: test
2021-11-19 02:01:59,173 - INFO - alpha: 0.05
2021-11-19 02:01:59,173 - INFO - resolution: 2
2021-11-19 02:01:59,173 - INFO - Options all listed.------------------
2021-11-19 02:01:59,173 - INFO - ckp file: assets/model/mmnet_fcnresnet101_pascal.pth
Traceback (most recent call last):
  File "/home/runner/MMNet/test.py", line 127, in <module>
    test(logger, options)
  File "/home/runner/MMNet/test.py", line 65, in test
    test_dataset = Dataset.CorrespondenceDataset(
  File "/home/runner/MMNet/data/PascalDataset.py", line 32, in __init__
    self.train_data = pd.read_csv(self.spt_path)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 586, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 482, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 811, in __init__
    self._engine = self._make_engine(self.engine)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1040, in _make_engine
    return mapping[engine](self.f, **self.options)  # type: ignore[call-arg]
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 51, in __init__
    self._open_handles(src, kwds)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/base_parser.py", line 222, in _open_handles
    self.handles = get_handle(
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/common.py", line 702, in get_handle
    handle = open(
FileNotFoundError: [Errno 2] No such file or directory: '/data/SC_Dataset/PF-PASCAL/test_pairs.csv'

    P.S. Output of executing ls /data/SC_Dataset/PF-PASCAL/:

    Annotations  html  index.html  JPEGImages  parsePascalVOC.mat  ShowMatchingPairs
    opened by tjyuyao 2
  • How to reproduce the reported test accuracy?

    How to reproduce the reported test accuracy?

    By running given following command with code on the main branch:

    python test.py --alpha 0.05 --backbone fcn-resnet101 --ckp_name assets/model/mmnet_fcnresnet101_spair.pth --resize 224,320 --benchmark spair

    I expect to get the reported accuracy in the Table.2 of paper, i.e. 50.4 "all" accuracy, or spair with fcn-resnet101 backbone([email protected]:46.6%): as noted in the README.md file. However I get the following output, finding nowhere the related results. Can you point out the steps to reproduce the test accuracy?

    2021-11-19 00:49:54,452 - INFO - Options listed below:----------------
2021-11-19 00:49:54,452 - INFO - name: framework_train
2021-11-19 00:49:54,453 - INFO - benchmark: spair
2021-11-19 00:49:54,453 - INFO - thresh_type: auto
2021-11-19 00:49:54,454 - INFO - backbone_name: fcn-resnet101
2021-11-19 00:49:54,455 - INFO - ms_rate: 4
2021-11-19 00:49:54,455 - INFO - feature_channel: 21
2021-11-19 00:49:54,456 - INFO - batch: 5
2021-11-19 00:49:54,456 - INFO - gpu: 0
2021-11-19 00:49:54,457 - INFO - data_path: /data/SC_Dataset
2021-11-19 00:49:54,457 - INFO - ckp_path: ./checkpoints_debug
2021-11-19 00:49:54,458 - INFO - visualization_path: visualization
2021-11-19 00:49:54,458 - INFO - model_type: MMNet
2021-11-19 00:49:54,459 - INFO - ckp_name: assets/model/mmnet_fcnresnet101_spair.pth
2021-11-19 00:49:54,459 - INFO - log_path: ./logs/
2021-11-19 00:49:54,460 - INFO - resize: 224,320
2021-11-19 00:49:54,460 - INFO - max_kps_num: 50
2021-11-19 00:49:54,461 - INFO - split_type: test
2021-11-19 00:49:54,461 - INFO - alpha: 0.05
2021-11-19 00:49:54,462 - INFO - resolution: 2
2021-11-19 00:49:54,462 - INFO - Options all listed.------------------
2021-11-19 00:49:54,463 - INFO - ckp file: assets/model/mmnet_fcnresnet101_spair.pth
2021-11-19 00:50:04,950 - INFO - [    0/12234]: 	 [Pair PCK: 0.333]	[Average: 0.333] aeroplane
2021-11-19 00:50:04,953 - INFO - [    1/12234]: 	 [Pair PCK: 0.100]	[Average: 0.217] aeroplane
2021-11-19 00:50:04,956 - INFO - [    2/12234]: 	 [Pair PCK: 0.308]	[Average: 0.247] aeroplane
2021-11-19 00:50:04,958 - INFO - [    3/12234]: 	 [Pair PCK: 0.364]	[Average: 0.276] aeroplane
2021-11-19 00:50:04,960 - INFO - [    4/12234]: 	 [Pair PCK: 0.000]	[Average: 0.221] aeroplane
2021-11-19 00:50:05,575 - INFO - [    5/12234]: 	 [Pair PCK: 0.200]	[Average: 0.217] aeroplane
2021-11-19 00:50:05,577 - INFO - [    6/12234]: 	 [Pair PCK: 0.250]	[Average: 0.222] aeroplane
2021-11-19 00:50:05,580 - INFO - [    7/12234]: 	 [Pair PCK: 0.308]	[Average: 0.233] aeroplane
2021-11-19 00:50:05,583 - INFO - [    8/12234]: 	 [Pair PCK: 0.182]	[Average: 0.227] aeroplane
2021-11-19 00:50:05,585 - INFO - [    9/12234]: 	 [Pair PCK: 0.636]	[Average: 0.268] aeroplane
2021-11-19 00:50:06,153 - INFO - [   10/12234]: 	 [Pair PCK: 0.667]	[Average: 0.304] aeroplane
2021-11-19 00:50:06,156 - INFO - [   11/12234]: 	 [Pair PCK: 0.385]	[Average: 0.311] aeroplane
2021-11-19 00:50:06,158 - INFO - [   12/12234]: 	 [Pair PCK: 0.455]	[Average: 0.322] aeroplane
2021-11-19 00:50:06,160 - INFO - [   13/12234]: 	 [Pair PCK: 0.250]	[Average: 0.317] aeroplane
2021-11-19 00:50:06,163 - INFO - [   14/12234]: 	 [Pair PCK: 0.615]	[Average: 0.337] aeroplane
2021-11-19 00:50:06,731 - INFO - [   15/12234]: 	 [Pair PCK: 0.000]	[Average: 0.316] aeroplane
...
2021-11-19 01:13:47,264 - INFO - [12216/12234]: 	 [Pair PCK: 0.222]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,265 - INFO - [12217/12234]: 	 [Pair PCK: 0.200]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,266 - INFO - [12218/12234]: 	 [Pair PCK: 0.250]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,268 - INFO - [12219/12234]: 	 [Pair PCK: 0.222]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,837 - INFO - [12220/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,838 - INFO - [12221/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,848 - INFO - [12222/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,850 - INFO - [12223/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,853 - INFO - [12224/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,422 - INFO - [12225/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,424 - INFO - [12226/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,425 - INFO - [12227/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,427 - INFO - [12228/12234]: 	 [Pair PCK: 0.333]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,429 - INFO - [12229/12234]: 	 [Pair PCK: 0.222]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,896 - INFO - [12230/12234]: 	 [Pair PCK: 0.333]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,899 - INFO - [12231/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,899 - INFO - [12232/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,901 - INFO - [12233/12234]: 	 [Pair PCK: 0.111]	[Average: 0.333] tvmonitor
    opened by tjyuyao 1
Releases(v0.1.0)
Owner
joey zhao
Master in Computer Sciences and Technology at Fudan University
joey zhao
GitHub Repository
EncT5: Fine-tuning T5 Encoder for Non-autoregressive Tasks

EncT5 (Unofficial) Pytorch Implementation of EncT5: Fine-tuning T5 Encoder for Non-autoregressive Tasks About Finetune T5 model for classification & r

Jangwon Park 34 Jan 01, 2023
🏆 The 1st Place Submission to AICity Challenge 2021 Natural Language-Based Vehicle Retrieval Track (Alibaba-UTS submission)

AI City 2021: Connecting Language and Vision for Natural Language-Based Vehicle Retrieval 🏆 The 1st Place Submission to AICity Challenge 2021 Natural

82 Dec 29, 2022
'Solving the sampling problem of the Sycamore quantum supremacy circuits

solve_sycamore This repo contains data, contraction code, and contraction order for the paper ''Solving the sampling problem of the Sycamore quantum s

Feng Pan 29 Nov 28, 2022
Pre-trained model, code, and materials from the paper "Impact of Adversarial Examples on Deep Learning Models for Biomedical Image Segmentation" (MICCAI 2019).

Adaptive Segmentation Mask Attack This repository contains the implementation of the Adaptive Segmentation Mask Attack (ASMA), a targeted adversarial

Utku Ozbulak 53 Jul 04, 2022
"Learning and Analyzing Generation Order for Undirected Sequence Models" in Findings of EMNLP, 2021

undirected-generation-dev This repo contains the source code of the models described in the following paper "Learning and Analyzing Generation Order f

Yichen Jiang 0 Mar 25, 2022
Pure python implementations of popular ML algorithms.

Minimal ML algorithms This repo includes minimal implementations of popular ML algorithms using pure python and numpy. The purpose of these notebooks

Alexis Gidiotis 3 Jan 10, 2022
AttentionGAN for Unpaired Image-to-Image Translation & Multi-Domain Image-to-Image Translation

AttentionGAN-v2 for Unpaired Image-to-Image Translation AttentionGAN-v2 Framework The proposed generator learns both foreground and background attenti

Hao Tang 530 Dec 27, 2022
An excellent hash algorithm combining classical sponge structure and RNN.

SHA-RNN Recurrent Neural Network with Chaotic System for Hash Functions Anonymous Authors [摘要] 在这次作业中我们提出了一种新的 Hash Function —— SHA-RNN。其以海绵结构为基础，融合了混

Houde Qian 5 May 15, 2022
Conversion between units used in magnetism

convmag Conversion between various units used in magnetism The conversions between base units available are: T - G : 1e4

0 Jul 15, 2021
CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP

CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP Andreas Fürst* 1, Elisabeth Rumetshofer* 1, Viet Tran1, Hubert Ramsauer1, Fei Tang3, Joh

Institute for Machine Learning, Johannes Kepler University Linz 133 Jan 04, 2023
Code of PVTv2 is released! PVTv2 largely improves PVTv1 and works better than Swin Transformer with ImageNet-1K pre-training.

Updates (2020/06/21) Code of PVTv2 is released! PVTv2 largely improves PVTv1 and works better than Swin Transformer with ImageNet-1K pre-training. Pyr

1.3k Jan 04, 2023
Code release for The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification (TIP 2020)

The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification Code release for The Devil is in the Channels: Mutual-Channel

PRIS-CV: Computer Vision Group 230 Dec 31, 2022
根据midi文件演奏“风物之诗琴”的脚本 "Windsong Lyre" auto play

Genshin-lyre-auto-play 简体中文 | English 简介 根据midi文件演奏“风物之诗琴”的脚本。由Python驱动,在此承诺， ⚠️ 项目内绝不含任何能够引起安全问题的代码。 前排提示：所有键盘在动但是原神没反应的都是因为没有管理员权限，双击run.bat或者以管理员模式

御坂17032号 386 Jan 01, 2023
Tensorflow solution of NER task Using BiLSTM-CRF model with Google BERT Fine-tuning And private Server services

Tensorflow solution of NER task Using BiLSTM-CRF model with Google BERT Fine-tuning

MaCan 4.2k Dec 29, 2022
Captcha-tensorflow - Image Captcha Solving Using TensorFlow and CNN Model. Accuracy 90%+

Captcha Solving Using TensorFlow Introduction Solve captcha using TensorFlow. Learn CNN and TensorFlow by a practical project. Follow the steps, run t

Jackon Yang 869 Jan 06, 2023
ESP32 python application to read data from a Tilt™ Hydrometer for homebrewing

TitlESP32 ESP32 MicroPython application to read and log data from a Tilt™ Hydrometer. Requirements A board with an ESP32 chip USB cable - USB A / micr

IoBeer 5 Dec 01, 2022
Scripts and outputs related to the paper Prediction of Adverse Biological Effects of Chemicals Using Knowledge Graph Embeddings.

Knowledge Graph Embeddings and Chemical Effect Prediction, 2020. Scripts and outputs related to the paper Prediction of Adverse Biological Effects of

Knowledge Graphs at the Norwegian Institute for Water Research 1 Nov 01, 2021
LiDAR Distillation: Bridging the Beam-Induced Domain Gap for 3D Object Detection

LiDAR Distillation Paper | Model LiDAR Distillation: Bridging the Beam-Induced Domain Gap for 3D Object Detection Yi Wei, Zibu Wei, Yongming Rao, Jiax

Yi Wei 75 Dec 22, 2022
The repository for our EMNLP 2021 paper "Finnish Dialect Identification: The Effect of Audio and Text"

Finnish Dialect Identification The repository for our EMNLP 2021 paper "Finnish Dialect Identification: The Effect of Audio and Text". We present a te

Rootroo Ltd 2 Dec 25, 2021
LeafSnap replicated using deep neural networks to test accuracy compared to traditional computer vision methods.

Deep-Leafsnap Convolutional Neural Networks have become largely popular in image tasks such as image classification recently largely due to to Krizhev

Sujith Vishwajith 48 Nov 27, 2022