Deep Dual Consecutive Network for Human Pose Estimation (CVPR2021)

Related tags

Deep LearningDCPose
Overview

Deep Dual Consecutive Network for Human Pose Estimation (CVPR2021)

Introduction

This is the official code of Deep Dual Consecutive Network for Human Pose Estimation.

Multi-frame human pose estimation in complicated situations is challenging. Although state-of-the-art human joints detectors have demonstrated remarkable results for static images, their performances come short when we apply these models to video sequences. Prevalent shortcomings include the failure to handle motion blur, video defocus, or pose occlusions, arising from the inability in capturing the temporal dependency among video frames. On the other hand, directly employing conventional recurrent neural networks incurs empirical difficulties in modeling spatial contexts, especially for dealing with pose occlusions. In this paper, we propose a novel multi-frame human pose estimation framework, leveraging abundant temporal cues between video frames to facilitate keypoint detection. Three modular components are designed in our framework. A Pose Temporal Merger encodes keypoint spatiotemporal context to generate effective searching scopes while a Pose Residual Fusion module computes weighted pose residuals in dual directions. These are then processed via our Pose Correction Network for efficient refining of pose estimations. Our method ranks No.1 in the Multi-frame Person Pose Estimation Challenge on the large-scale benchmark datasets PoseTrack2017 and PoseTrack2018. We have released our code, hoping to inspire future research.

Visual Results

On PoseTrack

Comparison with SOTA method

Experiments

Results on PoseTrack 2017 validation set

Method Head Shoulder Elbow Wrist Hip Knee Ankle Mean
PoseFlow 66.7 73.3 68.3 61.1 67.5 67.0 61.3 66.5
JointFlow - - - - - - - 69.3
FastPose 80.0 80.3 69.5 59.1 71.4 67.5 59.4 70.3
SimpleBaseline(2018 ECCV) 81.7 83.4 80.0 72.4 75.3 74.8 67.1 76.7
STEmbedding 83.8 81.6 77.1 70.0 77.4 74.5 70.8 77.0
HRNet(2019 CVPR) 82.1 83.6 80.4 73.3 75.5 75.3 68.5 77.3
MDPN 85.2 88.8 83.9 77.5 79.0 77.0 71.4 80.7
PoseWarper(2019 NIPS) 81.4 88.3 83.9 78.0 82.4 80.5 73.6 81.2
DCPose 88.0 88.7 84.1 78.4 83.0 81.4 74.2 82.8

Results on PoseTrack 2017 test set(https://posetrack.net/leaderboard.php)

Method Head Shoulder Elbow Wrist Hip Knee Ankle Total
PoseFlow 64.9 67.5 65.0 59.0 62.5 62.8 57.9 63.0
JointFlow - - - 53.1 - - 50.4 63.4
KeyTrack - - - 71.9 - - 65.0 74.0
DetTrack - - - 69.8 - - 65.9 74.1
SimpleBaseline 80.1 80.2 76.9 71.5 72.5 72.4 65.7 74.6
HRNet 80.0 80.2 76.9 72.0 73.4 72.5 67.0 74.9
PoseWarper 79.5 84.3 80.1 75.8 77.6 76.8 70.8 77.9
DCPose 84.3 84.9 80.5 76.1 77.9 77.1 71.2 79.2

Results on PoseTrack 2018 validation set

Method Head Shoulder Elbow Wrist Hip Knee Ankle Mean
AlphaPose 63.9 78.7 77.4 71.0 73.7 73.0 69.7 71.9
MDPN 75.4 81.2 79.0 74.1 72.4 73.0 69.9 75.0
PoseWarper 79.9 86.3 82.4 77.5 79.8 78.8 73.2 79.7
DCPose 84.0 86.6 82.7 78.0 80.4 79.3 73.8 80.9

Results on PoseTrack 2018 test set

Method Head Shoulder Elbow Wrist Hip Knee Ankle Mean
AlphaPose++ - - - 66.2 - - 65.0 67.6
DetTrack - - - 69.8 - - 67.1 73.5
MDPN - - - 74.5 - - 69.0 76.4
PoseWarper 78.9 84.4 80.9 76.8 75.6 77.5 71.8 78.0
DCPose 82.8 84.0 80.8 77.2 76.1 77.6 72.3 79.0

Installation & Quick Start

Check docs/installation.md for instructions on how to build DCPose from source.

PyTorch implementation of Deformable Convolution

PyTorch implementation of Deformable Convolution !!!Warning: There is some issues in this implementation and this repo is not maintained any more, ple

Wei Ouyang 893 Dec 18, 2022
1st ranked 'driver careless behavior detection' for AI Online Competition 2021, hosted by MSIT Korea.

2021AICompetition-03 본 repo 는 mAy-I Inc. 팀으로 참가한 2021 인공지능 온라인 경진대회 중 [이미지] 운전 사고 예방을 위한 운전자 부주의 행동 검출 모델] 태스크 수행을 위한 레포지토리입니다. mAy-I 는 과학기술정보통신부가 주최하

Junhyuk Park 9 Dec 01, 2022
The hippynn python package - a modular library for atomistic machine learning with pytorch.

The hippynn python package - a modular library for atomistic machine learning with pytorch. We aim to provide a powerful library for the training of a

Los Alamos National Laboratory 37 Dec 29, 2022
A simple editor for captions in .SRT file extension

WaySRT A simple editor for captions in .SRT file extension The program doesn't use any external dependecies, just run: python way_srt.py {file_name.sr

Gustavo Lopes 3 Nov 16, 2022
Multi Agent Path Finding Algorithms

MATP-solver Simulator collision check path step random initial states or given states Traditional method Seperate A* algorithem Confict-based Search S

30 Dec 12, 2022
Supervised Classification from Text (P)

MSc-Thesis Module: Masters Research Thesis Language: Python Grade: 75 Title: An investigation of supervised classification of therapeutic process from

Matthew Laws 1 Nov 22, 2021
Xi Dongbo 78 Nov 29, 2022
A script that trains a model to recognize handwritten digits using the MNIST data set.

handwritten-digits-recognition A script that trains a model to recognize handwritten digits using the MNIST data set. Then it loads external files and

Hamza Sayih 1 Oct 30, 2021
Wide Residual Networks (WideResNets) in PyTorch

Wide Residual Networks (WideResNets) in PyTorch WideResNets for CIFAR10/100 implemented in PyTorch. This implementation requires less GPU memory than

Jason Kuen 296 Dec 27, 2022
Deep Reinforcement Learning by using an on-policy adaptation of Maximum a Posteriori Policy Optimization (MPO)

V-MPO Simple code to demonstrate Deep Reinforcement Learning by using an on-policy adaptation of Maximum a Posteriori Policy Optimization (MPO) in Pyt

Nugroho Dewantoro 9 Jun 06, 2022
The official implementation of CircleNet: Anchor-free Detection with Circle Representation, MICCAI 2030

CircleNet: Anchor-free Detection with Circle Representation The official implementation of CircleNet, MICCAI 2020 [PyTorch] [project page] [MICCAI pap

The Biomedical Data Representation and Learning Lab 45 Nov 18, 2022
GUI for TOAD-GAN, a PCG-ML algorithm for Token-based Super Mario Bros. Levels.

If you are using this code in your own project, please cite our paper: @inproceedings{awiszus2020toadgan, title={TOAD-GAN: Coherent Style Level Gene

Maren A. 13 Dec 14, 2022
Official repository for "Exploiting Session Information in BERT-based Session-aware Sequential Recommendation", SIGIR 2022 short.

Session-aware BERT4Rec Official repository for "Exploiting Session Information in BERT-based Session-aware Sequential Recommendation", SIGIR 2022 shor

Jamie J. Seol 22 Dec 13, 2022
Implementing SYNTHESIZER: Rethinking Self-Attention in Transformer Models using Pytorch

Implementing SYNTHESIZER: Rethinking Self-Attention in Transformer Models using Pytorch Reference Paper URL Author: Yi Tay, Dara Bahri, Donald Metzler

Myeongjun Kim 66 Nov 30, 2022
ICNet and PSPNet-50 in Tensorflow for real-time semantic segmentation

Real-Time Semantic Segmentation in TensorFlow Perform pixel-wise semantic segmentation on high-resolution images in real-time with Image Cascade Netwo

Oles Andrienko 219 Nov 21, 2022
Magisk module to enable hidden features on Android 12 Developer Preview 1.

Android 12 Extensions This is a Magisk module that enables hidden features on Android 12 Developer Preview 1. Features Scrolling screenshots Wallpaper

Danny Lin 384 Jan 06, 2023
PyTorch implementation of Rethinking Positional Encoding in Language Pre-training

TUPE PyTorch implementation of Rethinking Positional Encoding in Language Pre-training. Quickstart Clone this repository. git clone https://github.com

Jake Tae 5 Jan 27, 2022
Keeping it safe - AI Based COVID-19 Tracker using Deep Learning and facial recognition

Keeping it safe - AI Based COVID-19 Tracker using Deep Learning and facial recognition

Vansh Wassan 15 Jun 17, 2021
paper: Hyperspectral Remote Sensing Image Classification Using Deep Convolutional Capsule Network

DC-CapsNet This is a tensorflow and keras based implementation of DC-CapsNet for HSI in the Remote Sensing Letters R. Lei et al., "Hyperspectral Remot

LEI 7 Nov 29, 2022
An addernet CUDA version

Training addernet accelerated by CUDA Usage cd adder_cuda python setup.py install cd .. python main.py Environment pytorch 1.10.0 CUDA 11.3 benchmark

LingXY 4 Jun 20, 2022