Deep Dual Consecutive Network for Human Pose Estimation (CVPR2021)

Related tags

Deep LearningDCPose
Overview

Deep Dual Consecutive Network for Human Pose Estimation (CVPR2021)

Introduction

This is the official code of Deep Dual Consecutive Network for Human Pose Estimation.

Multi-frame human pose estimation in complicated situations is challenging. Although state-of-the-art human joints detectors have demonstrated remarkable results for static images, their performances come short when we apply these models to video sequences. Prevalent shortcomings include the failure to handle motion blur, video defocus, or pose occlusions, arising from the inability in capturing the temporal dependency among video frames. On the other hand, directly employing conventional recurrent neural networks incurs empirical difficulties in modeling spatial contexts, especially for dealing with pose occlusions. In this paper, we propose a novel multi-frame human pose estimation framework, leveraging abundant temporal cues between video frames to facilitate keypoint detection. Three modular components are designed in our framework. A Pose Temporal Merger encodes keypoint spatiotemporal context to generate effective searching scopes while a Pose Residual Fusion module computes weighted pose residuals in dual directions. These are then processed via our Pose Correction Network for efficient refining of pose estimations. Our method ranks No.1 in the Multi-frame Person Pose Estimation Challenge on the large-scale benchmark datasets PoseTrack2017 and PoseTrack2018. We have released our code, hoping to inspire future research.

Visual Results

On PoseTrack

Comparison with SOTA method

Experiments

Results on PoseTrack 2017 validation set

Method Head Shoulder Elbow Wrist Hip Knee Ankle Mean
PoseFlow 66.7 73.3 68.3 61.1 67.5 67.0 61.3 66.5
JointFlow - - - - - - - 69.3
FastPose 80.0 80.3 69.5 59.1 71.4 67.5 59.4 70.3
SimpleBaseline(2018 ECCV) 81.7 83.4 80.0 72.4 75.3 74.8 67.1 76.7
STEmbedding 83.8 81.6 77.1 70.0 77.4 74.5 70.8 77.0
HRNet(2019 CVPR) 82.1 83.6 80.4 73.3 75.5 75.3 68.5 77.3
MDPN 85.2 88.8 83.9 77.5 79.0 77.0 71.4 80.7
PoseWarper(2019 NIPS) 81.4 88.3 83.9 78.0 82.4 80.5 73.6 81.2
DCPose 88.0 88.7 84.1 78.4 83.0 81.4 74.2 82.8

Results on PoseTrack 2017 test set(https://posetrack.net/leaderboard.php)

Method Head Shoulder Elbow Wrist Hip Knee Ankle Total
PoseFlow 64.9 67.5 65.0 59.0 62.5 62.8 57.9 63.0
JointFlow - - - 53.1 - - 50.4 63.4
KeyTrack - - - 71.9 - - 65.0 74.0
DetTrack - - - 69.8 - - 65.9 74.1
SimpleBaseline 80.1 80.2 76.9 71.5 72.5 72.4 65.7 74.6
HRNet 80.0 80.2 76.9 72.0 73.4 72.5 67.0 74.9
PoseWarper 79.5 84.3 80.1 75.8 77.6 76.8 70.8 77.9
DCPose 84.3 84.9 80.5 76.1 77.9 77.1 71.2 79.2

Results on PoseTrack 2018 validation set

Method Head Shoulder Elbow Wrist Hip Knee Ankle Mean
AlphaPose 63.9 78.7 77.4 71.0 73.7 73.0 69.7 71.9
MDPN 75.4 81.2 79.0 74.1 72.4 73.0 69.9 75.0
PoseWarper 79.9 86.3 82.4 77.5 79.8 78.8 73.2 79.7
DCPose 84.0 86.6 82.7 78.0 80.4 79.3 73.8 80.9

Results on PoseTrack 2018 test set

Method Head Shoulder Elbow Wrist Hip Knee Ankle Mean
AlphaPose++ - - - 66.2 - - 65.0 67.6
DetTrack - - - 69.8 - - 67.1 73.5
MDPN - - - 74.5 - - 69.0 76.4
PoseWarper 78.9 84.4 80.9 76.8 75.6 77.5 71.8 78.0
DCPose 82.8 84.0 80.8 77.2 76.1 77.6 72.3 79.0

Installation & Quick Start

Check docs/installation.md for instructions on how to build DCPose from source.

Eth brownie struct encoding example

eth-brownie struct encoding example Overview This repository contains an example of encoding a struct, so that it can be used in a function call, usin

Ittai Svidler 2 Mar 04, 2022
You Only Look Once for Panopitic Driving Perception

You Only 👀 Once for Panoptic 🚗 Perception You Only Look at Once for Panoptic driving Perception by Dong Wu, Manwen Liao, Weitian Zhang, Xinggang Wan

Hust Visual Learning Team 1.4k Jan 04, 2023
A programming language written with python

Kaoft A programming language written with python How to use A simple Hello World: c="Hello World" c Output: "Hello World" Operators: a=12

1 Jan 24, 2022
Unofficial PyTorch Implementation of Multi-Singer

Multi-Singer Unofficial PyTorch Implementation of Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpus. Requirements See re

SunMail-hub 123 Dec 28, 2022
Official Pytorch implementation for AAAI2021 paper (RSPNet: Relative Speed Perception for Unsupervised Video Representation Learning)

RSPNet Official Pytorch implementation for AAAI2021 paper "RSPNet: Relative Speed Perception for Unsupervised Video Representation Learning" [Suppleme

35 Jun 24, 2022
Library for fast text representation and classification.

fastText fastText is a library for efficient learning of word representations and sentence classification. Table of contents Resources Models Suppleme

Facebook Research 24.1k Jan 01, 2023
Distributed Evolutionary Algorithms in Python

DEAP DEAP is a novel evolutionary computation framework for rapid prototyping and testing of ideas. It seeks to make algorithms explicit and data stru

Distributed Evolutionary Algorithms in Python 4.9k Jan 05, 2023
Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation. In CVPR 2022.

Nonuniform-to-Uniform Quantization This repository contains the training code of N2UQ introduced in our CVPR 2022 paper: "Nonuniform-to-Uniform Quanti

Zechun Liu 60 Dec 28, 2022
Dyalog-apl-docset - Dyalog APL Dash Docset Generator

Dyalog APL Dash Docset Generator o alasa e kili sona kepeken tenpo lili a A Dash

Maciej Goszczycki 1 Jan 10, 2022
Simple API for UCI Machine Learning Dataset Repository (search, download, analyze)

A simple API for working with University of California, Irvine (UCI) Machine Learning (ML) repository Table of Contents Introduction About Page of the

Tirthajyoti Sarkar 223 Dec 05, 2022
Ludwig is a toolbox that allows to train and evaluate deep learning models without the need to write code.

Translated in 🇰🇷 Korean/ Ludwig is a toolbox that allows users to train and test deep learning models without the need to write code. It is built on

Ludwig 8.7k Dec 31, 2022
Official implementation of "Implicit Neural Representations with Periodic Activation Functions"

Implicit Neural Representations with Periodic Activation Functions Project Page | Paper | Data Vincent Sitzmann*, Julien N. P. Martel*, Alexander W. B

Vincent Sitzmann 1.4k Jan 06, 2023
Official PyTorch implementation of the paper "Deep Constrained Least Squares for Blind Image Super-Resolution", CVPR 2022.

Deep Constrained Least Squares for Blind Image Super-Resolution [Paper] This is the official implementation of 'Deep Constrained Least Squares for Bli

MEGVII Research 141 Dec 30, 2022
Pytorch implementation of MLP-Mixer with loading pre-trained models.

MLP-Mixer-Pytorch PyTorch implementation of MLP-Mixer: An all-MLP Architecture for Vision with the function of loading official ImageNet pre-trained p

Qiushi Yang 2 Sep 29, 2022
This repository contains tutorials for the py4DSTEM Python package

py4DSTEM Tutorials This repository contains tutorials for the py4DSTEM Python package. For more information about py4DSTEM, including installation ins

11 Dec 23, 2022
DIT is a DTLS MitM proxy implemented in Python 3. It can intercept, manipulate and suppress datagrams between two DTLS endpoints and supports psk-based and certificate-based authentication schemes (RSA + ECC).

DIT - DTLS Interception Tool DIT is a MitM proxy tool to intercept DTLS traffic. It can intercept, manipulate and/or suppress DTLS datagrams between t

52 Nov 30, 2022
JAX + dataclasses

jax_dataclasses jax_dataclasses provides a wrapper around dataclasses.dataclass for use in JAX, which enables automatic support for: Pytree registrati

Brent Yi 35 Dec 21, 2022
Implementation of the master's thesis "Temporal copying and local hallucination for video inpainting".

Temporal copying and local hallucination for video inpainting This repository contains the implementation of my master's thesis "Temporal copying and

David Álvarez de la Torre 1 Dec 02, 2022
A Python Reconnection Tool for alt:V

altv-reconnect What? It invokes a reconnect in the altV Client Dev Console. You get to determine when your local client should reconnect when developi

8 Jun 30, 2022
DeepLab is a state-of-art deep learning system for semantic image segmentation built on top of Caffe.

DeepLab Introduction DeepLab is a state-of-art deep learning system for semantic image segmentation built on top of Caffe. It combines densely-compute

Ali 234 Nov 14, 2022