Some bravo or inspiring research works on the topic of curriculum learning.

Overview

Awesome Curriculum LearningAwesome

Some bravo or inspiring research works on the topic of curriculum learning.

A curated list of awesome Curriculum Learning resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers, and awesome-architecture-search

Why Curriculum Learning?

Self-Supervised Learning has become an exciting direction in AI community.

  • Bengio: "..." (ICML 2009)

Biological inspired learning scheme.

  • Learn the concepts by increasing complexity, in order to allow learner to exploit previously learned concepts and thus ease the abstraction of new ones.

Contributing

Please help contribute this list by contacting me or add pull request

Markdown format:

- Paper Name.
  [[pdf]](link) 
  [[code]](link)
  - Key Contribution(s)
  - Author 1, Author 2, and Author 3. *Conference Year*

Table of Contents

Mainstreams of curriculum learning

Tag Det Seg Cls Trans Gen Other
Item Detection Semantic Image Classification Transfer Learning Generation other types
Issues (e.g.,) long-tail imbalance imbalance, noise long-tail, domain-shift collapose -

SURVEY

  • Curriculum Learning: A Survey. [pdf]
    • Soviany, Petru and Ionescu, Radu Tudor and Rota, Paolo and Sebe, Nicu. arxiv 2101.10382

2009

  • Curriculum Learning. [pdf]
    • "Rank the weights of the training examples, and use the rank to guide the order of presentation of examples to the learner."
    • Bengio, Yoshua and Louradour, J{'e}r{^o}me and Collobert, Ronan and Weston, Jason. ICML 2009

2015

  • Curriculum Learning. [pdf]
    • Jiang, Lu and Meng, Deyu and Zhao, Qian and Shan, Shiguang and Hauptmann, Alexander G. AAAI 2015

2017

  • Self-Paced Learning: An Implicit Regularization Perspective. [pdf]

    • Fan, Yanbo and He, Ran and Liang, Jian and Hu, Bao-Gang. AAAI 2017
  • Curriculum Dropout. [pdf] [code]

    • Morerio, Pietro and Cavazza, Jacopo and Volpi, Riccardo and Vidal, Ren'e and Murino, Vittorio. ICCV 2017
  • Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes. [pdf] [code]

    • Zhang, Yang and David, Philip and Gong, Boqing. ICCV 2017

2018

  • Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networks. [pdf]

    • "Sort the training examples based on the performance of a pre-trained network on a larger dataset, and then finetune the model to the dataset at hand."
    • Weinshall, Daphna and Cohen, Gad and Amir, Dan. ICML 2018
  • Self-Paced Deep Learning for Weakly Supervised Object Detection. [pdf]

    • Sangineto, Enver and Nabi, Moin and Culibrk, Dubravko and Sebe, Nicu. TPAMI 2018
  • MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks. [pdf] [code]

    • Jiang, Lu and Zhou, Zhengyuan and Leung, Thomas and Li, Li-Jia and Li, Fei-Fei. ICML 2018

MentorNet

  • CurriculumNet: Weakly Supervised Learning from Large-Scale Web Images. [pdf] [code]

    • Guo, Sheng and Huang, Weilin and Zhang, Haozhi and Zhuang, Chenfan and Dong, Dengke and Scott, Matthew and Huang, Dinglong. ECCV 2018
  • Unsupervised Feature Selection by Self-Paced Learning Regularization. [pdf]

    • Zheng, Wei and Zhu, Xiaofeng and Wen, Guoqiu and Zhu, Yonghua and Yu, Hao and Gan, Jiangzhang. Pattern Recognition Letters 2018
  • Learning to Teach with Dynamic Loss Functions. [pdf]

    • "A good teacher not only provides his/her students with qualified teaching materials (e.g., textbooks), but also sets up appropriate learning objectives (e.g., course projects and exams) considering different situations of a student."
    • Wu, Lijun Wu and Tian, Fei Tian and Xia, Yingce and Fan, Yang Fan and Qin, Tao and Lai, Jianhuang and Liu, Tie-Yan. NeurIPS 2018
  • Progressive Growing of GANs for Improved Quality, Stability, and Variation. Gen [pdf] [code]

    • "The key idea is to grow both the generator and discriminator progressively: starting from a low resolution, we add new layers that model increasingly fine details as training progresses. This both speeds the training up and greatly stabilizes it, allowing us to produce images of unprecedented quality."
    • Karras, Tero and Aila, Timo and Laine, Samuli and Lehtinen, Jaakko. ICLR 2018

Progressive growing of GANs

2019

  • Transferable Curriculum for Weakly-Supervised Domain Adaptation. [pdf] [code]

    • Shu, Yang and Cao, Zhangjie and Long, Mingsheng and Wang, Jianmin. AAAI 2019
  • Dynamic Curriculum Learning for Imbalanced Data Classification. [pdf][simple demo]

    • Wang, Yiru and Gan, Weihao and Wu, Wei and Yan, Junjie. ICCV 2019
  • Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation. [pdf]

    • Sakaridis, Christos and Dai, Dengxin and Gool Van Luc. ICCV 2019
  • On The Power of Curriculum Learning in Training Deep Networks. [pdf]

    • Hacohen, Guy and Weinshall, Daphna. ICML 2019
  • Balanced Self-Paced Learning for Generative Adversarial Clustering Network. [pdf]

    • Ghasedi, Kamran and Wang, Xiaoqian and Deng, Cheng and Huang, Heng. CVPR 2019

2020

  • Breaking the Curse of Space Explosion: Towards Effcient NAS with Curriculum Search. [pdf] [code]
    • Guo, Yong and Chen, Yaofo and Zheng, Yin and Zhao, Peilin and Chen, Jian and Huang, Junzhou and Tan, Mingkui. ICML 20

CNAS

  • BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition. [pdf] [code]
    • Zhou, Boyan and Cui, Quan and Wei, Xiu-Shen and Chen, Zhao-Min. CVPR 2020

BBN

  • Open Compound Domain Adaptation. [pdf] [code]
    • Liu, Ziwei and Miao, Zhongqi and Pan, Xingang and Zhan, Xiaohang and Lin, Dahua and Yu, Stella X and Gong, Boqing. CVPR 2020

OCDA

  • Curriculum Manager for Source Selection in Multi-Source Domain Adaptation. [pdf][code]

    • Yang, Luyu and Balaji, Yogesh and Lim, Ser-Nam and Shrivastava, Abhinav. ECCV 2020
  • Content-Consistent Matching for Domain Adaptive Semantic Segmentation. Seg [pdf] [code]

    • "to acquire those synthetic images that share similar distribution with the real ones in the target domain, so that the domain gap can be naturally alleviated by employing the content-consistent synthetic images for training."
    • "not all the source images could contribute to the improvement of adaptation performance, especially at certain training stages."
    • Li, Guangrui and Kang, Guokiang and Liu, Wu and Wei, Yunchao and Yang, Yi . ECCV 2020

CMM

  • DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search. [pdf]

    • "Our method is based on an interesting observation that the learning speed for blocks in deep neural networks is related to the difficulty of recognizing distinct categories. We carefully design a progressive data adapted pruning strategy for efficient architecture search. It will quickly trim low performed blocks on a subset of target dataset (e.g., easy classes), and then gradually find the best blocks on the whole target dataset."
    • Dai, Xiyang and Chen, Dongdong and Liu, Mengchen and Chen, Yinpeng and Yuan, Lu. ECCV 2020
  • Label-similarity Curriculum Learning. [pdf] [code]

    • "The idea is to use a probability distribution over classes as target label, where the class probabilities reflect the similarity to the true class. Gradually, this label representation is shifted towards the standard one-hot-encoding."
    • Dogan, Urun and Deshmukh, A. Aniket, and Machura, Marcin and Igel, Christian. ECCV 2020
  • Self-paced Contrastive Learning with Hybrid Memory for Domain Adaptive Object Re-ID. [pdf] [code] [zhihu]

    • Ge, Yixiao and Zhu, Feng and Chen, Dapeng and Zhao, Rui and Li, Hongsheng. NeurIPS 2020
  • Semi-Supervised Semantic Segmentation via Dynamic Self-Training and Class-Balanced Curriculum. [pdf] [code]

    • Feng, Zhengyang and Zhou, Qianyu and Cheng, Guangliang and Tan, Xin and Shi, Jianping and Ma, Lizhuang. arXiv 2004.08514
  • Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning. [pdf][code]

    • "Evolutionary Population Curriculum (EPC), a curriculum learning paradigm that scales up MultiAgent Reinforcement Learning (MARL) by progressively increasing the population of training agents in a stage-wise manner."
    • Long, Qian and Zhou, Zihan and Gupta, Abhibav and Fang, Fei and Wu, Yi and Wang, Xiaolong. ICLR 2020
  • Self-Paced Deep Reinforcement Learning. [pdf]

    • Klink, Pascal and D'Eramo, Carlo and Peters, R. Jan and Pajarinen, Joni. NeurIPS 2020
  • Automatic Curriculum Learning through Value Disagreement. [pdf]

    • " When biological agents learn, there is often an organized and meaningful order to which learning happens."
    • "Our key insight is that if we can sample goals at the frontier of the set of goals that an agent is able to reach, it will provide a significantly stronger learning signal compared to randomly sampled goals"
    • Zhang, Yunzhi and Abbeel, Pieter and Pinto, Lerrel. NeurIPS 2020

CMM

  • Curriculum by Smoothing. [pdf] [code]

    • Sinha Samarth and Garg Animesh and Larochelle Hugo. NeurIPS 2020
  • Curriculum Learning by Dynamic Instance Hardness. [pdf]

    • Zhou, Tianyi and Wang, Shengjie and Bilmes, A. Jeff. NeurIPS 2020

2021

  • Robust Curriculum Learning: from clean label detection to noisy label self-correction. [pdf] [online review]

    • "Robust curriculum learning (RoCL) improves noisy label learning by periodical transitions from supervised learning of clean labeled data to self-supervision of wrongly-labeled data, where the data are selected according to training dynamics."
    • Zhou, Tianyi and Wang, Shengjie and Bilmes, Jeff. ICLR 2021
  • Robust Early-Learning: Hindering The Memorization of Noisy Labels. [pdf] [online review]

    • "Robust early-learning: to reduce the side effect of noisy labels before early stopping and thus enhance the memorization of clean labels. Specifically, in each iteration, we divide all parameters into the critical and non-critical ones, and then perform different update rules for different types of parameters."
    • Xia, Xiaobo and Liu, Tongliang and Han, Bo and Gong, Chen and Wang, Nannan and Ge, Zongyuan and Chang, Yi. ICLR 2021
  • When Do Curricula Work? [pdf]

    • "We find that for standard benchmark datasets, curricula have only marginal benefits, and that randomly ordered samples perform as well or better than curricula and anti-curricula, suggesting that any benefit is entirely due to the dynamic training set size. ... Our experiments demonstrate that curriculum, but not anti-curriculum or random ordering can indeed improve the performance either with limited training time budget or in the existence of noisy data."
    • Wu, Xiaoxia and Dyer, Ethan and Neyshabur, Behnam. ICLR 2021 (oral)
  • Curriculum Graph Co-Teaching for Multi-Target Domain Adaptation. [pdf] [code]

    • Roy, Subhankar and Krivosheev, Evgeny and Zhong, Zhun and Sebe, Nicu and Ricci, Elisa. CVPR 2021

Curriculum Graph Co-Teaching

  • TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL. [pdf] [code]
    • Romac, Clément and Portelas, Rémy and Hofmann, Katja and Oudeyer, Pierre-Yves. ICML 2021

TechMyAgent

  • Self-Paced Context Evaluation for Contextual Reinforcement Learning. [pdf]

    • "To improve sample efficiency for learning on such instances of a problem domain, we present Self-Paced Context Evaluation (SPaCE). Based on self-paced learning, \spc automatically generates \task curricula online with little computational overhead. To this end, SPaCE leverages information contained in state values during training to accelerate and improve training performance as well as generalization capabilities to new instances from the same problem domain."
    • Eimer, Theresa and Biedenkapp, André and Hutter, Frank and Lindauer Marius. ICML 2021
  • Curriculum Learning by Optimizing Learning Dynamics. [pdf] [code]

    • Zhou, Tianyi and Wang, Shengjie and Bilmes, Jeff. AISTATS 2021

DoCL

  • FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling. [pdf] [code]
    • Zhang, Bowen and Wang, Yidong and Hou, Wenxin and Wu, Hao and Wang, Jindong and Manabu, Okumura and Takahiro, Shinozaki. NeurIPS 2021
Owner
Every tried, every failed, don't worry. Try again, fail again, fail better.
Tree LSTM implementation in PyTorch

Tree-Structured Long Short-Term Memory Networks This is a PyTorch implementation of Tree-LSTM as described in the paper Improved Semantic Representati

Riddhiman Dasgupta 529 Dec 10, 2022
ANN model for prediction a spatio-temporal distribution of supercooled liquid in mixed-phase clouds using Doppler cloud radar spectra.

VOODOO Revealing supercooled liquid beyond lidar attenuation Explore the docs » Report Bug · Request Feature Table of Contents About The Project Built

remsens-lim 2 Apr 28, 2022
Official Implementation of "Designing an Encoder for StyleGAN Image Manipulation"

Designing an Encoder for StyleGAN Image Manipulation (SIGGRAPH 2021) Recently, there has been a surge of diverse methods for performing image editing

749 Jan 09, 2023
TAP: Text-Aware Pre-training for Text-VQA and Text-Caption, CVPR 2021 (Oral)

TAP: Text-Aware Pre-training TAP: Text-Aware Pre-training for Text-VQA and Text-Caption by Zhengyuan Yang, Yijuan Lu, Jianfeng Wang, Xi Yin, Dinei Flo

Microsoft 61 Nov 14, 2022
SIR model parameter estimation using a novel algorithm for differentiated uniformization.

TenSIR Parameter estimation on epidemic data under the SIR model using a novel algorithm for differentiated uniformization of Markov transition rate m

The Spang Lab 4 Nov 30, 2022
This is a Python Module For Encryption, Hashing And Other stuff

EnroCrypt This is a Python Module For Encryption, Hashing And Other Basic Stuff You Need, With Secure Encryption And Strong Salted Hashing You Can Do

5 Sep 15, 2022
Multi-task yolov5 with detection and segmentation based on yolov5

YOLOv5DS Multi-task yolov5 with detection and segmentation based on yolov5(branch v6.0) decoupled head anchor free segmentation head README中文 Ablation

150 Dec 30, 2022
UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation

UnivNet UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation. Training python train.py --c

Rishikesh (ऋषिकेश) 55 Dec 26, 2022
audioLIME: Listenable Explanations Using Source Separation

audioLIME This repository contains the Python package audioLIME, a tool for creating listenable explanations for machine learning models in music info

Institute of Computational Perception 27 Dec 01, 2022
🌊 Online machine learning in Python

In a nutshell River is a Python library for online machine learning. It is the result of a merger between creme and scikit-multiflow. River's ambition

OnlineML 4k Jan 02, 2023
Repository for the COLING 2020 paper "Explainable Automated Fact-Checking: A Survey."

Explainable Fact Checking: A Survey This repository and the accompanying webpage contain resources for the paper "Explainable Fact Checking: A Survey"

Neema Kotonya 42 Nov 17, 2022
Detecting Blurred Ground-based Sky/Cloud Images

Detecting Blurred Ground-based Sky/Cloud Images With the spirit of reproducible research, this repository contains all the codes required to produce t

1 Oct 20, 2021
Instance Semantic Segmentation List

Instance Semantic Segmentation List This repository contains lists of state-or-art instance semantic segmentation works. Papers and resources are list

bighead 87 Mar 06, 2022
Python implementation of "Multi-Instance Pose Networks: Rethinking Top-Down Pose Estimation"

MIPNet: Multi-Instance Pose Networks This repository is the official pytorch python implementation of "Multi-Instance Pose Networks: Rethinking Top-Do

Rawal Khirodkar 57 Dec 12, 2022
Source code and data from the RecSys 2020 article "Carousel Personalization in Music Streaming Apps with Contextual Bandits" by W. Bendada, G. Salha and T. Bontempelli

Carousel Personalization in Music Streaming Apps with Contextual Bandits - RecSys 2020 This repository provides Python code and data to reproduce expe

Deezer 48 Jan 02, 2023
ZeroGen: Efficient Zero-shot Learning via Dataset Generation

ZEROGEN This repository contains the code for our paper “ZeroGen: Efficient Zero

Jiacheng Ye 31 Dec 30, 2022
This repository stores the code to reproduce the results published in "TiWS-iForest: Isolation Forest in Weakly Supervised and Tiny ML scenarios"

TinyWeaklyIsolationForest This repository stores the code to reproduce the results published in "TiWS-iForest: Isolation Forest in Weakly Supervised a

2 Mar 21, 2022
Code release for NeurIPS 2020 paper "Co-Tuning for Transfer Learning"

CoTuning Official implementation for NeurIPS 2020 paper Co-Tuning for Transfer Learning. [News] 2021/01/13 The COCO 70 dataset used in the paper is av

THUML @ Tsinghua University 35 Sep 23, 2022
Conversational text Analysis using various NLP techniques

PyConverse Let me try first Installation pip install pyconverse Usage Please try this notebook that demos the core functionalities: basic usage noteb

Rita Anjana 158 Dec 25, 2022
alfred-py: A deep learning utility library for **human**

Alfred Alfred is command line tool for deep-learning usage. if you want split an video into image frames or combine frames into a single video, then a

JinTian 800 Jan 03, 2023