Self-Supervised Learning

Last update: Dec 14, 2021

Overview

Self-Supervised Learning

Features

self_supervised offers features like

modular framework
support for multi-gpu training using PyTorch Lightning
easy to use and written in a PyTorch like style
supports custom backbone models for self-supervised pre-training

Supported Models

Supported Loss Function

Benchmarks

Currently implemented models and their accuracy on cifar10 and imagenette.

ImageNette

Model	Epochs	Batch Size	Test Accuracy
MoCo	800	256	0.827
SimCLR	800	256	0.847
SimSiam	800	256	0.827
BarlowTwins	800	256	0.801
BYOL	800	256	0.851

Cifar10

Model	Epochs	Batch Size	Test Accuracy
MoCo	200	128	0.83
SimCLR	200	128	0.78
SimSiam	200	128	0.73
BarlowTwins	200	128	0.84
BYOL	200	128	0.85
MoCo	200	512	0.85
SimCLR	200	512	0.83
SimSiam	200	512	0.81
BarlowTwins	200	512	0.78
BYOL	200	512	0.84
MoCo	800	128	0.89
SimCLR	800	128	0.87
SimSiam	800	128	0.80
MoCo	800	512	0.90
SimCLR	800	512	0.89
SimSiam	800	512	0.91

Tutorials

Want to jump to the tutorials and see lightly in action?

Self-Supervised Learning

Related tags

Overview

Self-Supervised Learning

Features

Supported Models

Supported Loss Function

Benchmarks

ImageNette

Cifar10

Tutorials

Further Reading

Owner

Robin

Stock-history-display - something like a easy yearly review for your stock performance

Code for Parameter Prediction for Unseen Deep Architectures (NeurIPS 2021)

🔮 Execution time predictions for deep neural network training iterations across different GPUs.

A TensorFlow implementation of Neural Program Synthesis from Diverse Demonstration Videos

DCGAN LSGAN WGAN-GP DRAGAN PyTorch

Official implementation for "Symbolic Learning to Optimize: Towards Interpretability and Scalability"

SpiroMask: Measuring Lung Function Using Consumer-Grade Masks

A PyTorch implementation of EventProp [https://arxiv.org/abs/2009.08378], a method to train Spiking Neural Networks

Rocket-recycling with Reinforcement Learning

code for "AttentiveNAS Improving Neural Architecture Search via Attentive Sampling"

This is a yolo3 implemented via tensorflow 2.7

Code for Boundary-Aware Segmentation Network for Mobile and Web Applications

Extending JAX with custom C++ and CUDA code

sense-py-AnishaBaishya created by GitHub Classroom

NeuTex: Neural Texture Mapping for Volumetric Neural Rendering

Pytorch implementation of face attention network

2021 Artificial Intelligence Diabetes Datathon

Detecting Blurred Ground-based Sky/Cloud Images

Official code for the CVPR 2021 paper "How Well Do Self-Supervised Models Transfer?"

CyTran: Cycle-Consistent Transformers for Non-Contrast to Contrast CT Translation