Learning Sparse Neural Networks through L0 regularization

Last update: Nov 10, 2022

Related tags

Overview

Example implementation of the L0 regularization method described at

Learning Sparse Neural Networks through L0 regularization, Christos Louizos, Max Welling & Diederik P. Kingma, https://openreview.net/pdf?id=BkdI3hgRZ

This code is provided as is and is not maintained / updated.

Owner

AMLAB

GitHub Repository

A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision

🤗 Accelerate was created for PyTorch users who like to write the training loop of PyTorch models but are reluctant to write and maintain the boilerplate code needed to use multi-GPUs/TPU/fp16.

3.5k Jan 08, 2023

OptNet: Differentiable Optimization as a Layer in Neural Networks

OptNet: Differentiable Optimization as a Layer in Neural Networks This repository is by Brandon Amos and J. Zico Kolter and contains the PyTorch sourc

428 Dec 24, 2022

The easiest way to use deep metric learning in your application. Modular, flexible, and extensible. Written in PyTorch.

News March 3: v0.9.97 has various bug fixes and improvements: Bug fixes for NTXentLoss Efficiency improvement for AccuracyCalculator, by using torch i

5k Jan 02, 2023

TorchShard is a lightweight engine for slicing a PyTorch tensor into parallel shards

TorchShard is a lightweight engine for slicing a PyTorch tensor into parallel shards. It can reduce GPU memory and scale up the training when the model has massive linear layers (e.g., ViT, BERT and

275 Nov 22, 2022

Pretrained ConvNets for pytorch: NASNet, ResNeXt, ResNet, InceptionV4, InceptionResnetV2, Xception, DPN, etc.

Pretrained models for Pytorch (Work in progress) The goal of this repo is: to help to reproduce research papers results (transfer learning setups for

8.7k Dec 31, 2022

This is an differentiable pytorch implementation of SIFT patch descriptor.

This is an differentiable pytorch implementation of SIFT patch descriptor. It is very slow for describing one patch, but quite fast for batch. It can

150 Dec 24, 2022

A Pytorch Implementation for Compact Bilinear Pooling.

CompactBilinearPooling-Pytorch A Pytorch Implementation for Compact Bilinear Pooling. Adapted from tensorflow_compact_bilinear_pooling Prerequisites I

169 Dec 23, 2022

A PyTorch repo for data loading and utilities to be shared by the PyTorch domain libraries.

878 Dec 30, 2022

A Closer Look at Structured Pruning for Neural Network Compression

A Closer Look at Structured Pruning for Neural Network Compression Code used to reproduce experiments in https://arxiv.org/abs/1810.04622. To prune, w

140 Dec 05, 2022

Over9000 optimizer

Optimizers and tests Every result is avg of 20 runs. Dataset LR Schedule Imagenette size 128, 5 epoch Imagewoof size 128, 5 epoch Adam - baseline OneC

405 Nov 27, 2022

A tiny package to compare two neural networks in PyTorch

Compare neural networks by their feature similarity

180 Dec 30, 2022

higher is a pytorch library allowing users to obtain higher order gradients over losses spanning training loops rather than individual training steps.

higher is a library providing support for higher-order optimization, e.g. through unrolled first-order optimization loops, of "meta" aspects of these

1.5k Jan 03, 2023

Learning Sparse Neural Networks through L0 regularization

Related tags

Overview

Owner

AMLAB

A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision

OptNet: Differentiable Optimization as a Layer in Neural Networks

The easiest way to use deep metric learning in your application. Modular, flexible, and extensible. Written in PyTorch.

TorchShard is a lightweight engine for slicing a PyTorch tensor into parallel shards

Pretrained ConvNets for pytorch: NASNet, ResNeXt, ResNet, InceptionV4, InceptionResnetV2, Xception, DPN, etc.

This is an differentiable pytorch implementation of SIFT patch descriptor.

A Pytorch Implementation for Compact Bilinear Pooling.

A PyTorch repo for data loading and utilities to be shared by the PyTorch domain libraries.

A Closer Look at Structured Pruning for Neural Network Compression

Over9000 optimizer

A tiny package to compare two neural networks in PyTorch

higher is a pytorch library allowing users to obtain higher order gradients over losses spanning training loops rather than individual training steps.

A pure Python implementation of Compact Bilinear Pooling and Count Sketch for PyTorch.

3D-RETR: End-to-End Single and Multi-View3D Reconstruction with Transformers

torch-optimizer -- collection of optimizers for Pytorch

PyTorch to TensorFlow Lite converter

Model summary in PyTorch similar to `model.summary()` in Keras

Differentiable ODE solvers with full GPU support and O(1)-memory backpropagation.

Distiller is an open-source Python package for neural network compression research.

Implementation of LambdaNetworks, a new approach to image recognition that reaches SOTA with less compute