Implementation of TransGanFormer, an all-attention GAN that combines the finding from the recent GanFormer and TransGan paper

Last update: Dec 06, 2022

Overview

TransGanFormer (wip)

Implementation of TransGanFormer, an all-attention GAN that combines the finding from the recent GansFormer and TransGan paper. It will also contain a bunch of tricks I have picked up building transformers and GANs for the last year or so, including efficient linear attention and pixel level attention.

Install

$ pip install transganformer

Usage

$ transganformer --data ./path/to/data

Citations

@misc{jiang2021transgan,
    title   = {TransGAN: Two Transformers Can Make One Strong GAN}, 
    author  = {Yifan Jiang and Shiyu Chang and Zhangyang Wang},
    year    = {2021},
    eprint  = {2102.07074},
    archivePrefix = {arXiv},
    primaryClass = {cs.CV}
}

@misc{hudson2021generative,
    title   = {Generative Adversarial Transformers}, 
    author  = {Drew A. Hudson and C. Lawrence Zitnick},
    year    = {2021},
    eprint  = {2103.01209},
    archivePrefix = {arXiv},
    primaryClass = {cs.CV}
}

You might also like...

Finding all things on-prem Microsoft for password spraying and enumeration.

msprobe About Installing Usage Examples Coming Soon Acknowledgements About Finding all things on-prem Microsoft for password spraying and enumeration.

205 Jan 9, 2023

Implementation of the 😇 Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient Visual Backbones

HaloNet - Pytorch Implementation of the Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient Visual Backbones. This re

189 Nov 22, 2022

Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"

Memory Efficient Attention Pytorch Implementation of a memory efficient multi-head attention as proposed in the paper, Self-attention Does Not Need O(

180 Jan 5, 2023

Implementation of Deformable Attention in Pytorch from the paper "Vision Transformer with Deformable Attention"

Deformable Attention Implementation of Deformable Attention from this paper in Pytorch, which appears to be an improvement to what was proposed in DET

128 Dec 24, 2022

A curated (most recent) list of resources for Learning with Noisy Labels

321 Jan 9, 2023

The official implementation of NeurIPS 2021 paper: Finding Optimal Tangent Points for Reducing Distortions of Hard-label Attacks

11 Nov 27, 2022

The official pytorch implementation of our paper "Is Space-Time Attention All You Need for Video Understanding?"

TimeSformer This is an official pytorch implementation of Is Space-Time Attention All You Need for Video Understanding?. In this repository, we provid

1k Dec 31, 2022

Unofficial PyTorch implementation of Fastformer based on paper "Fastformer: Additive Attention Can Be All You Need"."

Fastformer-PyTorch Unofficial PyTorch implementation of Fastformer based on paper Fastformer: Additive Attention Can Be All You Need. Usage : import t

126 Dec 6, 2022

Unofficial Tensorflow-Keras implementation of Fastformer based on paper [Fastformer: Additive Attention Can Be All You Need](https://arxiv.org/abs/2108.09084).

Fastformer-Keras Unofficial Tensorflow-Keras implementation of Fastformer based on paper Fastformer: Additive Attention Can Be All You Need. Tensorflo

10 Jan 30, 2022

Comments

kmeans_iters in attention

Hello first at all thank you for taking the time and port all of theses models to pytorch, you are really helping out people like me who have tensorflow as a second language :D

in line 689 of https://github.com/dorarad/gansformer/blob/main/training/network.py there is a loop over kmeans-iter in the transformerlayer, but i can t find this loop in your implementation of Attention, is it implicitly implemented with these einops - i am sadly not fluent with einops or tensorflow :/

regards martin

opened by martinpflaum 0
dual_contrast_loss argument - error while training

When trying to train a model, the dual_contrast_loss argument spits up an error on lightweight-gan.py. My guess is that it's not implemented yet.

At first I thought it was a mismatch (should it be dual_contrastive_loss btw like in lightweight-gan?) but apparently needed to be commented out for training to start successfully.

Very exciting by the way! Training seems to be super fast, about 28h for 150,000 iterations for a 256p model on my 2080 (although it's a very small dataset). I'll share some closer-to-final results once I have them.

Best, Theodore.

opened by TheodoreGalanos 2

Releases(0.0.17)

0.0.17(Apr 27, 2021)

Source code(tar.gz)
Source code(zip)
0.0.16(Apr 15, 2021)

Source code(tar.gz)
Source code(zip)
0.0.15(Apr 13, 2021)

Source code(tar.gz)
Source code(zip)
0.0.14(Apr 13, 2021)

Source code(tar.gz)
Source code(zip)
0.0.12(Apr 12, 2021)

Source code(tar.gz)
Source code(zip)
0.0.11(Apr 12, 2021)

Source code(tar.gz)
Source code(zip)
0.0.10(Apr 11, 2021)

Source code(tar.gz)
Source code(zip)
0.0.9(Apr 11, 2021)

Source code(tar.gz)
Source code(zip)
0.0.8(Apr 11, 2021)

Source code(tar.gz)
Source code(zip)
0.0.7(Apr 9, 2021)

Source code(tar.gz)
Source code(zip)
0.0.6(Apr 9, 2021)

Source code(tar.gz)
Source code(zip)
0.0.5(Apr 9, 2021)

Source code(tar.gz)
Source code(zip)
0.0.4(Apr 7, 2021)

Source code(tar.gz)
Source code(zip)
0.0.3(Apr 7, 2021)

Source code(tar.gz)
Source code(zip)
0.0.2(Apr 5, 2021)

Source code(tar.gz)
Source code(zip)
0.0.1(Apr 5, 2021)

Source code(tar.gz)
Source code(zip)

Owner

Phil Wang

Working with Attention. It's all we need.

GitHub Repository

TensorFlow implementation of the paper "Hierarchical Attention Networks for Document Classification"

Hierarchical Attention Networks for Document Classification This is an implementation of the paper Hierarchical Attention Networks for Document Classi

83 Dec 05, 2022

Learn the Deep Learning for Computer Vision in three steps: theory from base to SotA, code in PyTorch, and space-repetition with Anki

DeepCourse: Deep Learning for Computer Vision arthurdouillard.com/deepcourse/ This is a course I'm giving to the French engineering school EPITA each

113 Nov 29, 2022

Implementation of TransGanFormer, an all-attention GAN that combines the finding from the recent GanFormer and TransGan paper

Related tags

Overview

TransGanFormer (wip)

Install

Usage

Citations

You might also like...

Finding all things on-prem Microsoft for password spraying and enumeration.

Implementation of the 😇 Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient Visual Backbones

Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"

Implementation of Deformable Attention in Pytorch from the paper "Vision Transformer with Deformable Attention"

A curated (most recent) list of resources for Learning with Noisy Labels

The official implementation of NeurIPS 2021 paper: Finding Optimal Tangent Points for Reducing Distortions of Hard-label Attacks

The official pytorch implementation of our paper "Is Space-Time Attention All You Need for Video Understanding?"

Unofficial PyTorch implementation of Fastformer based on paper "Fastformer: Additive Attention Can Be All You Need"."

Unofficial Tensorflow-Keras implementation of Fastformer based on paper [Fastformer: Additive Attention Can Be All You Need](https://arxiv.org/abs/2108.09084).

Comments

kmeans_iters in attention

dual_contrast_loss argument - error while training

Releases(0.0.17)

0.0.17(Apr 27, 2021)

0.0.16(Apr 15, 2021)

0.0.15(Apr 13, 2021)

0.0.14(Apr 13, 2021)

0.0.12(Apr 12, 2021)

0.0.11(Apr 12, 2021)

0.0.10(Apr 11, 2021)

0.0.9(Apr 11, 2021)

0.0.8(Apr 11, 2021)

0.0.7(Apr 9, 2021)

0.0.6(Apr 9, 2021)

0.0.5(Apr 9, 2021)

0.0.4(Apr 7, 2021)

0.0.3(Apr 7, 2021)

0.0.2(Apr 5, 2021)

0.0.1(Apr 5, 2021)

Owner

Phil Wang

TensorFlow implementation of the paper "Hierarchical Attention Networks for Document Classification"

Learn the Deep Learning for Computer Vision in three steps: theory from base to SotA, code in PyTorch, and space-repetition with Anki

Colab notebook and additional materials for Python-driven analysis of redlining data in Philadelphia

torchsummaryDynamic: support real FLOPs calculation of dynamic network or user-custom PyTorch ops

Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers [CVPR 2021]

WRENCH: Weak supeRvision bENCHmark

Decoding the Protein-ligand Interactions Using Parallel Graph Neural Networks

Zero-shot Synthesis with Group-Supervised Learning (ICLR 2021 paper)

The official implementation of the research paper "DAG Amendment for Inverse Control of Parametric Shapes"

This is the repository for our paper Ditch the Gold Standard: Re-evaluating Conversational Question Answering

Two-Stage Peer-Regularized Feature Recombination for Arbitrary Image Style Transfer

Everything you need to know about NumPy( Creating Arrays, Indexing, Math,Statistics,Reshaping).

Anomaly detection related books, papers, videos, and toolboxes

Convert human motion from video to .bvh

GAN Image Generator and Characterwise Image Recognizer with python

Syntax-Aware Action Targeting for Video Captioning

Generate pixel-style avatars with python.

Code, environments, and scripts for the paper: "How Private Is Your RL Policy? An Inverse RL Based Analysis Framework"

This repository gives an example on how to preprocess the data of the HECKTOR challenge

This repository collects 100 papers related to negative sampling methods.