Repo for my Tensorflow/Keras CV experiments. Mostly revolving around the Danbooru20xx dataset

Last update: Dec 27, 2022

Related tags

Deep Learning SW-CV-ModelZoo

Overview

SW-CV-ModelZoo

Repo for my Tensorflow/Keras CV experiments. Mostly revolving around the Danbooru20xx dataset

Framework: TF/Keras 2.7

Training SQLite DB built using fire-egg's tools: https://github.com/fire-eggs/Danbooru2019

Currently training on Danbooru2021, 512px SFW subset (sans the rating:q images that had been included in the 2022-01-21 release of the dataset)

Reference:

Anonymous, The Danbooru Community, & Gwern Branwen; “Danbooru2021: A Large-Scale Crowdsourced and Tagged Anime Illustration Dataset”, 2022-01-21. Web. Accessed 2022-01-28 https://www.gwern.net/Danbooru2021

Journal

06/02/2022: great news crew! TRC allowed me to use a bunch of TPUs!

To make better use of this amount of compute I had to overhaul a number of components, so a bunch of things are likely to have fallen to bitrot in the process. I can only guarantee NFNet can work pretty much as before with the right arguments.
NFResNet changes should have left it retrocompatible with the previous version.
ResNet has been streamlined to be mostly in line with the Bag-of-Tricks paper (arXiv:1812.01187) with the exception of the stem. It is not compatible with the previous version of the code.

The training labels have been included in the 2021_0000_0899 folder for convenience.
The list of files used for training is going to be uploaded as a GitHub Release.

Now for some numbers:
compared to my previous best run, the one that resulted in NFNetL1V1-100-0.57141:

I'm using 1.86x the amount of images: 2.8M vs 1.5M
I'm training bigger models: 61M vs 45M params
... in less time: 232 vs 700 hours of processor time
don't get me started on actual wall clock time
with a few amenities thrown in: ECA for channel attention, SiLU activation

And it's all thanks to the folks at TRC, so shout out to them!

I currently have a few runs in progress across a couple of dimensions:

effect of model size with NFNet L0/L1/L2, with SiLU and ECA for all three of them
effect of activation function with NFNet L0, with SiLU/HSwish/ReLU, no ECA

Once the experiments are over, the plan is to select the network definitions that lay on the Pareto curve between throughput and F1 score and release the trained weights.

One last thing.
I'd like to call your attention to the tools/cleanlab_stuff.py script.
It reads two files: one with the binarized labels from the database, the other with the predicted probabilities.
It then uses the cleanlab package to estimate whether if an image in a set could be missing a given label. At the end it stores its conclusions in a json file.
This file could, potentially, be used in some tool to assist human intervention to add the missing tags.

You might also like...

Human head pose estimation using Keras over TensorFlow.

RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild.

71 Jan 5, 2023

Graph Neural Networks with Keras and Tensorflow 2.

Welcome to Spektral Spektral is a Python library for graph deep learning, based on the Keras API and TensorFlow 2. The main goal of this project is to

2.2k Jan 8, 2023

QKeras: a quantization deep learning library for Tensorflow Keras

QKeras github.com/google/qkeras QKeras 0.8 highlights: Automatic quantization using QKeras; Stochastic behavior (including stochastic rouding) is disa

437 Jan 3, 2023

Hyperparameter Optimization for TensorFlow, Keras and PyTorch

Hyperparameter Optimization for Keras Talos • Key Features • Examples • Install • Support • Docs • Issues • License • Download Talos radically changes

1.6k Dec 15, 2022

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.

MMdnn MMdnn is a comprehensive and cross-framework tool to convert, visualize and diagnose deep learning (DL) models. The "MM" stands for model manage

5.7k Jan 9, 2023

Deep GPs built on top of TensorFlow/Keras and GPflow

GPflux Documentation | Tutorials | API reference | Slack What does GPflux do? GPflux is a toolbox dedicated to Deep Gaussian processes (DGP), the hier

107 Nov 2, 2022

tf2onnx - Convert TensorFlow, Keras and Tflite models to ONNX.

tf2onnx converts TensorFlow (tf-1.x or tf-2.x), tf.keras and tflite models to ONNX via command line or python api.

1.8k Jan 8, 2023

Build tensorflow keras model pipelines in a single line of code. Created by Ram Seshadri. Collaborators welcome. Permission granted upon request.

deep_autoviml Build keras pipelines and models in a single line of code! Table of Contents Motivation How it works Technology Install Usage API Image

102 Dec 17, 2022

Advanced Deep Learning with TensorFlow 2 and Keras (Updated for 2nd Edition)

1.5k Jan 3, 2023

Releases(models_db2021_5500_2022_10_21)

models_db2021_5500_2022_10_21(Oct 21, 2022)

ConvNext B, ViT B16
Trained on Danbooru2021 512px SFW subset, modulos 0000-0899
top 5500 tags (2021_0000_0899_5500/selected_tags.csv)
alpha to white
padding to make the image square is white
channel order is BGR, input is 0...255, scaled to -1...1 within the model

| run_name | definition_name | params_human | image_size | thres | F1 | |:---------------------------------|:------------------|:---------------|-------------:|--------:|-------:| | ConvNextBV1_09_25_2022_05h13m55s | B | 93.2M | 448 | 0.3673 | 0.6941 | | ViTB16_09_25_2022_04h53m38s | B16 | 90.5M | 448 | 0.3663 | 0.6918 |
Source code(tar.gz)
Source code(zip)
ConvNextBV1_09_25_2022_05h13m55s.7z(322.58 MB)
ViTB16_09_25_2022_04h53m38s.7z(312.96 MB)
convnexts_db2021_2022_03_22(Mar 22, 2022)

ConvNext, T/S/B
Trained on Danbooru2021 512px SFW subset, modulos 0000-0899
alpha to white
padding to make the image square is white
channel order is BGR, input is 0...255, scaled to -1...1 within the model

| run_name | definition_name | params_human | image_size | thres | F1 | |:---------------------------------|:------------------|:---------------|-------------:|--------:|-------:| | ConvNextBV1_03_10_2022_21h41m23s | B | 90.01M | 448 | 0.3372 | 0.6892 | | ConvNextSV1_03_11_2022_17h49m56s | S | 51.28M | 384 | 0.3301 | 0.6798 | | ConvNextTV1_03_05_2022_15h56m42s | T | 29.65M | 320 | 0.3259 | 0.6595 |
Source code(tar.gz)
Source code(zip)
ConvNextBV1_03_10_2022_21h41m23s.7z(311.29 MB)
ConvNextSV1_03_11_2022_17h49m56s.7z(177.36 MB)
ConvNextTV1_03_05_2022_15h56m42s.7z(102.96 MB)
nfnets_db2021_2022_03_04(Mar 4, 2022)

NFNet, L0/L1/L2 (based on timm Lx model definitions) Trained on Danbooru2021 512px SFW subset, modulos 0000-0899 alpha to white padding to make the image square is white channel order is BGR, input is 0...255, scaled to -1...1 within the model

| run_name | definition_name | params_human | image_size | thres | F1 | |:---------------------------------|:------------------|:---------------|-------------:|--------:|-------:| | NFNetL2V1_02_20_2022_10h27m08s | L2 | 60.96M | 448 | 0.3231 | 0.6785 | | NFNetL1V1_02_17_2022_20h18m38s | L1 | 45.65M | 384 | 0.3259 | 0.6691 | | NFNetL0V1_02_10_2022_17h50m14s | L0 | 27.32M | 320 | 0.3190 | 0.6509 |
Source code(tar.gz)
Source code(zip)
NFNetL0V1_02_10_2022_17h50m14s.7z(94.98 MB)
NFNetL1V1_02_17_2022_20h18m38s.7z(157.97 MB)
NFNetL2V1_02_20_2022_10h27m08s.7z(210.49 MB)
nfnet_tpu_training(Feb 6, 2022)

Attaching the lists of files currently in use for training and validation, along with the selected labels.

Training and validation files have been selected from the 512px SFW subset.
Source code(tar.gz)
Source code(zip)
2021_0000_0899.7z(1.32 MB)
NFNetL1V1-100-0.57141(Dec 31, 2021)
NFNet, L1 (based on timm Lx model definitions), 100 epochs, F1 @ 0.4 at the end of the 100th epoch was 0.57141

Trained on Danbooru2020 512px SFW subset, modulos 0000-0599

320px per side

alpha to white

padding to make the image square is white

channel order is BGR, scaled to 0-1

mixup alpha = 0.2 during epochs 76-100

analyze_metrics on Danbooru2020 original set, modulos 0984-0999: {'thres': 0.3485, 'F1': 0.6133, 'F2': 0.6133, 'MCC': 0.6094, 'A': 0.9923, 'R': 0.6133, 'P': 0.6133}

analyze_metrics on image IDs 4970000-5000000: {'thres': 0.3148, 'F1': 0.5942, 'F2': 0.5941, 'MCC': 0.5892, 'A': 0.9901, 'R': 0.5940, 'P': 0.5943}

Source code(tar.gz)
Source code(zip)
NFNetL1V1-100-0.57141.7z(158.09 MB)

Owner

GitHub Repository

Code and data of the EMNLP 2021 paper "Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer"

StyleAttack Code and data of the EMNLP 2021 paper "Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer" Prepare Pois

19 Nov 20, 2022

Numerical-computing-is-fun - Learning numerical computing with notebooks for all ages.

As much as this series is to educate aspiring computer programmers and data scientists of all ages and all backgrounds, it is also a reminder to mysel

758 Dec 25, 2022

Code for the submitted paper Surrogate-based cross-correlation for particle image velocimetry

Surrogate-based cross-correlation (SBCC) This repository contains code for the submitted paper Surrogate-based cross-correlation for particle image ve

5 Jun 30, 2022

An API-first distributed deployment system of deep learning models using timeseries data to analyze and predict systems behaviour

Gordo Building thousands of models with timeseries data to monitor systems. Table of content About Examples Install Uninstall Developer manual How to

26 Dec 27, 2022

[CVPR 2021] NormalFusion: Real-Time Acquisition of Surface Normals for High-Resolution RGB-D Scanning

NormalFusion: Real-Time Acquisition of Surface Normals for High-Resolution RGB-D Scanning Project Page | Paper | Supplemental material #1 | Supplement

49 Nov 24, 2022

Official implementation of the paper 'High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Network' in CVPR 2021

LPTN Paper | Supplementary Material | Poster High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Network Ji

372 Dec 26, 2022

PyTorch code accompanying the paper "Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learning" (NeurIPS 2021).

HIGL This is a PyTorch implementation for our paper: Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learning (NeurIPS 2021). Our cod

20 Dec 14, 2022

Repo for my Tensorflow/Keras CV experiments. Mostly revolving around the Danbooru20xx dataset

Related tags

Overview

SW-CV-ModelZoo

Reference:

Journal

You might also like...

Human head pose estimation using Keras over TensorFlow.

Graph Neural Networks with Keras and Tensorflow 2.

QKeras: a quantization deep learning library for Tensorflow Keras

Hyperparameter Optimization for TensorFlow, Keras and PyTorch

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.

Deep GPs built on top of TensorFlow/Keras and GPflow

tf2onnx - Convert TensorFlow, Keras and Tflite models to ONNX.

Build tensorflow keras model pipelines in a single line of code. Created by Ram Seshadri. Collaborators welcome. Permission granted upon request.

Advanced Deep Learning with TensorFlow 2 and Keras (Updated for 2nd Edition)

Releases(models_db2021_5500_2022_10_21)

models_db2021_5500_2022_10_21(Oct 21, 2022)

convnexts_db2021_2022_03_22(Mar 22, 2022)

nfnets_db2021_2022_03_04(Mar 4, 2022)

nfnet_tpu_training(Feb 6, 2022)

NFNetL1V1-100-0.57141(Dec 31, 2021)

Owner

Code and data of the EMNLP 2021 paper "Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer"

Numerical-computing-is-fun - Learning numerical computing with notebooks for all ages.

Code for the submitted paper Surrogate-based cross-correlation for particle image velocimetry

An API-first distributed deployment system of deep learning models using timeseries data to analyze and predict systems behaviour

[CVPR 2021] NormalFusion: Real-Time Acquisition of Surface Normals for High-Resolution RGB-D Scanning

Official implementation of the paper 'High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Network' in CVPR 2021

PyTorch code accompanying the paper "Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learning" (NeurIPS 2021).

Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning (ICLR 2021)

dyld_shared_cache processing / Single-Image loading for BinaryNinja

IEEE-CIS Technical Challenge on Predict+Optimize for Renewable Energy Scheduling

Codes for the paper Contrast and Mix: Temporal Contrastive Video Domain Adaptation with Background Mixing

Official implementation of Rethinking Graph Neural Architecture Search from Message-passing (CVPR2021)

SOTA model in CIFAR10

4K videos with annotated masks in our ICCV2021 paper 'Internal Video Inpainting by Implicit Long-range Propagation'.

Rotation-Only Bundle Adjustment

Pathdreamer: A World Model for Indoor Navigation

Simple reimplemetation experiments about FcaNet

Implementation of our paper 'RESA: Recurrent Feature-Shift Aggregator for Lane Detection' in AAAI2021.

Deep Learning Tutorial for Kaggle Ultrasound Nerve Segmentation competition, using Keras

This is the official implementation for "Do Transformers Really Perform Bad for Graph Representation?".