CLIP: Connecting Text and Image (Learning Transferable Visual Models From Natural Language Supervision)

Last update: Jan 07, 2023

Overview

CLIP (Contrastive Language–Image Pre-training)

Experiments (Evaluation)

Model	Dataset	Acc (%)
ViT-B/32 (Paper)	CIFAR100	65.1
ViT-B/32 (Our)	CIFAR100	61.71
ViT-B/32 (Paper	CIFAR10	91.3
ViT-B/32 (Our)	CIFAR10	88.8

Overview

Training

Work In Process

Usage

Evaluation

python evaluation.py --dataset CIFAR100 --cuda True

args
- dataset (str): CIFAR10, CIFAR100 (default: CIFAR100)
- num_workers (int): default: 0
- batch_size (int): default: 128
- cuda (bool): False
Training
- Prepare Data
  - Visual Genome Dataset link
  - Download (images, region descriptions)
- training
```
python main.py --base_dir ./ --cuda True
```

Reference

paper link
Author: Alec Radford, Jong Wook Kim, Chris Hallacy, Girish Sastry, Amanda Askell, Pamela Mishkin, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Jack Clark, Gretchen Krueger, Ilya Sutskever
OpenAI

Owner

Myeongjun Kim

Computer Vision Research using Deep Learning

GitHub Repository

Accepted at ICCV-2021: Workshop on Computer Vision for Automated Medical Diagnosis (CVAMD)

Is it Time to Replace CNNs with Transformers for Medical Images? Accepted at ICCV-2021: Workshop on Computer Vision for Automated Medical Diagnosis (C

80 Dec 27, 2022

PyTorch implementation for "HyperSPNs: Compact and Expressive Probabilistic Circuits", NeurIPS 2021

HyperSPN This repository contains code for the paper: HyperSPNs: Compact and Expressive Probabilistic Circuits "HyperSPNs: Compact and Expressive Prob

8 Nov 08, 2022

Breast Cancer Classification Model is applied on a different dataset

1 Feb 04, 2022

Neural Koopman Lyapunov Control

Neural-Koopman-Lyapunov-Control Code for our paper: Neural Koopman Lyapunov Control Requirements dReal4: v4.19.02.1 PyTorch: 1.2.0 The learning framew

6 Dec 24, 2022

Large scale PTM - PPI relation extraction

Large-scale protein-protein post-translational modification extraction with distant supervision and confidence calibrated BioBERT The silver standard

1 Feb 25, 2022

Tensorflow implementation of Character-Aware Neural Language Models.

Character-Aware Neural Language Models Tensorflow implementation of Character-Aware Neural Language Models. The original code of author can be found h

751 Dec 26, 2022

An unofficial personal implementation of UM-Adapt, specifically to tackle joint estimation of panoptic segmentation and depth prediction for autonomous driving datasets.

Semisupervised Multitask Learning This repository is an unofficial and slightly modified implementation of UM-Adapt[1] using PyTorch. This code primar

11 Nov 25, 2022

Unified unsupervised and semi-supervised domain adaptation network for cross-scenario face anti-spoofing, Pattern Recognition

USDAN The implementation of Unified unsupervised and semi-supervised domain adaptation network for cross-scenario face anti-spoofing, which is accepte

11 Nov 03, 2022

Bottom-up Human Pose Estimation

Introduction This is the official code of Rethinking the Heatmap Regression for Bottom-up Human Pose Estimation. This paper has been accepted to CVPR2

108 Dec 01, 2022

This repository is for DSA and CP scripts for reference.

dsa-script-collections This Repo is the collection of DSA and CP scripts for reference. Contents Python Bubble Sort Insertion Sort Merge Sort Quick So

9 Nov 22, 2022

Solution of Kaggle competition: Sartorius - Cell Instance Segmentation

Sartorius - Cell Instance Segmentation https://www.kaggle.com/c/sartorius-cell-instance-segmentation Environment setup Build docker image bash .dev_sc

68 Dec 09, 2022

Pre-trained BERT Models for Ancient and Medieval Greek, and associated code for LaTeCH 2021 paper titled - "A Pilot Study for BERT Language Modelling and Morphological Analysis for Ancient and Medieval Greek"

Ancient Greek BERT The first and only available Ancient Greek sub-word BERT model! State-of-the-art post fine-tuning on Part-of-Speech Tagging and Mor

22 Dec 08, 2022

Image-to-image regression with uncertainty quantification in PyTorch

Image-to-image regression with uncertainty quantification in PyTorch. Take any dataset and train a model to regress images to images with rigorous, distribution-free uncertainty quantification.

25 Dec 26, 2022

Python PID Tuner - Based on a FOPDT model obtained using a Open Loop Process Reaction Curve

PythonPID_Tuner Step 1: Takes a Process Reaction Curve in csv format - assumes data at 100ms interval (column names CV and PV) Step 2: Makes a rough e

6 Jan 14, 2022

The official implementation of CVPR 2021 Paper: Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation.

Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation This repository is the official implementation of CVPR 2021 paper:

9 Nov 14, 2022

[CVPR2021] The source code for our paper 《Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learning》.

TBE The source code for our paper "Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Le

150 Dec 28, 2022

CLIP: Connecting Text and Image (Learning Transferable Visual Models From Natural Language Supervision)

Related tags

Overview

CLIP (Contrastive Language–Image Pre-training)

Experiments (Evaluation)

Overview

Training

Usage

Reference

Owner

Myeongjun Kim

Accepted at ICCV-2021: Workshop on Computer Vision for Automated Medical Diagnosis (CVAMD)

PyTorch implementation for "HyperSPNs: Compact and Expressive Probabilistic Circuits", NeurIPS 2021

Breast Cancer Classification Model is applied on a different dataset

Neural Koopman Lyapunov Control

Large scale PTM - PPI relation extraction

Tensorflow implementation of Character-Aware Neural Language Models.

An unofficial personal implementation of UM-Adapt, specifically to tackle joint estimation of panoptic segmentation and depth prediction for autonomous driving datasets.

Unified unsupervised and semi-supervised domain adaptation network for cross-scenario face anti-spoofing, Pattern Recognition

Bottom-up Human Pose Estimation

This repository is for DSA and CP scripts for reference.

Solution of Kaggle competition: Sartorius - Cell Instance Segmentation

Pre-trained BERT Models for Ancient and Medieval Greek, and associated code for LaTeCH 2021 paper titled - "A Pilot Study for BERT Language Modelling and Morphological Analysis for Ancient and Medieval Greek"

Image-to-image regression with uncertainty quantification in PyTorch

Python PID Tuner - Based on a FOPDT model obtained using a Open Loop Process Reaction Curve

The official implementation of CVPR 2021 Paper: Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation.

Tensorflow 2 implementation of our high quality frame interpolation neural network

Pytorch implementation of CoCon: A Self-Supervised Approach for Controlled Text Generation

Structured Data Gradient Pruning (SDGP)

The MATH Dataset

[CVPR2021] The source code for our paper 《Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learning》.