Contrastive Language-Audio Pretraining

Overview

CLAP

Contrastive Language-Audio Pretraining

In due time this repo will be full of lovely things, I hope.

Feel free to check out the Issues if you're interested in contributing. Leave a note saying what interests you. :)

from jax import random, numpy as np
from clap.clap import CLAP

key1, key2 = random.split(random.PRNGKey(0), 2)

text = random.randint(key1, (2, 16,), 0, 256)
audio = random.uniform(key1, (2, 8, 512))

text_mask = np.ones((2, 16), dtype = bool)
audio_mask = np.ones((2, 8), dtype = bool)

model = CLAP(
    text_vocab = 256,
    text_dim = 512,
    text_depth = 6,
    text_heads = 8,
    audio_dim = 512,
    audio_depth = 6,
    audio_heads = 8
)

params = model.init(key2, text, audio, text_mask, audio_mask)
loss = model.apply(params, text, audio, text_mask, audio_mask)

# after a lot of training

sim = model.apply(params, text, audio, text_mask, audio_mask, return_loss = False) # (2, 2)
Owner
Charles Foster
Charles Foster
Say "good morning" on Discord, in batch, one-click.

🌞 gm Good Morning! Usage Simply copy the channel_list to gm.py and fill authorization_list with authorization token(s). Enjoy. Authorization Please r

e 3 Nov 18, 2022
A decentralized messaging daemon built on top of the Kademlia routing protocol.

parakeet-message A decentralized messaging daemon built on top of the Kademlia routing protocol. Now that you are done laughing... pictures what is it

Jonathan Abbott 3 Apr 23, 2022
Code done for/during the course

Serverless Course Autumn 2021 - Code This repository contains a set of examples developed during, but not limited to the live coding sessions. Lesson

Alexandru Burlacu 4 Dec 21, 2021
Ap lokit lokit

🎵 FANDA MUSIC BOT Fanda Music adalah proyek bot telegram yang memungkinkan Anda memutar musik di obrolan suara grup telegram. a href="https://www.py

Fatur 2 Nov 18, 2021
An integrated information collection tool

infoscaner 环境配置 目前infoscaner仅支持在linux上运行,建议运行在最新版本的kali中 infoscaner是基于python3版本实现的,运行之前首先安装python库 如果同时存在python2和python3,请输入以下命令 pip3 install -r requi

CMACCKK 74 Sep 13, 2021
A fully responsive interface to manage all your favorite software on your HTPC.

Python 3 port of Hellowlol's HTPC Manager fork We made this an organization repository to be more independent from single developers. If you want to j

26 Jan 04, 2023
Reverse engineering multi-device WhatsApp Web.

whatsapp-web-multi-device-reveng In this repository, the research for reverse engineering multi-device WhatsApp Web takes place, see here for a descri

84 Jan 01, 2023
A Serverless Application Model stack that persists the $XRP price to the XRPL every minute as a TrustLine. There are no servers, it is effectively a "smart contract" in Python for the XRPL.

xrpl-price-persist-oracle-sam This is a XRPL Oracle that publishes external data into the XRPL. This Oracle was inspired by XRPL-Labs/XRPL-Persist-Pri

Joseph Chiocchi 11 Dec 17, 2022
Monitor your Binance portfolio

Binance Report Bot The intent of this bot is to take a snapshot of your binance wallet, e.g. the current balances and store it for further plotting. I

37 Oct 29, 2022
Multi-Branch CI/CD Pipeline using CDK Pipelines.

Using AWS CDK Pipelines and AWS Lambda for multi-branch pipeline management and infrastructure deployment. This project shows how to use the AWS CDK P

AWS Samples 36 Dec 23, 2022
Sniper for Anigame and Izzi discord bots!

Anigame Sniper Gen-3 Features Inbuilt Spammer Responds to your messages in discord (on/off) Snipes only where you want it to Set latency so that the b

22 Nov 13, 2022
Drcom-pt-client - Drcom Pt version client with refresh timer

drcom-pt-client Drcom Pt version client with refresh timer Dr.com Pt版本客户端 可用于网页认

4 Nov 16, 2022
Windows版本微信客户端(非网页版)自动化,可实现简单的发送、接收微信消息

wxauto Windows版本微信客户端自动化,可实现简单的发送、接收微信消息

357 Dec 29, 2022
Ethereum Gas Fee for the MacBook Pro touchbar (using BetterTouchTool)

Gasbar Ethereum Gas Fee for the MacBook Pro touchbar (using BetterTouchTool) Worried about Ethereum gas fees? Me too. I'd like to keep an eye on them

TSS 51 Nov 14, 2022
Kang Sticker bot

Kang Sticker Bot A simple Telegram bot which creates sticker packs from other stickers, images, documents and URLs. Based on kangbot Deploy Credits: s

Hafitz Setya 11 Jan 02, 2023
Role Based Access Control for Slack-Bolt Applications

Role Based Access Control for Slack-Bolt Apps Role Based Access Control (RBAC) is a term applied to limiting the authorization for a specific operatio

Jeremy Schulman 7 Jan 06, 2022
A simple message content sniping Discord bot which you can run yourself! Sniping API pulled from isobot and Arch bot

Discord Snipe Bot This is a bot made with the same message content sniping API from isobot and Arch bot. It's default prefix is -, however you can als

notsniped 5 Aug 11, 2022
An alternative to OpenFaaS nats-queue-worker for long-running functions

OpenFaas Job Worker OpenFaas Job Worker is a fork of project : OSCAR Worker - https://github.com/grycap/oscar-worker Thanks to Sebástian Risco @srisco

Sebastien Aucouturier 1 Jan 07, 2022
Fetch information about a public Google document.

xeuledoc Fetch information about any public Google document. It's working on : Google Docs Google Spreadsheets Google Slides Google Drawning Google My

Malfrats Industries 655 Jan 03, 2023
How to add reaction on message discord.py

BA / HR / RS: Python (discord.py) skripta pomocu koje dodajete reakciju na vasu poruku putem komande !v ili da se dodaje samo u nekoj odredjenoj sobi.

Seekii 3 Dec 23, 2021