efficient neural audio synthesis in the waveform domain

Last update: Dec 23, 2022

Overview

neural waveshaping synthesis

real-time neural audio synthesis in the waveform domain

paper • website • colab • audio

by Ben Hayes, Charalampos Saitis, György Fazekas

This repository is the official implementation of Neural Waveshaping Synthesis.

Model Architecture

Requirements

To install:

pip install -r requirements.txt
pip install -e .

We recommend installing in a virtual environment.

Data

We trained our checkpoints on the URMP dataset. Once downloaded, the dataset can be preprocessed using scripts/create_urmp_dataset.py. This will consolidate recordings of each instrument within the dataset and preprocess them according to the pipeline in the paper.

python scripts/create_urmp_dataset.py \
  --gin-file gin/data/urmp_4second_crepe.gin \ 
  --data-directory /path/to/urmp \
  --output-directory /path/to/output \
  --device cuda:0  # torch device string for CREPE model

Alternatively, you can supply your own dataset and use the general create_dataset.py script:

python scripts/create_dataset.py \
  --gin-file gin/data/urmp_4second_crepe.gin \ 
  --data-directory /path/to/dataset \
  --output-directory /path/to/output \
  --device cuda:0  # torch device string for CREPE model

Training

To train a model on the URMP dataset, use this command:

python scripts/train.py \
  --gin-file gin/train/train_newt.gin \
  --dataset-path /path/to/processed/urmp \
  --urmp \
  --instrument vn \  # select URMP instrument with abbreviated string
  --load-data-to-memory

Or to use a non-URMP dataset:

python scripts/train.py \
  --gin-file gin/train/train_newt.gin \
  --dataset-path /path/to/processed/data \
  --load-data-to-memory

efficient neural audio synthesis in the waveform domain

Related tags

Overview

neural waveshaping synthesis

real-time neural audio synthesis in the waveform domain

paper • website • colab • audio

Model Architecture

Requirements

Data

Training

Owner

Ben Hayes

A python library to artfully visualize Factorio Blueprints and an interactive web demo for using it.

Official repository of Semantic Image Matting

Deep High-Resolution Representation Learning for Human Pose Estimation

This is the official code for the paper "Learning with Nested Scene Modeling and Cooperative Architecture Search for Low-Light Vision"

NeurIPS 2021 Datasets and Benchmarks Track

Naszilla is a Python library for neural architecture search (NAS)

Implementation of momentum^2 teacher

Uncertainty Estimation via Response Scaling for Pseudo-mask Noise Mitigation in Weakly-supervised Semantic Segmentation

Official PyTorch implementation of "Edge Rewiring Goes Neural: Boosting Network Resilience via Policy Gradient".

[NeurIPS 2021] PyTorch Code for Accelerating Robotic Reinforcement Learning with Parameterized Action Primitives

Sharpened cosine similarity torch - A Sharpened Cosine Similarity layer for PyTorch

TriMap: Large-scale Dimensionality Reduction Using Triplets

Buffon’s needle: one of the oldest problems in geometric probability

Semi Supervised Learning for Medical Image Segmentation, a collection of literature reviews and code implementations.

LSTM Neural Networks for Spectroscopic Studies of Type Ia Supernovae

Official Pytorch implementation of the paper: "Locally Shifted Attention With Early Global Integration"

Quickly and easily create / train a custom DeepDream model

The author's officially unofficial PyTorch BigGAN implementation.

Fast, Attemptable Route Planner for Navigation in Known and Unknown Environments

Meli Data Challenge 2021 - First Place Solution