FAST-RIR: FAST NEURAL DIFFUSE ROOM IMPULSE RESPONSE GENERATOR
This is the official implementation of our neural-network-based fast diffuse room impulse response generator (FAST-RIR) for generating roomimpulse responses (RIRs) for a given rectangular acoustic environment. Our model is inspired by StackGAN architecture. The audio examples and spectrograms of the generated RIRs are available here.
Requirements
Python3.6
Pytorch
python-dateutil
easydict
pandas
torchfile
gdown
pickle
Embedding
Each normalized embedding is created as follows: If you are using our trained model, you may need to use extra parameter Correction(CRR).
Listener Position = LP
Source Position = SP
Room Dimension = RD
Reverberation Time = T60
Correction = CRR
CRR = 0.1 if 0.5
<0.6
CRR = 0.2 if T60>0.6
CRR = 0 otherwise
Embedding = ([LP_X,LP_Y,LP_Z,SP_X,SP_y,SP_Z,RD_X,RD_Y,RD_Z,(T60+CRR)] /5) + 1
Generete RIRs using trained model
Download the trained model using this command
source download_generate.sh
Create normalized embeddings list in pickle format. You can run following command to generate an example embedding list
python3 example1.py
Run the following command inside code_new to generate RIRs corresponding to the normalized embeddings list. You can find generated RIRs inside code_new/Generated_RIRs
python3 main.py --cfg cfg/RIR_eval.yml --gpu 0
Range
Our trained NN-DAS is capable of generating RIRs with the following range accurately.
Room Dimension X --> 8m to 11m
Room Dimesnion Y --> 6m to 8m
Room Dimension Z --> 2.5m to 3.5m
Listener Position --> Any position within the room
Speaker Position --> Any position within the room
Reverberation time --> 0.2s to 0.7s
Training the Model
Run the following command to download the training dataset we created using a Diffuse Acoustic Simulator. You also can train the model using your dataset.
source download_data.sh
Run the following command to train the model. You can pass what GPUs to be used for training as an input argument. In this example, I am using 2 GPUs.
python3 main.py --cfg cfg/RIR_s1.yml --gpu 0,1
Related Works
- IR-GAN: Room Impulse Response Generator for Far-field Speech Recognition (INTERSPEECH2021)
- TS-RIR: Translated synthetic room impulse responses for speech augmentation (IEEE ASRU 2021)
Citations
If you use our FAST-RIR for your research, please consider citing
@misc{ratnarajah2021fastrir,
title={FAST-RIR: Fast neural diffuse room impulse response generator},
author={Anton Ratnarajah and Shi-Xiong Zhang and Meng Yu and Zhenyu Tang and Dinesh Manocha and Dong Yu},
year={2021},
eprint={2110.04057},
archivePrefix={arXiv},
primaryClass={cs.SD}
}
Our work is inspired by
@inproceedings{han2017stackgan,
Author = {Han Zhang and Tao Xu and Hongsheng Li and Shaoting Zhang and Xiaogang Wang and Xiaolei Huang and Dimitris Metaxas},
Title = {StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks},
Year = {2017},
booktitle = {{ICCV}},
}
If you use our training dataset generated using Diffuse Acoustic Simulator in your research, please consider citing
@inproceedings{9052932,
author={Z. {Tang} and L. {Chen} and B. {Wu} and D. {Yu} and D. {Manocha}},
booktitle={ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
title={Improving Reverberant Speech Training Using Diffuse Acoustic Simulation},
year={2020},
volume={},
number={},
pages={6969-6973},
}
81 Nov 08, 2022
468 Jan 03, 2023
838 Jan 01, 2023
12 Jan 13, 2022
1 Oct 27, 2021
3.2k Jan 05, 2023
20 Oct 24, 2022
5 Dec 08, 2022
6 Dec 21, 2022
20 Oct 09, 2022
9 Nov 24, 2022
76 Jan 03, 2023
3 Feb 18, 2022
847 Jan 08, 2023
196 Jan 04, 2023
75 Nov 11, 2022
26 Dec 25, 2022
19.4k Jan 04, 2023
3 Oct 23, 2022
32 Jul 19, 2022