The Illinois repository for Climatehack (https://climatehack.ai/). We won 1st place!

Overview

Climatehack

This is the repository for Illinois's Climatehack Team. We earned first place on the leaderboard with a final score of 0.87992.

Final Leaderboard

An overview of our approach can be found here.

Example predictions:

Setup

conda env create -f environment.yaml
conda activate climatehack
python -m ipykernel install --user --name=climatehack

First, download data by running data/download_data.ipynb. Alternatively, you can find preprocessed data files here. Save them into the data folder. We used train.npz and test.npz. They consist of data temporally cropped from 10am to 4pm UK time across the entire dataset. You could also use data_good_sun_2020.npz and data_good_sun_2021.npz, which consist of all samples where the sun elevation is at least 10 degrees. Because these crops produced datasets that could fit in-memory, all our dataloaders work in-memory.

Best Submission

Our best submission earned scores exceeding 0.85 on the Climatehack leaderboard. It is relatively simple and uses the fastai library to pick a base model, optimizer, and learning rate scheduler. After some experimentation, we chose xse_resnext50_deeper. We turned it into a UNET and trained it. More info is in the slides.

To train:

cd best-submission
bash train.sh

To submit, first move the trained model xse_resnext50_deeper.pth into best-submission/submission.

cd best-submission
python doxa_cli.py user login
bash submit.sh

Also, check out best-submission/test_and_visualize.ipynb to test the model and visualize results in a nice animation. This is how we produced the animations found in figs/model_predictions.gif.

Experiments

We conducted several experiments that showed improvements on a strong baseline. The baseline was OpenClimateFix's skillful nowcasting repo, which itself is a implementation of Deepmind's precipitation forecasting GAN. This baseline is more-or-less copied to experiments/dgmr-original. One important difference is that instead of training the GAN, we just train the generator. This was doing well for us and training the GAN had much slower convergence. This baseline will actually train to a score greater than 0.8 on the Climatehack leaderboard. We didn't have time to properly test these experiments on top of our best model, but we suspect they would improve results. The experiments are summarized below:

Experiment Description Results
DCT-Trick Inspired by this, we use the DCT to turn 128x128 -> 64x16x16 and IDCT to turn 64x16x16 -> 128x128. This leads to a shallower network that is autoregressive at fewer spatial resolutions. We believe this is the first time this has been done with UNETs. A fast implementation is in common/utils.py:create_conv_dct_filter and common/utils.py:get_idct_filter. 1.8-2x speedup, small <0.005 performance drop
Denoising We noticed a lot of blocky artifacts in predictions. These artifacts are reminiscent of JPEG/H.264 compression artifacts. We show a comparison of these artifacts in the slides. We found a pretrained neural network to fix them. This can definitely be done better, but we show a proof-of-concept. No performance drop, small visual improvement. The slides have an example.
CoordConv Meteorological phenomenon are correlated with geographic coordinates. We add 2 input channels for the geographic coordinates in OSGB form. +0.0072 MS-SSIM improvement
Optical Flow Optical flow does well for the first few timesteps. We add 2 input channels for the optical flow vectors. +0.0034 MS-SSIM improvement

The folder experiments/climatehack-submission was used to submit these experiments.

cd experiments/climatehack-submission
python doxa_cli.py user login
bash submit.sh

Use experiments/test_and_visualize.ipynb to test the model and visualize results in a nice animation.

Owner
Jatin Mathur
Undergrad at UIUC. Currently working on satellites with LASSI (https://lassiaero.web.illinois.edu/). Previously @astranis, @robinhood, @fractal, @ncsa
Jatin Mathur
Differentiable simulation for system identification and visuomotor control

gradsim gradSim: Differentiable simulation for system identification and visuomotor control gradSim is a unified differentiable rendering and multiphy

105 Dec 18, 2022
Build fully-functioning computer vision models with PyTorch

Detecto is a Python package that allows you to build fully-functioning computer vision and object detection models with just 5 lines of code. Inferenc

Alan Bi 576 Dec 29, 2022
Official Pytorch implementation of Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inference (ICLR 2022)

The Official Implementation of CLIB (Continual Learning for i-Blurry) Online Continual Learning on Class Incremental Blurry Task Configuration with An

NAVER AI 34 Oct 26, 2022
Convolutional neural network web app trained to track our infant’s sleep schedule using our Google Nest camera.

Machine Learning Sleep Schedule Tracker What is it? Convolutional neural network web app trained to track our infant’s sleep schedule using our Google

g-parki 7 Jul 15, 2022
Final project code: Implementing BicycleGAN, for CIS680 FA21 at University of Pennsylvania

680 Final Project: BicycleGAN Haoran Tang Instructions 1. Training To train the network, please run train.py. Change hyper-parameters and folder paths

Haoran Tang 0 Apr 22, 2022
Repository for publicly available deep learning models developed in Rosetta community

trRosetta2 This package contains deep learning models and related scripts used by Baker group in CASP14. Installation Linux/Mac clone the package git

81 Dec 29, 2022
official code for dynamic convolution decomposition

Revisiting Dynamic Convolution via Matrix Decomposition (ICLR 2021) A pytorch implementation of DCD. If you use this code in your research please cons

Yunsheng Li 110 Nov 23, 2022
Investigating automatic navigation towards standard US views integrating MARL with the virtual US environment developed in CT2US simulation

AutomaticUSnavigation Investigating automatic navigation towards standard US views integrating MARL with the virtual US environment developed in CT2US

Cesare Magnetti 6 Dec 05, 2022
Focal Loss for Dense Rotation Object Detection

Convert ResNets weights from GluonCV to Tensorflow Abstract GluonCV released some new resnet pre-training weights and designed some new resnets (such

17 Nov 24, 2021
An implementation of the Contrast Predictive Coding (CPC) method to train audio features in an unsupervised fashion.

CPC_audio This code implements the Contrast Predictive Coding algorithm on audio data, as described in the paper Unsupervised Pretraining Transfers we

Meta Research 283 Dec 30, 2022
Implementing SYNTHESIZER: Rethinking Self-Attention in Transformer Models using Pytorch

Implementing SYNTHESIZER: Rethinking Self-Attention in Transformer Models using Pytorch Reference Paper URL Author: Yi Tay, Dara Bahri, Donald Metzler

Myeongjun Kim 66 Nov 30, 2022
Implementation of ICCV21 paper: PnP-DETR: Towards Efficient Visual Analysis with Transformers

Implementation of ICCV 2021 paper: PnP-DETR: Towards Efficient Visual Analysis with Transformers arxiv This repository is based on detr Recently, DETR

twang 113 Dec 27, 2022
Fuse radar and camera for detection

SAF-FCOS: Spatial Attention Fusion for Obstacle Detection using MmWave Radar and Vision Sensor This project hosts the code for implementing the SAF-FC

ChangShuo 18 Jan 01, 2023
Official repository of DeMFI (arXiv.)

DeMFI This is the official repository of DeMFI (Deep Joint Deblurring and Multi-Frame Interpolation). [ArXiv_ver.] Coming Soon. Reference Jihyong Oh a

Jihyong Oh 56 Dec 14, 2022
[CVPR 2021] MiVOS - Mask Propagation module. Reproduced STM (and better) with training code :star2:. Semi-supervised video object segmentation evaluation.

MiVOS (CVPR 2021) - Mask Propagation Ho Kei Cheng, Yu-Wing Tai, Chi-Keung Tang [arXiv] [Paper PDF] [Project Page] [Papers with Code] This repo impleme

Rex Cheng 106 Jan 03, 2023
Minimal deep learning library written from scratch in Python, using NumPy/CuPy.

SmallPebble Project status: experimental, unstable. SmallPebble is a minimal/toy automatic differentiation/deep learning library written from scratch

Sidney Radcliffe 92 Dec 30, 2022
DSAC* for Visual Camera Re-Localization (RGB or RGB-D)

DSAC* for Visual Camera Re-Localization (RGB or RGB-D) Introduction Installation Data Structure Supported Datasets 7Scenes 12Scenes Cambridge Landmark

Visual Learning Lab 143 Dec 22, 2022
VGGVox models for Speaker Identification and Verification trained on the VoxCeleb (1 & 2) datasets

VGGVox models for speaker identification and verification This directory contains code to import and evaluate the speaker identification and verificat

338 Dec 27, 2022
BirdCLEF 2021 - Birdcall Identification 4th place solution

BirdCLEF 2021 - Birdcall Identification 4th place solution My solution detail kaggle discussion Inference Notebook (best submission) Environment Use K

tattaka 42 Jan 02, 2023
ManipNet: Neural Manipulation Synthesis with a Hand-Object Spatial Representation - SIGGRAPH 2021

ManipNet: Neural Manipulation Synthesis with a Hand-Object Spatial Representation - SIGGRAPH 2021 Dataset Code Demos Authors: He Zhang, Yuting Ye, Tak

HE ZHANG 194 Dec 06, 2022