competitions-v2

Last update: Dec 02, 2022

Related tags

Overview

Codabench

(formerly Codalab Competitions v2)

Installation

$ cp .env_sample .env
$ docker-compose up -d
$ docker-compose exec django ./manage.py migrate
$ docker-compose exec django ./manage.py generate_data
$ docker-compose exec django ./manage.py collectstatic --noinput

You can now login as username "admin" with password "admin" at http://localhost:8000

If you ever need to reset the database, use the script ./reset_db.sh

Running tests

# Non "end to end tests"
$ docker-compose exec django py.test -m "not e2e"

# "End to end tests" (a shell script to launch a selenium docker container)
$ ./run_selenium_tests.sh

# If you are on Mac OSX it is easy to watch these tests, no need to install
# anything just do:
$ open vnc://0.0.0.0:5900

# And login with password "secret"

Example competitions

The repo comes with a couple examples that are used during tests:

v2 test data

src/tests/functional/test_files/submission.zip
src/tests/functional/test_files/competition.zip

v1.5 legacy test data

src/tests/functional/test_files/submission15.zip
src/tests/functional/test_files/competition15.zip

Other Codalab Competition examples

https://github.com/codalab/competition-examples/tree/master/v2/

Building compute worker

To build the normal image:

docker build -t codalab/competitions-v2-compute-worker:latest -f Dockerfile.compute_worker .

To build the GPU version:

docker build -t codalab/competitions-v2-compute-worker:nvidia -f Dockerfile.compute_worker_gpu .

Updating the image

docker push codalab/competitions-v2-compute-worker

Worker setup

# install docker
$ curl https://get.docker.com | sudo sh
$ sudo usermod -aG docker $USER

# >>> reconnect <<<

Start CPU worker

Make a file .env and put this in it:

# Queue URL
BROKER_URL=
   
    

# Location to store submissions/cache -- absolute path!
HOST_DIRECTORY=/your/path/to/codabench/storage

# If SSL is enabled, then uncomment the following line
#BROKER_USE_SSL=True

NOTE /your/path/to/codabench -- this path needs to be volumed into /codabench on the worker, as you can see below.

$ docker run \
    -v /your/path/to/codabench/storage:/codabench \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -d \
    --env-file .env \
    --restart unless-stopped \
    --log-opt max-size=50m \
    --log-opt max-file=3 \
    codalab/competitions-v2-compute-worker:latest

Start GPU worker

nvidia installation instructions

$ nvidia-docker run \
    -v /your/path/to/codabench/storage:/codabench \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v /var/lib/nvidia-docker/nvidia-docker.sock:/var/lib/nvidia-docker/nvidia-docker.sock \
    -d \
    --env-file .env \
    --restart unless-stopped \
    --log-opt max-size=50m \
    --log-opt max-file=3 \
    codalab/competitions-v2-compute-worker:nvidia

Worker management

Outside of docker containers install Fabric like so:

pip install fab-classic==1.17.0

Create a server_config.yaml in the root of this repository using:

cp server_config_sample.yaml server_config.yaml

Below is an example server_config.yaml that defines 2 roles comp-gpu and comp-cpu, one with gpu style workers (is_gpu and the nvidia docker_image) and one with cpu style workers

comp-gpu:
  hosts:
    - [email protected]
    - [email protected]
  broker_url: pyamqp://user:[email protected]:port/vhost-gpu
  is_gpu: true
  docker_image: codalab/competitions-v2-compute-worker:nvidia

comp-cpu:
  hosts:
    - [email protected]
  broker_url: pyamqp://user:[email protected]:port/vhost-cpu
  is_gpu: false
  docker_image: codalab/competitions-v2-compute-worker:latest

You can of course create your own docker_image and specify it here.

You can execute commands against a role:

❯ fab -R comp-gpu status
..
[[email protected]] out: CONTAINER ID        IMAGE                                           COMMAND                  CREATED             STATUS              PORTS               NAMES
[[email protected]] out: 1d318268bee1        codalab/competitions-v2-compute-worker:nvidia   "/bin/sh -c 'celery …"   2 hours ago         Up 2 hours                              hardcore_greider
..

❯ fab -R comp-gpu update
..
(updates workers)

See available commands with fab -l

competitions-v2

Related tags

Overview

Codabench

(formerly Codalab Competitions v2)

Installation

Running tests

Example competitions

v2 test data

v1.5 legacy test data

Other Codalab Competition examples

Building compute worker

Worker setup

Start CPU worker

Start GPU worker

Worker management

Owner

CodaLab

Cancer Drug Response Prediction via a Hybrid Graph Convolutional Network

Repository for the electrical and ICT benchmark model developed in the ERIGrid 2.0 project.

Implementing Graph Convolutional Networks and Information Retrieval Mechanisms using pure Python and NumPy

Multi-Scale Geometric Consistency Guided Multi-View Stereo

Defending graph neural networks against adversarial attacks (NeurIPS 2020)

CLOOB training (JAX) and inference (JAX and PyTorch)

Unsupervised Video Interpolation using Cycle Consistency

Pytorch implementation of U-Net, R2U-Net, Attention U-Net, and Attention R2U-Net.

Python interface for SmartRF Sniffer 2 Firmware

PyoMyo - Python Opensource Myo library

Code for Iso-Points: Optimizing Neural Implicit Surfaces with Hybrid Representations

Learning to Communicate with Deep Multi-Agent Reinforcement Learning in PyTorch

Retinal vessel segmentation based on GT-UNet

MassiveSumm: a very large-scale, very multilingual, news summarisation dataset

A Japanese Medical Information Extraction Toolkit

RoadMap and preparation material for Machine Learning and Data Science - From beginner to expert.

(Preprint) Official PyTorch implementation of "How Do Vision Transformers Work?"

Goal of the project : Detecting Temporal Boundaries in Sign Language videos

An Approach to Explore Logistic Regression Models

DLWP: Deep Learning Weather Prediction