Ladder network is a deep learning algorithm that combines supervised and unsupervised learning

Related tags

Game Developmentladder
Overview

This repository contains source code for the experiments in a paper titled Semi-Supervised Learning with Ladder Networks by A Rasmus, H Valpola, M Honkala, M Berglund, and T Raiko.

Required libraries

Install Theano, Blocks Stable 0.2, Fuel Stable 0.2

Refer to the Blocks installation instructions for details but use tag v0.2 instead. Something along:

pip install git+git://github.com/mila-udem/[email protected]
pip install git+git://github.com/mila-udem/[email protected]

Fuel comes with Blocks, but you need to download and convert the datasets. Refer to the Fuel documentation. One might need to rename the converted files.

fuel-download mnist
fuel-convert mnist --dtype float32
fuel-download cifar10
fuel-convert cifar10
Alternatively, one can use the environment.yml file that is provided in this repo to create an conda environment.
  1. First install anaconda from https://www.continuum.io/downloads. Then,
  2. conda env create -f environment.yml
  3. source activate ladder
  4. The environment should be good to go!

Models in the paper

The following commands train the models with seed 1. The reported numbers in the paper are averages over several random seeds. These commands use all the training samples for training (--unlabeled-samples 60000) and none are used for validation. This results in a lot of NaNs being printed during the trainining, since the validation statistics are not available. If you want to observe the validation error and costs during the training, use --unlabeled-samples 50000.

MNIST all labels
# Full
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 1000,1,0.01,0.01,0.01,0.01,0.01 --labeled-samples 60000 --unlabeled-samples 60000 --seed 1 -- mnist_all_full
# Bottom
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,0,0,0,0,0,0 --labeled-samples 60000 --unlabeled-samples 60000 --seed 1 -- mnist_all_bottom
# Gamma model
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,2 --labeled-samples 60000 --unlabeled-samples 60000 --seed 1 -- mnist_all_gamma
# Supervised baseline
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-0 --denoising-cost-x 0,0,0,0,0,0,0 --labeled-samples 60000 --unlabeled-samples 60000 --f-local-noise-std 0.5 --seed 1 -- mnist_all_baseline
MNIST 100 labels
# Full
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 1000,10,0.1,0.1,0.1,0.1,0.1 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_full
# Bottom-only
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 5000,0,0,0,0,0,0 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_bottom
# Gamma
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,0.5 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_gamma
# Supervised baseline
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-0 --denoising-cost-x 0,0,0,0,0,0,0 --labeled-samples 100 --unlabeled-samples 60000 --f-local-noise-std 0.5 --seed 1 -- mnist_100_baseline
MNIST 1000 labels
# Full
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,20,0.1,0.1,0.1,0.1,0.1 --f-local-noise-std 0.2 --labeled-samples 1000 --unlabeled-samples 60000 --seed 1 -- mnist_1000_full
# Bottom-only
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,0,0,0,0,0,0 --labeled-samples 1000 --unlabeled-samples 60000 --seed 1 -- mnist_1000_bottom
# Gamma model
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,10 --labeled-samples 1000 --unlabeled-samples 60000 --seed 1 -- mnist_1000_gamma
# Supervised baseline
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-0 --denoising-cost-x 0,0,0,0,0,0,0 --labeled-samples 1000 --unlabeled-samples 60000 --f-local-noise-std 0.5 --seed 1 -- mnist_1000_baseline
MNIST 50 labels
# Full model
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,20,0.1,0.1,0.1,0.1,0.1 --labeled-samples 50 --unlabeled-samples 60000 --seed 1 -- mnist_50_full
MNIST convolutional models
# Conv-FC
run.py train --encoder-layers convv:1000:26:1:1-convv:500:1:1:1-convv:250:1:1:1-convv:250:1:1:1-convv:250:1:1:1-convv:10:1:1:1-globalmeanpool:0 --decoder-spec gauss --denoising-cost-x 1000,10,0.1,0.1,0.1,0.1,0.1,0.1 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_conv_fc
# Conv-Small, Gamma
run.py train --encoder-layers convf:32:5:1:1-maxpool:2:2-convv:64:3:1:1-convf:64:3:1:1-maxpool:2:2-convv:128:3:1:1-convv:10:1:1:1-globalmeanpool:6:6-fc:10 --decoder-spec 0-0-0-0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,0,0,0,1 --labeled-samples 100 --unlabeled-samples 60000 --seed 1  -- mnist_100_conv_gamma
# Conv-Small, supervised baseline. Overfits easily, so keep training short.
run.py train --encoder-layers convf:32:5:1:1-maxpool:2:2-convv:64:3:1:1-convf:64:3:1:1-maxpool:2:2-convv:128:3:1:1-convv:10:1:1:1-globalmeanpool:6:6-fc:10 --decoder-spec 0-0-0-0-0-0-0-0-0-0 --denoising-cost-x 0,0,0,0,0,0,0,0,0,0 --num-epochs 20 --lrate-decay 0.5 --f-local-noise-std 0.45 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_conv_baseline
CIFAR models
# Conv-Large, Gamma
./run.py train --encoder-layers convv:96:3:1:1-convf:96:3:1:1-convf:96:3:1:1-maxpool:2:2-convv:192:3:1:1-convf:192:3:1:1-convv:192:3:1:1-maxpool:2:2-convv:192:3:1:1-convv:192:1:1:1-convv:10:1:1:1-globalmeanpool:0 --decoder-spec 0-0-0-0-0-0-0-0-0-0-0-0-gauss --dataset cifar10 --act leakyrelu --denoising-cost-x 0,0,0,0,0,0,0,0,0,0,0,0,4.0 --num-epochs 70 --lrate-decay 0.86 --seed 1 --whiten-zca 3072 --contrast-norm 55 --top-c False --labeled-samples 4000 --unlabeled-samples 50000 -- cifar_4k_gamma
# Conv-Large, supervised baseline. Overfits easily, so keep training short.
./run.py train --encoder-layers convv:96:3:1:1-convf:96:3:1:1-convf:96:3:1:1-maxpool:2:2-convv:192:3:1:1-convf:192:3:1:1-convv:192:3:1:1-maxpool:2:2-convv:192:3:1:1-convv:192:1:1:1-convv:10:1:1:1-globalmeanpool:0 --decoder-spec 0-0-0-0-0-0-0-0-0-0-0-0-0 --dataset cifar10 --act leakyrelu --denoising-cost-x 0,0,0,0,0,0,0,0,0,0,0,0,0 --num-epochs 20 --lrate-decay 0.5 --seed 1 --whiten-zca 3072 --contrast-norm 55 --top-c False --labeled-samples 4000 --unlabeled-samples 50000 -- cifar_4k_baseline
Evaluating models with testset

After training a model, you can infer the results on a test set by performing the evaluate command. An example use after training a model:

./run.py evaluate results/mnist_all_bottom0
Owner
Curious AI
Deep good. Unsupervised better.
Curious AI
GitHub Repository
GardenZen Prototype 2.5D Game

Relaxing 2.5D Game about gardening a small farm and growing food, plants and other things. Follow my Youtube Devlog for further details (see about me section)

Paul 3 Apr 15, 2022
MinMax Algo , Python

Write a PYTHON program to play the game of TIC-TAC-TOE on a 3×3 board with alternate inputs from user and computer.

Naman Anand 1 Nov 26, 2021
Building a Mario-like, classic platformer game in Python using the PyGame Library

Building a Mario-like, classic platformer game in Python using the PyGame Library

Clarence Vinzcent Reyes 1 Feb 06, 2022
AI based assitant for minecarft

Minecraft_AI_assistant AI-based assistant for Minecraft There are 4 steps to build 1-I'm using collecting_data.png as a structure to take shots with c

Murat Ali Avcu 13 Oct 16, 2022
Simple darts game using Tkinter and sqlite3. Also associated with Python.

Ever wanted to play a simple and fun game before, and it even keeps a database of your score? Well here it is!! Introducing; Darts! A simple and fun g

an aspirin 2 Dec 19, 2021
Backend application for a game to classify waste for recycling

Waste Organizer Game Backend application used in a game to classify trash for recycling. What is waste organizer game? It is a game developed during t

10 Jun 13, 2021
A simple yet powerful Snake Game made with myPygameWorkflow

snakeGame A simple yet powerful Snake Game made with myPygameWorkflow. Requirments python3 Python.org myPygameWorkflow Github Ripo Usage $ cd main $ p

DuskyElf 1 Dec 26, 2021
Lutris desktop client in Python / PyGObject

Lutris Lutris is an open source gaming platform that makes gaming on Linux easier by managing, installing and providing optimal settings for games. Lu

Lutris 6.1k Dec 30, 2022
Bingo game with python

bingo-game-with-python type of plays possible player vs computer player vs player computer vs computer game is built with 4 objects classes 1.game 2.b

1 Nov 27, 2021
Wordle is a word game reminiscent of mastermind

Wordle is a word game reminiscent of mastermind. The player tries to guess a five letter word within six attempts. After each attempt, hints are given. Green tiles show that the letter is in the righ

Sidharth Anand 1 Feb 09, 2022
Cricket game using PYQT

Cricket-game-using-PYQT This is a Fantasy cricket Desktop application build in p

Sanket Mane 1 Jan 03, 2022
XO game with server, client and visualizer for AI bots.

XO game with server, client and visualizer for AI bots.

Ali 4 Jul 14, 2022
Easily manage wine prefixes in a new way. Run Windows software and games on Linux

Bottles Easily manage wineprefix using environments Documentation · Forums · Telegram group · Funding 📚 Documentation Before opening a new issue, che

Bottles 4.1k Jan 09, 2023
My first Minecraft CPU. Created in collaboration with Peer Carnes as a final project in CS 281: Architecture and Assembly at the University of Puget Sound

Minecraft CPU This is my first ever Minecraft CPU, created in collaboration with Peer Carnes. We created a custom assembly language, including an asse

Andy Chamberlain 4 Oct 10, 2022
python script to convert .OBJ files into Minecraft, rendering them in game with a core shader.

samples: random notes about the tool general output format: (animation not supported yet but planned) vertex id Minecraft's gl_VertexID isn't per mode

199 Jan 02, 2023
Searches the word list in Wordle based on search pattern.

Wordle Searcher Searches the word list in Wordle based on search pattern. Warning: like all forms of cheating, it trivializes the game, and robs you o

Tyler Martin 1 Jan 29, 2022
The Original Snake Game. Maneuver a snake in its burrow and earn points while avoiding the snake itself and the walls of the snake burrow.

Maneuver a snake in its burrow and earn points while avoiding the snake itself and the walls of the snake burrow. The snake grows when it eats an apple by default which can be disabled in the setting

17 Nov 12, 2022
The game company we work for has two events that we want to track: buy an item and join a guild. Each of them has metadata characteristic of such events.

The game company we work for has two events that we want to track: buy an item and join a guild. Each of them has metadata characteristic of such events.

Caro Arriaga 1 Feb 04, 2022
A rhythm-based game that automatically generates obstacles based on a song's features.

DISCLAIMER: This is my first coding project, created in December 2019. The game may not be optimized, and looking back on it, there are a lot of chang

Kenneth Huang 1 Dec 27, 2021
Typing test and practice on command line without the need of any internet connection

Terminal-Typing-Test Typing test and practice on command line without the need of any internet connection About CLI based typing test and practice tha

Angad Deep Singh 3 Oct 19, 2022