Anonymous implementation of KSL

Last update: Nov 10, 2021

Related tags

Deep Learning ksl

Overview

k-Step Latent (KSL)

Implementation of k-Step Latent (KSL) in PyTorch.

Representation Learning for Data-Efficient Reinforcement Learning

[Paper]

Code is built on top of the DrQ repo from Denis Yarats.

Getting Started

First, create and activate conda env:

conda env create -f conda_env.yml

conda activate ksl

This repo relies on environments from DMControl, and therefore assumes that you can run MuJoCo.

From within ./ksl, simply run:

python train.py

Altering training schemes can be done by feeding additional args, such as:

python train.py env=cheetah_run lr=2e-4

For a full list of customizable args, see ./ksl/configs.yaml.

Observing Runs

Just as in the DrQ repo, train.py will produce the runs folder, where all the outputs are going to be stored including train/eval logs, tensorboard blobs, and evaluation episode videos. To launch tensorboard run

tensorboard --logdir runs

The console output is also available in a form:

| train | E: 5 | S: 5000 | R: 11.4359 | D: 66.8 s | BR: 0.0581 | ALOSS: -1.0640 | CLOSS: 0.0996 | TLOSS: -23.1683 | TVAL: 0.0945 | AENT: 3.8132

a training entry decodes as

train - training episode
E - total number of episodes
S - total number of environment steps
R - episode return
D - duration in seconds
BR - average reward of a sampled batch
ALOSS - average loss of the actor
CLOSS - average loss of the critic
TLOSS - average loss of the temperature parameter
TVAL - the value of temperature
AENT - the actor's entropy

while an evaluation entry

| eval  | E: 20 | S: 20000 | R: 10.9356

contains

E - evaluation was performed after E episodes
S - evaluation was performed after S environment steps
R - average episode return computed over `num_eval_episodes` (usually 10)

Anonymous implementation of KSL

Related tags

Overview

k-Step Latent (KSL)

Getting Started

Observing Runs

Owner

How to Become More Salient? Surfacing Representation Biases of the Saliency Prediction Model

Real-world Anomaly Detection in Surveillance Videos- pytorch Re-implementation

A cross-lingual COVID-19 fake news dataset

Official Implementation of "LUNAR: Unifying Local Outlier Detection Methods via Graph Neural Networks"

EmoTag helps you train emotion detection model for Chinese audios

VM3000 Microphones

Implementation of "GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddings" in PyTorch

TensorFlow Ranking is a library for Learning-to-Rank (LTR) techniques on the TensorFlow platform

Implementation of "Learning to Match Features with Seeded Graph Matching Network" ICCV2021

Learning Versatile Neural Architectures by Propagating Network Codes

Notebooks for my "Deep Learning with TensorFlow 2 and Keras" course

Learning Continuous Signed Distance Functions for Shape Representation

A small fun project using python OpenCV, mediapipe, and pydirectinput

Jarvis Project is a basic virtual assistant that uses TensorFlow for learning.

Camview - A CLI-tool used to stream CCTV online footage based on URL params

Pure python PEMDAS expression solver without using built-in eval function

An AI Assistant More Than a Toolkit

Mask2Former: Masked-attention Mask Transformer for Universal Image Segmentation in TensorFlow 2

PPO Lagrangian in JAX

Video Frame Interpolation without Temporal Priors (a general method for blurry video interpolation)