Gym Threat Defense

Overview

Gym Threat Defense

The Threat Defense environment is an OpenAI Gym implementation of the environment defined as the toy example in Optimal Defense Policies for Partially Observable Spreading Processes on Bayesian Attack Graphs by Miehling, E., Rasouli, M., & Teneketzis, D. (2015). It constitutes a 29-state/observation, 4-action POMDP defense problem.

The environment

The Threat Defense environment

Above, the Threat Defense environment can be observed. None of the notations or the definitions made in the paper will be explained in the text that follows, but rather the benchmark of the toy example will be stated. If these are desired, follow the link found earlier to the paper of Miehling, E., Rasouli, M., & Teneketzis, D. (2015).

Attributes

Of the 12 attributes that the toy example is built up by, two are leaf attributes (1 and 5) and one is a critical attribute (12). To give the network a more realistic appearance, the 12 attributes are intepreted in the paper as:

  1. Vulnerability in WebDAV on machine 1
  2. User access on machine 1
  3. Heap corruption via SSH on machine 1
  4. Root access on machine 1
  5. Buffer overflow on machine 2
  6. Root access on machine 2
  7. Squid portscan on machine 2
  8. Network topology leakage from machine 2
  9. Buffer overflow on machine 3
  10. Root access on machine 3
  11. Buffer overflow on machine 4
  12. Root access on machine 4

Actions

The defender have access to the two following binary actions:

  • u_1: Block WebDAV service
  • u_2: Disconnect machine 2

Thus we have four countermeasures to apply, i.e U = {none, u_1, u_2, u_1 & u_2}.

Cost Function

The cost function is defined as C(x,u) = C(x) + D(u).

C(x) is the state cost, and is 1 if the state, that is x, is a critical attribute. Otherwise it is 0.

D(u) is the availability cost of a countermeasure u, and is 0 if the countermeasure is none, 1 if it is u_1 or u_2 and 5 if it is both u_1 and u_2.

Parameters

The parameters of the problem are:

# The probabilities of detection:
beta = [0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.7, 0.6, 0.7, 0.85, 0.95]

# The attack probabilities:
alpha_1, alpha_5 = 0.5

# The spread probabilities:
alpha_(1,2), alpha_(2,3), alpha_(4,9), alpha_(5,6), alpha_(7,8), alpha_(8,9), alpha_(8,11), alpha_(10,11) = 0.8

alpha_(3,4), alpha_(6,7), alpha_(9,10), alpha_(11,12) = 0.9

# The discount factor:
gamma = 0.85

# The initial belief vector
pi_0 = [1,0,...,0]

Dependencies

  • OpenAI Gym
  • Numpy

Installation

cd gym-threat-defense
pip install -e .

Rendering

There are two possible rendering alternatives when running the environment. These are:

  • Render to stdout
  • A visual mode which prints the graph and indicate which nodes the attacker has taken over

To do a visual rendering, pass in 'rgb_array' to the render function.

env.render('rgb_array')

GUI rendering

Otherwise, for an ASCII representation to stdout, pass in 'human'.

env.render('human')

Example of the printing, where we can see that the agent took the block and disconnect action. The attacker has enabled five attributes, i.e. nodes, represented by ones, where the non-enabled attributes are represented by zeros. A node with parentheses is a leaf node, also known as an entry-point, a square bracket is a normal non-leaf node and a double bracketed node is a critical node.

Action: Block WebDAV service and Disconnect machine 2
(1) --> [1] --> [0] --> [0]
		      \--> [0] <-- [0] <-- [1] <-- [1] <-- (1)
			   \--> [0] <---/
				  \--> [0] --> [[0]]

By default the mode is set to printing to stdout.

Example

As an example on how to use the Threat Defense environment, we provide a couple of algorithms that uses both configurations of the environment. Read the README in the examples/ directory for more information on which algorithm works with which.

Template

How to create new environments for Gym

Inspiration

banana-gym

gym-soccer

gym-pomdp

Authors

Owner
Hampus Ramström
Hampus Ramström
Sequence modeling benchmarks and temporal convolutional networks

Sequence Modeling Benchmarks and Temporal Convolutional Networks (TCN) This repository contains the experiments done in the work An Empirical Evaluati

CMU Locus Lab 3.5k Jan 01, 2023
Pose estimation with MoveNet Lightning

Pose Estimation With MoveNet Lightning MoveNet is the TensorFlow pre-trained model that identifies 17 different key points of the human body. It is th

Yash Vora 2 Jan 04, 2022
This repository provides code for "On Interaction Between Augmentations and Corruptions in Natural Corruption Robustness".

On Interaction Between Augmentations and Corruptions in Natural Corruption Robustness This repository provides the code for the paper On Interaction B

Meta Research 33 Dec 08, 2022
Establishing Strong Baselines for TripClick Health Retrieval; ECIR 2022

TripClick Baselines with Improved Training Data Welcome 🙌 to the hub-repo of our paper: Establishing Strong Baselines for TripClick Health Retrieval

Sebastian Hofstätter 3 Nov 03, 2022
PatrickStar enables Larger, Faster, Greener Pretrained Models for NLP. Democratize AI for everyone.

PatrickStar: Parallel Training of Large Language Models via a Chunk-based Memory Management Meeting PatrickStar Pre-Trained Models (PTM) are becoming

Tencent 633 Dec 28, 2022
FPGA: Fast Patch-Free Global Learning Framework for Fully End-to-End Hyperspectral Image Classification

FPGA & FreeNet Fast Patch-Free Global Learning Framework for Fully End-to-End Hyperspectral Image Classification by Zhuo Zheng, Yanfei Zhong, Ailong M

Zhuo Zheng 92 Jan 03, 2023
PyTorch implementation for our paper "Deep Facial Synthesis: A New Challenge"

FSGAN Here is the official PyTorch implementation for our paper "Deep Facial Synthesis: A New Challenge". This project achieve the translation between

Deng-Ping Fan 32 Oct 10, 2022
Code for our WACV 2022 paper "Hyper-Convolution Networks for Biomedical Image Segmentation"

Hyper-Convolution Networks for Biomedical Image Segmentation Code for our WACV 2022 paper "Hyper-Convolution Networks for Biomedical Image Segmentatio

Tianyu Ma 17 Nov 02, 2022
Tensorflow implementation of "BEGAN: Boundary Equilibrium Generative Adversarial Networks"

BEGAN in Tensorflow Tensorflow implementation of BEGAN: Boundary Equilibrium Generative Adversarial Networks. Requirements Python 2.7 or 3.x Pillow tq

Taehoon Kim 922 Dec 21, 2022
A simple implementation of Kalman filter in single object tracking

kalman-filter-in-single-object-tracking A simple implementation of Kalman filter in single object tracking https://www.bilibili.com/video/BV1Qf4y1J7D4

130 Dec 26, 2022
Light-SERNet: A lightweight fully convolutional neural network for speech emotion recognition

Light-SERNet This is the Tensorflow 2.x implementation of our paper "Light-SERNet: A lightweight fully convolutional neural network for speech emotion

Arya Aftab 29 Nov 12, 2022
SelfRemaster: SSL Speech Restoration

SelfRemaster: Self-Supervised Speech Restoration Official implementation of SelfRemaster: Self-Supervised Speech Restoration with Analysis-by-Synthesi

Takaaki Saeki 46 Jan 07, 2023
Metrics to evaluate quality and efficacy of synthetic datasets.

An Open Source Project from the Data to AI Lab, at MIT Metrics for Synthetic Data Generation Projects Website: https://sdv.dev Documentation: https://

The Synthetic Data Vault Project 129 Jan 03, 2023
Implementation of the Point Transformer layer, in Pytorch

Point Transformer - Pytorch Implementation of the Point Transformer self-attention layer, in Pytorch. The simple circuit above seemed to have allowed

Phil Wang 501 Jan 03, 2023
SegTransVAE: Hybrid CNN - Transformer with Regularization for medical image segmentation

SegTransVAE: Hybrid CNN - Transformer with Regularization for medical image segmentation This repo is the official implementation for SegTransVAE. Seg

Nguyen Truong Hai 4 Aug 04, 2022
Neuralnetwork - Basic Multilayer Perceptron Neural Network for deep learning

Neural Network Just a basic Neural Network module Usage Example Importing Module

andreecy 0 Nov 01, 2022
Source code of the paper Meta-learning with an Adaptive Task Scheduler.

ATS About Source code of the paper Meta-learning with an Adaptive Task Scheduler. If you find this repository useful in your research, please cite the

Huaxiu Yao 16 Dec 26, 2022
HHP-Net: A light Heteroscedastic neural network for Head Pose estimation with uncertainty

HHP-Net: A light Heteroscedastic neural network for Head Pose estimation with uncertainty Giorgio Cantarini, Francesca Odone, Nicoletta Noceti, Federi

18 Aug 02, 2022
Pytorch Implementation of Value Retrieval with Arbitrary Queries for Form-like Documents.

Value Retrieval with Arbitrary Queries for Form-like Documents Introduction Pytorch Implementation of Value Retrieval with Arbitrary Queries for Form-

Salesforce 13 Sep 15, 2022