Human Action Controller - A human action controller running on different platforms.

Related tags

Deep Learninghac
Overview

Human Action Controller (HAC)

Goal

A human action controller running on different platforms.

Fun Easy-to-use
Accurate Anywhere

Fun Examples

Mouse Control

Mouse Control

Keyboard Control

Keyboard Control

Playing Game

Pikachu

Enhancing interaction

Gather Town

Solutions provided by HAC

Platform Module Progress Comment
PC / Win10 Mouse Control V
PC / Win10 Keyboard Control V
PC / Ubuntu Mouse Control
PC / Ubuntu Keyboard Control

Getting started

Installation

$ pip install pyhac

Run the demo of mouse control

$ git clone https://github.com/dabit-lucas/hac.git
$ cd hac
$ python demo.py

Recording custom actions

$ python recording.py -d {action name} -k True

Press key "r" to start recording, the data will be saved into ./data

Training a custom module

Here is an example of a config file of action set,

{
    "actions": [
        "r_five",
        "r_zero",
        "l_five",
        "l_zero",
        "two_index_fingers_up",
        "two_index_fingers_down",
        "33",
        "55",
        "sit"
    ],
    "type": "gesture_only"
}

These actions form a model by running a training process:

$ python train.py --conf {path_of_action} --model_name {name_of_model}

The generated model will become a module. Take mouse control module as an exmaple, it can create mappings among actions and controls by the following code:

mouse_module = hac.add_module("mouse_control")
hac.set_init_module(mouse_module)

# create mapping between controls and actions
mouse_module.add_mouse_mapping("mouse_left_down", ["r_five", "r_zero"])
mouse_module.add_mouse_mapping("mouse_left_up", "r_five")
mouse_module.add_mouse_mapping("mouse_right_down", ["l_five", "l_zero"])
mouse_module.add_mouse_mapping("mouse_right_up", "l_five")
mouse_module.add_mouse_mapping("right_move_diff", ["r_five", "r_five"])
mouse_module.add_mouse_mapping("right_move_diff", ["r_zero", "r_zero"])
mouse_module.add_mouse_mapping("left_move_diff", ["l_five", "l_five"])
mouse_module.add_mouse_mapping("left_move_diff", ["l_zero", "l_zero"])
mouse_module.add_mouse_mapping("roll_up", "two_index_fingers_up")
mouse_module.add_mouse_mapping("roll_down", "two_index_fingers_down")

If the five gesture with a right hand shows in consecutive two frames ["r_five", "r_five"], then do control right_move_diff, which means moving the mouse cursor. The above description can be represented by the following code:

mouse_module.add_mouse_mapping("right_move_diff", ["r_five", "r_five"])

Development guideline

The structure of HAC

Community

Welcome to ask any question in issues.

Contributing

Any contribution is welcomed. Please fork this repo and summit a pull request.

Owner
GitHub Repository
Multivariate Boosted TRee

Multivariate Boosted TRee What is MBTR MBTR is a python package for multivariate boosted tree regressors trained in parameter space. The package can h

SUPSI-DACD-ISAAC 61 Dec 19, 2022
DIR-GNN - Discovering Invariant Rationales for Graph Neural Networks

DIR-GNN "Discovering Invariant Rationales for Graph Neural Networks" (ICLR 2022)

Ying-Xin (Shirley) Wu 70 Nov 13, 2022
An MQA (Studio, originalSampleRate) identifier for lossless flac files written in Python.

An MQA (Studio, originalSampleRate) identifier for "lossless" flac files written in Python.

Daniel 10 Oct 03, 2022
Official Repo of my work for SREC Nandyal Machine Learning Bootcamp

About the Bootcamp A 3-day Machine Learning Bootcamp organised by Department of Electronics and Communication Engineering, Santhiram Engineering Colle

MS 1 Nov 29, 2021
AI that generate music

PianoGPT ai that generate music try it here https://share.streamlit.io/annasajkh/pianogpt/main/main.py or here https://huggingface.co/spaces/Annas/Pia

Annas 28 Nov 27, 2022
This project is a re-implementation of MASTER: Multi-Aspect Non-local Network for Scene Text Recognition by MMOCR

This project is a re-implementation of MASTER: Multi-Aspect Non-local Network for Scene Text Recognition by MMOCR，which is an open-source toolbox based on PyTorch. The overall architecture will be sh

Jianquan Ye 82 Nov 17, 2022
Implementation of our recent paper, WOOD: Wasserstein-based Out-of-Distribution Detection.

WOOD Implementation of our recent paper, WOOD: Wasserstein-based Out-of-Distribution Detection. Abstract The training and test data for deep-neural-ne

8 Dec 24, 2022
Replication Package for AequeVox:Automated Fariness Testing for Speech Recognition Systems

AequeVox Replication Package for AequeVox:Automated Fariness Testing for Speech Recognition Systems README under development. Python Packages Required

Sai Sathiesh 2 Aug 28, 2022
Real-world Anomaly Detection in Surveillance Videos- pytorch Re-implementation

Real world Anomaly Detection in Surveillance Videos : Pytorch RE-Implementation This repository is a re-implementation of "Real-world Anomaly Detectio

seominseok 62 Dec 08, 2022
Code of the paper "Deep Human Dynamics Prior" in ACM MM 2021.

Code of the paper "Deep Human Dynamics Prior" in ACM MM 2021. Figure 1: In the process of motion capture (mocap), some joints or even the whole human

Shinny cui 3 Oct 31, 2022
Original Pytorch Implementation of FLAME: Facial Landmark Heatmap Activated Multimodal Gaze Estimation

FLAME Original Pytorch Implementation of FLAME: Facial Landmark Heatmap Activated Multimodal Gaze Estimation, accepted at the 17th IEEE Internation Co

Neelabh Sinha 19 Dec 17, 2022
Official implementation of the paper "Topographic VAEs learn Equivariant Capsules"

Topographic Variational Autoencoder Paper: https://arxiv.org/abs/2109.01394 Getting Started Install requirements with Anaconda: conda env create -f en

T. Andy Keller 69 Dec 12, 2022
N-gram models- Unsmoothed, Laplace, Deleted Interpolation

N-gram models- Unsmoothed, Laplace, Deleted Interpolation

Ravika Nagpal 1 Jan 04, 2022
The source code for the Cutoff data augmentation approach proposed in this paper: "A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation".

Cutoff: A Simple Data Augmentation Approach for Natural Language This repository contains source code necessary to reproduce the results presented in

Dinghan Shen 49 Dec 22, 2022
Train DeepLab for Semantic Image Segmentation

Train DeepLab for Semantic Image Segmentation Martin Kersner, [email protected]

Martin Kersner 172 Dec 14, 2022
shufflev2-yolov5：lighter, faster and easier to deploy

shufflev2-yolov5: lighter, faster and easier to deploy. Evolved from yolov5 and the size of model is only 1.7M (int8) and 3.3M (fp16). It can reach 10+ FPS on the Raspberry Pi 4B when the input size

pogg 1.5k Jan 05, 2023
Machine Translation Implement By Bi-GRU And Transformer

Seq2Seq Translation Implement By Bidirectional GRU And Transformer In Pytorch Before You Run The Code You should download the data through the link be

He Wang 2 Oct 27, 2021
Emotion Recognition from Facial Images

Reconhecimento de Emoções a partir de imagens faciais Este projeto implementa um classificador simples que utiliza técncias de deep learning e transfe

Gabriel 2 Feb 09, 2022
With this package, you can generate mixed-integer linear programming (MIP) models of trained artificial neural networks (ANNs) using the rectified linear unit (ReLU) activation function

With this package, you can generate mixed-integer linear programming (MIP) models of trained artificial neural networks (ANNs) using the rectified linear unit (ReLU) activation function. At the momen

ChemEngAI 40 Dec 27, 2022
Scikit-event-correlation - Event Correlation and Forecasting over High Dimensional Streaming Sensor Data algorithms

scikit-event-correlation Event Correlation and Changing Detection Algorithm Theo

Intellia ICT 5 Oct 30, 2022