This code finds bounding box of a single human mouth.

Last update: Nov 27, 2022

Related tags

Overview

Swab AI

This code finds bounding box of a single human mouth. In comparison to other face segmentation methods, it is relatively insusceptible to open mouth conditions, e.g., yawning, surgical robots, etc. The mouth coordinates are found in a more certified way using two independent algorithms. Therefore, the algorithm can be used in more sensitive applications.

Sample Output

Selected images from YawDD and CelebAMask dataset.

Example Code

only for debugging purposes cv2.imwrite("output.jpg", cv2.rectangle(img, points[0], points[1], (0, 0, 255), 2)) print("output.jpg is written.") ">

import swab_ai
import cv2

img = cv2.imread("input.jpg")  # img is numpy array

points = swab_ai.Get_BoundingBox(img)  # returns bounding box coordinates of img

if points: # bounding box is found
    # save output file -> only for debugging purposes
    cv2.imwrite("output.jpg", cv2.rectangle(img, points[0], points[1], (0, 0, 255), 2))
    print("output.jpg is written.")

Python API

`swab_ai.py`

It loads required libraries and models. This module contains three functions:

GetFace(input_image, show=False, save=False, live=False): calls FaceInference.py and returns coordinates of face bounding box and two landmarks of mouth.
GetMouth(face_image, show=False, save=False, live=False): return mouth mask.
Get_BoundingBox(input_image, show=False, save=False, live=False): calls GetFace and GetMouth functions and checks if the bounding box could be certified by checking for sufficient overlap of two independent algorithms. This function returns the mouth bounding box and also prints the FPS in the output. In case mouth bounding box could not be found Frame Droppe is printed in the console.

`live_test.py`

For debugging purposes: The algorithm is run on live webcam video. The result of face detection and segmentation algorithms is visualized in real-time.

Dockerfile

We have used the latest version of pytorch image pytorch:1.9.0-cuda11.1-cudnn8-runtime as of now.

To build the docker image and run the container in one step:

 docker build -t swabai .
 docker run --runtime=nvidia swabai

Note that --runtime=nvidia is necessary to enable GPU.

If you see the message output.jpg is written, it means that everything is working correctly. To see the output image use this command to mount local working directory:

 docker run -v $PWD:/usr/src/app/ --runtime=nvidia swabai

Model

The first time that the code is run, model is downloaded automatically. Every time that the code restarts, it checks for model updates.

You might also like...

Pytorch based library to rank predicted bounding boxes using text/image user's prompts.

pytorch_clip_bbox: Implementation of the CLIP guided bbox ranking for Object Detection. Pytorch based library to rank predicted bounding boxes using t

50 Nov 27, 2022

Pytorch ImageNet1k Loader with Bounding Boxes.

ImageNet 1K Bounding Boxes For some experiments, you might wanna pass only the background of imagenet images vs passing only the foreground. Here, I'v

11 Oct 15, 2022

[CVPR2021] UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles

UAV-Human Official repository for CVPR2021: UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicle Paper arXiv Res

129 Jan 4, 2023

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, CVPR 2021

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors Human POSEitioning System (H

66 Dec 21, 2022

Human Action Controller - A human action controller running on different platforms.

Releases(v0.2)

v0.2(Dec 3, 2021)

Stage 3 (final stage) of the project
Source code(tar.gz)
Source code(zip)
v0.1(Sep 30, 2021)

Stage 2 of the project
Source code(tar.gz)
Source code(zip)

This code finds bounding box of a single human mouth.

Related tags

Overview

Swab AI

Sample Output

Example Code

Python API

swab_ai.py

live_test.py

Dockerfile

Model

You might also like...

Pytorch based library to rank predicted bounding boxes using text/image user's prompts.

Pytorch ImageNet1k Loader with Bounding Boxes.

[CVPR2021] UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, CVPR 2021

Human Action Controller - A human action controller running on different platforms.

Python scripts for performing 3D human pose estimation using the Mobile Human Pose model in ONNX.

StyleGAN-Human: A Data-Centric Odyssey of Human Generation

7th place solution of Human Protein Atlas - Single Cell Classification on Kaggle

"3D Human Texture Estimation from a Single Image with Transformers", ICCV 2021

Releases(v0.2)

v0.2(Dec 3, 2021)

v0.1(Sep 30, 2021)

Owner

iThermAI

PyKaldi GOP-DNN on Epa-DB

Generative Autoregressive, Normalized Flows, VAEs, Score-based models (GANVAS)

一个免费开源一键搭建的通用验证码识别平台，大部分常见的中英数验证码识别都没啥问题。

Repository for the COLING 2020 paper "Explainable Automated Fact-Checking: A Survey."

Code for CMaskTrack R-CNN (proposed in Occluded Video Instance Segmentation)

Code base of object detection

Implementation of average- and worst-case robust flatness measures for adversarial training.

An open-source outlier detection package by Getcontact Data Team

HandTailor: Towards High-Precision Monocular 3D Hand Recovery

The software associated with a paper accepted at EMNLP 2021 titled "Open Knowledge Graphs Canonicalization using Variational Autoencoders".

Semantic similarity computation with different state-of-the-art metrics

Source code for The Power of Many: A Physarum Swarm Steiner Tree Algorithm

Human segmentation models, training/inference code, and trained weights, implemented in PyTorch

HistoSeg : Quick attention with multi-loss function for multi-structure segmentation in digital histology images

Open source implementation of AceNAS: Learning to Rank Ace Neural Architectures with Weak Supervision of Weight Sharing

Diffusion Probabilistic Models for 3D Point Cloud Generation (CVPR 2021)

pytorch implementation of dftd2 & dftd3

The Body Part Regression (BPR) model translates the anatomy in a radiologic volume into a machine-interpretable form.

Turn based roguelike in python

ColBERT: Contextualized Late Interaction over BERT (SIGIR'20)

`swab_ai.py`

`live_test.py`