This code finds bounding box of a single human mouth.

Last update: Nov 27, 2022

Related tags

Overview

Swab AI

This code finds bounding box of a single human mouth. In comparison to other face segmentation methods, it is relatively insusceptible to open mouth conditions, e.g., yawning, surgical robots, etc. The mouth coordinates are found in a more certified way using two independent algorithms. Therefore, the algorithm can be used in more sensitive applications.

Sample Output

Selected images from YawDD and CelebAMask dataset.

Example Code

only for debugging purposes cv2.imwrite("output.jpg", cv2.rectangle(img, points[0], points[1], (0, 0, 255), 2)) print("output.jpg is written.") ">

import swab_ai
import cv2

img = cv2.imread("input.jpg")  # img is numpy array

points = swab_ai.Get_BoundingBox(img)  # returns bounding box coordinates of img

if points: # bounding box is found
    # save output file -> only for debugging purposes
    cv2.imwrite("output.jpg", cv2.rectangle(img, points[0], points[1], (0, 0, 255), 2))
    print("output.jpg is written.")

Python API

`swab_ai.py`

It loads required libraries and models. This module contains three functions:

GetFace(input_image, show=False, save=False, live=False): calls FaceInference.py and returns coordinates of face bounding box and two landmarks of mouth.
GetMouth(face_image, show=False, save=False, live=False): return mouth mask.
Get_BoundingBox(input_image, show=False, save=False, live=False): calls GetFace and GetMouth functions and checks if the bounding box could be certified by checking for sufficient overlap of two independent algorithms. This function returns the mouth bounding box and also prints the FPS in the output. In case mouth bounding box could not be found Frame Droppe is printed in the console.

`live_test.py`

For debugging purposes: The algorithm is run on live webcam video. The result of face detection and segmentation algorithms is visualized in real-time.

Dockerfile

We have used the latest version of pytorch image pytorch:1.9.0-cuda11.1-cudnn8-runtime as of now.

To build the docker image and run the container in one step:

 docker build -t swabai .
 docker run --runtime=nvidia swabai

Note that --runtime=nvidia is necessary to enable GPU.

If you see the message output.jpg is written, it means that everything is working correctly. To see the output image use this command to mount local working directory:

 docker run -v $PWD:/usr/src/app/ --runtime=nvidia swabai

Model

The first time that the code is run, model is downloaded automatically. Every time that the code restarts, it checks for model updates.

You might also like...

Pytorch based library to rank predicted bounding boxes using text/image user's prompts.

pytorch_clip_bbox: Implementation of the CLIP guided bbox ranking for Object Detection. Pytorch based library to rank predicted bounding boxes using t

50 Nov 27, 2022

Pytorch ImageNet1k Loader with Bounding Boxes.

ImageNet 1K Bounding Boxes For some experiments, you might wanna pass only the background of imagenet images vs passing only the foreground. Here, I'v

11 Oct 15, 2022

[CVPR2021] UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles

UAV-Human Official repository for CVPR2021: UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicle Paper arXiv Res

129 Jan 4, 2023

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, CVPR 2021

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors Human POSEitioning System (H

66 Dec 21, 2022

Human Action Controller - A human action controller running on different platforms.

Releases(v0.2)

v0.2(Dec 3, 2021)

Stage 3 (final stage) of the project
Source code(tar.gz)
Source code(zip)
v0.1(Sep 30, 2021)

Stage 2 of the project
Source code(tar.gz)
Source code(zip)

This code finds bounding box of a single human mouth.

Related tags

Overview

Swab AI

Sample Output

Example Code

Python API

swab_ai.py

live_test.py

Dockerfile

Model

You might also like...

Pytorch based library to rank predicted bounding boxes using text/image user's prompts.

Pytorch ImageNet1k Loader with Bounding Boxes.

[CVPR2021] UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, CVPR 2021

Human Action Controller - A human action controller running on different platforms.

Python scripts for performing 3D human pose estimation using the Mobile Human Pose model in ONNX.

StyleGAN-Human: A Data-Centric Odyssey of Human Generation

7th place solution of Human Protein Atlas - Single Cell Classification on Kaggle

"3D Human Texture Estimation from a Single Image with Transformers", ICCV 2021

Releases(v0.2)

v0.2(Dec 3, 2021)

v0.1(Sep 30, 2021)

Owner

iThermAI

Tiny Kinetics-400 for test

The all new way to turn your boring vector meshes into the new fad in town; Voxels!

Prototype for Baby Action Detection and Classification

Single-step adversarial training (AT) has received wide attention as it proved to be both efficient and robust.

In this project, we develop a face recognize platform based on MTCNN object-detection netcwork and FaceNet self-supervised network.

Methods to get the probability of a changepoint in a time series.

Chinese license plate recognition

Official PyTorch implementation of "Improving Face Recognition with Large AgeGaps by Learning to Distinguish Children" (BMVC 2021)

Fast and Easy Infinite Neural Networks in Python

Neighborhood Reconstructing Autoencoders

Algorithmic Trading using RNN

It's a implement of this paper：Relation extraction via Multi-Level attention CNNs

Data, notebooks, and articles associated with the RSNA AI Deep Learning Lab at RSNA 2021

Dimension Reduced Turbulent Flow Data From Deep Vector Quantizers

Large-Scale Unsupervised Object Discovery

Official implementation of the paper Label-Efficient Semantic Segmentation with Diffusion Models

A plug-and-play library for neural networks written in Python

Ground truth data for the Optical Character Recognition of Historical Classical Commentaries.

Pytorch-3dunet - 3D U-Net model for volumetric semantic segmentation written in pytorch

Google-drive-to-sqlite - Create a SQLite database containing metadata from Google Drive

`swab_ai.py`

`live_test.py`