Swab AI
This code finds bounding box of a single human mouth. In comparison to other face segmentation methods, it is relatively insusceptible to open mouth conditions, e.g., yawning, surgical robots, etc. The mouth coordinates are found in a more certified way using two independent algorithms. Therefore, the algorithm can be used in more sensitive applications.
Sample Output
Selected images from YawDD and CelebAMask dataset.
Example Code
import swab_ai
import cv2
img = cv2.imread("input.jpg") # img is numpy array
points = swab_ai.Get_BoundingBox(img) # returns bounding box coordinates of img
if points: # bounding box is found
# save output file -> only for debugging purposes
cv2.imwrite("output.jpg", cv2.rectangle(img, points[0], points[1], (0, 0, 255), 2))
print("output.jpg is written.")
Python API
swab_ai.py
It loads required libraries and models. This module contains three functions:
GetFace(input_image, show=False, save=False, live=False): callsFaceInference.pyand returns coordinates of face bounding box and two landmarks of mouth.GetMouth(face_image, show=False, save=False, live=False): return mouth mask.Get_BoundingBox(input_image, show=False, save=False, live=False): callsGetFaceandGetMouthfunctions and checks if the bounding box could be certified by checking for sufficient overlap of two independent algorithms. This function returns the mouth bounding box and also prints the FPS in the output. In case mouth bounding box could not be foundFrame Droppeis printed in the console.
live_test.py
For debugging purposes: The algorithm is run on live webcam video. The result of face detection and segmentation algorithms is visualized in real-time.
Dockerfile
We have used the latest version of pytorch image pytorch:1.9.0-cuda11.1-cudnn8-runtime as of now.
To build the docker image and run the container in one step:
docker build -t swabai .
docker run --runtime=nvidia swabai
Note that
--runtime=nvidiais necessary to enable GPU.
If you see the message output.jpg is written, it means that everything is working correctly. To see the output image use this command to mount local working directory:
docker run -v $PWD:/usr/src/app/ --runtime=nvidia swabai
Model
The first time that the code is run, model is downloaded automatically. Every time that the code restarts, it checks for model updates.
50 Nov 27, 2022
11 Oct 15, 2022
129 Jan 4, 2023
66 Dec 21, 2022
27 Jul 20, 2022
99 Dec 31, 2022
762 Jan 8, 2023
8 Jul 9, 2021
193 Dec 5, 2022
109 Dec 02, 2022
2.1k Dec 28, 2022
65 Dec 09, 2022
4 Aug 22, 2022
6 Jun 13, 2022
2.7k Jan 08, 2023
154 Dec 21, 2022
400 Dec 27, 2022
54 Nov 25, 2022
54 Dec 20, 2022
3 Dec 26, 2022
1 Jan 06, 2022
64 Dec 24, 2022
157 Dec 26, 2022
61 Jan 01, 2023
10 Aug 30, 2022
20 Jan 05, 2023
682 Jan 04, 2023
0 Sep 02, 2021