I am using trt_pose on a system with i5 cpu and GTX 1660 Super GPU. I am currently only getting 9 fps for an 1080 x 1920 video. Any ideas for how to improve FPS?

Hello everyone,

this issue is redundant, I re-post it due the misleading title in the original one #68

this is the live_demo with opencv, just change the the file format to ipynb (github doesn't allow ipynb files attached): live_demo.txt

I'm feeding the image to the execute method, but the count of people is always 0(which is wrong).

Hello All,

We are trying to develop an application which uses trt_pose, the model is too slow and the max we could extract was 15fps on Jetson Tx2, after nvpmodel and jetson_clocks. Also, I observed that through tegrastats, the code is not utilizing all the cores, and I believe the GPU cores have zero usage during runtime, even though it is enabled inside the code. Can you please help me in getting a boost in the performance? Thank you.

Hello, Has anyone happen to have gotten this error when importing trt_pose.coco before?

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
 in 
      1 import json
----> 2 import trt_pose.coco
      3 
      4 with open('human_pose.json', 'r') as f:
      5     human_pose = json.load(f)

/usr/local/lib/python3.6/dist-packages/trt_pose-0.0.1-py3.6-linux-aarch64.egg/trt_pose/coco.py in 
      7 import tqdm
      8 import trt_pose
----> 9 import trt_pose.plugins
     10 import glob
     11 import torchvision.transforms.functional as FT

ImportError: /usr/local/lib/python3.6/dist-packages/trt_pose-0.0.1-py3.6-linux-aarch64.egg/trt_pose/plugins.cpython-36m-aarch64-linux-gnu.so: undefined symbol: _ZTIN3c1021AutogradMetaInterfaceE

@jaybdub Hi, jaybdub, Thanks for your great work! When I was trainning the net, I find the trainning process on only one GPU(I have two), how can I train trt_pose on multi-GPU? And restore from a pretrained checkpoint?

Hi, I highly appreciate your repo. But I test your model in Jetson nano Tegra X1 it just got about 8FPS lower than 22 FPS in your report. I just follow your guide except I did not use USB camera, I used OpenCV to read image instead of. I want to know whether you use other kind of Jetson nano, which version of Pytorch or I made a mistake. Many tks.

While running cell-5 of live_demo.ipynb notebook import torch2trt model_trt = torch2trt.torch2trt(model, [data], fp16_mode=False, max_workspace_size=1<<25)
getting error below: RuntimeError Traceback (most recent call last) in 1 import torch2trt 2 ----> 3 model_trt = torch2trt.torch2trt(model, [data], fp16_mode=False, max_workspace_size=1<<25)

~/tlt-experiments/skeleton-env/lib/python3.6/site-packages/torch2trt-0.1.0-py3.6-linux-x86_64.egg/torch2trt/torch2trt.py in torch2trt(module, inputs, input_names, output_names, log_level, max_batch_size, fp16_mode, max_workspace_size, strict_type_constraints, keep_network, int8_mode, int8_calib_dataset, int8_calib_algorithm, int8_calib_batch_size, use_onnx) 553 ) 554 --> 555 engine = builder.build_cuda_engine(network) 556 557 module_trt = TRTModule(engine, input_names, output_names)

RuntimeError: NVRTC error:

Hi @jaybdub!

Simply awesome the way we can detect poses and plot it on vídeo!! Great job!!

I just couldn't find out how to get pose's features metadata programatically in python (eg.: left_eye, left_elbow, right_ankle, etc.)

Could you please clarify it?

Thanks!

Dear Nvidia team, I would like to do some tranfer learning with my own data based on the model already trained on COCO dataset. However, I only found that I can change the checkpoint interval in the training config file. But had no clue about how to continue training from a checkpoint file. Could you please give me some idea about this if possible? Many thanks!

errors occur when convert .pth to onnx . I just follow the deep_stream_trt_pose instrution use : ./export_for_isaac.py --input_checkpoint resnet18_baseline_att_224x224_A_epoch_249.pth erros: : File name too long import-im6.q16: not authorized re' @ error/constitute.c/WriteImage/1037. import-im6.q16: not authorizedtorch' @ error/constitute.c/WriteImage/1037. ./export_for_isaac.py: line 60: syntax error near unexpected token (' ./export_for_isaac.py: line 60:class InputReNormalization(torch.nn.Module):'

Hi All I successfully installed trt_pose on jetson nano board, very interesting, thanks! i would like to install on a pc with 2070rtx using nvcr.io/nvidia/tensorrt:20.12-py3 based custom docker but, during model conversion (early), it gives me this error

[TensorRT] ERROR: CUDA initialization failure with error 222. Please check your CUDA installation: http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html Traceback (most recent call last): File "live_demo.py", line 40, in model_trt = torch2trt.torch2trt(model, [data], fp16_mode=True, max_workspace_size=1<<25) File "/usr/local/lib/python3.8/dist-packages/torch2trt-0.1.0-py3.8.egg/torch2trt/torch2trt.py", line 498, in torch2trt TypeError: pybind11::init(): factory function returned nullptr

ubuntu: 20.04 Driver Version: 450.80.02 cuda: 11.1 torch: 1.7.1 torch2trt: 0.1.0 torchvision: 0.8.2

Thanks in advance Gianluca

The stack trace is as follows:

raceback (most recent call last): File "setup.py", line 52, in setup( File "/usr/local/lib/python3.8/dist-packages/setuptools/init.py", line 87, in setup return distutils.core.setup(**attrs) File "/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/core.py", line 185, in setup return run_commands(dist) File "/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/core.py", line 201, in run_commands dist.run_commands() File "/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/dist.py", line 969, in run_commands self.run_command(cmd) File "/usr/local/lib/python3.8/dist-packages/setuptools/dist.py", line 1208, in run_command super().run_command(command) File "/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py", line 74, in run self.do_egg_install() File "/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py", line 123, in do_egg_install self.run_command('bdist_egg') File "/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/usr/local/lib/python3.8/dist-packages/setuptools/dist.py", line 1208, in run_command super().run_command(command) File "/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/usr/local/lib/python3.8/dist-packages/setuptools/command/bdist_egg.py", line 165, in run cmd = self.call_command('install_lib', warn_dir=0) File "/usr/local/lib/python3.8/dist-packages/setuptools/command/bdist_egg.py", line 151, in call_command self.run_command(cmdname) File "/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/usr/local/lib/python3.8/dist-packages/setuptools/dist.py", line 1208, in run_command super().run_command(command) File "/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/usr/local/lib/python3.8/dist-packages/setuptools/command/install_lib.py", line 11, in run self.build() File "/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/command/install_lib.py", line 112, in build self.run_command('build_ext') File "/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/usr/local/lib/python3.8/dist-packages/setuptools/dist.py", line 1208, in run_command super().run_command(command) File "/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/usr/local/lib/python3.8/dist-packages/setuptools/command/build_ext.py", line 84, in run _build_ext.run(self) File "/usr/local/lib/python3.8/dist-packages/Cython/Distutils/old_build_ext.py", line 186, in run _build_ext.build_ext.run(self) File "/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/command/build_ext.py", line 346, in run self.build_extensions() File "/usr/local/lib/python3.8/dist-packages/torch/utils/cpp_extension.py", line 741, in build_extensions build_ext.build_extensions(self) File "/usr/local/lib/python3.8/dist-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions _build_ext.build_ext.build_extensions(self) File "/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/command/build_ext.py", line 468, in build_extensions self._build_extensions_serial() File "/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/command/build_ext.py", line 494, in _build_extensions_serial self.build_extension(ext) File "/usr/local/lib/python3.8/dist-packages/setuptools/command/build_ext.py", line 246, in build_extension _build_ext.build_extension(self, ext) File "/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/command/build_ext.py", line 549, in build_extension objects = self.compiler.compile( File "/usr/local/lib/python3.8/dist-packages/torch/utils/cpp_extension.py", line 562, in unix_wrap_ninja_compile _write_ninja_file_and_compile_objects( File "/usr/local/lib/python3.8/dist-packages/torch/utils/cpp_extension.py", line 1419, in _write_ninja_file_and_compile_objects _run_ninja_build( File "/usr/local/lib/python3.8/dist-packages/torch/utils/cpp_extension.py", line 1756, in _run_ninja_build raise RuntimeError(message) from e RuntimeError: Error compiling objects for extension

Sorry for the less than descriptive title, but I wasn't sure how else to title it.

I've got a 4gb Jetson Nano (SeeedStudio Jetson Recomputer J2010 carrier board) with a 128gb SSD as the root storage device. It's running JetPack 4.6 (output of 'apt-cache show nvidia-jetpack' below)

`$ sudo apt-cache show nvidia-jetpack Package: nvidia-jetpack Version: 4.6-b199 Architecture: arm64 Maintainer: NVIDIA Corporation Installed-Size: 194 Depends: nvidia-cuda (= 4.6-b199), nvidia-opencv (= 4.6-b199), nvidia-cudnn8 (= 4.6-b199), nvidia-tensorrt (= 4.6-b199), nvidia-visionworks (= 4.6-b199), nvidia-container (= 4.6-b199), nvidia-vpi (= 4.6-b199), nvidia-l4t-jetson-multimedia-api (>> 32.6-0), nvidia-l4t-jetson-multimedia-api (<< 32.7-0) Homepage: http://developer.nvidia.com/jetson Priority: standard Section: metapackages Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_4.6-b199_arm64.deb Size: 29368 SHA256: 69df11e22e2c8406fe281fe6fc27c7d40a13ed668e508a592a6785d40ea71669 SHA1: 5c678b8762acc54f85b4334f92d9bb084858907a MD5sum: 1b96cd72f2a434e887f98912061d8cfb Description: NVIDIA Jetpack Meta Package Description-md5: ad1462289bdbc54909ae109d1d32c0a8

Package: nvidia-jetpack Version: 4.6-b197 Architecture: arm64 Maintainer: NVIDIA Corporation Installed-Size: 194 Depends: nvidia-cuda (= 4.6-b197), nvidia-opencv (= 4.6-b197), nvidia-cudnn8 (= 4.6-b197), nvidia-tensorrt (= 4.6-b197), nvidia-visionworks (= 4.6-b197), nvidia-container (= 4.6-b197), nvidia-vpi (= 4.6-b197), nvidia-l4t-jetson-multimedia-api (>> 32.6-0), nvidia-l4t-jetson-multimedia-api (<< 32.7-0) Homepage: http://developer.nvidia.com/jetson Priority: standard Section: metapackages Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_4.6-b197_arm64.deb Size: 29356 SHA256: 104cd0c1efefe5865753ec9b0b148a534ffdcc9bae525637c7532b309ed44aa0 SHA1: 8cca8b9ebb21feafbbd20c2984bd9b329a202624 MD5sum: 463d4303429f163b97207827965e8fe0 Description: NVIDIA Jetpack Meta Package Description-md5: ad1462289bdbc54909ae109d1d32c0a8 ` I've set up a python 3.6 virtualenv and followed the installation instructions for all required packages. There were no errors during any of the installations and I've verified all of the packages import properly from the python command line. When I run the jupyter notebook 'live_demo.ipynb' I am able to run all of the steps up until the following step:

`import torch2trt

model_trt = torch2trt.torch2trt(model, [data], fp16_mode=True, max_workspace_size=1<<25)`

When I attempt to run that step the systems thinks about it for a bit and then a dialog pops up that says the python kernel has crashed and will be automatically restarted. I cannot get past this step.

To help debug/diagnose I took all of the code from the notebook and incrementally added it to a python file to see if I could reproduce the issue. The code I have so far is:

`import cv2 import json import trt_pose.coco import trt_pose.models import torch import torch2trt from torch2trt import TRTModule import time import torchvision.transforms as transforms import PIL.Image from trt_pose.draw_objects import DrawObjects from trt_pose.parse_objects import ParseObjects from jetcam.usb_camera import USBCamera from jetcam.csi_camera import CSICamera from jetcam.utils import bgr8_to_jpeg import ipywidgets from IPython.display import display

with open('human_pose.json', 'r') as f: human_pose = json.load(f)

topology = trt_pose.coco.coco_category_to_topology(human_pose)

num_parts = len(human_pose['keypoints']) num_links = len(human_pose['skeleton'])

model = trt_pose.models.resnet18_baseline_att(num_parts, 2 * num_links).cuda().eval() MODEL_WEIGHTS = 'resnet18_baseline_att_224x224_A_epoch_249.pth' model.load_state_dict(torch.load(MODEL_WEIGHTS))

WIDTH = 224 HEIGHT = 224

data = torch.zeros((1, 3, HEIGHT, WIDTH)).cuda() print("Calling torch2trt.torch2trt\n") model_trt = torch2trt.torch2trt(model, [data], fp16_mode=True, max_workspace_size=1<<25) print("Done\n") OPTIMIZED_MODEL = 'resnet18_baseline_att_224x224_A_epoch_249_trt.pth' print("Calling torch.save\n") torch.save(model_trt.state_dict(), OPTIMIZED_MODEL) print("Done\n")`

When I run this code I see "Calling torch2trt.torch2trt" in the console and then, after a pause, I see the following error repeated many times on the console:

[TensorRT] ERROR: 3: [builderConfig.cpp::canRunOnDLA::341] Error Code 3: Internal Error (Parameter check failed at: optimizer/api/builderConfig.cpp::canRunOnDLA::341, condition: dlaEngineCount > 0 )

After that the system will seem to hang. The desktop will display a low memory warning (If I run 'watch -n1 free -h' in another console window I can see free memory drop from 3+GB to as little as 96MB). After some time the process just reports "Killed" and exits back to the command line.

I am at a loss. Can you please provide any helpful information you have that might help me correct this issue and continue?

Thanks Marc

Hi @jaybdub ,

1. Does this project have the feature to resume training from a checkpoint?

2. Are we able to start training from a pretrained model on a new dataset?

Thank you!

my running code is failed after reinstall the dependency. my jet pack details :- NVIDIA Jetson UNKNOWN L4T 32.5.0 [ JetPack 4.5 ] Ubuntu 18.04.5 LTS Kernel Version: 4.9.201+ CUDA 10.2.300 CUDA Architecture: NONE OpenCV version: 4.1.1 OpenCV Cuda: NO CUDNN: 8.0.0.180 TensorRT: 7.1.3.0 Vision Works: 1.6.0.501 VPI: ii libnvvpi1 1.0.12 arm64 NVIDIA Vision Programming Interface library Vulcan: 1.2.70

Hi, i would like to ask if there is any available evaluation metrics for this model? There is a eval.ipynb notebook here but it seems incomplete and using normal object detection metrics for evaluating the model.

Is there any scripts available to use for evaluating the keypoint detection for metrics like PCK, PDJ, PCP as shown in this repo ?

Thanks!