How do I load a HigherHRNet in OpenCV C++? - c++

Question

I have recently started using the C++ implementation of OpenCV and have been running into a spot of trouble. I have been experimenting with trying to estimate 3D human pose with the video from my built in camera.
To start, I looked at a project like this which accomplishes a similar task by importing an ONNX model and loading it with cv::dnn::readNetFromONNX(modelPath);. However this model only performs 2D pose estimation. From this I concluded that if I could gather a model from an alternate source and, as long as it was in the ONNX format, then it would be able to be loaded by OpenCV.
I tried going to Google Colab to use OpenVino in a safe environment to grab a copy of the model with their model downloader and model converter. These commands ended up being:
!pip install openvino-dev[onnx]
!omz_downloader --name higher-hrnet-w32-human-pose-estimation
!pip install yacs
!omz_converter --name higher-hrnet-w32-human-pose-estimation
Through the course of these commands, we see:
========== Converting higher-hrnet-w32-human-pose-estimation to ONNX
Conversion to ONNX command: /usr/bin/python3 -- /usr/local/lib/python3.7/dist-packages/open_model_zoo/model_tools/internal_scripts/pytorch_to_onnx.py --model-path=/usr/local/lib/python3.7/dist-packages/open_model_zoo/model_tools/models/public/higher-hrnet-w32-human-pose-estimation --model-path=/content/public/higher-hrnet-w32-human-pose-estimation --model-name=get_net --import-module=model '--model-param=file_config=r"/content/public/higher-hrnet-w32-human-pose-estimation/experiments/higher_hrnet.yaml"' '--model-param=weights=r"/content/public/higher-hrnet-w32-human-pose-estimation/ckpt/pose_higher_hrnet_w32_512.pth"' --input-shape=1,3,512,512 --input-names=image --output-names=embeddings,heatmaps --output-file=/content/public/higher-hrnet-w32-human-pose-estimation/higher-hrnet-w32-human-pose-estimation.onnx
ONNX check passed successfully.
========== Converting higher-hrnet-w32-human-pose-estimation to IR (FP16)
Conversion command: /usr/bin/python3 -m mo --framework=onnx --data_type=FP16 --output_dir=/content/public/higher-hrnet-w32-human-pose-estimation/FP16 --model_name=higher-hrnet-w32-human-pose-estimation --reverse_input_channels '--input_shape=[1,3,512,512]' --input=image '--mean_values=image[123.675,116.28,103.53]' '--scale_values=image[58.395,57.12,57.375]' --output=embeddings,heatmaps --input_model=/content/public/higher-hrnet-w32-human-pose-estimation/higher-hrnet-w32-human-pose-estimation.onnx
which indicates the existence of higher-hrnet-w32-human-pose-estimation.onnx in our local storage. So I downloaded that file to my local device and ran in the above context. When I run, I get a cryptic error:
Unhandled exception at 0x00007FFB62FF4F69 in posedetect.exe: Microsoft C++ exception: cv::Exception at memory location
Is there a way to load a 3D pose estimation model in OpenCV using C++?
Alternate attempt with openpose
I tried following the recommendation for using openpose as an alternative to the HigherHRNet model as described by #B200011011 is the comments. To do this I went to the open pose github and performed the following:
git clone https://github.com/CMU-Perceptual-Computing-Lab/openpose.git
cd openpose/models
getModels.bat
cp pose/body_25/pose_iter_584000.caffemodel ../source/repos/posedetect/models/
When I try to load this caffe model with cv::dnn::readNetFromCaffe(modelPath); I get a similar cryptic error as trying to load the HigherHRNet Model:
Unhandled exception at 0x00007FFB62FF4F69 in posedetect.exe: Microsoft C++ exception: cv::Exception at memory location 0x000000E42CB0D660.
So how do I load a HigherHRNet (or other 3D pose estimation) model in OpenCV C++?