{"id":2473,"date":"2021-06-28T09:00:31","date_gmt":"2021-06-28T01:00:31","guid":{"rendered":"https:\/\/blog.ailia.ai\/uncategorized\/blazepose-skeletal-detection-model-capable-of-obtaining-3d-coordinates\/"},"modified":"2025-05-20T14:46:30","modified_gmt":"2025-05-20T06:46:30","slug":"blazepose-skeletal-detection-model-capable-of-obtaining-3d-coordinates","status":"publish","type":"post","link":"https:\/\/blog.ailia.ai\/en\/tips-en\/blazepose-skeletal-detection-model-capable-of-obtaining-3d-coordinates\/","title":{"rendered":"BlazePose : A 3D Pose Estimation Model"},"content":{"rendered":"\n

Overview<\/strong><\/h3>\n\n\n\n

BlazePose\u00a0<\/em>(Full Body) is a pose detection model developed by Google that can compute (x,y,z) coordinates of 33 skeleton keypoints. It can be used for example in fitness applications.<\/p>\n\n\n\n

\"\"
\u51fa\u5178\uff1ahttps:\/\/pixabay.com\/ja\/photos\/%E5%A5%B3%E3%81%AE%E5%AD%90-%E7%BE%8E%E3%81%97%E3%81%84-%E8%8B%A5%E3%81%84-%E3%83%9B%E3%83%AF%E3%82%A4%E3%83%88-5204299\/<\/a><\/figcaption><\/figure>\n\n\n\n

<\/a><\/p>\n\n\n\n

BlazePose: On-device Real-time Body Pose <\/a>tracking<\/a><\/a><\/p>\n\n\n\n

On-device, Real-time Body Pose Tracking with MediaPipe BlazePose<\/a><\/p>\n\n\n\n

BlazePose input and output<\/strong><\/h3>\n\n\n\n

BlazePose <\/em>consists of two machine learning models: a Detector <\/em>and an Estimator<\/em>. The Detector <\/em>cuts out the human region from the input image, while the Estimator <\/em>takes a 256×256 resolution image of the detected person as input and outputs the keypoints.<\/p>\n\n\n\n

BlazePose <\/em>outputs the 33 keypoints according the following ordering convention. This is more points than the commonly used 17 keypoints of the COCO dataset.<\/p>\n\n\n\n

\"\"
BlazePose keypoints (Source: https:\/\/developers.google.com\/ml-kit\/vision\/pose-detection<\/a>)<\/figcaption><\/figure>\n\n\n\n

Architecture<\/strong><\/h3>\n\n\n\n

The Detector <\/em>is an Single-Shot Detector(SSD) based architecture. Given an input image (1,224,224,3), it outputs a bounding box (1,2254,12) and a confidence score (1,2254,1). The 12 elements of the bounding box are of the form (x,y,w,h,kp1x,kp1y,\u2026,kp4x,kp4y), where kp1x to kp4y are additional keypoints. Each one of the 2254 elements has its own anchor, anchor scale and offset need to be applied.<\/p>\n\n\n\n

There are two ways to use the Detector<\/em>. In box mode<\/em>, the bounding box is determined from its position (x,y) and size (w,h). In alignment mode<\/em>, the scale and angle are determined from (kp1x,kp1y) and (kp2x,kp2y), and bounding box including rotation can be predicted.<\/p>\n\n\n\n

\"\"
Source: https:\/\/ai.googleblog.com\/2020\/08\/on-device-real-time-body-pose-tracking.html<\/a><\/figcaption><\/figure>\n\n\n\n

The\u00a0Estimator\u00a0<\/em>uses heatmap for training, but computes keypoints directly without using heatmap for faster inference.<\/p>\n\n\n\n

\"\"
Tracking network architecture: regression with heatmap supervision (Source: https:\/\/ai.googleblog.com\/2020\/08\/on-device-real-time-body-pose-tracking.html<\/a>)<\/figcaption><\/figure>\n\n\n\n

The first output of the Estimator<\/em> is (1,195) landmarks , the second output is (1,1) flags. The landmarks are made of 165 elements for the (x,y,z,visibility,presence<\/em>) for every 33 keypoints .<\/p>\n\n\n\n

The z<\/em>-values are based on the person\u2019s hips, with keypoints being between the hips and the camera when the value is negative, and behind the hips when the value is positive.<\/p>\n\n\n\n

The visibility <\/em>and presence <\/em>are stored in the range of [min_float,max_float<\/em>] and are converted to probability by applying a sigmoid function. The visibility <\/em>returns the probablity of keypoints that exist in the frame and are not occluded by other objects. presence <\/em>returns the probablity of keypoints that exist in the frame.<\/a><\/p>\n\n\n\n

Model Card BlazePose GHUM 3D.pdf<\/a><\/p>\n\n\n\n

Usage<\/strong><\/h3>\n\n\n\n

Use the following command to run\u00a0BlazePose (Full Body)<\/em>\u00a0with ailia SDK.<\/p>\n\n\n\n

$ python3 blazepose-fullbody.py -v 0<\/code><\/p>\n\n\n\n

<\/a>ailia-models\/pose_estimation_3d\/blazepose-fullbody at master \u00b7 axinc-ai\/ailia-models<\/a><\/p>\n\n\n\n

Here is a result on a sample video. The size of the circles at keypoints indicates the z-value.<\/p>\n\n\n\n

\n