Introduction to “MoveNet,” a machine learning model available in the ailia SDK. With ailia SDK and ailia MODELS, an edge inference framework, implementing AI functionality into applications becomes easy.
Introduction to MoveNet
Here’s an introduction to “MoveNet,” a machine learning model available in the ailia SDK. MoveNet is a pose detection model released by Google on May 17, 2021. It improves the accuracy of detecting poses in dynamic videos compared to conventional pose detection techniques. It is particularly suitable for live fitness and sports applications.
Next-Generation Pose Detection with MoveNet and TensorFlow.js
May 17, 2021 – Posted by Ronny Votel and Na Li, Google Research Today we’re excited to launch our latest pose detection…
MoveNet Architecture
MoveNet rapidly and accurately detects 17 2D keypoints. It consists of two models: Lightning and Thunder. Lightning is suitable for applications requiring speed, while Thunder is for applications requiring precision. Both Lightning and Thunder models can operate at over 30 FPS on desktop PCs, laptops, and smartphones.
The architecture is similar to CenterNet, with a FeatureExtractor based on MobileNetV2 with a Feature Pyramid Network (FPN). Setting the output stride to 4 allows for handling high resolutions.
The AI model outputs include Person center heatmap, Keypoint regression field, Person keypoint heatmap, and 2D per-keypoint offset field.
MoveNet is trained on both the COCO dataset and Google’s proprietary dataset. The COCO dataset lacks challenging data, such as poses with significant changes or motion blur, which may not be suitable for fitness or dance applications. Google’s internal dataset includes annotated yoga, fitness, and dance videos from YouTube, using only three frames per video to ensure dataset diversity.
How to Use MoveNet
To use MoveNet, use the following command. It enables recognition from a webcam.
$ python3 movenet.py -v 0
Example：
https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FhFUMD46Nugc%3Ffeature%3Doembed&display_name=YouTube&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DhFUMD46Nugc&image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FhFUMD46Nugc%2Fhqdefault.jpg&key=a19fcc184b9711e1b4764040d3dc5c07&type=text%2Fhtml&schema=youtube
ailia-models/pose_estimation/movenet at master · axinc-ai/ailia-models
(Image from…
ax Inc. is an AI solutions company developing ailia SDK for cross-platform, GPU-accelerated inference. We offer comprehensive AI solutions ranging from consulting to model creation, SDK provision, and AI-based app/system development and support. Feel free to contact us for more information.
