3DObjectDetectionPytorch : 3D Object Detection Model

Overview

3DObjectDetectionPytorch is a machine learning model that calculates 3D bounding boxes of objects. Other object detection models such as YOLOgenerally computes 2D bounding boxes, but this new model returns bounding boxes with depth information.

GitHub – sovrasov/3d-object-detection.pytorch

Architecture

3DObjectDetectionPytorch is capable of recognizing the following 9 classes.

OBJECTRON_CLASSES = (‘bike’, ‘book’, ‘bottle’, ‘cereal_box’, ‘camera’, ‘chair’, ‘cup’, ‘laptop’, ‘shoe’)

First, the 2D bounding box of the object is computed with MobileNetV2 SSD, and then the 3D bounding box is calculated with MobileNetV3 regression model. This regression model takes an image (1,3,224,224) as input and returns 9 keypoints (x,y) for each class (9,1,9,2).

3DObjectDetectionPytorch was trained on the Objectron dataset, which is publicly available from Google.

GitHub — google-research-datasets/Objectron

The Objectron dataset is a dataset developed for AR development and contains 15K annotated videos and 4M annotated images.

Usage

Use the following command to run 3DObjectDetectionPytorch with ailia SDK on a web camera video stream.

$ python3 3d-object-detection.pytorch.py -v 0

Here is the result you can expect.

ailia-models/object_detection_3d/3d-object-detection.pytorch

Related topics

ailia MODELS also contains Google’s mediapipe_objectron which is a model also trained using the Objectron dataset.

ailia-models/object_detection_3d/mediapipe_objectron

ax Inc. has developed ailia SDK, which enables cross-platform, GPU-based rapid inference.

ax Inc. provides a wide range of services from consulting and model creation, to the development of AI-based applications and SDKs. Feel free to contact us for any inquiry.