M-LSD : Machine Learning Model for Detecting Wireframes

Overview

M-LSD is a machine learning model developed by NAVER to detect wireframes of objects. Since it can accurately detect the contours of sheets of paper and books, it can be used for pre-processing of OCR.

Towards Real-time and Light-weight Line Segment Detection

Architecture

The classic approach to line detection is complex and made of with multiple modules, whereas M-LSD detects lines in a single shot, which allows for fast processing.

The model uses MobileNetV2 as backbone, with the addition of a block for generating heatmaps in the later stage.

Line segments are defined as Tri-Points (TP), as shown below. The line segment is defined by lc, which indicates the center point, ds which is the displacement vector to the start point, and de which is the displacement vector to the end point.

The output of the model is a (1,200,2) vector lc representing the center point of the line segments, a (1,200) vector which is the confidence of the line segments, and a displacement map (1,256,256,4) representing the displacement from the center point to the start and end points of the line segments. Line segments can then be calculated by adding the three vectors of center points, start points, and end points.

The data sets used for training are Wireframe and YorkUrban.

Usage

M-LSD can be used with ailia SDK 1.2.8 and later with the following command to detect wireframes from the webcam video stream.

$ python3 mlsd.py -v 0

ailia-models/line_segment_detection/mlsd at master · axinc-ai/ailia-models

Here is the result you can expect.

ax Inc. has developed ailia SDK, which enables cross-platform, GPU-based rapid inference.

ax Inc. provides a wide range of services from consulting and model creation, to the development of AI-based applications and SDKs. Feel free to contact us for any inquiry.