{"id":2512,"date":"2021-10-25T09:00:30","date_gmt":"2021-10-25T01:00:30","guid":{"rendered":"https:\/\/blog.ailia.ai\/uncategorized\/bytetrack-tracking-model-that-also-takes-into-account-the-low-accuracy-boundingbox\/"},"modified":"2025-05-20T17:41:26","modified_gmt":"2025-05-20T09:41:26","slug":"bytetrack-tracking-model-that-also-takes-into-account-the-low-accuracy-boundingbox","status":"publish","type":"post","link":"https:\/\/blog.ailia.ai\/en\/tips-en\/bytetrack-tracking-model-that-also-takes-into-account-the-low-accuracy-boundingbox\/","title":{"rendered":"ByteTrack : Tracking model that also considers low accuracy bounding boxes"},"content":{"rendered":"\n

Overview<\/strong><\/h3>\n\n\n\n

ByteTrack<\/em>\u00a0is a model for object tracking published in October 2021. By applying\u00a0ByteTrack\u00a0<\/em>to the bounding box of people detected by\u00a0YOLOX<\/em><\/a>, you can assign a unique ID to each person.\u00a0ByteTrack<\/em>\u00a0is currently the state-of-the-art and outperforms\u00a0SiamMOT\u00a0<\/em>and transformer-based tracking models.<\/p>\n\n\n\n

\"\"
Source: https:\/\/github.com\/ifzhang\/ByteTrack<\/a><\/figcaption><\/figure>\n\n\n\n

<\/a>ByteTrack: Multi-Object Tracking by Associating Every Detection Box<\/a><\/p>\n\n\n\n

<\/a>GitHub – ifzhang\/ByteTrack<\/a><\/p>\n\n\n\n

Architecture<\/strong><\/h3>\n\n\n\n

In Multi-Object Tracking (MOT), object detection is first performed using models such as YOLOX<\/em><\/a>, and a tracking algorithm is used to track objects in-between frames. However, in real-world applications, the result of object detection is sometimes incomplete, resulting in objects being ignored.<\/p>\n\n\n\n

Most object detection algorithms ignore bounding boxes with low confidence values. This is because there is a trade-off since accepting bounding boxes with low confidence values will improve the detection rate (True Positive), but will also cause False Positive.<\/p>\n\n\n\n

However, the question whether all bounding boxes with low confidence values should be removed or not is relevant. Even with a low confidence value, the object may still exist, and ignoring it would decrease the efficiency of the tracking model.<\/p>\n\n\n\n

The following figure illustrates this problem. In frame\u00a0t1<\/code>, four people with confidence values above 0.5 are tracked. However, at framest2<\/code>\u00a0and\u00a0t3<\/code>, the score of the person with the red bounding box drops from 0.8 to 0.4 and then further down from 0.4 to 0.1 due to occlusion. As a result, this person is ignored.<\/p>\n\n\n\n

\"\"
Source:\u00a0https:\/\/arxiv.org\/pdf\/2110.06864.pdf<\/a><\/figcaption><\/figure>\n\n\n\n

ByteTrack <\/em>solves this problem by using a motion model that manages a queue called tracklets<\/em> to store objects being tracked, and performs tracking and matching between bounding boxes with low confidence values.<\/p>\n\n\n\n

In the matching process, an algorithm called BYTE<\/em> is used. First, the positions in the next frame of objects in the tracklets<\/em> are predicted using the Kalman filter<\/em>, then they are matched with high-score detected bounding boxes using motion similarity<\/em>. With motion similality<\/em>, the score is computed by Interaction over Union (IoU), which indicates the amount of overlap between objects (step (b) in the above image shows the results of this first matching).<\/p>\n\n\n\n

Next, the algorithm performs a second matching. Objects in the tracklets <\/em>that could not be matched (eg. red boxes in the previous image), are then matched with detected bounding boxes with lower confidence values (step (c) in the above image shows the results of this second matching).<\/p>\n\n\n\n

The details of the algorithm are described below. It is a simple tracking algorithm using Kalman filter, therefore it is very fast.<\/p>\n\n\n\n

\"\"
Source: https:\/\/arxiv.org\/pdf\/2110.06864.pdf<\/a><\/figcaption><\/figure>\n\n\n\n

Despite the simplicity of the method,\u00a0ByteTrack\u00a0<\/em>achieves SoTA object tracking.<\/p>\n\n\n\n

\"\"
Source:\u00a0https:\/\/arxiv.org\/pdf\/2110.06864.pdf<\/a><\/figcaption><\/figure>\n\n\n\n

Here is an example of detection in each benchmark.<\/p>\n\n\n\n

\"\"
Source: https:\/\/arxiv.org\/pdf\/2110.06864.pdf<\/a><\/figcaption><\/figure>\n\n\n\n

And a performance comparison with conventional methods.<\/p>\n\n\n\n

\"\"
Source: https:\/\/arxiv.org\/pdf\/2110.06864.pdf<\/a><\/figcaption><\/figure>\n\n\n\n

The object detection step is based on\u00a0YOLOX<\/em><\/a>\u00a0trained on the\u00a0MOT19\u00a0<\/em>and\u00a0MOT20<\/em>\u00a0datasets, where the recognition resolution is 1440×800 for\u00a0MOT17\u00a0<\/em>and 1600×896 for\u00a0MOT20<\/em>. Therefore,\u00a0ByteTrack\u00a0<\/em>itself runs fast, but the object detection processing time is rather high.<\/p>\n\n\n\n

Comparison with DeepSort<\/strong><\/h3>\n\n\n\n

DeepSort<\/em><\/a> <\/em>uses ReID<\/em> identification model to link bounding boxes of detected people between frames, and for those who could not be linked, Sort<\/em> uses the prediction of bounding box movement calculated by Kalman filter to link them between frames. However, this is only done for bounding boxes with high confidence values.<\/p>\n\n\n\n

ByteTrack <\/em>does not use ReID<\/em>, but uses only the movement prediction of bounding boxes calculated using the Kalman filter to track people between frames. Therefore, it is technically similar to Sort<\/em> step used in DeepSort<\/em>. However, performance have been improved by splitting the processing in two steps, the first one targeting the bounding boxes with high confidence values, the second one for the ones with low confidence values.<\/p>\n\n\n\n

Usage<\/strong><\/h3>\n\n\n\n

ByteTrack<\/em>\u00a0can be used with ailia SDK by running the following command.<\/p>\n\n\n\n

$ python3 bytetrack.py -v 0<\/code><\/p>\n\n\n\n

You will need to install the\u00a0lap<\/code>\u00a0library as a dependency.<\/p>\n\n\n\n

pip3 install lap<\/code><\/p>\n\n\n\n

To run faster, use the\u00a0-m<\/code>\u00a0option to swap the object detection model for a lighter version of YOLOX, for example\u00a0yolox_s<\/code>\u00a0in the command below.<\/p>\n\n\n\n

$ python3 bytetrack.py -v 0 -m yolox_s<\/code><\/p>\n\n\n\n

<\/a>ailia-models\/object_tracking\/bytetrack<\/a><\/p>\n\n\n\n

ax Inc.<\/a> has developed ailia SDK<\/a>, which enables cross-platform, GPU-based rapid inference.<\/p>\n\n\n\n

ax Inc. provides a wide range of services from consulting and model creation, to the development of AI-based applications and SDKs. Feel free to\u00a0contact us<\/a>\u00a0for any inquiry.<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Overview ByteTrack\u00a0is a model for object tracking published in October 2021. By applying\u00a0ByteTrack\u00a0to the bounding box of people detected by\u00a0YOLOX, you can assign a unique ID to each person.\u00a0ByteTrack\u00a0is currently the state-of-the-art and outperforms\u00a0SiamMOT\u00a0and transformer-based tracking models. ByteTrack: Multi-Object Tracking by Associating Every Detection Box GitHub – ifzhang\/ByteTrack Architecture In Multi-Object Tracking (MOT), object detection is first performed using models such as YOLOX, and a tracking algorithm is used to track objects in-between frames. However, in real-world applications, the result of object detection is sometimes incomplete, resulting in objects being ignored. Most object detection algorithms ignore bounding boxes with low confidence values. This is because there is a trade-off since accepting […]<\/p>\n","protected":false},"author":5,"featured_media":2440,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[255],"tags":[266],"class_list":["post-2512","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tips-en","tag-ailiamodels-en"],"acf":[],"_links":{"self":[{"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/posts\/2512","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/comments?post=2512"}],"version-history":[{"count":1,"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/posts\/2512\/revisions"}],"predecessor-version":[{"id":2518,"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/posts\/2512\/revisions\/2518"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/media\/2440"}],"wp:attachment":[{"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/media?parent=2512"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/categories?post=2512"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.ailia.ai\/en\/wp-json\/wp\/v2\/tags?post=2512"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}