Transformer Affinity for Tracking: Efficient reidentification of anchor-based detections in non-constant frame rate conditions
Résumé
Multiple object tracking (MOT) is an essential task in computer vision. It comprises a motion prediction module and an appearance-based reidentification mechanism. Relying only on motion prediction without incorporating appearance features results in identity switches, whereas using only appearance without motion prediction leads to identity merges. In this work, we are targeting real-time object detection and tracking in RGB images in an energy-restricted environment. The challenge in such a task is the uneven processing time between frames processed by the system, depending on the time required on each frame for detection and matching operations. As a result, some frames may be skipped until the system is ready to process a new one even if the camera streams at a fixed frame rate. Besides, when objects move in the camera's direction, they can change scale, making intra-scale association insufficient. To tackle these issues, we propose Transformer Affinity for Tracking (TransAT), which unifies the motion prediction and appearance reidentification in a single network and performs cross-scale matching. Our experiments validate our contributions and show that TransAT achieves successful reidentification of objects even when they change scales. Moreover, our method is robust to a drastic change in the frame rate, unlike DeepSORT, StrongSORT, and RetinaTrack.
| Origine | Fichiers produits par l'(les) auteur(s) |
|---|---|
| Licence |