Bridging Domains: The Power of Fine-Tuned Visual-Text Alignment

The goal of domain-adaptive object detection (DAOD) is to help detectors work well in new, unlabeled areas by reducing the bias that comes from the original, labeled areas. Recent work has shown that vision-language models (VLMs) can be very helpful in this process. However, there are some issues that need to be addressed. Most existing methods focus on single-domain detection, which is different from the multi-domain tasks in DAOD. This can make it hard to align visual and text features across different domains in a detailed way. Some initial solutions might also overlook the importance of relational reasoning in prompts and the interaction between different types of information, which are key for fine-grained alignment. To tackle these challenges, a new approach called FGPro has been developed. This method focuses on fine-grained visual-text feature alignment in DAOD using prompt tuning. FGPro has three main levels. At the prompt level, a learnable domain-adaptive prompt is created, and a prompt relation encoder is built to understand the relationships between different parts of the prompt. At the model level, a bidirectional cross-modal attention mechanism is used to fully interact visual and textual fine-grained information. Additionally, a prompt-guided cross-domain regularization strategy is customized to inject domain-invariant and domain-specific information into prompts in a separate way. These designs help to align the fine-grained visual-text features of the source and target domains, making it easier to capture domain-aware information.

The effectiveness of FGPro has been tested in four different cross-domain scenarios. The results show notable performance improvements over existing methods. For example, in cross-weather scenarios, the average precision at 50% IoU (AP50) increased by 1. 0%. In simulation-to-real scenarios, the improvement was 1. 2% AP50. For cross-camera scenarios, the increase was 1. 3% AP50, and in industry settings, the improvement was 2. 8% AP50. These results validate the effectiveness of FGPro's fine-grained alignment. The success of FGPro highlights the importance of fine-grained visual-text feature alignment in DAOD. By addressing the paradigm discrepancies and focusing on relational reasoning and cross-modal information interactions, FGPro provides a novel and effective solution. The results demonstrate that this approach can significantly improve the performance of object detection in various cross-domain scenarios.

Actions