Enhancing Content-Based Retrieval Through an End-to-End Approach Utilizing Deep Learning and Multidimensional Indexing

Abstract

Recent advancements in technology, coupled with reductions in hardware and software costs, have propelled visual search applications into the spotlight, making them both popular and indispensable. Consequently, the rapid and precise retrieval of images from vast databases through image queries has become a critical task. We introduce a novel end-to-end retrieval architecture that significantly enhances retrieval performance when compared to a baseline system that conducts database searches at the video frame level. Leveraging a pre-trained convolutional neural network model, we employ unsupervised image retrieval processes to extract and store low-level features for efficient indexing. To facilitate swift and effective access, we implement a tree-based indexing structure known as VP-Tree. This structure utilizes the extracted low-level features. To make these features compatible with our system, we employ dimension reduction techniques to represent them in a lower-dimensional space. Our experiments, conducted on three benchmark datasets, demonstrate that VP-Tree consistently outperforms k-nearest neighbor (KNN) search in terms of retrieval accuracy and efficiency. Specifically, for image data set, VP-Tree achieves a precision of 56.3903, an F1-score of 68.703, and an area under the curve (AUC) of 93.518719, all slightly surpassing KNN. Similarly, for news video data set, VP-Tree attains a precision of 38.704011, an F1-score of 55.029674, and an AUC of 64.6412, again outperforming KNN. For documentary data set, VP-Tree achieves a notable improvement with a precision of 73.511723, an F1-score of 84.734013, and an AUC of 80.981328, demonstrating superior performance over KNN. In addition to accuracy, we evaluated retrieval time across different dataset sizes. While KNN performs slightly faster on smaller datasets, VP-Tree scales significantly better as dataset size increases. For 100,000 images, VP-Tree reduces retrieval time from 79.77 to 54.34 ms, and for 200,000 images, it improves performance from 108.75 to 44.63 ms, confirming its efficiency in large-scale retrieval scenarios. These results highlight VP-Tree as a robust and scalable alternative to traditional KNN-based methods, ensuring both accuracy and efficiency in large-scale image retrieval tasks.