Binary set embedding for cross-modal retrieval

Yu, Mengyang, Liu, Li and Shao, Ling (2017) Binary set embedding for cross-modal retrieval. IEEE Transactions on Neural Networks and Learning Systems, 28 (12). pp. 2899-2910. ISSN 2162-237X

Preview

PDF (Accepted manuscript) - Accepted Version Download (2MB) | Preview

Abstract

Cross-modal retrieval is such a challenging topic that traditional global representations would fail to bridge the semantic gap between images and texts to a satisfactory level. Using local features from images and words from documents directly can be more robust for the scenario with large intraclass variations and small interclass discrepancies. In this paper, we propose a novel unsupervised binary coding algorithm called binary set embedding (BSE) to obtain meaningful hash codes for local features from the image domain and words from text domain. Understanding image features with the word vectors learned from the human language instead of the provided documents from data sets, BSE can map samples into a common Hamming space effectively and efficiently where each sample is represented by the sets of local feature descriptors from image and text domains. In particular, BSE explores relationship among local features in both feature level and image (text) level, which can balance the sensitivity of each other. Furthermore, a recursive orthogonalization procedure is applied to reduce the redundancy of codes. Extensive experiments demonstrate the superior performance of BSE compared with state-of-the-art cross-modal hashing methods using either image or text queries.

Item Type:	Article
Faculty \ School:	Faculty of Science > School of Computing Sciences
Related URLs:	http://ieeexplore.ieee.org/document/7577...
Depositing User:	Pure Connector
Date Deposited:	09 Mar 2017 01:41
Last Modified:	22 Oct 2022 02:10
URI:	https://ueaeprints.uea.ac.uk/id/eprint/62905
DOI:	10.1109/TNNLS.2016.2609463

Downloads

Actions (login required)

View Item