Exploiting text-related features for content-based image retrieval

G. Schroth, S. Hilsenbeck, R. Huitl, F. Schweiger, E. Steinbach

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

22 Scopus citations


Distinctive visual cues are of central importance for image retrieval applications, in particular, in the context of visual location recognition. While in indoor environments typically only few distinctive features can be found, outdoors dynamic objects and clutter significantly impair the retrieval performance. We present an approach which exploits text, a major source of information for humans during orientation and navigation, without the need for error-prone optical character recognition. To this end, characters are detected and described using robust feature descriptors like SURF. By quantizing them into several hundred visual words we consider the distinctive appearance of the characters rather than reducing the set of possible features to an alphabet. Writings in images are transformed to strings of visual words termed visual phrases, which provide significantly improved distinctiveness when compared to individual features. An approximate string matching is performed using N-grams, which can be efficiently combined with an inverted file structure to cope with large datasets. An experimental evaluation on three different datasets shows significant improvement of the retrieval performance while reducing the size of the database by two orders of magnitude compared to state-of-the-art. Its low computational complexity makes the approach particularly suited for mobile image retrieval applications.

Original languageEnglish
Title of host publicationProceedings - 2011 IEEE InternationalSymposium on Multimedia, ISM 2011
Number of pages8
StatePublished - 2011
Event13th IEEE International Symposium on Multimedia, ISM 2011 - Dana Point, CA, United States
Duration: 5 Dec 20117 Dec 2011

Publication series

NameProceedings - 2011 IEEE InternationalSymposium on Multimedia, ISM 2011


Conference13th IEEE International Symposium on Multimedia, ISM 2011
Country/TerritoryUnited States
CityDana Point, CA


  • CBIR
  • text-related visual features
  • visual location recognition


Dive into the research topics of 'Exploiting text-related features for content-based image retrieval'. Together they form a unique fingerprint.

Cite this