Publication details

Large-scale similarity data management with distributed Metric Index

Authors

NOVÁK David BATKO Michal ZEZULA Pavel

Type Article in Periodical
Magazine / Source Information Processing and Management
MU Faculty or unit

Faculty of Informatics

Citation
Doi http://dx.doi.org/10.1016/j.ipm.2010.12.004
Field Informatics
Keywords Distributed data structures; Performance tuning; Similarity search; Scalability; Peer-to-peer structured networks; Metric space
Description Metric space is a universal and versatile model of similarity that can be applied in various areas of non-text information retrieval. However, a general, efficient and scalable solution for metric data management is still a resisting research challenge. In this work, we try to make an important step towards such management system that would be able to scale to data collections of billions of objects. We propose a distributed index structure for similarity data management called the Metric Index (M-Index) which can answer queries in precise and approximate manner. This technique can take advantage of any distributed hash table that supports interval queries and utilize it as an underlying index. We have performed numerous experiments to test various settings of the M-Index structure and we have proved its usability by developing a full-featured publicly-available Web application.
Related projects: