Informace o publikaci
ScaleText
| Autoři | |
|---|---|
| Rok publikování | 2016 |
| Druh | Software |
| www | Repositář projektu (neveřejný, přístup na vyžádání vůči podpisu NDA) |
| Popis | ScaleText version 0.7 is an early prototype of an innovative software for scalable semantic text search, aimed at evaluating the performance and technical feasibility of the project. The core of this result is a database engine, realized as a stand-alone package in the Python language, that implements document indexing and search using vectors for text representation. The vectors are created automatically from plain text using several methods for semantic analysis: LSI, LDA, TF-IDF. The documents go through several stages, from preprocessing, segmentation, vectorization to vector encoding and storage. Each step is realized by a dedicated component, with its output backed by a backend database engine for persistence. See the associated publication [RYGL, Jan, Petr SOJKA, Michal RŮŽIČKA and Radim ŘEHŮŘEK. ScaleText: The Design of a Scalable, Adaptable and User-Friendly Document System for Similarity Searches : Digging for Nuggets of Wisdom in Text. In Aleš Horák, Pavel Rychlý, Adam Rambousek. Proceedings of the Tenth Workshop on Recent Advances in Slavonic Natural Language Processing, RASLAN 2016. Brno: Tribun EU, 2016. p. 79-87, 9 pp. ISBN 978-80-263-1095-2.] for a deeper description of the design methodology, APIs and data flow. Release 0.7 includes the definition of class interfaces and dependencies, plus their instantiation using concrete algorithms of LSI and LDA, and a concrete database backend of Gensim, making the system fully end-to-end executable. |
| Související projekty: |