Informace o publikaci

Corpus Annotation Pipeline for Non-standard Texts



Rok publikování 2018
Druh Článek ve sborníku
Konference Text, Speech, and Dialogue, 21st International Conference, TSD 2018
Fakulta / Pracoviště MU

Fakulta informatiky

Klíčová slova Non-standard language; Interlingual homographs; Corpora annotation
Popis According to some estimations (e.g. [9]), web corpora contain over 6% of foreign material (borrowings, language mixing, named entities). Since annotation pipelines are usually built upon standard and correct data, the resulting annotation of web corpora often contains serious errors. We studied in depth annotation errors of the web corpus czTenTen 12 and proposed an extension to the tagger desamb that had been used for czTenTen annotation. First, the subcorpus was made using the most problematic documents from czTenTen. Second, measures were established for the most frequent annotation errors. Third, we established several experiments in which we extended the annotation pipeline so it could annotate foreign material and multi-word expressions. Finally, we compared the new annotations of the subcorpus with the original ones.
Související projekty: