Publication details

Corpus Annotation Pipeline for Non-standard Texts



Year of publication 2018
Type Article in Proceedings
Conference Text, Speech, and Dialogue, 21st International Conference, TSD 2018
MU Faculty or unit

Faculty of Informatics

Keywords Non-standard language; Interlingual homographs; Corpora annotation
Description According to some estimations (e.g. [9]), web corpora contain over 6% of foreign material (borrowings, language mixing, named entities). Since annotation pipelines are usually built upon standard and correct data, the resulting annotation of web corpora often contains serious errors. We studied in depth annotation errors of the web corpus czTenTen 12 and proposed an extension to the tagger desamb that had been used for czTenTen annotation. First, the subcorpus was made using the most problematic documents from czTenTen. Second, measures were established for the most frequent annotation errors. Third, we established several experiments in which we extended the annotation pipeline so it could annotate foreign material and multi-word expressions. Finally, we compared the new annotations of the subcorpus with the original ones.
Related projects:

You are running an old browser version. We recommend updating your browser to its latest version.

More info