Skip to main content
Skip header

Methods of Analysis of Textual Data

Type of study Follow-up Master
Language of instruction English
Code 460-4074/02
Abbreviation MATD
Course title Methods of Analysis of Textual Data
Credits 4
Coordinating department Department of Computer Science
Course coordinator doc. Mgr. Jiří Dvorský, Ph.D.

Subject syllabus

A brief outline of the lectures' topics:
- Introduction to information systems. The history and evolution of text retrieval. Differences between database systems and information retrieval (IR) systems. The general model of information retrieval system.
- Pattern matching. One sample pattern matching. Aho-Corasick algorithm. Regular expressions, finite automata. Algorithms for approximate pattern matching.
- Suffix trees. DAWG. Patricia and similar data structures.
- Primary processing of texts. Lexical analysis. Stemming. Lemmatization. Stop words.
- Construction of index systems. Zipf law and the estimated size of the index system. Indexing based on classification. Positional index systems. Methods for weighting terms. TF-IDF weight terms. Methods of compression index systems. Methods for encoding natural numbers.
- Query Languages​​. Relevance document. The degree of similarity between pairs of document-query. Relevance vs. similarity. The structure and query evaluation. Boolean DIS. IR system evaluation (accuracy, completeness, F-measure).
- Signature methods. Chained and layered coding signatures. Efficient evaluation of queries.
- Latent semantics. Methods for dimension reduction. Methods based on matrix decomposition. Random projection. Vector DIS. Construction and evaluation of the query vector. Other types of DIS (extended Boolean). Indexing, query structure, evaluation questions.
- Search the site. Analysis of hypertext documents, structural methods. PageRank and HITS. Metasearch and cooperative search. Application of computational intelligence and soft computing in processing a text search.
- Methods for automatic summarization: abstraction and extraction. Detection and evolution theme. Sentiment analysis, classification and clustering of documents.
- Parallel and distributed search. Decentralized P2P and search.
- Semantic and contextual search technology Hummingbird, Snapshot (Satori) and Graph Search.

Literature

1. Manning, C. D.; Raghavan, P. & Schutze, H. Introduction to Information Retrieval, Cambridge University Press, 2008
2. Witten I. H., Moffat A., Bell T. C.: Managing Gigabytes (2nd ed.): Compressing and Indexing Documents and Images, Morgan Kaufmann Publishers Inc., 1999, ISBN 1-55860-570-3 
3. Baeza-Yates R. A., Ribeiro-Neto B.: Modern Information Retrieval, Addison-Wesley Longman Publishing Co., Inc., 1999, ISBN 020139829X 
4. Feldman R., Sanger J.: The Text Mining Handbook: Advanced Approaches in Analyzing Unstructured Data, Cambridge University Press, 2006, ISBN 978-0521836579 
5. Berry M. W., Kogan J.: Text Mining: Applications and Theory, Wiley, 2010, ISBN 978-0470749821 
6. Weiss S. M., Indurkhya N., Zhang T.: Fundamentals of Predictive Text Mining, Springer, 2010, ISBN 978-1849962254 
7. Langville, A. N. & Meyer, C. D. Google's PageRank and Beyond: The Science of Search Engine Rankings Princeton University Press, 2006
8. Korfhage, R. R. Information Storage and Retrieval, John Wiley & Sons, 1997

Advised literature

1. Witten, I. H.; Gori, M. & Numerico, T. Web Dragons: Inside the Myths of Search Engine Technology, Morgan Kaufmann, 2006