Text as Data

Textual data have rapidly gained attention in the communication, social, and political sciences. Without their discussion, a companion to data management would be incomplete. This chapter starts with a discussion of basic operations on character strings, such as concatenation, search and replace. It then moves on to the management of corpora of text. It also discusses routine issues in the management of textual data, such as stemming, stop-word deletion, and the creation of term-frequency matrices.

Below is the supporting material for the various sections of the chapter.

Character Strings

Basic operations on character strings

Counting words in the UK party manifesto on occasion of the 2017 election

Text Corpora with the tm Package

A simple example for the usage of the tm package

An example for the use of tm on data from the Manifesto Project

Improvements Provided by the quanteda Package

A basic example for the usage of the quanteda package

Obtaining tokens in text documents using quanteda

Preparing Manifesto Project data using quanteda