Love Data Week

Links:

• Voyant Tools
• The Voyant Tools Help Page
• Project Gutenberg
• Voyant Tools demo video
• Advanced Voyant Tools demo with GutenTag

Demo video: Google N-Grams and HathiTrust Bookworm

Links to the tools:

• Google N-Grams
• HathiTrust Bookworm

Computational Text Analysis, Computer-aided Text Analysis, Text Mining, and the abbreviation TDM are broad terms for searching, organizing, and analyzing large amounts of text data.

TDM can help reveal new patterns or information from a large body of work - leading to the development of new knowledge, of a larger evidence-based practice. TDM enables researchers to analyze thousands of documents and terabytes of data, allowing for a comprehensive look into research questions.

The methods used to process corpora vary widely between disciplines, and are based on insights from machine learning, statistics, computational linguistics, sociology, and many other fields. 

Examples where researchers used text analysis to answer their research question:

• Sentiment Analysis on Twitter Data mining
• Historical Newspaper Text Analysis with TDM Platforms
• Extracting text from books and text visualization of term frequencies
• Authorship Attribution / Stylometric Analysis techniques

Much of the content of this page comes from the University of Pennsylvania's Text Analysis Guide by Jajwalya Karajgikar.

Common methods of text analysis include:

1. Sentiment Analysis: Sentiment analysis employs natural language processing techniques to identify and extract subjective information from text, such as opinions and emotions expressed in the textual data, and is commonly used to analyze social media posts, customer reviews, and other text data to determine the overall sentiment.

2. Text Classification: Text classification involves categorizing text data into predefined classes or categories based on the content of the text and is frequently used for tasks such as spam filtering, topic identification, and sentiment classification.

3. Topic Modeling: Topic modeling is a statistical method used to identify topics or themes that occur in a collection of documents, allowing hidden patterns and relationships within text data to be discovered. It is widely applied in fields such as social sciences and humanities.

4. Named Entity Recognition: Named Entity Recognition (NER) is the process of identifying and extracting named entities from text, such as names of people, places, and organizations. It is commonly used for information extraction, retrieval, and data analysis.

5. Text Clustering: Text clustering is the process of grouping similar documents together based on their content, which is frequently used to identify patterns and similarities in large text datasets, particularly in fields such as marketing and customer service.

6. Text Summarization: Text summarization involves creating a concise summary of a longer text document and can be used to quickly understand the main points and themes of a large document or set of documents.

7. Text Mining: Text mining involves extracting useful information from unstructured text data using techniques such as natural language processing, machine learning, and information retrieval to discover patterns, relationships, and trends in large text datasets.

8. Named Entity Disambiguation: Named Entity Disambiguation is the process of disambiguating named entities by distinguishing between entities with similar names or referring to the same real-world entity, thereby reducing ambiguity in text data.

9. Word Frequencies: Word frequency analysis involves counting the number of times each word appears in a text document or corpus to identify common words or phrases, which can provide insights into the content of the text data.

10. Visualization: Text visualization involves creating visual representations of text data, such as word clouds, topic models, and graphs, to identify patterns, trends, and relationships in the data and communicate insights to stakeholders in a clear and concise manner.