# 12. Natural Language Processing (NLP)¶

# Objectives¶

- Perform NLP tasks fundamental to many of our later case studies
- TextBlob, NLTK, Textatistic and spaCy libraries
- Tokenize text into words and sentences
- Parts-of-speech tagging
- Sentiment analysis for determining whether text is positive, negative or neutral
- Detect the language of text and translate between languages

# Objectives (cont.)¶

- Word roots via stemming and lemmatization
- Spell checking and correction
- Word definitions, synonyms and antonyms ===CURRENTLY DELETED AND IN DUMP FILE===
- Remove stop words from text
- Create word cloud visualizations
- Readability assessment
- Named entity recognition and similarity detection

- 12.1 Introduction
- 12.2 TextBlob
  - 12.2.1 Create a TextBlob — Sections 12.2.1-7 are in one notebook
  - 12.2.2 Tokenizing Text into Sentences and Words
  - 12.2.3 Parts-of-Speech Tagging
  - 12.2.4 Extracting Noun Phrases
  - 12.2.5 Sentiment Analysis with TextBlob’s Default Sentiment Analyzer
  - 12.2.6 Sentiment Analysis with the NaiveBayesAnalyzer
  - 12.2.7 Language Detection and Translation
  - 12.2.8 Inflection: Pluralization and Singularization
  - 12.2.9 Spell Checking and Correction
  - 12.2.10 Normalization: Stemming and Lemmatization
  - 12.2.11 Word Frequencies
  - 12.2.12 Getting Definitions, Synonyms and Antonyms from WordNet
  - 12.2.13 Deleting Stop Words
  - 12.2.14 n-grams
- 12.3 Visualizing Word Frequencies with Bar Charts and Word Clouds
  - 12.3.1 Visualizing Word Frequencies with Pandas
  - 12.3.2 Visualizing Word Frequencies with Word Clouds
- 12.4 Readability Assessment with Textatistic
- 12.5 Named Entity Recognition with spaCy
- 12.6 Similarity Detection with spaCy
- 12.7 Other NLP Libraries and Tools
- 12.8 Machine Learning and Deep Learning Natural Language Applications
- 12.9 Natural Language Datasets
- 12.10 Wrap-Up
- Exercises

©1992–2020 by Pearson Education, Inc. All Rights Reserved. This content is based on Chapter 5 of the book Intro to Python for Computer Science and Data Science: Learning to Program with AI, Big Data and the Cloud.

DISCLAIMER: The authors and publisher of this book have used their best efforts in preparing the book. These efforts include the development, research, and testing of the theories and programs to determine their effectiveness. The authors and publisher make no warranty of any kind, expressed or implied, with regard to these programs or to the documentation contained in these books. The authors and publisher shall not be liable in any event for incidental or consequential damages in connection with, or arising out of, the furnishing, performance, or use of these programs.