1.8 It’s the Libraries!

  • Existing libraries to help you avoid “reinventing the wheel”
  • Leverage your program-development efforts
  • Perform significant tasks with modest amounts of code

1.8.1 Python Standard Library

  • The Python Standard Library provides rich capabilities for
    • text/binary data processing, mathematics
    • functional-style programming
    • file/directory access
    • data persistence
    • data compression/archiving
    • cryptography
    • operating-system services
    • concurrent programming
    • interprocess communication
    • networking protocols
    • JSON/XML/other Internet data formats
    • multimedia
    • internationalization
    • GUI
    • debugging
    • profiling
    • and more
Some of the Python Standard Library modules we use in the book
collections—Additional data structures beyond lists, tuples, dictionaries and sets.
csv—Processing comma-separated value files.
datetime, time—Date and time manipulations.
decimal—Fixed-point and floating-point arithmetic, including monetary calculations.
doctest—Simple unit testing via validation tests and expected results embedded in docstrings.
json—JavaScript Object Notation (JSON) processing for use with web services and NoSQL document databases.
math—Common math constants and operations.
os—Interacting with the operating system.
queue—First-in, first-out data structure.
random—Pseudorandom numbers.
re—Regular expressions for pattern matching.
sqlite3—SQLite relational database access.
statistics—Mathematical statistics functions like mean, median, mode and variance.
string—String processing.
sys—Command-line argument processing; standard input, standard output and standard error streams.
timeit—Performance analysis.

1.8.2 Data-Science Libraries

  • Enormous and rapidly growing community of open-source developers in many fields
  • One of the biggest reasons for Python’s popularity is the extraordinary range of open-source libraries developed by its open-source community
  • The following table lists various popular data-science libraries
Popular Python libraries used in data science
Scientific Computing and Statistics
NumPy (Numerical Python)—Python does not have a built-in array data structure. It uses lists, which are convenient but relatively slow. NumPy provides the high-performance ndarray data structure to represent lists and matrices, and it also provides routines for processing such data structures.
SciPy (Scientific Python)—Built on NumPy, SciPy adds routines for scientific processing, such as integrals, differential equations, additional matrix processing and more. scipy.org controls SciPy and NumPy.
StatsModels—Provides support for estimations of statistical models, statistical tests and statistical data exploration.
Data Manipulation and Analysis
Pandas—An extremely popular library for data manipulations. Pandas makes abundant use of NumPy’s ndarray. Its two key data structures are Series (one dimensional) and DataFrames (two dimensional).
Visualization
Matplotlib—A highly customizable visualization and plotting library. Supported plots include regular, scatter, bar, contour, pie, quiver, grid, polar axis, 3D and text.
Seaborn—A higher-level visualization library built on Matplotlib. Seaborn adds a nicer look-and-feel, additional visualizations and enables you to create visualizations with less code.
Machine Learning, Deep Learning and Reinforcement Learning
scikit-learn—Top machine-learning library. Machine learning is a subset of AI. Deep learning is a subset of machine learning that focuses on neural networks.
Keras—One of the easiest to use deep-learning libraries. Keras runs on top of TensorFlow (Google), CNTK (Microsoft’s cognitive toolkit for deep learning) or Theano (Université de Montréal).
TensorFlow—From Google, this is the most widely used deep learning library. TensorFlow works with GPUs (graphics processing units) or Google’s custom TPUs (Tensor processing units) for performance. TensorFlow is important in AI and big data analytics—where processing demands are huge. You’ll use the version of Keras that’s built into TensorFlow.
OpenAI Gym—A library and environment for developing, testing and comparing reinforcement-learning algorithms.
Natural Language Processing (NLP)
NLTK (Natural Language Toolkit)—Used for natural language processing (NLP) tasks.
TextBlob—An object-oriented NLP text-processing library built on the NLTK and pattern NLP libraries. TextBlob simplifies many NLP tasks.
Gensim—Similar to NLTK. Commonly used to build an index for a collection of documents, then determine how similar another document is to each of those in the index.

©1992–2020 by Pearson Education, Inc. All Rights Reserved. This content is based on Chapter 1 of the book Intro to Python for Computer Science and Data Science: Learning to Program with AI, Big Data and the Cloud.

DISCLAIMER: The authors and publisher of this book have used their best efforts in preparing the book. These efforts include the development, research, and testing of the theories and programs to determine their effectiveness. The authors and publisher make no warranty of any kind, expressed or implied, with regard to these programs or to the documentation contained in these books. The authors and publisher shall not be liable in any event for incidental or consequential damages in connection with, or arising out of, the furnishing, performance, or use of these programs.