Python Standard Library Overview: Which Module to Use

What is in the Python standard library?

The Python standard library is the set of modules that ships with every Python installation, the "batteries included". It covers text (re, textwrap), data structures (collections, itertools), numbers (math, statistics, decimal), dates (datetime, zoneinfo), files (pathlib, shutil), formats (json, csv, sqlite3), concurrency (asyncio, threading), networking and testing (unittest).

Lesson

"Batteries included" is Python's oldest slogan, and the batteries are the reason a Python program that parses a date, reads a CSV, hashes a file and serves it over HTTP is a hundred lines rather than a thousand. The skill is knowing the map: which module holds the thing you need, so that you look there before writing it yourself. This lesson is that map — the modules grouped by job, with the one or two functions each is used for — plus how to read the documentation and how to explore a module from the REPL. Later modules of this track go deep on the areas that matter most; this one is the index.

Text

Data structures and functional tools

Numbers

Dates and times

Files, paths and the OS

Data formats and persistence

Concurrency and networking

Language, testing and tooling

Reading the documentation

Every module's page at docs.python.org/3/library/ follows one shape: a summary, the functions and classes with their signatures and version notes ("Changed in version 3.10"), then examples. Read the signature, then the version note if you support older interpreters, then the example. In the REPL, help(module) prints the docstrings, dir(module) lists the names, and help(module.function) shows the signature — enough for most questions without leaving the terminal. For a function whose behaviour you are unsure of, the fastest check is to call it on a small value and look.

The habit

Before writing a helper, ask: is this a text job (textwrap, difflib), a collection job (collections, itertools), a number job (math, statistics), a file job (pathlib, shutil), a format job (json, csv)? Ten minutes with the map saves an hour and a bug: the library's version has been tested on the edge cases you have not thought of yet.

Pitfalls

• Writing a mean, a deque, a permutations generator or a CSV parser by hand.
• pickle.load on data from outside your program.
• time.time where perf_counter (timing) or monotonic (timeouts) was meant.
• os.path string juggling where pathlib reads better.
• Reaching for a third-party package before checking the standard library.
• Guessing at behaviour instead of a two-line REPL experiment.

Key takeaways

• The standard library covers text, collections, numbers, dates, files, formats, concurrency, networking, testing and tooling; know the map before writing helpers.
• collections, itertools, functools, pathlib, json, csv, datetime, re, math, logging, unittest are the modules every program touches.
• help(), dir() and a small experiment in the REPL answer most questions; the docs show signature, version notes, examples.
• pickle is for trusted data only; secrets for anything security-related; perf_counter for timing.
• Prefer the library's tested implementation to a hand-written one.

Common questions

What does "batteries included" mean in Python?

It is Python's long-standing slogan for its large standard library: an installation already includes modules for parsing dates, reading CSV and JSON, hashing files, regular expressions, SQLite databases and serving HTTP, so many programs need no third-party packages at all.

How do I find what functions a Python module has?

In the REPL, dir(module) lists its names, help(module) prints its docstrings and help(module.function) shows one signature. The documentation at docs.python.org/3/library/ gives each module's functions with signatures, version notes such as "Changed in version 3.10" and examples. A two-line experiment on a small value settles the rest.

Is pickle safe to use in Python?

Only for data your own program wrote. Unpickling can run arbitrary code while it rebuilds objects, so never call pickle.load on data from outside a trust boundary. Use json for data exchanged with other programs, and tomllib or configparser for configuration files.

Which Python module should I use to time code?

time.perf_counter() for measuring elapsed time, since it has the highest available resolution and is not affected by changes to the system clock; time.monotonic() for timeouts; time.time() only for wall-clock timestamps. For benchmarking small snippets, the timeit module runs the code many times.

How do I run a shell command from Python?

Use subprocess.run with the command as a list of arguments, for example subprocess.run(["ls", "-l"], capture_output=True, text=True), which returns the exit code and the captured output. Passing a list rather than a string built from input avoids shell injection.

Exercises

import os.path as osp
from collections import Counter
import numpy as np, json
from shop.models import Product

stdlib: collections json os
other: numpy shop
statements 4

math is

callable: isclose isfinite isinf isnan isqrt
other: (none)

In this module: Modules, packages and imports

• Modules and imports — files, namespaces and sys.modules
• Packages — directories, __init__.py, relative imports and project layout
• The standard library map — where to look before you write it (this lesson)
• Virtual environments and packaging — venv, pip and pyproject.toml
• Scripts and the command line — argv, argparse, exit codes and streams
• Checkpoint — Modules, packages and imports