Data Handling and Standard Library#

Beyond core collections, Python's standard library provides battle-tested tools for time, serialization, CLI parsing, logging, and text processing. Knowing which module to reach for — and which to avoid — separates idiomatic code from reinvented wheels.

How to use this page

Reference before take-homes and tooling scripts. Basics: Basic Built-in Functions and Libraries. Files: File Handling.

At a glance


Track	Python Intermediate
Sections	15 major topics
Outline	Use the right-hand TOC to jump

Topics: Standard library mental map · datetime — dates and times · json — intermediate usage · csv — intermediate patterns · argparse — command-line interfaces · logging — not print() · configparser — INI files · collections — intermediate patterns · … (+7 more)

Standard library mental map
datetime — dates and times
json — intermediate usage
csv — intermediate patterns
argparse — command-line interfaces
logging — not print()
configparser — INI files
collections — intermediate patterns
itertools and functools — beyond basics
hashlib and secrets
statistics module
re — when string methods fail … and 3 more sections

Standard library mental map#

Need	Module	Notes
Paths	`pathlib`	Preferred over `os.path`
JSON	`json`	Interchange format
CSV	`csv`	Tabular text
Dates/times	`datetime`, `zoneinfo`	Timezone-aware in 3.9+
CLI args	`argparse`	stdlib CLI
Logging	`logging`	Structured app logs
Config INI	`configparser`	`.ini` files
Env vars	`os.environ`	Process environment
Regex	`re`	Pattern matching
Collections	`collections`	Counter, deque, etc.
Itertools	`itertools`	Combinatorics, chaining
Functools	`functools`	cache, partial
Math/stats	`math`, `statistics`	Numeric helpers
Hashing	`hashlib`	SHA-256, etc.
Secrets	`secrets`	Cryptographic random
Copy	`copy`	deep/shallow copy
Typing	`typing`, `collections.abc`	Static hints

`datetime` — dates and times#

from datetime import datetime, date, time, timedelta, timezone

now = datetime.now(timezone.utc)   # aware UTC
today = date.today()
delta = timedelta(days=7, hours=3)

future = now + delta
future.isoformat()               # '2026-07-14T03:30:00+00:00'

# Parsing ISO format
dt = datetime.fromisoformat("2026-07-07T12:00:00+05:30")

# Naive vs aware
naive = datetime.now()             # no tzinfo — avoid for global apps
aware = datetime.now(timezone.utc) # has tzinfo

`zoneinfo` (3.9+) — timezones#

from zoneinfo import ZoneInfo

ist = ZoneInfo("Asia/Kolkata")
dt = datetime(2026, 7, 7, 12, 0, tzinfo=ist)
dt.astimezone(timezone.utc)

Pitfall	Fix
Naive vs aware comparison	`TypeError` — make both aware
`datetime.today()` vs `now()`	`today()` is local date at midnight
DST gaps	Use `zoneinfo`, not fixed offsets

For interviews, datetime rarely appears unless parsing logs or timestamps.

`json` — intermediate usage#

import json
from pathlib import Path

data = {"items": [1, 2, 3], "meta": {"version": 1}}

# Pretty print
print(json.dumps(data, indent=2, sort_keys=True))

# Custom serialization
class Point:
    def __init__(self, x, y):
        self.x, self.y = x, y

def encode(obj):
    if isinstance(obj, Point):
        return {"x": obj.x, "y": obj.y}
    raise TypeError(f"not serializable: {type(obj)}")

json.dumps({"p": Point(1, 2)}, default=encode)

# Streaming large JSON — consider ijson (third-party) or line-delimited JSON

JSON numbers map to Python int/float only — no Decimal without custom encoder.

`csv` — intermediate patterns#

import csv
from io import StringIO

# In-memory CSV
buf = StringIO()
writer = csv.writer(buf)
writer.writerow(["a", "b"])
csv_text = buf.getvalue()

# Custom delimiter
reader = csv.reader(StringIO("a|b|c"), delimiter="|")

# Handle bad rows — strict in 3.12+ has more options

Always open files with newline="" — see File Handling.

`argparse` — command-line interfaces#

import argparse

def main():
    parser = argparse.ArgumentParser(description="Process input file")
    parser.add_argument("input", type=Path, help="input file path")
    parser.add_argument("-o", "--output", type=Path, help="output path")
    parser.add_argument("-v", "--verbose", action="store_true")
    parser.add_argument("-n", "--count", type=int, default=10)

    args = parser.parse_args()
    if args.verbose:
        print(f"reading {args.input}")
    process(args.input, args.output)

if __name__ == "__main__":
    main()

`action`	Effect
`"store_true"`	Flag becomes True if present
`"store"`	Default — saves value
`"append"`	Collect multiple values
`"count"`	`-vvv` → verbosity level

Useful for local test runners: python solution.py --input data.json.

Advanced: Building Command Line Interfaces.

`logging` — not print()#

import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)s %(name)s: %(message)s",
)
log = logging.getLogger(__name__)

log.debug("detail")
log.info("started")
log.warning("deprecated API")
log.error("failed")
log.exception("unexpected")   # includes traceback — inside except block

Level	Value	Use
DEBUG	10	Verbose diagnostic
INFO	20	Normal progress
WARNING	30	Something unexpected
ERROR	40	Failure
CRITICAL	50	Serious error

Why logging over print: levels, timestamps, module names, redirect to files, disable in production.

file_handler = logging.FileHandler("app.log")
log.addHandler(file_handler)

`configparser` — INI files#

; config.ini
[database]
host = localhost
port = 5432

[app]
debug = true

import configparser

config = configparser.ConfigParser()
config.read("config.ini")

host = config.get("database", "host")
port = config.getint("database", "port")
debug = config.getboolean("app", "debug")

For JSON config, use json directly. For env-based config, use os.environ.

`collections` — intermediate patterns#

Already covered in Basics — intermediate additions:

from collections import ChainMap, UserDict, UserList

# Layered config: first dict with key wins
settings = ChainMap(cli_args, env_config, defaults)

# Subclass to customize behavior
class SortedList(list):
    def append(self, item):
        super().append(item)
        self.sort()

Type	Use
`Counter`	Frequencies, multiset
`defaultdict`	Graph adjacency, grouping
`deque`	BFS, sliding window
`OrderedDict`	Explicit LRU move_to_end
`namedtuple`	Lightweight records
`ChainMap`	Scoped config lookup

`itertools` and `functools` — beyond basics#

from itertools import groupby, islice, chain, pairwise
from functools import reduce, cmp_to_key

# groupby requires sorted input for all groups
data = sorted(users, key=lambda u: u.dept)
for dept, group in groupby(data, key=lambda u: u.dept):
    print(dept, list(group))

# pairwise sliding window (3.10+)
list(pairwise([1, 2, 3, 4]))   # [(1,2), (2,3), (3,4)]

# reduce — explicit fold
from functools import reduce
product = reduce(lambda acc, x: acc * x, [1, 2, 3, 4], 1)

Prefer built-in sum, any, all, or explicit loops when clearer.

`hashlib` and `secrets`#

import hashlib

hashlib.sha256(b"data").hexdigest()

import secrets

token = secrets.token_urlsafe(32)
secrets.compare_digest(a, b)   # timing-safe string compare

Use hashlib for checksums; secrets for tokens — never random for security.

See Managing Secrets.

`statistics` module#

import statistics

statistics.mean([1, 2, 3, 4])
statistics.median([1, 2, 3, 100])
statistics.mode([1, 1, 2, 3])
statistics.stdev([2, 4, 4, 4, 5, 5, 7, 9])

For NumPy-scale numerics, use NumPy (third-party) — not in stdlib interviews unless allowed.

`re` — when string methods fail#

import re

# Common patterns
email_like = re.compile(r"[\w.+-]+@[\w-]+\.[\w.-]+")
digits = re.findall(r"\d+", "a1b22")
clean = re.sub(r"\s+", " ", text.strip())

# Verbose pattern
pattern = re.compile(r"""
    (?P<year>\d{4})
    -
    (?P<month>\d{2})
    -
    (?P<day>\d{2})
""", re.VERBOSE)

Prefer str.split, in, startswith when sufficient — regex adds complexity.

`os.environ` and `python-dotenv`#

import os

db_url = os.environ.get("DATABASE_URL", "sqlite:///local.db")
debug = os.getenv("DEBUG", "0") == "1"

# Third-party .env loading (not stdlib)
# from dotenv import load_dotenv
# load_dotenv()

Never commit secrets — use environment variables in production.

What to avoid or use carefully#

Module	Caution
`pickle`	Never load untrusted data
`eval` / `exec`	Code injection risk
`shell=True` in subprocess	Shell injection
`random`	Not cryptographic — use `secrets`
Global mutable state	Hard to test

Decision guide#

Read/write paths?        → pathlib
JSON config?             → json + Path.read_text
Tabular export?          → csv
CLI for local testing?   → argparse
Debug production code?   → logging
Count/group/graph?       → collections
Time with timezone?      → datetime + zoneinfo
Secure token?            → secrets
Parse complex text?      → re (or str methods first)

Interview traps (quick reference)#

Trap	What goes wrong	Safe approach
Naive datetime compare	TypeError or wrong ordering	Use aware UTC
JSON set/tuple	TypeError on dump	Convert to list
csv extra newlines	Missing `newline=""`	Follow csv docs
print in production	No levels/files	logging
pickle from network	RCE vulnerability	JSON only
groupby unsorted	Incomplete groups	Sort first

Mental model checklist#

When is pathlib preferred over os.path?
What types does JSON support natively?
What is the difference between logging.info and print?
Why must CSV files use newline=""?
When do you need zoneinfo vs plain datetime?

What's next#

Topic	Page
File formats in depth	File Handling
Modules	Modules and Packages
Advanced file I/O	File IO
CLI apps	Building Command Line Interfaces

Data Handling and Standard Library#

Standard library mental map#

datetime — dates and times#

zoneinfo (3.9+) — timezones#

json — intermediate usage#

csv — intermediate patterns#

argparse — command-line interfaces#

logging — not print()#

configparser — INI files#

collections — intermediate patterns#

itertools and functools — beyond basics#

hashlib and secrets#

statistics module#

re — when string methods fail#

os.environ and python-dotenv#