File Handling#

Reading and writing files is essential for take-home assignments, local test harnesses, and parsing input. Modern Python favors pathlib over string paths and with for reliable cleanup. This page covers text/binary I/O, common formats, and error patterns.

How to use this page

Use pathlib for paths, json/csv for structured data. Error patterns: Error Handling. Advanced I/O: File IO.

At a glance


Track	Python Intermediate
Sections	11 major topics
Outline	Use the right-hand TOC to jump

Topics: pathlib — object-oriented paths (preferred) · Opening files with open() · Text vs binary · JSON · CSV · Temporary files and directories · File system operations · Common errors and handling · … (+3 more)

pathlib — object-oriented paths (preferred)
Opening files with open()
Text vs binary
JSON
CSV
Temporary files and directories
File system operations
Common errors and handling
with statement — why it matters
Pickle — caution
Practical take-home layout

`pathlib` — object-oriented paths (preferred)#

from pathlib import Path

root = Path("/Users/me/project")
data_file = root / "data" / "input.txt"

data_file.exists()
data_file.is_file()
data_file.is_dir()
data_file.read_text(encoding="utf-8")
data_file.write_text("hello\n", encoding="utf-8")
data_file.read_bytes()
data_file.write_bytes(b"\x00\x01")

for path in root.glob("**/*.py"):   # recursive
    print(path)

for path in root.iterdir():
    print(path.name, path.suffix)

`str` path	`Path` equivalent
`os.path.join(a, b)`	`Path(a) / b`
`os.path.exists(p)`	`Path(p).exists()`
`open(p)`	`Path(p).open()` or `.read_text()`

Use Path in new code — clearer and cross-platform.

Opening files with `open()`#

from pathlib import Path

path = Path("data.txt")

with path.open("r", encoding="utf-8") as f:
    content = f.read()

with path.open("w", encoding="utf-8") as f:
    f.write("line1\n")
    f.writelines(["line2\n", "line3\n"])

Mode reference#

Mode	Meaning
`"r"`	Read text (default)
`"w"`	Write text — truncates existing file
`"x"`	Exclusive create — fails if exists
`"a"`	Append text
`"r+"`	Read and write
`"rb"`, `"wb"`, `"ab"`	Binary read/write/append
`"w+"`, `"a+"`	Read + write variants

Always specify encoding="utf-8" for text on Python 3 — platform default may differ on Windows.

Reading strategies#

# Entire file — fine for small files
text = path.read_text(encoding="utf-8")

# Line by line — memory efficient
with path.open(encoding="utf-8") as f:
    for line in f:
        process(line.rstrip("\n"))

# All lines as list
lines = path.read_text(encoding="utf-8").splitlines()

# Fixed-size chunks — large/binary files
with path.open("rb") as f:
    while chunk := f.read(8192):
        process(chunk)

Method	Memory	Use
`.read()`	Whole file	Small files
Iterate lines	O(1) extra	Logs, large text
`.read(n)` chunks	Bounded buffer	Binary / streaming

Text vs binary#

	Text mode (`"r"`, `"w"`)	Binary mode (`"rb"`, `"wb"`)
Type	`str`	`bytes`
Encoding	Required concept	Raw bytes
Newlines	Universal newline translation	No translation
Use	Source code, JSON, CSV	Images, pickles, compressed

data = "café"
b = data.encode("utf-8")
b.decode("utf-8")

path.write_bytes(b)
path.read_bytes()

Never mix str operations on binary streams without encode/decode.

JSON#

import json
from pathlib import Path

path = Path("config.json")

# Write
data = {"name": "Alice", "scores": [90, 85]}
path.write_text(json.dumps(data, indent=2), encoding="utf-8")

# Or
with path.open("w", encoding="utf-8") as f:
    json.dump(data, f, indent=2)

# Read
obj = json.loads(path.read_text(encoding="utf-8"))
# Or
with path.open(encoding="utf-8") as f:
    obj = json.load(f)

Function	Input/Output
`json.dumps` / `json.loads`	`str` ↔ Python objects
`json.dump` / `json.load`	file ↔ Python objects

JSON types map to: dict, list, str, int, float, bool, None — not set, tuple (becomes list), or custom classes without encoder.

json.dumps({"a": {1, 2}})   # TypeError — set not serializable

# Custom types
json.dumps(obj, default=str)   # fallback — use carefully

CSV#

import csv
from pathlib import Path

path = Path("data.csv")

# Read as dicts
with path.open(newline="", encoding="utf-8") as f:
    reader = csv.DictReader(f)
    for row in reader:
        print(row["name"], row["score"])

# Write
rows = [{"name": "Alice", "score": "90"}, {"name": "Bob", "score": "85"}]
with path.open("w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=["name", "score"])
    writer.writeheader()
    writer.writerows(rows)

Always pass newline="" when opening CSV files — documented csv requirement for correct line ending handling.

CSV is strings only — convert types yourself: int(row["score"]).

Temporary files and directories#

import tempfile
from pathlib import Path

with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".txt") as f:
    f.write("temp data")
    temp_path = Path(f.name)

temp_path.unlink()   # manual cleanup when delete=False

with tempfile.TemporaryDirectory() as tmpdir:
    work = Path(tmpdir) / "output.txt"
    work.write_text("data")
# tmpdir removed automatically

Use for tests and intermediate processing — not long-term storage.

File system operations#

from pathlib import Path
import shutil

src = Path("src.txt")
dst = Path("backup/src.txt")

dst.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(src, dst)           # copy with metadata
shutil.move(str(src), str(dst))    # move/rename

Path("empty_dir").mkdir(exist_ok=True)
shutil.rmtree("empty_dir")         # delete tree — dangerous

Common errors and handling#

Exception	Cause
`FileNotFoundError`	Path does not exist (read)
`FileExistsError`	`"x"` mode but file exists
`PermissionError`	OS denies access
`IsADirectoryError`	Opened directory as file
`UnicodeDecodeError`	Wrong encoding for text

from pathlib import Path

def read_config(path: Path) -> dict:
    try:
        return json.loads(path.read_text(encoding="utf-8"))
    except FileNotFoundError:
        return {}
    except json.JSONDecodeError as e:
        raise ValueError(f"invalid JSON in {path}") from e

Patterns: Error Handling.

`with` statement — why it matters#

with open("f.txt") as f:
    data = f.read()
# f.close() called even if exception inside block

File objects are context managers. Prefer with over manual try/finally/close.

Pickle — caution#

import pickle

pickle.dump(obj, open("obj.pkl", "wb"))
obj = pickle.load(open("obj.pkl", "rb"))

Never unpickle untrusted data — arbitrary code execution risk. Use JSON for interchange; pickle for trusted internal caches only.

Practical take-home layout#

project/
├── data/
│   ├── input.json
│   └── output.csv
├── src/
│   └── solution.py
└── tests/
    └── test_solution.py

from pathlib import Path

ROOT = Path(__file__).resolve().parent.parent
INPUT = ROOT / "data" / "input.json"

Using __file__ anchors paths regardless of cwd.

Interview traps (quick reference)#

Trap	What goes wrong	Safe approach
Relative path + wrong cwd	FileNotFoundError	Anchor with `__file__` or CLI arg
`"w"` mode	Silently truncates file	Confirm intent; use `"a"` to append
Missing encoding on Windows	Mojibake / decode errors	`encoding="utf-8"`
CSV without `newline=""`	Extra blank lines	Follow csv module docs
`read()` huge file	MemoryError	Iterate lines or chunks
Pickle untrusted input	Security vulnerability	Use JSON

Mental model checklist#

When should you use binary vs text mode?
What is the difference between json.load and json.loads?
Why use pathlib over string paths?
What does "w" do to an existing file?
Why always specify encoding for text files?

What's next#

Topic	Page
Stdlib data tools	Data Handling and Standard Library
Modules	Modules and Packages
Advanced file I/O	File IO
Error handling	Error Handling