Managing Secrets

Secrets — API keys, database passwords, tokens, private keys — must never live in source code or version control. Python applications should load credentials from environment variables, secret managers, or encrypted stores at runtime, rotate them regularly, and avoid leaking them in logs, tracebacks, or error messages.

How to use this page

Treat secrets as configuration, not code. Pair with File IO for permission-safe files, HTTP Requests for authenticated clients, and Unit Testing for mocking credentials in tests.

At a glance

Track	Python Advanced → Security and Safe Practices
Sections	13 major topics
Outline	Use the right-hand TOC to jump

Topics: Golden rules · Environment variables · .env files with python-dotenv · secrets module — cryptographic randomness · File-based secrets with strict permissions · keyring — OS credential store · Cloud secret managers (pattern) · Configuration layering · … (+5 more)

1. Golden rules
2. Environment variables
3. .env files with python-dotenv
4. secrets module — cryptographic randomness
5. File-based secrets with strict permissions
6. keyring — OS credential store
7. Cloud secret managers (pattern)
8. Configuration layering
9. HTTP clients and secrets
10. Logging and error hygiene
11. Testing without real secrets
12. Git hygiene … and 1 more sections

---

Golden rules

Rule	Rationale
Never commit secrets to git	History is forever — bots scan public repos
Never log secrets	Logs aggregate to insecure systems
Least privilege	Scope tokens to minimum permissions
Rotate on exposure	Assume compromise after any leak
Separate dev/prod secrets	Staging keys should not unlock production
Fail fast if missing	Crash at startup, not mid-request

---

Environment variables

import os

def require_env(name: str) -> str:
    value = os.environ.get(name)
    if not value:
        raise RuntimeError(f"Missing required environment variable: {name}")
    return value

api_key = require_env("API_KEY")
database_url = require_env("DATABASE_URL")

export API_KEY="sk-live-..."
python -m myapp

Pros	Cons
Simple, 12-factor friendly	Visible in process list on some OS
No files to manage	Easy to misconfigure in shells
Works in containers/K8s	No built-in rotation

In Docker/Kubernetes, inject via secrets mounted as env vars or files.

---

.env files with python-dotenv

For local development only — not a production secret store.

# pip install python-dotenv
from dotenv import load_dotenv
import os

load_dotenv()  # loads .env from cwd or specified path
api_key = os.environ["API_KEY"]

# .env — NEVER commit
API_KEY=dev-key-only
DATABASE_URL=postgresql://localhost/myapp
DEBUG=true

.gitignore:

.env
.env.*
!.env.example

Provide .env.example with placeholder keys:

# .env.example — safe to commit
API_KEY=your-key-here
DATABASE_URL=postgresql://user:pass@localhost/dbname

Interview trap — .env in production

Production should use vault/K8s secrets/AWS SM — not a .env file on disk in the image.

---

secrets module — cryptographic randomness

import secrets

# NOT for loading API keys — for GENERATING tokens
token = secrets.token_urlsafe(32)
password = secrets.token_hex(16)

# Compare secrets in constant time
import hmac
hmac.compare_digest(provided_token, expected_token)

Function	Use
token_urlsafe(n)	Session tokens, reset links
token_hex(n)	Hex-encoded random bytes
compare_digest(a, b)	Timing-safe string comparison

Do not use random.random() for security-sensitive tokens.

---

File-based secrets with strict permissions

import os
from pathlib import Path

def read_secret_file(path: Path) -> str:
    if oct(path.stat().st_mode & 0o777) != "0o600":
        raise PermissionError(f"Secret file {path} must be mode 600")
    return path.read_text(encoding="utf-8").strip()

key = read_secret_file(Path("/run/secrets/api_key"))
os.chmod(path, 0o600)  # set after writing

Docker and Kubernetes mount secrets as files under /run/secrets/ — read at startup, keep in memory only.

See File IO for atomic writes when generating local key material.

---

keyring — OS credential store

import keyring

keyring.set_password("myapp", "api_user", "secret-value")
password = keyring.get_password("myapp", "api_user")

Uses macOS Keychain, Windows Credential Locker, or Secret Service on Linux. Good for CLI tools on developer machines — less common in server deployments.

---

Cloud secret managers (pattern)

# Conceptual — AWS Secrets Manager via boto3
import boto3
import json

def load_db_credentials(secret_arn: str) -> dict:
    client = boto3.client("secretsmanager")
    response = client.get_secret_value(SecretId=secret_arn)
    return json.loads(response["SecretString"])

Provider	Service
AWS	Secrets Manager, SSM Parameter Store
GCP	Secret Manager
Azure	Key Vault
HashiCorp	Vault

Fetch once at startup, cache in memory, refresh on rotation signals.

---

Configuration layering

from dataclasses import dataclass
import os

@dataclass(frozen=True)
class Settings:
    api_key: str
    database_url: str
    debug: bool

def load_settings() -> Settings:
    return Settings(
        api_key=os.environ["API_KEY"],
        database_url=os.environ["DATABASE_URL"],
        debug=os.environ.get("DEBUG", "false").lower() == "true",
    )

Layer	Source
Defaults	Safe dev defaults in code (non-secret)
Environment	Production overrides
Local .env	Developer machines only
Secret manager	Production credentials

Validate all required fields before serving traffic — fail at import/startup.

---

HTTP clients and secrets

import os
import httpx

def create_client() -> httpx.Client:
    token = os.environ["API_TOKEN"]
    return httpx.Client(
        headers={"Authorization": f"Bearer {token}"},
        timeout=10.0,
    )

Never embed tokens in URLs (logged in access logs). Use headers. Details: HTTP Requests.

---

Logging and error hygiene

import logging

logger = logging.getLogger(__name__)

def safe_log_request(url: str, headers: dict) -> None:
    redacted = {k: "***" if k.lower() == "authorization" else v for k, v in headers.items()}
    logger.info("request url=%s headers=%s", url, redacted)

Leak vector	Prevention
print(os.environ)	Strip in code review
Exception messages with URL query tokens	Parse and redact
Debug middleware dumping headers	Disable in prod
Jupyter notebooks	Clear outputs before commit

---

Testing without real secrets

import pytest

@pytest.fixture(autouse=True)
def fake_secrets(monkeypatch):
    monkeypatch.setenv("API_KEY", "test-key-not-real")
    monkeypatch.setenv("DATABASE_URL", "sqlite:///:memory:")

Use dedicated test credentials with limited scope — never production keys in CI. See Unit Testing.

---

Git hygiene

.env
*.pem
*.key
credentials.json
secrets/

If a secret is committed:

1. Rotate the credential immediately
2. Remove from history (git filter-repo or BFG) — coordinate with team
3. Audit access logs for misuse

Pre-commit hooks (detect-secrets, gitleaks) catch accidents early.

---

CLI tools and secrets

import getpass

password = getpass.getpass("Database password: ")

Prompt interactively for local admin tools — avoid echoing to terminal. For automation, use env vars. See Building Command Line Interfaces.

---

Interview traps (quick reference)

Trap	What goes wrong	Safe approach
API key in source code	Leaked via git	Environment / vault
.env committed	Public exposure	.gitignore + .env.example
random() for session tokens	Predictable tokens	secrets.token_urlsafe
Logging full request headers	Token in log aggregator	Redact sensitive headers
Same prod key in dev	Blast radius on leak	Separate credentials
os.environ dump in debug	Accidental exfiltration	Structured debug flags
World-readable key file (chmod 644)	Local privilege escalation	0o600 permissions
Secrets in URL query strings	Logged by proxies	Authorization header

Mental model checklist

1. Where should production database passwords live?
2. What is the purpose of secrets.compare_digest?
3. Why is .env inappropriate as a production secret backend?
4. How do you fail fast when API_KEY is missing?
5. What file mode should a local PEM key use?
6. How do you test code that reads os.environ?

What's next

Topic	Page
Secure file reads	File IO
Authenticated HTTP	HTTP Requests
Mocking in tests	Unit Testing
CLI password prompts	Building Command Line Interfaces