Writing Scripts That Do Not Break

Nearly every script starts as something you ran by hand and it worked. Then someone else runs it, or it runs on a schedule, or the input file changes shape. These habits are what carry a script across that line, and none of them takes more than a few minutes to add.

Guard the entry point

Code at the top level of a file runs the moment anything imports it. Put the work behind a function and call it under a guard, and your script becomes importable, testable and safe to reuse.

def main():
    rows = load("input.csv")
    write(transform(rows), "output.csv")


if __name__ == "__main__":
    main()

Catch narrowly, or not at all

A bare except swallows everything, including the typo you just made and the interrupt you pressed to stop the run. Catch the specific failure you expect and let everything else surface loudly.

try:
    config = json.loads(path.read_text())
except FileNotFoundError:
    config = DEFAULTS
except json.JSONDecodeError as err:
    raise SystemExit(f"config is not valid JSON: {err}")

Make it rerunnable

Assume every script will be run twice by accident. Write to a temporary file and move it into place at the end, so an interrupted run never leaves half an output behind. Where the script writes to a database or an API, make the operation idempotent: check before insert, or use an upsert.

• Never overwrite the input file in place.
• Write output to a new name, then replace the old one as the last step.
• Where you must append, record what you already processed so a rerun skips it.

Print less, log more

Print statements are fine while you are writing. The moment a script runs anywhere other than your own terminal, you want timestamps and levels, and you want to be able to turn detail up without editing the file.

import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)s %(message)s",
)

logging.info("read %d rows", len(rows))
logging.warning("skipped %d rows with no date", skipped)

Keep configuration out of the code

Paths, credentials and thresholds change; logic does not. Read them from the environment or from arguments so the same file runs on your laptop and on a server without being edited, and so a password never reaches a repository.

import os
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("--input", required=True)
parser.add_argument("--limit", type=int, default=1000)
args = parser.parse_args()

API_KEY = os.environ["API_KEY"]   # fails loudly if it is missing