os and sys

Two different jobs

os is the operating system: files, folders, environment variables, processes. sys is the running interpreter: its version, its arguments, its streams.

Environment variables

The right place for configuration that differs between machines, and the only place for secrets:

import os

os.environ["APP_MODE"] = "demo"

print(os.environ["APP_MODE"])
print(os.environ.get("NOT_SET"))
print(os.environ.get("NOT_SET", "a default"))
print("APP_MODE" in os.environ)

demo
None
a default
True

os.environ behaves like a dictionary, so get() with a default is the safe read. Setting one affects this process only; it does not change your shell.

import os

token = os.environ.get("API_TOKEN")
if token is None:
    print("API_TOKEN is not set; refusing to continue")

API_TOKEN is not set; refusing to continue

Never put a key in your source. Read it from the environment and fail loudly when it is missing.

Folders and files

import os

os.makedirs("data/reports", exist_ok=True)

with open("data/reports/march.csv", "w", encoding="utf-8") as f:
    f.write("name,value\n")

print(sorted(os.listdir("data")))
print(os.path.isdir("data/reports"))
print(os.path.isfile("data/reports/march.csv"))
print(os.path.getsize("data/reports/march.csv") > 0)

['reports']
True
True
True

exist_ok=True means "make sure it exists" rather than "create it, and fail if it is there". Without it, running twice raises FileExistsError.

Walking a tree

import os

os.makedirs("tree/inner", exist_ok=True)
for path in ["tree/a.txt", "tree/inner/b.txt"]:
    with open(path, "w", encoding="utf-8") as f:
        f.write("x")

found = []
for folder, subfolders, files in os.walk("tree"):
    for name in files:
        found.append(os.path.join(folder, name).replace(os.sep, "/"))

print(sorted(found))

['tree/a.txt', 'tree/inner/b.txt']

os.walk gives you the folder, its subfolders and its files at every level. pathlib's rglob does the same thing more briefly, and the Paths chapter covers it.

os.path

import os

path = os.path.join("data", "reports", "march.csv")
print(path.replace(os.sep, "/"))
print(os.path.basename(path))
print(os.path.dirname(path).replace(os.sep, "/"))
print(os.path.splitext("march.csv"))

data/reports/march.csv
march.csv
data/reports
('march', '.csv')

os.sep is / on macOS and Linux and \ on Windows, which is why these examples normalise before printing. os.path.join is what stops you hard-coding either one.

sys: the interpreter

import sys

print(sys.version_info >= (3, 10))
print(isinstance(sys.version_info.major, int))
print(len(sys.path) > 0)
print(isinstance(sys.executable, str))

True
True
True
True

These are checked rather than printed because every one differs by machine. sys.executable is the interpreter running right now, and printing it is the fastest answer to "why does my import fail" — usually you installed into a different Python.

sys.argv

Everything typed after the script name:

# greet.py
import sys

print(sys.argv)

if len(sys.argv) < 2:
    print("usage: python greet.py NAME")
    sys.exit(1)

print("Hello,", sys.argv[1])

['greet.py', 'Ada']
Hello, Ada

sys.argv[0] is the script itself, so real arguments start at 1. For anything beyond one or two, use argparse — the Command-line Arguments chapter covers it.

Exiting, and the streams

import sys

sys.stdout.write("written straight to stdout\n")
print("print goes there too")

written straight to stdout
print goes there too

Errors and progress belong on the other stream:

import sys

sys.stderr.write("this goes to stderr\n")

this goes to stderr

Keeping them apart means piping a program's output into a file or another command captures the data without the chatter.

sys.exit(0) means success and any other number means failure, which is what a shell script checks.