Chapters
Python114 chapters

Beyond the basicsChapter 100 of 114

Packaging

Turn a folder of scripts into something installable.

What a package is

A folder with an __init__.py, and a pyproject.toml beside it saying what it is called and what it needs:

text
mytool/
├── pyproject.toml
├── README.md
├── src/
│   └── mytool/
│       ├── __init__.py
│       ├── core.py
│       └── cli.py
└── tests/
    └── test_core.py

The src/ layout is worth adopting. Without it, running tests from the project root imports your code from the working directory rather than from the installed copy, so the tests pass and the installed package is broken.

pyproject.toml

One file describes the whole project:

text
[project]
name = "mytool"
version = "0.1.0"
description = "Does a useful thing."
readme = "README.md"
requires-python = ">=3.10"
dependencies = [
    "requests>=2.31",
]

[project.optional-dependencies]
dev = ["pytest", "ruff"]

[project.scripts]
mytool = "mytool.cli:main"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project.scripts] is the interesting part: it creates a real mytool command that calls main() in mytool/cli.py. Your users never type python -m.

It replaces the old setup.py and setup.cfg, and every current tool reads it.

Packages are folders with __init__.py

Python
import sys, types

# what `import mytool` gives you: a module object with names attached
mytool = types.ModuleType("mytool")
mytool.__version__ = "0.1.0"
sys.modules["mytool"] = mytool

import mytool as installed
print(installed.__version__)

Output

0.1.0

Normally __init__.py does that for you. Keep it small: it runs on every import, so a slow line there makes every command that uses your package slow.

Python
# src/mytool/__init__.py
from mytool.core import process

__version__ = "0.1.0"
__all__ = ["process"]

Re-exporting the few names that matter means users write from mytool import process instead of reaching into your internals — which then leaves you free to rearrange them.

Installing while you work

bash
python -m pip install -e .
python -m pip install -e ".[dev]"

The -e is an editable install: the package is importable from anywhere, and your edits take effect immediately with no reinstall. It is how you should work on anything bigger than a script.

Building and publishing

bash
python -m pip install build twine
python -m build

That writes two files into dist/: a .whl wheel, which is what pip installs, and a .tar.gz source distribution.

bash
python -m twine upload --repository testpypi dist/*
python -m twine upload dist/*

Upload to TestPyPI first and install from it. A broken package on the real PyPI cannot be replaced — a version number can never be reused, only yanked.

Versions

MAJOR.MINOR.PATCH, and the promise is to your users:

  • patch — a bug fix, nothing else changed
  • minor — something added, existing code still works
  • major — something existing changed or was removed

Break that and people stop trusting your version numbers, which is the only thing making automatic upgrades safe.

What to include

text
# .gitignore
__pycache__/
*.egg-info/
dist/
build/
.venv/

A README.md shows on your PyPI page, and a LICENSE decides whether anyone may legally use your work. Without one, the default is nobody.

Do you need to publish?

Usually not. A package installable from a git URL covers most private cases:

bash
python -m pip install git+https://github.com/you/mytool

Publish when strangers should find it by name. Package without publishing, from the first day of any project bigger than one file — the layout and the pyproject.toml are what make it testable and installable, and those are worth having whether or not anyone else ever sees it.

Test yourself

2 questions

Why put your code under src/ rather than at the project root?

Show the answer

Otherwise tests import from the working directory rather than the installed copy — Without it the tests can pass while the installed package is broken.

What does pip install -e . give you?

Show the answer

An editable install, so your edits take effect with no reinstall — It is how you should work on anything bigger than a single script.

Next chapter

What Machine Learning Is

Finding the rules from examples instead of writing them yourself.