Tuning tman for pytest

Python

Chatty by default, quiet during collection; xdist workers orphan on a hard kill.

Recommended starting caps

These are starting points derived from how pytest behaves, not measurements of your machine. Every one of them should be replaced by a number you took from your own runs — the last section on this page is how.

alias "test" {
    command "pytest"
    args "-q" "--tb=short"
    max-time "20m"
}

alias "e2e" {
    command "pytest"
    args "tests/e2e" "--tb=short"
    max-time "30m"
    max-mem 4096
    max-parallel 1
}

pytest prints a character per test, so a running suite is never quiet for long and the stall backstop mostly sits idle. The interesting windows are before the first character and after the last one.

Collection is the quiet stretch

Before any test runs, pytest imports every test module. On a large suite with heavy imports — a framework, a machine-learning stack, anything that touches native libraries at import time — that is minutes of silence. It is not idle: the CPU is busy and files are being read, so on Linux tman sees activity and the backstop holds regardless. On macOS and Windows the tree is not walked, so give collection-heavy suites a wider stall there.

A session-scoped fixture that waits on a service — a container starting, a migration running — is the case that does look like a hang: zero output, zero CPU, waiting on a peer that has sent nothing. That is what max-time is for. Do not tighten stall to bound it; stall cannot tell that wait apart from a dead process, and a value low enough to catch one kills the other.

xdist and the reaper

pytest -n auto spawns worker processes. When the run is culled or killed, tman kills the whole tree, so the workers go with it — and any that survive a machine suspend are reaped by the next tman command rather than sitting on your CPU overnight. That is the case the reaper exists for.

Anything that binds a port gets `max-parallel 1`

An end-to-end alias that starts a server, a database, or a container should hold one slot, not two. The second copy will not fail cleanly — it will bind-conflict, or worse, share the database and produce a failure that looks like a flaky test. Buckets are scoped per name and directory, so max-parallel 1 on e2e does not slow down test at all.

Turn the starting point into a real number

Everything above is derived from how this tool behaves, not from your machine. The numbers that matter come from your own runs, and getting them takes one command:

tman status --json | jq '.runs[] | {name, exit, duration_ms}'

Set max-time from the slowest successful run you are willing to wait for, with real headroom — a cap that fires on healthy work teaches everyone to raise the cap, and after a few rounds of that it bounds nothing. Set stall from the longest stretch the command legitimately produces no output and no CPU, then double it.

Exit 124, 125, and 126 are findings, not flakes. 124 is a wall-clock timeout, 125 a stall, 126 a resource cull. Widening the cap that fired is the right response only after you know why it fired — the first two times a suite trips max-mem, the interesting question is what is holding the memory.

Frequently asked

Related tools

• Vitest
• Jest
• go test
• dotnet test
• cargo test
• RSpec
• Gradle & Maven

Sources

• pytest usage