Python backend folder philosophy

In Python web backends, frameworks rarely enforce a folder layout, so it falls to team agreement. A well-organized layout shortens onboarding for new members; without it, the same patterns get scattered everywhere.

1. The 4-layer starting point

A widely used starting point borrows the vocabulary of domain-driven design and uses four layers.

Layer	Responsibility
routers/ (or api/)	HTTP entry. Request/response transforms, auth, validation.
services/	Business rules. Transaction boundaries.
repositories/ (or dao/)	DB access. SQL · ORM calls.
schemas/	Pydantic models. Input/output serialization.

Folders like utils/ · crawlers/ · schedulers/ · clients/ · workers/ are added depending on operational character.

2. Domain separation

Splitting routers into per-domain packages prevents changes in one domain from leaking into others.

src/
├── routers/
│   ├── users/
│   │   ├── __init__.py        # router instance
│   │   ├── handlers.py
│   │   └── deps.py            # domain-specific dependencies
│   └── orders/
│       └── ...
├── services/
│   ├── users.py
│   └── orders.py
├── repositories/
│   ├── users.py
│   └── orders.py
├── schemas/
│   ├── users.py
│   └── orders.py
├── utils/
│   ├── http.py                # centralized HTTP utility
│   ├── time.py
│   └── logger.py
├── crawlers/
│   └── public_data/...
├── schedulers/
│   └── jobs.py
└── main.py

Rather than putting layers inside each domain, putting domain files inside layer folders keeps import paths short. There is no absolute right answer, and once the number of domains explodes, flipping to domain-first folders becomes natural.

3. Centralized HTTP utility

If many places call external APIs directly with requests.get(), the following gets scattered.

• User-Agent policy
• Default timeouts
• Retry/backoff
• Error logging and metrics
• Proxy and auth headers

Bundling these into a single entry point like utils/http.py keeps policy changes in one place.

# utils/http.py
import httpx
from tenacity import retry, wait_exponential_jitter, stop_after_attempt

DEFAULT_TIMEOUT = httpx.Timeout(10.0, connect=5.0)
USER_AGENTS = [...]

@retry(wait=wait_exponential_jitter(initial=1, max=10), stop=stop_after_attempt(3))
async def fetch(url: str, **kw) -> httpx.Response:
    async with httpx.AsyncClient(timeout=DEFAULT_TIMEOUT, headers={'User-Agent': pick_ua()}) as c:
        r = await c.get(url, **kw)
        r.raise_for_status()
        return r

Crawlers and external integrations should call through this utility whenever possible. As more places use httpx/requests directly, policy consistency breaks.

4. Direct dependency vs abstraction

It is fine for services/ to import functions from repositories/ directly. Introduce interface (Protocol) abstractions only where test doubles are frequently needed. Wrapping every place in abstraction only inflates the code.

5. HTTP client candidates

Library	First release	Character
requests	2011, Kenneth Reitz	Synchronous. The most universal. No async support.
aiohttp	2014	async client and server. Stable due to its age.
httpx	2019, encode	Both sync and async. Similar API to requests. HTTP/2 support.
urllib3	2008	Used inside requests. Low level.
aiohttp-retry · tenacity	—	Retry policy helpers.

httpx is often praised for offering a consistent API across mixed sync and async code. requests remains intuitive for simple scripts and synchronous backends.

6. init.py, settings module, DI

Each domain folder's __init__.py collects only the symbols meant to be exposed. Internal module structure can change without breaking external imports.

core/config.py or settings.py reads environment variables in one place via Pydantic Settings (pydantic-settings). Avoid scattering os.getenv calls across modules.

FastAPI Depends functions are gathered in a file like deps.py. Router files focus on the handler bodies.

Request/response Pydantic models are gathered in the schemas/ layer (a single schemas.py when domains are few). Leaving them as inline dicts inside router files splits the same schema. The global exception handler and middleware registration (CORSMiddleware, rate limiting) live in one place — where the app instance is created, usually main.py or an app factory under core/. The patterns of the handlers and middleware themselves are covered in a separate note; here we only fix their location in the folder layout.

7. Common pitfalls

Services that are effectively routers — if business logic is absent and the service only delegates to a repository, the layer loses its meaning. For simple CRUD, calling the repository directly from the router is fine; introducing services later, when rules grow, is not too late.

Circular imports — services/users.py ↔ services/orders.py. When cross-domain calls are needed, one side should import only an interface, or the integration service should live in a separate module.

utils/ becoming bloated — it tends to become the graveyard of uncategorized code. Past a threshold, promote and split it into a domain or adapter.

Bypassing the HTTP policy — as more places use requests.get directly to write something quickly, the value of the central utility evaporates. Backstop with code review and lint rules.

Closing thoughts

Folder layout rarely lands right on the first try. As one domain grows, places to split reveal themselves; as several domains mix, places to merge appear. 4 layers + domain folders + a central utils module is the simplest starting point for a small team.

Next

• sql-as-ssot
• api-handler-pattern

See httpx · requests · aiohttp · tenacity · Pydantic Settings · Architecture Patterns with Python.