144 lines
6.1 KiB
Python
144 lines
6.1 KiB
Python
"""
|
|
TEMPLATE — how to write a contract test. Intentionally empty: no test runs from
|
|
this file, and no tests are committed to the core repo at all.
|
|
|
|
Tests belong to a room:
|
|
|
|
cfg/<room>/soleprint/station/tools/tester/tests/
|
|
|
|
They get merged into the built instance and discovered from there. Core ships the
|
|
base class, the runner and the UI — never anyone's test bodies.
|
|
|
|
|
|
THE POINT
|
|
─────────
|
|
One suite, any environment. A test says what the API promises; it never says who
|
|
implements it or where it runs. Point the same file at a container on your laptop,
|
|
a namespace in the cluster, or a deployed host, and the answer should only differ
|
|
if the contract actually broke.
|
|
|
|
That constrains how tests are written, which is the second idea:
|
|
|
|
|
|
NO HELPER FRAMEWORK TOOLS
|
|
─────────────────────────
|
|
Deliberately plain. A test makes an HTTP call and asserts on the response.
|
|
|
|
- stdlib `unittest`, not pytest fixtures/plugins/parametrize machinery
|
|
- `httpx` against a URL, not a framework test client
|
|
- no factories, no DSL, no ORM access, no database setup
|
|
- no in-process shortcuts — if the test can reach it, so can a client
|
|
|
|
The moment a test needs framework helpers to express itself, it has stopped
|
|
testing the contract and started testing the implementation. It also stops being
|
|
portable: helper-bound tests only run where that framework runs.
|
|
|
|
(Django, where it appears at all, is an optional private DB editor via its admin —
|
|
never the framework, and never something a test reaches into.)
|
|
|
|
|
|
MODES OF EXECUTION
|
|
──────────────────
|
|
A mode is how a test reaches the system under test. Only the first is built.
|
|
|
|
http direct BUILT, and the one that matters. `ContractTestCase` below: pure
|
|
httpx over the wire. Discovered by
|
|
`unittest.TestLoader().discover(pattern="test_*.py")`, so a test
|
|
MUST subclass `ContractTestCase` to appear in the runner.
|
|
|
|
browser SCAFFOLDED, not wired end to end, and rarely the right tool.
|
|
`playwright/runner.py` shells out (`npx playwright test
|
|
--reporter=json`) and parses the report.
|
|
|
|
Browser tests earn their cost on complex UIs. Dashboards, monitors
|
|
and log views are not that — they render what an endpoint already
|
|
returned, so testing the endpoint tests the dashboard. Reach for
|
|
this only when behaviour lives in the browser and nowhere else;
|
|
data visualisation may eventually qualify. Not now.
|
|
|
|
any language INTENDED, not built. The browser adapter is the shape it would take:
|
|
a runner owes only a command to invoke and a machine-readable report
|
|
to parse. Nothing about that requires the test to be Python — R, Go,
|
|
k6 or a shell script satisfy the same contract.
|
|
|
|
Environment targeting is orthogonal to mode. `environments.json` holds the targets:
|
|
|
|
[{"id": "local", "name": "Local", "url": "http://localhost:8000",
|
|
"api_key": "", "description": "...", "default": true}]
|
|
|
|
Select one via `POST /tools/tester/api/environment/select`, or set the env directly:
|
|
|
|
CONTRACT_TEST_URL required — base URL of the target
|
|
CONTRACT_TEST_AUTH_TYPE bearer (default) | api-key | none
|
|
CONTRACT_TEST_TOKEN bearer token; fetched from the token endpoint if unset
|
|
CONTRACT_TEST_API_KEY required when auth type is api-key
|
|
CONTRACT_TEST_TOKEN_ENDPOINT default /api/token/
|
|
CONTRACT_TEST_USER / CONTRACT_TEST_PASSWORD used to fetch a token
|
|
|
|
CONTRACT_TEST_URL=http://localhost:8000 python -m tester run
|
|
|
|
|
|
WRITING ONE
|
|
───────────
|
|
The domain below is the in-repo invoicing fixture (cfg/sample) — deliberately
|
|
generic. Substitute your own; the shape is the part that transfers.
|
|
|
|
from ..base import ContractTestCase
|
|
|
|
|
|
class TestCustomers(ContractTestCase):
|
|
'''Customer endpoints.'''
|
|
|
|
def test_list_returns_customers(self):
|
|
'''A list endpoint returns a list.'''
|
|
response = self.get("/api/customers/")
|
|
self.assert_status(response, 200)
|
|
self.assert_is_list(response.data)
|
|
|
|
def test_create_then_read_back(self):
|
|
'''What was written is what comes back.'''
|
|
created = self.post("/api/customers/", {"name": "Acme Ltd"})
|
|
self.assert_status(created, 201)
|
|
self.assert_has_fields(created.data, "id", "name")
|
|
|
|
fetched = self.get(f"/api/customers/{created.data['id']}/")
|
|
self.assert_status(fetched, 200)
|
|
self.assertEqual(fetched.data["name"], "Acme Ltd")
|
|
|
|
def test_unknown_customer_is_404(self):
|
|
'''Absence is reported, not guessed at.'''
|
|
self.assert_status(self.get("/api/customers/00000000/"), 404)
|
|
|
|
Inherited from `ContractTestCase` (see ../base.py) — this is the whole surface:
|
|
|
|
self.get / post / put / patch / delete auth headers applied, .data parsed
|
|
self.assert_status(response, code)
|
|
self.assert_has_fields(data, *names)
|
|
self.assert_is_list(data, min_length=0)
|
|
self.base_url / self.token / self.api_key
|
|
|
|
Conventions that keep a suite portable:
|
|
|
|
- Group by area in subfolders; put multi-step sequences under `workflows/`.
|
|
- Keep paths in one `endpoints.py` per room so a URL change is a one-line diff.
|
|
- Create what you need through the API and assert on what comes back. A test that
|
|
depends on data already being there only passes in the environment it was
|
|
written against, which defeats the point.
|
|
- Skip, don't fail, when the target is simply unreachable — an unreachable
|
|
environment is not a broken contract.
|
|
|
|
|
|
GHERKIN (OPTIONAL)
|
|
──────────────────
|
|
A test can declare feature/scenario metadata in its docstring; the mapper reads it
|
|
to tie runs back to `.feature` files. Feature files are synced, not committed
|
|
(see features/.gitignore).
|
|
|
|
def test_create_then_read_back(self):
|
|
'''
|
|
Feature: Customer records
|
|
Scenario: A created customer can be read back
|
|
Tags: @smoke @customers
|
|
'''
|
|
"""
|