Most health endpoints lie.
They answer 200 because the web server is awake, not because the application behind it works. The connection pool is exhausted, the cache refuses writes, the disk filled up last night, and the load balancer keeps routing traffic because /health said ok. Then the pager goes off for something else entirely and somebody spends twenty minutes trusting a green dashboard.
A 200 that proves nothing
Almost every codebase starts with one line: res.json({ status: 'ok' }). No dependency is touched. The route returns a hardcoded string, which means the only failure it ever reports is a process so dead the framework stopped serving routes at all. For a liveness probe, that is fine and correct. For anything a load balancer consults before sending customer traffic, it is decoration.
The version this page writes opens a real connection per dependency. Each probe gets its own deadline, each one is timed, and a failure comes back as a name and an error string instead of a generic outage.
Look at what changes in the response. A passing service returns the dependency map with latency per entry. A failing one returns 503 with "redis": { "status": "down", "error": "redis exceeded 2000ms" }, and you already know where to look before opening a terminal.
Still running the one line version? Tick database and redis, keep the detailed shape, and compare the generated route against yours. The timeout handling is the part worth stealing, not the payload.
Watch a live endpoint insteadThree probes, three questions
Kubernetes made these names common, and plenty of teams run all three against the same URL. That is where restart loops come from.
- Liveness asks: is this process broken beyond recovery?
- Answer with the process and nothing else. A deadlocked event loop, a heap that never frees, a thread pool that stopped accepting work. If a liveness probe touches your database, one slow query restarts every pod in the deployment at the same time. The generated live route returns uptime and nothing more, on purpose.
- Readiness asks: should traffic reach this instance right now?
- This is where dependency probes belong. A pod that lost its Redis connection stays alive and gets pulled out of the service until the connection returns. No restart, no cold start, no lost in flight work. Pick the Kubernetes shape above and the dependencies move here automatically.
- Startup asks: has the slow boot finished?
- Migrations, JIT warmup, a cache prime that takes ninety seconds. Without a startup probe you set
initialDelaySecondshigh enough for the worst boot, which delays real failure detection for the rest of the pod's life. The generated YAML gives startup 30 attempts at 5 second intervals and leaves liveness tight.
Everything the endpoint returns is readable by anyone who reaches the route. Driver versions, internal hostnames, queue names, and raw exception text are a free reconnaissance report. Error strings from a failed database connect often carry the host and port.
Keep the liveness route open and unauthenticated. Put the detailed payload behind the cluster network, a private port, or a shared token. Never cache either route at a proxy or CDN, and never redirect them.
What each dependency box writes
| Box | Probe in the generated file | Reports down when | Worth knowing |
|---|---|---|---|
database | A SELECT 1 on the default connection | The driver throws, or the query passes the probe timeout | Proves the pool hands out a working connection. Says nothing about replication lag or disk space. |
redis | PING, plus a write and read back on Django and Flask | The reply is not PONG, or the key does not read back | A read only replica passes PING and fails writes, which is why the cache backed probes write first. |
external_api | GET https://api.toolexe.com/health | A non 2xx response or a client timeout | Swap the URL. Think hard before this one gates readiness, see the timeout section below. |
storage | Writes .health-probe, then deletes it | The write, the delete, or the permission check fails | Catches a full volume and a read only remount, which a stat call misses. |
queue | Passive declare on the default queue, or a worker ping | The broker refuses the declare, or no worker answers | Broker reachable and workers running are two different failures. Celery and RabbitMQ get different probes here. |
The timeout is the whole design
Default here is 2000ms per dependency, and the number matters more than the probe list.
Work it backwards. Your orchestrator gives the request a deadline, often 1 to 3 seconds. Every probe has to finish inside that window with room to spare, so the per dependency budget sits below the caller's timeout, and the Kubernetes YAML this page writes sets timeoutSeconds one second above your probe budget for that reason. Express and FastAPI run probes in parallel, so the endpoint costs about the slowest dependency. Laravel, Django, and Flask run them in sequence in the generated file, so five probes at 2000ms each is a 10 second worst case against a 3 second deadline, and the probe fails on timing alone while every dependency was fine.
Now the part nobody plans for. Aggregating dependencies means one shared dependency takes down the fleet. If forty pods all probe the same Redis and Redis hiccups, all forty report not ready, the service loses every endpoint, and you turn a degraded cache into a full outage. Readiness should list what this instance needs to serve its own traffic. A recommendation API that reads from cache and falls back to the database should not fail readiness on cache. An external payment provider almost never belongs in readiness, because you cannot restart your way out of their incident.
A quieter failure mode: the probe itself becomes load. Five dependencies polled every 5 seconds across forty pods is 2400 extra queries a minute against your database, forever. Cache the result for a few seconds, or drop boxes you do not need.
Where the generated file stops
- Client wiring is a placeholder. The Express file requires
../dband../redis, Flask imports fromapp.extensions, Spring assumes the beans are autowired. Point them at your real modules. - No result caching. Every poll runs every probe. Wrap the runner in a short lived cache before you ship it to a large deployment.
- Timeouts are best effort in PHP and Python. A blocking socket read ignores a wall clock deadline, so set
PDO::ATTR_TIMEOUT, Redisread_timeout, and request timeouts on the clients as well. - Spring Boot Actuator already ships liveness and readiness groups. Take the controller only when you need a payload Actuator will not produce, and do not run both on the same path.
- One instance, one view. Nothing here aggregates across a fleet or stores history.
- No auth, no rate limit, no response schema. Those are deployment decisions, and guessing them for you would be worse than leaving the hooks visible.
Once the endpoint is live, the follow up work sits elsewhere. Point the API Endpoint Tester at the route to confirm the status codes and payload before a probe runner ever sees it. When a check starts flapping, the timing breakdown in the Network Request Analyzer separates DNS and TLS cost from application cost, and the Error Log Analyzer groups the exception text your probes are now surfacing. Before a traffic event, the Load Test Script Generator is the honest way to find out whether readiness holds under concurrency, because a probe that passes at idle and fails at 200 requests per second is the one that pages you.
Generation runs entirely in your browser. Route paths, timeouts, and dependency selections stay on your machine, and no request is made to any host you name here.
