Skip to main content
Cluster problems announce themselves at different depths — a cluster that never provisions fails at the Cluster API layer, a crashing app fails three layers further in. Working top-down keeps you from reading pod logs for what is actually a machine that never booted.

​
The triage order

1

Fleet health — which cluster, how badly

GET /api/clusters/health (the cluster list page) gives per-cluster health across the supervisor. Start here to confirm which cluster is unhealthy and whether its API is reachable at all — an unreachable tenant makes every deeper read fail, and that is your answer, not an obstacle.
2

Machines and phase — does the cluster physically exist

GET /api/clusters/<cluster>/nodes lists the Cluster API machines. A cluster stuck in Provisioning, or a worker count that never converges on the desired replicas, is an infrastructure-provider problem: the cluster’s events page (/s/<supervisor>/clusters/<cluster>/events) carries the provider’s own reason. Nothing inside the tenant can be fixed until the machines run.
3

Events inside the tenant — what Kubernetes itself is complaining about

GET /api/clusters/<cluster>/events is the tenant’s own event stream. Warnings with high counts are the signal — this real excerpt points directly at a crash-looping container:
4

Pod events and logs — the actual failure

Follow the object the event names: GET /api/clusters/<cluster>/workload-pods?podNamespace=<ns> shows status and restart counts, .../workload-pod-events narrows to one pod, and .../workload-pod-logs reads what the container printed before dying. A CrashLoopBackOff with four-digit restarts has been failing for days — the first log lines after startup usually say why.
When no curated view covers the resource you suspect, GET /api/clusters/<cluster>/kubectl-get fetches any object’s full YAML — see the escape hatch.

​
Patterns that look like failures but aren’t

​
Metrics panel empty — available: false

GET /api/clusters/<cluster>/metrics returns 200 with "available": false and zeroed numbers when the tenant has no metrics source installed. This is the documented fallback, not an outage:
Fix by enabling the monitoring addon (or installing metrics-server in the tenant) — not by debugging connectivity.

​
Cluster reachable but views are empty — cold-start partial status

Right after the backend starts (or the first time a supervisor is queried), list endpoints can return 200 with an empty result while caches warm in the background — the health endpoint explicitly serves an empty list on a cold cache rather than blocking on a slow probe. The UI polls and fills in within moments. An empty cluster list or blank health column seconds after a deploy means “not warmed yet”; conclude an outage only if it persists across refreshes.

​
Creation rejected for overlapping CIDRs

Cluster creation validates pod/service CIDRs against the supervisor and every existing cluster, and rejects explicit overlaps — a guest whose pod CIDR collides with the supervisor’s is unreachable from the supervisor’s own pods, so the rejection is protecting you. Leave the CIDR fields blank and the allocator picks free subnets automatically. See Create a cluster.

​
Talosconfig download returns 404

GET /api/clusters/<cluster>/talosconfig returns 404 for any non-Talos cluster — the credential simply does not exist for that provider. Same shape for a vcluster instance kubeconfig whose export was never enabled (details).

​
When the platform layer is the suspect

If every cluster on a supervisor misbehaves the same way, stop debugging tenants and check the supervisor’s components — the CAPI engine, addon engine, and providers all report through the component-status model:
  • GET /api/supervisors/<supervisor>/capi/status — operator and provider readiness with the chart controller’s reason and message.
  • GET /api/supervisors/<supervisor>/capi/addons/status — the CAAPH engine and each HelmChartProxy; a proxy that is not ready explains a missing CNI or agent in new guests.
  • POST /api/supervisors/<supervisor>/capi/recover-rollback — recovers the engine from a stuck Helm rollback state when its status shows a failed release.

​
Permissions

Triage is read-only: cluster:GetHealth, cluster:GetNodes, cluster:GetEvents, and cluster:GetWorkloads cover the whole ladder, with cluster:GetResources for kubectl-get and capi:GetStatus for the engine checks.
Celum AI runs this same ladder on request — get_clusters_health, get_cluster_events, and get_cluster_pod_logs are the assistant’s versions of the steps above, so “why is demo-cl unhealthy?” walks the identical APIs.

Day-2 operations

Every read used above, with fuller payload examples.

Platform health

The component-status model the supervisor-level checks build on.

Create a cluster

The creation-time failure table, including CIDR and quorum mistakes.

Storage overview

The parallel triage ladder for Pending PVCs inside tenants.