evidence-based RCA for kubernetes

stop guessing
at root cause.

topo ai correlates your temporal k8s graph, git diffs, deploy events and live signals into a ranked list of hypotheses - each with supporting and counter evidence, and a confidence you can defend in an incident review.

on-prem LLM ready eBPF flows (PoC) slack · pagerduty · github
topo - zsh - 120×30
topo topo why checkout-api
  ~ pulling temporal graph @ t=14:32:08Z ± 60m
  ~ 27 signals · 4 diffs · 11 pod events · 1 deploy

H1 conf=0.91 Cache removal regression in price_quote()
    +0.42 diff: payments/checkout.py removed @lru_cache
    +0.31 temporal: deploy @ 14:28:41 → spike 14:32:08 (Δ+3m27s)
    +0.18 signal: 28 queries on pricing.quotes ↑ 18×
H2 conf=0.74 Memory limit exhaustion (OOMKilled)
H3 conf=0.18 Upstream pricing-svc latency

✓ ranked 3 hypotheses with evidence chains in 412ms

how it works

an incident is a story. not a graph.

topo ai is built around the narrative of an incident - what changed, when, what broke, and what evidence supports each theory. The topology view is secondary; the timeline + ranked hypotheses are the primary surface. Engineers spend less time spelunking, more time deciding.

Temporal graph

Every node + edge carries timestamps. State-over-time, not just now. `topo diff --since 1h`.

Identity resolution

Repo → image → helm → deployment → pod, with confidence scores. The backbone.

Evidence engine

Ranked hypotheses with supporting & counter-evidence. Not a single black-box answer.

Recovery loop

After a fix, topo watches the same signals to confirm - or flag a hidden second cause.

why evidence first

a confidence score you can defend in review.

Single-answer RCA tools are unfalsifiable. topo ai is built around an evidence ledger: every score breaks down into the diffs, signals and temporal correlations that contributed - and the counter-evidence that subtracted. Disagree with the verdict? Show your work.

// hypothesis #1 - confidence 0.91
+ 0.42 diff: payments/checkout.py removed @lru_cache
+ 0.31 temporal: deploy 14:28:41 → spike 14:32:08 (Δ+3m27s)
+ 0.18 signal: pricing.quotes db queries ↑ 18× (58/s)
- 0.05 counter: 2/11 pods show healthy memory
──────────────────────────────────────────────
= 0.91 (threshold for slack post: 0.85)

build roadmap

block by block, evidence-first.

Temporal history is in Block 1, identity resolution gets its own block, signals before blame, and Slack only after the RCA is explainable.

phase 1

foundation

01

K8s discovery + temporal graph model

Versioned snapshots, evidence metadata, CLI: topo discover, topo diff --since 1h

02

Identity resolution subsystem

Maps repo → image → helm release → deployment → pod → network identity with confidence.

03

Git + CI/CD layer + signal ingestion

GitHub/GitLab + Actions + ArgoCD + pod/k8s events on the graph with timestamps.

phase 2

intelligence

04

Evidence-based blame engine

Ranked hypotheses with supporting & counter-evidence. Confidence over single-answer.

05

Recovery verification loop

Confirms recovery against the same signals that triggered the alert. Builds ground truth.

phase 3

product surface

06

Incident workspace (narrative-first)

Timeline + evidence panel. Topology is secondary. Replay for past incidents.

07

Workflow integration

Slack / PagerDuty / PR comments only after RCA is explainable.

phase 4

depth

08

eBPF network discovery (PoC)

DaemonSet captures actual TCP/UDP flows. Ground truth, not inferred.

09

LLM reasoning layer

On-prem Llama/Mistral 70b over the graph + RAG over runbooks.

walk through
a live incident.

The demo plays out INC-4419 in real time - alert, correlation, ranked hypotheses with evidence, on-call action, and recovery verification. No setup, no auth.