Dark Code Is an Organizational Problem, Not a Security One
AI now ships code no human ever understood. Observability and better pipelines don't fix that — comprehension has to be engineered back in.
There is code running in production at companies you use every day that nobody can fully explain — not the engineer who shipped it, not the team that owns it, not the CTO. It works, it passed the tests, and no one on the payroll understands what it does or what would break if it stopped. The industry is starting to call this dark code: not buggy, not spaghetti, not technical debt, but code that was never understood by anyone, because AI wrote it, it passed automated checks, and it shipped. The comprehension step simply didn't happen, because the process no longer requires it to.
Why the obvious fixes fall short
Observability is worth having, but telemetry is not comprehension — it only measures which dark code is breaking. Better agent pipelines and guardrails matter too, but they add a layer to troubleshoot, not understanding. And "it's fine, the model knows what it wrote" gets more tempting as models improve, right up until you can't tell whether the AI is confident or correct. Worse, laying off engineers and asking the rest to ship more manufactures dark code faster.
Engineer comprehension back in
- Force understanding before the code exists. Spec-driven development — write the spec clearly enough that it doubles as the eval. After a costly outage, Amazon rebuilt its coding tool around exactly this.
- Make systems self-describing. Put structural and semantic context in the codebase itself, so behavior is legible to humans and agents rather than locked in someone's head.
- Add a comprehension gate at review that asks what a principal engineer would: why this dependency, why this structure, where are the seams.
Nobody is slowing down, and nobody should. But shipping faster than you understand is an organizational choice — and you can choose to fight it.