AI audit · StackedSP
1 of 18 systems had actually passed its own quality gate
A recruiting firm had built a genuinely sophisticated in-house AI agent system, but its own quality gate had quietly become optional under deadline pressure. The audit found the drift, sized it, and priced every fix in days, not weeks.
1 / 2
The problem
The team had designed real engineering discipline: a tiered review process, a defined testing method for grading each AI system's output, and clear rules for who signs off on what. On paper, nothing was allowed to ship as done without passing its own test.
In practice, only one of eighteen AI-driven systems in production had actually passed that test. The rest had been marked done because they were in daily use and seemed to work, not because anyone had graded them against the standard the team wrote for itself, including the two most business-critical ones.
The system also lived in two disconnected places. Changes made in one didn't reach the other automatically, which meant part of the team could be working from a stale, unreviewed version without knowing it.
What we proposed
The audit traced every claim back to source: a full read of the codebase and its history, cross-checked against the team's own tracking board and planning documents, to find exactly where practice had drifted from policy.
Every gap got a cheap, fast fix instead of a rebuild: split done into what actually passed testing versus what's just in daily use, stand up the missing test cases for the two most critical systems first, and collapse the two-world setup into one source of truth that updates everywhere automatically.
The clearest proof ran live on the call: one of the team's own AI reviewers, pointed at their own internal template, flagged the exact violations of the team's own house rules. Their system catching their own drift, on request.
The result
The methodology was already excellent. Getting the team to use it under pressure was the fix.
1 / 18
systems that had actually passed their own test
days
estimated time for every fix identified, not weeks
2
disconnected copies of the system, collapsed to one