proofing guardrail
AI-assisted editorial QA where every model suggestion passes a meaning-drift gate before a human sees it.
team one internal tool, deployed in production on the agency's internal aws/eks platform. rule packs and brand specifics sanitized.
Editorial and proofing errors slip through before delivery, and the obvious fix is worse than the problem: pure-LLM “fix my copy” tools quietly change meaning while fixing grammar. In client work, a tool that silently rewrites a legal disclaimer is not a productivity gain — it's a liability with a chat interface.
The risk isn't missing typos — it's an AI altering intent. So the gate, not the suggestion, is the product: every model suggestion must prove it didn't drift the meaning before a human ever sees it, and every decision the system makes must be traceable afterward.
Deterministic first, semantic second. Mode A runs without any LLM: linting engines, custom rules, glossary enforcement, and exact-match disclaimer checking. Mode B — LLM critique for fuzzy disclaimer matching, recurring feedback patterns, and brand voice — sees only the rules Mode A could not resolve, and may only flag issues that cite an active rule. No freelancing.
The meaning-drift gate is mostly deterministic too. Every Mode B suggestion is checked on token-level diff size, contiguous-word-run limits, and protected-entity preservation, with an optional semantic-similarity check — and lands as pass, review, or reject, with reasons recorded. Suggestions are minimal-span edits by design; anything that might alter meaning gets routed to a human.
Rules resolve through a hierarchy. Seven layers — legal and compliance at the top, down through global, agency, brand, product, campaign, and the individual deliverable — compile into one effective rule set per document, so a brand exception legitimately overrides an agency default and the system can say exactly which rule fired and from which layer.
Documents parse into anchored spans; the rule hierarchy compiles; Mode A resolves the mechanical; Mode B critiques the semantic through a server-side model gateway (provider SDKs and keys never reach the browser — CI fails the build if one appears in the bundle); the drift gate filters; humans review issue by issue — accept, reject, edit, or escalate; exports ship with an audit receipt. Provider call logs, rule hits, review actions, and drift checks are all first-class records.
Live in production with an 800+ test suite passing, and the accuracy bar written into the spec as hard gates: zero auto-applied edits, semantic drift under 1% among accepted suggestions, and 100% of issues anchored to the source and cited to a rule. The working estimate — stated as the spec's estimate, not a measured result — is 20–35% proofing time saved from the deterministic pass alone, pushing toward 35–50% with the gated semantic pass.
Trust in AI tools is an architecture decision. Deterministic rules first, semantic critique second, a drift gate between the model and the human — and no silent edits, ever.
the insight