john mark lowry
§ ai tools · 2026

proofing guardrail

AI-assisted editorial QA where every model suggestion passes a meaning-drift gate before a human sees it.

team one internal tool, deployed in production on the agency's internal aws/eks platform. rule packs and brand specifics sanitized.

01 · the moment

Editorial and proofing errors slip through before delivery, and the obvious fix is worse than the problem: pure-LLM “fix my copy” tools quietly change meaning while fixing grammar. In client work, a tool that silently rewrites a legal disclaimer is not a productivity gain — it's a liability with a chat interface.

02 · the reframe

The risk isn't missing typos — it's an AI altering intent. So the gate, not the suggestion, is the product: every model suggestion must prove it didn't drift the meaning before a human ever sees it, and every decision the system makes must be traceable afterward.

03 · the routes

Deterministic first, semantic second. Mode A runs without any LLM: linting engines, custom rules, glossary enforcement, and exact-match disclaimer checking. Mode B — LLM critique for fuzzy disclaimer matching, recurring feedback patterns, and brand voice — sees only the rules Mode A could not resolve, and may only flag issues that cite an active rule. No freelancing.

The meaning-drift gate is mostly deterministic too. Every Mode B suggestion is checked on token-level diff size, contiguous-word-run limits, and protected-entity preservation, with an optional semantic-similarity check — and lands as pass, review, or reject, with reasons recorded. Suggestions are minimal-span edits by design; anything that might alter meaning gets routed to a human.

Rules resolve through a hierarchy. Seven layers — legal and compliance at the top, down through global, agency, brand, product, campaign, and the individual deliverable — compile into one effective rule set per document, so a brand exception legitimately overrides an agency default and the system can say exactly which rule fired and from which layer.

04 · the architecture

Documents parse into anchored spans; the rule hierarchy compiles; Mode A resolves the mechanical; Mode B critiques the semantic through a server-side model gateway (provider SDKs and keys never reach the browser — CI fails the build if one appears in the bundle); the drift gate filters; humans review issue by issue — accept, reject, edit, or escalate; exports ship with an audit receipt. Provider call logs, rule hits, review actions, and drift checks are all first-class records.

Decision flow: document through deterministic Mode A, unresolved rules to LLM Mode B, every suggestion through the meaning-drift gate to the human review queue, export with audit receipt
the gate sequence — rejected suggestions never reach a human
05 · the outcome

Live in production with an 800+ test suite passing, and the accuracy bar written into the spec as hard gates: zero auto-applied edits, semantic drift under 1% among accepted suggestions, and 100% of issues anchored to the source and cited to a rule. The working estimate — stated as the spec's estimate, not a measured result — is 20–35% proofing time saved from the deterministic pass alone, pushing toward 35–50% with the gated semantic pass.

Trust in AI tools is an architecture decision. Deterministic rules first, semantic critique second, a drift gate between the model and the human — and no silent edits, ever.

the insight
ran onPython + FastAPI · Vale + LanguageTool · LLM gateway (server-side) · React

← all work