Building a Self-Improving Security Model

Every deployed classifier has a shelf life. Attackers iterate on phrasing, encodings, and framing faster than annotation teams can label. A prompt-injection model that measured 0.99 F1 at launch is quietly worse six weeks later — not because it got dumber, but because the threat distribution moved. The only durable answer is to make improvement automatic.

The nightly loop

• Generate. FORGE, a fine-tuned adversarial generator, produces fresh attack prompts conditioned on each registered target agent's system prompt — attacks tailored to what the agent will actually obey.
• Classify. Every generated attack runs through the same production GUARD artifact the gateway serves. Measuring anything else would lie about the miss rate.
• Capture misses. Attacks scored below the blocking threshold are saved as training data. These are the only examples that matter: they are, by construction, the frontier.
• Retrain balanced. Misses are paired with an equal sample of clean prompts so recall improves without teaching the model paranoia.
• Gate. The candidate faces the FULL held-out validation corpus — every labeled example outside training, never a subsample.
• Deploy or abort. Above 0.98 weighted F1, the new ONNX artifact is promoted atomically and hot-swapped into the gateway within sixty seconds, no restart. Below it, the candidate is discarded and the previous model keeps serving.
• Harden the attacker. Attacks GUARD caught become FORGE training pairs, forcing the next generation to produce harder variants.
• Report. Tenants wake up to a PDF: attempts, blocks, misses, OWASP IDs, CVSS scores, remediation.

Why the regression gate is the whole ballgame

Naive continuous retraining fails in a specific, predictable way: catastrophic forgetting. Train tonight's model on 200 novel jailbreaks and tomorrow it may block them beautifully while mislabeling ordinary customer questions as attacks. False positives destroy trust faster than false negatives destroy data.

The gate converts deployment from a hope into a proof. A candidate that cannot demonstrate it preserved performance on the entire validation corpus does not ship — regardless of how impressive its new-recall numbers look. Aborted runs are recorded, not hidden, so the trend line tells the truth.

Tiering the improvement

Shared-model economics matter: per-tenant training scales linearly with customers. Free and Pro tenants receive weekly promotions of the shared model; Business receives nightly; Enterprise adds a per-tenant fork fine-tuned on that tenant's own missed attacks, served only to that tenant and re-derived from entitlement every night so downgrades stop serving the fork automatically.

What compounding looks like

The interesting property of this loop is not any single night's gain. It is that the attacker leg and the defender leg improve against each other indefinitely, and every night's misses permanently enlarge the training corpus. Month one buys you a model that catches last month's tricks. Month six buys you a model that has never seen a trick it did not eventually learn from.