Book a call

WritingAI-ready design systemsPart 9 of 20 in Making your design system AI-ready

Guardrails for AI-generated UI: lint rules, tests and evals that keep agents on-system

Agents produce more UI than reviewers can check by eye. Layered guardrails catch off-system output automatically and tell the agent how to fix it, and evals show whether your design system work is paying off. Configs, scripts and a scoring method included.

Diagram of six guardrail layers: types, lint, tests, review, evals and telemetry, catching off-system output at different stages.
On this pageThe problem: silent success at scale
  1. The problem: silent success at scale
  2. Defence in depth: six layers
  3. Layer 1: types that refuse wrong answers
  4. Layer 2: lint rules that explain the fix
  5. Layer 3: tests that check what lint can't
  6. One check command
  7. Layer 4: what people should still review
  8. Layer 5: evals, measuring agents on your system
  9. Layer 6: telemetry across every repo
  10. Close the loop: fix the system, not just the code
  11. Common mistakes
  12. The short version
  13. Frequently asked questions
  14. Further reading

The short answer

Guardrails for AI-generated UI are automated checks that catch off-system output before a person has to, and tell the agent exactly how to fix it. Layer them: strict types reject invalid props; lint rules (ESLint, Stylelint) block deprecated imports, raw colours, pixel values and Tailwind defaults, with error messages that name the replacement; tests check accessibility (axe) and appearance (visual regression); one check command runs everything so agents run it before finishing; and evals (a fixed set of real tasks, run repeatedly and scored automatically) measure how often agents stay on-system as you improve tokens, metadata, docs and tools. Treat every failure as feedback about the system, not just the code.

  • A lint message is an instruction the agent will follow: "Use Select from @kestrel/harbor" beats "Import not allowed".
  • Run each eval task several times. Agents vary, and one good run proves little.
  • Part 9 of 20 in the series on AI-ready design systems.

The problem: silent success at scale

In part 2, the agent's first attempt passed every check and reported "Done!" with a legacy card, a hard-coded blue and its own currency formatter. That's the most damaging failure, because it moves the work of checking onto reviewers, and reviewers miss things, especially when agents produce several times more code than before.

An illustration: review fatigue

The situation

Three months after Kestrel's engineers adopted coding agents, the number of UI pull requests has roughly doubled. Reviewers skim. The two Harbor maintainers can't look at everything.

What slips through

A Harbor audit finds 140 new hard-coded colours, 22 new imports from @kestrel/ui-old, and four new date pickers, all merged since the agents arrived.

What works

Make the machine say no, clearly, every time: lint rules with fix-it messages, a check command agents run before finishing, and a weekly eval that shows whether things are getting better.

Defence in depth: six layers

No single check catches everything. Stack cheap, fast checks first and slower, broader ones later. Each layer catches what the previous one can't.

While writing
1 · TypesInvalid props and values fail to compile
2 · LintDeprecated imports, raw values, Tailwind defaults
Before merge
3 · TestsAccessibility (axe), visual regression, behaviour
4 · ReviewA person checks judgement: right pattern, right words
Over time
5 · EvalsReal tasks, run repeatedly, scored automatically
6 · TelemetryAdoption metrics across every repo
Guardrail layers. Layers 1–3 stop individual mistakes; layers 5–6 show whether the system as a whole is getting easier for agents to use.

Layer 1: types that refuse wrong answers

Covered in part 4: union types for every option prop, required props where the component can't work without them (an icon-only button's aria-label), and @deprecated tags. Make sure the type checker actually runs in the agent's loop: put it in the check command, and say so in AGENTS.md. Also make sure your tsconfig is strict enough to catch mistakes; "strict": true is the baseline.

Layer 2: lint rules that explain the fix

Lint is the most valuable guardrail for agents, for a simple reason: agents read error messages and act on them. A rule that says what to do instead turns a failure into a correction in the same session, with no person involved.

Block deprecated imports

// eslint.config.js (flat config)
export default [
  {
    files: ["src/**/*.{ts,tsx}"],
    rules: {
      "no-restricted-imports": ["error", {
        paths: [
          { name: "@kestrel/ui-old",
            message: "ui-old is being removed. Use @kestrel/harbor. Look components up with harbor.search_components." },
          { name: "@kestrel/harbor", importNames: ["Modal", "AlertDialog"],
            message: "Deprecated. Use Dialog, or ConfirmDialog to confirm an action." },
          { name: "@kestrel/harbor", importNames: ["BtnPrimary"],
            message: "Deprecated. Use <Button variant=\"primary\">." },
        ],
        patterns: [
          { group: ["**/legacy/**"], message: "Don't import from legacy folders. Use @kestrel/harbor." },
        ],
      }],
    },
  },
];
Tip

Generate the deprecated-component part of this config from your component metadata (every entry with status: deprecated and its replacedBy), the same way the rules file's "Never use" list is generated in part 6. Then deprecating a component updates the docs, the rules file and the lint rule in one release.

Block raw values in JSX

ESLint's built-in no-restricted-syntax rule matches code patterns using selectors, including regular expressions. Two selectors catch the most common agent habits: hex colours in inline styles, and Tailwind's default palette in class names.

{
  rules: {
    "no-restricted-syntax": ["error",
      {
        selector: "JSXAttribute[name.name='style'] Literal[value=/#[0-9a-fA-F]{3,8}\\b|rgba?\\(/]",
        message: "Raw colour in style. Use a semantic token, e.g. var(--color-text-secondary). Find one with harbor.search_tokens.",
      },
      {
        selector: "JSXAttribute[name.name='className'] Literal[value=/\\b(bg|text|border|ring)-(slate|gray|zinc|red|orange|amber|yellow|green|emerald|teal|sky|blue|indigo|violet|purple|pink|rose)-\\d{2,3}\\b/]",
        message: "Tailwind default colour. Use Harbor theme classes such as bg-surface or text-danger.",
      },
    ],
  },
}

Block raw values in CSS

For stylesheets, the Stylelint plugin stylelint-declaration-strict-value requires chosen properties to use variables or functions instead of literal values:

// stylelint.config.mjs
export default {
  plugins: ["stylelint-declaration-strict-value"],
  rules: {
    "scale-unlimited/declaration-strict-value": [
      ["/color$/", "background", "fill", "stroke", "/^padding/", "/^margin/", "gap", "border-radius", "font-size"],
      {
        ignoreValues: ["inherit", "transparent", "currentColor", "0", "auto", "none"],
        message: "Use a Harbor token (var(--…)) for ${property}. Find one with harbor.search_tokens.",
      },
    ],
  },
};
Tool call · runpnpm check
Resultsrc/features/payslips/PayslipSummary.tsx 4:10 error ui-old is being removed. Use @kestrel/harbor. Look components up with harbor.search_components. 12:22 error Raw colour in style. Use a semantic token, e.g. var(--color-text-secondary).
Tool call · harbor.search_tokens{ "need": "secondary text for a label", "type": "color" }
AgentReplaced the ui-old import with StatCard from Harbor and the hex value with var(--color-text-secondary). Lint and types now pass.
Illustration The lint message names the tool to call, so the agent fixes its own work within the same task.

Roll out lint without blocking everyone

An existing codebase will have hundreds of violations. Don't let that stop you:

  1. Start as warnings on existing code

    Report everything; fail nothing yet. The count becomes your baseline.

  2. Errors on changed lines only

    Run lint as errors on files changed in each pull request, so new code (including all agent code) is held to the standard while old code waits.

  3. Burn down with codemods

    Fix the mechanical cases in bulk (renamed imports, known colour-to-token swaps), then switch the rules to errors everywhere. See part 10.

Layer 3: tests that check what lint can't

Lint reads code; tests look at the result. Two kinds matter most for AI-generated UI.

ACCESSIBILITY

Automated axe checks

import { test, expect } from
  "@playwright/test";
import AxeBuilder from
  "@axe-core/playwright";

test("payslip page is accessible",
  async ({ page }) => {
  await page.goto("/payslips/2026-10");
  const { violations } =
    await new AxeBuilder({ page })
      .analyze();
  expect(violations).toEqual([]);
});

Catches missing labels, low contrast and invalid ARIA. It finds a portion of accessibility issues, not all; keyboard and screen reader checks still need people.

APPEARANCE

Visual regression

test("payslip summary", async ({ page }) => {
  await page.goto(
    "/storybook/iframe.html" +
    "?id=payslips-summary--default");
  await expect(page)
    .toHaveScreenshot();
});

Compares screenshots with approved ones. Catches a component that "works" but looks wrong in dark mode. Hosted services such as Chromatic do the same at scale for Storybook.

More on testing components in design system testing and accessibility in a design system.

One check command

Agents follow instructions like "run pnpm check before finishing" very reliably. Give them one command that runs everything relevant and prints clear output.

// package.json
{
  "scripts": {
    "lint": "eslint . && stylelint \"src/**/*.css\"",
    "typecheck": "tsc --noEmit",
    "test:ui": "playwright test --project=a11y",
    "check": "pnpm lint && pnpm typecheck && pnpm test:ui"
  }
}

Some agents can also run a command automatically after every edit (Claude Code's hooks are one example), which catches problems even earlier. Either way, run the same command in CI, so nothing merges without passing it.

Layer 4: what people should still review

Automation handles the mechanical checks so people can focus on judgement. A short checklist for reviewing AI-generated UI:

  • Right pattern for the job? A table where a list would be clearer; a dialog where an inline edit would do.
  • Right words? Labels, errors and empty states follow the content guidelines. See UX writing.
  • Did it build something new? Any new component or helper needs a reason. Often a system component already does the job.
  • Did it flag what it couldn't map? Read the agent's summary. Gaps it reports are design system backlog items.
  • Keyboard and screen reader basics on anything interactive: tab order, focus, announcements.

Layer 5: evals, measuring agents on your system

Everything above checks one change at a time. An eval (short for evaluation) answers a different question: across realistic tasks, how often does an agent build on-system UI with our design system, and is that number going up? It's how you know whether the work in parts 3 to 8 is paying off, and which part to improve next.

New words

Eval: a repeatable test of an AI system: fixed inputs, run several times, scored the same way each time. Golden task: one of those inputs, with a known good outcome. Pass rate: the share of runs that meet the scoring criteria.

  1. Write 10–20 golden tasks

    Use real work: the baseline tasks from part 2 plus tasks from recent tickets. For each, write the prompt and what a correct result must contain ("uses ConfirmDialog with tone=danger", "no new components").

  2. Run each task several times

    Start each run from a clean copy of a sample repo, in a fresh session, using the agent's non-interactive mode (for example claude -p or codex exec). Three to five runs per task shows how consistent the agent is.

  3. Score automatically

    A script inspects each run's changes: lint and type results, deprecated imports, raw values, whether expected components appear, axe violations. People spot-check a sample.

  4. Track the numbers over time

    Rerun after every change to tokens, metadata, docs, rules or the MCP server, and on each new agent or model version.

// evals/score.mjs: score one run from its diff (simplified)
import { execSync } from "node:child_process";

export function score(runDir, task) {
  const sh = (cmd) => { try { execSync(cmd, { cwd: runDir, stdio: "pipe" }); return true; } catch { return false; } };
  const diff = execSync("git diff HEAD", { cwd: runDir, encoding: "utf8" });
  const added = diff.split("\n").filter((l) => l.startsWith("+") && !l.startsWith("+++")).join("\n");
  return {
    task: task.id,
    lint: sh("pnpm -s lint"),
    types: sh("pnpm -s typecheck"),
    deprecatedImports: (added.match(/@kestrel\/ui-old|\/legacy\//g) ?? []).length,
    rawColours: (added.match(/#[0-9a-fA-F]{6}\b/g) ?? []).length,
    expectedComponents: task.expect.every((name) => added.includes(`<${name}`)),
    newComponentFiles: (diff.match(/^\+\+\+ b\/src\/components\//gm) ?? []).length,
  };
}
Stage of Harbor's workRuns passing lint + typesExpected components usedRaw colours per run
Baseline (part 2)38%41%4.6
+ AGENTS.md (part 6)61%58%2.1
+ token descriptions and Tailwind theme (part 3)70%60%0.4
+ metadata and MCP server (parts 4, 7)84%83%0.3
+ lint with fix-it messages (this part)96%87%0.0
Illustration Numbers are made up to show the shape of a typical eval history across 15 tasks × 4 runs. Your order of steps, and your results, will differ. The point is to measure each step.
Watch out

Don't tune your docs to the eval tasks until the numbers look good: that's teaching to the test. Keep a few tasks aside that you never look at when making changes, and refresh the task set every quarter from recent real work.

Layer 6: telemetry across every repo

The same measures that track design system adoption track agent behaviour, because agent code is now a large share of all code. Count these weekly across product repos:

MeasureHowTells you
Share of UI imports from HarborStatic analysis of importsWhether new code uses the system
Deprecated imports, trendThe lint rule's count, per weekWhether migration is moving or going backwards
Raw values, trendThe lint and Stylelint countsToken adoption
New local componentsFiles added under components/ in product reposGaps in the system, or agents ignoring it
Zero-result MCP searchesServer logs (hosted) or eval transcriptsWhat people ask for that you don't have
Agent-era adoption metrics. See design system metrics for the wider set.

Close the loop: fix the system, not just the code

When a guardrail catches a mistake, it's tempting to fix the code and move on. The bigger win is asking why the agent made it, and fixing the cause in the design system.

FailureEval: agent used Dialog for a delete confirmation
→
DiagnoseDialog's purpose says "confirm or edit"; overlaps ConfirmDialog
→
Fix the sourceSharpen Dialog's purpose; add avoidWhen → ConfirmDialog
→
GuardAdd a golden task for it
→
VerifyRerun evals; pass rate up
The feedback loop. Every repeated agent mistake points to a token, component, doc or rule that people were also probably finding confusing.

Common mistakes

MISTAKE

Error messages without a fix

"Restricted import" makes the agent guess. Name the replacement and the tool to find it.

MISTAKE

Checks that only run in CI

If the agent never runs them, it finishes with errors and someone else has to fix them. Put the command in AGENTS.md.

MISTAKE

Judging from one run

Agents vary run to run. Decide from several runs per task, not a lucky demo.

MISTAKE

Treating evals as a one-off

Models, agents and your system all change. An eval that isn't rerun stops telling you anything within a quarter.

The short version

  • Layer guardrails: types, lint, tests, review, evals, telemetry.
  • Write lint messages as instructions: the replacement, and the tool to find it.
  • Give agents one check command and tell them to run it; run the same in CI.
  • Measure with evals: real tasks, several runs each, scored automatically, rerun after every change.
  • When a check fails repeatedly, fix the design system's source, not only the code.

Next: How to make a legacy design system AI-ready, putting all of this into a phased plan for a system that's already in use.

Frequently asked questions

How do I stop AI from generating off-system UI?

Combine guidance with enforcement: rules files, described tokens and component metadata guide the agent; strict types and lint rules with clear fix-it messages catch what slips through; accessibility and visual tests check the result. Agents read error messages and fix their own work.

What lint rules should a design system have for AI-generated code?

At minimum: block imports from deprecated packages and components (no-restricted-imports), block raw colours in styles and default Tailwind palette classes (no-restricted-syntax), and require tokens for colours, spacing and type in CSS (stylelint-declaration-strict-value).

What is an eval for a design system?

A repeatable test of how well AI agents build with your system: a fixed set of realistic tasks, each run several times, with results scored automatically for things like lint errors, deprecated imports, raw values and expected components.

Can automated tests check accessibility of AI-generated UI?

Partly. Tools like axe-core catch many issues (missing labels, contrast, invalid ARIA) but not all. Keyboard behaviour and screen reader experience still need human checks.

How do I measure whether AI agents use my design system?

Track the share of UI imports from the system, deprecated imports and raw values over time, new local components in product repos, and pass rates on a regular eval run.

Further reading