Validation Pipeline

Validation Pipeline

The npm run validate command is the required pre-merge and pre-release quality gate. It runs a locked sequence of checks through scripts/validate.mjs — every step must pass for the pipeline to succeed. No step may be reordered, skipped, or downgraded without governance approval.

Who this is for

• Implementors adding new components or validation rules
• Developers running validation locally before commits
• AI agents understanding what checks their changes must pass

Command Sequence

Steps run in order, from fastest to slowest for early feedback:

#	Command	Tool	What it checks
1	format:check	Prettier	Code formatting consistency
2	lint	ESLint	Code quality, Astro/TS rules
3	lint:tokens	Custom script	Token naming, raw color violations
4	lint:token-overrides	Custom script	Site token override validation
5	lint:static-rules	Custom script	Source-level content quality (alt text, etc.)
6	generate:tokens:check	Custom script	Fluid token drift detection
7	lint:requirements	Custom script	Requirements catalog integrity
8	lint:content-ingestion	Custom script	Content frontmatter and cross-references
9	lint:ownership	Custom script	File ownership marker validation
10	lint:doc-coverage	Custom script	Documentation coverage checks
11	astro:check	astro check	TypeScript type checking
12	build	astro build	Full production build
13	test:browsing-smoke	Playwright + preview	Homepage -> articles -> article detail smoke flow
14	lint:a11y:axe	axe-core + Playwright	Automated accessibility audit

Unit tests for the static-rules framework itself live under npm test — a separate script run by upstream CI but not required for downstream sites. See Core unit tests below.

End-of-Run Summary

After all steps complete, the validation runner prints a final summary table with:

• Step name
• Pass/fail status
• Warning count
• Error count

This summary is intended to make warnings and failures easier to review after long runs. The pipeline still exits non-zero if any step fails.

Per-Step Specification

Step 1 — format:check

• Scope: All *.ts, *.astro, *.css, *.md, *.json files
• Pass: All files match Prettier formatting
• Fail: Any formatting difference
• Fix: npm run format auto-fixes all issues

Step 2 — lint

• Scope: All *.ts and *.astro files
• Pass: Zero ESLint errors (warnings allowed)
• Fail: Any ESLint error
• Config: Extends recommended Astro + TypeScript configs

Step 3 — lint:tokens

• Scope: Component and layout files (src/core/components/**, src/pages/**)
• Errors (release-blocking):
  • Raw color literals (hex, rgb, hsl, oklch, named colors) in component styles
  • Primitive color tokens (--pt-color-*) used directly in component styles — must use semantic tokens (--st-color-*)
• Warnings (non-blocking):
  • Undocumented semantic tokens (used but not defined in core style files)
• Note: Non-color primitives (--pt-space-*, --pt-radius-*, etc.) are allowed in component styles. Only color tokens require strict semantic enforcement.

Step 4 — lint:token-overrides

• Scope: Site token override files (src/site/styles/primitives.css, semantic.css, light.css, dark.css)
• Errors (release-blocking):
  • Wrong-location: token prefix does not match the site file (e.g., --pt-* in semantic.css)
  • Same-tier collision: same token name declared in multiple site files (excluding the expected light ↔ dark pair)
• Warnings (non-blocking):
  • Unknown token: site token not found in the core inventory (suppress with /* token:new */)
  • Mode completeness: --st-* token overridden in one mode file but not the other
• Note: Complements lint:tokens (step 3). lint:tokens checks token usage in components; lint:token-overrides checks token override declarations in site style files. See Override Precedence (DOC-00086) for the full contract.

Step 5 — lint:static-rules

• Scope: Routed markdown/MDX content, .astro templates, and any supporting inventories required by enabled rules
• Checks: Source-level rule findings such as alt/missing and broken-link
• Pass: Zero error-severity findings in default mode; warnings remain informational
• Fail: Any error-severity finding, or any warning-or-higher finding in --strict mode

Step 6 — generate:tokens:check

• Scope: Generated token CSS outputs
• Pass: Generated files are up to date
• Fail: Any drift between config inputs and generated token files

Step 7 — lint:requirements

• Scope: src/core/config/requirements.catalog.ts
• Checks: ID format (REQ-NNNNN) and uniqueness, valid status values, valid appliesTo references (DOC-NNNNN), valid relatedRequirements references, no orphaned dependsOn
• Pass: Zero integrity violations
• Fail: Any invalid reference or format error

Step 8 — lint:content-ingestion

• Scope: All content collections (src/content/**/*.md, src/content/**/*.mdx, src/core/content/**/*.md, src/core/content/**/*.mdx)
• Checks: Required frontmatter fields present and valid per collection schema, relatedDocs cross-references resolve, docId values unique, routed markdown heading baseline rules
• Pass: Zero schema violations or broken cross-references
• Fail: Any invalid frontmatter or dangling reference

Step 9 — lint:ownership

• Scope: Source files expected to carry ownership markers
• Checks: Required // CORE-OWNED or // SITE-OWNED markers
• Pass: Zero ownership marker violations
• Fail: Any missing or invalid ownership marker

Step 10 — lint:doc-coverage

• Scope: Core component inventory and theme-doc references
• Checks: Component documentation mappings stay complete
• Pass: All non-exempt components have documentation coverage
• Fail: Any required component lacks doc coverage

Step 11 — astro:check

• Scope: All Astro components and TypeScript files
• Pass: Zero type errors
• Fail: Any type error
• Note: Warnings are allowed during development

Step 12 — build

• Scope: Full production build
• Pass: Build completes with exit code 0
• Fail: Missing imports, broken routes, template errors
• Note: Build warnings are logged but do not fail the step

Step 13 — test:browsing-smoke

• Scope: Canonical public article-browsing journey at one desktop viewport and one mobile viewport
• Pass: Both journeys can start on the homepage, reach /articles/ through visible navigation UI, open a visible rendered article entry, and confirm a visible H1 plus a visible, non-empty article content region
• Fail: Missing or broken homepage navigation path, zero visible article entries, preview startup failure, or weak article-detail rendering
• How it works: Runs against the built site via astro preview. The script uses real UI interactions only; it does not jump directly to /articles/ or article-detail routes. The smoke step intentionally stays narrow: article browsing only

Step 14 — lint:a11y:axe

• Scope: Dedicated a11y test pages (showcase/sample pages, not production content) — tested in both light and dark mode
• Pass: Zero WCAG 2.2 AA violations on all test pages in both color modes
• Fail: Any violation in either mode
• How it works: Runs against production build output after the browsing smoke step. Each test page is loaded twice — once in light mode, once with data-theme="dark" — to catch mode-specific contrast and visibility regressions

Core unit tests

The npm test script runs unit tests for the theme’s internal frameworks — currently test:static-rules, which verifies the static-rules engine, route-manifest builder, and rule-specific fixtures under scripts/lib/static-rules/fixtures/.

Who runs them

• Upstream theme CI — runs npm test as part of its pre-release gate (npm run validate:all covers both the site pipeline and the core tests in one command).
• Downstream sites — npm test is not required. The core library is synced from upstream and is trusted to have passed its own tests before release. Site maintainers modifying core code locally should run npm test manually.

Why separate from npm run validate

npm run validate checks whether the current site’s content, tokens, and build output are correct. npm test checks whether the core library itself is correct. Mixing the two muddles failure semantics — a test failure means “core is broken” while a lint failure means “this site needs fixing.” Keeping them separate makes ownership obvious.

Accessibility Gate

WCAG 2.2 AA conformance is a release-blocking quality gate.

Test page strategy

Production content pages are not tested in the automated pipeline — they change with content and would make CI brittle. Instead, dedicated non-production test pages exercise every component variant and layout pattern:

• Component showcase pages — render every component variant (buttons, forms, cards, nav states, etc.)
• Sample draft pages — realistic page compositions using all major layouts and section types

These pages are draft/noindex (excluded from sitemap and production navigation) but included in the build. Each is tested in both light and dark mode.

Manual verification

Manual checks supplement automated testing:

• Keyboard navigation through all interactive patterns (menus, modals, forms, tabs)
• Screen reader spot-check on key templates
• Visible focus states on all interactive elements

Suppression policy

Suppressions are only allowed for documented false positives. Each requires:

• The specific axe rule ID and element/page affected
• Justification explaining why it’s a false positive
• Project owner approval
• Tracking in a dedicated config file (e.g., a11y-suppressions.json)

No blanket rule disabling — suppressions are per-element.

Performance Budgets

Performance targets are soft goals (recommended, not release-blocking) until automated CI performance checks are in place.

Lighthouse mobile targets

Category	Target
Performance	≥ 90
Accessibility	≥ 95
Best Practices	≥ 90
SEO	≥ 95

Core Web Vitals targets

Metric	Target	What it measures
LCP	≤ 2.5s	Largest Contentful Paint — loading performance
INP	≤ 200ms	Interaction to Next Paint — responsiveness
CLS	≤ 0.1	Cumulative Layout Shift — visual stability

Test routes: home page, plus the same dedicated showcase/sample pages used for a11y testing.

Run manual Lighthouse checks against production builds and record results in validation notes.

Add automated Lighthouse CI with budget assertions when performance budgets are moved into CI gates.

Exception Process

When a validation step fails and the failure cannot be immediately fixed:

1. Document — file, rule, and specific failure
2. Justify — explain why it cannot be fixed now
3. Scope — define exactly what is waived (specific rule + specific file/element)
4. Set expiry — follow-up task with a target work package
5. Get approval — project owner must approve before merge
6. Track — add to the relevant suppression config or exception list

Exceptions are never open-ended. Blanket waivers are not permitted.