Writing Auto-Verification Rules as Versioned YAML

Problem Statement

When auto-verification rules live inside application code or a LIMS admin screen, nobody can answer the question an inspector actually asks: what were the exact release criteria the day this result was signed out, and who approved them? Rules buried in code are invisible to clinical reviewers; rules edited through a vendor GUI often have no diff, no reviewable history, and no atomic promotion. Expressing the rules as a plain-text YAML file changes that — the rule set becomes a reviewable artifact under Git, every edit is a diff with an author and a timestamp, every release is a tag, and a continuous-integration pipeline can refuse to promote a file that does not parse, does not type-check, or introduces an ambiguous precedence. This page shows the concrete YAML format for auto-verification and delta-check rules, the Pydantic v2 loader that validates it, and the Git and CI workflow that turns a text file into a Part 11-defensible change-control record.

Prerequisites

Before adopting a YAML-backed rule set, confirm the baseline:

  • Runtime: Python 3.11+, pydantic>=2.6, and a YAML parser — ruamel.yaml or PyYAML>=6; the examples use yaml.safe_load so no arbitrary object construction is possible.
  • Canonical identity: analytes referenced by rules are already resolved to a stable (loinc, specimen, method_version) key through the Test Code Taxonomy & Standards service, so a rule binds to identity rather than an instrument-local test code.
  • Rule model: the field contract and most-restrictive-wins composition defined in Auto-Verification Rule Configuration; this page is the concrete authoring format for those rows.
  • Version control: a Git repository with protected-branch review, so merges require the dual approval the rule model records.

Step-by-Step Implementation

Step 1: Write the rule file as valid, reviewable YAML

Keep one file per analyte family so a diff is scoped and reviewable, and put the release metadata at the top of the file so the version and its effective date travel with the rules. Each rule is a mapping with a stable rule_id, the canonical analyte key, the check type, a declarative condition, and an action. Nothing in the file is executable — a condition is data in a small fixed grammar, never Python.

yaml
# rules/sodium.yaml — auto-verification rules for serum sodium
ruleset_version: "rs-2026-06-1"
effective_from: "2026-07-01"
analyte: "2951-2|serum|architect-c4000-v4"   # LOINC | specimen | method
author: "a.tech"
approver: "b.director"

rules:
  - rule_id: "na-range-adult"
    check: range
    condition: "value within reference"
    action: verify
    priority: 0

  - rule_id: "na-critical-low"
    check: critical
    condition: "value <= 120"        # mmol/L, UCUM-normalized
    action: hold
    priority: 30

  - rule_id: "na-critical-high"
    check: critical
    condition: "value >= 160"
    action: hold
    priority: 30

  - rule_id: "na-delta-abrupt"
    check: delta
    condition: "delta_abs >= 10 within 72h"
    action: hold
    priority: 20

  - rule_id: "na-interference-hemolysis"
    check: interference
    condition: "hemolysis_index >= 2"
    action: reject
    priority: 40

The delta rule (na-delta-abrupt) is expressed in the same grammar as the range and critical rules: a comparison against a named quantity (delta_abs) with an optional temporal window (within 72h). The engine, not the file, knows how to fetch the prior result and compute delta_abs; the file only declares the threshold and the action, which keeps the clinical decision reviewable in isolation from the delta-check machinery that evaluates it.

Step 2: Load and validate the file with Pydantic v2

Parse the file with yaml.safe_load, then validate it against a strict Pydantic v2 model so a malformed rule fails at load time — in CI — rather than at the bedside. extra="forbid" rejects typo’d keys; the condition grammar is checked with a regex so a nonsense predicate cannot slip through.

python
from __future__ import annotations

import re
from datetime import date
from enum import Enum, IntEnum
from pathlib import Path

import yaml
from pydantic import BaseModel, ConfigDict, Field, TypeAdapter, model_validator


class Check(str, Enum):
    RANGE = "range"
    DELTA = "delta"
    CRITICAL = "critical"
    REFLEX = "reflex"
    INTERFERENCE = "interference"


class Action(IntEnum):
    VERIFY = 0
    REFLEX_ORDER = 1
    HOLD = 2
    REJECT = 3


_CONDITION = re.compile(
    r"^(value|delta_abs|delta_pct|hemolysis_index|icteric_index)\s+"
    r"(within reference|[<>]=?\s*-?\d+(\.\d+)?)"
    r"(\s+within\s+\d+h)?$"
)


class Rule(BaseModel):
    model_config = ConfigDict(frozen=True, extra="forbid")

    rule_id: str
    check: Check
    condition: str
    action: Action
    priority: int = 0

    @model_validator(mode="after")
    def _grammar(self) -> "Rule":
        if not _CONDITION.match(self.condition):
            raise ValueError(f"{self.rule_id}: condition not in allowed grammar: "
                             f"{self.condition!r}")
        return self


class RuleSet(BaseModel):
    model_config = ConfigDict(frozen=True, extra="forbid")

    ruleset_version: str
    effective_from: date
    analyte: str
    author: str
    approver: str
    rules: list[Rule] = Field(min_length=1)

    @model_validator(mode="after")
    def _governance(self) -> "RuleSet":
        if self.author == self.approver:
            raise ValueError(f"{self.ruleset_version}: author and approver must differ")
        ids = [r.rule_id for r in self.rules]
        if len(ids) != len(set(ids)):
            raise ValueError(f"{self.ruleset_version}: duplicate rule_id in set")
        return self


def load_ruleset(path: Path) -> RuleSet:
    raw = yaml.safe_load(path.read_text(encoding="utf-8"))
    return RuleSet.model_validate(raw)

TypeAdapter is imported for the batch case where a directory of files is validated at once; a single file goes through RuleSet.model_validate. The _governance validator enforces two of the invariants the rule configuration model requires — distinct signers and unique rule identifiers — at parse time, so neither can reach a released version.

Step 3: Version the file in Git and tag the release

The file lives in a repository; the release is a tag. Author edits on a branch, open a pull request, and the protected-branch review is where the second identity approves — the same dual approval the approver field records. Merging to the main branch and tagging with the ruleset_version mints the immutable release; the tag is what the promotion step resolves.

bash
git switch -c na-critical-low-tightening
git add rules/sodium.yaml
git commit -m "sodium: tighten critical-low hold to 120 mmol/L (rs-2026-06-1)"
git push -u origin na-critical-low-tightening
# open PR; a second reviewer approves -> dual approval recorded in the merge
git switch main && git merge --ff-only na-critical-low-tightening
git tag -a rs-2026-06-1 -m "sodium ruleset effective 2026-07-01"
git push origin main --tags

Because the tag is immutable and the file at that tag is content-addressable, any historical result can be replayed against the exact rule text that was effective at its collection date — the reconstruction requirement CLIA retention imposes.

Step 4: Gate every change in CI

CI is the promotion boundary. It parses every rule file, validates it against the model, and runs the composition and coverage checks before a merge is allowed. A file that fails any check blocks the merge, so a bad edit degrades toward the last released version rather than shipping.

Config-to-promote flow for versioned YAML rules A left-to-right pipeline. An author edits a YAML rule file, which is validated in continuous integration against the Pydantic model. Valid files go to protected-branch review for dual approval, then a merge tags an immutable version, which is promoted to the effective-dated ruleset store. A failure branch returns files that do not parse or type-check from CI validation back to the author for revision. Edit rules.yaml on a branch CI validate pydantic · grammar Review dual approval Merge + tag immutable version Promote effective-dated parse / type fail Blocked — merge refused last version stays effective

A GitHub Actions job that runs the validator over the whole rules directory is enough to enforce the boundary:

yaml
# .github/workflows/validate-rules.yml
name: validate-rules
on: [pull_request]
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: pip install "pydantic>=2.6" "pyyaml>=6"
      - run: python -m ruleset.validate rules/

Verification & Testing

Prove the loader rejects the failure modes that matter, because a rule file that silently loads wrong is worse than one that fails loudly. Round-trip the model to confirm the parsed rule set is byte-stable through serialization — the property the audit replay depends on.

python
import pytest
from pydantic import ValidationError

from ruleset import RuleSet, load_ruleset


def test_rejects_self_approval(tmp_path):
    p = tmp_path / "bad.yaml"
    p.write_text(
        "ruleset_version: rs-x\n"
        "effective_from: 2026-07-01\n"
        "analyte: '2951-2|serum|v4'\n"
        "author: same.person\n"
        "approver: same.person\n"
        "rules:\n"
        "  - {rule_id: r1, check: range, condition: value within reference, action: verify}\n"
    )
    with pytest.raises(ValidationError, match="author and approver must differ"):
        load_ruleset(p)


def test_rejects_bad_condition_grammar(tmp_path):
    p = tmp_path / "bad2.yaml"
    p.write_text(
        "ruleset_version: rs-y\n"
        "effective_from: 2026-07-01\n"
        "analyte: '2951-2|serum|v4'\n"
        "author: a\n"
        "approver: b\n"
        "rules:\n"
        "  - {rule_id: r1, check: delta, condition: 'do something clever', action: hold}\n"
    )
    with pytest.raises(ValidationError, match="not in allowed grammar"):
        load_ruleset(p)


def test_round_trips(tmp_path):
    p = tmp_path / "ok.yaml"
    p.write_text(
        "ruleset_version: rs-z\n"
        "effective_from: 2026-07-01\n"
        "analyte: '2951-2|serum|v4'\n"
        "author: a\n"
        "approver: b\n"
        "rules:\n"
        "  - {rule_id: r1, check: critical, condition: 'value <= 120', action: hold}\n"
    )
    rs = load_ruleset(p)
    assert RuleSet.model_validate_json(rs.model_dump_json()) == rs

Expected: all three pass. The first two confirm the governance and grammar guards fail closed at load time; the third confirms the parsed set survives a serialize/deserialize round trip unchanged.

Compliance Note

Storing rules as versioned YAML directly serves 21 CFR Part 11 §11.10(e),(k) change control: every edit is an attributable, timestamped Git commit, every release is an immutable tag, and the protected-branch review supplies the two distinct approver identities the record requires. CLIA §493.1256 and CAP GEN.41350 require that auto-verification criteria be documented, validated before use, and reproducible on review — the YAML file is the documentation, the CI gate is the pre-use validation, and the tag makes the criteria that were effective at any past collection date reconstructable for the CLIA §493.1105 retention window. Because the merge and tag are recorded in the version-control audit trail, the change history aligns with the HIPAA-compliant audit trail expectations for the rest of the system.

Troubleshooting

A rule file loads but a typo'd key is silently ignored.

Root cause: the model does not forbid extra keys, so prioritty: 10 parses as an unknown field and the rule falls back to the default priority. Fix: set model_config = ConfigDict(extra="forbid") on both Rule and RuleSet; a misspelled key then raises a ValidationError in CI instead of shipping a rule with the wrong precedence.

YAML parses a bare threshold as the wrong type.

Root cause: unquoted YAML scalars coerce aggressively — value: 120 is an int, but a version like 1.10 becomes a float 1.1 and loses the trailing zero, and on/off become booleans. Fix: quote every string scalar (ruleset_version: "rs-2026-06-1"), keep numeric thresholds inside the quoted condition string where the grammar regex controls parsing, and use yaml.safe_load so no implicit object construction occurs.

Two branches tag the same ruleset_version and the promotion is ambiguous.

Root cause: ruleset_version was reused across two edits, so the immutable tag no longer identifies one file state. Fix: treat the version string as monotonic and unique — bump it on every release (rs-2026-06-1 → rs-2026-06-2) — and add a CI check that fails if the tag already exists, so a duplicate version can never be promoted.

A delta rule validates but never fires at runtime.

Root cause: the condition names a quantity the engine does not compute — for example delta_ratio when only delta_abs and delta_pct are supported — so the grammar regex must reject it. Fix: keep the condition grammar and the engine’s evaluable quantities in lock-step; add the quantity to both the regex and the evaluator together, and cover it with a test that loads a rule using it and asserts the engine produces a verdict.

Part of: Auto-Verification Rule Configuration.