Judgments

A Judgment is a single-shot AI evaluation of one record: the model reads the record fields you declare, and answers inside a typed output contract you declare. There is no agent loop and no tool use. You define which fields it may produce, which values each field may take, and how confident the model must be before anything happens. Every evaluation leaves a decision row recording what the model saw, what it answered, its confidence, and its stated reasoning.

When you’d use it

Set a value from unstructured text: priority from a description, category from an email body, sentiment from a comment.
Triage on arrival: bind a judgment to a business rule so every new record is classified the moment it is created.
Backfill an existing table: run the same judgment in bulk over a filtered record set, with all proposed writes collected into one reviewable plan.
Keep an outcome true, continuously: declare a standing goal (a drift predicate plus a cadence) and the platform fills the gaps on schedule (“every record’s priority stays consistent with its description”).
Keep a human in the loop: route every decision, or only the uncertain ones, to a review queue where a person applies, edits, or dismisses it.

What you manage

Judgments live at AI → Decisions (/admin/judgments). A judgment names a target table, the context fields the model sees, the output contract (fields, allowed values, types), a knowledge skill that carries the judging instructions, a confidence threshold, and one of three apply policies:

direct: an above-threshold result writes to the record immediately, audited.
batch_changeset: results become one reviewable plan; a human approves the whole set before anything is written.
human_queue: every decision waits in the review queue for a person.

The Review queue tab lists pending decisions: those queued by policy, and those that fell below the confidence threshold under any policy. For each item you see the proposed fields, the confidence against the threshold, and the model’s rationale. You can Apply the output as-is, Override the fields before applying (your edits are checked against the same output contract), or Dismiss without touching the record. The write is audited as you, not as the AI.

A judgment can also name an escalation investigator: an investigate-mode Agent that takes a deeper, read-only look at a pending decision when you ask for one. The queue shows an Investigate button on those items; the agent reads the record and its context, then returns a finding (summary, evidence, recommendation) you review next to the decision before resolving it. The investigation never writes anything, and resolving the decision remains your action.

You can also author a judgment conversationally: ask the chat to plan a judgment, review the validation diagnostics it shows, and confirm creation. Asking the chat why a judgment decided a value (“why P3?”) answers from the stored decision record, and asking whether a new version is ready to activate answers from its stored eval results.

Standing goals (scheduled reconcile)

A judgment can carry a reconcile section — a drift predicate, a cadence (hourly, daily, or weekly), and a per-tick chunk limit. That turns it into a standing goal: a declarative outcome the platform keeps true instead of a one-off run. On each due tick the predicate selects the records that are out of the desired state and the judgment evaluates them, under the same apply policy, contract, and audit as any other invocation.

The predicate uses the same filter vocabulary as record list views, including relative datetime values like now-7d for “older than a week” drift conditions, re-resolved at every tick. Two guards keep the goal honest and affordable:

Pending records are skipped. A record whose decision is waiting in the review queue, or riding a still-open changeset plan, is never re-judged — no duplicate queue items, no competing writes.
One evaluation per record per cadence. A record already judged within the current cadence window (including a failed attempt) waits for the next window instead of burning model calls every tick.

Every tick leaves one coverage row on the judgment page: how many records matched the predicate, how many were skipped by each guard, how many were eligible, processed, and remaining for the next tick. A tick that fails leaves a failed row with a distinct error code — never a silent gap. A Run now button ticks the goal on demand, and asking the chat for a judgment’s reconcile status answers from the same coverage rows.

Tip: make the drift predicate test the output field (for example priority is empty when the judgment fills priority), so records that are successfully filled stop matching. With apply_policy: direct this is what keeps a standing goal from re-selecting the same records every cadence — the editor warns about direct-apply standing goals for this reason.

Versions and the eval gate

Every save of a judgment records a version, so the history of what was live when is never lost. Editing and saving applies immediately, as before. When you want to test a change before it goes live, use Save as draft on the judgment page instead: the draft sits next to the live version, immutable, until you activate or reject it.

Eval cases are what drafts are tested against. The platform collects them for you from the review queue: when a reviewer overrides a decision, the record’s input and the reviewer’s corrected fields become a case; when a reviewer applies a decision as-is, that confirmation becomes a case too. You can also add cases by hand for edge situations you want covered. Cases can be disabled but never deleted, because past eval runs scored against them.

Run eval on a draft replays every enabled case through the draft’s exact configuration and scores the answers: passed, failed (with a per-field diff), or incompatible when the case’s expected label no longer fits the draft’s contract. Incompatible cases are reported but excluded from the accuracy figure, never silently counted either way.

The gate: if a judgment has at least one enabled eval case, a draft cannot be activated until an eval has been run against that exact draft. The gate guarantees the eval happened; you judge whether the score is good enough. A judgment with no cases yet activates drafts freely.

The crystallizer (learning from overrides)

Corrections compound instead of evaporating. When reviewers have overridden enough of a judgment’s decisions since its last proposed revision (three new override cases by default, tenant-configurable), the crystallizer drafts an improved version for you: it reads the current judging skill, the output contract, the override cases (what the model said, what the human corrected it to, and why), and a sample of confirmed decisions as “keep this working” anchors — then writes a revised skill body that generalizes the corrections.

Nothing goes live on its own. The improved instructions land as a draft skill plus a draft judgment version referencing it, marked with a crystallizer chip on the judgment’s Versions panel alongside a summary of what changed and which overrides drove it. It rides the same eval gate as any draft: run the eval (the override cases that motivated the draft are part of the case set, so the score shows whether the revision actually fixes them), review the drafted skill body, then activate, which flips the drafted skill live together with the version — or reject, which retires the drafted skill. After a rejection the crystallizer stays quiet until enough new override cases accumulate.

The judgment page’s Crystallizer panel shows the signal (new override cases against the threshold), the open draft if one exists, a Crystallize now button, and one honest row per run — drafted, skipped (with the distinct reason), or failed with an error code. The scheduler checks each judgment daily; only one crystallizer draft can be open at a time. Asking the chat whether a judgment is learning from its corrections answers from the same rows.

The crystallizer also watches for patterns that no longer need a model at all. When recent decisions show that a single field value predicts an output with high support and precision (twenty agreeing decisions at 95% by default), it proposes a deactivated business rule capturing the pattern — condition, don’t-overwrite guard, and set-field action, with the evidence counts in its description. Activating it in the rules admin makes that pattern deterministic and free; leaving it inactive costs nothing. Proposals are deduplicated, so declining one doesn’t mean seeing it again every day.

Good to know

The contract is hard. A model answer with an unknown field, a value outside the allowed list, or a missing confidence is rejected and recorded as a failed decision. Nothing out of contract ever reaches a record.
Below threshold never writes. A low-confidence decision is recorded and waits for a person, whatever the apply policy.
Bulk runs are one approval, not thousands. A bulk run under batch_changeset produces a single plan with one write step per record; below-confidence records are listed as exceptions rather than silently included.
Every decision is auditable. The decision row keeps the input snapshot, model, skill version, output, confidence, rationale, and, once resolved, who resolved it and what was actually written.
Human overrides obey the same contract. The review queue rejects an override value outside the declared allowed values, the same rule the model is held to.
Resolving queue items builds your eval set. Approvals and overrides in the review queue become eval cases automatically, so the judgments you review the most are also the ones best protected against a bad revision.
Eval runs are replays, not decisions. An eval never writes to records and never adds rows to the decision audit or the review queue; its per-case results live on the eval run itself.
One investigation at a time. Requesting an investigation for a decision that already has one in flight returns the existing run instead of dispatching a second.

How it connects

A judgment’s knowledge lives in a Skill, so the judging instructions are versioned and reviewable like any other prompt content. Batch results ride Plans and governance for approval and execution. On-event binding uses Business rules with the run-judgment action, and record writes follow the same audit trail as any other change. What a judgment may read and write is bounded by the Safety model.

Terms: every noun here is defined in the glossary.