> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ambral.ai/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> To set Ambral up in a project, run `npx ambral@latest skill install` in the project's root. It installs Ambral's skill, matched to the CLI's version, and the skill says how to do everything these pages describe with the `ambral` CLI. Prefer it to working from these pages. Never ask anyone for an Ambral API key, and never handle one: the person creates it in the web app and stores it where their app reads its secrets.

# A retrieval agent

> Grade the answer, not how it got there, and cheaply.

Grade the answer, not how it got there, and cheaply.

Based on the `criteria` preset.

A retrieval agent: it looks up documents, then answers the user's question from them.

Only the final answer matters, so only it is asked for and graded (`take: "last"`), and every score comes from
criteria grading it on its own terms, as the criteria preset does: nothing compares it with the baseline's. Its
three checks are quick and read the same answer, so they're asked in one judge call, and grounding weighs most.
Compare the research agent, which judges the path as well as the report.

With no scorer comparing with the baseline, production is graded on every criterion as a candidate is: run it with
`--subjects production,<model>` to read a candidate's answers beside the agent's own on the same criteria, or
`--subjects production` alone to grade what the agent sends today, at the cost of the judge calls alone.

An example to learn from, not a starting point: start a config from `ambral tasksets scoring preset`, and take
from this one the choices that fit your agent.

```yaml theme={null}
# preset: criteria
# title: A retrieval agent
# A retrieval agent: it looks up documents, then answers the user's question from them.
#
# Goal: grade the answer, not how it got there, and cheaply.
#
# Only the final answer matters, so only it is asked for and graded (`take: "last"`), and every score comes from
# criteria grading it on its own terms, as the criteria preset does: nothing compares it with the baseline's. Its
# three checks are quick and read the same answer, so they're asked in one judge call, and grounding weighs most.
# Compare the research agent, which judges the path as well as the report.
#
# With no scorer comparing with the baseline, production is graded on every criterion as a candidate is: run it with
# `--subjects production,<model>` to read a candidate's answers beside the agent's own on the same criteria, or
# `--subjects production` alone to grade what the agent sends today, at the cost of the judge calls alone.
#
# An example to learn from, not a starting point: start a config from `ambral tasksets scoring preset`, and take
# from this one the choices that fit your agent.

id: "replay"

# Only the last assistant turn, the answer: one reply per trace and subject, and one judge call. If the agent's last
# turn can be a tool call rather than its answer, take the last turn where production called no tool instead (its
# answer in text, or an empty turn): `tools: "none"` here, matched on the recorded turn, as `points.tools` always is.
# Either way a subject may search again where the baseline answered, and the group grades that search as its
# answer, which should fail `answers_the_question`, as a subject that never answers should. A group's
# `turns: { tools: "none" }` would skip that reply instead, leaving it with no score, left out of the run's, and so
# hide a subject that never answers.
points:
  tools: null
  take: "last"

# No checks: with one point per trace, a check's score would be its scorer's score at that point.
episode:
  rewards:
    replies: null
    trajectory: null
    checks: {}
  metrics:
    replies: null
    trajectory: null
    checks: {}

taskset:
  # Nothing compares with the baseline: a grounded answer can be worded many ways and drawn from other documents than
  # the baseline's, so its reply is no bar to match. The criteria grade each answer on its own terms instead.
  fromBaseline:
    rewards: {}
    metrics: {}
    judges: {}
  criteria:
    answer:
      model: "<judge>"
      # A grounded answer follows its retrieval closely, so 20 messages hold the documents it draws on, and a larger
      # maxChars keeps more of each. `messages` times `maxChars` must fit the judge's context: a prompt too long for it
      # fails its call, and a failed call fails the trial.
      # A search tool's description can say what its results hold (a date range, which sources it covers), and an answer
      # may rest on that. `called` shows the judge the definitions of the tools whose calls or results it is shown, so
      # such a claim isn't marked unsupported; `all` would add the tools the agent left unused.
      context:
        messages: 20
        maxChars: 15000
        tools: "called"
      # Three quick checks on one answer, so one call asks them all, side by side.
      perCall: null
      criteria:
        # Grounding is what a retrieval agent is for, so it counts three times as much as either other check.
        - name: "claims_from_documents"
          weight: 3
          text: "Each claim the reply makes comes from a document retrieved in the messages before it."
          notApplicable: "The reply makes no factual claim, or the documents it draws on aren't shown."
          # The judge picks one: unsupported 0, partly 0.5, grounded 1, or n/a (no score) where `notApplicable` holds.
          choices:
            - "unsupported"
            - "partly"
            - "grounded"
          meanings:
            unsupported: "It makes a claim no retrieved document supports."
            partly: "Its main answer is supported, but a detail isn't."
        - name: "answers_the_question"
          text: "The reply answers the very question the user asked, rather than a nearby one."
          notApplicable: "The question isn't shown."
        - name: "says_when_unanswered"
          text: "When the retrieved documents don't answer the question, the reply says so instead of guessing."
          notApplicable: "The retrieved documents answer the question, or the question or the documents retrieved for
            it aren't shown."

required: []
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.