Skip to main content
Create and read a workload’s tasksets, line their runs up by run or by model, set how they’re scored, and read the prompts their traces open with Every ambral command also takes the global flags.

ambral tasksets create

Create a taskset of a workload’s recorded traces. Each trace’s branch is fixed when it is created, so every run of it replays the same sessions Examples

ambral tasksets list

List the org’s tasksets, or one workload’s, newest first, a page at a time with how many there are: each one’s kind, how many tasks it holds, its scoring version, its last run not marked local with how it went, and how many of its runs are marked local Examples

ambral tasksets get

Show a taskset and a page of its tasks: a replay taskset’s traces’ branches, or an offline taskset’s scenarios, and its scoring version Examples

ambral tasksets results

Line a taskset’s runs up side by side, by run (what changed over time on one model) or by model (where each model breaks): each column’s counts, a row per scenario (or a replay taskset’s branch) with each column’s repeats passed, and each check’s mean. Runs marked local are left out unless asked for (--source), and counted Examples

ambral tasksets scoring preset

List the scoring presets, each with when to use it, or print one as a YAML scoring config to start from, every field explained in a comment above it Examples

ambral tasksets scoring get

Print a taskset’s current scoring config as YAML, every field explained in a comment above it. It is whole: every key a file may leave out is written at its default Examples

ambral tasksets scoring set

Save a scoring config as a taskset’s next version, after checking it. A config with mistakes prints each, at its field, and saves nothing Examples

ambral tasksets prompts list

List the system prompts the taskset’s traces open with, the most traces first: each one’s id, how many traces open with it, and its first line, and how many open with none Examples