Skip to main content
Judge the path, with rules that must hold at every reply. Based on the trajectory preset. A coding agent: it edits files, runs tests and commands, and says when the work is done. The session is judged as a whole, as the trajectory preset does: the judge compares the subject’s step at each point with the baseline’s. But some rules hold at every reply, so it adds them as criteria, which grade the subject’s reply alone, and scores its replies all-or-nothing beside the judge’s verdict and whether the session ends with tested work. Compare the research agent, which balances its verdict with a report’s score instead (mean). An example to learn from, not a starting point: start a config from ambral tasksets scoring preset, and take from this one the choices that fit your agent.