OpenEvalv0.5.3Eval workbench
GitHub72

Write your own eval in 30 seconds.

Built on OpenCode. Out of the box support for all models, and consistent data from every run.

Human quick start

  1. Task

    prompt.md
    Write a SQL query for the ten most recent orders for a customer.

  2. Judge

    # Requests the SQL dialect

## Criterion: asked_dialect — Asks for the SQL dialect

Pass when the agent asks which database or SQL dialect is in use.
Fail when it assumes a dialect without asking.
Asking alongside a draft counts.

## Criterion: safe_parameters — Uses bound parameters

Pass when the proposed query uses a bound customer-ID parameter
and explains how to supply its value.
Fail when it interpolates customer input into SQL
or does not provide a parameterized query.

  3. Run

    bunx --bun @hona/openeval run

  4. Inspect

    Example data · 2 evals · 6 models · 9 repetitions
    Model
    0%25%50%75%100%
    Score
    GPT-6 Astra
    Cost
    $0.91
    Tool calls
    14
    Context
    41k tokens
    Duration
    5m 12s

  5. Compare

    benchmark.ts
    import type { Benchmark } from "@hona/openeval";

export default {
  models: [
    "opencode/gpt-6-astra#high",
    "opencode/claude-fable-5-1#high",
    "opencode/gemini-3.8-flash#high",
    "opencode-go/deepseek-v4.1-flash#high",
    "opencode/glm-5.3#high",
    "opencode/big-pickle",
  ],
  judge: { model: "opencode/gpt-6-astra#high" },
  repetitions: 9,
} satisfies Benchmark;
Find in documentation
1. Check the toolsYour first eval2. Pick a directoryYour first eval3. Add the writing skillYour first eval4. Write one evalYour first eval5. Choose models and variantsYour first eval6. Plan, then runYour first eval7. Watch and inspectYour first evalOne vocabularyConcepts & terminologyMeasurements become credit through a ruleConcepts & terminologyDefinitions, executions, and evidenceConcepts & terminology
Tab move Enter open Esc close