Reward Atlas
All field guidesEsc to close
Sign in
LLM / AgenticPublicPublishedv1.0Advanced

SQL-Detective-v1

A multi-turn agent answers analytics questions by querying a read-only SQLite sandbox. Tool use, verifiable rewards, 12-turn budget. Maintained by Bhaskar & Jaswanth · updated 28 Aug 2026.

Dataset
2,400 q · 12 DBs
Action
tool call | final answer
Turn cap
12
Best baseline
GRPO-7B 61.4%
03

Observation space

What the policy sees

The observation is the running conversation. Tool results are appended as tool messages, truncated to 200 rows.

system
You are a data analyst. Use the tools to inspect the database, then call submit_answer with the exact result rows. You have 12 turns.
user
Which three product categories had the largest drop in revenue between Q1 and Q2 2024?
assistant
list_tables()
tool
["orders", "order_items", "products", "categories", "customers"]
assistant
describe_table("order_items")
tool
order_id INTEGER · product_id INTEGER · quantity INTEGER · unit_price REAL

Context window: 16k tokens. When exceeded, the oldest tool messages are replaced with [elided: N rows], and the system prompt and question are always kept.

Guide details
Version
Type
LLM / Agentic
API
OpenEnv
License
Apache-2.0
Seeds
3
Domains
tool-usedatacoding
Install
pip install reward-atlas-sql-detective

Issue with this step? Suggest an edit