the chance-level baseline
shrug.ai: stupid, fast
Benchmark your decision and judgment endpoints against chance. shrug.ai is a drop-in random-draw baseline for OpenAI Decisions and TypeSafe System One — if your model can’t beat a shrug, you have a problem.
Free up to 1,000 requests/month. No model, no latency, no excuses.
How it works
Drop in a baseline
Point your OpenAI Decisions or TypeSafe System One client at shrug.ai. Same request shape, same response shape — no code changes beyond the base URL and key.
Get a principled random draw
Every question is answered by sampling, not thinking: choices, predicates, and scores come back with calibrated probabilities. That is your chance-level floor.
Bias the dice (optional)
Add a shrug.distributions block to weight options by name or set p_true for a predicate. Normalized automatically, so you control the null hypothesis.