ScriptBench — the My Weird Prompts script-writer benchmark

ScriptBench is our own internal benchmark, open-sourced. It measures one workload: a ~28k-token brief turned into a ~3,800-word, two-host comedic dialogue for text-to-speech. It probably isn't the right benchmark for your use case. Fork it and tailor the task card, rubric and measures to yours.

My Weird Prompts is a podcast written entirely by language models. This page shows which model writes it now, how candidates scored on ScriptBench (formerly MWP-Script; one ~3,800-word, two-host script in a single pass), which ones failed on censorship or refusals, and informal verdicts on models that wrote aired episodes. The data is read live from the eval repo.

Related work: PodBench (Tencent, ACL 2026, arXiv 2601.14903; code, MIT) is the closest public benchmark in this space: 800 samples, inputs up to 21k tokens, multi-speaker podcast scripts, deterministic checks plus an LLM judge.

Current writer

Loading…

Leaderboard

Technical fails and exclusions

A technical fail is any refusal, by the model or by a hosting provider's filter cutting the output off. A technical fail is never ranked and is out of the production round robin.

Exclusions

Models not screened again, with the reason: technical fails, superseded generations (only the newest generation of a family is evaluated), geo-blocks, already aired.

Informal evaluations

Claude reads one or two aired scripts per model and gives a light verdict; Daniel's own impressions are quoted where he gave them. Production-trial evidence, not a benchmark.

Loading…