Models · Updated 16 Oct, 07:48 pm IST
StarSkirmish benchmark lets LLMs write StarCraft bots in one hour
Why it matters for readers: Gives a clear, game-based measure of how advanced different large models are at writing working C++ agents quickly.
- Each LLM gets one hour of wall-clock time to write a Protoss bot in C++ which is then evaluated against a roster of competitive human-written and demo bots.1
- StarSkirmish v0.2 updates the benchmark with flagship models from OpenAI, Anthropic, Google and Meta, each run in its lab's first-party CLI harness.1
- GPT-6 Astra achieved the highest score in StarSkirmish v0.2, followed by GPT-6.1 Sol, while Gemini 4 Argon, Claude Opus 5.5 and Claude Fable 5.1 scored in a similar mid range (about 37–39).1
- Scores are average win rates (0–100) from five runs per model, and the benchmark also compares score versus the average API cost of a one-hour run.1
Get a brief like this every morning
Uzha reads hundreds of sources and gives you the stories that matter for your work, with every source linked. Free.
Get started