Research · Updated 11 Oct, 04:20 am IST
Hacker News post explains 'decision models' and constrained outputs for calibrated answers
Why it matters for readers: It shows a practical technique to get structured outputs and why model confidence may not equal real-world accuracy.
- Structured Output can be used to guarantee valid JSON from a language model by constraining its outputs, but typical decoding still requires multiple token passes.1
- Decision models (called 'System one' in the post) assume a fixed set of selectable options and can emit only those tokens when the vocabulary is masked.1
- The example in the post shows a standard language model needing 11 passes to generate the final constrained output.1
- Masking the vocabulary so the model can only emit fixed options helps force selection but does not ensure the chosen option is correct or that token probabilities equal true answer confidence.1
Get a brief like this every morning
Uzha reads hundreds of sources and gives you the stories that matter for your work, with every source linked. Free.
Get started