## Sources

1. [Don’t Blame the Model](https://www.oreilly.com/radar/dont-blame-the-model/)

---

### "Don’t Blame the Model" by Sruly Rosenblat

**Main Arguments**
*   The reputation Large Language Models (LLMs) have for being unreliable—such as giving contradictory answers or struggling to follow formats—is not solely the fault of the models themselves; it is heavily influenced by the API endpoints and surrounding tooling provided by AI companies [1].
*   Model providers make policy decisions, rather than purely technical ones, to artificially limit the tools, visibility, and control that third-party developers have over the models [2, 3]. 
*   These restrictions directly impact what applications can be built, system reliability, and a developer's ability to steer outcomes, especially as LLMs are increasingly deployed in high-stakes fields like medicine and law [2, 4].
*   While increasing model intelligence is the most common approach to improving performance, enhancing developer control and system transparency is a crucial, yet neglected, avenue for ensuring reliability [5].

**Key Takeaways**
*   **The Problem with Chat Templates:** Modern LLM APIs rely heavily on chat templates that enforce a strict separation between user messages (input) and assistant messages (output) [2]. This setup prevents developers from prefilling model responses, which severely limits their ability to control the beginning of an output or partially regenerate incorrect answers [6]. 
*   **Lack of Output Controls:** Standard APIs often lack advanced control mechanisms like constrained decoding [7]. While some APIs offer structured JSON output, they do not allow developers to precisely restrict tokens for other formats, such as enforcing XML structure, writing without certain letters, or validating chess moves at inference time [7].
*   **Hidden Reasoning and Confidence Metrics:** Major AI providers like Anthropic, Google, and OpenAI intentionally hide crucial data from developers, such as full chain-of-thought reasoning tokens and log probabilities (logprobs) [1, 8, 9]. 
*   **The Risk of Over-Restriction:** While AI companies restrict access to prevent prompt injections, ensure safety, and stop competitors from mimicking their models via distillation, these restrictions hurt end users [10-13]. Failing to provide diagnostic tools is an AI safety concern in itself [4].
*   **The Appeal of Open-Weight Models:** Local, open-weight models maintain popularity because they allow developers to trade complexity for reliability. They offer full transparency into reasoning and logprobs, and they support features like prefilling and constrained decoding [1, 13]. 

**Important Details**
*   **Logprobs (Log Probabilities):** These represent the probability of all possible options for the model's next token [1]. They are one of the best signals for measuring a model's confidence [8]. Currently, Google provides only the top 20 logprobs, OpenAI no longer provides them for GPT 5 models, and Anthropic has never provided them [8].
*   **Debugging via Reasoning Traces:** When a model hallucinates or provides a wrong answer, a full reasoning trace helps developers pinpoint exactly where it failed—whether it misunderstood the prompt, made a logical error, or just chose the wrong final token [9]. Summaries of these traces, which are currently the industry standard for proprietary models, obscure this diagnostic capability [9, 10].
*   **Distillation and Security Concerns:** Distillation is a technique where developers use the outputs of a strong model to cheaply train another model [12]. Providers hide logprobs and reasoning tokens to make this process harder and less informative, though it is still suboptimally possible using top-K probabilities [5, 12]. DeepSeek R1 was cited as a model that gained massive popularity despite security concerns, largely due to its open nature [12].
*   **Prefill Attacks:** While allowing developers to prefill model responses can increase the risk of prompt injections, companies already use classification models to defend against these attacks, and similar safeguards could be used for prefilling [11].
*   **Actionable Recommendations:** The author urges model providers to release a separate, more complex API endpoint that provides developers with [3, 14]:
    *   Full reasoning traces (with safety violations handled in the final answer).
    *   At least the top 20 logprobs over the entire output.
    *   Constrained decoding via regular expressions (regex) or formal grammars.
    *   Full control to prefill, stop, or branch assistant responses mid-generation.