## Sources

1. [Linear Thinking, Nonlinear Costs](https://www.oreilly.com/radar/linear-thinking-nonlinear-costs/)

---

### **Linear Thinking, Nonlinear Costs – O’Reilly** by Nicole Koenigstein

**Main Arguments**

*   **Economical Unsustainability:** Many AI agent systems become financially unviable long before they reach technical maturity because teams focus on model choice and prompt design while ignoring the **missing optimization layer** [1].
*   **Nonlinear Cost Scaling:** Unlike traditional software, a single user request in an agent system often triggers a cascade of expensive steps—including routing, reasoning, reflection, and guardrail checks—that scale costs nonlinearly [2].
*   **Abstraction Trap:** Tools like Claude Code and Codex make generating agent workflows easier, but this abstraction hides the underlying mechanics, leading to code that passes unit tests but remains **economically wasteful** [1, 3, 4].
*   **The Return of "Old" Computer Science:** The article argues that the solution to modern AI agent inefficiencies lies in applying classical algorithmic principles like **backtracking, dynamic programming, and memoization** [2, 4, 5].

**Key Takeaways**

*   **Engineering Craft vs. Code Generation:** While coding agents can implement systems, the human engineer must still recognize repeated contexts, unpruned branches, and recomputed decisions to keep systems viable [4].
*   **Failure as Information:** In agent workflows, a "retry" should not simply be a repeated attempt; it must provide **structured feedback** to the model to change its state (prompt, tool choice, or retrieved evidence) for the next trial [6, 7].
*   **Optimization Follows Topology:** The specific optimization strategy must match the system's architecture (centralized, decentralized, independent, or hybrid) to effectively address its unique overhead and communication costs [5, 8].
*   **Reliability Math:** Improving a step with a 60% success rate to 99% reliability requires multiple retries, but because LLMs are not "fair dice," simply bumping the temperature is insufficient; the underlying state must evolve [6, 9].

**Important Details**

*   **Classical Optimization Patterns Applied to Agents:**
    *   **Memoization:** Storing the results of expensive reasoning. If an agent encounters a previously seen state, it should use a lookup instead of a new LLM call [10-12].
    *   **Pruning:** Eliminating search paths that will not lead to a solution, such as killing a reflection loop that no longer yields structural improvements [11, 12].
    *   **Dynamic Programming:** Breaking problems into overlapping subproblems and sharing results—such as sharing codebase analysis across multiple specialized agents [12, 13].
*   **Prompt Caching:** Recognized as the "first win" for optimization, it reduces the cost of repeated context (system prompts, tool definitions, etc.) but does not address repeated decision-making [10].
*   **Topology-Specific Strategies:**
    *   **Centralized Systems:** Focus on memoizing the orchestrator’s decisions [14].
    *   **Decentralized Systems:** Prioritize prompt caching on shared context and pruning unproductive peer exchanges [14].
    *   **Independent Swarms:** Rely heavily on memoization and pruning to prevent multiple agents from paying for the same answer to semantically equivalent questions [15].
    *   **Hybrid Systems:** Use dynamic programming for shared subproblems within clusters and memoization for coordinator routing decisions [15].