## Sources

1. [From Capabilities to Responsibilities](https://www.oreilly.com/radar/from-capabilities-to-responsibilities/)

---

### **From Capabilities to Responsibilities** by Artur Huk

This article argues that as AI moves from experimental demos to high-stakes production environments, the industry must transition from focusing on agent **capabilities** to defining and enforcing agent **responsibilities** [1, 2]. The author proposes the **Responsibility-Oriented Agent (ROA)** pattern as a framework for building governable AI systems that can execute critical real-world actions with deterministic oversight [3, 4].

#### **Main Arguments**
*   **The Scalability Trap of Human-in-the-Loop (HITL):** Traditional HITL models, where humans must approve every decision, become **operational bottlenecks** at scale [5]. This leads to **alert fatigue**, where humans begin "clicking through" approvals without verification, creating technical debt and governance failure [5].
*   **Capabilities vs. Responsibilities:** Most AI design asks what an agent *can* do (capabilities), but high-stakes systems must focus on what an agent is **authorized** to do (responsibilities) [6, 7]. A responsibility-oriented approach uses **hard boundaries** encoded in code rather than mere "suggestions" in prompts [8].
*   **Governance by Exception:** The scalable alternative to HITL is **Human-Over-The-Loop (HOTL)** [9, 10]. In this model, humans act as **Policy Designers** who define contracts, while the system operates autonomously within those bounds, escalating only truly exceptional cases [9, 10].
*   **Architecture Over Alchemy:** Production resiliency should not rely on the "alchemy" of prompt engineering or hoping for LLM alignment [4, 11]. Instead, it requires a **deterministic execution kernel**—a "Kernel Space" that validates every action before it affects the real world [12, 13].

#### **Key Takeaways**
*   **The Five Engineering Pillars of ROA:**
    1.  **Responsibility Contract:** A machine-readable, versioned contract that defines the hard boundaries of an agent's authority (e.g., maximum trade size or prohibited industries) [8, 14].
    2.  **Mission:** An immutable optimization objective that defines the "North Star" for the agent’s reasoning within its authorized space [15, 16].
    3.  **Epistemic Isolation:** Agents do not issue direct commands; they emit **Policy Proposals**—untrusted claims that the Runtime must validate before execution [17, 18].
    4.  **Epistemic Longevity:** Agents maintain a **decision trajectory** across cycles to prevent "decision amnesia," where an agent repeats previously rejected intents [19-21].
    5.  **Decision Telemetry:** Every proposal is bound to a **Decision Flow ID (DFID)**, creating an immutable audit trail that connects inputs, validation outcomes, and execution receipts [21, 22].
*   **Separation of Concerns:** ROA agents produce two distinct outputs: an **Explain** narrative for human auditors and a **Policy** proposal for machine enforcement [19, 23]. The system never parses natural language narratives for execution logic [23].
*   **Wrapping Existing Frameworks:** The ROA pattern is intended to **wrap**, not replace, frameworks like LangChain or AutoGen [24, 25]. By restricting these frameworks to a sandboxed tool for emitting proposals, developers maintain reasoning power while gaining deterministic governance [25].

#### **Important Details**
*   **Escalation Triggers:** Systems transition to an **ESCALATED** state if an agent exceeds authority limits, if confidence drops below a set threshold, or if external API errors occur [10, 26]. 
*   **Escalation Budgets:** To prevent "Escalation DDoS," agents have a rate-limited budget (e.g., 3 per hour). If exhausted, the agent is **SUSPENDED** until a human intervenes [27].
*   **Frozen Context and JIT Verification:** Operators review decisions based on the exact world-state (T0) the agent saw, but any eventual override is subjected to **Just-In-Time (JIT) Verification** to ensure the world hasn't drifted too far (T1) for the action to still be valid [28].
*   **Role-Based Scoping:** Defining a specific role (e.g., insurance underwriter) automatically narrows the **data context** the agent needs, which improves LLM reliability by avoiding the "Lost in the Middle" problem associated with massive context windows [29, 30].
*   **Costs of ROA:** Implementing this pattern introduces **engineering overhead**, including added latency for validation gates, the need for versioned contracts, and increased storage for detailed telemetry [31]. These costs are framed as necessary investments for high-risk systems where the downside of an incorrect action is high [4].