This [2024 paper about ADAS](https://arxiv.org/pdf/2408.08435) (Automated Design of Agentic Systems) has stuck with me as I think about my work. I think that the longevity of this paper is explained by its simplicity and its general approach to control flow and agentic system design.

## Background: LLMs are useful but unreliable

Generally, LLMs are useful because of their capacity to respond to the unknown. While some domains remain inaccessible to current models, increasingly complex tasks can be effectively broken down into manageable components.

Two significant approaches are task decomposition or "divide and conquer," where a task is broken down into a tree of subtasks, and chain-of-thought, where a model is induced to communicate intermediary "reasoning" responses to self-prompt. Different approaches imply different corresponding control flows in agentic systems.

Regardless of how systems are designed or how much better LLMs become, the basic problem today remains unchanged from GPT-1: LLM output remains indeterminstic and untrustworthy, but encoding constraints within agentic systems to improve performance and/or reliability within a specific domain requires engineer-time to build and maintain while limiting their capacity to solve novel problems.

Until determinism and/or explainability are solved, increases in model accuracy will not fundamentally change the current problem-space of agentic system design.

## ADAS is meta-agentic programming

This means that agentic system design is relatively stable (thus why a 2024 paper remains quite relevant) and worthy of significant attention. I think ADAS is a key concept within the domain.

Hu, Lu, and Clune's concept of ADAS is fairly self-explanatory: control flow of agentic systems can be "automated," that is to say determined by an agent.

The novelty of their approach is that they are not only interested in prompt design for the model decision/generation components of the systems that they are building, but also in having the model select code components. For code generation: when to run tests, or static analysis, or which tools to provide to each model components.

Essentially, ADAS as proposed is programming a meta-agent to generate and then use agentic systems.

## ADAS as a case study for building agentic systems

Hu, Lu, and Clune's paper doesn't require new technologies. It could theoretically work with GPT-1. Nor does it represent a novel domain or complex algorithm.

ADAS is "just" metaprogramming over different system designs and picking one approach or another via a classifier backed by an LLM. The code is about structuring the problem space so that architectural decisions can be more easily identified and deferred to a model.

ADAS isn't magic. It doesn't remove the requirement for deterministic components. And given the fundamental indeterminacy constraint, humans remain in the loop for code that has to run correctly (note: there is a lot of code that doesn't need to run correctly, but that's another topic). Nor does it make sense for humans to immediately implement overly general solutions to solve specific, often time-sensitive problems.

But I find this paper valuable both due to the domain and as a case-study. It encourages me as an engineer to design for existing LLM constraints, encode key invariants so they are statically enforced and understood by my colleagues and our coding assistants, and defer minutiae to automated systems. I don't need to control how a problem is solved (and indeed shouldn't); I only need the means to validate the solution.