[Understanding Agency] A Universal Framework for Understanding AI Agents and Autonomy
This framework will provide the conceptual foundation for AgentsDecoded. It will help us navigate the AI Agent space and provide a common set of terms to refer back to.
Introduction
A clear and simple framework to understand agency and autonomy will help navigate the space and bring order and clarity over the wide range of ideas and approaches. In particular it will:
Help us understand where the agent paradigm is relevant and where other software or application development paradigms are suitable. This can lead to better architecture choices and makes it simpler to identify hype and marketing speak.
Help evaluate and compare different approaches to agent development. This can guide in the choice of a specific framework or set of tools to best fit a domain or problem type.
Help capture patterns of agent systems and apply them to a variety of different domains. This can help advance the field and lead to more robust systems.
In the next few paragraphs we will provide definitions for agency, autonomy and a framework that can help in the analysis of anything claiming to be an AI Agent, irrespective of the technologies that were used to construct it.
What is an agent?
Anything that serves a goal for someone (including itself) can be described as an agent.
This was the view that attracted me to the theoretical framework of Luck and d'Inverno1 and shaped my understanding of AI agents (as well as providing the starting point for my PhD thesis and subsequent work2) . It was a view of agents that focussed on the end result: the what (a goal), not the how (the technologies that enable the agent to achieve the goal).
Technologies are, of course, important but cannot be the first port of call. Technologies, tools and methods change. We need to be able to compare and contrast agents irrespective of the underlying technology that makes them possible. In addition, definitions that depend on enumerating behavioural characteristics such as proactivity, reactivity and social ability become easier to define3 when based on some clear foundational concepts.
Luck and d'Inverno's view, centred on goals, enables us to consider anything from an agent lens with clarity. So much so, that the first example Luck and d'Inverno gave is that of a teacup acting as an agent. "How can a tea cup possibly be an agent? It is an inanimate object", I thought to myself.
If the teacup is sitting on its own on the table not serving any purpose there is no value in considering as an agent. However, a teacup holding my tea and keeping it warm and in a single place such that I can use it to drink my tea is an agent for me. It fulfils my goal to keep my tea is warm and in one place. It can be ascribed agency.
What is a goal?
A goal is a desirable state of affairs as described by a set of attributes (describable features) of the world.
Our tea cup is an agent with the goal of containing tea. Granted, it's a very simple agent. So simple it doesn't even know about its goal. If the tea spills it has no way of picking it up and putting it back in. However, the concept provides us with a a starting point. It grounds our thinking and enables us to focus on the what, not the how.
Any object that has some features and capabilities (e.g. it is solid, round and can hold liquids) and can be ascribed a goal by someone can be considered an agent.
By focussing on the what and not the how we free ourselves from the endless debates about what is the right combination of the right technologies. Agency is not about how something is built but about what something is able to do for someone (or for itself). Arguably, if something can achieve the goal with the least amount of complexity possible then it should be celebrated, not accused of not displaying sufficient AI capabilities!
In general, we can think of an agent as something that serves a goal and has certain capabilities (for simplicity we will split them in sensors and actuators) that can somehow influence the environment to achieve that goal.
If we think of the current development-focussed view of agency where we may be using LLMs + tools to perform actions we can map that functionality to the diagram above. Consider the typical scenario of a weather agent. The goal is provided by the user through asking for the current weather. The agent will use a tool (a sensor) to read information from a weather service. It may then use an actuator to format that response and show it back to the user - achieving the desirable state of displaying the weather to the user. Just like the teacup, this is not a very interesting agent. Of course it uses an LLM to understand the request but it anyway can only execute one plan - which is to call the available tool to retrieve information and display it. Nevertheless, it is more sophisticated than a teacup and moves our understanding of agency forward.
With agency defined as goal-directed behaviour we can now move on to talk about different types of agents, from a simple tea cup or weather agent to an all-knowing singularity that could run the world!
Types of agency
Agency can be passive or active, and on a continuum from self-directed to autonomous
Passive agents
A teacup is what we will call a passive agent. It has no understanding of the goal it is serving and no capability to effect change in its environment independently. Agency, in this case, is entirely in the eye of the beholder. The agent is fundamental in achieving the goal, but all the reasoning is done by the entity (the tea drinker in this case) manipulating the agent.
A passive agent has no internal representation of a goal and no internal reasoning capabilities to track progress towards a goal.
Now, you may think this is too much of an abstract concept, almost philosophical, and certainly not useful to every day AI agent development. Remember, the goal is to ground our thinking and build up. A passive agent is our grounding. It also describes most software in existence today!
Most software has some capabilities but no specific goals of its own. It has no way of effecting change until someone comes along and manipulates it in some way. The user of the agent is doing most of the planning and reasoning but the agent remains crucial in achieving the goal.
Active agents
If we have passive agents, we can also have active agents.
An active agent is one that has some internal representation of a goal and can move from goal definition to a series of actions that will help it achieve that goal, potentially making some of its own choices about how to achieve the goal.
While a passive agent is ascribed a goal by its user an active agent is assigned a goal. The distinction is crucial. A spreadsheet may hold all our financial data and can provide a forecast of sales but it has no internal understanding of what it is doing. The goal of sales projection is ascribed to it by its user.
An agent that is able to access discrete transaction data and then use an understanding of how sales forecasts are put together to produce a forecast has been assigned a goal and may make some of its own choices about how to process the data in order to achieve that goal
The issue of "making its own choices" brings us to the next set of concepts. Self-direction and autonomy.
Self-direction and Autonomy
So far we talked about an agent being ascribed a goal by the user (the tea cup) or being assigned a goal (the sales forecast agent). Now, we turn our focus to how an active agent achieves the goal in the latter case.
Here we will draw a distinction between self-direction and autonomy.
Self-direction describes an agent's ability to make choices about how to accomplish a goal. It cannot, however, select what goal to accomplish.
Take our sales forecast agent from above. We provide it with sales data and ask it to make a prediction about what sales may look like in the future. The goal, that desirable state of affairs, is clear and indisputable. We need to go from a world where we do not have sales projections to one where we have them. The agent cannot choose to not achieve that goal or to produce a forecast of the environmental cost of doing those sales instead. The only space where it may have some choice is how it will get there.
Now, an agent that has access to a single function called "sales forecast" that accepts sales data and provides a forecast does not have a great deal of self-direction. On the self-direction continuum it sits right at zero. It has only one choice and it will execute that choice.
Most AI Agents out there today are exactly this. They will accept a request from a user and make use of a single tool to generate a result and display it back to the user. Even if they use multiple tools, if those tools are simply wired in a single path the agent still has no great choice. There is absolutely nothing wrong with that. As long as the agent is providing some clear value to the user and can achieve that simple I will call it a win. However, from a design perspective it is important that we are able to distinguish between different degrees of self-direction.
The more self-direction we require of an agent, the more AI techniques we will need to employ to achieve it. Consider a notification agent for your phone. The goal is to display a notification when an event happens. If we are to draw a continuum of self-direction for this agent it may go something like this:
For each one of those choices we may be deploying simple rules or sophisticated AI models. Each layer of choice or decision-making gives our agent more self-direction. What it does not do, however, is change the end goal which is to deliver notifications to a user.
Autonomy describes an agent's ability to generate its own goals.
Where self-directed agents can make choices about how to achieve goals, autonomous agents can generate their own goals. This is the difference between a Sales Forecast agent that can crunch data to provide a sales forecast and a CFO agent that is directed to optimise the output of the sales team and decides to generate a Sales Forecast as a way to better understand how the team is performing.
There is something intrinsically different between an agent that is simply trying to achieve a goal (described as a desirable explicit state in the world) and an agent that is trying to satisfy more vague optimisation requests (such as "improve the performance of the sales team").
To mechanism that can lead from simple fixes goals to more interesting optimisation requests (that can lead to the generation of goals) we will term a motivation.
Motivations are desires or preferences that drive the agent to set its own agenda, as opposed to having specific goals assigned.
Autonomy requires some form of motivation, while for self-direction a goal can suffice4.
Reactivity, proactivity and social ability
Before we draw a line let's discuss the concepts of reactivity, proactivity and social ability. At the start we mentioned that these concepts are hard to define in the absence of some foundational grounding. With the grounding now in place we can revisit them.
Passive agents do not have any inherent reactivity, proactivity or social ability.
Self-directed and autonomous agents are reactive if they are able to adjust their actions based on external stimuli.
In the pursuit of their goals, if they are able to sense their environment and make different choices because of the changes in the environment they can be deemed reactive.
Self-directed and autonomous agents are proactive when they are pursuing goals over time, potentially adjusting planning and approach as required. This can be expressed in a couple of ways. An agent can be proactive in the sense that it is always on and it is always trying to maintain a certain state in the environment, such as a smart temperature control system constantly monitoring a building and taking decisions so that the temperature is at a certain level. An agent can also be proactive in that it can attempt to achieve a goal in multiple ways, not just giving up after the first attempt.
Finally, self-directed and autonomous agents are social when they are able to interact with other agents and humans while working towards their goals. It is important to include humans here because humans can very much play the role of an additional agent. Imagine the scenario where a user requests that a self-directed agent submit an insurance claim on their behalf. That agent identifies that they need to some additional information and decides that the best way to get that information is to ask that same human that assigned the original goal. That human in now serving a goal for the agent that the original goal was assigned to! If the human does not provide the information, that goal cannot be achieved.
Bringing it all together
While there is more nuance and we can layer concepts in a few different ways the framework that we described so far gives us a useful tool through which to compare and contrast different AI Agents and different AI Agent frameworks, tools and methodologies. We can focus on the end result they are trying to bring about, what types of agents with what types of characteristics they can support and discuss how they go about achieving it. These tables provides a summary of everything we discussed.
Passive Agent
Goals ascribed to them
Agency depends on a viewing agent (a user)
No internal reactivity, proactivity or social ability
Active (Self-Directed) Agent
Goals assigned to them
Reactive in adjusting actions to changes in the environment
Proactive in pursuing goals over time or executing multiple attempts to achieve goal
Socially able in contacting other agents in order to achieve its goals
Autonomous Agent
Exhibits all the characteristics of self-directed agents
Goals generated by agent based on motivations (preferences, utility functions, etc)
Reactive in adjusting motivations to changes in the environment
Proactive in influencing the environment over the long-term to optimise how it satisfies its motivations
Socially able to contact other agents to accomplish its goals or satisfy its motivations
In reviewing a lot of the AI Agent work out there we will see that we are mostly still either dealing with passive agents and different variations of active agents. We can then compare and contrast the degree of self-direction of agents and their level of reactivity, proactivity and social ability. Some agents may also be autonomous and it will be interesting to discovering what real levels of autonomy are out there today.
Mark d’Inverno, Michael Luck (2004). The SMART Agent Framework. In: Understanding Agent Systems. Springer Series on Agent Technology. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-10702-7_2
Ronald Ashri, Michael Luck, Mark d’Inverno (2005). From SMART to agent systems development, Engineering Applications of Artificial Intelligence, Volume 18, Issue 2, 2005, ISSN 0952-1976,
Michael Wooldridge, Nicholas R. Jennings (1995). Intelligent Agents: Theory and Practice, The Knowledge Engineering Review, Vol 10:2,
It is worth saying that these concepts can be stacked. I may have a fixed top-level goal (so exhibit no autonomy there) but can use preferences and utility functions to generate sub-goals. While it tempting to call that autonomy, for the purposes of this analysis we remain in the real of self-direction.