What Are AI Agents and How Do They Work?

"Agent" is one of the most overused words in AI. Some people mean a chatbot with a fancy name, and others mean software that runs a business process on its own. This article gives you a precise definition, explains how agents work, and shows when they are worth the extra risk. It is one of the building blocks described in our AI engineering overview.

A precise definition

Anthropic's engineering guide, Building effective agents, draws a useful line between two kinds of systems:

• Workflows are "systems where LLMs and tools are orchestrated through predefined code paths."
• Agents are "systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks."

The difference is who decides the next step. In a workflow, your code does: first summarise, then classify, then route. In an agent, the model does: it looks at the task, picks a tool, looks at the result and chooses what to do next.

The building blocks

The same guide describes the foundation as an "augmented LLM", a model with three added capabilities.

• Retrieval: searching for information. See RAG explained.
• Tools: choosing and calling actions, such as an API or a database query.
• Memory: retaining information across steps or conversations.

Microsoft's AI-103 study guide covers the same ideas from the developer side. It lists defining "agent roles, goals, conversation-tracking approach, and tool schemas", building agents that "integrate retrieval, function-calling, and conversation memory", and implementing orchestrated multi-agent solutions.

How an agent works, step by step

1. A goal arrives, such as "find this customer's order and draft a ticket about it."
2. The model plans a next step and chooses a tool, described to it by a name, parameters and what the tool does.
3. Your code runs the tool and returns the result to the model.
4. The model looks at the result and decides: is the goal met, or is another step needed?
5. The loop repeats until the model finishes or a limit is reached.

The important design choices are in the tools. A good tool has a clear name, precise parameters and an honest description of its side effects. Separating tools that only read from tools that change something is one of the most valuable habits you can build.

When to use an agent

Anthropic's advice is restrained: start with simple prompts, optimise them with comprehensive evaluation, and add multi-step agentic systems only when simpler solutions fall short. It notes that agents involve higher latency, increased costs and the potential for compounding errors, so they suit problems where it is difficult or impossible to predict the number of steps needed.

In practice that means:

• Use a single prompt when one model call can do the job.
• Use a workflow when the steps are known and fixed.
• Use an agent when the path genuinely depends on what the model finds along the way.

Many things that are called agents would work better as workflows, which are cheaper, faster and easier to test.

The risks to design around

Giving a model the ability to act creates risks that a plain chatbot does not have. The OWASP Top 10 for LLM Applications (2025) includes several that apply directly:

• Prompt Injection (LLM01). Text in the model's input, including text hidden in a document or a web page, tries to steer it into doing something else.
• Excessive Agency (LLM06). The agent has more access or freedom than the task needs.
• Improper Output Handling (LLM05). Code trusts what the model produced without checking it.
• Unbounded Consumption (LLM10). Loops or retries run up costs.

The AI-103 outline reflects these concerns, listing "safeguards and approval flow controls" for autonomous workflows and governing agent behaviour with "oversight modes, constraints, and tool-access controls".

Design habits that help

• Give the least access that works. A read-only tool cannot cause damage.
• Require approval for anything that changes the world, especially spending money, deleting data or contacting people.
• Set limits on steps, time and cost.
• Log every tool call so you can see what happened and why.
• Treat everything the model reads as untrusted, because it may contain instructions from someone else.
• Test with evaluations, not by trying a few examples by hand. See AI guardrails and safety.

Learn agents by building one

Reading about agents only goes so far. The concepts click when you build a small one, give it a read tool and a write tool, watch it make a mistake, and add an approval step. The AI-103 guide shows where agents sit in a full certification syllabus, and learning by doing explains why we teach this way. In Code Nexus, agents appear in the AI seasons as hands-on tasks with real consequences. Create a free account to start with the fundamentals.