Hey everyone, Alex here. Welcome back to another edition of Coding with Alex.
If you’ve spent any time on tech Twitter, GitHub trending, or Hacker News lately, you’ve probably noticed that the phrase "AI Agent" has crossed the threshold from exciting buzzword to absolute exhaustion. We are constantly bombarded with promises of autonomous agents that will write our code, manage our cloud infrastructure, and handle customer support while we drink margaritas on a beach. But as software engineers, we don’t buy into magic. We buy into architecture, state machines, API calls, and data flows.
When you strip away the marketing gloss, what does an AI Agent actually look like from the inside? How do we transition from a basic LLM prompt-and-response loop to a deterministic, reliable system that can autonomously execute complex tasks? Today, we are going inside the black box. We’ll deconstruct the concrete architectural patterns, code, and state management strategies required to build a production-grade AI agent.
The Fallacy of the "Single Prompt" Agent
When most developers start playing with LLMs, they write something like this:
const response = await openai.chat.completions.create({
model: "gpt-4o",
messages: [{ role: "user", content: "Analyze this database schema and write a migration script." }]
});
This is a simple request-response cycle. It’s stateless, ephemeral, and entirely dependent on the model getting everything right in a single pass. If the model makes a syntax error, hallucinated a table name, or ran out of output tokens, the process fails.
An AI Agent is different. An agent is an architectural pattern where the LLM is placed inside a loop—often called the ReAct (Reasoning and Acting) loop. The agent evaluates an input, decides on an action, executes that action via external tools (APIs, databases, file systems), observes the outcome, and repeats the process until a goal is met. To make this work reliably, we need a robust software wrapper around the LLM.
The Core Blueprint of an Agentic System
To build an agent, we need to orchestrate four core architectural pillars:
- The Brain (The LLM + System Prompt): Responsible for planning, decision-making, and parsing tool outputs.
- Memory (State Management): Short-term memory (the conversation history and intermediate step logs) and long-term memory (vector databases for semantic retrieval).
- Tools (The Action Engine): Strictly typed functions that the LLM can decide to execute (e.g., executing a SQL query, hitting a REST API, reading a local file).
- The Execution Loop (The Controller): The deterministic code (usually written in Python or TypeScript) that coordinates the state, calls the LLM, parses its intent, runs the chosen tools, and feeds the results back into the LLM.
Let's look at how these components interact in a standard execution loop:
+-----------------------------------------------------------------------+
| THE EXECUTION LOOP |
| |
| +------------------+ System Prompt +--------------------+ |
| | | + User Input | | |
| | | --------------------> | | |
| | | | | |
| | | <-------------------- | | |
| | | Tool Call | | |
| | State/Memory | (e.g., run_sql) | LLM (Brain) | |
| | | | | |
| | | Tool Result | | |
| | | <------------------- | | |
| | | | | |
| | | --------------------> | | |
| +------------------+ Final Answer +--------------------+ |
| ^ |
+------------|----------------------------------------------------------+
| Run Tool (Sandbox)
v
+---------------+
| External APIs |
+---------------+
Under the Hood: Building an Agent from Scratch
Let’s write some concrete TypeScript code to build a light, transparent agent runner. We won't use heavy frameworks like LangChain or AutoGen here; building it from scratch is the best way to understand the underlying mechanics.
We want our agent to have access to two tools: a tool to read system logs, and a tool to execute a mock shell command. First, let’s define the interface for our tools:
interface Tool {
name: string;
description: string;
parameters: object; // JSON Schema representing the tool's inputs
execute: (args: any) => Promise<string>;
}
Now, let's implement our specific tools. Note that tools must return raw data (usually serialized as JSON or plain string) that the agent can read in its next planning step.
const logReaderTool: Tool = {
name: "read_system_logs",
description: "Reads the recent error logs from the server application.",
parameters: {
type: "object",
properties: {
lines: { type: "number", description: "Number of trailing lines to read" }
},
required: ["lines"]
},
execute: async ({ lines }) => {
// In production, this would read from CloudWatch, Datadog, or a local log file
console.log(`[Tool Execute] Reading last ${lines} lines of system logs...`);
return JSON.stringify([
{ timestamp: "2023-11-04T12:00:01Z", level: "ERROR", message: "Database connection timed out." },
{ timestamp: "2023-11-04T12:00:05Z", level: "INFO", message: "Attempting reconnect..." },
{ timestamp: "2023-11-04T12:00:10Z", level: "ERROR", message: "Database connection failed. Port 5432 unreachable." }
]);
}
};
const portCheckerTool: Tool = {
name: "check_port_status",
description: "Checks if a local port is open and listening.",
parameters: {
type: "object",
properties: {
port: { type: "number", description: "The port number to check" }
},
required: ["port"]
},
execute: async ({ port }) => {
console.log(`[Tool Execute] Checking status of port ${port}...`);
if (port === 5432) {
return JSON.stringify({ port, status: "CLOSED", process: "none" });
}
return JSON.stringify({ port, status: "LISTEN", process: "node" });
}
};
const toolsRegistry: Record<string, Tool> = {
read_system_logs: logReaderTool,
check_port_status: portCheckerTool
};
The Orchestrator Loop
Now, let's look at the actual orchestrator loop. The orchestrator must handle state management (storing the chat history) and handle the "tool call" callbacks generated by the LLM. In this example, we’ll use OpenAI's Function Calling API, which is the industry standard for mapping natural language intent to structured schema calls.
import OpenAI from 'openai';
const openai = new OpenAI();
async function runAgent(userGoal: string) {
// Initialize state with system prompt and the user's explicit goal
const messages: any[] = [
{
role: "system",
content: `You are an autonomous systems engineering agent.
Your goal is to diagnose server errors using the tools provided.
You must iterate: analyze logs, test environments, and find the root cause.
When you have identified the root cause, present it clearly to the user.`
},
{ role: "user", content: userGoal }
];
let iteration = 0;
const maxIterations = 5;
while (iteration < maxIterations) {
console.log(`\n--- Iteration ${iteration + 1} ---`);
// Convert our Tools registry to OpenAI format
const openAiTools = Object.values(toolsRegistry).map(t => ({
type: "function" as const,
function: {
name: t.name,
description: t.description,
parameters: t.parameters
}
}));
// Step 1: Query the Brain
const response = await openai.chat.completions.create({
model: "gpt-4o",
messages: messages,
tools: openAiTools,
tool_choice: "auto"
});
const responseMessage = response.choices[0].message;
messages.push(responseMessage); // Save the LLM's raw response to state
// If the LLM didn't request a tool, it means it has formulated the final answer
if (!responseMessage.tool_calls) {
console.log("[Agent Finished] Final Answer from LLM:");
console.log(responseMessage.content);
return;
}
// Step 2: Handle tool execution requests
for (const toolCall of responseMessage.tool_calls) {
const toolName = toolCall.function.name;
const toolArgs = JSON.parse(toolCall.function.arguments);
console.log(`[Agent Decision] Wants to run tool: "${toolName}" with args:`, toolArgs);
const requestedTool = toolsRegistry[toolName];
if (!requestedTool) {
throw new Error(`Agent attempted to call registered tool "${toolName}" but it does not exist.`);
}
// Execute tool (observing the result)
const toolOutput = await requestedTool.execute(toolArgs);
console.log(`[Tool Result] Data returned: ${toolOutput}`);
// Feed the result back into the message history state
messages.push({
role: "tool",
tool_call_id: toolCall.id,
name: toolName,
content: toolOutput
});
}
iteration++;
}
console.log("[Agent Timeout] Failed to find solution within execution limit.");
}
// Kick off our agent to diagnose why the system is down!
runAgent("The application is down. Figure out why it's failing and tell me.");
The Engineering Hard Parts: What Happens in Production?
The code above is simple enough to understand, but translating this to a production-grade enterprise system reveals a host of deep engineering challenges. When your agent runs inside an actual developer platform, you have to architect defenses against some major failure modes.
1. Sandboxing and Security
If you give an AI Agent the ability to execute terminal commands, run Python scripts, or query raw databases, you have effectively created an intentional Remote Code Execution (RCE) vulnerability. A prompt injection attack could trick the agent into running rm -rf / or exfiltrating environment variables containing API keys.
The Solution: The runtime environment must be heavily sandboxed. Never let an agent run tools on your primary host or inside a shared cluster. Use micro-VMs like AWS Firecracker or isolated Docker containers with gVisor runtimes. These instances should have restricted egress networking, short lifespans, and strict resource limits (CPU/Memory limits) to prevent runaway processes or crypto-mining scripts.
2. The Cost and Latency Death Spiral
If you notice in the execution loop above, every single tool execution appends data to the message history and sends the entire history back to the LLM. In an agent run that takes 15 iterations, the prompt context size expands quadratically. This has two bad side effects: your LLM API bill skyrockets, and your latency crawls to a halt as the prompt processing time increases.
The Solution: Implement a state-pruning worker. You can't let your chat context grow infinitely. You must summarize older tool executions, prune highly repetitive logs, or transition historical context into a RAG (Retrieval-Augmented Generation) system, only retrieving the relevant tool results when requested by the model.
3. Deterministic Fallbacks for Infinite Loops
LLMs are probabilistic. It is highly common for an agent to get stuck in a loop: it reads a log, encounters an error, runs the port checker tool, gets a closed port response, gets confused, reads the log again, runs the port checker again... and runs up a $50 bill in ten minutes.
The Solution: Do not rely solely on the LLM to manage loop termination. Implement deterministic circuit breakers in your orchestrator code. Keep track of the actions taken by the agent. If the exact same tool with the exact same arguments is called three times sequentially without modifying state, trip the breaker and hand off the task to a human developer.
Conclusion: The Era of the Platform Engineer-Agent Hybrid
As software engineers, we shouldn't view AI agents as magical black boxes destined to replace us. Instead, we should view them as highly complex state machines that we must architect, write code for, sandbox, and optimize. The true value of AI agents lies in how well we write their wrapper systems: the validation pipelines, the tool execution sandboxes, and the deterministic guards that keep them on track.
Building agents forces us to become better platform engineers. We have to design clean APIs, strict schemas, and isolated operating environments so that our automated workflows can run safely and reliably.
What are your thoughts on agentic workflows? Have you built custom ReAct loops in your team’s pipelines, or are you holding off until the ecosystem matures? Let me know in the comments below!
Until next time, happy coding!