AI coding agents have become one of the biggest advancements in software development. Tools like Claude Code, OpenAI Codex, Cursor, and Pi are changing how developers write, debug, and maintain code.

Unlike traditional AI chatbots that simply answer questions, coding agents can explore your project, understand your codebase, modify files, execute terminal commands, and solve programming tasks with very little manual effort.

But before building an AI agent, it's important to understand the concepts that make these systems work.

In this article, we'll cover the foundation of AI agents, starting from Large Language Models (LLMs), how they generate responses, why tokens matter, and how system prompts influence their behavior.

By the end, you'll understand the building blocks used in almost every modern AI application.

Why Learn These Concepts?

Many developers jump directly into frameworks like LangChain or start building AI applications using APIs.

While that works, it often creates confusion when something doesn't behave as expected.

Understanding the basics helps you:

• Build your own AI agents from scratch.

• Debug AI applications more effectively.

• Write better prompts.

• Understand how tools like Claude Code or Codex work internally.

• Design AI-powered applications instead of only using existing ones.

Think of these concepts as the operating system behind every AI application.

What is a Large Language Model (LLM)?

A Large Language Model (LLM) is an AI model trained on massive amounts of data to understand and generate human language.

Instead of storing answers like a database, an LLM learns patterns from billions of examples.

Whenever you ask it a question, it predicts the most likely next piece of text based on everything it learned during training.

A Simple Example

Suppose you type:

The capital of India is

Most people instantly think of Delhi.

An LLM does something similar.

It doesn't search the internet in real time. Instead, it predicts that Delhi is the most likely next answer based on patterns learned during training.

LLMs Don't Memorize Information

A common misconception is that AI models work like search engines.

They don't.

Instead of remembering every sentence from the internet, they learn relationships between concepts.

For example, during training, the model learns patterns like these:

Concept	Relationship Learned
Delhi	Capital of India
React	JavaScript library
Docker	Container platform
PostgreSQL	Relational database

When you ask a question, the model predicts an answer using these learned relationships.

Where Does an LLM Get Its Knowledge?

Modern LLMs are trained using enormous datasets collected from publicly available sources.

These include:

• Books

• Documentation

• Research papers

• Technical blogs

• Programming tutorials

• Public GitHub repositories

• Forums

• Educational websites

The model studies this information during training to learn language, reasoning, and coding patterns.

It doesn't continuously learn from the internet after deployment unless it's retrained with newer data.

Why Is Data Becoming a Problem?

One interesting point discussed in the video is that AI companies have already consumed most of the publicly available internet data.

This creates a new challenge.

If every company is training on nearly the same public information, where will future training data come from?

To solve this problem, companies are now investing in:

Source	Purpose
Synthetic Data	AI-generated training examples
Human Labeling	Creating high-quality datasets manually
Licensed Content	Access to premium data
Private Codebases	Better programming knowledge

Some companies even pay developers for access to large, high-quality private codebases because they provide valuable training data.

What Makes Modern AI So Powerful?

Most modern language models are built using a neural network architecture called the Transformer.

Before Transformers, AI struggled to understand long conversations or complex documents.

The Transformer architecture introduced a concept called Attention, allowing models to understand how different words relate to each other.

For example:

"The developer fixed the bug because he understood the code."

The model learns that he refers to the developer.

This ability to understand context is one of the biggest reasons modern AI performs so well.

Note: You don't need to deeply understand Transformer mathematics before building AI applications. However, knowing that modern LLMs use the Transformer architecture gives you useful background knowledge.

What Are Tokens?

Humans read complete words.

LLMs don't.

Instead, they break text into smaller pieces called tokens.

A token can be:

• A complete word

• Part of a word

• A punctuation mark

• A number

• Even a space in some tokenizers

For example, the word:

Programming

might be divided into:

Program
ming

Similarly,

Artificial Intelligence

might become several individual tokens depending on the tokenizer used.

Why Are Tokens Important?

Every interaction with an AI model is measured in tokens.

Tokens determine:

• How much text the model can remember

• API usage costs

• Maximum conversation length

• Response size

This is why you'll often hear terms like:

• Context Window

• Input Tokens

• Output Tokens

These all refer to how many tokens the model is processing.

How AI Generates Responses

One of the most fascinating things about an LLM is that it never writes an entire paragraph at once.

Instead, it generates one token at a time.

For example, if you ask:

Explain Docker.

The model might internally generate something like:

Docker

↓

is

↓

a

↓

containerization

↓

platform

Each new token is predicted using all the previous tokens that have already been generated.

This process continues until the model generates a special end token, indicating that the response is complete.

Although this happens token by token, it's so fast that it feels like the AI is writing naturally.

What Is a System Prompt?

Whenever you use an AI application like ChatGPT or Claude, there's usually an invisible instruction sent before your message.

This instruction is called the System Prompt.

Its purpose is to tell the AI how it should behave.

For example, a system prompt might instruct the model to:

• Answer politely

• Keep responses short

• Write code only

• Avoid unsafe content

• Explain concepts for beginners

The user never sees this prompt, but it strongly influences the model's responses.

Example

System Prompt

You are an expert programming tutor.

Always explain concepts in simple language.

User Prompt

Explain React Hooks.

Because of the system prompt, the AI responds like a teacher rather than simply defining React Hooks.

System Prompt vs User Prompt

System Prompt	User Prompt
Defines how the AI should behave	Contains the user's actual request
Usually written by the application	Written by the user
Applied before every conversation	Changes with each query

Both are combined before being sent to the language model.

Chat Applications vs APIs

There are two common ways to interact with an LLM.

Using a Chat Application

Applications like ChatGPT or Claude provide:

• Predefined system prompts

• Safety checks

• Built-in conversation history

• Extra features like web search or memory

Everything is managed for you.

Using an API

When using an API, developers control everything.

They can decide:

• The system prompt

• Which model to use

• Conversation history

• Temperature

• Tool access

• Response format

This flexibility is what makes APIs ideal for building AI applications, assistants, and coding agents.

Streaming vs One-Shot Responses

When you ask ChatGPT a question, have you noticed that the response appears word by word instead of waiting until the entire answer is ready?

That's called streaming.

Instead of generating the complete response and sending it all at once, the AI sends small chunks as they are generated.

One-Shot Response

In a one-shot response:

1. You send a request.

2. The model generates the complete answer.

3. The server sends everything back at once.

The user has to wait until the response is fully generated.

Streaming Response

With streaming:

1. You send a request.

2. The model starts generating tokens.

3. Each token is immediately sent back.

4. The UI updates continuously.

This makes the application feel much faster, even though the total generation time is almost the same.

Why Streaming Improves User Experience

Imagine asking an AI to generate a 500-word explanation.

With a one-shot response, you might wait 10 seconds before seeing anything.

With streaming, you start reading within a second while the remaining content is still being generated.

This creates a much smoother and more interactive experience.

That's why almost every modern AI application uses streaming.

How Does Streaming Actually Work?

The AI model generates one token at a time.

Instead of collecting every token before sending the response, the backend forwards each token to the frontend as soon as it's available.

The flow looks like this:

1. User sends a request.

2. Backend forwards the request to the AI model.

3. The model generates tokens one by one.

4. The backend streams those tokens to the frontend.

5. The frontend displays them immediately.

The AI itself isn't aware that the response is being streamed.

Streaming is handled by the backend application.

Image

AI Image Prompt

Create a flow diagram showing User → Backend Server → LLM. The LLM generates multiple tokens, and each token is streamed back through the backend to the frontend in real time. Show small text bubbles appearing one after another to represent streaming. Minimal flat design with modern developer aesthetics.

How Does the Frontend Receive Streaming Data?

Since responses are continuously arriving, a normal HTTP request-response cycle isn't enough.

Instead, most AI providers use Server-Sent Events (SSE).

SSE allows the server to keep a connection open and continuously send updates until the response is complete.

This is why AI applications can display responses in real time.

Server-Sent Events (SSE)

Server-Sent Events are a communication protocol built on top of HTTP.

Instead of sending a single response, the server keeps sending multiple events over the same connection.

For AI applications, those events usually contain newly generated tokens.

Simple Flow

User
   ↓
Frontend
   ↓
Backend
   ↓
LLM

LLM → Token 1
LLM → Token 2
LLM → Token 3
LLM → Token 4

Backend streams every token back to the UI.

Why Not Use WebSockets?

Many developers assume WebSockets are required for AI streaming.

However, that's usually not the case.

The instructor explains that Server-Sent Events are a better fit because communication only needs to happen in one direction—from the server to the client.

SSE vs WebSockets

Server-Sent Events	WebSockets
One-way communication	Two-way communication
Built on HTTP	Separate protocol
Easier to scale	More complex
Perfect for AI responses	Better for chat apps, games, collaborative editing

Since the user isn't constantly sending data while the AI is responding, SSE is usually the simpler and more efficient solution.

What Happens Behind the Scenes?

When you click Send in ChatGPT or Claude, several things happen before the AI even starts generating a response.

The backend typically:

• Authenticates the user.

• Applies safety checks.

• Adds the system prompt.

• Sends the request to the LLM.

• Streams the generated tokens back to the UI.

This entire process happens within seconds.

What is an AI Agent?

Now that we understand how AI communicates, let's answer an important question.

What exactly is an AI agent?

An AI agent is more than just a chatbot.

It's a software system that receives a goal and uses available tools to complete that goal.

Unlike a traditional LLM that only generates text, an AI agent can interact with the outside world.

For example, it can:

• Read files

• Search code

• Execute terminal commands

• Call APIs

• Browse the web

• Update databases

• Write new files

The AI doesn't just answer questions—it performs actions.

Assistant vs AI Agent

These two terms are often used interchangeably, but they're not exactly the same.

AI Assistant

An assistant primarily focuses on conversation.

Examples include:

• ChatGPT

• Claude

• Gemini

Their main job is to answer questions and provide information.

AI Agent

An agent receives a goal and actively works toward completing it.

Instead of simply explaining how to fix a bug, it can:

• Locate the relevant file

• Edit the code

• Run tests

• Verify the fix

This ability to perform actions is what makes an agent different.

Comparison

AI Assistant	AI Agent
Answers questions	Completes tasks
Mostly conversational	Goal-oriented
Limited interaction with external systems	Uses tools and external services
Generates text	Generates text and performs actions

Is ChatGPT an AI Agent?

The answer depends on how it's being used.

If ChatGPT is simply answering your questions, it's acting like an assistant.

However, if it's connected to tools such as:

• Web Search

• File System

• Calendar

• Email

• Database

• APIs

then it starts behaving much more like an AI agent.

The distinction isn't always black and white.

It's more of a spectrum.

Coding Agents

Coding agents are a specialized type of AI agent.

Instead of helping with general tasks, they're designed specifically for software development.

They can:

• Read project files

• Understand repository structure

• Search code

• Modify existing code

• Create new files

• Run terminal commands

• Execute tests

• Fix bugs

This makes them significantly more capable than a standard chatbot.

Why Are Terminal-Based Coding Agents Becoming Popular?

The transcript highlights an interesting trend.

Many developers are moving away from traditional AI chat interfaces and using terminal-based coding agents instead.

The reason is simple.

Developers already spend most of their time inside the terminal.

Instead of copying code between a browser and an editor, they can interact with AI directly from their development environment.

This creates a much smoother workflow.

Why Do Simple Agents Sometimes Perform Better?

One surprising point discussed in the video is that more complexity doesn't always lead to better performance.

Some AI coding agents use multiple sub-agents, memory systems, and complex orchestration pipelines.

Others, like Pi, follow a much simpler approach.

Despite having a lightweight architecture, they can still perform extremely well because modern language models have become much better at reasoning and exploring codebases on their own.

This is an important lesson for developers.

Instead of immediately building a complex multi-agent system, start with a simple agent that uses a capable model effectively.

In the previous part, we learned the difference between an AI assistant and an AI agent. We also saw how AI applications stream responses and why modern coding agents are becoming increasingly popular.

However, one question still remains:

How can an AI model perform actions instead of just generating text?

For example:

• How does ChatGPT search the web?

• How does Claude check the weather?

• How does Claude Code edit files on your computer?

• How can an AI execute terminal commands?

The answer lies in Tools, also known as Function Calling.

This is one of the most important concepts you'll learn if you want to build AI agents.

Why LLMs Alone Are Limited

A Large Language Model only knows what it learned during training.

It cannot:

• Browse the internet

• Read your local files

• Execute shell commands

• Access databases

• Send emails

• Call APIs

• Know today's weather

• Know the latest stock prices

It simply predicts text based on its training data.

For example, if an LLM was trained six months ago, it doesn't know what happened yesterday unless someone provides that information.

A Simple Example

Imagine asking:

"What's the weather in Delhi right now?"

Without internet access, the model can only guess.

It might respond with something like:

"Delhi is generally warm during this season."

Notice something?

That's not the current weather.

It's simply making an educated prediction.

Why Do AI Applications Give Better Answers?

When you ask ChatGPT or Claude the same question, they often return the live weather.

How?

Not because the model suddenly learned today's weather.

Instead, the application gives the model access to a Weather Tool.

The model requests the latest weather data, receives it, and then uses that information to generate its response.

What Are Tools?

A Tool is simply a function that allows an AI model to interact with something outside its own knowledge.

Think of tools as bridges between the AI and the real world.

Examples include:

• Web Search

• Weather API

• Calculator

• Database

• Terminal

• File System

• GitHub

• Gmail

• Calendar

Without tools, an LLM can only generate text.

With tools, it can perform real-world tasks.

Image

AI Image Prompt

Create a modern educational illustration showing an LLM in the center connected to multiple tools. Around the LLM place icons for Web Search, Terminal, Database, Weather API, File System, Email, GitHub, and Calendar. Show arrows between the LLM and each tool. Use a flat vector style with blue and purple gradients, clean typography, white background, and a modern developer documentation aesthetic.

How Tools Change an AI Agent

Without tools, an AI behaves like this:

User
   ↓
LLM
   ↓
Text Response

With tools, the flow becomes:

User
   ↓
LLM
   ↓
Tool Request
   ↓
External Service
   ↓
Result
   ↓
LLM
   ↓
Final Answer

This extra step is what transforms a chatbot into an intelligent assistant.

Real-World Examples of Tools

User Request	Tool Used
What's the weather today?	Weather API
Search the latest AI news	Web Search
Read this project	File System
Fix this bug	Terminal + File System
Send an email	Gmail API
Check GitHub issue	GitHub API

Every modern AI application relies on these kinds of tools.

How Does an LLM Know It Should Use a Tool?

This is where things get interesting.

An LLM cannot directly call an API.

It cannot execute JavaScript.

It cannot make an HTTP request.

It only generates text.

So how does it ask for a tool?

The answer is surprisingly simple.

The model has been trained to generate a special structured response whenever it decides a tool is needed.

Instead of answering the user directly, it first says something equivalent to:

"Please execute this function."

The application receives that request, runs the tool, and sends the result back to the model.

Understanding the Tool Calling Process

Imagine you ask:

"What's the weather in Delhi?"

Here's what happens behind the scenes.

Step 1

The user sends the question.

What's the weather in Delhi?

Step 2

The LLM realizes it doesn't know the current weather.

Instead of guessing, it asks for the Weather Tool.

Something similar to:

Call Weather Tool

City: Delhi

Step 3

Your backend executes the Weather API.

The API responds:

31°C

Sunny

Step 4

That result is added back into the conversation.

Now the model knows the answer.

Step 5

Finally, the model generates a natural response.

For example:

"The current weather in Delhi is 31°C with sunny conditions."

The user never sees the tool call.

They only see the final answer.

Image

AI Image Prompt

Design a flowchart showing Tool Calling. Steps: User asks "What's the weather in Delhi?" → LLM requests Weather Tool → Backend calls Weather API → Weather API returns 31°C → LLM generates final answer. Use colorful arrows, modern flat vector style, white background, minimal developer documentation look.

Why Doesn't the LLM Call the API Directly?

Because the model isn't an application.

It's simply a prediction engine.

It doesn't know:

• How to send HTTP requests

• How to authenticate APIs

• Where your Weather API lives

• Which API key to use

Those responsibilities belong to your backend.

The LLM simply decides what it needs.

Your application decides how to get it.

The Backend Plays an Important Role

Your backend acts like the coordinator.

Its responsibilities include:

• Sending prompts to the model

• Executing requested tools

• Managing API keys

• Handling authentication

• Validating inputs

• Returning tool results

Without the backend, the LLM cannot interact with external systems.

How Does the LLM Know Which Tools Exist?

Whenever your application sends a request to the model, it also includes information about the available tools.

For example:

Tool	Description
Web Search	Search the internet
Weather	Get live weather information
Calculator	Perform calculations
Read File	Read project files
Run Command	Execute terminal commands

The model reads these descriptions before answering.

If it believes one of them can help solve the user's request, it asks to use that tool.

Can an LLM Use Multiple Tools?

Yes.

Suppose a user asks:

"Find today's AI news and summarize it."

The agent may perform several steps:

1. Use Web Search.

2. Collect relevant articles.

3. Read their content.

4. Summarize everything.

The user sees only the final answer, but multiple tool calls happened in the background.

Coding Agents Use Even More Tools

A coding agent typically has access to tools like:

Tool	Purpose
Read File	Understand code
Write File	Modify source code
Terminal	Execute commands
Search Files	Find relevant code
Git	Check repository status
Tests	Validate changes

These tools allow the AI to work almost like a developer.

Why Tool Calling Is So Important

Without tool calling, an AI assistant can only explain how to solve a problem.

With tool calling, it can actually solve it.

For example:

Instead of saying:

"Run npm install."

The agent can actually execute:

npm install

Instead of explaining how to fix a file, it can edit the file itself.

That's the biggest difference between a chatbot and a coding agent.

What is an Agent Loop?

An Agent Loop is a cycle where the AI repeatedly:

1. Understands the current situation.

2. Decides what to do next.

3. Uses a tool.

4. Observes the result.

5. Decides the next action.

6. Repeats until the task is complete.

Instead of producing an answer immediately, the AI keeps thinking and acting until it reaches its goal.

This is what makes an AI agent feel intelligent.

Image

AI Image Prompt

Create a modern infographic illustrating the AI Agent Loop. Show a circular flow with six steps: Goal → Think → Select Tool → Execute Tool → Observe Result → Repeat Until Complete. Use blue and purple gradients, clean vector icons, white background, minimal documentation style, arrows connecting each step, 16:9.

Understanding the Agent Loop with an Example

Suppose you ask:

"Fix the login bug in my project."

A coding agent might internally perform these steps.

Step 1: Understand the Task

The agent first reads your request.

It understands that the goal is to fix a login issue.

No code has been modified yet.

Step 2: Explore the Project

Next, it tries to understand the project.

It may:

• Read the README file.

• Check the project structure.

• Identify authentication-related files.

At this stage, the agent is gathering context.

Step 3: Read Relevant Files

After finding possible files, it starts reading them.

For example:

• auth.ts

• login.ts

• middleware.ts

The agent doesn't randomly read every file.

Instead, it explores the project just like a developer would.

Step 4: Decide the Fix

Once it has enough information, the LLM reasons about the problem.

It identifies:

• The likely cause.

• The required changes.

• The files that need modification.

Step 5: Edit the Code

Now the Write File tool is used.

The agent updates the necessary files.

Step 6: Verify the Fix

A good coding agent doesn't stop after editing code.

It often runs:

• Tests

• Build commands

• Linters

If something fails, it continues the loop.

If everything succeeds, the task is complete.

The Entire Flow

User Goal
      ↓
Understand Request
      ↓
Explore Project
      ↓
Read Files
      ↓
Reason About Solution
      ↓
Edit Code
      ↓
Run Tests
      ↓
Task Complete

Notice that several tool calls happened before the final response.

This entire workflow is the Agent Loop.

Why Doesn't the AI Solve Everything in One Step?

Because real-world software development is iterative.

When developers fix a bug, they rarely know the solution immediately.

Instead, they:

• Read documentation.

• Search files.

• Understand existing code.

• Make changes.

• Test those changes.

AI coding agents follow a very similar process.

Thinking Like a Developer

One interesting point from the transcript is that modern language models have become very good at exploring codebases independently.

Instead of requiring complicated workflows, they often perform surprisingly well by simply repeating this process:

1. Read a file.

2. Learn something.

3. Decide what to read next.

4. Repeat.

This is similar to how an experienced developer approaches an unfamiliar project.

Why Pi Performs Surprisingly Well

One of the main discussions in the video is about Pi, an open-source terminal coding agent.

At first glance, Pi looks extremely simple.

It has:

• A small codebase.

• Minimal architecture.

• Very little abstraction.

Yet it performs remarkably well.

Why?

Because today's LLMs are already excellent at exploring codebases and reasoning about code.

The orchestration layer has become less important than it used to be.

Complex Agents vs Simple Agents

The instructor compares two approaches.

Complex Agent

A complex coding agent might:

• Spawn multiple sub-agents.

• Maintain long-term memory.

• Create task planners.

• Generate summaries.

• Coordinate several workflows simultaneously.

This architecture is powerful but also much more complicated.

Simple Agent

A simple agent usually follows one loop:

1. Think.

2. Use a tool.

3. Observe.

4. Repeat.

Despite its simplicity, it often performs just as well because modern LLMs are capable of handling much of the reasoning themselves.

Comparison

Simple Agent	Complex Agent
Easier to build	Difficult architecture
Easy to debug	More moving parts
Lower maintenance	Higher maintenance
Works well with strong LLMs	Useful for advanced workflows

The key takeaway is that simplicity often wins.

Multi-Agent Systems

Some coding agents divide work among multiple specialized agents.

For example:

Agent	Responsibility
Planner	Understand the task
Explorer	Search the codebase
Coder	Modify files
Reviewer	Verify changes
Tester	Run tests

Each agent has its own responsibility.

They collaborate to solve a larger problem.

This approach can improve performance for very large or complex tasks, but it also increases implementation complexity.

Do You Always Need Multiple Agents?

Not necessarily.

The transcript emphasizes that many modern coding agents perform extremely well without complicated multi-agent orchestration.

Today's models are strong enough to handle many tasks using a single reasoning loop.

If you're building your first AI agent, start with one agent before introducing multiple specialized agents.

How Coding Agents Explore a Codebase

When you ask a coding agent to fix a bug, it doesn't magically know where the problem is.

Instead, it explores the project step by step.

For example:

Read README
      ↓
Understand Project
      ↓
Search Authentication Files
      ↓
Read auth.ts
      ↓
Read middleware.ts
      ↓
Identify Bug
      ↓
Fix Code

This process closely resembles how human developers work.

Image

AI Image Prompt

Create a software architecture diagram showing how an AI coding agent explores a project. The flow should be: User Task → Read README → Search Files → Read Relevant Files → Understand Context → Modify Code → Run Tests → Success. Use folder icons, file icons, terminal icons, clean arrows, flat vector illustration, white background, developer documentation style.

Why Understanding the Agent Loop Matters

If you're building an AI-powered application, the Agent Loop is one of the first architectural decisions you'll make.

Questions you'll need to answer include:

• When should the loop stop?

• Which tools should the AI have access to?

• How many iterations are allowed?

• Should the AI verify its own work?

• Should it ask for user confirmation before making changes?

These decisions define how reliable and useful your agent will be.

Applying This in Your Own Projects

Suppose you're building an AI code review tool.

Your agent loop might look like this:

Step	Action
1	Read Pull Request
2	Read Modified Files
3	Analyze Changes
4	Generate Suggestions
5	Return Feedback

Or imagine building a documentation assistant.

The loop could be:

Step	Action
1	Read Documentation
2	Search Related Files
3	Collect Context
4	Generate Documentation
5	Save Markdown File

The Agent Loop stays the same.

Only the tools and objectives change.

Why Supporting Multiple AI Providers Is Difficult

At first glance, switching between AI providers seems simple.

You might think changing the API URL is enough.

In reality, every provider has its own way of handling:

• Requests

• Responses

• Streaming

• Tool calls

• Authentication

• Error handling

This means every integration requires additional work.

Different Providers, Different APIs

Some popular AI providers include:

• OpenAI

• Anthropic

• Google Gemini

• DeepSeek

• Grok

• OpenRouter

Even though they all expose language models, their APIs are structured differently.

For example:

Feature	Can Differ Between Providers
Request Body	Yes
Streaming Format	Yes
Tool Calling Format	Yes
Authentication	Yes
Response Structure	Yes

Because of these differences, developers can't simply replace one provider with another without modifying their code.

Why This Becomes a Problem

Imagine you're building an AI coding assistant.

Today, you're using OpenAI.

Tomorrow, you decide to switch to Anthropic.

If your application directly depends on OpenAI's response format, you'll need to rewrite large parts of your code.

Now imagine supporting five different providers at the same time.

The complexity grows quickly.

The Solution: Provider Abstraction

Instead of writing provider-specific logic throughout the application, coding agents introduce a provider abstraction layer.

This layer acts as a translator.

It converts different provider APIs into one common interface.

Your application only communicates with that interface.

The abstraction layer handles everything else.

Image

AI Image Prompt

Create a software architecture diagram. Show multiple AI providers (OpenAI, Anthropic, Gemini, DeepSeek) connected to a central "Provider Abstraction Layer". From there, connect to an AI Coding Agent. Use modern flat vector design, white background, clean arrows, blue and purple developer theme, 16:9 aspect ratio.

Benefits of a Provider Abstraction Layer

Using an abstraction layer offers several advantages.

• Switching providers becomes easier.

• The rest of the application remains unchanged.

• Adding new providers requires less effort.

• Maintenance becomes simpler.

Instead of writing custom logic for every provider, developers only need to update the abstraction layer.

Why Pi Uses This Approach

The instructor explains that Pi supports multiple AI providers.

Each provider behaves differently.

Instead of exposing those differences throughout the codebase, Pi converts every provider into a common format.

This allows the rest of the application to remain provider-independent.

As a result, developers can switch models without redesigning the entire agent.

Tool Calling Isn't Standardized

Earlier, we learned how tool calling works.

Unfortunately, there isn't a universal standard for how providers implement it.

For example:

• One provider may return tool calls in one JSON format.

• Another provider may use a completely different structure.

• Some providers even use different streaming formats.

This means the abstraction layer has to normalize all these differences before the agent can use them consistently.

Why This Matters

Imagine your coding agent expects tool calls like this:

{
  "tool": "read_file",
  "arguments": {
    "path": "auth.ts"
  }
}

Now imagine another provider returns:

{
  "function": "read_file",
  "params": {
    "file": "auth.ts"
  }
}

Both represent the same action.

However, your application must understand both formats.

That's exactly what the provider abstraction layer handles.

Streaming Responses Also Differ

Streaming isn't implemented identically by every provider.

Some stream plain text.

Others stream structured JSON events.

Some include additional metadata alongside generated tokens.

Without a common abstraction, your frontend would need custom handling for every provider.

A well-designed provider layer converts all of them into one consistent stream.

Supporting New Models

Suppose tomorrow a brand-new AI company releases an impressive language model.

If your application already has a provider abstraction layer, adding support usually involves:

1. Creating a new provider implementation.

2. Mapping requests.

3. Mapping responses.

4. Registering the provider.

The rest of the application remains unchanged.

That's one of the biggest advantages of good software architecture.

Skills vs Tools

Another concept discussed in the video is Skills.

Although they sound similar, Skills and Tools solve different problems.

Tools

Tools perform actions.

Examples include:

• Reading files

• Executing terminal commands

• Searching the web

• Calling APIs

They interact with external systems.

Skills

Skills don't perform actions.

Instead, they provide additional knowledge or instructions to the AI.

Think of them as reusable documentation.

For example, a company might create a deployment skill explaining:

• How deployments work

• Infrastructure requirements

• Naming conventions

• Internal workflows

Instead of repeating those instructions in every prompt, the agent loads the relevant Skill whenever it's needed.

Tools vs Skills

Tools	Skills
Perform actions	Provide knowledge
Execute APIs	Provide context
Interact with systems	Guide the model
Dynamic execution	Static documentation

Both improve an AI agent, but they serve different purposes.

Real Example

Suppose your company has a custom deployment process.

Instead of writing a huge prompt every time, you create a Skill describing:

• Deployment steps

• Kubernetes configuration

• Environment variables

• Domain setup

• Load balancer configuration

Whenever the AI needs to deploy an application, it loads this Skill before starting.

This gives the model project-specific knowledge without changing the underlying LLM.

Why Skills Are Useful

Skills help reduce prompt size while improving consistency.

Instead of embedding hundreds of lines into every request, the agent loads only the information relevant to the current task.

This approach keeps prompts smaller and makes responses more reliable.

Understanding AI Coding Agents from Scratch (Part 6)

Introduction

Throughout this guide, we've learned how AI coding agents understand tasks, use tools, interact with files, and continuously improve their responses using the Agent Loop.

Now let's look at two final topics discussed in the video:

• How AI coding agents are evaluated.

• Why simple agent architectures often outperform complex multi-agent systems.

We'll also wrap everything together by understanding how you can apply these concepts when building your own AI-powered applications.

How Do We Know if an AI Coding Agent is Good?

When a new language model or coding agent is released, one obvious question arises:

How do we measure its performance?

The answer is through benchmarks.

A benchmark is a collection of standardized tasks used to compare different AI models and coding agents under the same conditions.

Instead of relying on opinions, benchmarks provide measurable results.

Why Benchmarks Matter

Imagine comparing two developers.

Instead of asking who is better, you give both developers the same set of programming tasks.

The one who solves more problems correctly performs better.

Benchmarks work in exactly the same way.

They provide:

• Fixed tasks

• Common evaluation criteria

• Comparable scores

This allows researchers and developers to compare different AI systems fairly.

Common AI Benchmarks

The video mentions several well-known benchmarks.

Benchmark	Purpose
SWE Bench	Evaluates software engineering tasks
Terminal Bench	Evaluates terminal-based coding agents

Each benchmark focuses on different abilities.

For coding agents, Terminal Bench has become particularly important because it measures how well an agent can solve real software development problems.

What is Terminal Bench?

Terminal Bench evaluates coding agents that operate inside a terminal.

Instead of asking simple programming questions, it measures whether an AI can complete realistic development tasks.

These tasks may include:

• Exploring a project

• Reading source code

• Editing files

• Running commands

• Fixing bugs

• Completing coding workflows

This makes it much closer to real-world software development.

Why Pi Performs Surprisingly Well

One interesting observation from the video is that Pi, despite being a relatively simple coding agent, performs exceptionally well on Terminal Bench.

This surprises many developers because they expect more complex systems to perform better.

However, that's not always true.

Why Simplicity Often Wins

Earlier generations of AI models required complex orchestration.

Developers built systems with:

• Memory modules

• Multiple planners

• Sub-agents

• Reflection loops

• Long reasoning pipelines

These additions compensated for weaker language models.

Modern LLMs are much stronger.

They can often reason through complex tasks without requiring heavy orchestration.

As a result, a lightweight architecture paired with a capable model can perform remarkably well.

Complex Doesn't Always Mean Better

It's tempting to believe that adding more components automatically improves an AI system.

In practice, every additional component introduces:

• More complexity

• More maintenance

• More debugging

• More failure points

A simpler system is often:

• Easier to understand

• Easier to extend

• Easier to maintain

• More reliable

This is one of the biggest lessons from the transcript.

Image

AI Image Prompt

Create an infographic comparing two AI coding agent architectures. On the left, show a complex architecture with Planner, Memory, Multiple Agents, Reflection, and Orchestrator connected together. On the right, show a simple architecture with User → LLM → Tools → Agent Loop. Highlight that the simpler architecture performs efficiently with modern LLMs. Use a clean developer documentation style with blue and purple accents.

Where Do Frameworks Like LangChain Fit In?

The instructor also shares an important industry perspective.

Frameworks such as LangChain and LangGraph are useful because they simplify development.

However, many companies build their own orchestration layers instead of depending entirely on these frameworks.

Why?

Because every product has unique requirements.

Custom implementations provide:

• Better control

• Higher flexibility

• Easier optimization

• Lower dependency on external libraries

This doesn't mean frameworks are bad.

They're excellent for learning and rapid development.

But understanding the underlying concepts is much more valuable than memorizing framework APIs.

Focus on Fundamentals

One of the strongest messages throughout the video is this:

Learn how agents work, not just how to use an agent framework.

If you understand:

• LLMs

• Tokens

• Prompts

• Streaming

• Tool Calling

• Agent Loops

then learning any framework becomes much easier.

The reverse is not always true.

Someone who only knows a framework may struggle when debugging or designing custom AI systems.

Building Your Own AI Coding Agent

By now, you have enough conceptual knowledge to build a basic coding agent.

A simplified architecture might look like this:

User
   ↓
Frontend
   ↓
Backend
   ↓
System Prompt
   ↓
LLM
   ↓
Agent Loop
   ↓
Tools
   ↓
File System / Terminal / APIs
   ↓
LLM
   ↓
Streaming Response

Every concept we've covered fits somewhere in this workflow.

Image

AI Image Prompt

Create a complete architecture diagram for an AI coding agent. Show the flow: User → Frontend → Backend → System Prompt → LLM → Agent Loop → Tools (File System, Terminal, Web Search, APIs) → Tool Results → LLM → Streaming Response → User. Use a modern software architecture style, clean arrows, white background, flat vector illustrations, and blue-purple gradients.

How Can You Apply These Concepts?

These ideas aren't limited to coding assistants.

The same architecture can power many AI applications.

Project	Tools You Might Use
Customer Support Bot	CRM, Email, Knowledge Base
AI Research Assistant	Web Search, PDFs, Databases
AI Code Reviewer	GitHub, File System, Terminal
Documentation Generator	Source Code, Markdown Files
DevOps Assistant	Kubernetes, Docker, Shell Commands
Personal Productivity Agent	Calendar, Email, Notes

The only thing that changes is the set of tools available to the agent.

The overall architecture remains largely the same.

Complete Learning Journey

Let's quickly recap everything we've covered throughout this guide.

Topic	What You Learned
Large Language Models	How AI predicts text
Tokens	How models process language
System Prompts	How AI behavior is controlled
APIs	How applications communicate with models
Streaming	Why responses appear gradually
Server-Sent Events	How streaming is implemented
AI Agents	Goal-oriented AI systems
Tools	How AI interacts with external systems
Tool Calling	How LLMs request actions
Agent Loop	Continuous reasoning and execution
Coding Agents	AI systems designed for software development
Provider Abstraction	Supporting multiple AI providers
Skills	Injecting project-specific knowledge
Benchmarks	Measuring coding agent performance

These concepts form the foundation of nearly every modern AI agent.