01 What agentic AI actually is
Start here since this is new territory. Everything later in this module assumes these fundamentals.
The one-sentence definition
A regular LLM call is a single request-response: you send a prompt, it sends back text, done. An agent is an LLM wired into a loop where it can decide to take actions (call a tool, search, run code, ask a follow-up), look at the result, and decide what to do next — repeating until it thinks the task is done. The model isn't just generating text anymore, it's making decisions about what to do.
Plain LLM call
Like asking a very well-read colleague a question over email. One message in, one message back. They can't go check anything.
Agent
Like giving that colleague a laptop, database access, and telling them "figure out why the metric dropped and report back." They'll look things up, form a hypothesis, check it, revise, and only respond once they've actually done the work.
The core loop (this is the whole idea — everything else is detail)
Click through the steps. This think → act → observe cycle, repeated, is what "agentic" means. It's called the ReAct pattern (Reason + Act) in the literature.
1. Thought
2. Action
3. Observation
4. Repeat or answer
Thought
The model reasons in text about what it knows and what it needs next. Example, for "why did churn spike this week": "I should first check if this is one segment or all users." This step is just the model thinking out loud — no side effects yet.
Is this new? What's actually different from a normal script
The loop itself (do something, check result, decide next step) is exactly what a while loop or a state machine does in regular software. The genuinely new part is that the decision of what to do next is made by the model, in natural language, at runtime — not hard-coded by a developer in advance. That's the entire delta. It's powerful because it handles tasks whose steps you couldn't fully enumerate ahead of time; it's risky for exactly the same reason — you've handed runtime control flow to something nondeterministic.
What agentic AI is not
A chatbot that answers questions is not an agent — no actions, no loop. A RAG pipeline (retrieve docs, stuff into prompt, generate) is not an agent either by the strict definition — it's a fixed two-step pipeline with no decision-making about whether to retrieve again or do something else. It becomes agentic the moment the model can decide, on its own, to search again, try a different tool, or stop. Keeping this distinction sharp matters in interviews — people who use "agent" for any LLM-touching product tend to get caught out on a follow-up question.
Where you'll actually see this used
Coding agents
Reads a codebase, writes code, runs tests, reads the failure, fixes it, reruns — a real loop where each step depends on the last.
Customer support triage
Looks up the customer's account, order history, and past tickets across systems before deciding how to respond or who to escalate to.
Data / metrics investigation
"Why did this KPI move" — queries different tables, forms a hypothesis, checks it, backtracks if wrong. Directly relevant to your marketing DS world.
Ops / SRE agents
Reads logs and metrics dashboards, correlates an incident to a recent deploy, proposes or takes a remediation action.
Research / deep-research assistants
Searches the web across many rounds, reads sources, decides what's still unknown, searches again, then synthesizes.
Personal task assistants
Books things, drafts and sends emails, checks calendars — multi-step tasks across tools with real-world side effects.
Vocabulary you need before the rest of this module makes sense
LLM
The underlying model (GPT, Claude, etc.) that does the actual language reasoning at every step of the loop.
Context window
Everything the model can "see" for one call — system prompt, conversation history, tool results so far. It's finite; this limit is why memory management matters.
System prompt
The standing instructions that shape the agent's behavior and persona, set once, present on every call.
Tool / function calling
A model's ability to output a structured request ("call `search(query)`") instead of plain text, which your code then actually executes and feeds the result back in.
Orchestrator
The code (or another LLM call) that decides which agent or tool runs next — the traffic controller of the loop.
Memory
What persists across steps or sessions — short-term (this conversation) vs. long-term (facts saved for future runs, usually in a database or vector store).
RAG (retrieval-augmented generation)
Fetching relevant documents and inserting them into the prompt before generating — often a component used inside an agent, not the same thing as an agent.
Embeddings / vector database
A way to turn text into numeric vectors so "find documents similar in meaning" becomes a fast nearest-neighbor search — the retrieval mechanism behind most RAG and long-term memory.
Multi-agent system
Several agents, each with a narrower role, coordinated by an orchestrator — covered in depth in the next section.
Connecting the dots: memory vs. context window vs. vector DB
The fastest way in is a computer-architecture analogy, since it maps directly onto systems intuition you already have.
Context window
RAM. Everything the model sees for one call — system prompt, conversation so far, tool results. Finite, fast, must fit or it doesn't exist to the model right now.
Memory (concept)
Not a mechanism — the general idea of "information that needs to survive beyond what fits in the context window." Short-term = accumulated conversation (really just context window contents that haven't fallen off yet). Long-term = facts that must persist across sessions or exceed what context could ever hold.
Vector DB
Disk. One specific way to implement long-term memory (and RAG) — external, larger, slower, requires an explicit "load" step (similarity search) before anything reaches the model. Never read by the model directly; it feeds the context window on demand, one relevant slice at a time.
One trace, start to finish: a session starts with just the system prompt and task in the context window. The agent loops — thought, action, observation — and each step gets appended, so the context window keeps growing turn by turn (this growth is what "short-term memory" actually is). Eventually one of two pressures hits: the context window nears its token limit, or the task needs a fact that lives outside this session entirely. That's when you reach for long-term memory — summarize and drop older raw turns to control size, and/or query a vector DB to pull back only the handful of relevant facts, spliced into the context window just for this turn. The vector DB holds everything; the context window only ever holds the small slice that's relevant right now. The memory-bloat failure mode from section 04 is exactly what happens when someone conflates these three and just keeps appending everything to the context window instead of offloading to a vector DB or summarizing.
Worked example: answering a question from a book too large for the context window
This is the canonical RAG pattern, and it's the same vector DB mechanism as long-term memory — just applied to a static document instead of an agent's own history.
1. Chunk
Split the PDF into overlapping passages (a paragraph or a page or two each) — small enough that a handful of them comfortably fit in a context window together.
2. Embed
Run every chunk through an embedding model, turning each into a vector that represents its meaning. This happens once, up front, not per question.
3. Store
Save each chunk's vector plus its original text in the vector DB. The whole book now lives on "disk" — none of it is in any context window yet.
4. Query
When a real question comes in, embed the question the same way, and ask the vector DB for the nearest-neighbor chunks — the handful of passages most semantically similar to the question.
5. Assemble
Splice just those few retrieved chunks (not the whole book) into the context window alongside the question, then generate the answer.
The model never "reads the whole book" for any single question — it only ever sees the few passages the vector DB judged most relevant, which is exactly why this scales to books, entire document corpora, or codebases far larger than any context window could hold. The trade-off to know cold for interviews: retrieval quality is only as good as the chunking and embedding choices — if the answer actually depends on information spread across distant, unrelated chunks (e.g. a plot detail from chapter 2 needed to explain chapter 40), naive top-k similarity search can miss it, since it retrieves by semantic similarity to the question, not by narrative or logical connection. That's a known limitation, and part of why more advanced RAG setups add a query-rewriting or multi-hop retrieval step — the agent notices the first retrieval was insufficient and searches again with a reformulated query, which is the ReAct loop from section 01 applied specifically to retrieval.
The one thing to internalize before moving on: agentic AI is not a new model or a new algorithm. It's an architecture pattern — a loop plus tools plus memory wrapped around a model that already exists. Everything that follows in this module (failure modes, evaluation, architecture) is really just "software engineering for a component that reasons in natural language and can be subtly wrong." That framing is exactly the angle that plays to your strengths.
02 Anatomy of a multi-agent system
Click each component. The point isn't the box — it's what fails when it's built naively.
Orchestrator / planner
decides who acts next
Worker agents
specialized reasoning loops
Tool layer
function calls, APIs, code exec
Memory
short + long term state
Evaluation / guardrails
checks output before acting
Observability
trace, replay, cost tracking
Orchestrator / planner. Routes tasks to agents — either a fixed graph (LangGraph state machine) or an LLM-decided router (supervisor pattern). Naive version: an LLM call that free-forms "which agent should go next" with no bound on hops. That's how you get infinite loops. A systems engineer's instinct here is right: treat this like a state machine with a max-depth / max-iteration circuit breaker, same as you'd cap retries on a queue consumer.
Worker agents. Each is a reasoning loop (usually ReAct-style: think → act → observe → repeat) scoped to a narrow responsibility. The failure most people miss: giving one agent too many tools/responsibilities degrades reasoning quality — the model's effective "attention budget" gets diluted, same intuition as god-classes in software design.
Tool layer. Function calling / MCP servers / code execution. This is where your backend background matters most: tools need idempotency keys, timeouts, and typed schemas the model can't get subtly wrong. Untyped tool outputs are the single biggest source of silent agent failure.
Memory. Short-term = conversation window (bounded, decays). Long-term = vector store / structured facts (unbounded, needs eviction). Naive systems just keep appending to context — this is the agent equivalent of an unbounded cache with no TTL. It degrades both cost and reasoning quality (lost-in-the-middle effect).
Evaluation / guardrails. Structural validation (does the tool call match schema), semantic validation (does the output satisfy intent), and safety checks — ideally before an action executes, not just after. This is the agent-world equivalent of input validation and a canary check before rollout.
Observability. Full trace of every LLM call, tool call, and intermediate reasoning step, replayable. Without this, debugging "why did the agent do that" is pure archaeology. Maps directly to distributed tracing (OpenTelemetry) — same problem, LLM nondeterminism just makes it worse.
The judgment call that separates you: most candidates can draw this diagram. Few can say which of these six components is over-engineered for a given use case, and which is the one that will actually break in production. That's the question to rehearse answering out loud.
03 When multi-agent, when single-agent, when no agent at all
The most senior answer to "should this be agentic" is usually "no." Walk the tree.
Is the task's structure known and repeatable in advance?
Yes, fixed steps
No, needs dynamic reasoning
Don't build an agent. A deterministic pipeline (plain code, maybe one LLM call for the fuzzy step) is cheaper, faster, and fully debuggable. Agentic overhead buys you nothing if the control flow doesn't need to vary. This is the single most common interview trap — people reach for agents when a DAG would do.
Keep going. Dynamic reasoning over unknown-shape tasks is where agents earn their keep — e.g. "investigate why this metric moved," where the number of steps and which tools get used isn't knowable upfront.
Do distinct sub-tasks need genuinely different context, tools, or expertise?
Yes, clearly separable
No, one coherent context
Multi-agent may be justified — e.g. a research agent, a coding agent, a critique agent with genuinely different tool access and system prompts. The separation should mirror a real division of responsibility, not be invented for architecture-diagram appeal.
Single agent with more tools beats multi-agent with more coordination overhead. Every extra agent adds a communication/handoff failure surface. Default to the simplest thing that works — this is the same instinct as not microservice-ing a monolith prematurely.