Generative AI

LLM Systems: Architecture First, Framework Second

LangChain, LangGraph, Claude ADK, Google ADK, PydanticAI — different names, different abstractions, same promise. This article cuts through the framework noise and identifies the five architectural properties that actually determine whether an LLM system survives production.

Cyril Noirot
By Cyril NoirotApril 1, 20266 min read
Share on

If you have spent even a few hours in the GenAI ecosystem recently, you have probably hit the same wall.

LangChain. LangGraph. Claude ADK. Google ADK. PydanticAI.

Different names. Different abstractions. Same promise:

build production-ready AI systems.

And yet — confusion is the dominant outcome.

This article cuts through the noise.

You think you are choosing a framework

You are not.

You are trying to solve a much more fundamental problem:

How do I integrate a probabilistic system (an LLM) into a deterministic product?

Every framework you encounter is just a different answer to that question. The sooner you internalize this, the sooner the landscape stops being overwhelming and starts being legible.

The real stack — without the marketing

All LLM systems, regardless of tooling, collapse into four layers:

1. Model layer OpenAI, Claude, Gemini. This is a function: input → output. Nothing more.

2. Orchestration layer How calls are chained, routed, retried. This is where LangChain, LangGraph, and the ADKs sit.

3. Reliability layer Validation, typing, guardrails. This is where Pydantic and PydanticAI become critical.

4. Application layer Your business logic. APIs, pipelines, decision systems — everything that actually matters to the business.

The frameworks compete on layer 2. Most production failures happen at layer 3. And the value lives entirely in layer 4.

Understanding this hierarchy changes how you evaluate tools.

Why LangChain feels powerful — and breaks in production

LangChain became popular because it abstracts everything: prompts, tools, memory, agents. You can go from zero to a working demo in an afternoon.

But that is exactly the problem.

Too much abstraction removes control. In production systems, you need to trace decisions, not just log responses. You need deterministic fallbacks when the LLM fails. You need predictable execution flows that can be tested, debugged, and explained.

LangChain hides too much of that logic behind convenience wrappers.

It is a good sandbox. It is a weak foundation.

The shift: from "agents" to "systems"

The industry is quietly moving away from the idea that you should let the agent figure it out — toward designing explicit systems where the LLM is one constrained component among many.

This is the context in which LangGraph emerged.

LangGraph forces you to think in states, transitions, and explicit flows. Instead of the implicit:

prompt → magic → result

You define:

state → node → decision → next node

This matters because you can debug it. You can test it. You can explain to a stakeholder exactly why the system produced a given output.

This is not about building agents. This is about building execution infrastructure.

The most underrated layer: reliability

This is where most teams fail.

Raw LLM output is inconsistent, brittle, and sometimes wrong in structure — not just in content. If your downstream system receives:

{"price": "twenty euros"}

instead of:

{"price": 20.0}

You do not have a system. You have a liability.

This is where Pydantic — and by extension PydanticAI — becomes non-negotiable. The key idea is simple but load-bearing:

LLM outputs must be treated as typed objects, not text.

You define a schema:

class Product(BaseModel):
    name: str
    price: float

And you enforce it. Every call. Without exception.

This single design choice is what separates demos from production systems. It is not glamorous. It is essential.

Claude ADK and Google ADK: the ecosystem play

Both Anthropic and Google are doing the same thing: providing orchestration tools tightly integrated with their models, pushing toward full-stack ownership.

These are not just frameworks. They are strategic ecosystems.

They make sense if you are committed to one provider and value deep integration over flexibility. But they introduce a constraint that is worth naming explicitly:

Your architecture becomes tied to a vendor.

For some teams, that is an acceptable trade-off. For others — particularly those operating in regulated environments or managing multi-model strategies — it is a structural risk.

What actually matters (and what does not)

What does not matter:

  • Which framework you pick
  • Whether you use LangChain or an ADK
  • Which abstraction looks cleaner in a tutorial

These are surface-level decisions. They feel important because they are visible. But they are not where production systems succeed or fail.

What actually matters:

  1. Control — Can you explain exactly why your system produced a given output?
  2. Determinism — Do you have fallback logic when the LLM fails or returns garbage?
  3. Observability — Can you trace decisions, not just responses?
  4. Validation — Are outputs enforced against a schema, or just trusted?
  5. Replaceability — Can you swap the model without rewriting your system?

If you can answer yes to all five, your framework choice is a detail. If you cannot, no framework will save you.

The winning architecture pattern

The systems that survive production follow one principle:

The LLM is a constrained function inside a deterministic pipeline.

Not an autonomous agent running the show.

A clean architecture looks like this:

API / Backend
    ↓
Orchestration (explicit flow control)
    ↓
LLM calls (isolated, typed, retriable)
    ↓
Schema validation (strict, enforced)
    ↓
Business logic (deterministic)

Every layer has a clear responsibility.

Practical recommendation

If you strip everything down to what holds in production:

  • Use graph-based orchestration (LangGraph or equivalent) for control flow
  • Use Pydantic for output validation — every time, no exceptions
  • Work within your ecosystem constraints, but avoid coupling core logic to a single model interface.
  • Avoid over-abstracted agent frameworks that trade debuggability for convenience

And most importantly:

Design your system first. Add the LLM second.

The system should work — at least as a skeleton — before any LLM is involved. If removing the LLM collapses everything, you have not built a system. You have built a dependency.

Final thought

The GenAI ecosystem is still in its framework inflation phase. New tools will keep appearing. APIs will change. Abstractions will evolve.

But the core constraint will remain:

You are integrating uncertainty into systems that require reliability.

About the author

Cyril Noirot

Cyril Noirot

Lead Data Scientist

Freelance data scientist. I design and ship decision systems — forecasting, pricing, marketing measurement, optimization.

Newsletter

Technical writing on forecasting, pricing, and decision systems. No fixed schedule, no spam.

Enter your email
Subscribe