Fullstack Development

Real Latency vs Perceived Latency in GenAI Systems

Raw latency and perceived latency are different engineering problems. Production GenAI systems feel fast when they expose progress early, overlap backend work, and avoid silent waiting.

Cyril Noirot

May 13, 2026

Share on

8 min read

Every production GenAI system eventually runs into the same uncomfortable question:

The answer is that raw latency and perceived latency are different engineering problems.

Raw latency is backend execution time. It includes retrieval, reranking, orchestration, model inference, queueing, serialization, and network overhead.

The two are related, but they are not the same metric. A silent 1.5-second wait can feel worse than a streamed 2.5-second answer because the first interface creates uncertainty and the second creates progress.

About the author

Cyril Noirot

Lead Data Scientist

Freelance data scientist. I design and ship decision systems — forecasting, pricing, marketing measurement, optimization.

Why Operational AI Systems Break When Granularity Scales

Enterprise AI systems often work at the first level of granularity, then become fragile when the business asks for more precision. A field lesson from pharmaceutical supply optimization on why incremental architectures matter.

Fullstack Development12 min

Why LLM-Only Parsing Breaks in Production — And What to Do Instead

LLMs can extract structured data from anything — until they cannot. This article documents the failure modes of LLM-only parsing in production pipelines, and presents a layered architecture where determinism comes first and the LLM is used only where it is structurally irreplaceable.

Fullstack Development6 min

Stop Talking About Agentic Commerce. Start With Agentic RAG.

Most companies are not ready for autonomous commerce agents. The practical starting point is Agentic RAG: systems that retrieve business context, reason over it, and produce decision-ready outputs.

Newsletter

Technical writing on forecasting, pricing, and decision systems. No fixed schedule, no spam.

Enter your email