Fullstack Development

Real Latency vs Perceived Latency in GenAI Systems

Raw latency and perceived latency are different engineering problems. Production GenAI systems feel fast when they expose progress early, overlap backend work, and avoid silent waiting.

Cyril Noirot

May 13, 2026

Share on

8 min read

The answer is that raw latency and perceived latency are different engineering problems.

Raw latency is backend execution time. It includes retrieval, reranking, orchestration, model inference, queueing, serialization, and network overhead.

A silent 1.5-second wait can feel worse than a streamed 2.5-second answer because the first interface creates uncertainty and the second creates progress.

This distinction matters because often we optimize the wrong number, focusing only on total execution time, when the user is often reacting to something more specific:

About the author

Cyril Noirot

Lead Data Scientist

Freelance data scientist. I design and ship decision systems — forecasting, pricing, marketing measurement, optimization.

The Hidden Cost of Precision in Operational AI

Enterprise AI systems often work at the first level of granularity, then become fragile when the business asks for more precision. A field lesson from pharmaceutical supply optimization on why incremental architectures matter.

Fullstack Development12 min

Why LLM-Only Parsing Breaks in Production — And What to Do Instead

LLMs can extract structured data from anything — until they cannot. This article documents the failure modes of LLM-only parsing in production pipelines, and presents a layered architecture where determinism comes first and the LLM is used only where it is structurally irreplaceable.

Fullstack Development6 min

Stop Talking About Agentic Commerce. Start With Agentic RAG.

Most companies are not ready for autonomous commerce agents. The practical starting point is Agentic RAG: systems that retrieve business context, reason over it, and produce decision-ready outputs.

Newsletter

Technical writing on forecasting, pricing, and decision systems. No fixed schedule, no spam.

Enter your email