Lessons from running systems in production — war stories and hard-won patterns.
OpenAI and Anthropic both document outages, overload codes, and best-effort availability. The survival kit: LLM-aware circuit breakers, same-model-first fallback chains, and degraded modes designed before you need them.
One agent, one context window, one serial loop — until it stalls at 40 minutes and 180K tokens. The orchestrator pattern fans work out to isolated sub-agents in parallel, then synthesizes. Here's the backend, in compiling Go.
Go 1.26 makes the Green Tea garbage collector the default. What actually changed, how to measure GC CPU before and after on your own workload, and the short window you have to opt out.
2,500 API endpoints in one MCP server without blowing context windows. The Code Mode pattern uses search + execute to cut token cost by 1,000x.
AI agents calling tools via MCP create new attack surfaces: prompt injection through tool responses, credential leakage, and unauthorized execution.
Handle unpredictable JSON in Go: map[string]any, json.RawMessage, type switches, and defensive patterns for shifting schemas.
Go graceful shutdown: SIGTERM handling, health probe coordination, and Kubernetes drain patterns for zero dropped requests.
Stop fighting java.util.Date. Master LocalDateTime, ZonedDateTime, Instant, and Duration — predictable, thread-safe time handling.
Production LLM API patterns: streaming, function calling, retries, token budgets, cost optimization, and observability for backend engineers.
Build RAG systems that work in production: chunking strategies, embedding selection, pgvector ops, and retrieval quality evaluation.