Practical field notes on the parts of AI systems that have to survive production: agents, evaluation, security, memory, retrieval, model engineering, and infrastructure. Recursive Harness Engineering follows one path: build the harness, evaluate its behavior independently, then automate the search for improvements. Start with one of the series below, or browse every article by topic.
Series · Parts: 3
Harness engineering first, then recursive improvement.
Swipe through the parts →
Series · Parts: 6
In order. Start at the loop.
Swipe through the parts →
Evaluation
Evaluate agent behavior and every stage of a retrieval-augmented generation system.
Retrieval & language
Build ranking pipelines, choose document AI systems, and structure language reliably.
Fast Answers
Choosing a model, tool, or approach? Get a short recommendation, the conditions that change it, and a link to the full explanation.
AI Agents
The same support tasks as an agent, an agent with approval, an agent in a workflow, a router and a workflow with no agent, compared on one test set.
· 23 min
AI Agents
How task requirements shape an agent harness: choosing control flow, tools, code execution, state, permissions and evidence before choosing a framework.
· 29 min
AI Agents
A prologue to recursive harness engineering: test an LLM's sequential changes to a memory tool, then check whether simple search finds the same result.
· 20 min
Developer Tooling · AI Agents
Map pydantic to Zod, FastAPI to Hono, and Celery to a Postgres queue, then build a streaming agent service in a pnpm TypeScript monorepo.
· 25 min
AI Agents
How to accept a tested artifact, recover uncertain tool calls, hand work to another session, and measure whether a harness component helps.
· 20 min
Model Engineering · AI Infrastructure
A practical guide to choosing LLM and diffusion-model quantization paths by bottleneck, hardware, runtime kernel, calibration data, and deploy checks.
· 33 min
AI Agents
Why production AI agents need schema-guided memory with temporal validity, provenance, and structured conflict handling instead of raw text vector recall.
· 16 min
AI Agents
How to evaluate production AI agents by turning traces into versioned regression datasets, trajectory metrics, calibrated judges, and CI checks.
· 27 min
AI Agents
Keep an agent run alive across worker failures, reconcile uncertain tool effects, and choose a runtime: queue and worker, Temporal, sandboxes, or hosted agents.
· 30 min
Retrieval and Language Systems
RAG evaluation metrics for parsing, retrieval, reranking, generation, citations, and production telemetry, with code and failure modes.
· 44 min
AI Agents
How to authorize an agent's proposed action, contain its executor, and test the boundaries that model guardrails cannot enforce.
· 26 min
Retrieval and Language Systems
NER in 2026 means choosing between GLiNER, spaCy, Transformers, and LLM extraction for latency, accuracy, and schema control.
· 41 min
AI Agents
Design an agent's action and result contract, then choose JSON calls, MCP, CLI, or code execution for the work it needs to do.
· 23 min
Model Engineering · AI Infrastructure
A practitioner's reference to 45 LLM engineering concepts for production systems, spanning inference, training, architecture, deployment, and operations.
· 73 min
Retrieval and Language Systems
OCR in 2026 means choosing between classical pipelines and VLMs for text, layout, tables, and document extraction.
· 22 min
AI Infrastructure
How Polars, DataFusion, Daft, Ray Data, and Spark run tabular and multimodal workloads, with code, a like-for-like benchmark, and decision rules.
· 19 min
AI Agents
Choose agent memory by the failure it must prevent, then pick the tool: checkpoint savers, key-value stores, pgvector or Qdrant, files, or a memory framework.
· 26 min
Retrieval and Language Systems
How to build a search ranking stack with BM25, dense embeddings, hybrid RRF, cross-encoder reranking, and LLM listwise reranking on Amazon ESCI.
· 28 min
AI Agents
Choose between fixed workflows, ReAct, ReWOO, and Plan-and-Execute by asking when tool results must change the next action.
· 15 min
Model Engineering
A technical walkthrough of DeepSeek's Manifold-Constrained Hyper-Connections (mHC), residual stream width scaling, Sinkhorn routing, and training stability.
· 13 min
AI Agents
What public Enterprise RAG Challenge 3 submissions reveal about prompt iteration, validation, context strategy, tool wrappers, and agent architecture.
· 13 min
Model Engineering
When to fine-tune LLMs, when to use RAG or prompting, and how LoRA, QLoRA, DoRA, Unsloth, Axolotl, datasets, evals, and deployment fit.
· 29 min
AI Agents · Model Engineering
How Schema-Guided Reasoning uses vLLM, XGrammar, Pydantic schemas, and constrained decoding to enforce structurally valid LLM outputs.
· 16 min
Model Engineering · AI Infrastructure
Evaluate and deploy LoRAX for long-tail LoRA serving, with current APIs, Kubernetes chart hardening, adapter security, cache-aware routing, and vLLM trade-offs.
· 16 min
AI Agents
How domain-driven design helps AI agents use shared language, bounded contexts, services, repositories, and deterministic business rules.
· 16 min
AI Agents
How to design context engineering for AI agents: context windows, instruction hierarchy, retrieval, memory, tools, guardrails, provenance, and compression.
· 25 min
AI Agents · Developer Tooling
Build a custom MCP server with uv and FastMCP, expose ML feature-store tools, test them locally, and connect the server to Claude Desktop.
· 10 min
Model Engineering
How to read open-weight model names by separating training role, architecture, quantization method, storage format, runtime, and hardware fit.
· 14 min
Developer Tooling
Compare local image-generation tools on macOS by setup, model support, workflow control, reproducibility, extensions, and maintenance status.
· 12 min
Model Engineering · Developer Tooling
Compare local LLM tools on macOS by interface, model format, memory headroom, API exposure, and Apple Silicon development workflow.
· 12 min
Developer Tooling
How pyproject.toml works for build systems, project metadata, runtime dependencies, development groups, CLI entry points, and tool configuration.
· 8 min
Developer Tooling
Zsh startup files explained: when ~/.zprofile and ~/.zshrc load on macOS and Linux, what belongs in each file, and how to diagnose the running shell mode.
· 7 min
AI Infrastructure
What foundation models changed in MLOps: release units, retrieval, tools, evaluation, tracing, serving, and the tools that do each job.
· 16 min
Model Engineering · AI Infrastructure
How to scale LLMs across GPUs and nodes with data, sharded, tensor, pipeline, context, and expert parallelism, and which framework implements each.
· 18 min
Developer Tooling
A MacBook setup checklist for AI engineering: Xcode tools, Homebrew, Python with uv, Docker, terminal setup, and VS Code.
· 7 min
Developer Tooling
How I use uv on macOS for Python installs, project dependencies, lock files, inline scripts, and one-off CLI tools.
· 9 min
Nothing here yet.
