https://slavadubrov.com/Practical AI engineering by Slava Dubrov.enhttps://slavadubrov.com/blog/2026/08/15/typescript-for-python-ml-engineers/https://slavadubrov.com/blog/2026/08/15/typescript-for-python-ml-engineers/# TypeScript for Python ML engineers: build an agent service This is a fast onboarding guide for experienced Python engineers who need to ship AI services in TypeScript and Node. It is written for ML engineers, data scientists, and backend developers who do not need a beginner JavaScript course. I did that onboarding myself over the past few months after working in Python and Java. Most guides I found started with basic programming or front-end DOM work. This article starts from Python service concepts. By the end, you can map a Python service stack to TypeScript and recognize the Python habits that cause JavaScript bugs. The running example follows a streaming agent service from schema to deployment.Sat, 15 Aug 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/07/22/ai-agent-harness-engineering/https://slavadubrov.com/blog/2026/07/22/ai-agent-harness-engineering/# Harness Engineering for AI Agents: Designing Control Loops An agent's reasoning loop chooses the next action. Its **harness** supplies context, validates proposed tool calls, authorizes them, dispatches what it accepts, records the results, and decides whether the task is complete. The first three of those jobs already have an article. [Memory](/blog/2026/02/14/ai-agent-memory-architecture/) supplies the context, [tool use](/blog/2026/03/24/ai-agent-tool-use/) defines what can be proposed, and [security](/blog/2026/04/20/ai-agent-security/) decides what runs. They ran as separate posts because they are separate engineering problems. Choosing between Qdrant and pgvector has nothing to do with writing a PreToolUse deny rule. They also share a moment: the gap between the model naming an action and the machine performing it. Each answers a question about that gap, and the harness is the code that holds it open long enough to ask all three. For engineers building or reviewing coding-agent harnesses, the remaining job is to decide whether finished work is actually complete and to prove that each control in the loop earns what it costs. [Part 5](/blog/2026/05/26/ai-agent-runtime/) covered the runtime that keeps the process alive underneath.Wed, 22 Jul 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/07/05/model-quantization-in-2026-from-foundations-to-production-serving/https://slavadubrov.com/blog/2026/07/05/model-quantization-in-2026-from-foundations-to-production-serving/# Model Quantization Guide: Foundations to Production Serving Quantization uses fewer bits to represent model values. A serving path may quantize [weights, activations, the KV cache, or some combination](https://nvidia.github.io/TensorRT-LLM/features/quantization.html), and each target solves a different serving problem. Choose the target from the current bottleneck: [weight memory, activation compute, KV-cache size, kernels, hardware, calibration data, or quality](https://docs.vllm.ai/en/latest/features/quantization/). A 4-bit model may fit in VRAM yet run slowly with an unoptimized kernel, as the [JarvisLabs vLLM benchmark](https://jarvislabs.ai/blog/vllm-quantization-complete-guide-benchmarks) demonstrates. FP8 works well on NVIDIA Hopper hardware with a compatible runtime and offers no native benefit on unsupported GPUs. TensorRT-LLM's [hardware matrix](https://nvidia.github.io/TensorRT-LLM/features/quantization.html#hardware-support-matrix) shows the supported paths. For long contexts or high concurrency, [KV-cache quantization](https://docs.vllm.ai/en/stable/features/quantization/quantized_kvcache/) may save more memory than weight quantization. ML and platform engineers can use the bottleneck to select a candidate format and runtime, then validate that exact serving path before deployment.Sun, 05 Jul 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/06/20/schema-guided-agent-memory/https://slavadubrov.com/blog/2026/06/20/schema-guided-agent-memory/# AI Agent Memory: Schema-Guided State and Provenance Long-running agents often retrieve stale facts because ordinary semantic memory has no rule for deciding which value is current. When a user changes a passport deadline from July 15 to June 30, vector search can retrieve both statements. A stateful memory layer must record that the June 30 value supersedes the earlier one. For engineers building long-running or multi-tenant agents, that stale-fact failure is why memory must survive separate runs while enforcing current state and tenant boundaries. The design below shows how to write typed records and test current versus point-in-time reads without treating vector recall as the source of truth.Sat, 20 Jun 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/06/10/agent-evals-traces-to-test-suites/https://slavadubrov.com/blog/2026/06/10/agent-evals-traces-to-test-suites/# AI Agent Evaluation in Production: Traces to Test Suites A final answer can say that a refund is complete while its trace shows that `verify_identity` never ran, `issue_refund` retried 17 times, or the agent declared success before the database changed. Answer-only grading hides those failures. For engineers operating tool-using agents in production, the fix is to turn repeatable traces into bounded regression cases: deterministic checks enforce tool order, arguments, loops, and invariants; calibrated judges handle the decisions that require interpretation. The result is a versioned suite that catches the same failure before the next release.Wed, 10 Jun 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/05/26/ai-agent-runtime/https://slavadubrov.com/blog/2026/05/26/ai-agent-runtime/# Long-Running AI Agent Runtime: Sessions and Checkpoints An agent run may last for hours, while its worker process may restart at any time. The model still chooses the next action, but the runtime must preserve state, control execution, and recover from a failure in the middle of a tool call. This post defines the runtime boundary. [Part 6](/blog/2026/07/22/ai-agent-harness-engineering/) then opens the one component in it that decides — the harness, which is where memory, tool contracts, and permission checks stop being three topics and become one program.Tue, 26 May 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/05/10/rag-evaluation-metrics/https://slavadubrov.com/blog/2026/05/10/rag-evaluation-metrics/# RAG Evaluation Metrics: Retrieval, Reranking, Generation A RAG system with broken filters can run for months without triggering an operational alert. It still returns answers and meets its latency target, but the answers rely on incomplete evidence. Recall@k against the original gold set exposes the loss. Latency and availability dashboards do not. For engineers operating or evaluating multi-stage RAG systems, this reference maps failures in document parsing, filtering, retrieval, reranking, and generation to the metric that identifies each one, then shows where that measurement belongs in a release or monitoring gate.Sun, 10 May 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/04/20/ai-agent-security/https://slavadubrov.com/blog/2026/04/20/ai-agent-security/# AI Agent Security: Permissions, Sandboxes, and MCP Threats Agent security begins after the model proposes an action and before the machine performs it. The question is which control gets the last word when the action reaches credentials, files, networks, or an external side effect. The program that holds that gap open is the **harness** — the control loop that builds each prompt, decides which proposed tool calls actually run, and reads the results back. Most of the controls below live there, because the moment before execution is the last point at which a check is still cheap. The rest sit on either side of it. Once the command runs, what is left is the sandbox around it, the credentials it was handed, and whatever you can undo afterwards — and some of the incidents in this article never reached a model at all. AI agent security is broader than LLM safety. Early guardrail products inspected the input and output of one model call. They could filter toxic text, redact personal data, block jailbreaks, and reject off-topic answers. That boundary was useful while the model could only return text. Tool loops added filesystems, shells, Model Context Protocol (MCP) servers, and credentials. That expanded the threat model from unsafe text to unsafe actions. The six incidents examined below were not failures that a better output filter could prevent; the surrounding system had been compromised. When an agent can read a repository, call a tool, or move data to a third party, engineers need to map each proposed action to the control that can actually stop it. The sections below make that map: permissions, hooks, sandboxes, credentials, and human review each belong at a different boundary.Mon, 20 Apr 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/04/02/ner-guide/https://slavadubrov.com/blog/2026/04/02/ner-guide/# NER Guide 2026: GLiNER, spaCy, Transformers, and LLMs Named entity recognition (NER) now spans compact encoders, open-vocabulary models, and LLM-based extraction. In the cited CrossNER evaluation, a 300M-parameter GLiNER model exceeds the reported zero-shot F1 of 13B UniNER. A newer bi-encoder reports **up to 130 times the throughput** of the comparable `gliner_small-v2.5` uni-encoder at 1,024 entity types when labels are pre-computed. The paper measured this on a single H100 with batch size 1 across 64-, 256-, and 512-token inputs. These results motivate experiments, not a universal production ranking. The companion repository provides runnable examples for GLiNER, ONNX export, LLM-generated training labels, and structured extraction. For workloads dominated by explicit spans, compact encoders are usually the faster and cheaper option. LLMs remain useful for producing training data and handling cases that require inference or normalization. This guide is for engineers choosing and evaluating NER architectures for RAG, agent, document, or privacy pipelines. You will leave with a model-selection path, a bounded evaluation plan, and a three-tier design for combining encoders with LLM extraction. > **Companion repo:** [**ner-field-guide**](https://github.com/slavadubrov/ner-field-guide), with runnable demos for GLiNER, ONNX export, the LLM-as-teacher pipeline, and structured extraction with Instructor.Thu, 02 Apr 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/03/24/ai-agent-tool-use/https://slavadubrov.com/blog/2026/03/24/ai-agent-tool-use/# AI Agent Tool Use: MCP, CLI, Skills, and Code Execution [Part 1](/blog/2026/01/31/ai-agent-reasoning-loops/) covered reasoning loops, and [Part 2](/blog/2026/02/14/ai-agent-memory-architecture/) covered memory. This article adds the action layer: how an agent exposes, selects, and runs tools. Memory decides what the agent knows going into a turn; this layer decides what it can do about it. [Part 4](/blog/2026/04/20/ai-agent-security/) covers the check that decides whether a named call runs at all, and [Part 6](/blog/2026/07/22/ai-agent-harness-engineering/) covers the harness that runs both the call and the check. Throughout this article, the **harness** means the control program that assembles each prompt, dispatches the tool calls it accepts, and decides when the task is done — everything around the model that is ordinary code you write. The tooling story changed in 2025–2026. **MCP**, the Model Context Protocol, gave vendors one shared way to expose external services. Code-executing agents showed that a model can sometimes compose a small program more efficiently than it can issue a long sequence of JSON calls. Anthropic reported a **98.7% token reduction** for one Google Drive-to-Salesforce workflow, and the CodeAct paper reported gains of up to **20%** across its benchmark setup. Those results describe their tasks and harnesses, not a universal advantage for code execution. I compare JSON tool calling, MCP, Skills, CLI tools, and code execution in that order. A later section applies Agent-Computer Interface (ACI) design principles to the [Market Analyst Agent](https://github.com/slavadubrov/market-analyst-agent), a small LangGraph research agent I built for Part 1 that fetches market data and writes an analyst report.Tue, 24 Mar 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/03/13/llm-engineering-guide/https://slavadubrov.com/blog/2026/03/13/llm-engineering-guide/# LLM Engineering Guide: 45 Concepts for Production Systems An LLM service can miss its latency SLO because decode is limited by memory bandwidth, because the KV cache has consumed the batch budget, or because the queue is masking both. A fine-tuning run can fail for a different version of the same reason: model state no longer fits the hardware. The right fix follows from the bottleneck, not from the longest list of techniques. This is a reference for engineers who already know basic ML and systems concepts, and need to connect a production symptom to the relevant part of the stack. It covers 45 concepts across hardware, inference, training, deployment, applications, and operations. Use an entry to identify the mechanism, its practical consequence, and the condition that limits the cited result; then use the linked deep dive or your own benchmark to make the decision.Fri, 13 Mar 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/03/04/ocr-guide/https://slavadubrov.com/blog/2026/03/04/ocr-guide/# OCR in 2026: Classical Pipelines, VLMs, and Document AI OCR leaderboards disagree because they test different documents, outputs, and judges. An early-2026 snapshot of OmniDocBench and OCR Arena produced sharply different model orderings; the scores are not interchangeable, but the disagreement is useful. A production choice needs documents and metrics from the actual workload. Vision-language models (VLMs) can handle layout, handwriting, tables, and degraded images that break a plain text-recognition pipeline. Traditional engines remain competitive on clean print, especially when CPU latency and operating cost matter. The dated snapshot below includes [PaddleOCR-VL 1.6](https://www.paddleocr.ai/main/en/version3.x/pipeline_usage/PaddleOCR-VL.html) and [dots.mocr](https://github.com/rednote-hilab/dots.mocr), with different hardware, privacy, and output trade-offs. Check each project before making a current choice. > **Companion repo:** [**The OCR Gauntlet**](https://github.com/slavadubrov/ocr-gauntlet) contains three notebooks. Its main notebook compares up to five OCR engines on five downloaded samples and reports CER, WER, ANLS, latency, and estimated cost. The checked-in run contains Tesseract, Docling + Tesseract, Mistral OCR v3, and a Gemini run; dots.ocr was unavailable. Do not use the Gemini row for model comparison: the runner requests `gemini-2.5-flash`, but the checked-in notebook labels that output Gemini 3 Flash. This guide is for engineers choosing an OCR or document-AI pipeline for real workloads where text, layout, tables, or extracted fields matter. After reading it, you should be able to choose a baseline, compare models on task-specific metrics, and route uncertain or high-risk fields to validation or review.Wed, 04 Mar 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/02/21/modern-data-processing-engines/https://slavadubrov.com/blog/2026/02/21/modern-data-processing-engines/# Data Processing Engines: Polars, DataFusion, Ray, and Spark For years, **Pandas** handled in-memory tabular work and **Apache Spark** handled the distributed kind. That split worked fine when the data was structured. Modern pipelines also process images, audio, and video. In these workloads, CPU decoding can starve GPU inference, while JVM garbage collection and Python's Global Interpreter Lock limit throughput. New engines use **Rust** and **Apache Arrow** to reduce those costs. To compare the options, I benchmarked Pandas, Polars, DataFusion, Daft, and native Rust on two real datasets. Spark and Ray are covered in separate distributed notebooks. NYC taxi trips represent tabular work; Food-101 images represent a multimodal pipeline. The [engine-comparison-demo repository](https://github.com/slavadubrov/engine-comparison-demo) contains the code. Run it on your own hardware because engine performance depends on the machine, dataset, and workload.Sat, 21 Feb 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/02/14/ai-agent-memory-architecture/https://slavadubrov.com/blog/2026/02/14/ai-agent-memory-architecture/# AI Agent Memory Architecture: Checkpoints and Vector Stores A reasoning loop only survives one request unless its state is stored outside the worker. Without **agent memory**, the agent cannot resume a paused plan, recover after a crash, or recall a preference from an earlier session. [Part 1](/blog/2026/01/31/ai-agent-reasoning-loops/) covered the control flow. This post identifies which state each later turn needs and where that state should live. I'll use the [Market Analyst Agent](https://github.com/slavadubrov/market-analyst-agent) — a small LangGraph agent that fetches market data and writes an analyst report — to anchor the hot-checkpoint discussion. The cold-vector and raw-Markdown sections are independent illustrative designs that show extensions the current project does not yet implement. Then I'll cover when PostgreSQL, Redis, Qdrant, key-value stores, and plain Markdown files each make sense.Sat, 14 Feb 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/02/08/search-ranking-stack/https://slavadubrov.com/blog/2026/02/08/search-ranking-stack/# Search Ranking Stack: BM25, Embeddings, and Reranking Search must satisfy both exact and semantic intent. A query for "wireless headphones" should match those words, but the final order may also depend on product quality, user preferences, and availability. No single ranking method handles all of those signals well. This post builds the stack one stage at a time: BM25 retrieval, dense embeddings, Reciprocal Rank Fusion, cross-encoder reranking, and finally LLM listwise ranking. A [companion demo repository](https://github.com/slavadubrov/search-ranking-stack) contains runnable code for the stages on a sampled slice of Amazon ESCI product-search data.Sun, 08 Feb 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/01/31/ai-agent-reasoning-loops/https://slavadubrov.com/blog/2026/01/31/ai-agent-reasoning-loops/# AI Agent Reasoning Loops: ReAct, ReWOO, Plan-and-Execute An **agent reasoning loop** is the control flow that decides when a model plans, calls a tool, reads the result, and stops. For an engineer building an agent, that choice is also a budget: it determines how often the model runs, how much history each call carries, and whether a surprising result can change the next action. This article compares **ReAct**, **ReWOO**, and **Plan-and-Execute** through a LangGraph Market Analyst Agent I built. You will leave with a routing rule and implementation shapes to adapt, rather than three names to add to a diagram. The loop is the innermost layer of the series. Memory, tools, security, runtime, and acceptance checks surround it; they do not replace it.Sat, 31 Jan 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/01/21/manifold-constrained-hyper-connections/https://slavadubrov.com/blog/2026/01/21/manifold-constrained-hyper-connections/# DeepSeek mHC: Manifold-Constrained Hyper-Connections Modern deep learning rests on the residual connection. Hyper-Connections (HC) explore another architectural dimension: widen the residual state into several interacting streams. DeepSeek's **[Manifold-Constrained Hyper-Connections (mHC)](https://arxiv.org/abs/2512.24880)** paper studies how to keep that routing stable at larger training scales. This post starts with standard residual connections, then adds Hyper-Connections and the instability they create. The mHC constraint and its implementation cost come at the end.Wed, 21 Jan 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/01/11/enterprise-rag-challenge-agent-architectures/https://slavadubrov.com/blog/2026/01/11/enterprise-rag-challenge-agent-architectures/# Enterprise RAG Challenge 3: Lessons from Public Entries Enterprise RAG Challenge 3 (ERC3) asked agents to complete business tasks against a simulated company API. The frozen prize leaderboard is unusually useful because many entrants published more than a score: architecture, model mix, cost, and failure notes. I reviewed those public descriptions to answer a narrower question: which design choices recurred in strong submissions, and which of them are useful outside this benchmark? By the end, you should be able to turn these observations into design hypotheses for your own agent traces, then test them against your task mix and failure costs.Sun, 11 Jan 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/01/04/llm-fine-tuning-guide/https://slavadubrov.com/blog/2026/01/04/llm-fine-tuning-guide/# LLM Fine-Tuning Guide: LoRA, QLoRA, Unsloth, Axolotl Most fine-tuning failures are decision failures. A team trains before proving that prompting, retrieval, or constrained decoding cannot solve the problem. Another evaluates on the training distribution, or discovers after training that the artifact is awkward to serve. This guide treats adaptation as an experiment with an operational exit. It starts with the decision boundary, then follows one path through data, LoRA or QLoRA, task-specific evaluation, export, and serving.Sun, 04 Jan 2026 00:00:00 GMThttps://slavadubrov.com/blog/2025/12/28/schema-guided-reasoning-vllm/https://slavadubrov.com/blog/2025/12/28/schema-guided-reasoning-vllm/# Schema-Guided Reasoning: vLLM, XGrammar, and Pydantic This article is for Python engineers who need model output that downstream code can validate. You will learn how to define a Pydantic schema, request structured output from vLLM, and add application checks for meaning and policy. Retrying an LLM call does not guarantee valid JSON. The next sample may fail in the same way, and repeated calls add latency and cost. Schema-Guided Reasoning (SGR) enforces a schema while the model generates each token. You define the required fields with Pydantic, and the inference engine blocks tokens that would violate that structure. The result is syntactically valid by construction rather than by retry.Sun, 28 Dec 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/10/22/lorax-serving-guide-lora-adapters-kubernetes/https://slavadubrov.com/blog/2025/10/22/lorax-serving-guide-lora-adapters-kubernetes/# LoRAX Serving Guide: LoRA Adapters on Kubernetes at Scale One base model and many LoRA adapters create an unusual serving problem. The base weights are shared, but every request may need a different set of adapter weights. A conventional one-deployment-per-variant design wastes GPU memory when most variants are idle. [LoRAX](https://github.com/predibase/lorax) addresses that long tail. It loads adapters on demand, batches requests for different adapters, and moves adapter weights between GPU and CPU memory. The attractive headline is "thousands of fine-tuned models on one GPU." The engineering question is narrower: given your active adapter set, arrival pattern, and latency target, does exchange scheduling help enough to justify another serving runtime? This guide is for inference and platform engineers who need to serve many LoRA variants from one base model. It shows how to test the documented APIs, size the working set, and turn the starter Helm chart into an explicit production plan.Wed, 22 Oct 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/10/20/domain-driven-design-ai-agents/https://slavadubrov.com/blog/2025/10/20/domain-driven-design-ai-agents/# Domain-Driven Design for AI Agents: Contexts and Rules Agent projects become difficult to change when prompts, code, and business processes use different terms. Compliance asks for a "policy check," while the implementation exposes `process_data()`. The vague name hides which rule is being applied, who owns it, and where a change belongs. Domain-Driven Design (DDD) puts that business language and ownership at the center. For an agent, the model can interpret a request and propose a typed command. An application service then supplies trusted context, and the domain model accepts or rejects the state change. This guide connects the vocabulary and boundary work to that execution path. This guide is for engineers building agents that change business state and need explicit ownership of domain rules. You will learn how to map a model proposal to an authorized application command and where the illustrative flow still requires a concrete transaction implementation.Mon, 20 Oct 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/10/05/context-engineering-ai-agents/https://slavadubrov.com/blog/2025/10/05/context-engineering-ai-agents/# Context Engineering for AI Agents: Memory and Tools Context engineering is the pipeline that selects what a model sees before each decision: instructions, examples, knowledge, memory, tool definitions, observations, and guardrails. An agent does not act on everything the system knows. It acts on the working set assembled for the next model call. That selection is where many failures begin. A stale preference looks current. Retrieved text contains an instruction. A long tool result hides the failed precondition. A summary remembers the decision but loses which file changed. For engineers building or operating AI agents, retrieval systems, and tool-using applications, the practical job is to assemble the smallest sufficient working set for each agent decision while preserving scope, provenance, and permissions. This article shows how to design a per-decision context pipeline, find its trust boundaries, test whether its working set supports the task, and make failures inspectable rather than blaming a model that only sees the assembled input.Sun, 05 Oct 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/06/10/mcp-server-tutorial-uv-fastmcp/https://slavadubrov.com/blog/2025/06/10/mcp-server-tutorial-uv-fastmcp/# MCP Server Tutorial: Build with Python, uv, and FastMCP Python developers can turn a local feature-store workflow into a version-pinned FastMCP server for Claude Desktop, using **uv** to create the environment, expose tools and a resource, and verify the server locally before the desktop connection.Tue, 10 Jun 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/05/11/open-source-llm-formats/https://slavadubrov.com/blog/2025/05/11/open-source-llm-formats/# Open-Weight LLM Variants, Formats, and Quantization Names such as `Model-32B-A3B-Instruct-AWQ` look dense because they combine several independent decisions: family and size, architecture, training role, and quantization. A repository may package those weights as sharded Safetensors, while a community conversion of the same checkpoint appears as `Q4_K_M.gguf`. Those labels do not share one category: GPTQ and AWQ are quantization methods, GGUF is a container and runtime ecosystem, and MoE is an architecture. Reading the layers separately makes the download choice easier. When choosing between a `Q4_K_M.gguf` download for local inference and an AWQ Safetensors repository for GPU serving, start with the checkpoint behavior and architecture your task needs. Then confirm that its numeric representation, package layout, and target runtime fit the available hardware, memory, and serving constraints.Sun, 11 May 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/05/10/stable-diffusion-macos-tools/https://slavadubrov.com/blog/2025/05/10/stable-diffusion-macos-tools/# Stable Diffusion on macOS: Local Image Tools Compared Running image models locally on macOS is mostly a tooling choice. Draw Things, DiffusionBee, ComfyUI, AUTOMATIC1111, and Fooocus overlap, but they differ in setup, model support, workflow control, extensions, and automation. A drag-and-drop app gets you started quickly; a node graph takes longer to learn but exposes more of the pipeline. On Apple Silicon, the application, model family, precision, image size, and workflow all affect whether generation fits in memory. Choose the interface from the work you need to reproduce, then test the exact model and settings.Sat, 10 May 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/05/10/local-llms-on-macos/https://slavadubrov.com/blog/2025/05/10/local-llms-on-macos/# Local LLMs on macOS: Ollama, LM Studio, MLX, llama.cpp Engineers running local models on an Apple Silicon Mac need to size the complete workload, not just choose a model that downloads. Apple Silicon can run everyday prompts without a remote API because the CPU and GPU share one [unified memory pool](https://ml-explore.github.io/mlx/build/html/usage/unified_memory.html), but model weights compete there with the KV cache, runtime workspace, and macOS. Once the intended model, context, and memory headroom fit, choose the control surface that matches the workflow: Ollama for a managed local service, LM Studio for desktop inspection, llama.cpp for direct GGUF execution, or MLX-LM for Apple-native Python. That keeps model-quality testing separate from tool choice and gives you a fair, memory-aware benchmark for the stack you will run.Sat, 10 May 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/05/08/pyproject-toml-guide/https://slavadubrov.com/blog/2025/05/08/pyproject-toml-guide/# pyproject.toml Guide: Packaging, Dependencies, and Tools Older Python projects often spread configuration across `setup.py`, `setup.cfg`, requirements files, `MANIFEST.in`, and separate files for development tools. `pyproject.toml` gives those concerns a common home, although it does not replace every project file or make every section part of one standard. This guide walks through the file from build configuration to project metadata and tool settings. Four owners keep the sections distinct: what builds the project, what the project publishes, what contributors need locally, and what individual tools configure.Thu, 08 May 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/05/07/zprofile-vs-zshrc/https://slavadubrov.com/blog/2025/05/07/zprofile-vs-zshrc/# Zsh Startup Files: .zprofile vs .zshrc on macOS and Linux If your terminal feels slow, or an environment variable appears in one terminal but not another, the cause is often Zsh's startup mode. The two files most people reach for are `~/.zprofile` and `~/.zshrc`, but neither filename alone tells you whether a line belongs there. Zsh chooses startup files from two independent properties: whether the shell is a **login shell** and whether it is **interactive**. Terminal applications on macOS and Linux can choose different combinations, so inspect the running shell instead of assuming the operating system decides for you.Wed, 07 May 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/05/06/mlops-vs-llmops/https://slavadubrov.com/blog/2025/05/06/mlops-vs-llmops/# MLOps vs LLMOps: Infrastructure for Foundation Models When a foundation-model release regresses, the weight file may be unchanged: a prompt, retrieval index, tool permission, route, or policy can be the moving part. For ML, platform, and AI application engineers who already operate conventional MLOps, the practical artifact is therefore a release manifest that names those dependencies and points to the first control to inspect when a failure appears. This is an editorial operating model, not a claim that every application needs the same platform. Keep MLOps lineage, staged delivery, monitoring, and rollback; expand the unit you version when generated answers or tool actions make behavior depend on more than the weights.Tue, 06 May 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/05/04/scaling-llms-multi-gpu-multi-node/https://slavadubrov.com/blog/2025/05/04/scaling-llms-multi-gpu-multi-node/# Scaling LLMs with Multi-GPU and Multi-Node Parallelism Large-model workloads move beyond one GPU for different reasons. A training job may run out of memory on optimizer state. Another may run out on long-sequence activations. A model that fits may still miss its throughput target. Each problem calls for a different partition and communication pattern. This is a practical walk-through of the main parallelism strategies and the constraints behind them, informed by Hugging Face's [Ultra-Scale Playbook](https://huggingface.co/spaces/nanotron/ultrascale-playbook). The goal is to show what each split buys, what it communicates, and when combinations become necessary.Sun, 04 May 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/04/19/macbook-ai-engineering-setup/https://slavadubrov.com/blog/2025/04/19/macbook-ai-engineering-setup/# MacBook Setup for AI Engineering: macOS Tools and Workflow This is the setup I apply to a fresh MacBook before doing AI engineering work. It covers the command-line tools macOS does not include, Homebrew, Python with uv, the shell, terminal and editor choices, Docker, and local AI tools. The useful boundary is between required foundations and personal preferences. Xcode Command Line Tools, Homebrew, Git, and a Python workflow are foundations. Warp, iTerm2, Powerlevel10k, Cursor, and local model runners are choices. The commands below install the foundation; the later sections record the choices I make on top. This is a documented setup sequence, not a bit-for-bit reproducible machine image.Sat, 19 Apr 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/04/17/uv-python-on-macos/https://slavadubrov.com/blog/2025/04/17/uv-python-on-macos/# uv on macOS: Managing Python Versions, Projects, and Tools I switched my Python workflow to uv. It replaced the tool switching I used to do between pip, virtual environments, pip-tools, pipx, and project managers. One executable now covers most of that work. The workflows are still different. For Python developers moving to uv on macOS from pip, virtual environments, pip-tools, or pipx, a project, an inline-metadata script, a one-off CLI, and an installed CLI each belong to a different environment and lifecycle. The commands below show how to choose among them. ## Quick start ```bash # Install uv with Homebrew brew install uv # Start a project uv init example-app cd example-app uv add httpx uv run python main.py # Run an isolated CLI without installing it permanently uvx ruff check . ```Thu, 17 Apr 2025 00:00:00 GMT