Tech questions, answered
45 short answers from 9 guides. Each set links to the full guide for the worked examples and the reasoning behind the answer.
Fine-Tuning Datasets and LLM Evals: A Practical Guide
What is the difference between SFT and DPO?
Supervised fine-tuning (SFT) trains a model to predict the assistant replies you want, using next-token cross-entropy on the assistant turns. Direct Preference Optimization (DPO) shows two answers to one prompt and raises the probability of the preferred one relative to the rejected one, compared with a frozen reference copy. OpenAI recommends running SFT on the preferred responses first, then DPO on the pairs.
How many examples do I need to fine-tune an LLM?
OpenAI asks for at least 10 training examples and reports clear improvements from 50 to 100 well-chosen ones. That matches the general experience that a few hundred consistent, high-quality rows beat thousands of noisy ones when fine-tuning for behaviour and format. Coverage of the hard cases the base model gets wrong matters more than raw volume.
Can fine-tuning teach a model new facts?
Fine-tuning is a poor way to add new facts to a language model. A model fine-tuned on a few hundred examples of a fact rarely recalls it reliably, so facts are better supplied at run time through retrieval. Supervised fine-tuning works well for teaching format and behaviour, such as tone, policies, labels, JSON extraction or a house style.
What is the difference between exact match and F1 score?
Exact match scores 1 only when the normalised output and expected answer are equal, after SQuAD normalisation lower-cases, removes punctuation and the articles a, an and the, and collapses whitespace. Token F1 gives partial credit for overlapping words: "Canberra is the capital" against "Canberra" has precision 1/3, recall 1 and an F1 of 0.5.
What is ROUGE-L used for in LLM evaluation?
ROUGE-L, from Lin (2004), scores the longest common subsequence of words between a model output and a reference, rewarding the same words in the same order. It was built for summaries. Like other overlap metrics, ROUGE-L can punish a correct answer phrased differently and reward a wrong answer that repeats the right words, so read a sample of failures.
How to Build an MCP Server in TypeScript or Python
What is an MCP server?
An MCP server is a small program that gives AI applications actions they can take, data they can read and prompt templates people can reuse. Once it speaks the Model Context Protocol, the same server works in Claude, Cursor, VS Code, Codex and other clients. It can offer three kinds of thing: tools, resources and prompts.
What is the difference between MCP tools, resources and prompts?
Tools are actions the model chooses to call, each with a name, a description and a JSON Schema for its arguments. Resources are read-only data addressed by a URI, and the application decides when to load them into context. Prompts are templates a user picks from a menu in the client. Most MCP servers start with tools.
Which npm package do I install for the MCP TypeScript SDK?
The MCP TypeScript SDK's v2 line splits the old @modelcontextprotocol/sdk package into @modelcontextprotocol/server and @modelcontextprotocol/client, and uses Zod 4 for schemas. To build a server, install @modelcontextprotocol/server and zod, add typescript and @types/node as dev dependencies, and set "type": "module" in package.json.
How should an MCP server handle secrets?
A stdio MCP server should read keys from environment variables and let each user set them in their client's config. Never take secrets as tool arguments, because the model would see them and they would end up in logs. A hosted HTTP server can use the OAuth-based authorization flow in the MCP specification and should check the caller's permission on every call.
How do I write MCP tool descriptions that models use well?
A model never sees an MCP server's code, only tool names, descriptions and schemas, so describe when to use each tool, not just what it does. "Search orders by status and date. Use when the user asks about past or pending orders." beats "Searches orders." Constrain inputs with enums, trim responses to the fields that matter and write errors the model can recover from.
How to Make Your Website Visible to AI Assistants and AI Search
How do I block AI training crawlers but still appear in AI search?
Give training crawlers and AI search crawlers separate robots.txt groups, because operators document them as separate user agents. Disallow GPTBot, ClaudeBot, CCBot, Meta-ExternalAgent, Google-Extended and Applebot-Extended, while allowing OAI-SearchBot, Claude-SearchBot and PerplexityBot. Under RFC 9309 a crawler follows only the group that names it, so copy your general rules into each AI group.
Do AI crawlers run JavaScript?
Many AI crawlers and fetchers read the HTML the server returns and do not run JavaScript, or do so inconsistently. A single-page app that sends an empty root div and a script tag can look blank to them, even if Googlebot indexes it fine. Server-side rendering or prerendering fixes this; check with View source, not the Elements panel.
Does blocking Google-Extended remove a site from AI Overviews?
No. Google-Extended is a control token that never fetches anything: it tells Google whether content Googlebot already fetched may be used for Gemini model training. Blocking it does not affect search inclusion or rankings. For AI Overviews and AI Mode, Google points site owners to the normal Search controls, robots rules for Googlebot and snippet settings.
Do I need special structured data to appear in Google AI Overviews?
No. Google's guidance says there is no special schema.org markup or other special optimisation needed to appear in AI Overviews or AI Mode, and normal SEO practice applies. Add JSON-LD because it describes the page accurately. Also check templates for nosnippet, data-nosnippet, max-snippet or noindex, which limit what Google's AI features show.
Do AI crawlers respect robots.txt?
robots.txt is a voluntary request. OpenAI, Perplexity, Google and Meta say their user-triggered fetchers may not follow it, because a person asked for the page, while Anthropic says its agents, including Claude-User, respect it. Unknown scrapers ignore it entirely, so use server or CDN rules, ideally with operators' published IP ranges, for anything that must be blocked.
How to Reduce LLM Token Usage and API Cost
How much does prompt caching save on LLM API costs?
Prompt caching bills a repeated prompt prefix at a fraction of the input price. At prices checked on 1 November 2026, an Anthropic cache hit costs 10 percent of the input price on most models and 5 percent on Claude Opus 5.5 and Sonnet 5.5. OpenAI's cached rates run from 50 percent on GPT-4o down to 5 percent on GPT-6.1 Sol.
Why is my prompt cache not getting hits?
Prompt caches only match when the entire prefix is identical, so one changed character near the start makes every following token a cache miss. A common cause is putting the current date or the user's name at the top of the system prompt. Keep stable instructions and reference material first, and put anything that changes, such as the question, last.
How much does a batch API save?
OpenAI, Anthropic and Google all offer a batch API at 50 percent off input and output tokens, in exchange for asynchronous processing with results within 24 hours. It suits evaluations, nightly classification, back-filling summaries and data extraction over archives. Batch discounts can stack with prompt caching, which Anthropic's pricing page confirms and OpenAI reflects in separate batch rates for cached input.
Why do output tokens cost more than input tokens?
Output tokens are priced at 4 to 8 times input on most current models; Claude Sonnet 5.5, for example, charges $2 per million input tokens and $10 per million output. In a chat request with 2,500 tokens in and 350 out, output is 41 percent of the cost despite being 12 percent of the tokens, so limiting answer length pays off first.
Does minifying JSON reduce LLM tokens?
Yes. In a test with 20 customer records of five fields each, pretty-printed JSON was 842 tokens, the same JSON minified was 482, and CSV with a header row was 248, less than a third of the original. CSV drops the field names from each row, though, so check that the model still reads the data correctly.
How to Set Up MCP Servers in Claude, Cursor, VS Code, Codex and Gemini CLI
What is the difference between local and remote MCP servers?
A local MCP server runs over stdio: the client starts it as a child process, it runs with your user account's permissions, needs no network port and gets secrets from environment variables. A remote server runs elsewhere over streamable HTTP and the client connects to a URL, authenticating with a header token or an OAuth login. Use local servers for files and shells, remote for hosted services.
How do I add an MCP server to Claude Code?
Run claude mcp add in the terminal, where everything after -- is the command that starts the server, or use --transport http with a URL for a remote one. Put --env before --transport, or the CLI reads the server name as another KEY=value pair. Check the connection with claude mcp list, or /mcp inside a session.
Where is the Cursor MCP config file?
Cursor reads MCP servers from .cursor/mcp.json in a project and from ~/.cursor/mcp.json for all projects, both under an mcpServers key. Remote servers use url and optional headers. Values can pull from the environment with ${env:NAME}, which keeps tokens out of the file, and ${workspaceFolder} points at the project root.
Where does OpenAI Codex CLI store MCP servers?
Codex CLI uses TOML in ~/.codex/config.toml, with one [mcp_servers.<name>] table per server, and a trusted project can have its own .codex/config.toml. Local servers take command, args, env and env_vars, while remote servers take url plus bearer_token_env_var. Start-up times out after 10 seconds and tool calls after 60 unless those timeouts are set.
Why won't my MCP server connect?
Run the exact command and arguments in a terminal first, since a missing runtime, a package typo or a bad path shows up immediately. Use absolute paths, make sure a stdio server prints nothing but protocol messages to stdout, and restart the client, because most read MCP config only at start-up. MCP Inspector tests the server alone, separating server bugs from config mistakes.
How to Write Better AI Prompts: Structure, Examples and Format
What are the parts of a good AI prompt?
A good AI prompt usually covers six parts: the task (what should be produced), context (why, and where it will be used), the audience, the input material to work from, examples of a correct answer, and the output format, including shape, length and tone. A one-sentence role such as "You are a careful financial analyst" also helps focus behaviour and tone.
Why does AI ignore my instructions?
AI models often miss instructions that are buried in a long paragraph, or treat a line in pasted material, such as "please ignore the previous message", as an instruction to themselves. Wrapping instructions, background and data in separate sections, with XML tags or Markdown headings, fixes this. Conflicting rules like "short but detailed" also force the model to choose, and it may choose differently each time.
How do I reduce AI hallucinations with a prompt?
Give the model explicit permission to admit uncertainty, for example "If you are not sure of a fact, say so rather than guessing", which Anthropic lists as a basic way to reduce hallucinations. Supply the facts it should use and tell it to leave out anything not listed. Asking for statistics with no source and no way out invites invented numbers.
Why should a prompt explain the reason behind an instruction?
With a reason, the model can generalise the rule. Anthropic's guide notes that "NEVER use ellipses" works less well than explaining that the response will be read aloud by a text-to-speech engine that cannot pronounce them. Knowing that, the model also avoids other things a speech engine would stumble over. Stating rules as what to do, rather than shouting prohibitions, works better too.
How do I get reliable JSON output from an AI model?
Ask for "only valid JSON that matches this schema, with no text before or after it", and include the schema and one example. Say what to do with missing values, such as using null, and fix formats that vary, such as dates as YYYY-MM-DD. In production, use the API's structured output or JSON mode and validate the result in code, because a prompt alone cannot guarantee valid JSON.
LLM GPU Memory and Training Compute, Worked Out
What is the KV cache memory formula?
KV cache memory = 2 × layers × KV heads × head dimension × bytes per value × tokens × batch. Use num_key_value_heads from config.json, not the attention heads. For Llama 3.1 8B in BF16, with 32 layers and 8 KV heads of 128 dimensions, that is 128 KiB per token, so an 8,192-token context takes 1 GB.
Why does training with Adam need 16 bytes per parameter?
Mixed-precision Adam keeps 2 bytes for the 16-bit weights, 2 bytes for the 16-bit gradients and 12 bytes of optimizer state: an FP32 master copy of the weights plus Adam's two moment estimates at 4 bytes each. That totals 16 bytes per parameter, the ZeRO paper's figure, so an 8B model needs 120 GB before any activations.
What is the 6ND formula for LLM training compute?
The 6ND formula estimates the cost of training a dense transformer as C ≈ 6 × N × D FLOPs, where N is the parameter count and D the training tokens. The forward pass costs about 2N FLOPs per token and the backward pass about 4N. Chinchilla's 70 billion parameters on 1.4 trillion tokens works out at 5.88 × 10²³ FLOPs.
What is the Chinchilla scaling law?
The Chinchilla scaling law, from Hoffmann et al. (2022), says that for a fixed compute budget the loss-minimising model grows in step with its data, at roughly 20 tokens per parameter. Substituting D = 20N into C = 6ND gives N = √(C / 120), so 10²² FLOPs points to about 9.1 billion parameters on 183 billion tokens.
What is model FLOPs utilization (MFU)?
Model FLOPs utilization (MFU) is the share of a GPU's dense peak throughput that a real training run achieves after communication and memory stalls, typically 30 to 50 percent. An H100 SXM has a dense BF16 peak of 989 TFLOPS, so at 40 percent MFU it does 396 TFLOPS of useful work. Budget at 30 to 40 percent unless you have measured better.
The Chief of Staff Agent: Designing Multi-Agent Systems That Finish the Job
What is the orchestrator-workers pattern for AI agents?
In the orchestrator-workers pattern, one orchestrator agent decides the subtasks at run time, delegates them to specialist agents and merges their results. It is also called the chief-of-staff pattern, and it fits work whose subtasks are not known in advance, such as research or a product launch. Anthropic's "Building effective agents" article, from December 2024, describes it.
What should an AI agent delegation message include?
An AI agent delegation message should include a task id, sender and recipient, the objective, the full context, any inputs, the expected deliverable, acceptance criteria, constraints, a budget such as maximum turns and cost, and whether approval is required. A specialist cannot see the orchestrator's conversation, so free-text messages like "look into competitors" fail quietly.
Where should human approval checkpoints go in a multi-agent system?
Put a human approval checkpoint in front of anything published, sent, paid for or deleted, and in front of decisions that change the rest of the work, such as approving positioning before copy is written. At each checkpoint the orchestrator should present what needs approval, the options and its recommendation, then wait, and silence must not count as approval.
How do you evaluate a multi-agent system?
Keep a set of 20 to 50 realistic goals with known good outcomes and run the multi-agent system on them after every prompt change. Record task success against acceptance criteria, cost per run, wall-clock time, escalations and checkpoint rejections, then compare against a single-agent baseline. If the team is not clearly better, the single agent wins on cost and simplicity.
What are common failure modes in multi-agent systems?
Common multi-agent failure modes are lost context, where a specialist was never told a decision; silent guessing, where it invents a plausible answer; endless polishing between reviewer and writer; duplicate work; unchecked merges; and over-delegation of trivial steps. The fixes are complete task messages, a "blocked" status, written criteria with a revision cap, a status board and an explicit check step.
Tracing LLM and Agent Apps with OpenTelemetry, Plus Audit Logs
What is the difference between LLM traces and AI audit logs?
A trace is engineering data: sampled, kept for days or weeks, and optimised for latency and debugging questions. An AI audit record answers accountability questions months or years later, such as who acted, with which model and prompt version, what checks ran and what the system did. It must exist for every decision, not a sample, and be hard to change.
What does EU AI Act Article 12 require for logs?
For high-risk AI systems, Article 12 of the EU AI Act requires that the system technically allows automatic recording of events over its lifetime. The logging must help identify situations that may present a risk or lead to a substantial modification, and support post-market monitoring and the deployer's monitoring. Article 12(3) sets minimum fields for remote biometric identification systems.
How long must AI system logs be kept under the EU AI Act?
Under the EU AI Act, Article 19 requires providers and Article 26(6) requires deployers of high-risk AI systems to keep logs under their control for a period appropriate to the intended purpose and of at least six months, unless other Union or national law, in particular data protection law, provides otherwise. Sector rules in finance and health often require longer.
How do I make an AI audit log tamper-evident?
Store AI audit records in an append-only table with a hash chain, where each record stores the previous record's hash, so edits and deletions become detectable. Copy the latest hash somewhere the database owner cannot change. Include versioned model and prompt details, tool calls, human approvals and a trace id so investigations can reach the detailed trace.
How should I sample OpenTelemetry traces for an AI agent?
Use the parentbased_traceidratio sampler, set with OTEL_TRACES_SAMPLER and a ratio such as 0.25 in OTEL_TRACES_SAMPLER_ARG. It keeps a fixed share of traces and makes child spans follow the root span's decision, so you never record half an agent run. Audit records, unlike traces, should not be sampled.