The Difference Between Prompt Engineering and Context Engineering

The on-call engineer gets paged: the sales copilot started refusing discount requests it approved yesterday. Someone "fixed" it by rewriting the system prompt overnight. The rewrite worked in staging. Production still fails — because the CRM snapshot appended to every request now includes a conflicting pricing table from a sandbox org. Nobody changed the instruction. They changed the cargo.

That incident is the boundary in one story. Prompt engineering designs what you tell the model — role, constraints, output format, tone. Context engineering designs what you feed the model — retrieved documents, tool results, history, metadata, and the order they appear in a finite window. Teams collapse the two because both show up in the same API call. Production reliability depends on keeping them separate in ownership, tooling, and incident response.

Where prompt design ends#

Prompt engineering covers instructions the model should treat as authoritative for how to respond:

• System role and task framing
• Output schema hints and refusal rules
• Few-shot examples that demonstrate format, not confidential content
• Tool-use instructions when the model chooses among declared tools

It ends where the prompt author no longer controls the bytes. Dynamic user input, retrieval results, webhook payloads, and conversation history are not prompt engineering — they are inputs to context assembly. You can write "only use provided documents" in the system message. If assembly ships twelve irrelevant documents, the prompt did not fail. The pipeline did.

Confusing the two produces a predictable failure mode: every context problem becomes a prompt rewrite. The system prompt swells. Policy duplicates across regions. Latency rises. The model still misses the one paragraph that mattered because it was item eleven in an unbounded list.

Where context orchestration begins#

Context engineering begins when you ask: what enters the window, from where, under what limit, and what gets dropped?

That includes:

1. Retrieval packing — score thresholds, max chunks, deduplication, citation IDs
2. Memory policy — what persists across turns vs what is summarised or discarded
3. Tool output shaping — truncate, schema-filter, or store-by-reference large payloads
4. Ordering — high-signal evidence before boilerplate; task instruction before history
5. Cross-request stability — policy in system role, volatile evidence in user role

A fraud-review workflow illustrates the split. Prompt engineering defines: "Classify this transaction as approve, review, or block. Return JSON with reason codes from the allowlist." Context engineering ensures the model sees the last three transactions, the merchant risk score, and the relevant policy excerpt — not the entire event stream, not raw Kafka messages, not last week's unrelated alerts.

import { createHash } from "node:crypto";

interface AssembledContext {
  systemPrompt: string;       // prompt engineering owns this string
  evidenceBlocks: Evidence[]; // context engineering owns selection + order
  packHash: string;
}

interface Evidence {
  id: string;
  text: string;
  score: number;
  source: "retrieval" | "tool" | "memory";
}

function assembleContext(
  basePrompt: string,
  candidates: Evidence[],
  budgetTokens: number,
): AssembledContext {
  const sorted = [...candidates].sort((a, b) => b.score - a.score);
  const kept: Evidence[] = [];
  let used = 0;

  for (const block of sorted) {
    const tokens = estimateTokens(block.text);
    if (used + tokens > budgetTokens) break;
    kept.push(block);
    used += tokens;
  }

  const evidenceText = kept
    .map((b) => `[${b.source}:${b.id}]\n${b.text}`)
    .join("\n\n");

  const packHash = createHash("sha256")
    .update(basePrompt + evidenceText)
    .digest("hex")
    .slice(0, 12);

  return {
    systemPrompt: basePrompt,
    evidenceBlocks: kept,
    packHash,
  };
}

Notice basePrompt is stable versioned text. evidenceBlocks varies per request. Mixing them in one undocumented string destroys your ability to trace regressions.

Ownership in real teams#

Prompt engineering should sit close to product behaviour — what the assistant must never do, what format downstream APIs expect. Context engineering should sit close to data — retrieval indexes, CRM integrations, agent tool wrappers.

When the same person owns both without tooling, you get prompt bloat: retrieval fixes implemented as "also mention X in the system message." When two teams own each side without a contract, you get integration fights: backend ships full JSON blobs; prompt team adds "be concise" and hopes.

The contract between them is an assembly interface:

• Prompt team publishes basePrompt v3.2 with required output schema
• Platform team guarantees evidence region never exceeds 4k tokens and always includes merchant_id metadata line
• Incidents tag layer: prompt | context before anyone edits text

Scenarios misdiagnosed as prompt problems#

In each case, prompt edits add words without adding information. Context edits change what the model can actually access.

When prompt engineering is the right lever#

Not every failure is assembly. Prompt engineering is the correct tool when:

• Output format drifted after a model upgrade — schema hints need updating
• Refusal behaviour is wrong given correct evidence — safety or tone instructions
• Tool selection is incorrect despite clean tool descriptions in context — reorder or clarify tool definitions
• Few-shot examples no longer match production input distribution

Run a controlled test: freeze assembly from a failing trace, swap only the prompt. If behaviour changes materially, prompt engineering owns the fix. If not, stop editing wording and inspect the pack.

Building both without collapsing them#

Practically:

1. Keep systemPrompt in a versioned file or registry — no dynamic concatenation hidden inside it
2. Implement assembly in typed code with explicit region budgets
3. Add pre-flight validation: task instruction present, evidence non-empty when required, total tokens < model limit minus completion reserve
4. Train incident responders to log prompt_version and pack_hash before any hotfix

Your eval harness should include cases that vary evidence while holding prompts constant, and cases that vary prompts while holding evidence constant. Most teams only test the second. That is why retrieval regressions ship unnoticed.

Rollout pattern that works#

Teams that formalise context engineering without a big-bang rewrite usually follow this sequence:

1. Instrument — log tokens by region and pack hash on every call for two weeks
2. Freeze prompts — stop editing system text while assembly is fixed; reduces variables
3. Cap evidence — enforce max chunks and dedupe; measure quality and latency delta
4. Assign owners — prompt version owned by product/ML; assembly owned by platform
5. Add gates — block calls on empty required evidence or budget overflow

Within a sprint you typically see clearer incidents ("evidence region empty" vs "model hallucinated") and faster fixes because the layer is named.

Treat prompt and context docs as separate runbooks. Prompt runbooks cover tone, refusals, and schema version. Context runbooks cover retriever thresholds, eviction order, memory summarisation jobs, and integration field allowlists. Mixing them guarantees every incident becomes a prompt edit.

Summary#

Prompt engineering shapes model behaviour. Context engineering shapes the information behaviour operates on. The API call merges both, but production ownership should not. Stop treating every missing fact as a prompt rewrite. Start treating every overloaded window as an assembly bug — with budgets, eviction, and traces — and prompt work returns to what it does best: clear instructions on a clean pack.