Foundation Model vs Fine-Tuned Derivative: What's Inherited

Enterprise procurement decks love fine-tuned models with domain names in the title — LegalGPT, FinAssist-7B, InternalCoder. The pitch implies bespoke intelligence. Reality is messier: a fine-tuned derivative inherits most of its behavior, limits, and failure modes from the foundation model it started from. The fine-tune adds a thin behavioral layer. Teams that ignore lineage overpay for branding and under-estimate inherited risks.

Defining the lineage#

A foundation model is the large pretrained (and usually instruction-aligned) checkpoint vendors or labs ship as the root of a family. It defines architecture, tokenizer, context window implementation, pretraining knowledge cutoff, and broad capability envelope.

A fine-tuned derivative starts from those weights and continues training on a narrower dataset — often LoRA adapters, full-weight SFT on domain dialogs, or preference optimization on enterprise examples. Parameters change partially or fully; architecture and tokenizer typically do not.

Think inheritance, not replacement:

The fine-tune adjusts priorities on the output distribution. It does not give the model eyes it never had.

What fine-tuning actually improves#

Fine-tunes earn their keep in narrow, stable regimes:

• Format adherence — your JSON shape, ticket categories, CRM field vocabulary
• Style and tone — brand voice, concise support replies
• Domain phrasing — insurance endorsements, internal acronyms
• Classification boundaries — subtle intent differences on your taxonomy

They rarely fix:

• Bad retrieval or missing documents
• Tool schemas the parent never learned to call reliably
• Multilingual gaps in the tokenizer/pretrain mix
• Reasoning tasks the foundation model fails at zero-shot
• Knowledge after the foundation cutoff unless RAG or continued pretrain supplies it

If your gap is "model does not know our 2026 product line," fine-tuning on 500 examples is the wrong tool. Update retrieval or pretrain on approved corpus — then optionally fine-tune for format.

Enterprise risk when selecting fine-tunes#

Capability claims without eval lineage#

A vendor fine-tune marketed for healthcare may only be SFT on public medical QA pairs atop a general instruct model. It inherits hallucination under clinical phrasing. Due diligence means tracing: parent model, data sources, eval suites, and whether adapters are merged or served separately.

Safety regression#

Domain fine-tunes optimized for "always answer" can erode refusals the parent had. Internal fine-tunes on ticket data accidentally include customer PII in training — the derivative memorizes and regurgitates. Inherited alignment is not guaranteed after additional training unless you re-verify.

Lock-in through adapters#

LoRA adapters tie you to a specific foundation revision. Upgrading the parent without retraining adapters breaks behavior silently. Document merge policies and version pins as strictly as you would for base models.

License and compliance inheritance#

Foundation model licenses flow to derivatives. Some prohibit certain domains; some require attribution; some ban using outputs to train competitors. A fine-tune does not reset legal terms.

Operational envelope#

Context limits, rate limits, and multimodal support are parent properties. A "32k legal fine-tune" on an 8k-effective parent still degrades at 8k if eval says so — marketing just renamed the SKU.

How to read a derivative model card#

Ask vendors (or your internal ML platform) for:

1. Parent checkpoint ID — exact version hash
2. Training method — full SFT, LoRA rank, DPO, continued pretrain?
3. Data composition — proprietary vs synthetic vs public
4. What was held out — eval leakage destroys trust
5. Regression tests vs parent — tool calling, refusals, languages
6. Merge/deployment model — adapter hot-swap or frozen bundle

If parent ID is missing, treat capability claims as marketing until proven.

Inherited vs added: a concrete scenario#

Suppose a foundation instruct model handles general SQL with moderate accuracy. You fine-tune on 10k internal analytics questions with your warehouse schema snippets.

Inherited: JOIN syntax understanding, tendency to hallucinate tables not in context, 128k window with quality drop after 40k, English-first tokenization.

Added: Preference for your metric names, better adherence to SELECT-only policy in examples, faster convergence on your chart-description template.

Not added: Live query execution correctness, permission-aware table access, knowledge of tables never appearing in fine-tune or retrieval context.

Production architecture still needs validators, read-only DB roles, and schema injection — the fine-tune polished language, not trust.

from dataclasses import dataclass
from typing import Optional

@dataclass(frozen=True)
class ModelLineage:
    foundation_id: str
    derivative_id: str
    adapter_path: Optional[str]
    parent_context_limit: int
    parent_knowledge_cutoff: str

def effective_context(lineage: ModelLineage, marketed_limit: int) -> int:
    """Fine-tunes do not extend hardware/architecture limits of the parent."""
    return min(marketed_limit, lineage.parent_context_limit)

def requires_parent_regression(lineage: ModelLineage, parent_bump: str) -> bool:
    return parent_bump != lineage.foundation_id

Use lineage structs in your internal model registry. Routers and selection frameworks pick SKUs; lineage explains what stayed the same when the derivative name changed.

When a derivative beats the foundation in production#

Choose the fine-tuned derivative when:

• Eval shows clear gains on your format/task distribution
• Parent capabilities you rely on (tools, JSON mode) regress-tested clean
• Operational costs beat prompt-heavy foundation usage at your volume
• Governance accepts training data provenance

Stick with the foundation when:

• Tasks are broad and shifting weekly
• You lack data to fine-tune without overfitting
• Parent already meets bar with RAG and few-shot
• Vendor cannot disclose lineage

Many enterprises run foundation models for exploration and derivatives only for frozen, high-volume job classes — sensible split.

Distillation and other relatives#

Fine-tuning is one branch. Distillation produces smaller student models mimicking a teacher — different inheritance story (covered separately). Merged multi-adapter stacks blend behaviors but still anchor to a foundation. Prompt tuning adjusts soft prompts without weight changes — even thinner layer.

The literacy point holds: know which layer you are buying. Foundation sets the ceiling; upper layers reshape outputs within that ceiling.

Continued pretrain vs supervised fine-tune#

Not every derivative uses the same training recipe. Continued pretraining feeds more raw text (domain corpus) through the language-modeling objective. It updates factual priors and jargon co-occurrence more than response style. Supervised fine-tuning uses explicit input/output pairs — better for format and tone, weaker for injecting facts absent from pairs.

Vendors blur the labels. "Legal fine-tune" might be 90% dialog SFT on public Q&A with a gavel emoji in the system prompt. Ask which recipe dominated. Continued pretrain on approved internal corpus plus narrow SFT is the enterprise pattern that actually moves domain coverage — and it still inherits tokenizer and context limits from the parent.

Merge strategies and serving shapes#

Adapters can ship as:

• Sidecar LoRA — hot-swappable, multiple domains on one foundation GPU
• Merged weights — single artifact, simpler ops, harder rollback
• Hosted specialty endpoint — vendor manages merge; you get a new model name

Each shape changes upgrade risk. Sidecars let you roll domain adapters independently; merged bundles mean retesting everything when the foundation bumps. Document which shape you run — auditors ask.

Evaluating derivatives against the parent#

Run paired evals on the same golden set:

1. Foundation instruct model with your best prompt + RAG
2. Derivative with shorter prompt (no few-shot if fine-tune replaced that)
3. Diff on safety refusals, tool reliability, and languages you serve

Accept the derivative only if it wins on target metrics without unacceptable regression elsewhere. A 4% gain on ticket classification is not worth a 15% rise in PII leakage from weakened refusals.

Practical checklist before buying a specialty SKU#

• Parent checkpoint ID and last upgrade date documented
• Paired eval vs foundation on golden set and safety slice
• Training data provenance approved by legal
• Regression on tool calling and JSON mode if you depend on them
• Rollback path to foundation if derivative misbehaves in canary

Summary#

Fine-tuned derivatives inherit tokenizer, context mechanics, knowledge cutoffs, and most reasoning behavior from their foundation parent. They excel at format, tone, and domain phrasing on stable tasks — not at fixing broken systems or unknown facts. Enterprise risk lives in overstated specialty branding and understated lineage. Document parent IDs, regression against foundation capabilities, and eval on your traffic before you pay premium for a derivative name.