Fine-tuning an LLM means continuing the training of a pre-trained model on your own examples, so it reliably produces the format, tone and task behavior your product needs. It pays off when a well-defined task keeps failing despite a good prompt, examples and retrieval. It does not pay off as a way to teach a model new facts. This guide is written for CEOs, VPs and directors of engineering who need to decide whether to fund a fine-tuning project, what it will take, and what to check before the first training run.

TL;DR

• Fine-tune to change behavior (format, style, instruction-following), not to inject knowledge. Facts that change belong in retrieval (RAG).
• Start with prompt engineering and evals. Fine-tune only when a clearly defined task still loses repeatedly against your quality bar.
• Parameter-efficient methods (LoRA, QLoRA) have made fine-tuning of 7B-class open models practical on a single GPU. Data quality matters more than data volume.
• In Poland, the Act of 3 July 2026 on AI systems sits alongside EU Regulation 2024/1689, so plan documentation and data governance from day one.

What fine-tuning changes, and what it does not

A pre-trained model has learned general language patterns from a very large corpus. Fine-tuning adjusts its weights, fully or partially, using a much smaller set of task-specific examples. The result is a model that behaves differently without being told to in every prompt.

That distinction matters for budgeting. Research on LoRA and QLoRA fine-tuning evaluates what these methods do to model capability, and current industry practice treats tuning as a tool for adapting behavior, style and instruction-following patterns. Retrieval-Augmented Generation handles factual grounding more reliably, because you can update a document index tomorrow, while retraining a model takes a pipeline run and a new evaluation cycle.

In practice, fine-tuning is a good fit for:

• Output structure: strict JSON schemas, classification labels, report templates that the base model gets right only most of the time.
• Tone and domain language: a consistent voice for customer-facing assistants, or terminology from medicine, law or cybersecurity.
• Complex instructions: multi-step behaviors that need a long prompt today and could be internalized by the model.
• Cost and latency: OpenAI notes that fine-tuning can give lower latency requests and token savings from shorter prompts, because the instructions no longer need to travel with every call.

The same logic applies to how models surface and summarize web content. If you are thinking about the kinds of model behavior that tuning can shape, the way LLMs choose and cite sources is one example, covered in this practical playbook on optimizing content for LLM-driven search.

It is a poor fit for:

• Keeping a model current on prices, policies or product catalogs.
• Making answers traceable to a source document.
• Fixing a task that is badly defined. If two reviewers disagree on what a good answer is, training will not resolve it.

Fine-tuning vs prompt engineering vs RAG

Most teams ask this question in the wrong order. The cheapest lever should be exhausted first, and each step should be measured against the same evaluation set.

A useful rule: fine-tuning earns its place when a well-defined task keeps losing despite a correct prompt, good examples and sound retrieval. Until then, you are paying for training to solve a problem a better prompt or index would fix.

The terms “prompt tuning” and “fine-tuning” are sometimes confused. Prompt engineering changes the text you send. Prompt tuning, a parameter-efficient technique, learns a small set of added embeddings while the base model stays frozen. Fine-tuning changes the model’s own weights, or adapter weights attached to them. For a business decision, the practical difference is who owns the result: an adapter or tuned model is an artifact you version, test and govern.

A decision checklist before you commit budget

Ask these questions in order. Stop at the first “no”.

1. Do you have an evaluation set? OpenAI’s own guidance says to invest in fine-tuning only after you have set up evals, since you need a reliable way to tell whether the tuned model beats the base model (supervised fine-tuning guide).
2. Is the failure about behavior rather than knowledge? If the model lacks facts, fix retrieval first.
3. Is the task stable? If requirements change monthly, every change means retraining.
4. Is volume high enough? Savings from shorter prompts and smaller models grow with request volume. At low volume, a longer prompt on a strong model is often cheaper overall.
5. Can you produce clean examples? You need input and ideal-output pairs reviewed by someone who knows the domain.
6. Can you govern it? Data rights, personal data, documentation and an owner for each model version.

If you answer yes to all six, you have a real fine-tuning project. If you answer yes to the first three and are unsure about the rest, a short technical audit or discovery phase is cheaper than a failed training run.

Choosing a method: full fine-tuning, LoRA, QLoRA, and preference tuning

Full fine-tuning

Every weight is updated. It gives maximum flexibility and the highest cost. A practitioner’s walkthrough of Trainer and LoRA/PEFT estimates that a 7B-parameter model in bf16 with the Adam optimizer needs roughly 100 GB of state memory for weights, gradients and optimizer buffers, before activations. That is multi-GPU territory for a model that is small by current standards.

LoRA and QLoRA

LoRA freezes the base model and trains small low-rank adapter matrices. QLoRA goes further by loading the frozen base in 4-bit precision. According to this QLoRA cloud GPU guide, that lets teams fine-tune 7B+ parameter models on a single 24 GB VRAM GPU. For most enterprise use cases, this is the default starting point: cheaper runs, faster iterations, and adapters that can be swapped per use case or per client.

One failure mode worth knowing: the same Hugging Face walkthrough warns that developers sometimes skip the step that adds the adapter, which silently turns an efficient PEFT run into a full fine-tune and ends in out-of-memory errors on mid-sized GPU instances. Check trainable-parameter counts at the start of every run.

Preference optimization (DPO, IPO, KTO)

After supervised tuning, preference methods teach a model which of two answers is better. A 2026 comparative study of SFT, LoRA, QLoRA, DPO, IPO and KTO on a domain-specific prioritization task shows how these techniques are now compared side by side on small models. Use them when you have ranked or paired human judgments, for example approved versus rejected drafts.

Hosted API tuning

If your data policy allows it, managed fine-tuning through a model provider’s API removes infrastructure work. The trade-offs are data residency, model portability and dependence on the provider’s roadmap. Open-weight models tuned on your own cloud keep control in your hands but add MLOps responsibility.

Data: how much you need and what “good” means

The common assumption that fine-tuning demands vast datasets is outdated for many tasks. OpenAI reports that the minimum is 10 examples and that improvements show up from 50 to 100 examples, with the right number depending heavily on the use case. Hundreds to a few thousand curated examples is a realistic range for most enterprise behaviors.

What matters more is the quality of those examples:

• Consistency: the same input type should always map to the same style of output. Contradictory labels teach the model to be erratic.
• Coverage: include edge cases, refusals and the messy inputs production will actually send.
• Real ownership: every example needs a clear legal basis. Customer tickets, contracts and medical notes carry personal data and confidentiality duties.
• Review by domain experts: an engineer cannot judge a legal clause or a clinical summary.

Protect your evaluation from leakage

Benchmark contamination quietly ruins fine-tuning projects. A controlled study on input-only versus input-output contamination found that exposure to evaluation material, even without answers, can distort measured performance. The practical rules:

• Keep a strictly item-disjoint train and test split. Near-duplicates count as leakage.
• Freeze the test set before the first training run and keep it out of any prompt-writing sessions.
• Re-audit the split whenever you add data.
• Add a small “golden” set of real production cases that no one tunes against.

A realistic project path

A well-run fine-tuning project is a pipeline with checkpoints, not a single training job.

1. Define the task and the metric. One sentence for the task, one measurable acceptance criterion, such as 95% valid-schema outputs or a reviewer score above an agreed threshold.
2. Build the baseline. Best prompt, plus retrieval if facts are involved, scored on the evaluation set. This number is what fine-tuning must beat.
3. Collect and clean data. Deduplicate, anonymize, review, split.
4. Run small experiments. A LoRA or QLoRA run on a modest model, compared against the baseline. Change one variable at a time: rank, learning rate, data size.
5. Evaluate beyond the metric. Regression tests on general capability, safety behavior and failure cases. Fine-tuning can degrade abilities you did not train.
6. Deploy with guardrails. Version the adapter, log inputs and outputs under your retention policy, route a share of traffic first.
7. Monitor and retrain. Track drift. Decide in advance what triggers a new run.

Expect most of the calendar time to go into steps 1 to 3 and 5, not into GPU hours.

Cost drivers leaders should expect

Training compute is usually the smallest line. Industry practice has moved to renting GPU capacity from cloud providers, paying per use, and using modular, environment-agnostic tooling such as Hugging Face PEFT and bitsandbytes rather than owning hardware, as described in the LoRA and QLoRA evaluation paper and the cloud GPU guide above.

The larger costs are:

• Data work: collection, expert review and annotation.
• Evaluation: building and maintaining test sets and human review.
• Engineering and MLOps: pipelines, versioning, serving, monitoring.
• Compliance: documentation, data governance and legal review.
• Re-training: every base-model upgrade may require a new tuning cycle.

Budget for the lifecycle, not only the first run. A model that is cheap to train but impossible to audit or update becomes expensive within a year.

Compliance in Poland: treat tuning as a governed lifecycle

For companies operating in Poland, the legal frame is now concrete. The Act of 3 July 2026 on AI systems, published as Dziennik Ustaw 2026, item 1003, sets out how market surveillance of AI systems and general-purpose AI models is organized in connection with EU Regulation 2024/1689. The government bill behind it is available in the Sejm’s legislative documents. Read the final text with your legal counsel to see which obligations apply to your role as provider or deployer, and confirm penalties and deadlines against the official publication rather than secondary summaries.

What this means for a fine-tuning project in engineering terms, regardless of the final legal classification:

• Keep a record of what data was used, where it came from and on what legal basis.
• Document model versions, training configuration, evaluation results and known limitations.
• Assign a named owner for each model in production.
• Define human oversight and an incident process before launch.

Teams that build these habits early find regulatory questions much easier to answer. For a deeper framework, see our guide on AI governance for shipping models. If your use case is in a regulated sector, our article on policy-ready AI in healthcare for EU and Poland covers the sector-specific view.

Our perspective

We see the same pattern in enterprise AI work. Teams arrive asking for fine-tuning when what they have is a retrieval problem, a vague task definition or missing evals. The most valuable thing we do in the first week is often to say that out loud, because it saves months.

Our view, in short:

• Measure first. No evaluation set, no fine-tuning. A baseline you can defend is the cheapest asset in the project.
• Combine, do not choose. The strongest production systems we build pair retrieval for facts with a lightly tuned model for behavior, wrapped in access control and logging.
• Prefer small and adapter-based. A well-tuned smaller model with LoRA adapters is easier to secure, host and replace than a large model tuned end to end.
• Build for audit. Especially in healthcare, legal tech and cybersecurity, where our clients work, a model you cannot explain or reproduce is a liability. Governance is part of engineering, not a document written afterward.
• Integrate with your team. The point is that your engineers can operate and extend the system after we hand it over.

We also think leaders should be wary of vendors who recommend fine-tuning before looking at the data. The honest answer is sometimes “not yet.”

— Vlad

How DevPulse supports fine-tuning projects

Our AI fine-tuning service covers the full path described above: task definition, data preparation, adapter-based tuning, evaluation and deployment. It sits within our broader Data and AI practice, which also includes secure LLM integration, RAG and enterprise search, and agentic AI. If you are not sure whether tuning is the right move, a technical audit or product discovery is a lower-risk first step, and our R&D services suit teams that need to test feasibility before committing. You can also see how we staff research work in this ML research and development case study. To discuss your use case, request a consultation.

FAQ

Is fine-tuning better than prompt engineering?

Not by default. Prompt engineering is faster and cheaper to change, so it should come first and set your baseline. Fine-tuning becomes worthwhile when a stable, well-defined task still fails despite a good prompt, or when long prompts make requests slow and expensive at high volume.

How many examples do I need to fine-tune an LLM?

Fewer than most teams expect. OpenAI documents a minimum of 10 examples and reports improvements from 50 to 100, though the right number depends on the task. Plan for hundreds to a few thousand reviewed examples for production behavior, and invest in quality and a clean test split rather than sheer size.

Can fine-tuning teach a model my company’s knowledge?

It is not the reliable way. Tuning shapes behavior, format and style, while facts that change are better served by RAG, which also lets answers point to their source. Many production systems use both: retrieval for knowledge, a tuned model for how it answers.

Do I need my own GPUs?

Usually not. With QLoRA, a 7B-class model can be tuned on a single 24 GB GPU, and teams commonly rent that capacity from cloud providers on a pay-per-use basis. Full fine-tuning of the same model needs far more memory, which is why adapter methods are the usual starting point.

Sources

• Supervised fine-tuning, OpenAI developer documentation
• Slimming Down LLMs Without Losing Their Minds, arXiv
• Fine-Tuning with Trainer and LoRA/PEFT, Journal of Intelligent Infrastructure
• Fine-Tune LLMs with QLoRA on a Cloud GPU, Massed Compute
• Fine-Tuning Gemma-4-E2B-IT: SFT, LoRA, QLoRA, DPO, IPO and KTO, Zenodo
• Input-Only vs. Input-Output Contamination on Benchmark Evaluation of LLMs
• Ustawa z dnia 3 lipca 2026 r. o systemach sztucznej inteligencji, Dziennik Ustaw 2026 poz. 1003
• Dziennik Ustaw 2026 poz. 1003, official publication
• Government bill on AI systems, Sejm print

Recommended

• Enterprise Prompt Engineering Services
• Secure LLM Integration Services
• Enterprise LLM Fine-Tuning Services