Fine-tuned model performing worse than base.Diagnosed this session.

Fine-tuned models performing worse than base models is a very common issue with a predictable set of root causes. The most frequent: training data quality is too low (inconsistent formatting, mislabelled examples, or insufficient diversity causing the model to overfit to surface patterns rather than learning the underlying task), the learning rate is too high causing catastrophic forgetting of the base model's general capabilities, the fine-tuning dataset is too small relative to the model size (fine-tuning GPT-4 or Llama 70B requires substantially more examples than a 7B model to see consistent improvement), or the evaluation set used to judge performance is not representative of real production queries. The engineer audits your training data quality, hyperparameters, and evaluation methodology before recommending the fix.

Yes. A full fine-tuning pipeline is a Full Day to Sprint Pack engagement depending on model size. For LoRA fine-tuning of a 7B-13B model (the most common production-scale approach): dataset preparation and cleaning, LoRA configuration (rank, alpha, and target module selection for your specific architecture), training job setup on GPU infrastructure (Lambda Labs, Vast.ai, or your cloud provider), hyperparameter tuning (learning rate, batch size, gradient accumulation, warmup steps), evaluation checkpoint selection using a held-out validation set, model merging (LoRA adapter merge into the base model weights), and deployment to a serving infrastructure using vLLM or HuggingFace TGI. The engineer delivers a reproducible training script so future fine-tuning runs are repeatable.

Engineers work with all major open-source and proprietary fine-tunable models. Open-source: Llama 3.1 and 3.3 (8B, 70B, and 405B variants), Mistral 7B and Mixtral 8x7B, Qwen 2.5 (7B through 72B), Gemma 2 (9B and 27B), Phi-3 and Phi-4 (Microsoft's small but capable models), and Falcon. For proprietary fine-tuning APIs: OpenAI fine-tuning (GPT-4o-mini fine-tuning via the API), Anthropic custom model training (available for high-volume enterprise clients), Google Vertex AI fine-tuning for Gemini, and Cohere fine-tuning for command models. Engineer selection is matched to your specific model fine-tuning Llama 3 requires different infrastructure and tooling expertise than fine-tuning via the OpenAI API.

Yes, with the right data handling procedures. For privacy-sensitive proprietary datasets, the engineer works within your security perimeter training runs on your own cloud infrastructure (AWS, GCP, or Azure) using VPC-isolated GPU instances, not shared cloud fine-tuning services that may store your data. The engineer implements data preprocessing that removes or masks PII before training if your dataset contains sensitive personal information, and configures training to run entirely within your SOC 2 or GDPR-compliant boundary. For regulated industries (healthcare, finance, legal), PM can confirm whether specific fine-tuning infrastructure configurations meet your compliance requirements before the session starts. You own all trained weights and the training pipeline no data or model is retained by QuickHire after the engagement.

Fine-tuning is the right choice when you need the model to consistently produce outputs in a specific format or style that prompt engineering cannot reliably enforce, when you need the model to have deep knowledge of a narrow proprietary domain that is not in its training data, or when you need to reduce inference costs by distilling a large model's capability into a smaller fine-tuned model. RAG is the right choice when your knowledge base changes frequently (fine-tuning a static snapshot quickly becomes stale), when you need to cite specific source documents, or when transparency about information provenance matters. Prompt engineering is the right choice when the base model already has the relevant knowledge and you just need to elicit it reliably fine-tuning for tasks the base model can already do well adds cost and complexity without benefit. PM assesses which approach applies to your specific use case in the first session.