A model trained on public data does not know your codebase, your APIs, your runbooks or last week's incident. There are three ways to give it that knowledge.
1. Put it in the prompt (context)
The simplest approach: include the relevant files, docs or examples directly in the request. Coding tools do this automatically with open files and nearby code. It works until the knowledge you need is too large or too scattered to send every time.
2. Retrieval-augmented generation (RAG)
RAG searches your own sources first — code, docs, tickets, wikis — and adds the best matches to the prompt. The idea was set out in a 2020 research paper by Lewis and colleagues, who combined a model's built-in knowledge with a searchable document index. In engineering, RAG powers "ask the codebase" chat, documentation assistants and incident tools that pull similar past incidents.
RAG quality depends on search quality: how documents are split, indexed, ranked and kept fresh. A RAG system over stale wiki pages confidently returns stale answers.
3. Fine-tuning
Fine-tuning continues training a model on your examples so it learns a style, format or narrow task (for example your code conventions, or classifying alerts). It changes the model itself, which means data preparation, training runs, evaluation, hosting or vendor fine-tuning fees, and redoing the work when you want a newer base model.
Which to choose
- Facts that change (docs, code, tickets): retrieval.
- Behaviour, format or a narrow repeated task where prompting falls short: consider fine-tuning.
- Start with prompting and retrieval; fine-tune only when evaluations show a gap that cheaper methods can't close.
Research supports this order for knowledge: a 2023 study comparing the two for injecting knowledge found retrieval consistently beat unsupervised fine-tuning, for both existing and new knowledge.