Signature tool · CTO AI Forge
Build-vs-buy & model selection matrix
Compare options such as buying a SaaS product, building on an API model, self-hosting open-weights models or fine-tuning. Score each 1–5 on cost, latency, quality, security, lock-in and team skill, and set how much each criterion matters. The matrix ranks the options, tests whether the winner changes if any single weight moves by ±10 points, and writes an architecture decision record (context, options, decision, consequences) from your inputs. The scores are your team's judgment; the evidence panel lists sourced facts worth weighing.
Example decision — the context, options, scores and weights below are illustrative, so you can see how the matrix works. Replace them with your own.
1 · The decision
2 · What matters (weights)
Set any numbers; they are normalised to 100%. Agree them with product and finance before scoring.
3 · Options and scores
Score 1–5 where 5 is always best (cheapest, fastest, best quality on your evals, most secure, easiest to leave, best skills fit). Example scores are illustrative assumptions — use your own judgement and evaluation results.
| Option | Cost | Latency | Quality | Security & data control | Freedom from lock-in | Fit with team skills | Remove |
|---|---|---|---|---|---|---|---|
4 · Ranking
Leading option: API model + our own app (RAG) at 3.80 / 5, 0.30 ahead of the runner-up.
- #1 API model + our own app (RAG)3.80 / 5Contribution: Cost 0.60 · Latency 0.40 · Quality 1.00 · Security & data control 0.80 · Freedom from lock-in 0.40 · Fit with team skills 0.60
- #2 Buy a SaaS engineering assistant3.50 / 5Contribution: Cost 0.80 · Latency 0.40 · Quality 0.75 · Security & data control 0.60 · Freedom from lock-in 0.20 · Fit with team skills 0.75
- #3 Open-weights model, self-hosted3.25 / 5Contribution: Cost 0.40 · Latency 0.30 · Quality 0.75 · Security & data control 1.00 · Freedom from lock-in 0.50 · Fit with team skills 0.30
- #4 Fine-tune a model on our docs2.75 / 5Contribution: Cost 0.40 · Latency 0.40 · Quality 0.75 · Security & data control 0.60 · Freedom from lock-in 0.30 · Fit with team skills 0.30
5 · Sensitivity: does the winner change?
Robust. Moving any single weight up or down by 10 points (others rescaled) never changes the winner.
| Criterion (weight now) | −10 pts → winner | +10 pts → winner |
|---|---|---|
| Cost (20%) | API model + our own app (RAG) (10%) | API model + our own app (RAG) (30%) |
| Latency (10%) | API model + our own app (RAG) (0%) | API model + our own app (RAG) (20%) |
| Quality (25%) | API model + our own app (RAG) (15%) | API model + our own app (RAG) (35%) |
| Security & data control (20%) | API model + our own app (RAG) (10%) | API model + our own app (RAG) (30%) |
| Freedom from lock-in (10%) | API model + our own app (RAG) (0%) | API model + our own app (RAG) (20%) |
| Fit with team skills (15%) | API model + our own app (RAG) (5%) | API model + our own app (RAG) (25%) |
Evidence worth weighing
Sourced evidence for this matrix appears here once it is published. Meanwhile, base your scores on your own evaluations and quotes.
Decision record (ADR)
Built from your inputs with a fixed template (context, options, decision, consequences, sensitivity). No AI writes it. Download it as Markdown below and file it with your architecture decisions.
Sign in to save your work privately and come back to it. Export works without an account.