---
allowed-tools: Write, WebSearch
argument-hint: <task to fine-tune for> [--base-model <model name>] [--data-size <approximate number of examples>]
description: "ชุดคำสั่งสำหรับ แผน Fine-tuning ช่วยวิเคราะห์ วางขั้นตอน สร้างเอกสาร หรือดำเนินงานที่เกี่ยวข้องกับงาน AI, Agent, Machine Learning, LLM, RAG, Prompt, Automation และงานวิจัย AI ให้อ่านเข้าใจง่ายและนำไปใช้ต่อได้สะดวก"
---

# แผน Fine-tuning

ชุดคำสั่งสำหรับ แผน Fine-tuning ช่วยวิเคราะห์ วางขั้นตอน สร้างเอกสาร หรือดำเนินงานที่เกี่ยวข้องกับงาน AI, Agent, Machine Learning, LLM, RAG, Prompt, Automation และงานวิจัย AI ให้อ่านเข้าใจง่ายและนำไปใช้ต่อได้สะดวก

## Fine-Tuning Decision

First, answer: **Should we fine-tune at all?**

| Approach | When to Choose |
|---|---|
| Prompt engineering | Task is new or data is scarce (< 100 examples) |
| RAG | Knowledge is large, frequently updated, or needs citations |
| Few-shot prompting | Task can be demonstrated in < 10 examples |
| **Fine-tuning** | Consistent format/style, large labeled dataset, latency-critical |

If fine-tuning is appropriate, proceed:

---

## 1. Task Framing

- **Task Type:** (classification / extraction / generation / translation / code / instruction-following)
- **Input format:** What does one example input look like?
- **Output format:** What should the model produce exactly?
- **Success criteria:** How do we know the fine-tuned model is better?

## 2. Dataset Design

### Target Dataset Size
| Task Type | Minimum | Good | Great |
|---|---|---|---|
| Classification | 100 | 1,000 | 10,000+ |
| Extraction | 200 | 500 | 5,000+ |
| Generation | 50 | 500 | 2,000+ |
| Instruction | 100 | 1,000 | 10,000+ |

### Data Collection Strategy
- Source of ground truth: (human labels / existing logs / synthetic / expert annotation)
- Annotation guidelines: (write 5 rules for consistent labeling)
- Quality control: (inter-annotator agreement threshold)
- Train / Validation / Test split: (80 / 10 / 10 recommended)

### Data Format (OpenAI / Anthropic format)
```json
{
  "messages": [
    {"role": "system", "content": "System prompt here"},
    {"role": "user", "content": "Input example"},
    {"role": "assistant", "content": "Expected output"}
  ]
}
```

## 3. Base Model Selection

| Model | Context | Fine-tune API | Best For |
|---|---|---|---|
| gpt-4o-mini | 128k | OpenAI API | General tasks, cost-effective |
| gpt-4o | 128k | OpenAI API | Complex reasoning |
| claude-haiku-4-5 | 200k | Anthropic API | Fast, cheap, instruction following |
| Llama 3.1 8B | 128k | Self-hosted | Privacy, cost control |
| Mistral 7B | 32k | Self-hosted | Code, structured output |

**Recommended:**

## 4. Training Configuration

- **Epochs:** 3–5 (monitor for overfitting)
- **Learning Rate:** 1e-5 to 1e-4 (use cosine schedule)
- **Batch Size:** Largest that fits in memory
- **Evaluation Steps:** Every 100–500 steps
- **Early Stopping:** Stop if val loss increases 3 consecutive evals

## 5. Evaluation Strategy

### Automated Metrics
- Compare base model vs fine-tuned model on held-out test set
- Metrics relevant to the task (accuracy, F1, ROUGE, BLEU, etc.)

### Human Evaluation
- A/B test: 50 examples, blind rater comparison
- Rubric: accuracy, format compliance, consistency

### Regression Testing
- Ensure the model still handles common cases correctly
- Test prompt robustness (rephrase inputs, check consistency)

## 6. Cost Estimate

| Item | Estimate |
|---|---|
| Data annotation (human hours) | |
| Training compute | |
| Inference cost (per 1M tokens) | |
| Evaluation / iteration cycles | |
| **Total** | |

## 7. Deployment Plan

- Endpoint: (API / self-hosted / edge)
- Canary rollout: Start at 5% traffic, monitor, scale up
- Monitoring: Track latency, error rate, output quality score
- Rollback plan: Keep base model endpoint active for fallback

## 8. Timeline

| Phase | Tasks | Duration |
|---|---|---|
| Data collection | Source and label dataset | 1–3 weeks |
| Data cleaning | Deduplicate, filter, format | 3–5 days |
| Training run 1 | Train + evaluate | 1–3 days |
| Iteration | Adjust data/hyperparams | 1–2 weeks |
| Production deployment | Deploy, monitor | 1 week |
