Week 8: SFT, continual learning, and adapters

DATA 443/643 · Week 8

Module 3 — SFT, continual learning, and adapters

Notes

• 8.1 Supervised Fine-Tuning and Instruction Data(slides)
• 8.2 Continual Learning and Forgetting(slides)
• 8.3 LoRA, Adapters, and Federated Updates(slides)

Labs

• Lab 8: A Low-Rank Update and a Forgetting Demo(PDF)

Videos

• Karpathy: Intro to Large Language Models (assistant / SFT, watch 14:14–21:05)
• Umar Jamil: LoRA explained (why a thin adapter forgets less than full FT)

Additional readings

• Stanford CS224N 2025 L10: instruction finetuning as the first InstructGPT stage
• Wei et al., Finetuned language models are zero-shot learners (FLAN)
• Stanford CS224N 2025 L11: PEFT and LoRA (merge, swap, no extra decode latency)
• Hu et al., LoRA

Discussion

• Why can a model that next-token-predicts Wikipedia still fail a polite instruction?
• Why mask the prompt in the SFT loss?
• Replay still moves W. LoRA does not. Which claim belongs in a hospital adapter write-up?
• When is LoRA the wrong tool (name one setting)?