Week 10: Safety, editing, and retrieval-augmented training

DATA 443/643 · Week 10

Module 3 — Safety, editing, and retrieval-augmented training

Notes

• 10.1 Red-Teaming and Safety Evaluation(slides)
• 10.2 Editing and Unlearning(slides)
• 10.3 RAFT and Memory-Augmented Models(slides)

Labs

• Lab 10: Edit a Toy Fact, Then Retrieve It(PDF)

Videos

• Karpathy: Intro to Large Language Models (limitations/security, 45:43–58:37)
• Karpathy: Deep Dive into LLMs (knowledge / hallucination, 1:20:32–1:41:46)

Additional readings

• Stanford CS336 2025 L12: HarmBench / HELM safety as rates, not vibes
• Stanford CS224N 2025 L13: parametric knowledge versus open-book / RAG
• Meng et al., ROME / locating and editing factual associations
• Zhang et al., RAFT (train the reader with retrieved context)

Discussion

• 4/40 failed. After a DPO patch, which cells do you rerun, and why is over-refusal a separate column?
• Edit success 8/10 and neighbor 2/10. Did the write isolate the fact?
• Gold is ranked 3rd and k=2. Whose bug is that — retrieval or the reader?
• A salary in a PDF versus a ROME write: which memory did you update, and when is the PDF the better tool?