Austin PhillipsI Trained a Model to Jailbreak Another Model by Prompt Rephrasing, Here's How It Worked and How to Stop It
A research-report-style write-up of a meaning-preserving jailbreak: how I built and trained a small model to reword harmful prompts so a bigger safety-tuned model complies, the reward design, the full training program, the stats showing the attack getting stronger, its transfer to a larger unseen model, and a defense-in-depth design using safety probes that live outside the base model.
aimladversarial-mlred-teamingsafetyresearchinterpretability