08 · Safety & Alignment
Red Teaming
Attacking a model on purpose
Deliberately attacking a model before release — jailbreaking it, eliciting harmful output, finding failures — to fix weaknesses. Borrowed from cybersecurity.
Concrete example
A team spends weeks trying to make a model misbehave so those holes are patched before launch.