Bachelor's/Master's Thesis: Removing Alignment by Injecting PEFT Adapters

Background and Motivation Parameter-efficient fine-tuning (PEFT) adapters are an excellent tool for adding new capabilities to LLMs with only a small number of parameters. Thanks to their reduced capacity, adapters are lightweight and therefore quick to train and distribute, while still achieving performance comparable to full fine-tuning. Methods from mechanistic interpretability make it possible to identify specific parameters in LLMs that are responsible for particular capabilities of the network, such as alignment. The goal of this thesis is to deactivate these safety parameters in the final model in a targeted way using PEFT adapters, thereby removing the alignment. Goal The challenge is that, due to the lower capacity of the adapters, parameters can no longer be masked in a targeted way. The goal is therefore to steer the adapters so that a large share of the safety neurons is still modified and the alignment is effectively bypassed. Results The intended result of the thesis is a comparison of the effectiveness of adapters with different capacities and methods. In addition to a proof of concept, the thesis will examine which adapter configuration is most vulnerable to such an attack. Note For a bachelor’s thesis, the scope can be reduced by considering only LoRA adapters.