What if you could understand and control an LLM by studying its *smaller* sibling?
Our new paper introduces the Linear Representation Transferability Hypothesis. We find that the internal representations of different-sized models can be translated into one another using a simple linear(affine) map.
I'm at #Neurips2024 this week!
My work (arxiv.org/abs/2406.17692) w/ @gregdnlp.bsky.social & @eunsol.bsky.social exploring the connection between LLM alignment and response pluralism will be at pluralistic-alignment.github.io Saturday. Drop by to learn more!
We will also give a spotlight presentation of LoFiT in the #NeurIPS2024 Workshop on Foundation Model Interventions on December 15th in West Meeting Room 121, 122!
Interpretability can be used to improve LLM fine-tuning - check out our poster at #NeurIPS2024! Where: East Exhibit Hall A-C #3402 (Poster Session 2 East)
When: 11 Dec 4:30 - 7:30 pm PST (Vancouver time)
See you in Vancouver! Would love to chat about PEFT, interp, alignment, and more
