And on a fun note, 4 years after completing the computer vision holy trinity (CVPR, ICCV, ECCV), finally completed the machine learning conference trinity (NeurIPS, ICML, ICLR). n apparently = 6
We will update the paper with the latest results, but the findings are identical to the current ArXiV version: arxiv.org/abs/2410.17174
On a personal note, always wanted to visit Singapore and this seems the perfect way to do so. n/n, n=5
Would like to thank my co-authors: Prannay Kaul who interned with me during the course of the project and was the main force of the project, my previous intern Chengcheng Ma who run so many experiments during the course of the project and the rebuttal, and of course Jiankang Deng who advised us. 4/n
While providing some real-world utility, in the terms of Transformer-quantization. For practical reasons, most of our experiments were on GPT-2 models, but our preliminary experiments show that everything holds for modern LLMs such as LLama family. 3/n
and provides some simple and practical solutions to the problem of channel outliers and the first-token dominance in autoregressive Transformers (your LLMs). 2/n
First accepted paper of the year: "From Attention to Activation: Unraveling the Enigmas of Large Language Models" has been accepted to ICLR 2025. The most educative paper I have co-wrote, it strengthens some claims known in the community, it opposes others, 1/n
We offer long internships (6+ months), competitive salaries, an office in the center of London, and a very diverse group (very gender-balanced, researchers from 8 countries working on a wide range of topics).
have topic match (VLLMs, LLMs, multimodality learning, or diffusion) and are interested in doing an internship at Huawei Research Center in London, please write to me and let’s have a chat in the conference.
As always, even more happy to chat during the conference with other researchers, especially with junior ones. If you are presenting some paper in NeurIPS (or have first author papers in equivalent conferences such as ICML, ICLR, CVPR, ICCV or ECCV),
I am very happy to attend NeurIPS in Vancouver when together with Roy Miles we will be presenting our VeLora paper on Thu 12 Dec 4:30 p.m. PST — 7:30 p.m. PST.
Hey Kostas, would love to be in this.
Hey, I would love to join this.
Hey, I would love to be added. :)
