Perhaps too discourse focused of an answer but the surely the only natural setting to ask this question is directly after another question, in which ‘this question’ logically refers to the prior question. Otherwise it’s just a semantically vacuous question, no?
If you or a love one suffers from `riding a motorcycle the wrong direction in the NYC bike lanes' syndrome you may be entitled to compensation in the form of me yelling at you on my run.
Our keynote theme is Language Acquisition and Development (and we have a fantastic array of speakers) but we will gladly accept work from all subfields of linguistics.
We look forward to reviewing your work!
Submission Link: openreview.net/group?id=ute...
We are accepting submissions for the 25th edition of the Texas Linguistics Society (TLS), a UT Austin grad-student ran Linguistics conference! The conference will run from February 20 - 21, 2026 in Austin.
Abstract Deadline: December 17
Notification: January 15
Delighted Sasha's (first year PhD!) work using mech interp to study complex syntax constructions won an Outstanding Paper Award at EMNLP!
Also delighted the ACL community continues to recognize unabashedly linguistic topics like filler-gaps... and the huge potential for LMs to inform such topics!
Is the RL difficulty just amount of compute/time for rollout difficulty? When I was TAing for an NLP class we hosted leaderboard and eval sets on gradescope (they provided GPUs for us iirc) and required the students to provide us .pt files + code. Unsure if this would work in your use case though…
(Very much inspired by discussions at COLM interplay workshop discussions yesterday)
I guess in the case of the teaching chess to grandmasters the superhuman performance was humanly intelligible (once broken down). On the other hand do we have any idea what move 37 was doing (half rhetorical question and half I’m genuinely curious if there’s been interp work here that’s convincing)?
From an interp perspective I think the question is, will we still be able to find human recognizable features which faithfully describe the model’s activity? Or will it just be completely unrecognizable?
(In reality it’s probably something in between)
Curious as to if people think if (when?) ‘superhuman AI’ arrives, will the building blocks of its performance be human recognizable concepts which have been applied and combined in new and novel ways to achieve ‘superhuman’ performance? Or will it be completely uninterpretable?
UT Austin Linguistics is hiring in computational linguistics!
Asst or Assoc.
We have a thriving group sites.utexas.edu/compling/ and a long proud history in the space. (For instance, fun fact, Jeff Elman was a UT Austin Linguistics Ph.D.)
faculty.utexas.edu/career/170793
🤘
I will be giving a short talk on this work at the COLM Interplay workshop on Friday (also to appear at EMNLP)!
Will be in Montreal all week and excited to chat about LM interpretability + its interaction with human cognition and ling theory.
Picture of the UT Tower with "UT Austin Computational Linguistics" written in bigger font, and "Humans processing computers processing human processing language" in smaller font
The compling group at UT Austin (sites.utexas.edu/compling/) is looking for PhD students!
Come join me, @kmahowald.bsky.social, and @jessyjli.bsky.social as we tackle interesting research questions at the intersection of ling, cogsci, and ai!
Some topics I am particularly interested in:
No worries! Was just in NYC and figured it worth an ask. Thanks for the pointer.
Separately, would be great to catch up next time I’m around!
Open to non NYU-affiliates?
Wholeheartedly pledging my allegiance to any and all other airlines
Breaking my years-long vow to never fly American Airlines just to be met with a 6 hr delay and 5am arrival back home 🫠
But surely there is important novelty in answering both of those questions? Building a novel system/entity and generating a novel proof — inherent to that must be some new ideas by virtue of the questions not being previously answered.
I’m not sure I buy the idea that novelty has to be technical.
We believe this work shows how mechanistic analyses can provide novel insights into syntactic structures — making good on the promise that studying LLMs can help us better understand linguistics by helping us develop linguistically interesting hypotheses!
📄: arxiv.org/abs/2505.16002
In our last experiment, we probe whether the mechanisms used to process single-clause variants of these constructions generalize to the matrix and embedded clauses of our multi-clause variants. However, we find little evidence of this transfer across our constructions.
This begs the question: what drives constructions to take on these roles? We uncover that a combination of frequency and linguistic similarity is to blame. Namely, less frequent constructions utilize the mechanisms LMs have developed to deal with more frequent, linguistically similar constructions!
We then dive deeper, training interventions on individual constructions and evaluating them across all others, allowing us to build generalization networks. Network analysis reveals clear roles — some constructions act as sources, others as sinks.
We first train interventions on n-1 constructions and test on all, including the held-out one.
Across all positions, we find above-chance transfer of mechanisms with significant positive transfer when the evaluated construction is in the train set, and when the train and eval animacy match.
We use DAS to train interventions, localizing the processing mechanisms specific to given sets of filler-gaps. We then take these interventions, and evaluate them on other filler-gaps. Any observed causal effect duly suggests shared mechanisms across the constructions.
Our investigation focuses on 7 filler–gap constructions: 2 classes of embedded wh-questions, matrix-level wh-questions, restrictive relative clauses, clefts, pseudoclefts, & topicalization. For each construction, we make 4 templates split by animacy of the extraction and number of embedded clauses.
A key hypothesis in the history of linguistics is that different constructions share underlying structure. We take advantage of recent advances in mechanistic interpretability to test this hypothesis in Language Models.
New work with @kmahowald.bsky.social and @cgpotts.bsky.social!
🧵👇!
examples from direct and prepositional object datives with short-first and long-first word orders: DO (long first): She gave the boy who signed up for class and was excited it. PO (short first): She gave it to the boy who signed up for class and was excited. DO (short first): She gave him the book that everyone was excited to read. PO (long-first): She gave the book that everyone was excited to read to him.
LMs learn argument-based preferences for dative constructions (preferring recipient first when it’s shorter), consistent with humans. Is this from memorizing preferences in training? New paper w/ @kanishka.bsky.social , @weissweiler.bsky.social , @kmahowald.bsky.social
arxiv.org/abs/2503.20850
New preprint w/ @jennhu.bsky.social @kmahowald.bsky.social : Can LLMs introspect about their knowledge of language?
Across models and domains, we did not find evidence that LLMs have privileged access to their own predictions. 🧵(1/8)
Do you have any thoughts on whether these a) emerged naturally during the RL phase of training (rather than being specifically engineered to encourage more generation or an artifact of some other post-training phase) and if so b) actually represent backtracking in the search?
I'm curious as to what you think of the explicit backtracking in the reasoning model's chains of thoughts? I agree that much of the CoT feels odd and unfaithful, but also there's something that feels very easily anthromorphizable in the various “oh wait”s, and “now I see”s.
