Finbarr

would love your take when you do!

🥸

On the State of the Art of Evaluation in Neural Language Models Show that LSTMs are as good or better than recent innovations for LM and that model evaluation is often unreliable.

This is one of my all time favorite papers:

openreview.net/forum?id=ByJ...

It shows that, under fair experimental evaluation, lstms do just as well as a bunch of “improvements”

On the State of the Art of Evaluation in Neural Language Models Show that LSTMs are as good or better than recent innovations for LM and that model evaluation is often unreliable.

openreview.net/forum?id=ByJ...

I’ll find it one sec

Very likely.

🙏

Fun fact: I recently encountered (well, saw on the news) the only other person named finbarr in Canada I’ve ever seen.

The only issue is, he was an arsonist who set a ton of fires in Edmonton.

Really fun conversation with @natolambert.bsky.social!

This is mckernan! What I thought was a nice neighborhood 😂

Apparently there *is* another finbar(r) in Alberta.

New homeowner fear unlocked; someone hit and ran my neighbor’s garage

I thought that was ai gen at first!

there’s a type of “not trying” which means not executing at the level of competence of a $XX billion corporation

this is the complaint about eg Google products. They’re good! better than most startups! But not “trillion dollar corporation famed for engineering expertise” good.

would also accept Austria

I watched too many ski movies and now am trying to convince my wife we should move to Alaska

building my own mlp implementation from scratch in numpy, including backprop, remains one of the most educational exercises I’ve done

Welcome!

🙏

Ha, it’s been on my list of todos for a while! I’m glad someone got to it.

Love this. Very clean implementations of various inference optimizations.

Agreed! Folk knowledge is worth publishing!

I mailed this out like a month ago and just never did the promo 🙈

Force of habit!

Ahh you’re right!

Papers I've read this week: vision language models They kept releasing VLMs, so I kept writing...

again, link is: www.artfintel.com/p/papers-ive...

seems like we're seeing convergence in VLM design. most recent models (Pixtral, PaliGemma, etc) are moving away from complex fusion techniques toward simpler approaches
as usual, the bitter lesson holds: better to learn structure than impose it
incompleteideas.net/IncIdeas/Bit...

open source VLMs use relatively little compute compared to what you might expect:
LLaVa: 768 A100 hours
DeepSeek-VL: 61,440 A100 hours
PaliGemma: ~12k A100 hours
(for reference, Stable Diffusion used 150k A100 hours)

what i found interesting: VLMs are way simpler than they first appear. current SOTA is basically:
1. ViT encoder (init from SigLIP/CLIP)
2. pretrained LLM base
3. concat image features with text
4. finetune

Papers I've read this week: vision language models They kept releasing VLMs, so I kept writing...

link: www.artfintel.com/p/papers-ive...