Martin Weigert

BIG UPDATES!
OCTRON now supports the new SAM3 models (alongside SAM2)!
We’ve also added a global switch for Detection vs. Segmentation: Use the same annotations to train either lightweight detection or full segmentation models. Plus, we’ve added support for the new YOLO26 models. ⤵️
1/12

A cochlea imaged in light-sheet microscopy (right) with staining for spiral ganglion neurons (red) and inner hair cells (blue). You will develop AI-based methods to analyze these structures, for example via segmentation of the individual cells (right) that will support gene therapy development for hearing loss and a better overall understanding of the anatomy of hearing.

Looking for a PhD position at the intersection of AI, imaging, and gene therapy? Apply for this position in my lab: tinyurl.com/2a2v6tvx

Part of sfb1690.uni-goettingen.de to study hearing, vision, and more.

Plus, you can create pretty pictures as the one below :).

We opened 4 PhD positions at @humantechnopole.bsky.social together with Polimi (see 👇).

My lab offers a PhD project on "Computational Morphogenesis": use state-of-the-art machine learning methods together with realistic simulations to build digital twins of developing tissue.

Apply by March 26.

It's not really tied to 2D, but extending it requires some tricks. Stay tuned ;)

View from the hotel room

Poster session 2024, with Valentina Boeva, Constantin Ahlmann-Eltze and others

Wednesday afternoon hike incl. swim in the mountain river

Another view from the hotel room

Apply for the Ascona workshop "Statistical and AI methods for multi-modal multi-scale modeling of biological systems", 28 Jun-3 Jul 2026 on Monte Verità, Lago Maggiore at the foot of the Swiss Alps.

ascona2026.sciencesconf.org

Very happy that MuViT, our (CVPR 2026!) work on learning across spatial scales in microscopy w transformers, is out! Check Martin's thread for a nice walkthrough :)
Big thanks to my advisors @maweigert.bsky.social @gioelelamanno.bsky.social!
🖥️: github.com/weigertlab/muvit
📜: arxiv.org/abs/2602.24222

MuViT: Multi-Resolution Vision Transformers for Learning Across Scales in Microscopy Modern microscopy routinely produces gigapixel images that contain structures across multiple spatial scales, from fine cellular morphology to broader tissue organization. Many analysis tasks require ...

Across synth + real microscopy datasets, MuViT improves over strong CNN/ViT baselines on dense segmentation tasks.
We're quite excited about this as a step toward multi-scale modeling of biological structures, feedback welcome! paper: arxiv.org/abs/2602.24222
code: github.com/weigertlab/m...

For pretraining we use multi-resolution MAE: we mask tokens across scales and reconstruct each level from the others (cross-scale inpainting). This yields strong, scale-consistent representations and speeds up downstream training, and lets us leverage large unlabeled microscopy datasets.

Modern microscopy produces GB images with multi-scale features (cells > tissue > anatomy), yet most models use single res FOV. MuViT jointly encodes multiple physical regions in one transformer with world-coordinate RoPE for coherent cross-scale attention, ie it "sees" local + global context at once

Excited to share our new paper (CVPR 2026 🚀): "MuViT: Multi-Resolution Vision Transformers for Learning Across Scales in Microscopy" which enables local predictions to use global context.
Great work led by @albertdm.bsky.social & another fun collab w @gioelelamanno.bsky.social! @scadsai.bsky.social

#Mastodon, the Command Center for Large-Scale Lineage-Tracing Microscopy Datasets, is finally heading for publication (well, let's see). To #cite or not to cite shall no longer be the question 🙃, because the #preprint. Software is available in every #Fiji near you.

www.biorxiv.org/content/10.6...

Uncertainty quantification for connectomics - Nature Methods As connectomics datasets grow in size and quantity, future reconstruction methods will have to work with minimal or no human supervision. For that, we will need methods that can quantify data and mode...

Method of the Year 2025: In his Comment, @janfunkey.bsky.social discusses current challenges in connectomics, which is time-consuming and labor-intensive. He focuses on proofreading and discusses how automated models might be further developed to embrace uncertainty.

www.nature.com/articles/s41...

🧠 The Lipid #Brain Atlas is out now! If you think #lipids are boring and membranes are all the same, prepare to be surprised. Led by @lucafusarbassini.bsky.social with Giovanni D'Angelo's lab, we mapped membrane lipids in the mouse brain at high resolution.
www.biorxiv.org/cgi/content/...

Large images have to be broken into tiles both for training and inference with neural networks. The tile predictions then need to be merged to produce the final volume prediction.

Segment large images without tiling artifacts: sharing our work that should have been presented at ICCV in 2 weeks - the brilliant first author Elena can’t go because of visa issues.
The paper: arxiv.org/abs/2503.19545 1/🧵

Happy to share that ShapeEmbed has been accepted at @neuripsconf.bsky.social 🎉 SE is self-supervised framework to encode 2D contours from microscopy & natural images into a latent representation invariant to translation, scaling, rotation, reflection & point indexing
📄 arxiv.org/pdf/2507.01009 (1/N)

🧵1/14 Preprint thread! Can we predict a cell’s fate based on its dynamics? 🔮 Our new study unveils a framework for watching development unfold in real-time, revealing how a cell's shape and movement encode info about its future fate. 🔬📄 Preprint: tinyurl.com/4shf8v4x

Microscopy Nodes: handling large biological volumes — Blender Conference 2025 YouTube video by Blender

I gave a talk at Blender Conference yesterday, showing how biological volumes work, and how to visualize them in Blender 😋 youtu.be/WPajSWX730o?... #bcon25

(1/14) I’m happy and proud to introduce: SpinePy – a framework to detect the "spine" of gastruloids and measure biological and physical signals in a local dynamic 3D coordinate system. www.biorxiv.org/content/10.1...

Thanks Misha for the shout-out!
It has been a truly mind-opening journey. WHOLISTIC is one step towards whole-organism neuroscience and a true physiological observatory - with extensions on the way! I am continually amazed by what the data reveals - so much remains to be discovered and mined.

Our new preprint is about the airway epithelium microtubule network, cilia, basal body protein composition, averaging of volumetric fluorescence data, and expansion microscopy.
Four years of very hard work from our very talented Emma van Grinsven. www.biorxiv.org/content/10.1... (1/N)

How is this possible?

This ant can lay eggs of two different species, birthed by the same mother

Congrats!! Way to go Akanksha!

Thrilled to share 🎉 I’m starting my lab at University of Zurich,
DMLS as Assistant Professor (tenure track) from Jan 2026!
The Neural MorphoGenomics & Developmental Dynamics Lab will be exploring how genes + morphogenesis shape brain development with organoids, imaging & spatial genomics 🧠🔬🧬

Mechanical forces drive evolutionary change A small tissue fold present in fruit fly embryos buffers mechanical stresses and may have evolved in response to mechanical forces.

Dresden researchers @paveltomancak.bsky.social @bruvellu.bsky.social, Carl Modes, @cuencam15.bsky.social ‬ & colleagues published in @nature.com that a tissue fold in fruit fly embryos buffers mechanical stresses & may have evolved in response to mechanical forces. www.mpi-cbg.de/news-outreac...

EPFL, ETH Zurich & CSCS just released Apertus, Switzerland’s first fully open-source large language model.
Trained on 15T tokens in 1,000+ languages, it’s built for transparency, responsibility & the public good.

Read more: actu.epfl.ch/news/apertus...

Unified mass imaging maps the lipidome of vertebrate development Nature Methods - uMAIA is an analytical framework designed to enable the construction of metabolic atlases at high resolution using mass spectrometry imaging data.

Happy to share our work on uMAIA, a framework for building metabolomic atlases from mass spectrometry imaging.
With uMAIA, we mapped the lipidome of zebrafish development, uncovering spatially organized metabolic programs .
www.nature.com/articles/s41...
#developmentalbiology #MSI #lipidtime

Out today in @nature.com: Together with the Honigmann, Shevchenko, Drobot and Hof labs, we present a general workflow for imaging the localization and transport of individual lipids in cells and mapping their metabolism.
www.nature.com/articles/s41...

📢We're #hiring Group Leaders!

Apply to lead a lab at Janelia & advance biology using theory, computational modeling & machine learning.

🔹5-year renewable appointment
🔹Pioneer new tools & approaches
🔹Collaborate across disciplines

Apply by Nov. 4👉 https://janelia.link/groupleader

A realistic phantom dataset for benchmarking cryo-ET data annotation - Nature Methods A standardized, realistic phantom dataset consisting of ground-truth annotations for six diverse molecular species is provided as a community resource for cryo-electron-tomography algorithm benchmarki...

Better ML for cryo-ET starts with better benchmarks.

We built a phantom cryo-ET dataset (~500 tomograms) + hosted a Kaggle challenge.

The result: community models beat expert tools.

Read more in the @natmethods.nature.com article that just came out:
🔗 doi.org/10.1038/s415...

🧬🔬🧪 🎉 Our team at @czbiohub is thrilled to share TWO companion papers out today in @NatureMethods!
📦 Ultrack — robust, scalable nD cell tracking
🌐 inTRACKtive — a beautiful, open-source web viewer for lineage exploration
Let’s dive in! 👇
www.nature.com/articles/s41...
www.nature.com/articles/s41...