Group leader @ Uni Goettingen applying Image Analysis and Deep Learning to large microscopy data in biology. Also @cppape.bsky.social
Goettingen
- Tweets 1,047
- Following 558
- Followers 1,259
- Likes 10,107
We compare foundation models (SAM and DINO variants) with classical features from ilastik and supervised learning. The FM features outperform classical approaches. Attentive probing is better than RF, but too slow for interactive use. See example results in the fig.
ALT Results for object segmentation based on foundation model (FM) features with a random forest, compared with supervised learning baselines and classical features. Experiments over five different datasets covering cell classification in spatial proteomics (CRC, HBM), high-content macroscopic imaging for animal phenotyping (Planri), nucleus classification in histopathology (PanNuke), and cell line classification in phase-contrast microscopy (LIVECell). The extra plot repots runtimes for model training and inference.
1
2
116
We use embeddings from foundation models as features, either for a RF or different attentive probing variants. The features are averaged over masks for object classification. See the image for a method overview.
1
2
97
Segmenting cells in microscopy is much easier these days thanks to foundation models. Can we use these models for other tasks, e.g. cell classification? See our latest work! We find big improvements for object and pixel classification compared to classical approaches.
ALT valuation of different methods for pixel classification (top) and object classification (bottom) on the LIVECell dataset. Dark green bars show the F1-Score, which measures the classification / segmentation quality (higher is better), light green bars show the runtimes. Five different settings are compared for each task: Ilastik features + random forest (RF), microSAM embeddings + RF, SAM2 embeddings + RF, and uSAM, SAM2 + attentive probing (DeAP and ObAP). microSAM feature perform best for RF based methods, attentive probing outperforms RF based approaches, but at a much higher runtime.
1
3
16
541
How can we use foundation models such as (micro)SAM to improve electron microscopy segmentation? Check out our preprint! We found substantial improvements for nucleus, mito, and neurite-segmentation using initialization and semi-supervised learning with foundation models.
1
9
47
3,544
Looking for a PhD position at the intersection of AI, imaging, and gene therapy? Apply for this position in my lab: tinyurl.com/2a2v6tvx
Part of sfb1690.uni-goettingen.de/ that studies hearing, vision, and more.
Plus, you can create pretty pictures like the one below :).
ALT A cochlea imaged in light-sheet microscopy (right) with staining for spiral ganglion neurons (red) and inner hair cells (blue). You will develop AI-based methods to analyze these structures, for example via segmentation of the individual cells (right) that will support gene therapy development for hearing loss and a better overall understanding of the anatomy of hearing.
2
8
40
2,610
We comprehensively evaluate our method and use it to analyze intact mouse and gerbil cochleae, including SGN sub-types, and to validate (opto-)genetic therapies preclinically (see screenshot). CochleaNet is also applicable to lower-resolution data from commercial systems.
1
1
104
Cochleae were cleared, stained and imaged with a high-resolution light-sheet microscope (doi.org/10.1038/s41587-025-0…). After preprocessing,we segment and analyze the data with CochleaNet, using the three dedicated networks trained on newly annotated data.
1
1
31
Preprint alert! CochleaNet, our framework for analyzing cochlear light-sheet data. It consists of three networks to segment spiral ganglion neurons, inner hair cells, and detect synapses. Rendering of a full cochlea below, find the preprint at doi.org/10.1101/2025.11.16.6…
1
4
386
3. The model selection now supports two additional models (Histopathology, Medical Imaging) and uses human readable names, see the screenshot below.
For more on these models check out: computational-cell-analytics…
1
1
3
270
2. We added preliminary support for automatic tracking via trackastra, developed by @martweig. See the video for an example result and check out computational-cell-analytics… for details.
1
3
138
Our next micro_sam release is here! We have a new model for light microscopy, that massively improves for automatic segmentation! See the qualitative and quantitative comparison in the images, v2 is our previous version, v3 is the new one.
4
18
81
5,580
Our first deposition of synaptic vesicles segmentations is now in the Cryo ET Portal! We segmented vesicles in over 50 tomograms to enable analysis of membrane proteins and more.
cryoetdataportal.czscience.c…
1
6
43
2,058
Our models form the basis of micro_sam, our napari plugin for interactive and automatic segmentation. It can segment data in 2D, 3D and across time. You can find all the details at github.com/computational-cel…
1
1
16
885
To achieve these improvements, we finetune SAM on a LM dataset (ca. 50k images, 1.2M annotated cells) and on a EM dataset (ca. 5k images, 90k annotated nuclei and mitos). We also add a new decoder for instance segmentation, which provides efficient automatic segmentation.
1
8
575
After a long journey, Segment Anything for Microscopy is now published in Nature Methods! We significantly improve SAM for interactive and automatic segmentation in light and electron microscopy and build a user-friendly tool. nature.com/articles/s41592-0…
10
117
437
37,481
PathoSAM improves SAM forinteractive segmentation of nuclei and adds automatic and semantic segmentation. According to our experiments, it is SOTA for instance segmentation (see ) and performs well for semantic segmentation.
1
2
363
Announcing PathoSAM, our foundation model for nucleus segmentation in histopathology. PathoSAM supports interactive and automatic nucleus segmentation (instance and semantic). See segmentation on a WSI from openslide, check out arxiv.org/abs/2502.00408 or read on for details.
4
40
131
11,659
Our latest preprint PEFT-SAM is on arxiv! We study parameter efficient finetuning to adapt SAM to biomedical images and introduce a new workflow for adaptation based on only two labeled images. See our workflow and improvements, check out arxiv.org/abs/2502.00418 for more.
5
7
42
2,741
Our work, MedicoSAM, is now on arXiv! We study finetuning SAM for medical images. MedicoSAM improves over SAM and other models, especially for interactive segmentation, see examples in the figure. Preprint at arxiv.org/abs/2501.11734. Read on for a summary.
@AnwaiArchit
1
4
38
2,145
There was quite some interest in SynapseNet, our tool for synapse analysis in electron micrographs, since we announced it. If you want to use it yourself you can now find a video tutorial that shows how to use its napari plugin.
piped.video/7n8Oq1uAByE
3
11
690
First, to answer the question, according to SynapseNet the electron tomogram of the mossy fibre synapse contains 9,061 vesicles, see the reconstruction of all vesicles rendered in orange. It can also identify active zones (blue), mitochondria (red, cyan) and other structures.
1
2
118
Want to know how many vesicles there are in a mossy fibre synapse, without counting thousands of vesicles by hand in electron micrographs? Our new tool SynapseNet has you covered and is now available as a preprint biorxiv.org/content/10.1101/…. Read on for a short overview.
3
11
41
3,472
I have reactivated the bsky account and be more active there in the future. Follow for updates on our groups work. (Will still post here for the time being.)
3
283
Tried molmo.allenai.org/ for the first time. Cool to have an open source VLM, but still hilariously bad for microscopy and gaslights you about what it does :D.
8
1,129
Is GPT 4o a better microscopy image analyst than earlier versions? Doesn't seem so, at least not when it comes down to identifying mitochondria in EM
1
4
46
4,269
Announcing our new and stable release of Segment Anything for Microscopy!
We introduce new features and improve the tool to segment microscopy data with a few clicks, like the mitochondria in the video, first segmented automatically and then corrected interactively.
6
54
280
45,385
E.g. gets count for this plate from AGAR dataset right (123 colonies). However, when prompted to explain approach is very vague, and also can't/wont give json with exact colony locations. (Have not tried to circumvent this yet).
1
3
511
Automatic 3d segmentation works in 2 steps:
First segment all objects in a given slice, then segment those objects in the volume.
1
4
245
Release 0.3.0 for micro_sam is there!
- Support for polygon and ellipse prompts
- Automatic instance segmentation in 3d
- Faster fine-tuning
See the advantage of polygon over box annotation for a cell with complex shape below!
1
17
92
11,298
Everything at a glance:
- preprint: biorxiv.org/content/10.1101/…
- software: github.com/computational-cel…
- documentation: computational-cell-analytics…
- tutorials: piped.video/watch?v=ket7bDUP…
1
2
8
890
... and the video tutorial for the tracking annotation tool: piped.video/Xi5pRWMO6_w.
1
2
217
... the video tutorial for the 3d annotation tool: piped.video/PEy9-rTCdS4, ...
1
1
232
... in the video tutorial for the 2d annotation tool: piped.video/ket7bDUP9tI, ...
1
1
256
One of our main finding: finetuning the SAM model significantly improves performance for light microscopy datasets, including image settings **we have not trained on**. Italic font datasets in the image were used for training (results on a test split), normal font not!
1
3
382
Our micro_sam preprint is out! We finetune Segment Anything for Microscopy and build interactive napari annotation tools based on it. See preprint at biorxiv.org/content/10.1101/…, software at github.com/computational-cel… and more info in this thread.
4
53
156
17,077
Another big micro_sam update! We have released the first batch of finetuned models for microscopy and the code for training/finetuning your own models. See the improvements due to the LM model in the figure! (cyan=prompt, yellow=correct segmentation, red=SAM prediction).
1
41
142
20,541
Segmenting neurites in EM works quite good, see the example video for our example data from github.com/computational-cel…. However, it has a limitation for branching structures, where it doesn't work well.
2
2
117
And we integrate the first fine-tuned models from @AnwaiArchit. See how they improve segmentation compared to the standard segment anything below! (Standard on the left, fine-tuned on the right)
1
2
412
We improved the automatic instance segmentation to enable instance rerun when changing parameters, which makes it much easier to find good settings:
1
3
419
New release alert! We published a big update for segment anything for microscopy github.com/computational-cel…. Now supporting tiled prediction, better instance segmentation and initial fine-tuned models.
See how smooth interactive annotation runs now for a large image!
6
25
173
27,029
I am looking for a PhD student to work on applications of vision foundation models in biology:
Make segmentation and tracking for biology effortless! (See the gif for our initial attempts). Apply at uni-goettingen.de/de/644546.…. Please RT!
1
66
115
24,818
The tracking annotator allows automatically tracking cells over time, and has support for dividing cells.
1
2
21
1,682
The 2d annotator also support using the automatic segmentation functionality of #SegmentAnything to initialize the segmentation (see gif below), and bulk segmentation from bounding boxes (see the gif in the first tweet).
1
8
1,361
We support interactive 2d, 3d segmentation and tracking from bounding box and point annotations. See for example segmenting a mitochondrion in 3d from point annotations in a single slice:
1
13
1,421
Introducing #SegmentAnything for microscopy, our napari based tools for interactive microscopy annotation: github.com/computational-cel…
8
103
470
62,542
We also study different adaptation strategies (joint vs. two-stage training ) and demonstrate our approach for three applications: cell segmentation in phase-contrast microscopy, mitochondria segmentation in EM and lung segmentation in X-Ray.
1
3
359
We build on current self-training methods for semi-supervised learning and domain adaptation (FixMatch, AdaMatch, MeanTeacher) and comine them with the probabilistic UNet. We use samples from the PUNet to estimate pseudo-label masks and show how this improves domain adaptation.
1
3
391
If you need to visualize, analyze and share large and multi-modal microscopy data read on:
MoBIE has now been published in Nature Methods: nature.com/articles/s41592-0…
Particular thanks to @Sci_Wanderlust and @tischitischer who co-developed the core MoBIE functionality.
9
51
151
23,760
Glad I am not applying for the Walter-Benjamin-Program! (CV needs exact school dates from the first grade...)
1
1
9
1,790
"Drawing of a cute Platynereis dumerilii larva with googly eyes" <- Dall-E mini draws a segmeted worm, whereas SD looks more like a cucumber.
1
1
Dall-E Mini / Craiyon seems to be quite a bit better at generating microscopy related images than (vanilla) Stable Diffusion: "Cell culture imaged with fluorescenec microscopy"
1
1
7
New MoBIE release with support for spatial transcriptomics data is out. Check out piped.video/1dDaxOAZ9Sg for details, and stay tuned for an updated version of the preprint explaining all new features.
7
21