Research Scientist at Google DeepMind, Berlin. stronglyconvex.com
Berlin, DE
- Tweets 333
- Following 149
- Followers 2,489
- Likes 615
Pinned Tweet
Introducing SMERF: a streamable, memory-efficient method for real-time exploration of large, multi-room scenes on everyday devices. Our method brings the realism of Zip-NeRF to your phone or laptop!
Project page: smerf-3d.github.io
ArXiv: arxiv.org/abs/2312.07541
(1/n)
21
188
835
188,539
I had the pleasure last month of giving slightly provocative talk at Bliss AI, a a wonderful student-run organization here in Berlin. The best part: the talk is online for everyOne to enjoy! Behold: "NeRF is dead"
piped.video/FtaC5lh8hxs?si=dnJ0…
1
7
3,532
Daniel Duckworth retweeted
Are you at #SIGGRAPH2024 and want to learn how to reconstruct meshes from multi-view images that contain details like individual strands of grass? Then come today to the "Radiance Field Processing" session at 2 p.m. in Mile High 1.
binary-opacity-grid.github.i…
31
184
18,463
Daniel Duckworth retweeted
InterNeRF: Scaling Radiance Fields via Parameter Interpolation
@clintonjwang, @PeterHedman3, Polina Golland, @jon_barron, @duck
tl;dr: use camera origin to assign camera region to partitioned parameter grid and compute bilinear interpolation weights
arxiv.org/pdf/2406.11737
2
9
44
7,603
Daniel Duckworth retweeted
Googleがリアルタイムに高精度なレンダリングを実現する新技術「SMERF」を発表
gigazine.net/news/20231214-s…
23
38
21,939
Daniel Duckworth retweeted
Seeing the amazing new SMERF technology immediately made me imagine a time when we can walk around in environments like this, styled to be anything we can imagine. All happening in realtime, shaped with voice prompts or virtual paint or sculpt marks.
Here's a test of using some of the SMERF demo scenes to drive a generative AI image stream in Krea.. incredible! It's like seeing the world through a new camera lens of imagination.
I'll post a link to the incredible SMERF paper by @duck and his colleagues below.. what a time to be alive!
#ai #art
15
45
297
36,340
Daniel Duckworth retweeted
Thoughts NeRFs were dead? Google DeepMind just dropped SMERF — streamable, multi-room NeRFs with cm-level detail. Oh and it works realtime on mobile 🤯
It’s a sweet spot between the speed of Gaussian Splatting and the quality of Zip-NeRF.
More below ⬇️
Daniel Duckworth
12
88
546
72,430
Daniel Duckworth retweeted
SMERF from Google Research (again) achieves Zip-NeRF quality, operating at a remarkable 60fps on everyday devices like smartphones and laptops.
🔗 neuralradiancefields.io/the-…
4
67
345
40,162
Introducing SMERF: a streamable, memory-efficient method for real-time exploration of large, multi-room scenes on everyday devices. Our method brings the realism of Zip-NeRF to your phone or laptop!
Project page: smerf-3d.github.io
ArXiv: arxiv.org/abs/2312.07541
(1/n)
21
188
835
188,539
The result: a set of compact, streaming-ready submodels ready to run at up to 60 fps in your browser. The best part: you can try it out yourself: smerf-3d.github.io
(7/n)
1
1
16
3,094
This has been a joint work with my amazing collaborators: @PeterHedman3, @chrisjreiser, @PeterZhizhin, @jfthibert, @mariolucic_, @RSzeliski, and @jon_barron
Learn more and try SMERF out yourself at smerf-3d.github.io
(8/n)
2
4
26
2,417
From the moment NeRF was first published, the research community knew it would be something game-changing. I'm proud to be part of the team turning this amazing line of work into a real product experience!
Immersive View gives users a virtual, close-up look at indoor spaces in 3D! Learn how it uses neural radiance fields to seamlessly fuse photos to produce realistic, multidimensional reconstructions of your favorite businesses and public spaces → goo.gle/3X6L9G8
2
1
5,725
Daniel Duckworth retweeted
Code finally released for our CVPR 2022 papers (mip-NeRF 360/Ref-NeRF/RawNeRF)! You can also find links for each paper's dataset on its project page.
The code has some nice new camera utilities for larger real scenes, like this one.
github.com/google-research/m…
We've finally released code for three of our CVPR2022 papers: mip-NeRF 360, Ref-NeRF, and RawNeRF. Instead of three separate releases, we've done something a little unusual and merged them into a single repo. Excited to see what people do with this! github.com/google-research/m…
16
95
544
I'm stoked to be a contributor on Object SRT, a new method for unsupervised, posed-images-to-3D-scene representation and segmentation! It's crazy fast and, while far from perfect, is leaps and bounds better than anything I've seen yet :)
So excited to share Object Scene Representation Transformer (OSRT):
OSRT learns about complex 3D scenes & decomposes them into objects w/o supervision, while rendering novel views up to 3000x faster than prior methods!
🖥️ osrt-paper.github.io
📜 arxiv.org/abs/2206.06922
1/7
7
I'm proud to announce the release of our new paper relating Whitening, Newton's Method, and Generalization! tl;dr whitening w/o regularization significantly reduces a model's ability to generalize.
Work with @negative_result @sschoenholz @ethansdy @jaschasd
Whitening and second order optimization both destroy information about the dataset, and can make generalization impossible: arxiv.org/abs/2008.07545 We examine what information is usable for training neural networks, and how second order methods destroy exactly that information.
3
6
But all is not lost! We also find that *regularized* second-order optimization leads to better generalization than un-regularized second-order optimization or gradient descent.
2
3
None of this would be possible without my amazing collaborators! @negative_result, @sschoenholz, @ethansdyer, and @jaschasd.
2