Mixed Reality + Diffusion prototype as a tool for exploring concepts, styles, and moods by transforming real-world surroundings into alternate realities. #MixedReality #MR #AI #StableDiffusion #Quest3

Calibrating…

Quick look at The Great Eye, our in-house tool for handling and merging sensor data to make spaces interactive. Here it’s running with 6 Orbbec Femto Mega cameras and 3 Hokuyo 2D LiDARs, all feeding into the same system. We’ve been developing and evolving this tool for a while now, mostly as the sensor layer behind our interactive installations. Built with @openframeworks #openFrameworks #ComputerVision #LiDAR #DepthCamera

A new Dormant Memories exploration, this time using Carveout.... Coiffure Mile End. One of the reasons I built Carveout was to explore 3D captures as both places and data. Explore this Dormant Memory yourself ↓ #GaussianSplatting #3DGS #ComputerVision #segmentation

Carveout is available here: github.com/smallfly/Carveout

Carveout is out. I started Carveout as a research project to explore how objects in existing 3D Gaussian Splatting scenes could be identified, labelled Carveout renders synthetic views from an existing 3DGS, uses SAM 3 to segment objects in those views, then lifts the 2D masks back onto the individual Gaussians. The result is a set of labelled 3D objects with positions, bounding boxes, supporting views and their own Gaussian subsets. No retraining of the scene, and no original capture images are needed. A big part of the project was also about the process itself: learning how all these pieces fit together, while developing Carveout from the beginning with Claude Opus 5.1 and Fable, and later bringing in OpenAI Astra as a challenger. I have tested it on rooms, shops, objects and outdoor captures, at different scales and orientations. It definitely won’t work perfectly on everything though, and that is part of why I’m releasing it. The code is now available for anyone who wants to experiment with it, break it, or take it somewhere else. Link in the thread. #GaussianSplatting #3DGS #ComputerVision #segmentation

New scene in the Dormant Memories series. A bouquet, some light, depth and time. #GaussianSplatting #3DGS #pointcloud #memories

I’ve been testing Carveout on a much wider range of 3DGS captures. Different scales, tilted scenes, low ceilings, interiors, objects, exteriors… each exposed assumptions that were fine on my original test scenes but didn’t generalize well enough. So over the last few days I’ve been doing my best to fix those properly rather than adding scene-specific workarounds. The main changes: - Scene scale is now handled explicitly, including captures with unknown or incorrect scale. - Tilted captures can be levelled without modifying the original 3DGS. - Camera placement and view selection are more robust across very different spaces. - The VLM can be selected per scene, with a smaller option when VRAM is limited. - The UI now does a better job of guiding each stage and showing what is running or needs attention. The bigger gain is that Carveout is becoming much less dependent on how a scene was captured or exported. Though it’s still better to start with a scene that is properly oriented and scaled 😶‍🌫️ 3DGS capture by Seeget3D, shared on SuperSplat. #GaussianSplatting #3DGS #ComputerVision