Unity Catalog is the industryโ€™s only universal catalog for data and AI.

New Open Lakehouse + AI episode now available! Stop shipping 3 versions of annual revenue. ๐Ÿ‘‡ Scott Haines and Lisa Cao on metric views in Spark 4.2. ๐Ÿ“„ One YAML KPI. Catalog serves it globally. ๐Ÿค– Same number for dashboards, agents, and AI/BI. ๐Ÿ”— Native interop with Unity Catalog. ๐ŸŽฅ piped.video/JJkkOBLPge4 #openlakehouse #unitycatalog #apachespark
1
2
217
Ask three teams for last quarterโ€™s earnings. Get three numbers. Metric views in Unity Catalog define the KPI once (YAML: source, joins, filters, dimensions, measures). Humans and agents query the same number. Spark uses MEASURE(). Try it on Unity Catalog 0.6 with Apache Spark 4.3. ๐Ÿ”— Learn more: unitycatalog.io/blogs/uc-metโ€ฆ #UnityCatalog #MetricViews #ApacheSpark #OpenSource
1
94
Introducing the UC Delta API in Unity Catalog 0.5 ๐Ÿ‘‡ ๐Ÿ”น Versioned, discoverable, atomic REST surface. ๐Ÿ”น Native Delta wire format and intent-based commits. ๐Ÿ”น Run it locally with Spark or DuckDB. Read the full post: unitycatalog.io/blogs/unity-โ€ฆ #UnityCatalog #DeltaLake
1
3
295
[Ep 1] Open Lakehouse + AI: The Catalog Layer, Interoperability & AI-Agent Governance Learn how composable lakehouses center the catalog for metadata, versioning, and commit coordination. ๐Ÿ‘‡ ๐Ÿ”ธ Interop: Iceberg REST, Unity Catalog (OSS), Spark, Delta Lake, Delta-RS, Iceberg. ๐Ÿ”ธ Governance: credential vending, row/col filters, column masks, audits, and why โ€œno governance was the easiest governance.โ€ ๐Ÿ”ธ Agents as data customers (Temporal). Fine-grained access beats blanket credentials on 24/7 workloads. ๐ŸŽฅ Full episode: piped.video/watch?v=dEFAkS7vโ€ฆ #OpenLakehouse #UnityCatalog @ApacheSpark @ApacheIceberg @DeltaLakeOSS
1
2
5
365
Simplified Governance with Catalog-Managed Tables ๐ŸŒ Unity Catalog 0.4.0 introduces support for UC managed tables, enabling data teams to centrally govern, discover, access, and audit their data through Unity Catalog. Instead of relying on scattered storage paths, separate credentials, and manual maintenance, teams can rely on Unity Catalog as the single logical system of record for their data estate. Leverage UC managed tables to strengthen governance, improve performance, and build on the most modern open catalog for the data and AI era. ๐Ÿ“– Check out the announcement and implementation details on the Unity Catalog blog: lnkd.in/eZWkMBaR #unitycatalog #governance #opensource #catalogs #deltalake
1
2
143
With UC managed tables, teams unlock: ๐Ÿ”ธ ๐—จ๐—ป๐—ถ๐—ณ๐—ถ๐—ฒ๐—ฑ ๐—ด๐—ผ๐˜ƒ๐—ฒ๐—ฟ๐—ป๐—ฎ๐—ป๐—ฐ๐—ฒ:ย Unity Catalog centralizes access control, replacing fragmented storage-level policies. This simplifies how teams ensure all engines access data in a governed, consistent manner. ๐Ÿ”ธ ๐—ฆ๐˜๐—ฎ๐—ป๐—ฑ๐—ฎ๐—ฟ๐—ฑ๐—ถ๐˜‡๐—ฒ๐—ฑ ๐—ฑ๐—ถ๐˜€๐—ฐ๐—ผ๐˜ƒ๐—ฒ๐—ฟ๐˜†: Unity Catalog provides stable logical table identifiers, eliminating the need for clients to depend on physical storage paths for discovery. ๐Ÿ”ธ ๐—˜๐—ณ๐—ณ๐—ผ๐—ฟ๐˜๐—น๐—ฒ๐˜€๐˜€ ๐˜๐—ฎ๐—ฏ๐—น๐—ฒ ๐—ผ๐—ฝ๐˜๐—ถ๐—บ๐—ถ๐˜‡๐—ฎ๐˜๐—ถ๐—ผ๐—ป๐˜€: By automating storage tuning and credential management, Unity Catalog removes the burden of manual operational maintenance from data teams. ๐Ÿ”ธ ๐—›๐—ผ๐—น๐—ถ๐˜€๐˜๐—ถ๐—ฐ ๐—ฎ๐˜‚๐—ฑ๐—ถ๐˜๐—ฎ๐—ฏ๐—ถ๐—น๐—ถ๐˜๐˜†:ย Metadata and permissions are centralized in a single interface, allowing for high-level oversight of ownership and access instead of parsing low-level storage logs. ๐Ÿ”ธ ๐—˜๐—ป๐—ณ๐—ผ๐—ฟ๐—ฐ๐—ฒ๐—ฎ๐—ฏ๐—น๐—ฒ ๐—ฐ๐—ผ๐—ป๐˜€๐˜๐—ฟ๐—ฎ๐—ถ๐—ป๐˜๐˜€: Unity Catalog can authoritatively validate or reject schema and constraint changes, preventing incompatible updates that could compromise data integrity or break downstream workloads. ๐Ÿ”ธ ๐—™๐—ฎ๐˜€๐˜๐—ฒ๐—ฟ ๐—พ๐˜‚๐—ฒ๐—ฟ๐˜† ๐—ฝ๐—น๐—ฎ๐—ป๐—ป๐—ถ๐—ป๐—ด ๐—ฎ๐—ป๐—ฑ ๐—ณ๐—ฎ๐˜€๐˜๐—ฒ๐—ฟ ๐˜„๐—ฟ๐—ถ๐˜๐—ฒ๐˜€: Unity Catalog delivers table metadata directly to Delta clients, bypassing cloud storage requests to significantly reduce metadata latency and accelerate query planning and writes.
68
We are excited to announce ๐—จ๐—ป๐—ถ๐˜๐˜† ๐—–๐—ฎ๐˜๐—ฎ๐—น๐—ผ๐—ด ๐Ÿฌ.๐Ÿฐ.๐Ÿฌ which includes exciting new features and many bug-fixes and improvements! ๐ŸŽŠ Check out some of the highlights ๐Ÿ‘‡ ๐—จ๐—– ๐—ฆ๐—ฒ๐—ฟ๐˜ƒ๐—ฒ๐—ฟ โš™๏ธ Storage Credentials for AWS โš™๏ธ External Locations for AWS โš™๏ธ Managed storage location for catalogs and schemas ๐—จ๐—– ๐—ฆ๐—ฝ๐—ฎ๐—ฟ๐—ธ ๐—–๐—ผ๐—ป๐—ป๐—ฒ๐—ฐ๐˜๐—ผ๐—ฟ โšก Credential Renewal Enabled by Default โšก Support for Spark 4.1 and Delta 4.1 โšก Atomic CTAS for Delta Tables in UCSingleCatalog ๐—จ๐—– ๐—”๐—œ ๐Ÿค– DSPY Integration with AI Functions A huge thank you to the awesome community who made this release possible! ๐Ÿ“– Full release notes: github.com/unitycatalog/unitโ€ฆ #unitycatalog #opensource #oss
1
119
Weโ€™re excited to announce the release of Unity Catalog v0.3.1!ย ๐ŸŽ‰ This release includes exciting new features and many bug-fixes and improvements. Version 0.3.1 focuses on three major areas: ๐Ÿ”น ๐—œ๐—บ๐—ฝ๐—ฟ๐—ผ๐˜ƒ๐—ฒ๐—ฑ ๐—๐—ฎ๐˜ƒ๐—ฎ ๐—ฐ๐—น๐—ถ๐—ฒ๐—ป๐˜ ๐—”๐—ฃ๐—œ ๐˜„๐—ถ๐˜๐—ต ๐—ข๐—”๐˜‚๐˜๐—ต ๐˜€๐˜‚๐—ฝ๐—ฝ๐—ผ๐—ฟ๐˜: designed for reliability and extensibility in production environments. ๐Ÿ”น ๐—”๐˜‚๐˜๐—ผ๐—บ๐—ฎ๐˜๐—ถ๐—ฐ ๐—ฐ๐—ฟ๐—ฒ๐—ฑ๐—ฒ๐—ป๐˜๐—ถ๐—ฎ๐—น ๐—ฟ๐—ฒ๐—ป๐—ฒ๐˜„๐—ฎ๐—น: to support long-running workloads cross cloud platforms. ๐Ÿ”น ๐—จ๐—–-๐—บ๐—ฎ๐—ป๐—ฎ๐—ด๐—ฒ๐—ฑ ๐——๐—ฒ๐—น๐˜๐—ฎ ๐˜๐—ฎ๐—ฏ๐—น๐—ฒ๐˜€: this enables Unity Catalog to coordinate table storage and commits centrally. This release is the result of contributions from our growing open-source community. A big thank-you to everyone who reported issues, submitted pull requests, reviewed code, and shared feedback! ๐Ÿ”— Dive into the release notes for the full list of highlights:ย github.com/unitycatalog/unitโ€ฆ #unitycatalog #opensource #oss #catalog
82
Managing data pipelines at scale is complicated, often resulting in the data silo problemโ€”where valuable assets are spread across systems. This makes it difficult to track, secure access, and scale cleanly. The solution is a clear structure paired with centralized governance. โœ… The Medallion Architecture structures your data into three distinct layers: ๐Ÿฅ‰ ๐—•๐—ฟ๐—ผ๐—ป๐˜‡๐—ฒ: Raw, ingested data. ๐Ÿฅˆ ๐—ฆ๐—ถ๐—น๐˜ƒ๐—ฒ๐—ฟ: Cleaned, enriched data. ๐Ÿฅ‡ ๐—š๐—ผ๐—น๐—ฑ: Business-level data, ready for reporting. Pair this framework with Unity Catalog, and you get a unified system to manage, govern, and organize your entire data flow. ๐Ÿ”— Walk through how this works: unitycatalog.io/blogs/buildiโ€ฆ #UnityCatalog #MedallionArchitecture #DataGovernance #OpenSource
1
1
86
โšก๏ธ From vector DB to multimodal lakehouse at petabyte scale. Join us in Mountain View on Nov 13 for โ€œScaling Multimodal AI Lakehouse with Lance & LanceDBโ€ with Chang She (@lancedb) at the Open Lakehouse + AI Mini Summit! Learn: ๐Ÿ”น Lowโ€‘latency random access + search APIs (vectors, text, binaries) ๐Ÿ”น Schema primitives across blobs + metadata for feature engineering ๐Ÿ”น Hybrid search (vector + fullโ€‘text) for training/fineโ€‘tuning at scale Two exciting tracks, one epic afternoon. ๐Ÿ‘ #OpenLakehouse #AI #LanceDB #VectorDB #DataEngineering
2
7
454
Unity Catalog retweeted
Join us at Open Lakehouse + AI Paris on November 24, 6:30โ€“10PM โ€” co-located with the Forward Data Conference! ๐Ÿ‡ซ๐Ÿ‡ท Secure your spot now โฌ‡๏ธ luma.com/OLM-1124 Weโ€™re bringing together data innovators and open source contributors for an evening packed with insight and inspiration. Hear talks from: โœ… Alexandre BERGERE (@DataGalaxy / Datalex) on Building a Scalable Usage Insights Platform with Delta Sharing โœ… Bartosz Konieczny (@waitingforcode) on Design Patterns for the Open Lakehouse โœ… Youssef Mrini & El Ghali Benchekroun (@Databricks) on The Future of Open Table Formats & @unitycatalog_io Food, drinks, networking โ€” and plenty of new ideas (& swag) to take home. ๐ŸŒŸ #opensource #deltalake #oss #apacheiceberg #unitycatalog #lakehouse #openlakehouse #ai
5
8
689
Missed it liveโ“See how an observabilityโ€‘first Telemetry Lake correlates data movement with system behavior to detect issues, diagnose root causes, and adapt in real timeโ€”why correlation > coverage and why the first ~150 characters of signal matter. Watch: piped.video/watch?v=r04iCvm6โ€ฆ Whatโ€™s inside: @OpenLineage + @opentelemetry + an LLM reasoning layer that turns noisy lineage and traces into prioritized actions to cut TTD/TTR and reduce blast radius. #openlakehouse #datalineage #observability #AI
1
2
104
Where does the feature platform fit in a world increasingly dominated by AI? At Open Lakehouse + AI Mini Summit, Hao Xu (Apple) shares how Feast is evolving beyond a feature store into a full feature platform for AIโ€”bridging data, models, and applications through innovations like Compute Engine, Feast for RAG, and On-Demand Feature Views. Donโ€™t miss this deep dive into how foundational feature architecture continues to drive real-world AI innovation. ๐Ÿ“ Mountain View, CA ๐Ÿ—“๏ธ Nov 13 ๐Ÿ•ฆ 12:00 - 4:30PM PT ๐Ÿ”— Secure your spot: luma.com/OLMS-1113 #opensource #oss #unitycatalog #openlakehouse #ai
2
3
483
๐Ÿšจ Exciting news: @starburstdata has announced GA support for Unity Catalog, enabling users to read and write to any UC managed table โ€” @DeltaLakeOSS or @ApacheIceberg โ€” using industry-standard open APIs! ๐Ÿš€ What you should know: โœ… ๐—–๐—ฒ๐—ป๐˜๐—ฟ๐—ฎ๐—น๐—ถ๐˜‡๐—ฒ๐—ฑ ๐—ด๐—ผ๐˜ƒ๐—ฒ๐—ฟ๐—ป๐—ฎ๐—ป๐—ฐ๐—ฒ ๐—ฒ๐˜ƒ๐—ฒ๐—ฟ๐˜†๐˜„๐—ต๐—ฒ๐—ฟ๐—ฒ โ€” UC permissions are enforced wherever you query, with Starburst authenticating via OAuth 2.0 for per-user, secure access. โœ… ๐—จ๐—ป๐—ถ๐—ณ๐—ถ๐—ฒ๐—ฑ ๐—ฐ๐—ผ๐—บ๐—บ๐—ถ๐˜ ๐—ฐ๐—ผ๐—ผ๐—ฟ๐—ฑ๐—ถ๐—ป๐—ฎ๐˜๐—ถ๐—ผ๐—ป โ€” UC acts as the single commit coordinator for Delta Lake catalog-managed writes, enabling consistency and multi-table transactions across engines. โœ… ๐—ข๐—ป๐—ฒ ๐—ผ๐—ฝ๐—ฒ๐—ป, ๐—ด๐—ผ๐˜ƒ๐—ฒ๐—ฟ๐—ป๐—ฒ๐—ฑ ๐—น๐—ฎ๐—ธ๐—ฒ๐—ต๐—ผ๐˜‚๐˜€๐—ฒ โ€” A single catalog for multi-engine reads/writes, advanced governance, and open table formats โ€” no silos, no lock-in. The open lakehouse continues to grow through community-driven innovation. With Starburst joining engines and tools like @duckdb, @ClickHouseDB, @anyscalecompute, @langchain, and @daftengine, developers can now operate with consistent governance, broad interoperability, and transparent metadata standards across the entire data and AI lifecycle. ๐Ÿ”— Learn more: starburst.io/blog/starburst-โ€ฆ #opensource #oss #unitycatalog #starburst
2
87
Unity Catalog retweeted
The open source and data community is coming together in Mountain View, CA! ๐Ÿ’ฅ Join us Nov 13 (12โ€“4:30PM PT) for the ๐—ข๐—ฝ๐—ฒ๐—ป ๐—Ÿ๐—ฎ๐—ธ๐—ฒ๐—ต๐—ผ๐˜‚๐˜€๐—ฒ + ๐—”๐—œ ๐— ๐—ถ๐—ป๐—ถ ๐—ฆ๐˜‚๐—บ๐—บ๐—ถ๐˜, featuring two tracks packed with insights on AI infrastructure, context engineering, and the future of interoperable data systems. Lunch, swag, and great conversations included! Stick around for the @ApacheSpark Happy Hour (5โ€“6:30PM PT) โ€” the perfect way to wrap up a day of learning and community. ๐Ÿ˜Ž ๐ŸŽŸ๏ธ RSVP here: luma.com/OLMS-1113 #opensource #oss #unitycatalog #apachespark #deltalake #apacheiceberg #ai
3
9
1,031
๐Ÿ“ฃ Get ready for the nextย Open Lakehouse + AIย webinar! Joinย @wslulciuc, Co-Founder & CEO ofย @OleanderHQ, hosted byย @lisancao from @databricks, will explore why the future of data lineage isnโ€™t another graphโ€”itโ€™s aย reasoning layerย that connects lineage, telemetry, and AI. Learn how Oleanderโ€™sย Telemetry Lakeย unifies @OpenLineage, @opentelemetry, and #LLM reasoning to help data teams detect, diagnose, and adapt in real timeโ€”building theย always-on-call data engineer. ๐Ÿš€ ๐Ÿ—“๏ธ October 30 ๐Ÿ• 9:00AM PT ๐Ÿ”— Register: luma.com/openlakehouse-1030 #opensource #oss #openlakehouse #ai #oleander
3
194
Unity Catalog retweeted
Open Lakehouse + AI spotlight: Hannes Mรผhlheisen shares how anyone can contribute to @duckdb and DuckLakeโ€”both MIT-licensed, open source, and welcoming PRs and community engagement on GitHub. Check out this clip to hear how easy it is to get involved and start contributing today. ๐Ÿš€ Whatโ€™s next?โ€‹ ๐Ÿ”น October 30: Webinar โ€” The Failed Promises of Data Lineage: Why More Metadata Isnโ€™t the Answerโ€‹ ๐Ÿ”น November 13: Mini Summit โ€” Mountain Viewโ€‹ ๐Ÿ”น November 24: Meetup โ€” Parisโ€‹ ๐Ÿ”— RSVP and details:ย luma.com/openlakehouse #opensource #oss #openlakehouse #ai #ducklake #duckdb
1
4
439
Open Lakehouse + AI brings builders and practitioners together to share how open lakehouse and AI meet in the real worldโ€”through adoption stories, handsโ€‘on patterns, and collaborative learning. ๐ŸŒŽ Join a global community shaping the future of data and AI! Whatโ€™s next? ๐Ÿ”น ๐—ข๐—ฐ๐˜๐—ผ๐—ฏ๐—ฒ๐—ฟ ๐Ÿฏ๐Ÿฌ: Webinar โ€” The Failed Promises of Data Lineage: Why More Metadata Isnโ€™t the Answer ๐Ÿ”น ๐—ก๐—ผ๐˜ƒ๐—ฒ๐—บ๐—ฏ๐—ฒ๐—ฟ ๐Ÿญ๐Ÿฏ: Mini Summit โ€” Mountain View ๐Ÿ”น ๐—ก๐—ผ๐˜ƒ๐—ฒ๐—บ๐—ฏ๐—ฒ๐—ฟ ๐Ÿฎ๐Ÿฐ: Meetup โ€” Paris ๐Ÿ”— RSVP and details: luma.com/openlakehouse A look back at our recent Open Lakehouse + AI Amsterdam meetupโ€”huge thanks to the speakers, volunteers, and everyone who joined us! ๐Ÿ‘‡๐Ÿ“ธ #openlakehouse #ai #lakehouse #unitycatalog #apacheiceberg #deltalake #apachespark #oss #openlakehouseai
1
1
638