Working on @xtdb_com #ToolsForThought obsessive, 4GL maximalist (SQL is alright), always happy to chat :)
UK
- Tweets 739
- Following 4,754
- Followers 1,342
- Likes 5,212
I'm giving an online tech talk with @FINOSFoundation this Wednesday (4pm GMT) on 'as-of reporting' in financial services - but really this applies to all organisations who care about the auditability of systems. XTDB helps you to quickly figure out how data has changed and why /1
1
4
8
528
Ultimately though I suspect the 'best' protocol will be very data & situation dependent. It feels like all the same set of trade-offs you face when thinking about optimal LSM-trees and compression more generally
2
272
> recalculate the indexes on each insert
In principle this shouldn't be a blocker. Simpler is better if there's any hope for a database to accelerate the workload.
Like it might be okay *if* you could tune an LSM index to match the workload.
> no free lunch
aka RUM Conjecture
1
83
Just in case anyone was in doubt about the stability argument, Q1 (1974) still works great: dl.acm.org/doi/pdf/10.1145/8…
4
535
SQL is turning 50 years old later this week 🎉
In your opinion, which are the best bits? Which are the worst?
5
23
94
15,602
Related - I've only just realised that @MapsAsCode now supports Iterations 😍 docs.onlinewardleymaps.com/2…
1
6
748
Datomic bakes in some very strong opinions, and isn't OSS, but agreed it's totally worth taking the time to understand RIch's criticisms of regular SQL systems piped.video/watch?v=Cym4TZwT…
1
62
"What is @xtdb_com?" well, here's my latest <100s edit...let me know what you think! 🙏
1
9
18
1,430
Sometimes taking a hit on storage costs is worth it if you can decrease compute costs, but do your systems offer levers to arbitrage between the two?
...just one of the many things we'll be discussing this Thursday about columnar data & processing - sign up below to hear more 🙂
Still got time to RSVP!✍️
🗓️@refset's webinar on Columnar Data — Thursday, Nov 30, 16:00 UK (11 am ET).
Columnar storage isn't just a trend; it's an essential response to the soaring need for efficient data analytics.
Join us to learn more:
us06web.zoom.us/webinar/regi…
ALT RSVP here: https://us06web.zoom.us/webinar/register/8916989428777/WN_eY7vrdCjRfS-A-Q9jS29mw
1
3
7
1,193
Still, it sure would be nice if there was a better language for designing a database than SQL though.
1
2
5
448
"You should definitely design your database and *then* maybe consider using an ORM. Don't use the ORM to design your database" - @ThePrimeagen
Solid advice, fun watch 👇
piped.video/watch?v=nWchov5D…
2
2
11
1,995
DuckDB's latest benchmarks look seriously impressive!
In other news, I'm running a webinar on the principles and trends that underpin the world of columnar data storage & processing tech later this month. If that sounds interesting, you can sign up here us06web.zoom.us/webinar/regi…
2
2
15
1,746
"This shift in hardware balance is probably the biggest change to fundamental assumptions in database architecture since SSDs started to become a thing, and most people haven't internalized the implications yet." - @andrew_rogers
The race now is to optimise for latency.
9
33
210
54,542
Possibly a good moment to mention: I delayed diving into Clojure for ages purely because of the JVM dependency, but Clojure dialects are available for many different hosts nowadays. An LLVM-backed future with @jeayewilkerson's ‘Jank’ has me rather excited piped.video/watch?v=ncYlHfK2…
1
1
7
812
SQL has *massive* inertia and is certainly worth learning in favour of ORMs, but it feels far from unassailable given all the divides in opinions - thanks @ThePrimeagen for this entertaining reminder piped.video/watch?v=bpGvVI7N…
Something better than SQL can still succeed, surely...?
8
2
35
8,927
I've spent much of the last 4 years getting to grips with bitemporality. Luminaries like Kent Beck and Martin Fowler have been talking about it recently too. Curious to hear how bitemporality applies to your own Data Architecture? Please join my webinar: attendee.gotowebinar.com/reg…
2
18
22
7,537
@omnigres looks really promising! The idea of "moving code to data" is about embracing the physics of modern technology. Any architecture defying this new reality should be (will be) considered heretical piped.video/watch?v=ICqgQ5nv…
2
2
400
For the record I actually rather like LMDB :) and we've used it _within_ XT successfully (~3x read performance vs. RocksDB!)...but it's worth noting that XT is catering for a much higher level of abstraction, and all abstractions have a cost. See also cs.brown.edu/~ugur/fits_all.…
1
3
65
With storage costs decreasing ~indefinitely keeping history should be the default choice for everyone (certainly businesses, anyway).
2
1
109
If you like this talk you'll probably enjoy the book too. Easily my all-time favourite conference souvenir 🤘
5
It's been a decade since Craig Baumunk (RIP) outlined "Bitemp 2.0" ...but the revolution is still ahead of us! piped.video/watch?v=R7kQO5Z2…
I'm really hyped to be talking about these ideas at @strangeloop_stl this week in the context of our next-gen DB engine: xtdb.com/core2
7
22
I am *positively* excited for @hytradboi tomorrow — a day full of 10m *lightning* talks to melt your brain ⚡️🧠 …also featuring @xtdb_com 😎
Starting at 9am PT / 12pm ET / 5pm UK
ALT A person rubs a database against their head like a balloon, creating static charges.
"Have you tried rubbing a database on it?"
hytradboi.com/
A day of 10 minute talks on strange and unusual uses of databases and other data-centric ideas.
5
22
I'll be talking about @xtdb_com (as always 🙂) at the @hytradboi conference next month, which I think has a pretty awesome line-up! In anticipation of that, I thought I'd share a little about my journey down the rabbit hole...
14
40
Thank you @Avisi_IT for this wonderful overview of
@xtdb_com - a truly biertemporal database 🍻
piped.video/watch?v=dQpeArsN…
Er staat weer een nieuwe Avisi TechLab online! Dit keer gaat het over XTDB: de tijdreizende database. Jordi en Marcel leggen je graag uit hoe jij een productiewaardige backend kan bouwen met @xtdb_com. Klik hier: bit.ly/3lT782l #IThappenshere #XTDB #Backend
It's awesome to see the LA meetup group enjoying the Crux Datalog tutorial so much - good motivation to continue developing it 🙏
And kudos as ever to @usenextjournal for their excellent browser-based notebook platform that made the experience possible!
piped.video/watch?v=L7ajO6uD…
Second part of the Los Angeles #clojure Meetup's journey to learn Crux Datalog together. Thanks to everyone who joined!
piped.video/watch?v=L7ajO6uD…
5
18
piped.video/gcUWTa16Jc0 check out 13:50 for a particularly extreme example!
2
Reminds me of when I first learned about Jira workflows!
Being a better Jira than Jira would be a solid milestone :)
1
1
Heads up that "cool VR 3D stuff" is probably a distraction. Higher dimensions are best be encoded into appropriate visualisation channels. See vimeo.com/26205288
3
2
Very neat! You should check out @nikitonsky's similar conclusions and overview of "Alabaster" piped.video/l1b7Da2DnPo?t=832
Jeremy Taylor retweeted
"Escaping Captivity: Social Apes making Software in the Wild" piped.video/watch?v=HI04gW3V… @izs @nodedublin
1
1