Expert in tackling complex software challenges, with deep knowledge of TCP, MySQL and PostgreSQL kernels, and a passion for AI, history, math, and physics.
Beijing, China
- Tweets 3,298
- Following 136
- Followers 17,787
- Likes 1,427
Pinned Tweet
After nine months of hard work, this Postgres course is almost complete. You don't have to buy it, but if you want to truly understand Postgres internals, this is one of the most detailed courses you'll find. wangbin579.gumroad.com/l/pos…
5
52
526
36,385
Using arrays well is not easy. It takes skill, and you need to really understand your data and how it's structured. That's why programming is truly an art. In the future, poorly made databases will be everywhere.
In performance optimization, the data structure I least want to use is a hash (only as a last resort), and the one I most want to use is an array. When a database is full of hashes, there's probably a lot of optimization opportunity hiding in there.
2
3
83
4,627
In performance optimization, the data structure I least want to use is a hash (only as a last resort), and the one I most want to use is an array. When a database is full of hashes, there's probably a lot of optimization opportunity hiding in there.
11
9
232
16,804
I think I've got the evidence now: in tokio's async model, Rust is far weaker than C in some critical situations. Full breakdown in the Discord group next week. Don't miss it.
Why C? Four reasons. 1) Postgres is written in C too, and I want to reuse its WAL replay code. 2) I don't want to change my taste. I just don't like Rust, which misses a 't' 3) I don't want slow builds or big binaries. 4) I'm not smart enough for Rust.
5
67
9,261
I don't even like C++ and I don't want to use any C standard library, let alone Rust. Most of the time, good optimization is just about getting your algorithms and data structures right, which is something you have to figure out yourself, case by case.
Why C? Four reasons. 1) Postgres is written in C too, and I want to reuse its WAL replay code. 2) I don't want to change my taste. I just don't like Rust, which misses a 't' 3) I don't want slow builds or big binaries. 4) I'm not smart enough for Rust.
1
63
2,836
Why C? Four reasons. 1) Postgres is written in C too, and I want to reuse its WAL replay code. 2) I don't want to change my taste. I just don't like Rust, which misses a 't' 3) I don't want slow builds or big binaries. 4) I'm not smart enough for Rust.
Our product will be written in pure C (sorry, I don't trust Rust). Even if nobody ever uses it, it'll still show you what the art of optimization really looks like.
5
2
58
16,974
After comparing the trace info from various Rust extensions vs Postgres C code, it's pretty obvious which one performs better.
Our product will be written in pure C (sorry, I don't trust Rust). Even if nobody ever uses it, it'll still show you what the art of optimization really looks like.
2
21
2,452
Our product will be written in pure C (sorry, I don't trust Rust). Even if nobody ever uses it, it'll still show you what the art of optimization really looks like.
The course will keep getting deeper, since I'm growing fast. After studying Neon carefully, I'll use TigerBeetle's ideas to build a faster version that separates storage and compute, making the course even deeper. Technically, I'm sure I can build a great database product.
5
1
85
12,603
The course will keep getting deeper, since I'm growing fast. After studying Neon carefully, I'll use TigerBeetle's ideas to build a faster version that separates storage and compute, making the course even deeper. Technically, I'm sure I can build a great database product.
Me soon, and thank you so much for the course, it really helped me build a mental model of what goes on behind the scenes
1
45
9,955
If you've been at Postgres for over two years and still don't get it, you're using the wrong approach.
Two years ago, I was a Postgres newbie. I didn't even know how to install it. Now I feel like I really understand Postgres. Learning Postgres is only hard for DBAs. For developers, it's easy, and the ideas behind it are even easier, as long as you use the right approach.
3
27
2,361
Two years ago, I was a Postgres newbie. I didn't even know how to install it. Now I feel like I really understand Postgres. Learning Postgres is only hard for DBAs. For developers, it's easy, and the ideas behind it are even easier, as long as you use the right approach.
4
7
155
8,508
I spent 20 days tracing Neon and digging into all kinds of perf bottlenecks, and I learned a lot. Building a database on top of Postgres is actually simple, and most of you can do it too. You just need the courage to learn Postgres and go build one yourself.
3
62
3,347
You're right. From my experience, the most useful tool is just logs, but most people don't know how to use them effectively. No matter how hard the perf problem is, or how wild the butterfly effect gets, you can find the root cause of nearly 100% of them through careful logging.
Partially correct and i beg to differ a bit.
Flamegraphs are excellent for identifying where CPU time is spent. They become insufficient when the bottleneck comes from cache misses, memory bandwidth, lock contention, I/O stalls, scheduler behaviour, NUMA effects or tail latency amplification.
For difficult performance problems, use flamegraphs as the starting map and then add hardware counters, allocation profiles, off-CPU traces, lock analysis, I/O telemetry and workload evel measurements. The tool is not useless but the mistake is expecting one profile to explain the entire system.
3
2
47
3,716
This proves my point about Rust again: if you don't use it well, the performance is a disaster. It also confirms that TigerBeetle's architecture is excellent.
I think I've found one of the root causes of Neon's poor performance. If you use Rust the way Neon does, there's no way you'll get good performance. We'll analyze this as a classic case study in our Discord group.
5
6
93
8,504
If a company can't optimize better than I can, it definitely can't be called a tech company.
I think I've found one of the root causes of Neon's poor performance. If you use Rust the way Neon does, there's no way you'll get good performance. We'll analyze this as a classic case study in our Discord group.
2
31
2,946
I think I've found one of the root causes of Neon's poor performance. If you use Rust the way Neon does, there's no way you'll get good performance. We'll analyze this as a classic case study in our Discord group.
5
5
75
18,055
I can honestly tell you a story. Without tcpcopy back then, I'd probably have gotten an anxiety disorder, because I worried every day about my deployment crashing in production. After tcpcopy, I could say with confidence: my system won't break, and I don't need any alerts.
Writing server services in C is easy for me. I've been doing it for 10+ years and I've never gotten in trouble for my mistakes. Memory, async, performance, none of that is a problem. With C, I can write code however I want, because tcpcopy keeps me from worrying about going live.
1
1
17
3,264
Writing server services in C is easy for me. I've been doing it for 10+ years and I've never gotten in trouble for my mistakes. Memory, async, performance, none of that is a problem. With C, I can write code however I want, because tcpcopy keeps me from worrying about going live.
2
57
8,997
Did you ever wonder why paxos clusters recommend six nodes? As someone who's improved on some variant of paxos before, I'll admit I never thought about this.
TigerBeetle's architecture is definitely worth a look. A lot of the optimization ideas are worth learning from, and it really broadened my previous experience. Anyone who loves databases shouldn't miss it: github.com/tigerbeetle/tiger…
2
1
53
4,215
To really get the most out of TigerBeetle's architecture, you'd better have some relevant experience. A lot of the optimization techniques it covers are things I've already used myself, so it really resonates with me.
TigerBeetle's architecture is definitely worth a look. A lot of the optimization ideas are worth learning from, and it really broadened my previous experience. Anyone who loves databases shouldn't miss it: github.com/tigerbeetle/tiger…
1
2
48
4,509