Just watched
@adamtaggart's interview with
@edzitron on AI. Let me put on my AI hat...
"the early days" was mentioned several times. I studied AI in the late 80's -- that was just past the very early age, I had the privilege of reading work of the pioneers, working with some of them. Folks rolled their eyes, had not idea what I was talking about when I mentioned something like "neutral network". There was some cool stuff we could do back then, but we also learned the limitations of this fascinating technology.
Incidentally, part of my research was on natural language processing, with Eugene Charniak as one of my professors, a pioneer in the field.
It was the early days. I recall sitting in at a PhD thesis defense ca 1991, and someone in the audience pointed out the computational complexity, i.e. that the computing cost would be tremendous to do what he proposed. He brushed it off, saying "burn the Mips" -- "Million instructions per second" implying that over time, advanced computing power would solve the issue. FWIW, when computer scientists talk about computational complexity, it's an abstract measure of whether something is, say, linear or quadratic. If you stuff something into a huge model and computational complexity is linear, you will solve this with more advanced chips. If, in contrast, it's quadratic (or worse), you are bound to run into issues as your data set scales.
After getting my MSc in Computer Science with focus on AI, I switched universities to work on a PhD (which I never completed) in a non-AI field, but I knew bright students who worked on optimizing neutral networks (AI is more than neutral networks, but that's the dominant focus). I was mightily impressed about the increased efficiency of these models.
AI has amazing use cases - historically, these use cases were applied by highly specialized software engineers. However, "AI" the way we talk about it these does has no clue about causation -- because most commercial work is not focused on it. That means, AI models are about correlations, associations. This is, of course not intelligence.
FWIW, we have a similar issue in statistics. Statistics gives you relations, not predictions. That said, we've tweaked statistics enough over the years that we should, at least in theory, be able to determine the quality of statistical results. Except, if you look at much work in academia, a lot of folks use statistics that don't really know their math and make mistakes, drawing conclusions that aren't really there.
If you just take one field - economics - how on earth is it that we have academics that can prove their political bias with their models? Minimum wage is good. Minimum wage is bad. Just design your study and you'll be proven right.
To be clear, there are efforts to get causation into AI.
@yudapearl is a leader in the field. I had the pleasure of studying some of his pioneering work in the 80s. In 2018, he published "The Book of Why" where he half jokingly says he was all wrong back then, we don't even have language of causation, then proposes one. My view is that if you want AGI, you cannot do it with a model of causation -- everything else is hype.
The huge change that came a few years ago is that a consumer friendly natural language interface was released to the public, giving the general public access to the power of neutral networks. Suddenly, you have millions of people with no understanding of the underlying technology access an incredibly complex tool - except it doesn't appear complex. It's like Goethe's Sorcerer's Apprentice has entered the room.
What you have done is give every child around the world access to something that was formerly only accessible to a small circle of trained engineer.
The world thinks they are in a candy store. That's because they are. What can you expect to get? You can expect to get a lot of garbage because a lot of people just don't know how to use the technology. But you can also expect to get some amazing results because you'll have very smart people around the world that would not have used AI otherwise be able to use the technology we could have never imagined.
The point is: we don't know is how this will play out. We did not know the internet would create millions of Uber drivers - that's how we little we know about AI.
Ed says AI models aren't getting some basic stuff right. That's correct, but that's not the yardstick. It was never intended to get basic stuff right. Over time it will, but that's not the design of these models.
Anyone who has spent some time with models, I trust will say:
* I've used AI models and gotten crappy results;
* I've used AI models and gotten great results;
* Results I've received have improved substantially over the past year
My biggest concern is the amazing energy inefficiency. Investing in power generation is good, but if we invested the trillions into causation, we might actually get true AGI -- and importantly, a causation based model is orders of magnitude more energy efficient. That's not happening - at least not yet.
Is AI a lie? No. Is it a bubble? There's bound to be a bust at some point - we don't know when, but when that happens, the increased power capacity added will stay, and while the chips at the data centers will be outdated possibly faster than the depreciation on corporate balance sheets, data center infrastructure is part of the future, and even if a player goes bust, the data center will stay and someone else will pick it up.
The short of it: find a useful use case for AI for yourself. Exploration is part of learning - including all kinds of useless learning.
Think about the cat videos on social media - proof that not everything technology enables you increases productivity. But cat videos don't prove that all things are bad about the internet. The same with AI: we are in an exploration phase.
Finally, since Ed mentioned internet speed decades ago: when I started using the internet, I had a 300 bps connection, at times slower. That's 0.0003 Mbps. But I'm dating myself.
piped.video/watch?v=4xztOygL…