Creative AI Lead @ Supercell - working on the future of mobile entertainment AI Artist: "LIWA" + more. Creating @beatbanditai - AI-powered writers' room.

Helsinki, Finland
Fuck it creating a microdrama I can produce good quality drama now in a sane speed + price so doing a chapter (35 minutes). I see some people posting "I made an ad for 8 minutes for 8 euro" and I see other people posting "I made a short film for 20000 euro with a crew of 15 people" - every approach is fine! ... but the middle ground is interesting to me -- what's the best one person can create without still breaking the budget or spending months to do so. If you have a story to tell, can you produce it by yourself? I have 20 minutes created so far -- it's incredibly fun and challenging at the same time. quick teaser from episode 3 The series is fundamentally basically 'people talking' - no dragons, explosions, spacecraft or cameras whipping around people skateboarding -- I do have a feeling that if you're creating anything longer, it's not sustainable. You can't have a story that's always on 11. let's see where this goes why not?
Made with AI
41
9
269
32,265
Trained a new ( 374-million-parameter diffusion transformer ) model using my own art as part of training material -- 🧵 Pieces Still Life With Fruit On A Table
1
1
27
1,387
Winter Sun Through Trees
1
4
148
I mean really a diffusion model trained from scratch. (Not just finetuned or Lora.) Why go through the trouble -- Why not? Through a discussion with LLM as how to train it so that it doesn't try to be like everything else: doesn't aim for realism. 'deliberately undertrained by design: confident about how paint, tone and light behave, only faintly certain about what things are, with subjects that emerge or dissolve depending on how hard you push the guidance', as Fable summarizes the intent here for the art
5
135
Roope Rainisto retweeted
By popular demand, I present to you.. my magnum opus.
455
2,675
31,073
1,836,081
Roope Rainisto retweeted
The World is Changing: AI For Creativity By Jeffrey Katzenberg A few months ago, I sat in my office in Silicon Valley and watched as a tech founder showed me something extraordinary. On the screen was a fully realized, beautifully lit, well-composed animated scene. It was stunning and it made me feel exactly what I felt in 1986 watching Luxo Jr. That was the first time I watched a computer-animated 3D character take a breath and seem, against all reason, to have life. It left me in awe. Later that day, I received a text from an artist I've known for thirty years, 350 miles to the south, in the city where I spent most of my career. After seeing a similar video, she texted: "Is this the end of us?" My answer was, "Certainly not.” I have spent the better part of the last decade in Silicon Valley, but the heart of my career has been in Hollywood. Being deeply connected to both worlds means I have deep loyalties to each and a responsibility to speak honestly to both. In 2023, I said that these new AI tools would cut the time and cost of producing world-class animation by as much as ninety percent within three years. Some colleagues were alarmed, many were furious. There is growing fear and resistance surrounding AI within the creative community. I deeply understand it, because I've spent countless hours walking through animation studios watching gifted artists bent over their desks, rebuilding a single second of film for the tenth time because the ninth version wasn't quite right. I've sat in screening rooms where four years of people's labor played out in minutes, and I knew the name of every person that had spent countless hours bringing those images to life. The creative process is a calling, there's really no other way to describe it. From the outside some see resistance. From the inside, it is love. People do not fight this hard for things they don't care about. The pushback coming out of Hollywood represents the collective effort of people who are deeply passionate about their craft. Is History Repeating Itself? The history here is more complicated than either side may realize. In 1906, the most famous composer in America, John Philip Sousa, published an essay titled “The Menace of Mechanical Music." He warned that the phonograph would become "a substitute for human skill, intelligence and soul." Sousa's fight was not really about the machine, it was about money. The machines were playing his compositions, and the men who built them weren't paying him a cent. His campaign helped create the Copyright Act of 1909. He did not stop the technology. He changed the terms under which it could use his work. A hundred years ago, sound came to the movies. We remember it now as a miracle, and it was. What we forget is who paid for it. Before sound, tens of thousands of musicians made their living in the orchestra pits of movie houses, scoring every film live, every night, in towns all over the world. When the soundtrack arrived, the work of one composer and one orchestra was recorded for a film that went into thousands of theaters. The union fought back with everything it had, taking out newspaper ads across the country warning against the menace of "canned music," one of them showing a mechanical man tearing the strings out of a harp while an angel wept. They were not fools, and they were not Luddites. They were right. Those pit jobs did not come back. And yet (this is the part we have to be brave enough to admit), sound gave us the movie musical, the modern score, sfx, sound design, audio engineering, and an art form vastly larger than the one it disrupted. And it helped keep Hollywood in the forefront of world entertainment for the rest of the century and into the next. The loss was real. And yet the art form expanded. This is a story that has been told over and over again. To resist technology is to risk irrelevance. Just look at Kodak or Blockbuster. To embrace technology is to open doors of new possibility. Just consider Apple and Netflix. What I Learned From Walt Disney In the mid-1980s, I was tapped to lead Disney's animation division at a moment when the studio was at an inflection point. Animation wasn't just another business unit. It was the soul of the company, a medium revered because of Walt's genius and his passion. But the production system was cumbersome and unforgiving. A single movie was 125,000 individual hand-drawn and painted cels, photographed one frame at a time. Every revision carried a cost measured in months. These degrees of difficulty shaped the kinds of stories we could tell. We found our way forward in an unexpected place: Walt himself. The Disney archives held astonishing recordings of Walt explaining his creative process. His own writings. His notes and storyboards. Work product captured at every stage of his process. This was truly a gift. Listening, reading, sitting with the work itself, we heard him talk about character, about emotion, about how an audience feels when a character truly comes alive. He talked about making bold choices and refining a scene until it genuinely moved people. We didn't hear a word about pencils or paintbrushes. In fact, Walt was famous for being a technologist, forever hunting for state-of-the-art tools, often inventing them himself to achieve the images he saw in his head. But he never defined animation by the tools. He defined it by whether the audience believed the character. His principles were timeless. The tools were not. That realization changed everything. We co-developed the Computer Animation Production System (CAPS) with a young Northern California company called Pixar, replacing hand-painted cels with CGI. In The Little Mermaid, the final scene shimmered with a dimensionality and light that the old process simply couldn't achieve. In Beauty and the Beast, the ballroom sequence moved with a cinematic sweep that placed the audience inside the emotion of the moment. In Aladdin, the Cave of Wonders felt vast and alive, and the Magic Carpet became an intricate, compelling character all its own. In The Lion King, the stampede carried a scale and intensity that raised the emotional stakes beyond anything we'd done before. Technology didn't diminish the craft, it expanded the canvas. It gave artists more room to create. A decade later, the canvas expanded again. When Disney released Pixar's Toy Story, it wasn't simply a technical milestone. It was proof that a fully computer-animated film could carry real emotional weight, that it could make audiences laugh, cry, and believe. At DreamWorks, we made the difficult decision to sunset hand-drawn animation and become a fully computer-animated studio. It was the right thing to do, but it was not without pain. It cost talented people their place in an industry where they had worked their whole lives. Some made the leap to the new tools and did the finest work of their careers. Some never did. Tools are never the point. The instruments change with every generation. What endures is taste and imagination. The magical ability to make an audience feel. One of the greatest storytellers of our generation, George Lucas, succinctly captured the eternal essence of this issue: “It’s not the how, it’s the why.” A Distinction With a Difference I asked one of the leading AI models a question that has been challenging me for months. What is the difference between reasoning and creating? Its answer changed how I think about almost everything happening in this industry. It said . . . Reasoning and creating are two distinct cognitive modes, though they also work together. Reasoning is fundamentally evaluative and analytical. It operates on what already exists: facts, premises, evidence. It moves toward a conclusion that was in a sense already implied by the input. Reasoning is constrained by logic and truth. Its goal is to arrive somewhere correct, not to invent somewhere new. Creating is fundamentally generative. It produces something that didn't exist before. And crucially, there's no single right answer waiting to be found. A blank page has infinite valid responses. Creation involves choices that can't be fully justified by logic alone. Taste, intuition and vision fill the gap where deduction runs out. Reasoning is what Silicon Valley has been perfecting. Creating is what Hollywood has been practicing for more than a century. AI today operates almost entirely on the reasoning side of the line. It can deduce, evaluate, optimize, and pattern-match brilliantly. And while it can create, there is a real distinction to being creative. What it doesn’t yet have is those things that make us human: empathy, devotion, serendipity, the kind of creativity that comes from a person trying to say something only they could say. When the bot generates a piece of art, it is not trying to communicate anything. It is statistics, not soul; it is emulating things that have been done. By contrast, human creativity isn’t about repeating patterns of zeros and ones; it is about doing something new. One day, AI may close this gap. Three years ago, the leaders building AI would have called what they are achieving today, improbable, if not impossible. Impossible is no longer improbable. Today, the line between reasoning and creating is real. Even the leading technologists acknowledge we are not there yet. There is no scientific path to crossing this divide that anyone in the field can articulate today. Understanding that gap is where we will find common ground. A Path Forward In 2016, I closed one chapter in Hollywood with the sale of DreamWorks and opened another in Northern California, co-founding WndrCo. We’ve backed more than 50 founders building the next generation of technology and watched how breakthroughs in Silicon Valley emerge, first as experiments, then as platforms, and finally as infrastructure that reshapes entire industries. It's worth remembering that the last great revolution in animation also came from the north. Pixar was a Northern California company, forged not in the conventions of the Hollywood studio system, but in the technological breakthroughs of Silicon Valley. I've spent years on both sides of this bridge. For sure, I don’t have all the answers (take Quibi, for one!). But, from my past and present vantage points of my long career, here is what I see . . . Brilliant people in Northern California building this technology have made something extraordinary. They have earned the right for the rest of us to be, if not believers, at least optimistic that what comes next will be remarkable. But they have not made an artist. The tools are powerful, but they are not what makes a story matter. That knowledge lives 350 miles to the south, inside people whose life's work has informed the very models you are building. The right path forward includes them by design, with credit, with consent, and with compensation. Build this with the storytellers. Not on top of them. Taste is not something that can be synthesized, it is uniquely human. At the same time, Hollywood needs to accept that AI is not going away. The energy they are spending trying to make it disappear is energy they are not spending deciding the terms on which it will exist. And the terms are everything. The north needs something from it that they cannot build and cannot buy: creativity. The kind that takes a blank page and conjures a single right answer where there was none and has held audiences for a century. Without it, the most powerful reasoning engine ever invented will still be missing the only thing that makes a story worth telling. The artists who learn to wield these new instruments will do things the engineers never dreamed of. They always have. Edison invented the motion picture but made terrible movies. It took Chaplin, Lloyd, Keaton and so many others to make movies emotional. Now, the canvas is about to expand yet again. We should decide now that we intend to paint on it. There are so many valuable lessons in history. This has happened many times before, and it was never settled by the technology. It was settled by the terms. Sousa did not stop the phonograph; he helped write the law that made sure composers got paid. And two years ago, when the writers and the actors walked out, they were fighting for the very things Sousa was fighting for in 1906. Consent, compensation, the basic recognition that human creative work has a price that must be paid. The terms of that fight are still being negotiated, but the principle is older than any of us. The tools-versus-no-tools argument is a trap. First, we must all agree that there should be terms. Then we can have the crucial debate about what fairness requires. What I Learned From Steve Jobs Years ago, Steve Jobs said, "It's in Apple's DNA that technology alone is not enough. It's technology married with the liberal arts, married with the humanities, that yields us the result that makes our hearts sing." He was describing a device. But he could just as easily have been describing this tale of two cities. What I See Coming Soon As the barriers and the costs come down, more films will get made, not fewer. Studios will get to take more risks. There will be more seats at the table, and very soon entirely new forms of storytelling. In the 1980s, animation was dismissed as a niche corner of the business. Today it is one of the most beloved and profitable forms of storytelling in the world. In live action, filmmakers like Steven Spielberg, James Cameron and Peter Jackson embraced new visual tools not as shortcuts, but as instruments, and expanded cinema in the process. Every time storytelling has met a genuine technological shift, from synchronized sound to color to computer animation, it has redefined the boundaries of the medium and grown larger in the process. Assuredly, I don’t have all the answers, but I am confident that the creative opportunities will expand yet again. How we come through this is a choice. The north has the new tools. The south has the creative soul. The best future will draw on the best of both worlds.
811
1,397
7,817
8,004,495
This is what is looks like when Astra is in charge Once I have the scene fully written, and first version generated, I review it, and give notes of everything I would want to fix (you can write sloppily here) The Astra gets to work, using Runway MCP and BeatBandit MCP, working to fix all the issues I flagged. Generating, looking at the results, regenerating, patching. It's all pretty incredible. The Premiere UXP Bridge at the end then edits the fixes back into my Premiere timeline. In the not too distant feature all of this capability will be integrated into Premiere, I believe. (Or whatever the future One App for this will be that people will use.) But this is reality even today as long as you're comfortable with using MCP's.
Fuck it creating a microdrama I can produce good quality drama now in a sane speed + price so doing a chapter (35 minutes). I see some people posting "I made an ad for 8 minutes for 8 euro" and I see other people posting "I made a short film for 20000 euro with a crew of 15 people" - every approach is fine! ... but the middle ground is interesting to me -- what's the best one person can create without still breaking the budget or spending months to do so. If you have a story to tell, can you produce it by yourself? I have 20 minutes created so far -- it's incredibly fun and challenging at the same time. quick teaser from episode 3 The series is fundamentally basically 'people talking' - no dragons, explosions, spacecraft or cameras whipping around people skateboarding -- I do have a feeling that if you're creating anything longer, it's not sustainable. You can't have a story that's always on 11. let's see where this goes why not?
13
9
124
12,190
it's kind of embarrassing to notice how AI has ruined my typing - since you can leave typos in, I just leave them.
4
344
Creative writer experiment with Jev Use Jev to prime what is likely to happen in the story next: pause writing for a second, and it gives you candidates as to how the story would proceed. "unexpected ally", "a small accident", "unease", "use the room", "a telling detail" The fast LLM doesn't need to think, it then just needs to execute. Now, whether this makes any sense is debatable - you can also just ask a fast LLM to do this directly. There's an argument to be made as to this might give better results versus Luna alone just "try to figure it out yourself". Or then this is a bad idea! :)
5
2
39
3,142
Cemetery Service
1
12
1,492
Union Station Hall 30208 x 22656 🧵 see details
3
23
2,374
about 5 hours to render each artwork
4
514
We're Just Normal Men
1
12
1,643
Full text (translated by Astra-xhigh, I'm out of everything else): I Have No Choice but to Bury My Talent in Yesterday A few days ago, DeepSeek v4.1 was released, raising the ceiling of what small models can do by yet another notch. AI has advanced far faster than anyone expected. From the earliest version of ChatGPT, which could do little more than stumble through conversations like a child learning to speak and had a context window of only a few thousand tokens, to reasoning-capable models such as OpenAI o1, DeepSeek R1, and Kimi K1.5 Thinking, took only two short years. From reasoning models to the agents we have today—able to work fluidly with all kinds of tool harnesses, execute commands, and complete complex tasks—has taken only another year and a half. It is hard to imagine what AI will look like another one, two, or three years from now: how powerful it will be, whether it will already have acquired the ability to improve itself, and how deeply it will have spread into areas such as embodied intelligence. AI Is Getting Better and Better at Writing Kernels AI has been advancing just as quickly in my own field: the design and implementation of high-performance kernels. In the space of only a year, it has gone from being a little assistant that could help me look up documentation, read code, and find bugs to something approaching a kernel expert in its own right: capable of reading CUDA, PTX, and SASS code independently, using specialized tools to analyze the stalls associated with individual instructions, and then optimizing kernels on its own. I believe that before long, it will also be able to design kernel schedules independently, evaluate the performance of different scheduling strategies, implement them, and optimize the result. Of course I am proud of DeepSeek v4.1’s success. After all, I wrote its main Attention kernels [1], and the fact that the model performs so well is also, in a sense, a validation of my work. But the times keep moving forward, and no one can stop technological progress. I know very well that in another six months or a year, the kernels written by AI will probably be every bit as good as mine—and perhaps better. AI can reason at 300 tokens a second, type out a command in half a second, and produce a piece of code in twenty seconds. I cannot. AI can keep increasing its model depth, reasoning effort, tool-call budget—the frequency with which it interacts with its environment—and even its degree of parallelism. I cannot. Humanity has never shown much hesitation when it comes to destroying itself. So why, when I know perfectly well that “the better the kernels I write, the faster our new models will train and run inference; the faster the models improve, the sooner I myself will be replaced,” do I still do everything I can to optimize them? Partly because writing kernels is like playing a game to me. I get an enormous amount of pleasure from it. Whenever I invent a new technique, or see one of my kernels become faster, the excitement I feel is no less intense than what a speedrunner feels after breaking their own record. And when I see one of my kernels dramatically outperform the hardware vendor’s official implementation, I feel an equally powerful sense of pride. But there is a more important reason. Even if I simply gave up and started coasting—or deliberately put obstacles in the way to slow down model training—other companies’ models would continue advancing as usual, and in the end they would make me obsolete just the same. “Of course I would rather not be swept away by the revolution. But if I have to be, then I would rather be the one who revolutionizes myself.” When everyone is this determined to engineer their own obsolescence, I have little choice but to join this brutal arms race. And What About Me? When the day really comes that AI is better at writing kernels than I am, what will happen to me then? My own judgment is this: I probably will not lose my job, but I will have to change what I do. I should still be able to make a living. But I may no longer have the chance to do the work I once loved. I once came to a conclusion about the pace of change and my own place in the future. The world is changing so quickly—the development of AI above is a perfect example—that I have no way at all to predict what things will look like five or ten years from now. But whatever happens, I believe that with my breadth of vision, judgment, initiative, and intelligence, I will be able to keep a seat at the table and find my way back to the leading edge of the times. But that conclusion can only reassure me that I will not become unemployed. It cannot reassure me that I will never have to change professions. If anything, it tells me that changing professions may be precisely how I avoid unemployment. And what does changing professions mean? It means giving up the field of kernel design, implementation, and optimization that I have spent so long cultivating and have come to love so deeply, and instead becoming a “mech pilot” for AI agents. Before, three things were largely aligned: what interested me, what I was good at, and what industry needed. Now AI has taken the thing I am good at and become even better at it. At the same time, industry demand has drifted from “people who can write high-performance kernels” to “people who can use AI to produce high-performance kernels faster.” To keep up with what industry needs, I will inevitably have to leave behind the direction I once loved and move into some unknown new one. I believe that with my understanding of engineering, of the requirements of higher-level models, and of low-level hardware, I will still be able to produce high-quality kernels efficiently. I also know that I may come to love this new direction. Or I may not. But there is something genuinely painful about having the thing you love taken away from you. That quiet contentment of sitting at my workstation, settling in, and spending an entire afternoon writing kernels may sing its swan song this summer. I have no choice but to bury my talent in yesterday and become a mech pilot. There are more gears in my hands now, but fewer rhythms in my heart. An analogy might make this easier to picture. Suppose you are a master knitter. You are especially skilled at weaving intricate patterns and matching different colors. The sweaters you make are durable and beautifully patterned, and wealthy people from all the surrounding towns and villages come to ask you to make sweaters for them. You make a good living from it. And you genuinely love the work itself. You love sitting by the window, brewing a pot of tea, looking out at the green hills, clear water, cattle and sheep, and wisps of cooking smoke in the distance, and quietly spending an afternoon knitting. Then one day, someone invents a miraculous machine. Give it yarn and a pattern, and it can automatically knit the sweater for you. The quality and texture are every bit as good as what you could make by hand, and it works far faster than you ever could. You know perfectly well that your peers can use this machine to reach, effortlessly, the level you once spent years attaining. So you have no choice but to use it as well. You also know that with the twenty years of knitting experience you have accumulated, even once everyone has access to the same machine, you will still be able to produce better sweaters, faster, than your peers. But the pleasure of sitting by the window listening to the rain, guiding needle and thread, and letting the hours pass slowly has, in the end, been crushed beneath the roar of the machine. I know there is something deeply helpless about all of this, but there is no real way around it. I can probably keep my livelihood, but I will most likely have to give up an old love. I am the sort of person who keeps reason and emotion fairly compartmentalized. When something needs to be handled rationally, I can be very rational. But I also have a sentimental side. I remember that when I moved out of an apartment I had lived in for a year, I cried hard because I could not bear to part with all the memories tied to that place. Saying goodbye today to the age when kernels were written by hand and optimized in the human mind is undoubtedly more painful still. I do not know whether any readers have felt something similar. But I suppose there is no other way for this to go. And What About Everyone Else? As AI continues to improve, I also find myself worried about a few questions: Are students today increasingly likely to use AI to do their assignments, especially hands-on work such as labs? Imagine having two choices in front of you. One is to spend eight miserable hours struggling through a lab and perhaps not even get full marks. The other is to launch an AI model, spend a few cents and a few minutes, and have it write code that earns full marks for you. Which one are most students going to choose? The point above may leave large numbers of students with seriously underdeveloped engineering ability: the ability to organize code, build systems, anticipate future needs and design for them in advance, create good abstractions, and so on. As AI becomes more capable, will those “engineering skills” still be necessary? Will they gradually become obsolete, the way fluency in handwritten x86 assembly largely has? Or will they remain permanently valuable, like understanding the entire computing stack from software to systems to hardware? If it is the latter, then we may be in trouble. Put AI in the hands of someone with poor engineering judgment, and they can now produce mountains of terrible code several times faster than before, burying all kinds of hidden problems inside systems and making the world even more of a ramshackle operation held together by improvisation. In the society of the future, will power matter more than technical ability or intelligence? Perhaps these are questions that only the times themselves can answer. Conclusion As AI develops, the society of the future may be pulled toward one of two extremes: communism or Cyberpunk 2077. In the former, productive capacity is liberated on an enormous scale, and people’s standard of living rises substantially. (I’ll leave it at that, or I’m afraid this might not make it past moderation.) In the latter, a handful of technology companies control most of society’s resources. Only a tiny number of people have access to the most advanced AI and other technologies and are able to achieve something approaching “mechanical ascension,” while most people are left with only weak, second-rate AI. Moving from one social class to another would become harder and harder: you would first need access to the strongest AI in order to climb the class ladder, creating a self-reinforcing trap. Suppose Anthropic were to retain control of the most advanced AI in the world indefinitely. Which way do you think society would go—communism or 2077? Take a guess. That is why I still believe that frontier intelligence should be made available to everyone openly and affordably. I do not trust Anthropic or OpenAI to do that. In particular, I do not want Anthropic to control the world’s most advanced artificial intelligence or AGI. To put it dramatically, I think the stakes would be comparable to Hitler obtaining the atomic bomb before the Allies did. That is also why I chose to stay at DeepSeek, and why I have continued to stay. We work on AI that is powerful, fast, and accessible to everyone, and we open-source it. Perhaps that can pull the world at least a little farther away from the 2077 end of the spectrum. I hope the world we are heading into turns out all right. May all that is good and beautiful endure. [1] By “main Attention,” I mean only MQA attention with head dim = 512. This does not include the indexer used to select the top-k important tokens. That part was written by other colleagues—who are every bit as skilled—together with their AI agents. ----- Original: 我不得不把才华埋葬在昨天 前几天,DeepSeek v4.1 发布了,将小模型能力的高度又向上推进了一个档次。 AI 发展的速度远远超过了所有人的预期。从那个只会咿呀学语地聊天、上下文长度只有几千 token 的初版 ChatGPT,到具有推理能力的 OpenAI o1、DeepSeek R1 与 Kimi K1.5 Thinking,只不过短短两年;从推理模型到如今能够流畅地在各类 harness 工具中执行命令、完成复杂任务的智能体,也不过一年半。很难想象,倘若再等上一年、两年、三年,彼时的 AI 会成为什么样子,会有多么强大,会不会已经具备了自我进化的能力,并深度渗透进了具身智能等领域。 AI 越来越会写算子了 AI 在我所从事的算子设计、编写这一领域同样进步飞速,在短短一年的时间内,他已经从一个只能帮我查查文档、读读代码、找找 bug 的小助手,蜕变成了一位能够独立阅读 CUDA、PTX 与 SASS 编码、通过专业工具分析每条指令的停顿时间、进而独立优化算子的算子大师。相信在不久的未来,它也能拥有自己独立设计算子调度、评估不同调度方案的性能、将其实现并优化的能力。 我当然为 DeepSeek v4.1 的成功而骄傲 —— 毕竟它的主 Attention 算子都是我写的 [1],它的优秀正是对我的算子的一份肯定。但是,时代的车轮滚滚向前,技术的发展无人能挡。我很清楚,再过上半年或者一年,AI 写的算子大概率就会和我写得同样优秀,甚至将我超越。AI 能一秒思考 300 个 token、半秒敲出一行命令、二十秒写完一份代码,而我不行;AI 能在模型深度、思考强度、工具调用量(和环境交互的频率)、甚至并行度等方面都能不断提升,而我不能。 人类在毁灭自己这件事情上,自古以来都表现得毫不犹豫。为什么在明知“我算子写得越好,我们的新模型的训练、推理速度就会越快,模型能力进步就会更快,我就会更早地被取代”的情况下,我仍然选择尽力优化算子呢?一方面确实是因为写算子对我来说就像打游戏一样,能为我提供极大的快感。我在发明了一种新技术、或者看到自己算子的性能上升的那一刻,心中的激动程度不亚于游戏的速通玩家打破了自己过往的记录。同时,当看到自己的算子的性能远超厂商官方的算子时,我心中也会萌生极大的自豪感。但除此之外,一个更重要的原因是,哪怕我就此“摆烂”甚至故意下绊子耽误模型训练,其它家的模型也会照常发展并最终将我照杀不误。“我当然希望自己不要被革命,但如果非被革命不可的话,我希望革我自己命的人是我自己”。在大家都这么执着于毁灭自己的时候,我也不得不加入这场残酷的军备竞赛。 那我呢 等到 AI 写算子的水平真的高于我的那天,届时的我会怎么样呢? 我的判断是:我不至于会“失业”,但必须要“转业”。我的饭碗尚且能保住,但这可能会导致我再也没机会从事那份我曾热爱过的工作。 我曾经对时代的变化与我个人在未来的处境做出过一个判断:由于时代变化真的太快(上文的 AI 发展就是一个很好的例子),我完全无法预知五年、十年后会发生什么,但不论如何,我相信凭借着自己的眼界、判断力、主观能动性与智力,留在时代的牌桌上,并重新立于时代的潮头。但是,这个判断只能保证我不会“失业”,而无法保证我不需要“转业”,倒不如说这个判断鼓励我通过转业来避免失业。 那转业代表什么呢?它代表着我需要放弃我深耕已久并充满热爱的算子设计、编写、优化领域,转而去做 Agent 的“机甲驾驶员”。在之前,我的兴趣、我所擅长的、以及工业界所需要的,三者是基本对齐的;而现在,AI 让我所擅长的变成了它更擅长的,也让工业界的需求从“会写高性能算子的人”漂移到了“能用 AI 更快地产出高性能算子的人”。为了适应工业界的需求,我势必要放弃之前那个我热爱的方向,转向一个未知的新方向。我相信我能凭借着自己对于工程学、上层模型需求和底层硬件的理解,继续高质量、高效率地产出算子,我也知道我可能会热爱这个新方向(也可能不会),但被夺走热爱的感觉,确实不太好受。那份坐在工位上静心写上一下午算子的清欢,可能会在这个夏天成为绝唱。我不得不把才华埋葬在昨天,去做一位机甲驾驶员。我的手中多了些齿轮,但心中少了些节拍。 可以打个形象的比方:你精通织毛衣技术,尤其擅长各种图案的织造与各色色彩的搭配。你所织出的毛衣质量过硬且花纹美观,十里八乡的富人都来请你为他们织毛衣,你借此赚到了不少钱。同时,你十分享受着那种坐在窗边,沏一壶清茶,望着窗外的青山、绿水、牛羊与炊烟,静静地织上一下午毛衣的感觉。但有一天,有人发明出了一台神奇的机器,只需提供毛线与图案,便可自动织出毛衣,质量与纹理都不亚于你亲手织造的,且速度远快于你。你很清楚,你的同行可以凭着这台机器轻松达到你曾经的水平,因此你不得不也去用它。你也知道,凭借着你过去二十年攒下的织毛衣技术,哪怕大家都有机器,你织毛衣的速度与质量也还能超过同行。但那份临窗听雨、引针穿线、慢度光阴的意趣,终究还是被机器的轰鸣碾碎了。 我知道这很无奈,但没办法。饭碗可以保住,但旧日的热爱大概率是要放弃的。我是一个理性和感性分离得比较开的人,在需要用理性处理问题时可以很理性,但有时也会表现出感性的一面。我记得我在搬离住了一年的出租屋时,还大哭了一场,舍不得和过去的记忆分别。今天和之前那个手写算子、人脑优化的时代告别,无疑比这更加残酷。 不知道有没有读者有类似的感受,但我想这事儿也只能这样了。 那人们呢 在 AI 不断进步的同时,我也对一些问题表示担忧: 现在的学生是不是大概率会更倾向于使用 AI 完成作业,特别是偏向于实践的各种 Lab?想象一下,如果面前有两个选择,一个是苦哈哈地用八小时时间完成一个 Lab,或许还拿不到满分;另一个则是启动 AI 模型,用几毛钱的成本、几分钟的时间,直接让 AI 编写满分代码,那大部分学生会选择哪个呢? 上面一点会导致大量学生的工程能力严重不足,包括组织代码的能力、构建系统的能力、思考未来潜在需求并提前在设计上应对的能力、抽象的能力等等。那么在 AI 能力不断变强的背景下,这部分“工程能力”是否还是必须的呢?这些工程能力是会向旧日的“熟练编写 x86 汇编”的能力那样逐渐被时代抛弃,还是会像“理解从软件到系统再到硬件的整套计算机系统”的能力那样永远具有价值?如果是后者的话,那就危险了 —— 一个工程能力很差的人,在搭配上 AI 后,产出屎山的效率可以达到先前的数倍,进而给系统埋下各式祸患,让这个世界变得更加草台。 在未来社会中,权力(power)是不是会比技术或智商更加重要? 这些问题,或许就需要时代本身来回答了。 结语 伴随着 AI 的发展,未来的社会可能会趋向于两个极端:共产主义与赛博朋克 2077。在前者中,生产力得到极大的解放,人们的生活水平有了明显的提高(就写这些吧不然我怕过不了审);而在后者中,少数科技公司控制着大部分资源,只有极少数人能够使用最先进的 AI 和各式科技,获得接近“机械飞升”的效果,大部分人则只能用上很孱弱的 AI。阶层跨越将越来越难实现:你得先有最强的 AI,才能跨越阶层,形成了一种死循环。 你猜猜如果 Anthropic 公司永远掌握着这个世界上最先进的 AI,未来社会是会变成共产主义还是 2077 呢?你猜? 所以,我还是相信,最前沿的智能应该以一种开放、廉价的方式,供应给所有人。我不信任 Anthropic 或者 OpenAI 能这样做,特别是不希望 Anthropic 掌握最先进的人工智能或 AGI,夸张点说其严重性不亚于让希特勒先于盟军掌握原子弹技术。这也是为什么我选择并坚持留在了 DeepSeek:我们研究强大、快速、普惠的人工智能并将其开源,或许能把世界从 2077 那端拉回来一些。 愿未来的世界一切安好。May all the beauty be blessed. [1] “主 Attention”仅包括 head dim = 512 的 MQA attention,不包括用于选出 top-k 重要的 token 的 indexer,那部分是由其他(水平也非常强的)同事(以及他们的 AI Agent)编写的。
93
443
2,630
996,959
Coaling The Steamer
2
17
1,499
Thaw On The River
1
5
194
Searchlights
2
144