The linear model of innovation says progress starts at universities, then moves to industry, then reaches customers.
In a recent talk,
@cmuratori presented the 1960s software crisis. Dijkstra, a researcher and professor who actually changed practice, was depressed. In 1972, he put it this way:
« As long as there were no machines, programming was no problem at all; when we had a few weak computers, programming became a mild problem; and now we have gigantic computers, programming has become an equally gigantic problem. »
He wanted more planning and more formal verification.
Universities pushed that line for decades, at great expense.
It was the wrong bet, for reasons practitioners such as Fred Brooks could see. A complete formal spec of what the software ought to do sounds good in theory. In practice you often discover the requirement by trying ideas and iterating.
We are much better today at building reliable software (although this knowledge is not as widespread as I would hope). The progress took many years, and was driven by several actors, only few of them being high IQ professors.
We are in a new mess. AI-assisted programming produces what people call slop. Some are depressed. Others push forward. I do not know where we should go.
This innovation (AI programming) was NOT designed on campus. Programmers glued large language models to their tools and got agentic AI.
Now we have to think it through. A professor may invent a framing that changes how we see agents. Or not.
That is why the linear model is false. Innovation does not systematically start on campus. Customers and working programmers often matter more than intellectuals.
You may say software is a special case, not nuclear physics. Is it?
If analysis was the foundation of science in the early 20th century, software is the foundation of science in the early 21st. Software ate the world, including science.
@ericweinstein recently argued on the All-In podcast that we should give selected scientists so much money they cannot be managed, and wait for another Einstein. Fine. But in 1905 Einstein was a patent clerk, not a professor with infinite funding. Princeton came later. It is not obvious that the later work was better.
My home country, Canada, has programs that look like what Eric is advocating. There are hundreds of Canadian professors who have the equivalent of what Eric advocates. Some have their own buildings, dedicated to their work.
Whether that turned them into breakthrough producers is another story.
You cannot engineer great software with a neat up-front design. Trial and error is all you have. The same is true of breakthrough science. Nobody is smart enough to design a government program that reliably produces new Einsteins. Your program may soon be captured.
Around 1905, a patent clerk reinvented physics while two bicycle mechanics built an airplane. Stop looking only up. The next big thing often comes from misfits your program would have excluded.
We have to accept the fact that progress is somewhat unpredictable and that it cannot be engineered up front. Cool down the hubris and let us get serious.
Links
•Dijkstra, The Humble Programmer (1972):
cs.utexas.edu/~EWD/ewd03xx/E…
•Brooks, The Mythical Man-Month (1975)
•Casey:
piped.video/watch?v=hpj6r6Cj…
•Eric:
piped.video/watch?v=y84ypzEL…