Nice, but ;-)
There is one nice point in the paper about how geniuses work. I happened to run into a few Turing award winners and more than one Nobel laureate, and the way I see their work is not "they changed the question." The way I see their work is: they realized that the current state-of-the-art has arrived at the end of a dead end and backtracked.
Admittedly, that is, I believe, what only humans can do. Or maybe we should give it a try and check with one of the research assistants and see whether they can get out of such a concrete dead end if we ask them to.
But never mind how you get out of the dead end, you need to get into a new corridor, and THAT is something an AI can accelerate marvelously. More or less: provided the right guidance, it can re-arrange the giants whose shoulders we are standing upon. It can help us find not just a needle in a haystack, but almost any number of matching needles in a needle stack. We just have to ask the right questions, give the right guidance.
So... it's not going to be AI alone who accelerates developments, it's going to be human+AI.
I think of my AIs as my ghostwriters, ghost-researchers etc.
Necessarily, our AIs (LLMs is what I'm mostly thinking) are, in a way, the average of all human knowledge, not the outlier, the weirdo, the guy going against the flow. We are spending OUTRAGEOUS amounts of compute to make them go WITH the flow, and that's hard enough.
BUT if we humans add that missing 1% of weirdness that takes a field into a new corridor, we'll be flying down the corridor.
For example: If Einstein had had AI, he would still have to say "the speed of light is constant in all frames of reference" in order to break out of conventional thinking. But then he can hand it off: "dear AI, what would that mean? How would that change physics? Ask me everything you need to know." and I think he'd be in for a hell of a ride. That's at least my experience.
Too true to be good...
