When you stop growing you start dying.
― William S. Burroughs
Moore’s law isn’t a law, but rather an empirical observation that has held sway for far longer than could have been imagined fifty years ago. In some way shape or form, Moore’s law has provided a powerful narrative for the triumph of computer technology in our modern World. For a while it seemed almost magical in its gift of massive growth in computing power over the scant passage of time. Like all good things, it will come to an end, and soon if not already.
Its death is an inevitable event, and those who have become overly reliant upon its bounty are quaking in their shoes. For the vast majority of society Moore’s law has already faded away. Our phones and personal computers no longer become obsolete due to raw performance every two or three years. Today obsolescence comes from software, or advances in the hardware’s capability to be miserly with power (longer battery life on your phone!). Scientific computing remains fully in the grip of Moore’s law fever! Much in the same way that death is ugly and expensive for people, the death of Moore’s law for scientific computing will be the same.
Nothing can last forever. There isn’t any memory, no matter how intense, that doesn’t fade out at last.
― Juan Rulfo
One of the most pernicious and difficult problems with health care is end of life treatment (especially in the USA). An enormous portion of the money spent on a person’s health care is focused on the end of life (25% or more). Quite often these expenses actually harm people and reduce their quality of life. Rarely do the expensive treatments have a significant impact on the outcomes, yet we spend the money because death is so scary and final. The question I’m asking is whether we are about to do exactly the same thing with Moore’s law in scientific computing?
Moore’s law is certainly going to end. In practical terms it may already be dead with it holding only in the case of completely impractical stunt calculations. If one looks at the scaling of calculations with modest practical importance such as the direct numerical simulation of turbulence the conclusion is that Moore’s law has passed away. The growth in capability has simply fallen dramatically off the pace we would expect from Moore’s law. If one looks at the rhetoric in the national exascale initiative, the opposite case is made. We are going forward guns blazing. The issue is just the same as end of life care for people, is the cost worth the benefit?
Its hard to die. Harder to live
― Dan Simmons
The computers that are envisioned for the next decade are monstrosities. They are impractical and sure to be nearly impossible to use. They will be unreliable. They will be horrors to program. Almost everything about these computers is utterly repulsive to contemplate. Most of all these computers will be immense challenges to conduct any practical work on. Despite all these obvious problems we are going to spend vast sums of money acquiring these computers. All of this stupidity will be in pursuit of a vacuous goal of the fastest computer. It will only be the fastest in terms of a meaningless benchmark too.
Real dishes break. That’s how you know they’re real.
― Marty Rubin
For those of us doing real practical work on computers this program is a disaster. Even doing the same things we do today will be harder and more expensive. It is likely that the practical work will get harder to complete and more difficult to be sure of. Real gains in throughput are likely to be far less than the reported gains in performance attributed to the new computers too. In sum the program will almost certainly be a massive waste of money. The plan is for most of the money going to the hardware and the hardware vendors (should I think corporate welfare?). All of this will be done to squeeze another 7 to 10 years of life out of Moore’s law even though the patient is metaphorically in a coma already.
The bottom line is that the people running our scientific computing programs think that they can sell hardware. The parts of scientific computing where the value comes from can’t be persuasively sold. As a result modeling, methods, algorithms and all the things that make scientific computing actually worth doing are starved for support. Worse yet, the support they do receive is completely swallowed up by trying to simply make current models, methods and algorithms work on the monstrous computers we are buying.
What would be a better path for us to take?
Let Moore’s law die, hold a wake and chart a new path. Instead of building computers to be fast, build them to be useful and easy to use. Start focusing some real energy on modeling, methods and algorithms. Instead of solving the problems we had in scientific computing from 1990, start working toward methodologies that solve tomorrow’s problems. All the things we are ignoring have the capacity to add much more value than our present path.
For nothing is evil in the beginning.
― J.R.R. Tolkien
The irony of this entire saga is that computing could mean so much more to society if we valued the computers themselves less. If we simply embraced the evitable death of Moore’s law we could open the doors to innovation in computing instead of killing it in pursuit of a foolish and wasteful extension of its hold.
The most optimistic part of life is its mortality… God is a real genius.
― Rana Zahid Iqbal