Physics is becoming so unbelievably complex that it is taking longer and longer to train a physicist. It is taking so long, in fact, to train a physicist to the place where he understands the nature of physical problems that he is already too old to solve them.
— Eugene Paul Wigner
The past decade has seen the rise of commercial modeling and simulation tools with seemingly great capabilities. Computer power has opened vistas of simulation to the common engineer and scientist. Advances in other related technologies like visualization have provided an increasingly “turn-key” experience to users who can do seemingly credible cutting-edge work on their laptops. These advances also carry with them some real dangers most acutely summarized as a “black box” mentality toward the entire modeling and simulation enterprise.
artificial intelligence is no match for natural stupidity
Black box thinking causes problems because people get answers without understanding how those answers are arrived at. When problems are simple and straightforward this can work, but as soon as the problems become difficult issues arise. The models and methods in a code can do funny things that only make sense knowing the inner workings of the code. The black box point of view usually comes with too much trust of what the code is doing. This can cause people to accept solutions that really should have been subjected to far more scrutiny.
The missing element in the black box mentality is the sense of collaboration needed to make modeling and simulation work. The best work is always collaborative including elements of computation and modeling, but also experiments, mathematics and its practical applications. This degree of multi-disciplinary work is strongly discouraged today. Ironically, the demands of cost accounting often work steadfastly to undermine the quality of work by dividing people and their efforts into tidy bins. The drive to make everything accountable discourages the ability to conduct work in the best way possible. Instead our current system of management encourages the black box mentality.
Another force for pushing black box thinking is education. Students now run codes whose interface is easy enough to bear some resemblance to video games. Of course with a generation of scientists and engineers raised on video games this could be quite powerful. At the same time the details of the codes are not generally emphasized, and instead they tend to be viewed as black boxes. In classes when the details of the codes are unveiled, eyes glaze over and it becomes clear that the only thing they are really interested in is getting results, not knowing how the results were arrived at.
One way this current trend is being blunted is the adoption of verification and validation (V&V) in modeling and simulation. V&V encourages a distinct multidisciplinary point-of-view in its execution particularly when coupled to uncertainty quantification. To do V&V correctly requires a significant amount of deep knowledge of many technical areas. This is really difficult. Instead of engaging deeply in the technical work necessary for good V&V is simply beyond the capacity of most people’s capabilities and tolerance for effort. People paying for modeling and simulation for the most part are unwilling to pay for good V&V. They would rather have V&V that is cheap and fools people into confidence.
Computers are incredibly fast, accurate, and stupid: humans are incredibly slow, inaccurate and brilliant; together they are powerful beyond imagination.
― Albert Einstein
Two elements are leading to this problem. No one is willing to pay for high-quality technical work either the development or use of simulation codes. Additionally no one is willing to pay for the developers of the code and the users to work together. The funding, environment and tolerance to support the sort of multi-disciplinary activities that produce good modeling and simulation (and by virtue of that goo V&V) is shrinking with each passing year. Developing professionals who do this sort of work well is really expensive and time-consuming. When the edict is to simply crank out calculations with a turnkey technology, the appetite for running issues to ground necessary for quality simply doesn’t exist.
A couple of issues have really “poisoned the well” of modeling and simulation. The belief is that the technology is completely trustworthy and mature enough for novices to use is an illusion. Commercial codes are certainly useful, but need to be used with skill and care by curious, doubtful users. These codes often place a serious premium on robustness over accuracy, and cut lots of corners to keep their users happy. A happy user is usually first and foremost someone with a completed calculation regardless of the credibility of that calculation. We also believe that everything is deeply enabled by almost limitless computing power.
Think? Why think! We have computers to do that for us.
— Jean Rostand
Computing power doesn’t relieve us of the responsibility to think about what we are doing. We should stop believing that the computational tools can be used like magic, black magic in black boxes that we don’t understand. If you don’t understand how you got your answers, you probably shouldn’t trust that answer until you do.
A computer lets you make more mistakes faster than any other invention with the possible exceptions of handguns and Tequila.
— Mitch Ratcliffe