What I cannot create, I do not understand.
― Richard Feynman
The Richard Feynman quote “what I cannot create I do not understand” appeared on his chalkboard at his death. I realized that I have generally run my professional life by this principle. It isn’t enough merely to demonstrate rote knowledge; one needs to understand the principles underlying the knowledge. One way to demonstrate the mastery over knowledge is utilize the current knowledge and then extend the knowledge in that area to something new. This gets to the core of our current problem in science, we are not being asked to extend knowledge, and we are asked to curate knowledge. As a result we are losing the ownership that denotes mastery.
A core example of the issue is the capability to completely understand the material they are responsible for. If we are implementing things in a computer code can we rederive all the expression in the code? Do we understand the assumptions, conditions and caveats with the actual expressions? Can we extend or modify the expressions as the situation calls for it? Today in very many cases the answers to these questions is no. This is true for models we use, methods used for solution and algorithms we depend upon. This broad-based lack of intellectual ownership is a direct threat to our ability to ably provide stewardship of our missions dependent upon these products.
I learned very early the difference between knowing the name of something and knowing something.
― Richard Feynman
I want to be very clear about what I’m commenting on, we have substantial intellectual ownership of the code we write, but not necessarily what that code does. What we often do not have ownership of is the contents of that code, the models, the methods solving those models, and the algorithms the methods are based upon. We own the contents of the implementation, the software libraries, and the mapping of all of these to modern computing architectures. Because of the demise of Moore’s law we are exploring a myriad of extremely exotic computing approaches. These exotic computer architectures are causing implementations to become similarly exotic. In a sense my concern is that the difficulty of simply using these computers has the effect of sucking “all the oxygen” from the system and leaves precious little resource behind for any other creative endeavor, or risk taking. As a result we have no real progress being made in any of the activities in modeling and simulation beyond mere implementations.
A large part of my argument hinges upon the intellectual core of the value proposition associated with modeling and simulation. The question is whether progress in modeling and simulation is most greatly benefitted by greater computing power? Or improved models? Or improved methods? Or improved algorithms? The answers to these questions are not uniform by any means. There are times when the greatest need for modeling and simulation is the capacity of the computing hardware. At other times the models, methods or algorithms are the limiting factors. The question we should answer is what is the limiting factor today? It is not computing hardware. While we can always use more computing power, it is not limiting us today. I believe we are far more limited by our models of reality, and the manner in which we create, analyze and assess these models. Despite this lack of need for improved hardware, computing hardware is the focus of our efforts.
The program I work under is called stockpile stewardship. We act far more like stockpile curators. The general state of affairs is rapidly evolving to a state where no one really has the intellectual ownership of the content of their work. There is frightfully little freedom in defining a path toward greater understanding. The inertia of the “this is the way things are done around here” is so strong that it derails a great degree of progress. Intellectual ownership and progress are intimately related. The true ownership of knowledge and the capacity to produce progress are often, if not almost always one and the same.
In computational modeling we are largely in the business of stewarding legacy codes full of knowledge being curated. The choice of legacy codes is predicated on the ability of the codes to simulate issues of interest from an application point of view. It is a safe way to proceed in the short term while immensely dangerous in the long term. These codes are immensely complicated and full of wide swaths of both explicit and implicit knowledge. In many ways the models, methods and algorithms in the code are most completely documented in code, and other forms of documentation are woefully incomplete. Much of the knowledge of the decisions made in defining the code is contained only in the head’s of the people who originally wrote the code. Over time those people go away and the logic and rationale for the code’s form and function begins to fade away. We often find that certain things in the code can never be changed lest the code become non-functional. We are left with something that looks and feels like magic. It works and we don’t know why or understand how it works, but it does.
As I stated above I run my professional like by recreating existing the things. This makes dead certain that I understand something, and the only way to validate this understand is to move past the strictures and barriers of existing understanding. To truly own knowledge is equivalent to expanding that knowledge. You confirm your ownership of knowledge by the process of creation. This creative process is part of the essence of research. Thus by the virtue of the creation of legacy codes we are confirming the lack of research in the areas of knowledge vital to stewarding our stockpile.
I think nature’s imagination Is so much greater than man’s, she’s never going to let us relax.
― Richard Feynman
With the stewardship program we are in a very precarious position. We are not allowed to design new weapons. We are not allowed to fully test our ideas either. An important part of the space of knowledge is taken off the table and relegated to being purely curated. This full demonstration has the role of providing an important feedback of reality to the work being done. Reality is very good at injecting humility into the system when it is most needed. When the knowledge is curated we rapidly remove important and essential aspects of stewardship. We have immense issues associated with the long-term responsibility of caring for a stockpile. New issues arise that are beyond the set of conditions the systems were originally designed for. All of this needs a fertile intellectual environment to be properly stewarded. We are not doing this today. Instead the intellectual environment is actually being steadily eroded in favor of curating knowledge. In computing, the creation of legacy codes is a key symptom of this environment.
It doesn’t matter how beautiful your theory is, it doesn’t matter how smart you are. If it doesn’t agree with experiment, it’s wrong.
― Richard Feynman
In taking the raw intellectual material going into a production code many things must be simultaneously integrated (by production code I mean the codes we use to do serious application modeling for applied programs, like stockpile stewardship). This integration of intellectual material includes fundamental models, the closure of those models, the methods of solving those models and the utilization of algorithms for those methods. Furthermore there is a specific implementation in computer code that is suitable or better yet, optimized for the computing hardware.
A number of tricks, or practical accommodations are made in getting the models, methods and algorithms to work effectively for solving “real” problems with all of their dirty and realistic aspects. Most of these tricks are not something people are proud of, or can even explain in a coherent way. In many cases, the trick simply works and often a number of other tricks were tried first, and didn’t produce the desired outcome. The trick itself is rarely documented as such, and the tricks that didn’t work are usually undocumented. Many of the tricks are far more obvious and logical to use, and their failure is usually unexplained. Hence the production code works on the basis of tricks of the trade that are often history dependent, and rarely explained, yet utterly essential.
It is this point where the real horror show of legacy codes unfolds. The production code becomes a key aspect of executing an applied program, and the tricks necessary to make the code work are encoded into all of the results. The author of the tricks gets older and ultimately retires or dies, or gets a better job. When this happens the tricks become part of legend or lore and their capacity to make the code work achieves a magical status. The code’s results depend on all of the tricks, and the custody of the code passes to new people. These new people can’t change the tricks because they don’t understand the tricks. In very many cases, the tricks don’t look any different than the rest of the code and are indistinguishable from the parts of the methodology that are coherent and logical. It is all one big mess of coherent and incoherent ideas that simply gets stewarded much in the manner that monks steward old religious texts.
This is a generically awful situation that happens over and over again. In the programs I work for it is the standard way things unfold. The reason this happens is because creating a new production code is a risky thing. Most of the time the effort fails. The creation of the code requires a good environment that nurtures the effort. If the environment is not biased toward replacing older codes with new codes (i.e., progress and improving technology), the inertia of the status quo will almost invariably win. This inertia is based on the very human tendency to base correctness on what you are already doing. The current answer has a great deal greater propriety than the new answer. In many cases the results of existing codes provide the strongest and clearest mental image of what a phenomena looks like to people utilizing modeling and simulation especially in fields where experimental visuals do not exist.
For a successful technology, reality must take precedence over public relations, for nature cannot be fooled.
― Richard Feynman
Why should things be easy to understand?
― Thomas Pynchon