In preparing for battle I have always found that plans are useless, but planning is indispensable.
― Dwight D. Eisenhower
Many of you might be familiar with the PCMM (Predictive Capability Maturity Model). Most commonly it is thought of as structured way to assess (i.e., judge) the quality of a computational investigation. Since no one likes to be judged, it isn’t greeted with the greatest warmth. I’d like to provide another way of looking at PCMM, as a rubric for doing computational science.
In this sense it provides a catalog of things that you should worry about, decide on your intent and shape the work to meet specific goals and intended uses. The concept of broad categories of competence with graded levels of quality allows one to make sure all the necessary activities are being done. Too often a program will be executed almost solely within a single category and the other aspects of computation would be cobbled together. The intended quality of the work is an afterthought.
With the traditional structure of the PCMM as a starting point, there is no problem in removing or adding new categories to suit the purposes of the project. The usual set of categories: geometry/representation, physical modeling, code/software verification, solution verification/error estimation, validation, and uncertainty quantification are a good place to start, but inadequate for many projects. An example might be weather or climate modeling where data assimilation is important enough to warrant its own category. In computation of social science, the geometry is irrelevant and needs to be replaced with an appropriate description of the environment things like agents are placed in. In other cases the experimental work is sufficiently complex and focused that it should be expanded into far greater detail included a data focus. The point is that PCMM is not a fixed framework, but an idea of how to organize your activity as to not leave important things out.
An idea can only become a reality once it is broken down into organized, actionable elements.
― Scott Belsky
How might I use PCMM to do something that isn’t V&V related first? Say, like writing a new code?
I would consider what the application of the code is intended to be and how much further than the original intent might be supported? How essential is the geometric fidelity to the quality of simulations? How well are the basic physical models, and supporting constitutive relations established? Is the numerical method and the equations supported by mathematical rigor? Are numerical errors well understood where the equations and method are to be applied? What experiment exist for validation, and will new validation experiments be conducted? What sort of quantities of interest are needed and how will their uncertainty be assessed? For every question how critical is the quality of the answer, and what is the level of decision to be made with the results? Might any of this change over time, and can those changes be accommodated in the desired code?
Notice that this is the application of PCMM classic in the form of a set of questions to be asked at the outset of the work. If these questions are proactively addressed the code will be able to be assessed for V&V quality with great ease. Ultimately this should lead to an appropriate statement of the computational credibility of the code and its suitability for intended use.
The key is that PCMM isn’t just for V&V any more, and if it is used proactively V&V can be so much easier.
The essence of community, its heart and soul, is the non-monetary exchange of value; things we do and share because we care for others, and for the good of the place.
― Dee Hock