I read a lot. Technical papers. Blogs. You name it. The other day I came across an article that didn’t grab me immediately, but I soldiered on, and deep into the article (http://pchiusano.github.io/2014-10-13/worseisworse.html, I mentioned it yesterday where it describes the issues with applied mathematics) a thought arose in me, “this is some good shit!” The concepts applied so much more broadly than the original and primary focus on software, but to broad swaths of science, technology, business and the human existence in general.
The recent blog post rightly focused on the problems with culture and a bit more commentary might be useful. The core of the problem is one of local optimization. One can choose a solution that is optimal for one’s self or a small group that has awful consequences more broadly. If the implicit or explicit reward system isn’t carefully constructed and the consideration of the global consequences are ignored, something comfortable, but ultimately awful will be allowed to persist as a solution. For software this is common because of its niche status even today. For most organizations a legacy system provides value today and they aren’t willing to pay or wait for a better solution. This is part of essence of “worse is better”. It is part of the reason that the status quo always has a “home field advantage”.
The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.
George Bernard Shaw
At a deeper level the issue is associated with the notion of winner and losers in any resource battle. While the current economy is demonstrably worse for a vast majority of Americans (or Westerners), it is much better for a select cadre of the wealthiest. These wealthy are powerful and work to influence the gears of politics and power to maintain this status quo. It is difficult to move toward a different system when those with true power oppose it. This is in spite of a general conclusion that a more equitable system that favored a much larger segment of the population would actually result in more for most of the wealthy too. Decisions favor the incumbent. It is always more clear to defend what you already have rather than what you might have.
There are three classes of men; the retrograde, the stationary and the progressive.
― Johann Kaspar Lavater
These forces are always working against progress. Usually the thing that represents progress (a new language, new code, new theory…) is actually lower in performance than the incumbent, which has been optimized (or calibrated). Those who are not invested in the progress will usually choose the status quo for completely rational reasons including its performance, and the relative comfort of the known product. Thus for many, if not most, worse is really better (or easier). If the system is not carefully designed to blunt this outcome, progress will be especially painful. When the system is designed to stop progress, progress is almost impossible.
Better never means better for everyone… It always means worse, for some.
There can be no progress without head-on confrontation.
For reference there is a Wikipedia entry, and some reflections on Gabriel’s original essay: