What is ironic is that at the origin of the socialist / communist / anarchist movements (which were not at the time clearly distinguished and often overlapped) in the latter half of the 19th century, many of their founders and adherents were admirers of science and engineering and their accomplishments in mass production, construction, and efficiency through division of labour. They argued that the same principles of scientific rationalisation could and should be applied to society, replacing antiquated institutions passed down from antiquity, resulting in the same kind of improvements as had been achieved in the Industrial Revolution.
But what they failed to see is that the processes of science and engineering do not flow from theory down but from observation, trial and error, competition among different approaches, and incremental correction based upon experience. The would be social engineers were looking at the results of a long process and assuming they could jump directly there without the extended process of experimentation that produced those results.
Unfortunately, and often tragically, more than a century of failure does not seem to have taught them that it doesn’t work that way. Because of the local knowledge problem, the only way to make a large, complex system composed of intelligent actors who are always adjusting their behaviour based upon the evolution of the system is a distributed, organic, adaptive mechanism such as, for example, the free market in economics. Anything imposed from the top down will, before long, blow up because it will provoke unforeseen consequences and reactions that nullify its intentions.
What is dismaying is that this top-down approach has leaked from the realm of bad public policy into big science and big engineering, where what I’ve been calling the “cult of design” since the 1980s, where theory, untethered from experience and decoupled from feedback, has the arrogance to overrule the distributed wisdom and free choice of individuals.
At the same time, engineers building complex, distributed data networks and computer operating systems have learned from experience that top-down designs work poorly and do not scale gracefully, and that decision-making must be pushed down as close as possible to the transaction level just as it is in a free market. If only that experience and wisdom were able to percolate back into the realm of “public policy”.