The electric light did not come from the continuous improvement of candles. It did not come from gas companies making better mantles. It came from a different axis of value entirely — cleanliness, safety, convenience. The early bulbs were fragile, unreliable, expensive, and had no distribution infrastructure. On every metric the gas companies measured, the electric light was worse. By 1885, Edison held 75% of the U.S. market. The gas mantle was irrelevant.
Why did the gas companies not see it coming? Why did they not respond? Why did the disruption feel sudden when it had been building for years? Three thinkers, three pieces of the answer. Schumpeter named the force. Christensen named the mechanism. Thom named the moment. Together, they explain why most software "revolutions" are just better candles — and what the real electric light might be.
Schumpeter: the perennial gale
Joseph Schumpeter published Capitalism, Socialism, and Democracy in 1942. Chapter VII — six pages — introduced the concept that now bears his name.
"The fundamental impulse that sets and keeps the capitalist engine in motion comes from the new consumers' goods, the new methods of production or transportation, the new markets, the new forms of industrial organization that capitalist enterprise creates."
This is not an abnormal state. It is the normal state. Capitalism is change. Stability is the illusion.
"This process of Creative Destruction is the essential fact about capitalism. It is what capitalism consists in and what every capitalist concern has got to live in."
Creative Destruction. Not "innovation." Not "disruption." Destruction — and creation. Simultaneous. Inseparable. The new does not arrive after the old departs. The new is the departure of the old. You cannot have the electric light without destroying the gas industry. The destruction is not a side effect. It is the mechanism.
"Every piece of business strategy acquires its true significance only against the background of that process and within the situation created by it. It must be seen in its role in the perennial gale of creative destruction; it cannot be understood irrespective of it or, in fact, on the hypothesis that there is a perennial lull."
The perennial gale. Not a storm that passes. The wind that always blows. Strategy that assumes calm is strategy that assumes away the defining property of the environment. Most software architecture strategy assumes calm. The assumption is wrong.
"In capitalist reality as distinguished from its textbook picture, it is not that kind of competition which counts but the competition from the new commodity, the new technology, the new source of supply, the new type of organization — competition which strikes not at the margins of the profits and the outputs of the existing firms but at their foundations and their very lives. This kind of competition is as much more effective than the other as a bombardment is in comparison with forcing a door."
A bombardment, not forcing a door. Incumbents defend the door. The disruptor bombs the foundations. The door is irrelevant. The gas companies fortified their doors — better mantles, better distribution, better customer relationships. Edison bombed the foundations — a different kind of light, a different distribution model, a different value proposition. The gas companies never lost a door battle. They lost the foundation while they were reinforcing the door.
Schumpeter gives us the why. Why does the perennial gale blow? Because the fundamental impulse of the system is creation-through-destruction. The new good, the new method, the new market. Not better performance on existing metrics. New metrics. New markets. New foundations.
Christensen: the mechanism
If Schumpeter named the force, Clayton Christensen named the mechanism. The Innovator's Dilemma (1997) explains how great firms fail despite doing everything right.
"Generally, disruptive innovations were technologically straightforward, consisting of off-the-shelf components put together in a product architecture that was often simpler than prior approaches. They offered less of what customers in established markets wanted and so could rarely be initially employed there. They offered a different package of attributes valued only in emerging markets remote from, and unimportant to, the mainstream."
The electric light was simpler than the gas infrastructure. It offered less of what gas customers wanted — brightness, reliability, cost. It offered a different package valued by people who weren't gas customers. It was worse on the incumbents' metrics. Better on its own. The incumbents' customers didn't want it. The incumbents listened to their customers. They were destroyed.
"It was as if the leading firms were held captive by their customers."
"Blindly following the maxim that managers should keep close to their customers can be a fatal mistake."
The captivity is structural. The firm's resource-allocation processes reward sustaining innovations — better products for existing customers at higher margins. They penalize disruptive investments — worse products for nonexistent customers at lower margins. The managers are rational. The processes are rational. The outcome is fatal.
"The way decisions get made in successful organizations sows the seeds of eventual failure."
Christensen's prescription: create independent organizations, small enough to be excited by small markets, shielded from the parent's customers and cost structures. The disruption cannot be managed within the incumbent. It must be separated. The organizational structure is the problem. The organizational structure must be changed.
"Disruptive technology should be framed as a marketing challenge, not a technological one."
The technology is simple. The market is hard. The electric light was not a technology problem. Edison solved the filament in 1879. The problem was building a market that didn't exist for a product that was worse than the alternative on every dimension the existing market measured. That took twelve years to turn a profit. The incumbents' processes couldn't tolerate twelve years of losses on an inferior product. The disruptor's could. That is the mechanism.
Thom: the moment
If Schumpeter named the force and Christensen named the mechanism, René Thom named the moment. Catastrophe theory — introduced in Structural Stability and Morphogenesis (1972) — is the mathematics of discontinuous change. Continuously changing forces. Sudden, discontinuous effects.
"Catastrophe theory favors a dialectical, Heraclitean view of the universe, of a world which is the continual theatre of the battle between 'logoi,' between archetypes." — Thom
The universe as a theater of conflict between fundamental forms. Quantitative changes accumulate. Qualitative transformation erupts. The gas industry accumulated quantitative improvements — better mantles, better burners, better distribution — while the qualitative threat accumulated invisibly in a different market. The gas industry looked stable. The control variables were shifting. The catastrophe was approaching. Nobody could see it because the metrics they tracked were the old metrics. The new metric — "percentage of homes with electrical wiring" — was not on their dashboard.
The cusp catastrophe is the simplest model. One behavior variable. Two control factors: a normal factor and a splitting factor. As the splitting factor increases, the system develops two possible stable states. A small change in the normal factor can cause a sudden jump between them. The system exhibits five properties:
Bimodality. Two stable states exist simultaneously. Gas lighting and electric lighting coexisted for years. The market could support both. Until it couldn't.
Catastrophe. The jump between states is sudden. The market didn't shift gradually from gas to electric. It flipped. One year, gas was dominant. A few years later, it was irrelevant. The transition was not linear.
Hysteresis. The jump-down point differs from the jump-back-up point. Once the market flipped to electric, returning to gas would require electric to become much worse than gas was when gas was dominant. The threshold for switching back is higher than the threshold for switching forward. The new equilibrium is sticky. Disruption, once complete, is hard to reverse.
Inaccessibility. Intermediate states are unstable. You cannot be half-gas and half-electric for long. The transition is not a smooth gradient. It is a jump. The organization that tries to do both — sustain the old while developing the new — is in the inaccessible region. It falls to one side or the other. Most fall to the old side. The old side has revenue, customers, and organizational gravity.
Divergence. Small initial differences lead to dramatically different outcomes. Two gas companies, identical in 1879. One experiments with electric lighting, creates an independent division, shields it from the main business. The other doubles down on mantles. In 1885, one is an electric company. The other is bankrupt. The initial difference was small. The final divergence was total.
E.C. Zeeman, who popularized catastrophe theory in the 1970s, described the pattern directly:
"A gradual change in the control can cause a catastrophic sudden change in behavior. In all of nature we observe continuous changes giving rise to discontinuous jumps. In economics, a gradual relaxation after compression can cause a sudden inflationary explosion. People suddenly change opinion, and suddenly lose composure. Nations suddenly go to war."
Gradual change in the control variables. Catastrophic jump in the behavior. This is the moment Christensen's incumbents miss. They track the control variables they understand. The splitting factor — the new technology's trajectory, the new market's growth, the new axis of value — accumulates invisibly. The normal factor — market share in the old market, customer satisfaction among existing customers — looks fine. The system appears stable. The catastrophe is already determined. It just hasn't happened yet.
The three lenses on software
Schumpeter, Christensen, and Thom give us three ways to see software architecture decisions.
Schumpeter's lens: what is the perennial gale doing to your stack?
"This process of Creative Destruction is the essential fact about capitalism."
The essential fact about software is that the environments E-type systems serve are in perennial gale. Lehman's Law I — continuing change — is Schumpeter restated for code. The system must change or die because the world it models is being creatively destroyed. The new business model, the new regulation, the new user behavior — each is a Schumpeterian innovation that destroys the old assumptions the code was built on. The code does not age. The assumptions age. The assumptions are being bombarded. The code sits on the foundations.
Christensen's lens: are you improving candles or installing electric light?
Most software "innovation" is sustaining. Better monoliths. Better microservices. Better CI/CD. Better gas mantles. The customers — internal teams, product managers, the business — want better mantles. They reward better mantles. The engineers who build better mantles get promoted. The organization is held captive by its customers exactly as Christensen described.
The electric light — the true disruption — would serve non-consumers. Applications that can't be built under the current model. Users who can't afford the current model. Problems too small to justify the current cost structure. Dark factories may be electric light. Serverless was supposed to be. Most microservices migrations were better mantles on a monolith architecture. The mantle improved. The foundation didn't change.
Thom's lens: when does the catastrophe arrive?
The cusp catastrophe maps directly onto architecture evolution. The normal factor is the performance of the existing architecture on existing metrics — latency, throughput, developer productivity. The splitting factor is the divergence between what the architecture assumes and what the business needs. As the business changes, the assumptions embedded in the architecture become less valid. The splitting factor increases. The system develops bimodality — the old architecture and a possible new architecture exist as alternative stable states. The organization is in the inaccessible region between them. A small additional change in the business requirement — a new regulation, a new integration, a new scale threshold — triggers the catastrophe. The old architecture cannot accommodate it. The organization flips. The flip feels sudden. The splitting factor had been accumulating for years.
This is why architecture rewrites are always "unexpected" and "overdue" simultaneously. The catastrophe is visible in retrospect. The splitting factor — accumulated technical debt, architectural mismatch, assumption decay — was tracked by nobody. The normal factor — uptime, feature velocity — was tracked by everyone. The system looked stable. The catastrophe was already determined. It just hadn't happened yet.
The connection to Parnas
Parnas argued that modules should hide design decisions likely to change. This is Thom's catastrophe theory applied to software structure. The design decision is a control variable. If it changes, and the change propagates, the system experiences a catastrophe — a sudden large-scale restructuring triggered by a local change. Information hiding prevents propagation. The change stays inside the module. The catastrophe is contained.
"The criteria for module decomposition should be based on minimizing the propagation of change." — Parnas
Minimizing the propagation of change is minimizing the catastrophe surface. Every module boundary is a containment wall. The splitting factor rises inside the module. The catastrophe, when it comes, is local. One module flips. The rest of the system doesn't notice. This is the architecture of systems that survive the perennial gale. Most systems don't have it. The splitting factor rises globally. The catastrophe, when it comes, is total. The rewrite is announced. The organization flips. The old system is retired. The new system inherits the assumptions of its moment. The cycle begins again.
Schumpeter's gale blows. Christensen's incumbents fall. Thom's catastrophes erupt. Parnas's modules contain them. The four thinkers are one argument. The argument is about whether your architecture can survive the inevitable — not whether the inevitable can be avoided. It cannot.
References:
- Joseph A. Schumpeter, Capitalism, Socialism, and Democracy, Harper & Brothers, 1942. Chapter VII: "The Process of Creative Destruction."
- Clayton M. Christensen, The Innovator's Dilemma: When New Technologies Cause Great Firms to Fail, Harvard Business School Press, 1997.
- René Thom, Structural Stability and Morphogenesis, W.A. Benjamin, 1972. (Translated by D.H. Fowler, 1975.)
- E.C. Zeeman, "Catastrophe Theory," Scientific American, Vol. 234, No. 4, April 1976, pp. 65-83.
- David L. Parnas, "On the Criteria to Be Used in Decomposing Systems into Modules," Communications of the ACM, Vol. 15, No. 12, December 1972.
- Related posts: Brooks on Software Design series, Lehman's Software Evolution, Parnas's Information Hiding, Software dark factories, Henney's Microservices
Engineering is the discipline of building things that work within constraints. Every topic on this blog — operating systems, AI models, trading infrastructure, research labs, innovation economics — is examined through the lens of systems design. The lens is engineering. The method is: understand the constraints, design within them, verify the design works, iterate. The domain provides the specifics. The method is universal.
The electric light was not a better candle. It was a different category of thing, valued along a different axis. The incumbents who improved their candles did everything right and were destroyed anyway. The innovator's dilemma is not a failure of management. It is a property of the structure.