The Open Source model continues to confuse and frustrate theorists and managers alike on exactly where its strength lies.  What is its ability to innovate and provide lasting value in the marketplace?  The success of Linux, Apache, Perl and other software projects poses a conundrum to management science, economics and social scientists in general, threatening to overthrow their carefully constructed apple carts.

Free obscures the paddling under the water

Aurora 224 2ch Open Source DJ Mixer

It is surprising to see so many experts befuddled by impassioned entrepreneurs that rally around a cause and forge a future. Settlers in any number of cultures have been building churches, post offices, jails and garbage dumps long before any government took over the task and began directing them about its timing, zoning and purpose.

Do not get the wrong idea that the above examples mean open source equals community service, or is antithetical to profit. But the lack of payment has confused the discussion.

What those examples teach us is that free association is a pervasive one. We could also use examples of multiple suppliers enjoining to form a firm, but the problem there is the opposite one; people are confused by the seeming profit motive rather than the rallying motivation based on common interest.

What are the strengths of hierarchy?

To get to the heart of the matter, Open Source advocates could argue very forcibly that governments have done a worse job of managing cities than their unorganized fore bearers did. In fact, the management of complex and open systems is well documented and miserable. It is very difficult if not impossible to do, and it becomes the issue in conservation areas, large parks, global warming theories, and city planning.

Large organizations with command and control hierarchies would do well to consider these and Open Source examples and learn from them. If upper management’s first fiduciary duty in large organizations is to do no harm, their long term track record is abysmal, primarily because bureaucracies assume permanence and entrench it with structure and inertia.

Part of the strength of Open Source is that since it has no bureaucracy, it is inherently adaptable. One could say it listens well, but that is an inherently hierarchical statement. It does not listen as much as it communicates what it finds since the members are both employees and customers, to use the vocabulary of the firm.

Self-organization and peer selection

Perhaps we are all too jaded to realize the power of self organized, engaged peer production. Perhaps we are all so accustomed to being herded and shepherded and organized from our first hours in kindergarten through decades of passive rote memorization and then structured Taylorean work places that we have forgotten that self initiated collaboration is the natural human state of affairs. Perhaps we have intentionally impugned competition for so many decades that we can no longer identify it when we see it.

The simple fact is that auto and peer selection is more accurate than supervisor allocation, and that engagement and creativity are bound with mastery and purpose, not obedience. Did we need social science experiments to prove that job performance is a function of both ability and motivational or dispositional factors, and that self-selection inherently favors the confluence of those functions?

The concept of self-selection is under-emphasized in the discussion of Open Source. As Eric Raymond says,

“…contributions are received not from a random sample, but from people who are interested enough to use the software, learn about how it works, attempt to find solutions to problems they encounter, and actually produce an apparently reasonable fix. Anyone who passes all these filters is highly likely to have something useful to contribute[1].”

That is a great deal of filtering, and arguably more effective than any hiring process. The cognitive effect on motivation of all that work hardly warrants mention. And coupled with peer selection, we have a hugely powerful foundation that is difficult to duplicate in a hierarchy without getting seriously out of the box.

Redundancy as a feature

Raymond also coins the ‘Linus Law’ which makes another fundamental point that most analysis of open source misses; “Given enough eyeballs, all bugs are shallow.” What he means is that the comparatively large number of programmer / debuggers makes what traditional programming finds difficult quite easy; each programmer tends to have their own perspective, thus increasing the probability that a bug will be matched to existing expertise to solve it.

There are fundamental lessons here for management science. First, it reinforces the concept of multi-disciplinary teams and specialization structured around a common purpose. Second, it portrays the strengths of complex systems; adaptability and emergence as a construct of self-organizing redundancy.

In traditional processes and hierarchies we tend to think in terms of efficiency which requires simplicity which demands single process points. Simplicity is efficient. But it struggles in the face of adversity. And it utterly fails when it faces novelty because it can not respond. Open source structures thrive on redundancy. They count on it.

Another key to the Linus Law is that someone characterizes a problem, and someone else understands it and answers it. Interestingly, most people agree that identifying the problem is the more difficult of the two sides of the coin. Here again, the answer is redundancy.

Communication as a role, not a management trait

As to structure, Open Source is not just a group of people jammed together willy-nilly. There is a great deal of method to the madness, with a core group of ‘halo’ developers that take on an organizing role which keeps overhead low. Hierarchical organizations tend to promote information aggregators; some organizations see it as a key management trait despite technology’s growing ability to automate most of the function. In Open Source, there is never confusion about roles because there is no hierarchy to climb.

Emulating complexity and scale free networks

But the central message regarding open source must return to redundancy. Innovation even in group projects revolves around the concept that design, enhancement and perfection are iterative, and occur in parallel spaces. That is a feature, not a bug.

Exploration loves connected duplication and diffusion. Communication is central, allowing chunky adaptations and almost immediate exploitation. What hierarchy views as waste, open source views as essential.

What is engagement?

To conclude, we return to the example of free association, where the Amish build a church or barn. There is no commander or middle management shouting orders or allocating resources, or motivating the troops. Like an ant colony, each person finds a place to contribute and does so. There is no queen or strategic CEO imparting vision or developing execution capability for the well being of the project or the organism. And even without the benefit of technology, the Amish will put up a barn as fast or faster at less cost than any group of construction workers similarly equipped with charts and plans and cranes and managers.

To be fair, the gist of the above paragraph may be too broad; a barn is a simple construct. But certainly no hierarchical organization can inject the same ‘empowerment’ or ‘engagement’ from the top down as a self-selected team working to pursue their passion. How then, does an organization imitate that? Actually, quite easily if one turns the question on its head.

On the other side of the coin, there is a great deal of useless information generated on the Internet by thousands of people working together. That may not be open source per se, but it is freely associated creative destruction, and the point is that bad ideas naturally degrade into nothingness in the unregulated, unmanaged atmosphere of the Internet, measured only by use, than under a government or managerial hospice nurtured by heroic egos and coalitions of inertia.

Inherent cognitive dissonance of hierarchy

The concept of Open Source embraces the idea that people, free of censure from superiors and the very explicit threat of a pay check, almost immediately become more engaged. And once engaged, peer pressure acts more effectively than any command and control structure.

The lessons learned from Open Source are not just of collaborative software. The primary lesson of Open source is that the benefit of hierarchies remains elusive, and their systemic tendencies of failure avoidance and permanence are rarely compensated by direction, execution, and vision, if they were ever strengths at all.

[1] Eric S. Raymond, The Cathedral & the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary, 1st ed. (O’Reilly Media, 2001).

Cathedral and the Bazaar  Eric S. Raymond

Here Comes Everybody: The Power of Organizing Without Organizations  Clay Shirky

Friedrich Hayek characterized complex systems with two main properties; the number of variables, and the connection between them[i]. It was these properties that predominantly resulted in emergence. Hayek was describing the properties of scale free networks.

Beginning in the 1990s, scientists revived the study of network research, at least partly after they noticed that scale free networks predominate in virtually every discipline or instance where nature constructs complex systems[ii]. Examples of networks with a scale free topology include protein networks, the Internet, genetics, traffic patterns, and social networks[iii].

A scale free network follows rather simple rules. The probability P(k) that an arbitrary element of the network is connected to exactly k other elements has the form P(k) = Ck−γ, where γ is usually called the scale-free exponent.  In other words, scale free networks conform to Power Laws which stipulate that a smaller number of nodes are very highly connected while most nodes are poorly connected.

The Power Law, often conceptualized as the 80/20 rule or Pareto Principle, is characterized by a very long tail unlike the more popular and better known Bell Curve. Trends conforming to Power Laws are found everywhere including wealth distribution, the size of meteorites, research citations, stock returns, gene variation, river systems, and all the examples of scale free networks listed above. It is nature’s most prevalent topology.

The question immediately arises as to why complex systems take on a scale free topology. Closer inspection seems to indicate it provides optimal levels of both reliability and adaptability compared to alternatives[iv].

Barabási and Albert stipulate that power law generated networks are the result of two complimentary phenomena; naturally occurring networks expand by adding new vertices, and by their nature, new vertices show preference for attaching to nodes that are already well connected[v].

Reliability

Since failures occur at random and the vast majority of nodes are comparatively small in size and connection, the likelihood that a larger hub would be affected is almost negligible. Even if a central hub is destroyed, the network will not lose its connectedness since the remaining hubs guarantee connection.

The weakness is of course that multiple hubs fail at the same time, which would destroy the network and render it a series of disconnected nodes. Notice that for most instances and circumstances, the chance of this occurrence is much lower than the odds of destroying other network topologies. It is possible to construct a network without this weakness, but it is comparatively inefficient and much more costly.

Clustering implication of the Power Law

Topology of scale free network

The diagram illustrates that as the nodes become smaller, so do the clusters they enjoin (The phenomenon is also illustrated in the Leis Network introduction). The hubs by their nature are the main connection points to smaller ‘communities’ of nodes or networks.

It is the Power Law that demands this structure, much as well connected people in business or politics might connect small neighborhoods of people.

In the diagram above, there are relatively few connections between smaller nodes. In reality, it is common for smaller network nodes to have their own connections to other small nodes. Think of the Internet as an example, with links to pages as node connections. In this scenario, even destroying a central hub would introduce only temporary strain on the network.

Adaptability

We now also see the reason for scale free network adaptability.  Failed nodes are quickly replaced by smaller connected nodes either providing the same function, or growing to meet demand, or both.  It is the robustness, or redundancy of the complex system, that provides for its ability to adapt to changing circumstance.

Summary

Scale free networks are nature’s favorite form. There is a simple reason. They are the most economical (i.e., cost effective and efficient) expression of reliance and adaptivity.

End Notes

[i] Friedrich Hayek, New Studies in Philosophy, Politics, Economics and the History of Ideas (Univ of Chicago Pr (T), 1985), 25-34.

[ii] Albert-László Barabási, Linked: The New Science of Networks, 1st ed. (Basic Books, 2002).

[iii] Ibid.

[iv] Maximino Aldana, “Boolean dynamics of networks with scale-free topology,” Physica D: Nonlinear Phenomena 185, no. 1 (October 15, 2003): 45-66, http://www.sciencedirect.com/science/article/B6TVK-4B1RYJP-4/2/e5f0a84cdf2e12e95188aa14636b378b.

[v] Albert-László Barabási and Réka Albert, “Emergence of Scaling in Random Networks,” Science 286, no. 5439 (October 15, 1999): 509-512, http://www.sciencemag.org/cgi/content/abstract/286/5439/509.

Further Reading

Scale free Networks at Scholarpedia

Scale free Networks at Wikipedia Photo courtesy

The outmoded centralized mainframe model is a good analogy for organizational trends. Managers continue to learn that centralization and control surrounded by robotic job descriptions may economize keystrokes and mechanics, but fall short in the much more important spectrum of decentralized, networked discernment where innovation, adaptability and peer pressured ethics flourish. 

They put a man on the moon with IBM 7090s

Centralized software systems struggle to efficiently understand that Mary (the customer’s bookkeeper) runs payables on the last Friday of each month, and that sending out a faceless, automated late notice to her firm only engenders negative vibrations.   More importantly, the centralized system will never replace the local relationship of the firms’ two bookkeepers, engendered because they take their children to the same day care and most importantly, the emergent ideas they come up with over coffee on how they can work more seamlessly together. There is little future in centralizing payables if it takes an emergent agent off the field.  Ant colonies would never dream of it.  Neither would any adaptive, creative complex system.

Adaptation, productivity and self-actualization are local, just like evolving complex systems and markets. Lately large organizations have been leveraging that power by loosening their structures and providing safe harbor for movement, community, and activity. Everyone has the innate ability to innovate, create, fail, and adapt if only the rules allow it.  Given the proper combinations of elements, reliability and creativity can amplify together, much as reconstituted wood chips form a bond stronger than the original tree.

References

Photo courtesy of Library of Congress

There is evidence that organizations will continue to decentralize beyond multi-disciplinary teams and matrix structures. For it cannot be coincidence that technology continues to find its value in individual empowerment.  When it introduced the personal computer (PC) in the early 1980s, IBM threatened its own existence by subcontracting its birthright to Microsoft and Intel in one of the most spectacularly failed bets against empowerment in the technological age.  It marked one of the only times it did not use its own internally developed hardware and operating systems.  IBM quickly headed towards bankruptcy as the PC and the resulting explosion of technology forced the computer industry into a massive restructuring that left the mini-computer industry in a shambles; companies like DEC, Prime, HP, Control Data, Wang and TI saw their product lines obliterated with their own viability along with them.

IBM under Louis Gerstner saved itself from ruin in one of the most astounding turnaround examples of decentralization ever attempted. It may be that IBM’s present structure now mirrors many organizational fundamentals laid out on this website partly because it at one time so firmly rejected them.  The organization restructured from a hierarchical, unidirectional corporate culture to a broader based, decentralized one in less than 5 years.  They rejected their blue suits, sold buildings and ended up with almost half their workforce working from home in their pajamas.  They re-centered and refocused, and reversed their services and hardware mix of business on services, and sales and profits exploded.

Now IBM has obliterated most of its corporate hierarchy.  It relies on multi-disciplinary teams that reflect combinations of geography, function, customer segment and industry focus, depending on the situation.  They involve themselves in the open source communities which would have been unthinkable in their proprietary days of yesteryear.  But surely none of this expansion of adaptability and profitability and productivity would have occurred if their competitors and the markets had not forced them to react to preserve their viability.

To be fair, who could have predicted the explosion of creativity and innovation of turning millions of PC users into active entrepreneurs? After all, computing economics clearly show that centralized networks with dumb terminals are much less expensive to design and maintain.  That was the overwhelming organization of the computer industry even in the mini-computer world.  That hierarchy looks great on paper and proves more inexpensive to produce, purchase, maintain and deliver computing power.  Those economics remain true to this day.

But one huge factor was left out of the equation.  What was missing was the idea of autonomy. What was missing was the degree that control suppresses creativity. What was missing was an understanding of developmental psychology. What was missing was an in-depth appreciation of just how intrinsic complex systems (in this case free markets) are to innovation.  What was missing was just how incredibly productive users could be if they weren’t sitting around waiting for centralized IT department programmers and instead could themselves develop applications and mash-ups and innovations.

Corporate organizational structure is still hampering productivity, creativity and reliability.  Organizations both in their structure and their strategy often make bets against the individual, and they continue to do so in pendulum like swings in both micro and macro trends. They always eventually lose.  If you don’t believe me, ask IBM.

References

Photo courtesy Wikipedia Commons

The above picture symbolizes both technology and organizational change. Still wet behind the ears, in 1991 I was purchasing used mini-Vax systems as servers for Novell networks. They were faster and more inexpensive than PC’s, and cold room costs were sunk. Those pricing trends foreshadowed mini-computer markets.

If I remember correctly, we were the first in the state to cobble Vax systems together with Novell networks. Back then Peter Norton, not Bill Gates, was a G~d to teenage nerds everywhere, driving Microsoft and Intel innovation, and no self-respecting 80286 PC went without a QEMM-386 to make it effective.

This period also marked the last attempt by IBM’s misguided proprietary corporate policies regarding PS/2, which all nerds knew would inevitably fail.  It signified IBM’s continued denial regarding the structural change PC’s represented for technology.  No programmer I ever knew willingly dealt with it.

Maslow considered physical needs as the most basic. Obviously if a person is hungry or cold, they have no time for higher conceptual arguments, friendship or philosophy.

Physical needs, or base needs also presuppose a stable, holistic foundation, and it is in this analogy that we wish to develop our concepts. We wish to stress the holistic qualities of physicality. We think first of our bodies, and then its parts.  Our bodies are a wondrous example of a complex system evolving through time to its nature and environment.  Humans well know the penalty of depriving one of its components;  the whole system pays.

Connected with this concept of physicality, there is the concept of adaptability, potentiality and expression. Our bodies are not only what we make them; they are the extension of our lives and purpose. The strategies and tactics of those purposes change and so must our physical bodies to react to the realities and practicalities of our lives.

In terms of an organization, we immediately draw attention to the associated idea of multi-disciplinary groups. As we shall see, it is a key component of how we learn, and how we think about learning. We naturally see the whole as a sum of discrete and component parts.

However, our physicality goes beyond that. Our physicality demands we act. It means our organizational structures matter a great deal. It means they are likely not nearly as fluid as they should be. It means without our explicit exercise, we shall surely decline.

Specialization

The world is increasingly specialized. However, various disciplines have been researching and learning about essentially the same phenomena for decades, incubated by the silos of their own disciplinary languages and research.

Too often, our executives no longer think in terms of the whole organization. They cannot. There is little formal training to do so. The Renaissance man is dead.

Perhaps part of the challenge is not to train the executive to consider the organization, but to bring the organization to the executive. The organization, the hive must begin to act like one. It must structure itself in order to do so. It must divest itself of its hierarchy even while it buttresses it in a much more aggressive manner, not so much in developing strategic endeavors, or financial decision making, but in the spiritual and emotional well being of the organization.

What does that mean?  What does the individual need to learn to increase their effectiveness? For starters, we must begin with the idea of meaning or mission.  Rarely is there an effective map of the organization that describes its purpose as an organism. Functional departmental organizational structures give almost no clue about an organization’s purpose, much less how it accomplishes its goals. A leg cannot be an effective leg until it understands how it relates to the rest of the body. Besides understanding the meaning or essential purpose of the body, components must understand how they fit. No real meaningful or creative contribution obtains until it does.

To illustrate, each cell contains all the DNA necessary to replicate the entire organism.  No manager or queen or sergeant must give instruction to the ants in a colony.  No manager needs to give instructions to the pipe team at Nucor Steel.

We speak of empowerment as an action rather than as a natural state of the organism.  No ant needs to be empowered.  Neither does any human unless constrained in the first place.  No cell or Internet node needs to be encouraged to become engaged.  Neither does any human unless constrained by conditioning or organizational structure in the first place.

This idea fits well with what we know about creativity; to imagine something new, we need to understand the big picture. However, specialization obscures the big picture. Many of our disciplines demand a level of specialization that will only grow deeper through time. Therefore, our structures must accommodate those specializations. We individually will never know everything.

To paraphrase Frankl, we must stop asking ourselves what we wish to do to enrich ourselves, and ask ourselves what the organization demands of us. What does the organization need to do to fulfill its meaning? If it does so well, it will do so profitably.

We therefore come back to the idea of engendering each associate, each team member, each component of the complex system, with the fairly simple meaning or purpose of their team. The specialists learn together driven by a common meaning or purpose, just as the purpose directs our body cells. Interestingly, that concept also describes one of the ways we learn.

Cognition as Distributed

The components of our physicality exhibit comparisons to one of the ways we learn. There is a well-known story going around the Internet that describes the wondrous and surprising tale of making a pencil; the tale describes the fact that no one person can make one.[i] No single organization makes it. It is the product of self-organizing specialists:

My family tree begins with what in fact is a tree, a cedar of straight grain that grows in Northern California and Oregon. Now contemplate all the saws and trucks and rope and the countless other gear used in harvesting and carting the cedar logs to the railroad siding. Think of all the persons and the numberless skills that went into their fabrication: the mining of ore, the making of steel and its refinement into saws, axes, motors; the growing of hemp and bringing it through all the stages to heavy and strong rope; the logging camps with their beds and mess halls, the cookery and the raising of all the foods. Why, untold thousands of persons had a hand in every cup of coffee the loggers drink!

In his research on cognitive learning, Hutchins describes distributed cognition in the navigation of a U.S. Navy ship out of the harbor, where six people using sophisticated tools made it possible to accomplish tasks beyond the capabilities of any individual member.[ii] The active distribution itself allows adaptability and excellence that no individual can perform.

Another familiar example is of the specialized members of a sports team. The team requires the input and contribution of a highly interactive group; in hockey, positions consist of goalie, defense, wing and center. There are six players on the ice and four specializations; this high degree of specialization relative to the number of players is common in sports, music, and teams in general. It is interesting to note that in football, where specialization is the most advanced, one of the key methods of confusing the opponent is to rearrange the specializations in new configurations and duties so the opponent cannot predict actions or outcome. Incidentally, doesn’t that sound like an effective business tactic?

Further, cognition is not merely distributed. As the psychologist Moreno noticed in self-organizing groups, as complexity theory stipulates, and as Adam Smith described in ‘the Invisible Hand,’ individual actions often occur within situations where spontaneous order evolves.  No central plan or construct could possibly envision a blueprint detailed enough to replicate it. The order is the product of a never-ending series of adjustments made by individuals to the constantly changing environment in which they participate.

The creators of the group depend on this adaptability or reaction. A basketball team depends on the creative forces of this group dynamic. The game is not a never-ending series of pre-arranged plays and pre-choreographed ‘chess moves,’ so-to-speak, it is the artful positioning of the team that through its coordination and reactions creates successful situations.

Likewise, innovation in free markets is the product of members interacting in continual searches for improvement, survival and efficiency. No planner can reproduce this activity. They can only nurture the circumstances or environment.

Lastly, no parent engineer an adult. A person grows through a unique experiential journey, tempered by meaning and personal choice. Individual, group and organizational behavior exemplify these dynamics.

The conclusion to our examples is that organizations heavily dependent on hierarchical structures must be fully aware of the naturally suppressant effect that top-down choreography has on adaptability and creativity.  The physical organization of a firm and its attendant culture exerts a fundamental effect on the complexity and ability of its members to successfully perform.

Synergy of the Physical Body

Perhaps because of a synthesis of the apparent dichotomy of components and the whole, Maslow noticed that actualized people exhibited an ability to appreciate a freshness or renewal of well-worn experiences and events. They appreciate their environment in new and wonderful ways. They find solace and beauty in familiar places. They rejoice in situations that for most of us inspire only semi-conscious awareness. The actualized person, on the other hand, finds at times an almost mystical wonder in these habitual and well-known circumstances.

Organizationally, the recurring theme of team actualization, the synergy of each specialized body part, of each specialized player, inspires well-adapted and sometimes new methods and approaches. Our psychology demands cohesive groups and our functional hierarchies often stand in its way.

References

[i] Leonard E. Read, I, Pencil: My Family Tree as told to Leonard E. Read (Irvington-on-Hudson, NY: Foundation for Economic Education, 1999), http://www.econlib.org/library/Essays/rdPncl0.html.

[ii] E. Hutchins, “The Social Organization of Distributed Cognition,” Perspectives on socially shared cognition (1991): 283-307.