For the past several months, I’ve been immersed in nineteenth century history. Specifically, the history of interchangeability in technology between 1765, when the Système Gribeauval, the first modern technology doctrine based on the potential of interchangeable parts, was articulated, and 1919, when Frederick Taylor wrote The Principles of Scientific Management.

Here is the story represented as a Double Freytag diagram, which should be particularly useful for those of you who have read Tempo. For those of you who haven’t, think of the 1825 Hall Carbine peak as the “Aha!” moment when interchangeability was first figured out, and the 1919 peak as the conclusion of the technology part of the story, with the focus shifting to management innovation, thanks in part to Taylor.

The unsung and rather tragic hero of the story of interchangeability was John Harris Hall (1781 – 1841), inventor of the Hall carbine.  So I am naming my analog to Moore’s Law for the 19th century Hall’s Law in his honor.

The story of Hall’s Law is in a sense a prequel to the unfinished story of Moore’s Law. The two stories are almost eerily similar, even to believers in the “history repeats itself” maxim.

Why does the story matter? For me, it is enough that it is a fantastically interesting story. But if you must have a mercenary reason for reading this post, here it is: understanding it is your best guide to the Moore’s Law endgame.

So here is my telling of this tale. Settle in, it’s going to be another long one.

Onion Steel

In A Brief History of the Corporation, I argued that there were two distinct phases — an early mercantile-industrial phase that was primarily European in character, extending from about 1600 to 1800, and a later Schumpeterian-industrial phase, extending from about 1800-2000, that was primarily American and Russian in character.

Each phase was enabled by a distinct technological culture. In the early, British phase, a scientific sensibility was the exception rather than the rule. The default was the craftsman sensibility. In the later,  American-Russian phase, the scientific sensibility was the rule and the craftsman sensibility the exception (it is notable that the American-Russian phase was inspired by French thought rather than British; call it Napoleon’s revenge).

What was this (much romanticized today) craftsman sensibility?

Consider this passage about the state of steel-making in Sheffield, the leading early  nineteenth century technology center for the industry, before the rise of American steel. The quote is from Charles Morris’ excellent book The Tycoons, my primary reference for this post (it is nominally about the lives of Rockefeller, Carnegie, J. P. Morgan and Jay Gould, but is actually a much richer story about the broad sweep of 19th century technology history; I am not done with it yet, but it has been such a stimulating read that I had to stop and write this post):

Making a modest batch of steel could take a week or more, and traditional techniques were carefully passed down from father to son; one Sheffield recipe started by adding “the juice of four white onions.”

Morris attributes the onion story to Thomas Misa’s Nation of Steel, which is now on my reading list.

American steel displaced British steel not because it was based on the Bessemer and open hearth processes (Bessemer was English), but because the industry was built from the ground up along scientific lines, with no craftsman-baggage slowing it down.

The interesting thing about this recipe for onion steel is that it illustrates both the strengths and the weaknesses of the craftsman sensibility. You can only imagine the tedious sort of uninformed experimentation it took to consider adding onions to a steel recipe. There is something beautiful about the absence of preconceived notions in this sensibility. No modern metallurgist would even think to add onions to a metal recipe.

On the other hand, if a modern metallurgist were faced with data showing that onions improved the properties of steel, he or she would not rest until they’d either disproved the effect, or explained it in less bizarre terms. The recipe would certainly not get passed down from “father to son” (“mentor to mentee” today) unexplained.

What America brought to manufacturing was a wholesale shift from craftsman-and-merchant thinking about technology and business to engineer-and-manager thinking. The shift affected every important 19th century business sector: armaments, railroads, oil, steel, textile equipment. And it created a whole new sector: the consumer market.

But this was not the result of an abstract, ideological quest for scientific engineering and manufacturing, or a deliberate effort to replace high-skill/high-wage craftsmen with low-skill/low-wage/interchangeable machine operators.

It was a consequence of a relentless pursuit of interchangeability of parts, which in turn was a consequence of a pursuit of greater scale, profits and competition for market share (which drove greater complexity in offerings) on the vast geographic canvas that was America. Craft was merely a casualty along the way.

So why was interchangeability of parts a holy grail in this pursuit?

Interchangeability, Complexity and Scaling

The problem is that even the highest-quality craft does not scale. When something like a rifle is mass-produced using interchangeable parts, breakdowns can be fixed using parts cannibalized from other broken-down rifles (so two broken rifles can be mashed-up to make at least one that works) or with spare parts shipped from an warehouse. Manufacturing can be centralized or distributed in optimal ways, and constantly improved. Production schedules can be decoupled from demand schedules.

A craftsman-made rifle on the other hand, requires a custom-made/fitted replacement part. The problem is especially severe for an object like a rifle: small, widely-dispersed geographically, and liable to break down in the unfriendliest of conditions. Conditions where minimizing repair time is of the essence, and skilled craftsmen are rather thin on the ground. It is no surprise that the problem was first solved for guns.

Let’s do some pidgin math to get a sense of what a true mathematical model might look like.

Roughly speaking, scaling production for any mechanical widget involves three key dimensions: production volume V, structural complexity S (the number of  interconnections in an assembly is a good proxy measure for S, just like the number of transistors on a chip is a good proxy for its complexity) and operating tempo of the machine in use, T (since the speed of operation of a machine determines the stress and wear patterns, which in turn determines breakdown frequency; clock-rate is a similar measure for Moore’s Law).

For complex widgets, scaling production isn’t just (or even primarily) about making more new widgets; it is about keeping the widgets in existence in the field functioning for their design lifetime through post-sales repair and maintenance.  The greater the complexity and cost, the more the game shifts to post-sales.

You can combine the three variables to get a rough sense of manufacturing complexity and how it relates to scaling limits. Something like C=SxT provides a measure of the complexity of the artifact itself. Breakdown rate B is some function of complexity and production volumes, B=f(C, V). At some point, as you increase V, you get a corresponding increase in B that overwhelms your manufacturing capability. To complete this pidgin math model, you can think in terms of some B_max=f(C, V_max) above which V cannot increase without interchangeability.

Modern engineers use much more sophisticated measures (this crude model does not capture the tradeoff between part complexity and interconnection complexity for example, or the fact that different parts of a machine may experience different stress/wear patterns), but for our purposes, this is enough.

To scale production volume above V_max without introducing interchangeability, you have to either lower complexity and/or tempo or increase the number of skilled craftsmen. The first two are not options when you are trying to out-do the competition in an expanding market. That would be unilateral disarmament in a land-grab race. The last method is simply not feasible, since education in a craft-driven industrial landscape means long, slow and inefficient (in the sense that it teaches things like onion recipes) 1:1 apprenticeship relationships.

There is one additional method that does not involve interchangeability: moving towards disposability for the whole artifact, which finesses the parts-replacement problem entirely. But in practice, things get cheap enough for disposability to be a workable strategy only after mass production is achieved. Disposability is rarely a cost-effective strategy for craft-driven manufacturing, though I can think of a few examples.

These facts of life severely limited the scale of early nineteenth century technology. The more machines there are in existence, the greater the proportion of craftsmen whose time must be devoted to repair and maintenance rather than new production.  Since breakdowns are unpredictable and parts unique, there is no way to stockpile an inventory of spare parts cheaply. There is little room for cannibalization of parts in the field to temporarily mitigate parts shortages.

What was needed in the 19th century was a decoupling of scaling problems from manufacturing limitations.

Interchangeability and the Rise of Supply Chains

Interchangeability of parts breaks the coupling between scaling and manufacturing capacity by substituting supply-chain limits for manufacturing limits. For a rifle, you can build up a stockpile of spare parts in peace time, and deliver an uninterrupted supply of parts to match the breakdown rate. There is no need to predict which part might break down in order to meaningfully anticipate and prepare. You can also distribute production optimally (close to raw material sources or low-cost talent for instance), since there is no need to locate craftsmen near the point-of-use.

So when interchangeability was finally achieved and had diffused through the economy as standard practice (a process that took about 65 years), demand-management complexity moved to the supply chain, and most problems could be solved by distributing inventories appropriately.

These happy conditions lasted for nearly a century after widespread interchangeability was achieved, from about 1880 to 1980, when supply chains met their own nemesis, demand variability (that problem was partially solved using lean supply chains, which relied in turn on the idea of interchangeability applied to transportation logistics: container shipping. But I won’t get into that story here, since it is conceptually part of the unfinished Moore’s Law story).

The price that had to be paid for this solution was that the American economy had to lose the craftsmen and work with engineers, technicians and unskilled workers instead. This creates a very different technology culture, with different strengths and weaknesses. For example the scope of innovation is narrowed by such codification and scientific systematization of crafts (prima facie nutty ideas like onion steel are less likely to be tried), but within the narrower scope, specific patterns of innovation are greatly amplified (serendipitous discoveries like penicillin or x-rays are immediately leveraged to the hilt).

Why must craft be given up? Even the best craftsmen cannot produce interchangeable parts. In fact, the craft is practically defined by skill at dealing with unique parts through carefully fitted assemblies.  (“Interchangeability” is of course a loose notion that can range from functional replaceability to indistinguishability, but craft cannot achieve even the coarsest kind of interchangeability at any meaningful sort of scale).

Put another way, craft is about relative precision between unlike parts. Engineering based on interchangeability is about objective precision between like parts. One requires human judgment. The other requires refined metrology.

From Armory Practice to the American System

It was the sheer scale of America, the abundance of its natural resources (and the scarcity of its human resources), that provided the impetus for automation and the interchangeable parts approach to engineering.

As agriculture moved westward through New York, Pennsylvania and Michigan, the older settled regions began to turn to manufacturing for economic sustenance. The process began with the textile industry, born of stolen British designs around what is now Lowell, Massachusetts. But American engineering in the Connecticut river valley soon took on a distinct character.

Like the OSD/DARPA/NASA driven technology boom after World War II, the revolution was driven by the (at the time,  fledgling) American military, which had begun to acquire a mature and professional character after the war of 1812 (especially during the John Quincy Adams administration).

The epicenter of the action was the Springfield Armory, the PARC of its day, and outposts of the technology scene extended as far south as Harper’s Ferry, West Virginia.

John Hall was among the hundreds of pioneers who swarmed all over the Connecticut valley region, dreaming up mechanical innovations and chasing local venture capitalists, much like software engineers in Silicon Valley today.

There were plenty of other extraordinary people, including other mechanical engineering geniuses like Thomas Blanchard, inventor of the Blanchard gun-stock lathe (which was actually a general solution for turning any kind of irregular shape using what is known today as a pattern lathe). By the time he was done with gun stocks, a bottleneck part in gun-making, with all sorts of “subtle curves along multiple axes” he had created a system of 16 separate machines at the Springfield Armory that pretty much automated the whole process, squeezing out all craft of what had been the single most demanding component in gun-making.

British gun-making was like British steel-making before people like Blanchard and Hall blew up the scene. Here is Morris again:

The workings of the British gun industry were reasonably typical of the mid-nineteenth-century manufacturing. It was craft-based and included at least forty trades, each with its own apprenticeship system and organizations. The gun-lock, the key firing mechanism, was the most complicated, while the most skilled men were the lock-filers…[who]… spent years as apprentices learning to painstakingly hand-file the forty or so separate lock pieces to create a unified assembly… When the Americans breezily described machine-made stocks, and locks that required no hand fitting, they sounded as if there were smoking opium.

Among the opium-smoking geniuses, Blanchard at least enjoyed a good deal of success. Hall did not.

He put together almost the entire “American System” through his single-minded drive, in the technology-hostile Harper’s Ferry location far from the Connecticut Valley hub. When he was done, he had created an integrated manufacturing system of dozens of machines that produced interchangeable parts for every component of his carbine. Even parts from production runs from different years could be interchanged, a standard some manufacturing operations struggle to reach even today.

The achievement was based on relentless automation to eliminate human sources of error, increasingly specialized machines, and rigorous and precise measurements (there were three of every measurement instrument, one for production use, one for calibration, and a master instrument to measure wear on the other two).

It was a massive systems-engineering accomplishment. The Hall carbine was the starter pistol for the American industrial revolution.

Overtake, Pause, Overdrive

Hall did not reap much of the rewards. Thanks to unfortunate exploitative relationships (in particular with a shameless patent troll, William Thornton, a complete jerk by Morris’ account), he was banished to Harper’s Ferry rather than being allowed to work in Springfield. And his work, when completed, was acknowledged grudgingly, and with poor grace. The Hall carbine itself was obsolete by the time his system was mature, and others who applied it to newer products reaped the benefits.

Between 1825 and the 1910s, the methods pioneered by Hall spread through the region and beyond, and were refined and generalized. In the process, first America, and then the world, experienced a Moore’s Law type shock: rapidly increasing standards of living provided by an increasing variety of goods whose costs kept dropping.

Culturally, the period can be divided into three partially overlapping phases: an overtake phase (1851 – 1876)  when America clearly pulled ahead of Britain as the first nation in the technology world, a “pause” represented by the recession of the 1870s, and finally an over-drive phase beginning in the 1880s and continuing to the beginning of World War I, when the American model became the global model (and in particular, the Russian model, as Taylorism morphed into state doctrine).

Overtake: 1851 — 1876

The overtake phase has a pair of useful bookend events marking it. It began with the 1851 Crystal Palace Exhibition, the first of the great 19th century world fairs, when the world began to suspect that America was up to something (McCormick’s harvester and Colt’s revolver were among the items on display), and ended with the 1876 Centennial World Fair in Philadelphia, when all remaining doubt was erased and it became obvious that America had now comprehensively overtaken Britain in technology.

When Britain finally caught on and hastily began copying American practices following the Philadelphia fair, the result was a revitalization of British industry that produced, among other things, the legendary Enfield rifle (the rifle subplot in the story of interchangeability has an interesting coda that is shaping the world to this day, the Russian AK-47, as pure an example of the power of interchangeability-based mass manufacturing as has ever existed).

It wasn’t just guns. In every industry America began to show up Britain. Much of the credit went to showboating hustlers who claimed credit for interchangeability and the American System/Armory Practice, and made a lot of money without actually contributing very much to core technological developments. These included Eli Whitney of cotton gin fame, the McCormicks of the harvester, Samuel Colt (revolvers) and Isaac Singer (sewing machines). While they certainly contributed to the development of individual products, the invention of the American model itself was due to technologists like Blanchard and John Hall.

In the initial decades of the overtake, fueled in part by opportunity (and profiteering) associated with the Civil War and government subsidized building out of the railroad system, much of the impact was invisible. But by the 1890s, as the infrastructure phase was completed, the same methods were unleashed on everyday life, creating modern consumer culture and the middle class within the short space of a single generation.

The Pause: the 1870s

The Civil War looms large as the major political-economic event in this history (1861 – 1865), but the bulk of the impact was felt in the decade that followed, once the dust had settled and interrupted infrastructure projects were completed.

This impact took the form of the rather strange long recession of the 1870s, which was very culturally very similar to the one we are currently experiencing (increased economic uncertainty and fall in nominal incomes, hidden technology-driven increases in standard of living, foundational shifts in the nature of money — back then it was a greenbacks vs. gold thing).

One way to understand this process is that the infrastructure phase had created both tycoons and an extremely over-leveraged economy. It was the uncertain gap between “build it” and “they will come.”  It was a huge, collective pause, a national decade of breath-holding as people wondered whether the chaos unleashed by the new infrastructure would create a better social order or destroy everything without creating something new in its place.

Starting in the 1880s, the bet began paying off in spades. The recession ended and the over-drive boom began, as people figured out what to do with the newfound capabilities in their environment.

Overdrive: 1880s — 1913

A good early marker here is probably the first Montgomery Ward catalog in 1872, the first major sign that the new infrastructure allowed old businesses to be rethought, leading to the creation of the modern consumer economy.

The mail-order catalog was by itself a simple idea (the first catalog was just a single page), but the reason it disrupted old-school merchants was that it relied on all the infrastructure complexity that now existed.

Trains that ran on reliable schedules, to deliver mail, telegraph lines that brought instant price updates on western grain to the East Coast, steel to build everything, oil and electricity to light up (and later, fuel) everything, new financial systems to move money around, and of course, the application of interchangeability technology to everything in sight.

It took Sears, starting in 1888, to scale the idea and truly take down the merchant elites who had defined the old business culture, but by World War I, middle-class consumer culture had emerged and had come to define America. In another 50 years, it would come to define the world.

It was such a powerful boom that globally, it lasted a century, with two world wars and a Great Depression failing to arrest its momentum (as an aside, I wonder why people pay so much attention to the 1930s depression to make sense of the current recession; the 1870s recession makes for a far more appropriate comparison).

What ultimately killed it was its own success. Semiconductor manufacturing probably represents the crowning achievement of the Armory Practice/American System that began with a lonely John Hall pushing ahead against all odds at Harper’s Ferry.

Moore’s Law was born as the last and greatest achievement of the parent it ultimately devoured: Hall’s Law.

Hall’s Law

When you step back and ponder the developments between 1825 and 1919, it can be hard to make sense of all the action.

There is the pioneering work in manufacturing technology. There is the explosion of different product types as the American System diffused through the industrial landscape. There is the story of the rise of the first tycoons. There is the rise of consumerism and the gradual emergence of the middle class. There is the connectivity by steam and telegraph.

Then there is the increasingly confident and strident American presence on the global scene (especially through the World Fairs, two of which I already talked about). And of course, you have the Civil War, the California Gold Rush, the cowboy culture that existed briefly (and permanently reshaped the American identity) before Jay Gould killed it by finishing the railroad system.

There was the rise of factory farming and the meatpacking and refrigerator-car industries together killing the urban butcher trade and suddenly turning Americans into the greatest meat eaters in history. Paycheck economics took over as the tycoon economy killed the free agent.

In fact, there was a lot going on, to put it mildly. And that was just America. The rest of the world wasn’t exactly enjoying peace and stability either. Perry had kicked down the doors of Japan, Opium wars had ravaged China, the East India Company (the star of my History of Corporations post) had been quietly put out to pasture and the Mughal empire had collapsed. The Ottomans were    starting on a terminal decline. Continental Europe had begun its century-long post-Napoleon march towards World War I (the US Civil War served as a beta test for the post-Bismarck model of total war, just as the Spanish Civil war served as a beta test for World War II).

But just as Moore’s Law provides something of a satisfying explanatory framework for almost everything that has happened in the last 50 years, the drive towards the holy grail of interchangeability provides a satisfying explanatory framework for much of this action. Here’s my attempt at capturing what happened (someone enlighten me if something like this has already been proposed under a different name) :

Hall’s Law: the maximum complexity of artifacts that can be manufactured at scales limited only by resource availability doubles every 10 years. 

I believe this law held between 1825 and 1960, at which point the law hit its natural limits.

Here, I mean complexity in the loose sense I defined before: some function of mechanical complexity and operating tempo of the machine, analogous to the transistor count and clock-rate of chips.

I don’t have empirical data to accurately estimate the doubling period, but 10 years is my initial guess, based on the anecdotal descriptions from Morris’ book and the descriptions of the increasing presence of technology in the world fairs.

Along the complexity dimension, mass-produced goods increased rapidly got more complex, from guns with a few dozen parts to late-model steam engines with thousands.  The progress on the consumer front was no less impressive, with the Montogmery Ward catalog  offering mass-produced pianos within a few years of its introduction for instance. By the turn of the century, you could buy entire houses in mail-order kit form. The cost of everything was collapsing.

Along the tempo dimension, everything got relentlessly faster as well. Somewhere along the way, things got so fast thanks to trains and the telegraph, that time zones had to be invented and people had to start paying attention the second hand on clocks.

There is a ton of historical research on all aspects of this boom, but I suspect nobody has yet compiled the data in a form that can be used to fit a complexity-limit growth model and figure out the parameters of my proposed Hall’s Law, since it is the sort of engineering-plus-history analysis that probably has no hope of getting any sort of research funding (it would take some serious archaeology to discover the part-count, operating speed and production volumes for a sufficient number of sample products through the period to fit even my simple model, let alone a model that includes things like breakdown rates and actual, as opposed to theoretical, interchangeability).

But even without the necessary empirical grounding, I am fairly sure the model would turn out to be an exponential, just like Moore’s Law. Nothing else could have achieved that kind of transformation in that short a period, or created the kind of staggering inequality that emerged by the Gilded Age.

Break Boundaries and Tycoon Games

Both Moore’s Law and Hall’s Law in the speculative form that I have proposed, are exponential trajectories. These trajectories generally emerge when some sort of runaway positive-feedback process is unleashed, through the breaking of some boundary constraint (the term break boundary is due to Marshall McLuhan).

The positive-feedback part is critical (if you know some math, you can guess why: a “doubling” law in a difference/differential equation form has to be at least a first-order process; something like compound interest, if you don’t know what the math terms mean).

Loosely speaking, this implies a technological process that can be applied to itself, improving it. Better machines with interchangeable parts also means better machine tools that are themselves made with interchangeable parts and therefore can run continuously at higher speeds, with low downtime. Computers can be used to design more complex computers.  This is not true of all technological processes. Better plastics do not improve your ability to make new plastics, for instance, since they do not play much of a role in their own manufacturing processes.

This is the inner, technological positive-feedback loop (think of an entire technology sector engaging in a sort of 10,000 hours of deliberate practice; a major sign is that the most talented people turn to tool-building: Blanchard and Hall for Hall’s Law, people like the late Dennis Ritchie and Linus Torvalds for Moore’s Law).

But the technological positive-feedback loop requires an outer financial positive-feedback loop around it to fuel it. You need conditions where the second million is easier to make than the first million.

This means tycoons who spot some vast new opportunity and play land-grabbing games on a massive scale.

Both Hall’s Law and Moore’s Law led to wholesale management and financial innovation by precisely such new tycoons.

For Hall’s Law, the process started with Cornelius Vanderbilt, the hero of A. J. Stiles’ excellent The First Tycoon, who figured out how to tame the strange new beast, the post-East-India-Company corporation and in the process sidelined old money.

It is revealing that Vanderbilt was blooded in business through a major legal battle for steamboat water rights: Gibbons vs. Ogden (1824) that helped define the relationship of corporations to the rest of society. From there, he went from strength to strength, inventing new business and financial thinking along the way. Only in his old age did he finally meet his match: Jay Gould, who would go on to become the archetypal Robber Baron, taking over most of Vanderbilt’s empire from his not-so-talented children.

Vanderbilt was something of a transition figure. He straddled both management and finance, and old and new economies: he was a cross between an old-economy merchant-pirate in the Robert Clive mold (he ran a small war in Nicaragua for instance) and a new-economy corporate tycoon.  He transcended the categories that he helped solidify, which helped define the next generation of tycoons.

Among the four tycoons in Morris’ book, Rockefeller (Chernow’s Titan on Rockefeller is another must-read) and Carnegie appear on one side, as the archetypes of modern managers and CEOs. Both were masters of Wall Street as well, but were primarily businessmen.

On the financial side, we find the Joker-Batman pair of Gould and Morgan. Jay Gould was the loophole-finder-and-destabilizer; J. P. Morgan was the loophole-closer and stabilizer.  While Gould was a competent, if unscrupulous manager during the brief periods that he actually managed the companies he wrangled, he was primarily a financial pirate par excellence.

It makes for a very good story that he made his name by giving the elderly Vanderbilt, who pretty much invented the playbook along with his friends and rivals, the only financial bloody nose of his life (though Vanderbilt exacted quite a revenge before he died).   Through the rest of his career, he exposed and exploited every single flaw in the fledgling American corporate model, turning crude Vanderbilt-era financial tactics into a high art form. When he was done, he had generated all the data necessary for J. P. Morgan to redesign the financial system in a much stronger form.

Morgan’s model would survive for a century until the Moore’s Law era descendants of Gould (the financial pirates of the 1980s)  started another round of creative destruction in the evolution of the corporate form.

From Hall’s Law to Moore’s Law

Hall’s Law was the prequel to Moore’s Law in almost every way.  The comparison is not a narrow one based on just one dimension like finance or technology. It spans every important variable. Here is the corresponding Double Freytag:

I’ll save my analysis of the Moore’s Law era for another day, but here is a short point-by-point mapping/comparison of fundamental dynamics (i.e. things that were a consequence of the fundamental dynamics rather than historical accidents).

1. Obviously Hall’s Law maps to Moore’s Law
2. Increasing interchangeability in mechanical engineering maps to increasing transistor counts in semiconductor manufacturing. Increasing machine speeds map to increasing chip clock-rates.
3. Both technologies radically drove down costs of goods and created de facto higher standards of living
4. Both technologies saw the emergence of a new breed of tycoons within a few leadership generations. Jack Welch maps to Cornelius Vanderbilt. Bill Gates and Michael Dell map to Rockefeller and Carnegie. Jeff Bezos maps to Montgomery Ward and Sears.
5. The newer, younger “digital native” tycoons, starting with Zuckerberg, map to the post 1890 3rd generation innovators who were native to the new world of interchangeability rather than pioneers, similar to the early 20th century automobile and airplane industry tycoons (it is revealing that the Wrights were bicycle mechanics; bicycles were the first major consumer product to be designed around interchangeability from the ground up; the airplane was a result of the careful application of exactly the precise sorts of careful scientific measurement, experimentation and optimization that had been developed in the previous 75 years).
6. Each era was punctuated in the middle by a recessionary decade marked by financial excesses, as the economy retooled around the new infrastructure. The 1870s maps to the 2000s.
7. Each era enabled, and was in turn fueled by, new kinds of warfare, exemplified by major wars that disturbed a balance of power that had been maintained by old technology. The American Civil War maps to the Cold War, while the wars of the 1990s and 2000s are analogous to World War I.
8. Guns (including high-tempo machine guns) with interchangeable parts map to nuclear weapons. John Hall’s stint at Harper’s Ferry was the Manhattan Project of its day (here the mapping is not exact, since semiconductors were spawned by the military-industrial research infrastructure around electronics that emerged after World War II, rather than through the Manhattan project itself).
9. Lincoln’s assassination is eerily similar to Kennedy’s. Just checking to see if you are still paying attention. The first person to call bullshit on this point gets a free copy of The Tycoons.
10. The Internet and container shipping taken together are to Moore’s Law as the railroad, steamship and telegraph networks taken together were to Hall’s Law. The electric power grid provides the continuity between Hall’s Law and Moore’s Law.
11. Each era changed employment patterns and class structures wholesale. Hall’s Law destroyed nobility-based social structures, created a new middle class defined by educational attainments and consumer goods, and created paycheck employment. Moore’s Law is currently destroying each of these things and creating a Trading Up class, a new model of free agency, and killing education-based reputation models.
12. A new mass entertainment model started in each case. With Hall’s Law it was Broadway (which led on to radio, movies and television). With Moore’s Law, I’d say the analogy is to reality TV, which like Broadway represents new-era content in an old-era medium.
13. At the risk of getting flamed, I’d say that Seth Godin is arguably the Horatio Alger of today, but in a good way. Somebody has to do the pumping-up and motivating to inspire the masses to abandon the old culture and embrace the new by offering a strong and simple message that is just sound enough to get people moving, even if it cannot withstand serious scrutiny.
14. Hall’s Law led on to the application of its core methods to people, leading to new models of high-school and college education and eventually the perfect interchangeable human, The Organization Man. Moore’s Law is destroying these things, and replacing them with Y-Combinator style education and co-working spaces (this will end with the Organization Entrepreneur, a predictably-unique individual, just like everybody else).
15. Hall’s Law led to the industrial labor movement. Moore’s Law is leading to a new labor movement defined, in its early days, by things like standardized term-sheets for entrepreneurs ( the 5 day/40 hour week issue of our times; YC-entrepreneurs are decidedly not the new capitalists. They are the new labor. That’s a whole other post).
16. And perhaps most importantly, each era suffered an early crisis of financial exploitation which led first to loophole closing, and then to a new financial system and corporate governance model. Jay Gould maps to the architects of the subprime crisis. No J. P. Morgan figure has emerged to really clean up the mess, but new corporate models are already emerging that look so unlike traditional ones that they really shouldn’t be called corporations at all (hence the pointless semantic debate around my history of corporations post; it is really irrelevant whether you think corporations are dying or being radically reinvented. You are talking about the same underlying creative-destruction reality).

The New Gilded Age

When Mark Twain coined the term Gilded Age, he wasn’t exactly being complimentary. For some reason, the term seems to be commonly used as a positive one today, by those who want to romanticize the period.

I started to read the book and realized that Twain had completely missed the point of what was happening around him (the focus of the novel is political corruption; an element that loomed large back then, but was ultimately a sideshow), so I abandoned it.

But he got one thing right: the name.

Hall’s Law created a culture that was initially a layer of fake gloss on top of much grimmer realities. Things were improving dramatically, but it probably did not seem like it at the time, thanks to the anxiety and uncertainty. Just as you and I aren’t exactly celebrating the crashing cost of computers in the last two decades, those who lived through the 1870s were more worried about farming moving ever westward (outsourcing) and strange new status dynamics that made them uncertain of their place in the world.

It took time for Gilded to turn into Golden (about 50 years by my estimate, things became truly golden only after World War II). There were decades of turmoil which made the lives of transitional generations quite miserable. The 1870s were a you’ll-thank-me-later decade, but for those who lived through the decade in misery, that is no consolation.

I abandoned The Gilded Age within a few pages. It is decidedly tedious compared to Tom Sawyer and Huckleberry Finn. Sadly, Twain’s affection for a vanishing culture, which made him such an able observer of one part of American life, made him a poor observer of the new realities taking shape around him.

He makes a personal appearance in the stories of both Vanderbilt and Rockefeller, and appears to have strongly disliked the former and admired the latter, though both were clearly cut from the same cloth.

To my mind, Twain’s best stab at describing the transformation (probably A Connecticut Yankee in King Arthur’s Court — note the significance of Connecticut) is probably much worse than the attempts of younger writers like Edith Wharton and later, of course, everybody from Horatio Alger to F. Scott Fitzgerald.

We are clearly living through a New Gilded Age today, and Bruce Sterling’s term “Favela Chic”  (rather unfortunately cryptic ; perhaps we should call it “Painted Slum”) is effectively analogous to “Gilded Age.”

We put on brave faces as we live through our rerun of the 1870s. We celebrate the economic precariousness of free agency as though it were a no-strings-attached good thing.  We read our own Horatio Alger stories, fawn over new Silicon Valley millionaires and conveniently forget the ones who don’t make it.

New Media tycoons like Arrington and Huffington fight wars that would have made the Hearsts and Pulitzers of the Gilded Age proud, while us lesser bloggers go divining for smaller pockets of attention with dowsing rods, driven by the same romantic hope that drove  the tragicomic heroes of P. G. Wodehouse novels to pitch their plays to Broadway producers a century ago.

History is repeating itself. And the rerun episode we are living right now is not a pleasant one.

The problem with history repeating itself of course, is that sometimes it does not. The fact that 1819-1880 map pretty well to 1959-2012 does not mean that 2012-2112 will map to 1880-1980. Many things are different this time around.

But assuming history does repeat itself, what are we in for?

If the Moore’s Law endgame is the same century-long economic-overdrive that was the Hall’s Law endgame, today’s kids will enter the adult world with prosperity and a fully-diffused Moore’s Law all around them.

The children will do well. In the long term, things will look up.

But in the long term, you and I will be dead.

 Some thanks are due for this post. It was inspired in part by Chris McCoy of YourSports.com, who badgered me about the Internet = Railroad analogy enough that I was motivated to go hunt for the best place to anchor a broader analogy. His original hypothesis is now the generalized point 10 of my list. Thanks also to Nick Pinkston for interesting discussions on the future of post-Moore’s Law manufacturing; the child may resurrect its devoured parent after all. Also thanks to everybody who commented on the History of Corporations piece. 

First, let me get an announcement out of the way: Tempo is now out on the Kindle. Buy it, give it as a gift, tweet it etc. Whew! That’s a big, high-priority item checked off my to-do list.

Speaking of priorities. I had one of my weirder Aha! moments: you can use the well-known Towers of Hanoi game as a metaphor to understand the behavior of Maslow’s hierarchy of needs (or any similar hierarchy of priorities) under changing life circumstances, and the role of compartmentalization as a costly coping strategy. Here’s a picture:

If the details and implications of the metaphor aren’t immediately obvious, read on for the help-text.

The Basic Metaphor

The game involves moving a pyramidal stack of disks from one of three pegs to another without ever letting a larger disk rest on a smaller one. The number of moves approximately doubles each time you add a disk (so the full 5-level Maslow hierarchy would take 31 moves, where this simplified 3-disk version takes 7 moves).

The metaphor works like this: at any given time, each stack of disks represents a life compartment. Sometimes your life is more compartmentalized (work vs. life or work vs. life vs. health), and sometimes, it is all a single beautiful symphony.

The “never stack a big disk on a small one” models the idea that most of us don’t consciously violate priorities in obviously avoidable ways. If we do, we recognize our “moment of weakness.”

But this is only a local sort of consistency: we apply prioritization ideas via pair-wise comparisons/tradeoffs while making specific decisions. You rarely have more than two priorities butting up against each other in any given decision.

So long as your life situation is stable, you can gradually integrate the various compartments and get back to a single stack. We call this getting into a “routine.”

But when your life shifts, fragmentation and compartmentalization necessarily intrude. If your life changes faster than you can get back to the healthy stack of priorities, you’ll end up a mess, unless you level-up your game and redefine your idea of stability (think George Clooney in Up in the Air).

Integrated vs. Fragmented Priorities

In a fragmented, compartmentalized state, disks on the same stack represent integrated priorities while disks on different stacks represent fragmented priorities.

Integrated priorities reinforce each other via strong positive-feedback loops (lots of small trade-off decisions, with healthy behaviors being reinforced each time you make the healthy choice), and you can manage them consciously.

Fragmented priorities cannot be managed consciously, and can lead to messed-up behaviors that are not aligned with your actual priorities. You also lose any mutual reinforcement effects via positive feedback loops.

This happens because, in a compartmentalized life, you only have opportunities to trade off priorities that are on the same stack, through individual, micro-level decisions

Your ability to manage tradeoffs between stacks is much more limited. You are reduced to vague ideas like “I need to spend more time with my kids” or “I need to go to the gym more.” I suspect most people allocate attention amongst compartmentalized stacks based on how many disks the stack has (which determines how hard the stack is to compute with) rather than the priority of the stack.

To take a simple example “work life balance” is extremely hard in a traditional industrial environment, where “work” happens at the workplace, and “life” happens at home.

But it is much easier in (say) a Googleplex-like workplace with childcare, gyms and healthy food options available right near your office.

So instead of having to think in terms of “I need to spend more time with my kids” you can trade off “get coffee between meetings right now, or play a game of ping-pong with my kid in the childcare room.”

Priority management turns into a bunch of bite-sized decisions.

Implications of the Metaphor

The metaphor suggests several interesting ideas:

1. Consciously violating priorities (putting a bigger disk on a smaller one)  can help you stabilize much faster (exponentially faster) in a new situation, but at the risk of bigger disks permanently damaging/crushing smaller ones.
2. Given a specific definition of priorities/disks, there is a maximum frequency of life changing disruptions you can handle while still getting to a stable integrated stack at least briefly between disruptions. If your life changes any faster, it will be in a perennially unstable state.
3. Fusing layers simplifies the game. Manage fewer categories. So working for a moving company fuses work and physical health.
4. How you frame priorities is crucial. A stack of disks that needs to be moved under a life transition may not need to be moved at all if you redefine them to be robust to such transitions. If you are a nomad, capable of living out of a suitcase or camel-pack, your life will resist disruption due to physical moves. If you live in a country with portable, government-provided healthcare, your health will resist disruption due to changing jobs.
5. You are not entirely in control of your life stack. The organization of society plays a huge role.
6. Trying to keep unfused priorities stable in transit (moving multiple discs at once) is a balancing act.

The part of the metaphor that interests me the most is the idea that there are positive feedback loops within compartments. The more you compartmentalize your life, the more you lose the benefit of such loops. I think this relates to how 10x dynamics can be catalyzed in your life. It also relates to the more comprehensive analysis of such stuff that Greg Rader recently posted.

There is a saying that goes back to Milo of Croton: lift a calf everyday and when you grow up, you can lift a cow. The story goes that Milo, a famous wrestler in ancient Greece, gained his immense strength by lifting a newborn calf one day when he was a boy, and then lifting it every day as it grew. In a few years, he was able to lift the grown cow. The calf grew into a cow at about the rate that Milo grew  into a man. A rather freakish man apparently, since grown cows can weigh over  1000 lb.  The point is, the calf grew old along with the boy.

I have been pondering this story for a couple of years, and it has led me to a very fertile idea about product design and entrepreneurship.

I call it the Milo Criterion: products must mature no faster than the rate at which users can adapt. Call that ideal maximum rate the Milo rate.

It seems like a simple and almost tautological thought, but it leads to some subversive consequences, which is one reason I have been reluctant to talk about it. The most subversive effect is that it has led me to abandon lean startup theory, which is now orthodoxy in the startup world.

As a consequence, I have mostly abandoned notions like product-market-fit, minimum viable product, pivots and the core value of “lean.”  I only use the terms to communicate with people who think in those terms.  And I can’t communicate very much within that vocabulary.

Physical Products and Services

Commercial airline travel is an example of a service product that satisfied the Milo Criterion during its evolution. In the early days, the user experience was not very different from riding a train or bus. Airport designers modeled their early efforts on railroad stations leading to familiar experience for early air travelers.

But modern air travel, which has evolved over nearly a century, is a very different complex of behaviors that has drifted far from bus and train travel. Just look at the enormous number of complex behaviors we’ve learned:

1. Checking in (online and off)
2. Security checks and rules about carrying liquids
3. Gates and air-bridges that look nothing like railroad stations
4. Checked baggage and hand baggage rules
5. Seat belts and rules about staying seated at certain times
6. Baggage carousels for retrieving luggage
7. Dealing with layovers
8. Online bidding for cheap ticket deals
9. Airport parking and car rental options
10. Duty-free shopping
11. Visas, passports, immigration, customs
12. Rules about when you can use electronic devices

We’ve been able to get this far successfully because we took our time. By a happy coincidence, the physical constraints of the technology limited the rate at which airline travel could evolve.

Another example is driving, which is estimated to involve close to 1500 separate sub-skills. It took us about a century to get to modern driving, GPS, zipcars and all, starting with horse-drawn carriages.

This sort of long evolution trajectory is generally the case for physical products and services. They are naturally rate-limited by a variety of factors, so they tend to evolve and mature in ways that naturally satisfy the Milo Criterion.

Web Products

Thanks to the lack of physical constraints, Web products can go from paper napkin to fully realized vision in months rather than decades.  They can evolve at rates that far exceed the Milo rate.

It takes decades to build out airline infrastructure in a country. Even the Chinese government cannot move arbitrarily fast.

For Web products though, there appear to be no real limits, other than typing speed, to how fast  you can build things. And thanks to certain pathological externalities, they perversely go as fast as they can. In fact going faster and faster has become the motto of the sector.

Successful Web products do seem to satisfy the Milo criterion though. I tried applying the criterion to a whole bunch of products, and it turns out to be a pretty reliable way to sort the two classes. Google Wave fails the criterion. Google+ satisfies it.

The criterion seems to be descriptive. Is it prescriptive?

Consider modern software development. A set of behaviors have emerged in the last decade that appear to increase the success rate of Web products:

1. Starting small and simple
2. Building, testing and iterating rapidly
3. Testing with active users as frequently as possible, starting as early as possible

It is important to note that these principles were discovered bottom-up by the practitioners, rather than prescribed top-down by the theorists. Theoretical codification followed rather than led. So it is possible to criticize the theories while accepting the empirically validated practices.

I have come to believe that much of the theorizing about why these methods work is mostly noise.  These theories — including lean startup theory — are mostly a set of just-so explanations that serve to motivate practically effective behaviors, the way religions motivate moral behavior.

So even though the theories lead to the diffusion of useful behaviors, the flaws limit their potential.

I won’t attempt a full critique here but offer a basic axiom for an alternative theory:

The primary reason these behaviors are effective is that they slow down the process of software development and maintain the optimal behavior modification rate for humans.

In other words, the Milo Criterion is not just descriptive. It is prescriptive.  It is the dominant dynamic for successful products.

It leads to alternative explanations for why the effective practices work. It leads to building blocks that are different from the ones recommended by lean startup theory.

In fact it is a pretty fertile starting point for a whole different approach to thinking about entrepreneurship and product development. I’ve been developing these ideas, mostly in private, and applying them to my own business decisions.

Slow Marketing

I don’t like being cryptic, but in this case, I am not going to elaborate further (at least not right now) because the very thought of the tedious and potentially acrimonious arguments that might result is enough to turn me off. I don’t have enough skin in the game to make it worthwhile. Perhaps I am getting old and conflict-averse.

So I am not going to share my explanations or alternative building blocks. In fact, I deleted a couple of much longer draft posts, something I rarely do, since I hate wasting writing effort.

I wrote this post primarily as a way of saying hello to others who might already be thinking along the same lines I am. If you are, chances are the Milo Criterion will spark some productive thinking for you. If not, at least you learned the story of Milo of Croton, for use at cocktail parties.

I will share one more clue: I’ve started calling my developing theory slow marketing. Read into that what you will.

I find myself feeling strangely uncomfortable when people call me a generalist and imagine that to be a compliment.  My standard response is that I am actually an extremely narrow, hidebound specialist. I just look like a generalist because my path happens to cross many boundaries that are meaningful to others, but not to me. If you’ve been reading this blog for any length of time, you know the degree to which I keep returning to the same few narrow themes.

I think I now understand the reason I reject the generalist label and resonate far more with the specialist label. The generalist/specialist distinction is an extrinsic coordinate system for mapping human potential.  This system itself is breaking down, so we have to reconstruct whatever meaning the distinction had in intrinsic terms. When I chart my life course using such intrinsic notions, I end up clearly a (reconstructed) specialist.

The keys to this reconstruction project are: the much-abused idea of 10,000 hours of deliberate practice, the notion of grit, and an approach to keeping track of your journey through life in terms of an intrinsic coordinate system. Think of it as replacing compass or GPS-based extrinsic navigation with accelerometer and gyroscope-based  inertial navigation.

I call the result “the calculus of grit.” It is my idea of an inertial navigation system for an age of anomie, where the external world has too little usable structure to navigate by.

The Generalist-Specialist Distinction

The generalist/specialist distinction constitutes an extrinsic coordinate system. We think of our environment as containing breadth and depth dimensions. The breadth dimension is chopped up by disciplinary boundaries (whether academic, trade-based or business-domain based), while the depth dimension is chopped up by markers of validated progressive achievement. What you get is a matrix of domains of endeavor: bounded loci within which you can sustain deepening practice of some skilled behavior.

The boundedness is key. Mathematicians do not suddenly discover, in the 10th year of their practice, that they need advanced ballroom dancing skills to progress further. Ballroom dancers do not suddenly encounter a need for advanced aircraft engine maintenance skills after a few years of practice. Based on your strengths, you can place fairly safe bets early on about what you will/will not need to do if you make your home somewhere in the matrix.

Or at least, you used to be able to. I’ll get to how these expectations from the twentieth century are breaking down.

There is a social structure that conforms to these breadth/depth boundaries as well. A field of practitioners in each domain, stacked in a totem pole of increasing expertise, that legitimizes the work of individuals and provides the recognition needed for both pragmatic ends (degrees and such) and existential ends (“recognition” in the sense of say, Hegel).

In his book Creativity, Mihaly Csikzentmihalyi made up exactly such a definition of extrinsically situated creativity as the behavior of an individual within a field/domain matrix.

We are now breaking away from this model. Ironically, Csikzentmihalyi’s own work makes little sense within this model that he helped describe in codified ways; his work makes a lot more sense if you don’t attempt to situate it within his nominal home in psychology.

Extrinsically situated creativity with reference to some global, absolute scheme of generalist/specialist dimensions is unworkable. At best we can hope for local, relative schemes and an idea of intrinsically situated individual lives.

The Vacuity of Multi-Disciplinarity

The problem with this generalist/specialist extrinsically situated creativity model is that the extrinsic frames of references are getting increasingly dynamic, chaotic and murky. To the point that the distinction is becoming useless. Nobody seems to know which way is up, which way is down, and which way is sideways. If you guess and get lucky, the answers may change next year, leaving you disoriented once more.

The usual response to this environment is to invoke notions of multi-disciplinarity.

Unfortunately, this is worse than useless. In the labor market for skilled capabilities, and particularly in academia, multi-disciplinarity is the equivalent of gerrymandering or secession on an already deeply messed-up political map.  Instead of votes, you are grubbing for easily won markers of accomplishment. Its main purpose (in which it usually fails) is to create a new political balance of power rather than unleash human potential more effectively.

The purpose is rarely to provide a context for previously difficult novice-to-master journeys.

How do I know this? It’s patently obvious. If it takes 10,000 hours (K. Anders Ericsson’s now-famous threshold of deliberate practice, thanks to Gladwell, which translates to about 10 years typically) to acquire mastery in any usefully bounded domain, and you assume that there is at least one generation of pioneers who blazed that path to a new kind of mastery, what are you to make of fields that come and go like fruit flies in 2-3 years, in sync with business or funding cycles? The suspicious individual is right to suspect faddishness.

I have come to the conclusion that if I cannot trace a coherent history of at least 20 years for something that claims the label “discipline,” it isn’t one.

The problem with this though is that increasing amounts of valuable stuff is happening outside disciplines by this definition. It isn’t multi-disciplinary. It isn’t inter-disciplinary. It is simply non-disciplinary. It’s in the miscellaneous folder. It is so fluid that it resists extrinsic organization.

So given that most excitement centers around short-lived fruitfly non-disciplines, how do people even manage to log 10,000 deliberate practice hours in any coherent journey to mastery? Can you jump across three or four fruit-fly domains over the course of a decade and still end up with mastery of something, even if you cannot define it?

Yes. If you drop extrinsic frames of reference altogether.

The Compass and the Gyroscope

We are used to describing movement in terms of x, y and z coordinates, with respect to the Greenwich meridian, the Equator and sea level. Our sense of space is almost entirely based on such extrinsic coordinate systems (or landmarks within them). Things that we understand via spatial metaphors naturally tempt us into metaphoric coordinate systems like the depth/breadth one we just talked about. In academic domains, for instance, you could say the world is mapped with reference to an origin that represents a high-school graduate, with disciplinary majors and years of study forming the two axes that define further movement.

Somewhere in graduate school, I encountered an idea that blew my mind: you can also describe movement entirely intrinsically. Actually, I had encountered this idea before, in vague popular science treatments of Einstein’s general theory of relativity, but learning the basics of the math is what truly blows your mind.

The central idea is not hard to appreciate:  imagine riding a complicated roller coaster and keeping track of how far along you are on the track, how you’ve been turning, and how you’ve been twisting. That much is easy.

What is not easy is appreciating that that’s all you need. You can dispense with extrinsic coordinate systems entirely. Just keeping track of how those three variables (known as arc-length, curvature and torsion if my memory serves me) are changing, is enough. For short periods, you can roughly measure them using just your intrinsic sense of time and how your stomach and ears feel. To keep the measurements precise over longer periods, you need a gyroscope, an accelerometer and a watch.

If you want motifs for the two modes of operation, think of it as the difference between a magnetic compass and a gyroscope (these days, GPS might be a better motif for the former, but the phrase “the compass and the gyroscope” has a certain ring to it that I like).

We need another supporting notion before we can construct an intrinsic coordinate system for human lives.

Behavioral Boundedness

Remember that the primary real value of an extrinsically defined discipline in a field/domain matrix is predictable boundedness. Mathematicians can trust that they won’t have to suddenly start dancing halfway through their career to progress further.

This predictability allows you to form reasonable expectations for decades of investment, and make decisions based on your upfront assessment of your strengths, and expectations about how those strengths will evolve as you age.

If I decide that I have certain strengths in mathematics and that I want to bet on those strengths for a decade, to get to mastery, I shouldn’t suddenly stumble into a serious weakness along the way that blocks me, like a lack of natural athleticism.

So a disciplinary boundary is very useful if it provides that kind of predictability. I call this behavioral boundedness. An expectation that your expected behaviors in the future won’t wander too far out of certain strengths-based comfort zones you can guess at fairly accurately, upfront. Before putting in 10,000 hours.

What happens when that sort of predictability breaks down? It is certainly happening all over the place. For instance, I didn’t realize I lacked the strengths needed for a typical career in aerospace engineering (the sort high-school kids fantasize about when they first get interested in airplanes and rockets) until well into a PhD program in the subject. Fortunately, I was able to pivot and head in another direction with almost no wasted effort. Few people are that lucky.

There are domains where the boundedness is very weak indeed. The upfront visible boundedness is a complete illusion. Marketing is one such domain. You might get into it because you love creative messaging or talking to people. You may discover the idea of positioning two years into the journey and realize that creativity in messaging is a sideshow, and the real job is somewhat tedious analysis of the mental models of prospects. A further two years down the road, you may discover that to level-up your game once more, you need to become a serious quantitative analytics ninja and database geek.

This can also work out in positive ways. You might wake up one fine day and realize that your life, which makes no sense in nominal terms, actually adds up to expertise in some domain you’d never identified with at all. That actually happened to me with respect to marketing.  On paper, I am the opposite of a marketer. I have a PhD in aerospace engineering, am introverted, and write in long-winded and opaque ways rather than in catchy sound-bytes.

Nevertheless, at some point I realized with a shock that I had accidentally logged several thousand hours along a marketing career path without realizing it. I had just completely misunderstood what “marketing” meant based on the popular image the field presents to novices.

When I went free-agent a few months ago, most of my consulting leads I had coming in had to do with marketing work. This did not surprise me, but it certainly surprised my father and several close friends, who assumed I was doing some sort of technical consulting work around computational modeling and scientific computing.

I’d never thought of myself as a marketer. A computational modeler, yes. A hustler perhaps. A fairly effective corporate guerrilla, yes. A marketer, not really. I viewed my previous marketing work as the work of a curious tourist in a strange land. I viewed my marketing writing as outsider-anthropology amongst strange creatures. But apparently, that’s not how others view me.

Looking back, and trying to make sense of my life in retrospect as “the training of an accidental marketer,” it makes sense though: I’ve logged the right mix of complementary experiences. Marketing is still not my primary identity though (that would mean returning to a Procrustean bed of disciplinary identity).

Many people luck out like me, accidentally. We recognize what particular path to mastery we’re on, long after we actually get on it.

Many do not. They bum around in angsty anomie, craving structure where none exists, and realizing after a decade of wandering that they’ve unfortunately gotten nowhere.

Is it possible to systematically do things to put yourself on a path to mastery, and know you’re on one, without actually knowing what that path is until you’re already far down it?

Inside and Outside Views of Grit

If there is no external frame of reference, how do you know where you are, where you are going and whether you are progressing at all, as opposed to bumming around?

Can you log any old time-sheet of 10,000 hours, slap a label on it, and claim mastery?

Thankfully, intrinsic navigation is not quite that trite.

A clue to the mystery is the personality trait known as grit, probably the best predictor of success in the modern world.

Grit is the enduring intrinsic quality that, for a brief period in recent history, was coincident with the pattern of behavior known as progressive disciplinary specialization.

Grit has external connotations of extreme toughness, a high apparent threshold for pain, and an ability to keep picking yourself up after getting knocked down. From the outside, grit looks like the bloody-minded exercise of extreme will power. It looks like a super-power.

I used to believe this understanding of grit as a superhuman trait.  I used to think I didn’t possess it. Yet people seem to think I exhibit it in some departments. Like reading and writing. They are aghast at the amount of reading I do. They wonder how I can keep churning out thousands of words, week after week, year after year, with no guarantee that any particular piece of writing will be well-received.

They think I must possess superhuman willpower because they make a very simple projection error: they think it is hard for me because it would be hard for them. Well of course things are going to take superhuman willpower if you go after them with the wrong strengths.

For a while, I went around calling this faux-grit. The appearance of toughness. But the more I looked around me at other people who seemed to display grit in other domains, the more I realized that it wasn’t hard for them either. What they did would merely be superhuman effort for me. Faux grit and true grit are the same thing (the movie True Grit is actually quite a decent showcase of the trait; it showcases the superhuman outside/fluid inside phenomenon quite well).

So what does the inside view of grit look like? I took a shot at describing the subjective feel in my last post on the Tempo blog. It simply feels like mindful learning across a series of increasingly demanding episodes that build on the same strengths.

But the subjective feel of grit is not my concern here. I am interested in objective, intrinsically measurable aspects of grit that can serve as an internal inertial navigation system; a gyroscope rather than  GPS.

The Grit Gyroscope: Reworking, Referencing, Releasing

In physical space, latitude, longitude and altitude get replaced by arc-length, curvature and torsion when you go intrinsic.

In endeavor space, field, domain and years of experience get replaced by three variables that lend themselves to a convenient new 3Rs acronym: reworking, referencing, releasing (well, technically, it is internal referencing and early-and-frequent releasing, but let’s keep the phrase short and alliterative). I believe the new 3Rs are as important to adults as the old ones (Reading, wRiting and aRithmetic) are for kids.

Reworking

I stumbled upon rework as a key variable when I tried to answer a question on Quora: what are some tips for advanced writers?

Since writing is something everybody does, logging 10,000 writing hours is something anyone can do. My aha! moment came when I realized that it isn’t the writing hours that count, it is the rewriting hours. Everybody writes. People who are trying to walk the path towards mastery rewrite. I won’t say more about this variable. If you want a worked example, read my Quora answer. If you want a quick and pleasant read on the subject, Jason Fried’s Rework gets at some of the essential themes (though perhaps in a slightly gimmicky way).

Referencing

For referencing, my clue was my recent discovery that new readers of this blog often dive deep into the archives and read nearly everything I’ve written in the last four years. I dubbed it the ribbonfarm absurdity marathon because I didn’t understand what would possess anyone to undertake it.

But then I realized that I write in ways that practically demand this reading behavior if people really want to get the most value out of what I am talking about: I reference my own previous posts a lot. Not to tempt people into reading related content, but out of sheer laziness. I don’t like repeating arguments, definitions or key ideas. So I back-link. I do like most of my posts to be stand-alone and comprehensible to a new reader though, so I try to write in such a way that you can get value out of reading a post by itself, but significantly more value if you’ve read what I’ve written before. For example, merely knowing what I mean by the word legibility, which I use a lot, can increase what you get out of some posts by 50%. This is one reason blogging is such a natural medium for me. The possibilities of hyperlinking make it easy to do what would be extremely tedious with paper publishing.

The key here is internal referencing. I use far fewer external reference points (there’s perhaps a dozen key texts and a dozen papers that I reference all the time). It sounds narcissistic, but if you’re not referencing your own work at least 10 times as often as you’re referencing others, you’re in trouble in the intrinsic navigation world.  Instead of developing your own internal momentum and inertia, you are being buffeted by external forces, like a grain of pollen being subjected to the forces of Brownian motion.

Releasing

And finally, releasing. As in the agile software dictum of release early and often. In blogging, frequency isn’t about bug-fixing or collaboration. It isn’t even about market testing (none of my posts are explicitly engineered to test hypotheses about what kind of writing will do well).  It is purely about rational gambling in the dollar-cost averaging sense. It is the investing advice “don’t try to time the market” applied to your personal work.

If the environment is so murky and chaotic that you cannot strategically figure out clever moves and timing, the next best thing you can do is just periodically release bits of your developing work in the form of gambles in the external world. I think there’s a justifiable leap of faith here: if you are work admits significant reworking and internally-referencing, you’re probably on to something that is of value to others.

If a post happens to say the right thing at the right time, it will go viral. If not, it won’t. All I need to do is to keep releasing. This realization incidentally, has changed my understanding of phenomena like iteration in lean startups and serial entrepreneurs who succeed on their fifth attempt. It’s mostly about averaging across risk/opportunity exposure events, in an environment that you cannot model well.  I am pretty sure you can apply this model beyond blogging and entrepreneurship, but I’ll leave you to figure it out.

These three variables together can measure your progress along any path to mastery. What’s more, they can be measured intrinsically, without reference to any external map of disciplinary boundaries. All you have to do is to look for an area in your life where a lot of rework is naturally happening, maintain an adequate density of internal referencing to your own past work in that area, and release often enough that you can forget about timing the market for your ouput.

What does navigating by these three variables look like from the outside?

If you only do a lot of internal referencing, that’s like marching along a straight, level road.

If you do a lot of internal referencing and a lot of rework, that’s like marching along a steady uphill road that’s gradually getting steeper from an external point of view (in other words, you are on your own exponential path of progress). What you are doing will look impossible to observers. It may look like you are marching up a vertical cliff. A great example is the Silicon Valley archetype of the 10x engineer.

And finally, if you are releasing frequently, that’s like turning and twisting: spiraling around an increasingly steep mountain (or zig-zagging up via a series of switchbacks).

The Path of Least Resistance

Navigating with the 3Rs as an adult isn’t enough. You still have to recover the value the old disciplinary model provided: behavioral boundedness. Whether you are navigating intrinsically or extrinsically, suddenly running into a mountain — a major weakness — is just as bad.

The key here is very simple and very Sun Tzu: with respect to the external world, take the path of least resistance.

Why? Think of it this way. The disciplinary world very coarsely measured your aptitudes and strengths once in your lifetime, pointed you in a roughly right direction and said “Go!” The external environment had been turned into a giant obstacle course designed around a coarse global mapping of everybody’s strengths.

So there was no distinction between the map of the external world you were navigating, and the map of your internal strengths. The two had been arranged to synchronize. If you navigated through a map of external achievement, landmarks and honors, you’d automatically be navigating safely through the landscape of your internal strengths.

But when you cannot trust that you’ve been pointed in the right direction in a landscape designed around your strengths, you cannot afford to navigate based on a one-time coarse mapping of your own strengths at age 18.

If you run into an obstacle, it is far more likely that it represents a weakness rather than a meaningful real-world challenge to be overcome, as a learning experience.

Don’t try to go over or through. It makes far more sense to go around. Hack and work around. Don’t persevere out of a foolhardy superhuman sense of valor.

Hard Equals Wrong

If it isn’t crystal clear, I am advocating the view that if you find that what you are doing is ridiculously hard for you, it is the wrong thing for you to be doing.  I maintain that you should not have to work significantly harder or faster to succeed today than you had to 50 years ago. A little harder perhaps. Mainly, you just have to drop external frames of reference and trust your internal navigation on a landscape of your own strengths.  It may look like superhuman grit to an outsider, but if it feels like that inside to you, you’re doing something wrong.

This is a very contrarian position to take today.  Thomas Friedman in particular has been beating the “harder is better” drum for a decade now, most recently in his take on the London riots, modestly titled A Theory of Everything (Sort Of):

Why now? It starts with the fact that globalization and the information technology revolution have gone to a whole new level. Thanks to cloud computing, robotics, 3G wireless connectivity, Skype, Facebook, Google, LinkedIn, Twitter, the iPad, and cheap Internet-enabled smartphones, the world has gone from connected to hyper-connected.

This is the single most important trend in the world today. And it is a critical reason why, to get into the middle class now, you have to study harder, work smarter and adapt quicker than ever before. All this technology and globalization are eliminating more and more “routine” work — the sort of work that once sustained a lot of middle-class lifestyles.

The environment that really matters isn’t the external world. It is pretty much pure noise. You can easily find and process the subset that is meaningful for your life. It isn’t about harder, smarter, faster. If it were, I’d be dead. I’ve been getting lazier, dumber and slower. It’s called aging. I think Friedman is going to run out of superlatives like “hyper-” before I run out of life. If I am wrong, the world is going to collapse before he gets around to writing The World is Hyper-Flatter-er.  Humans are simply not as capable as Friedman’s survival formula requires them to be.

Exhortation is pointless. Humans don’t suddenly become super-human just because the environment suddenly seems to demand superhuman behavior for survival. Those who attempt this kill themselves just as surely as those dumb kids who watch a superman movie and jump off buildings hoping to fly.

It is the landscape of your own strengths that matters. And you can set your own, completely human pace through it.

The only truly new behavior you need is increased introspection. And yes, this will advantage some people over others. To avoid running faster and faster until you die of exhaustion, you need to develop an increasingly refined understanding of this landscape as you progress.  You twist and turn as you walk (not run) primarily to find the path of least resistance on the landscape of your strengths.

The only truly new belief you need is that the landscape of disciplinary endeavors and achievement is meaningless. If you are too attached to degrees, medals, prizes, prestigious titles and other extrinsic markers of progress in your life, you might as well give up now. With 90% probability you aren’t going to make it. It’s simple math: even if they were worth it, as our friend Friedman notes with his characteristic scare-mongering, there simply isn’t enough to go around:

Think of what The Times reported last February: At little Grinnell College in rural Iowa, with 1,600 students, “nearly one of every 10 applicants being considered for the class of 2015 is from China.” The article noted that dozens of other American colleges and universities are seeing a similar surge as well. And the article added this fact: Half the “applicants from China this year have perfect scores of 800 on the math portion of the SAT.”

If you’re paying attention to the Chinese kids who score a perfect 800, you’re paying attention to the wrong people. I mean, really? You should worry about some Chinese kid terrorized into achieving a perfect-800 math score by some Tiger Mom, and applying to Grinnell College?

It’s the Chinese kids who are rebelling against their Tiger Moms, completely ignoring the SAT, and flowing down the path of least resistance that you should be worried about.  After all Sun Tzu invented that whole idea.

So rework, reference, release. Flow through the landscape of your own strengths and weaknesses. Count to 10,000 rework hours as you walk. If you aren’t seeing accelerating external results by hour 3300, stop and introspect. That is the calculus of grit. It’s the exponential human psychology you need for exponential times. Ignore everything else.

Factoid: this entire 4000-plus word article is a working out of a 21-word footnote on page 89 of Tempo. That’s how internally-referenced my writing has become. Never say I don’t eat my own dogfood.

Post your comments over on the original post on the Tempo blog.

I am in Memphis, where I plan to meet up with Daniel Pritchett, some local entrepreneurs at a startup incubator, and anyone else who might be around. Next stop, St. Louis on Tuesday. As far as I know, I have no readers there, but I wanted to check out the Billy Goat chip company, maker of my favorite chips. If anybody is out there, it’d be great to meet up. From St. Louis I head to Omaha and after that, the road-trip basically goes into a sights-over-people mode, since my destinations in Nebraska and South Dakota (North Platte for a second visit to Bailey Yard, Alliance for Carhenge and Rapid City for Deadwood) aren’t places I am likely to find any readers. I’d be shocked to find somebody beyond Omaha. After South Dakota, I head to Jackson Hole in the heart of Yellowstone, where oddly enough I do have someone to stay with. After that, depending on how much time I have left, I might dawdle or dash my way to Vegas, the end point for this leg.

Posts from Week 2

• Strategies, Counter-examples and the UnAha! Experience
• On Ritual Time
• The Author’s Journey and the Blogger’s Journey
• Functional Fixedness and Kata Learning
• Freytag Staircases in Nashville
• The Car/Truck Ratio

This is a guest post by Stefan King, who has been bootstrapping his intriguing new blog, RecordedViews. I happened to mention in an offline discussion that I’d had Paglia’s classic, Sexual Personae, on my reading list for a while. Stefan offered to write a review/summary/introduction. To my surprise, he found some ideas from Tempo to be a useful framework for analysis. I totally did not bribe him to come to that conclusion. This post should make stand-alone sense even if you haven’t read Tempo. He also has another post up on his blog right now that should interest ribbonfarm readers.

In 1990, the art historian Camille Paglia provoked feminists and post-modernists with her controversial book Sexual Personae.  Paglia’s goal was to show the pagan patterns of continuity in western culture, and to expose feminist ideals as misguided wishful thinking. Now, two decades later, it is time to dig Sexual Personae out of the cultural compost heap and see if something interesting has grown there. Paglia has a highly sensitive intuition about great works of art, and she is a talented psychoanalyst of artists. The value of the book lies in those intuitions, which we can now study with the benefit of hindsight.

The Venus of Willendorf

The grand narrative of western archetypes, or “sexual personae” as Paglia calls them, starts with the Venus of Willendorf, a small statuette from the Stone Age. It is a faceless lump of feminine flesh, possibly a fertility talisman. It contrasts perfectly with anything civilized: there is no line, no shape, no stillness, and no Apollonian light. In those times, nature’s domination of humanity was total.

The Bust of Nefertiti

The next phase comes as the Egyptians develop their imagination in myth, and worshiped the gods of the sky and the earth alike. The human image becomes more conceptualized. To run an empire and canalize the flood of the Nile, they needed abstract thought and symbols. The balance between a cult of the demonic earth and sunlit clarity reaches the first height of archetypal beauty: the bust of Nefertiti. It is a created ideal of shape and form, still, with a sphinx-like androgynous face. It reveals an artistic strategy of denying chaos. Art is an attack on nature, that tries to push its random cruelty away. It is a frozen beauty, rendered legible to the intellect. In contrast to Dionysian identifications such as drinking and dancing, Apollonian beauty appears when aggressive eyes dominate nature. “Paganism is pictorialism plus the will-to-power. It is ritualism, grandiosity, colossalism, sensationalism.” That last sentence is an example of a dubious type of claim Paglia likes to make: “X is Y”-propositions. More on that later.

The techniques of form and shape are refined by the ancient Greeks, who made sense of the artistic possibilities. The god Apollo, originally depicted as a virile wolf-man, became an immature, beautiful and narcissistic boy who stares in the distance with dreamy eyes. That epoch ends with the spectacles of the ancient Romans. Art is an escape route from ethics.

The Mona Lisa

Then the arts reach a phase of stability in centuries of latent homoerotic religious icons, until Renaissance gives enough artistic momentum to a Florentine crucible of gay geniuses, who use pagan images to create an explosion of sexual personae. Examples are the Femme Fatale, Greek heroes, and angels such as the cherubs and seraphs. These creations culminate into a second artistic height: the Mona Lisa. “Leonardo’s Mona Lisa is the premiere sexual persona of western art. She is the Renaissance Nefertiti, eternally watching.”

The richness of sexual personae left behind by the Renaissance is then ritualised and refined by the romantic poets and painters. Paglia notes that every work of art implies a vision of human nature, and of nature itself. The duality is between the visions of Rousseau and de Sade. Rousseau believed that man is naturally good but corrupted by society, while de Sade sees humans as naturally cruel, driven by inner demonic forces. Romanticism is the struggle between these visions.

The early romantic poets approach natural beauty carefully, as they become aware of the closeness of nature to sex and cruelty. Just like their predecessors, they embrace androgyny as the only solution. Sexual chaos is excised or repressed, Apollonian beauty is taken as far as possible, to the point where a sexual personae is fully objectified for ritual worship by the artist. An example of the grotesque end state is The Picture of Dorian Gray by Oscar Wilde, another gay genius. Paglia even counts Edvard Munch’s The Scream as a decadent work, where a fetus is cast as a sexless sexual persona.

Narrative Structure and Sexual Personae

Disclaimer: I’m not burdening myself with a discussion of her views on post-modernism and feminism. Instead, I am trying to reconstruct Paglia’s perspective into something that is legible to those with a more scientific perspective. I’m not dealing with the book as a whole.

Paglia’s view on western culture seems correct to me, but she doesn’t express it very well. She fills page after page with “X is Y” statements that define a unfalsifiable Freudian space where anything goes. Like Nietzsche, Freud and Jung, she sees much, although you shouldn’t call it science.

We can interpret the art discussed in Sexual Personae as a single narrative with a double Freytag Triangle, and you can see the subconscious ‘grand strategy’ within the Western artistic tradition, and understand better what makes certain archetypes beautiful (click for larger image; the two peaks are the Bust of Nefertiti and the Mona Lisa respectively, while the beginning and end are the Venus of Willendorf and Modernism respectively).

(Note from Venkat: the double Freytag triangle is a model of narrative structure I made up in Tempo to represent the rise and fall of dramatic tension/narrative entropy in individual “decision stories” based on the classical Freytag triangle, and incorporating elements of Joseph Campbell’s monomyth model. Stefan has used it in this post to represent the structure of a grand narrative rather than an individual narrative).

If it is true that the history of western archetypes is a unified whole, then the narrative will have the emotional tones of a story. Every line in the double Freytag Triangle is an epoch with a characteristic tempo. In the summary I hinted at the emotional tones of the phases, but now we can explore them more directly and see what we find.

The left side of the first triangle is the exploration phase with a volatile, dissipative tempo. Recall that the Egyptians worshiped both the earth and the sky. The Egyptian gods are obviously energetic, with their animal heads. The mythology is rough and violent.

The top of the first triangle is the point where the exploration leads up to the recognition of an exploitable pattern: in this case, the bust of Nefertiti. The artist discovers that he can use his eyes to fixate nature into a light, conceptual stillness. He can locally repress the demonic powers of nature. Apollonian beauty is found.

The right side of the first triangle is always a decrescendo with emotional relief. The pattern fits: the sexual persona of Apollo becomes the archetypal beautiful boy, who is calm and dreamy. Unlike an Egyptian god, he is too weak to get things done, and he doesn’t have to.

When the treadmill of the religious icons is broken, culture enters the second crescendo which, according to Tempo, is characterized by a “high effort, low-coherence increase in momentum.” The works of the renaissance are energetic, interesting and diverse. I’ve never been bored in museums that have paintings from that age.

After the Mona Lisa – Leonardo’s externalisation of his theory of nature – art enters a retrospective phase with a “mix of joy and sorrow.” After the romantic poets figured out the rules, decadent painters ritualise the array of archetypes of the Renaissance into calm dreams, languid poses, androgynous and useless gentlemen, and dead bodies lying in the brambles. The decadent poets invent the genre of the gothic novel, with a sombre rhythm and impotent ghosts.

Art and Instinct

Paglia shows in Sexual Personae that art is never innocent; that there is no such thing as frivolous beauty. Archetypes are serious business, apparently, because of how human instinct responds to nature. Not ‘nature’ as in trees and flowers and lions, but ‘nature’ as in biology: the natural world is simultaneously our nurturing mother and our destroyer. The fear of death and the desire for sex are the primal motivational currents, that drag thoughts, eyes, and actions along with them.

This is a given for evolutionary psychologists and biologists, but not so long ago there was not much proof. Then, it took the intuitions of Nietzsche and Freud to figure it out. Camille Paglia channels the same spirit: she sees how the instinct of sex, and the fear of death, inevitably take shape in the archetypes that populate great art.

Post your comments over on the original post on the Tempo blog.

I am in Ann Arbor, MI as I write this, preparing to head south tomorrow. The plan is to wander down to New Orleans over the week, and then start up along the Mississippi next week. For the coming week, I have Atlanta plans nailed down and Nashville and New Orleans plans almost nailed down. According to Google Maps, Dayton, Cincinnati, Lexington, Knoxville, Montgomery and Mobile are along the route. If you suspect you are within a reasonable band off this route, give me a holler.

Here are links to my the posts I liveblogged on the Tempo blog during the first week. Delay-blogged rather.

• Time Travel for Ghosts
• Darwin, Some Rationalists and the Joker
• Peak Oil and the Tempo of the Earth
• Talking Temporal Illegibility in Montreal (video)
• Why Some Drives are Fun (video)
• The One Way of the Beginner
• Haircuts and the Guy Clock
• An Evening of Pace, Pace, Lead with Chuck
• Island Time vs. Mainland Time (video)
• The Tempo of Food
• A Moment of Silence with John Boyd

Some reflections on Week 1 follow, for those interested in the metatext.

So week 1 of the road trip was, hmm… how do I put it? Surreal. Equal parts enlightening, puzzling and just plain odd. This road trip project was based on the idea that I could combine book promotion with a sort of literary-performance-art approach to market research. The term “market research” somehow seems inappropriate for a guy bumming around sleeping on people’s couches, but since the net effect is that I am slowly beginning to understand who reads my writing and why, I guess you could call it that. Maybe there’s a second book waiting to come out of this road trip that I could call “market research for bloggers.”

I am experimenting with a lot of things at once: high-tempo/short-length posting, video, photography, interviews and other random things. So prepare to be somewhat befuddled. I know I am. One thing I’ve learned already is that when you are generating raw material very quickly, it becomes far harder to process it into insight, especially in real time. In other words, this road trip is turning out to be far more interesting than might be coming through in the writing (the video in particular is probably awful; very new medium for me, at least in the improv self-recording sense). I dislike reality-TV style blogging (my life, even on a road trip is not a good spectator sport), so I’ve been trying instead to focus on the key memes that I am encountering in my conversations and solo experiences, rather than the people.

But really rich and dense conversations and experiences don’t yield either value or key memes immediately. The real-time story, rearranged with the benefit of hindsight into a remembered story, makes for much more interesting reading. But on the other hand, liveblogging does seem to be more “live” and immediate, and that’s worth something. At least to me. By capturing a view of the journey in real time, ambiguity, dissonances and all, I am sort of creating a reference narrative which I hope to return to later to jog my memory when I need to.

There is one specific way in which meeting people in real life is interesting and immediately valuable. In my online interactions with people, I usually see only a tiny slice of who they are, and that in the context of my life, since the interactions are in places like the comments section of this blog, where I play host. Meeting them in person as a real-world guest instead of virtual host allows me to see why and how my writing fits into various individual lives. It is particularly interesting to see my writing in relation to the other things in people lives, and how people integrate the information they consume from different sources into a whole that makes sense in their lives. I admit the view is somewhat unsettling, since I am only used to seeing my thinking within the context of my own life. The process feels at least a little voyeuristic, since my ideas seem to provide a more intimate look into people’s lives than strangers normally get. It felt weird, for instance, to hear a reader explain the Gervais Principle to a friend who had never heard of me.

This piece was originally published in 1999, and is based on a 1996 camping trip. My thoughts have been drifting back to this experience lately, so I thought I’d share it. It’s a little overwrought, but it is significant for me personally because my writing voice first started emerging with this piece. Besides a few copy-editing and internationalization touches, I haven’t changed anything.

- One -

Namdapha, in an obscure corner of the subcontinent. Unobtrusive in a list of National Parks, among more famous names like Kaziranga and Corbett.

There is magic here.

I mean it. Many people know about it, and they carefully try to keep the place safe, by calling it a “National Park”. Not because there are tigers here, not because there are snow leopards, but because there is magic. There are other places that are wild — but nowhere else is there magic. You ride your bus through quaint places with names like Digboi and Miao, quaint but not magical; you pass through miles of lightly wooded country, green and natural, but again, not magical.

And then you enter.

The forest we are driving through, Wada explains over the noise in the cramped jeep, is secondary growth. “Real sub-tropical rainforest takes several thousand years to develop,” he yells. He waves at the yellow patches visible between clumps of trees, “When we get into the real thing, you’ll see the trees cover the land completely, with no gaps. There will also be dense undergrowth.” The jeep trundles through a rickety gate. We are now inside Namdapha National Park.

Namdpaha, land of four big cats. Land of the Hoolock Gibbon. The barren and cold Eastern Himalayas to the north, beyond the Dapha Bum Ridge. To the south, a troubled Burma across the Patkai Ranges. In the middle, the valley of the Noa Dehing and the Namdapha, the last magic kingdom on Earth.

At Deban, the trail starts. It cuts straight through 1985 square kilometers of untamed jungle, a relic of a time when ambitious men tried to build a road to Vijaynagar on the eastern end of the reserve. They underestimated the steady power of relentless rain, nine months a year.

I stand on the river bank. The others are still chatting excitedly about the sambar tracks we saw on the other side of the river. I look up; the Andromeda Galaxy is startlingly easy to pick out. The first piece of magic. Or is it just the clear forest skies? I am uncomfortable here, I do not really fit in.

We are ready to start, early the next morning. The consultations in torchlight over the planned route ended last evening. Hazrababu, our guide, has promised us at least a glimpse of the Hoolock Gibbon, the only ape native to India. The porters, Tiger, Gurav and Gangura, help each other hoist the heavy packs of rations onto their backs. They are silent, with impassive faces. Hazrababu makes up for their taciturnity with incessant chatter.

Crunch-snap-crunch, one step, another step, another step, the hike has started. Occasional laughter as some of the others exchange jokes. We stop every now and then, as someone spots a fluttering flash of color in the undergrowth. Salim Ali’s Pictorial Guide to the Birds of the Indian Subcontinent is pulled out hastily and the binoculars handed around. A list is slowly growing, Black Drongo, Himalayan Treepie, Himalayan Whistling Thrush. Joshi has become a butterfly fanatic. “Did, you know,” he exclaims, peering at his guidebook, “there are some 3000 butterfly species in this region? That’s nearly half the world’s total!” Another list is growing as well, Dark Judy, Leopard Lacewing, Common Map. Wada is not as good with the butterflies as he is with the birds.

Five miles to the 22^nd mile camp. I take off my shoes and discover that I have my first leech bite. A drop of saline and the little bugger drops off, leaving a wet, red smear. Need to lace up my boots and leechguards better next time.

Have the others noticed too? That the silence is very different here? They must have. Conversation has died down to a bare minimum. The crunch of gravel underfoot, the swish of huge leaves, Tiger’s steady breath as he walks ahead of me. Nobody has realized that this is a part of the magic, steadily growing stronger.

- Two –

Everybody is slightly constipated the next morning. Maggi Noodles and condensed milk; good camping food. Another five miles scheduled for today, on to the 27^th mile camp. A third list is growing very slowly, Malayan Giant Squirrel, Eastern Mole (dead), Tiger ( pugmarks).

Someone wails, “When do we actually get to see the Famous Four Big Cats?”

“This isn’t like Corbett, where the tigers practically pose for photographs,” Wada smiles indulgently. This is his second trip to Namdapha. He says the illegal logging operations have already changed the landscape noticeably in just the two years since his last visit. Apparently rich businessmen buy up the logging rights of the locals and fake the paperwork, logging many times the allowed quotas.

The core area is still untouched though. Nobody knows for how long.

We are used to the silence now, as we trudge steadily. Occasionally the trail draws near the river. The view is becoming increasingly impressive, as we slowly climb higher up the valley. The trees are close together, an impenetrable wall of green a few feet from the trail. A fourth list is easy, Hollong, Hollock, Wild Banana. The bird list is growing the fastest.

“Rainforests like this, they reach such incredible stability over the centuries, the number of trees per square mile of any species stays nearly perfectly constant. Things get much more complicated than the equatorial forests though — those have no undergrowth.” Another illuminating nugget from Wada. He has been urging everybody who will listen, to read Daniel Quinn’s Ishmael, the cult classic on the philosophy of conservation.

Millennia of evolution, incredible stability. Food chains and nutrient cycles drawn taut by the creeping and inexorable power of natural selection. Nature using a stochastic search to move to ever higher local maxima on the fitness landscape. The blind and impersonal power of evolutionary optimization. I smile at my own thoughts. The words sound silly here.

Early tropical sunset. We make camp at the damp, abandoned shack at 27^th mile. Everybody has encounters with the squiggly little leeches. Saline is passed around. I have far more leeches crawling over my khaki leech guards than the others, there is something wrong with my shoelaces.

I watch Tiger impassively flick a leech from his foot with the tip of a machete, as he relaxes silently after several hours of trudging with the heaviest of the packs. The porters fetch water from the river in long bamboo tubes, far more efficient than our clumsy pails and jugs. We listen open-mouthed as they tell us about the flavor of bamboo rice, rice cooked inside bamboo stems over an ordinary brush fire. Khichdi and aachar today. Amit is a revelation. He cooks like a professional.

Surely the others have noticed it now? Time and space are different here, rhythmic somehow. There is a startling homogeneity here that seems to capture all of infinity and eternity in a tiny box around me, just a few cubic feet in space and a few silent minutes in time. Time and space, they are unmarked here. Anonymous, unpunctuated by alarm clocks and the throb of cars pulsing through asphalt grids. There is a stillness here. You know that the place has been this way for centuries.

- Three -

We are moving faster now, not breathing so hard. There is an odd alertness to everybody’s walk. It takes fifteen minutes for the stiffness in my muscles to give way to a feeling of incredible power. With a shock I realize that despite sleeping on bumpy ground in a stifling sleeping bag, my nose open to the night chill, I have just had the best night’s sleep in several years.

I try to remember, there was no tossing and turning last night. I fell asleep within minutes of crawling into the sleeping bag, The sleep was heavy, dreamless, deathlike. And when I awoke, I awoke instantly, completely alert. No tired red eyes. No memories — it’s as if ten hours have simply vanished. Is it just the exercise? Can’t be. Even hours of swimming workouts under the glare of Coach Reddy’s eyes never bought me this sort of sleep. More magic. There is an odd loss of identity here. Am I a part of this forest now? A slave to its rhythm and tempo? It is a strange thought, reassuring and terrifying at the same time.

The seven miles to the 34^th mile camp are being covered at a record pace. The bird list is still growing. We heard the Hoolock gibbons screaming last evening. More tiger pugmarks. No tigers, no clouded leopards. A heavy thrashing deep in the undergrowth brings us to a stop. We peer hopefully through the sea of green, might be sambar or gaur. We see nothing, the heavy silence settles again, instantly. The interruption is forgotten in a few minutes, Namdapha wraps a tight straitjacket of the present around us again.

Ajay and I have been walking faster than the others. We reach the 34^th mile camp and have twenty minutes to ourselves, while the others catch up. We are on the edge of a cliff, with a magnificent view of the Noa Dehing flowing hundreds of yards below us, deceptively lazy. The entire sky is visible after a very long time. We can see vultures riding updrafts far above us. “Himalayan griffon,” Ajay hazards. We are not sure. The birds are too far above to identify clearly. Ajay collapses on the unexpected patch of grass just on the edge of the cliff. I sit down beside him and then get up with a yell. A fat greenish leech is hanging determinedly from my finger. It is fatter and bigger than the cousin we met earlier — the black leech. This one is more than an inch long. We decide to remain standing.

The others arrive. Gurav looks at my leech, now dead in a puddle of saline, and grins. He tells us that the green leech is more annoying — it sometimes drops down on people from branches overhead, unlike the black leech, which only crawls up from the grass. We grin uneasily at each other. Khichdi again today. I am not really that hungry. Ration master Tambe has increased the daily chocolate ration from one bar to one and a half and I have already eaten all of mine. I sit on a rock for a few minutes before turning in, staring at the river. The vultures have left.

There are dead leaves, dry branches, leeches floating dead in saline, yet the atmosphere is charged with an exuberant life energy, unbridled and oppressive. Trees, trees, trees all over, as far as I can see. A landscape of deathlike stillness, yet almost numbing with the smell of untouched life, miles and miles of it.

Why does this overabundance of life have such a still, deathlike facade? The place seems startlingly clean. Wait a minute. Clean? There is damp rotting stuff all around, there is a smelly green puddle just yards away. Would you sleep with a tub of compost in your room? No, this is not a clean place, not a tidy place. What is it then, this odd feeling of cleanliness?

- Four -

Six miles to the rest house at 40^th mile. The view is getting so stunning that we can just stare. At some points on the trail, we have a view of nearly the entire valley, some hundred miles long, with the Dapha Bum peak dimly visible through the shifting clouds. We cross one treacherous stretch, where a landslide has destroyed the trail. We cross gingerly, slipping on the dangerous, shifting rubble. The landslide is fresh, so there is no tough shrubbery to grab for support. It takes nearly an hour for everyone to get across. Most of us are scratched and mildly bruised all over. Fortunately the weather has been bright and sunny for several days now, Hazrababu says the trail is nearly useless during a large part of the rainy season.

The rest house pops up suddenly around a bend. It looks ridiculous, sitting incongruously in the middle of the forest. The Public Works Department built it in the same bout of enthusiasm in which they built the trail. The trail was supposed to be motorable, but a few years of regular landslides put an end to that ambition, and the rest house stands, largely unused, a relic of that effort.

We have reached earlier than scheduled. There are dry rooms with level floors to sleep on, for the first time in days. We stroll around, exploring the terraced gardens, a pathetic island of order in the middle of chaotic, exultant greenery. Somebody shouts. A strange tree has been discovered, laden with succulent yellow citrus fruits the size of small soccer balls. I make my one contribution to the flood of biology trivia that comprises most of our nerdy conversation , “That’s babli mas,” I inform the others. Not Latin, but still, a label. We had a babli mas tree growing in the garden of the house I lived in as a kid. The others have never tasted the fruit.

Gurav has found a patch of chili plants. He carefully chops some up, tosses them with the babli mas pulp, and offers it to us with an innocent grin. It is liquid fire, deliciously sweet and cool citrus, mingling with chilies so hot, it is nearly inedible. We gasp and choke and reach for more. It is completely irresistible. We sprawl out in the sun, on the improbable lawns, eating. Gurav is delighted. He informs us that we are eating Chakma Jong.

Food tastes good here, I think lazily. No spices, no delicate flavoring. Eating is a messy business, juice streams down my chin. It is not just that I am hungry, no. Chocolate, condensed milk, they taste odd here.

A voice pipes up, “Hey, anybody else have this horrible itch on your feet?” Joshi is scratching furiously, his feet a splotchy red. Everybody has it, more or less severe. “Dumding,” Gurav says helpfully, pointing at the tiny fly like creatures swarming overhead. Somebody murmurs, “Damn Things.” Laughter. The name sticks. It will take a week for the Damn Things to be finally forgotten.

Inside concrete walls, the outside world makes a brief reappearance. Jokes are exchanged till late into the evening, biology and Namdapha forgotten for a brief moment.

We sleep an extra hour. We are not moving today, but exploring the trail a little way ahead, where a broken bridge across the river marks our turning point. On the way there we come across the highlight of the trip — the severed leg of a sambar deer. There are drag marks all around, and distinct pug marks. “Leopard,” says Hazrababu with assurance, “Last night.” We stare at the leg for a while before moving on.

The bridge is in shambles. We sit about on the rocks for a while, watching the river, which is narrow and rapid over here, as it flows between huge boulders. The bird list is still growing. Daurian Redstart, Plumbous Redstart, Little Forktail.

Another night on level ground. The khichdi has turned out especially well today. Large dollops of ghee for everyone. To bed.

The severed leg of a sambhar deer. Exuberant life, death in plain sight. Death does not hide here, I think sleepily. Death is necessary here, as necessary as life. I think of urban images of death: dry plants in moldy gardens, a squirrel plastered to the road by a speeding car, men chanting dolefully as they carry a body to the ghat. Death is ugly in the city, perhaps because we try to avoid thinking of it as necessary. Inevitable yes, undesirable, yes, necessary, no. Death hides in the city, and life looks tired and insubstantial. In Namdapha, death does not hide. Odd then, that life should still dominate the senses so fully here.

- Five -

Quick march to Embeong. We go back along the trail, to a fork at 33^rd mile, where one branch, the one we hiked up, leads back to Deban. The other branch slopes down rapidly to the river bank, Embeong. The going is good, since the trail is now all downhill. We cover the entire ten odd miles to the riverbank by late afternoon. A record number of leeches: the last part of the trail has been unusually damp, with several pools and small streams adding some variety to the otherwise easy trail.

Up close, the river doesn’t seem quite so lazy. I have to walk briskly to keep up with it. We sit for a while waiting for the sudden, early sunset. Two rather interesting additions to the bird list: a flock of merganser ducks and a grayheaded fishing eagle, the largest bird we’ve seen so far. Up on the slopes of the valley, the birds are small and inconspicuous.

“This is probably the worst place to start birdwatching,” Wada says, “Mountain birds are extremely difficult to spot. If you want an easy, rewarding start, you should try someplace like Bharatpur, where you’ll spot two hundred birds on a single tree.”

Nothing about Namdapha is easy. Corbett and Kanha are better if you actually want to see a tiger, but this, densely wooded and shadowy, is the tiger’s real home. The largest of the big cats is a solitary stalker, preferring to hide in dense, wet undergrowth. Weighing in at close to 500 pounds, large tigers are much larger than the largest lions. Perhaps because they are so elusive, tigers have not been romanticized the way lions have. I suddenly remember the tigers I saw in Patna zoo, on the way to Guwahati. I remember I was slightly shocked at the size of the cats. It was the first time I had seen a tiger with adult eyes. The impassive stare through notched pupils, the restless pacing in an enclosure a thousand odd square feet in area. I tried to imagine the tiger here in Namdapha, staring at me from fifteen feet away, with no deep moat separating us.

Fifty odd tigers in nearly seven hundred square kilometers, giving each a territory of close to fourteen square kilometers. It is a shocking thought: this magnificent creature that needs so much room to be content, is now slowly vanishing, as another creature spreads inexorably, demanding more land than even this lordly five hundred pounds of raw power. Lesser creatures adapt better. Leopards, tiny by comparison, manage to survive very well on the outskirts of human settlements, reduced to lowly scavenging, but still, surviving. No such compromise for these proud striped loners though; they are disappearing, unable to come to terms with the new lords of the planet. And so they pace, restless and angry, in pathetic concrete recreations of their original homes, a thousandth the size, with a few dispirited, straggling shrubs where dense luxuriant and damp forest should be.

The others are setting up a makeshift tent. It looks like rain and there is no abandoned shack here. There is no way we could have carried enough tents for twelve. All we have is one two-man tent. We cut up our barsaatis, large doubled up hoods of green plastic, that we have been carrying as raincoats. Eight barsaatis and several bamboo poles make a nice cuboidal enclosure, about eight feet by fifteen feet. We crawl in after a special meal of kheer, made with rich condensed milk, rice, cashews and raisins. It turned out thick, rich and extremely heavy, and we have all eaten too much.

An ambitious plan for the next day: cross the Noa Dehing, walk a few hundred yards west where the Namdapha joins the Dehing, cross that, and then make our way back to Deban along the northern bank.

We get across the Dehing easily; the water is swift and knee-deep, but easy to cross if you are careful.

It has started to rain gently, visibility is down to a few dozen yards and we have a glimpse of the true Namdapha, the Namdapha of the nine months of heavy, incessant rain. It takes us over half an hour to get through a wide patch of wild banana, to the banks of the Namdapha. We stop.

The Namdapha is extremely swift. Tiger struggles across with the first of the backpacks and comes back struggling, with a wide grin on his face. “Tough,” he says, “Very tough.” It doesn’t take a long time to decide that this is beyond us. Watching the surefooted Tiger, veteran of the mountains, struggle grimly across twenty odd feet of raging water, we realize this is more than we can manage. Tiger is furious, he has to go back and fetch the first backpack. He is furious with Hazrababu, not with us. The porters have not bothered to conceal their contempt for Hazrababu so far, but now they are outright angry. They think he is not fit to be a forest service guard or trail guide.

Hazrababu, protector of the magic. A dozen odd guards for 1985 square kilometers of magic. Most of those are busy providing personal security for government officials. There are effectively just two guardians of the magic, and they double as peons. What keeps the magic safe then? Something does, for there are no poachers here, no noisy tourists. Can the magic hold out much longer?

We head back, but decide to cross the Dehing right there, at the confluence. It is swift, but looks manageable, and nobody is in a mood to struggle through the wet banana patch again. The southern bank is at least open and sandy.

Joshi crosses first. He is solidly built and moves steadily. He seems to hesitate for a moment, but then goes on. He reaches the other bank and waves to us, shouting something that we can’t make out. The roar of the Namdapha pouring in is difficult to shout over. I go next. I am much lighter than Joshi, so I have to be careful. No shoes on this wet day — we’ve been wearing rubber flip flops to avoid delays at the river crossings. I carry mine in my hand. The water is really cold.

I start to cross. And halfway across, I stumble. The water snatches away the flip flops I am carrying. I struggle to stand up. All of a sudden I realize that the hard round stones hurt my bare feet. I fight to stay upright; the water is only knee-deep, but fast enough to sweep me away if I try to sit down. But staying upright is tough with the water whipping around my legs and the backpack making me top-heavy. So I stand frozen there, unable to move. The current is strong enough to drag me off balance if I try lifting a foot to take a step. So I wait, mind as frozen as feet.

It occurs to me that I could die in the next few minutes.

Gurav casually grabs me by the arm as he comes up behind me, and supports me the rest of the way across.  He lets go of my arm as we reach the other bank, grins and walks off. I still don’t have my voice back. Joshi comes over and says, “I was trying to tell you guys not to try crossing here. The current is too swift.” I just nod dumbly.

The others have given up the shortcut and are headed back to the banana patch on the opposite bank. Joshi and I walk slowly back to camp. He spots a butterfly and yells at me to come look. I don’t think he realizes how scared I was. I still can’t believe it. I could have been swept away to the rocks a hundred feet downstream.

Suddenly, I realize why I felt uneasy on my first day here. It was not because I did not fit in, it was because I did. Namdapha does not lie to me. It tells me that I am part of it all, does not allow me to forget that I am mortal, that it is necessary that I must die someday, so that Namdapha can remain green.

Tomorrow I will be back in the real world. With that thought, another idea pops into my mind uninvited. Time is not unnatural here, in this unchanging place, it is the time in the outside world that is strange. That time has direction, purpose, a goal. Surely, that is not natural? Which one is the real world?

Epilogue

We made it back to Deban in one day, walking fast. Amit and I walked all the way, silent most of the time. The others stopped at 22^nd mile, where they got a ride on a truck. The PWD still hasn’t given up hope of making the road motorable, and trucks are struggling up, laden with men and material.

Namdapha National park is in the Changlang district of Arunachal Pradesh, tucked away in the easternmost corner of India. It is not frequented by tourists due to its inaccessibility, however with proper planning, it is possible to enjoy this most unusual of the Indian National Parks. Though it has more tigers and other wildlife than many more famous reserves, it is so well preserved that it is actually difficult to see anything here. The only large animal we managed to see was a barking deer. If you want to see a truly wild place though, Namdapha is probably the finest example in India.

Photo credits: Yogesh Wadadekar (Wada), who also provided feedback on the original piece.

We spend a lot of our lives doing nothing. Doing nothing is usually viewed as wasting time, and there are two ways it can be done. When you waste your own time, it’s called idleness. When others waste your time, it’s called waiting. I enjoy idleness.  I don’t like waiting.

Wasted time is not empty time. Empty time is meditation. You could argue that meditation is about subjective time standing still. Your productive potential, in theory, is either preserved or enhanced through empty do-nothing.  Wasted time is also not the same as recovery, relaxation or recharge time. That’s about using this minute to make another minute more potent.

Wasting time requires putting pointlessly dissipative activity into it. An annoying argument with an idiot about something that doesn’t matter, that ends up frustrating you, is a good example. You actively destroy the productive potential of time.  I like doing that sometimes.

Many are disturbed and offended by the very idea of wasting time. There is a beautiful bit in John Updike’s Rabbit series, where Rabbit Angstrom’s young girlfriend Jill  accuses him of having a “Puritan fear of waste.”

I used to be like Rabbit. Over the years, I’ve gotten increasingly comfortable with wasting time through idleness. I am still not comfortable with others wasting my time by having me wait, however. I suppose I am not evolved or ego-free enough for that.

Our culture of work is designed around wasting time for others. And it is not just waiting in queues, or waiting for important people who are running late for their appointment with you. That’s merely status-waiting. Robert Levine, in The Geography of Time, has a beautiful discussion of the interplay of status and waiting in different cultures.

I am talking about more deep-rooted ways of wasting time.

Paychecks for creative information work, unlike paychecks for hands-on or routine information work, are not about buying work. A paycheck represents an option, but not an obligation, on the part of an employer, to get value out of purchased time.

This so offends the work ethic of many that they’d rather manufacture highly-energetic and apparently productive ways of wasting time than enjoyable and openly pointless ways. Those who insist on the former love busywork. Those who prefer the latter are part of the  retired at work movement. Stanley Bing is the Messiah of this movement.

Employers recognize this deep-seated need to be “productive,” and encourage cultures of busywork over cultures of retired-at-work. This despite the fact that enlightened employers of information workers know they are buying time options rather than time stock, and that those comfortable being retired at work are more productive. So they take on the burden of creating busywork support systems.

Since busywork is by definition unproductive, you cannot find material evidence that it produces anything of value. You must look for social proof. Those who manufacture busywork therefore, do so in social ways, creating collective anxiety complexes to validate the value of each others output in circular ways.

Those who are caught in busywork economies usually recognize that they are really in a holding pattern, waiting for something to actually happen in their lives. But most of the time, they manage to forget it. That’s why they need a good deal more recharge/recovery time than either the productive or the idle. Busywork is vastly more exhausting than either. Waiting is existentially costly. Fred Wilson once said that the iPad is about reducing waiting costs, and he is right. The device lowers status-waiting costs in queues and waiting rooms.

Isn’t it ironic, by the way, that the industrial age created a proliferation of “waiting rooms?”

Unfortunately the iPad does nothing to lower busywork costs.

Fortunately, idleness costs nothing. You do not need to lower the costs of idleness because there aren’t any. If I ever run a big organization, I’ll have idling rooms at random locations, instead of waiting rooms in front of the offices of important executives.

The non-paycheck world is not different, since the same work-ethic anxieties operate everywhere.  There is enough room in the murky art of setting hourly consulting rates and pricing project proposals that you can adopt any time-wasting philosophy your anxieties demand. I’ve met consultants who bombard their clients with more busywork output than the most refined bureaucrat, and I’ve met other consultants who spend most of their time doing nothing, but still earn about the same as the ones who manufacture paper. For the latter model, you don’t need to rely on complex global labor arbitrage schemes of dubious morality to create your 4-hour work week. Most kinds of creative information work allow you to do so.

In the world of work, the great divide between those who choose to waste time by waiting, and those who choose to do so through idleness, is far more important than the more nominal divide between paycheck types and free agent types.

On the idle side of the fence, I have met many who like 10 hours of idleness for 30 hours of productive work. These are the high-energy dynamos. I’ve also met others, like myself, who like a ratio of about 30 hours of idleness to every 10 hours of productive work. I am fine with people who follow either pattern, though the low-idleness types tend to run ahead and leave me behind.

It is waiting that bothers me. Others usually don’t know how to waste my time properly. I have to do it myself.

Our methods of measuring and valuing time are very primitive. As information work becomes increasingly about creativity and ideas, the time-value-of-money equation is breaking down badly. But those who are comfortable with idleness have an advantage over those who enjoy waiting.

Okay, I’ve been stringing you guys along, promising a book, for nearly two years now. You could be forgiven for thinking that the project has fallen by the wayside. On the contrary, in spite of the insane pressures of leading a product launch at Xerox and writing this pretty demanding blog, by some miracle, I’ve actually been making steady progress on the book. I thought I’d share a few details. First, we’ve nearly locked down the cover. The ‘we’ includes my good friend and very talented designer/finance quant, Adam Hogan, who is doing the cover for me (while bumming around somewhere in the Czech republic). You’ll hear more about the talented Mr. Hogan on this blog soon.

As you can see, I’ve also nearly locked down a subtitle: timing, tactics and strategy in opportunistic, narrative-driven decision making. If that sounds like a bit of a mouthful, that’s because these are exciting days in book publishing. One of the things you have to do is Aim for Amazon, and strike a delicate balance between a great title/subtitle for humans and a search-friendly one for indexes and search engines. One of the proven strategies that has emerged is to optimize the subtitle for online discovery. The skill is not unlike the skill needed to think up great blog headlines, a game I enjoy a lot. I haven’t completely locked this down, but it’s getting close.

Let me share a few more details, including the final table of contents. And don’t forget to sign up for the announcement list, if you haven’t already.

The Table of Contents

Writing a book is an exercise in wrangling complexity, and one of the hardest parts is to getting the structure, sequence and number of chapters right. Over the two-odd years that I’ve wrestled with the book project, it has had as many as 12 and as few as 4 chapters. I finally have a structure I am happy with:

1. Introduction
2. A Sense of Timing
3. Momentum and Mental Models
4. Narrative Rationality
5. Universal Tactics
6. The Clockless Clock
7. Conclusion

I hope the chapter titles intrigue you. I’ll share more in a future update, assuming I haven’t actually finished the book by then.

Three Remarks

The manuscript now weighs in at about 50,000 words, which is about right for its category: somewhere between a business book to a pop-science book, and I am doing my final heavy-edit pass.  I thought some of you might enjoy pondering a few things I’ve discovered. This might be especially interesting to those of you who are planning to write your own books (something I strongly encourage, see my post Everybody Should Write a Book).

First, word counts are very deceptive.

I often write 4000-6000 word posts here on the blog, and have been clocking between 80,000 to 100,000 words every year. So based on pure word count, the book should have taken me 6-8 months to finish. Why has it taken me more than 2 years, and counting?

Developing a coherent narrative that long, and structuring and sequencing it properly for effective and stimulating reading, is extremely hard. Especially if you are trying to get the thing done mainly on Saturday mornings, and every week you have to sit down and develop situation awareness of a bigger manuscript each time. Programmers say that a 1000 line program is not 10 times as harder to write than a 100 line program, it is 100 times harder. Difficulty scales as the square of length. So by that measure, this book has not been 8-9 times harder to write than my longest blog post. It has been 64-81 times as hard.

Second, book writing is not like blogging.

The longest series on this blog, the Gervais Principle series, currently weighs in at about 24,000 words, and will close at about 28,000 when I finish the last part. Since the narrative there is not as tightly integrated, it has not been as hard to scale. So though I do eventually plan to publish that in “book form,” I really won’t dignify it with the noun book. When I get done with that, it will be a blook, and though there are people out there ready to tell you that any half-coherent collection of 10-12 blog posts is a legitimate book, I now know that’s just too low a standard. It isn’t the publishing vs. self-publishing distinction or the book vs. ebook distinction that matters. A serious book project is intrinsically a far harder intellectual challenge than a blog series.  At least if you’ve challenged yourself and are attempting to write something worthwhile and ambitious. Tempo will have its share of flaws, but lack of ambition won’t be among them.

Third, a book is not software.

It was hard for me to finally come to terms with this, but I finally realized that continuous beta, release early and often, test-driven development, and similar concepts, are bad ideas when it comes to a book project like mine.

When I started the project, I was very enthusiastic about applying these ideas. I did a few rounds of early beta reviews of the first few chapters with a selected beta-testing panel. But I found that it was just not worth the management overhead. A book is something that really gestates best in solitude and privacy. It is really hard to involve others, no matter how well-intentioned they are, continuously. I found I was spending more time managing, disambiguating and synthesizing a pile of mutually contradictory review comments than making progress on the manuscript. Some highly modular book projects may benefit from this consultative process. But most, I suspect, will not, and will mature better when driven by a more dictatorial author. Some day, when this whole project is a distant memory, I’ll reflect and try to figure out why. My suspicion is that a book with a tight narrative, like mine, is a single atomic unit, no matter how big it gets. It can only meaningfully be “tested” in its entirety.

Partly as a result, I merged my beta-sign up and announcement-email sign up lists. I may still solicit a very limited round of feedback on the complete manuscript, but I will only be asking a small group of trusted friends.

So yes, I will be releasing the book sans bold experiments and clever “Version 0.9, beta” tags. Good or bad, flawed or not, it is going to be an old-fashioned “first edition.” If it turns out the ideas need to evolve and be repaired, there may be a “second edition” at some distant date, but don’t expect “weekly builds” or new micro-editions every week. I suspect, once I get this out the door, I’ll be too exhausted to even look at it again for at least 6 months. That said, I did adopt one software practice: the book manuscript (typeset as a set of LaTeX source files) is evolving as a Subversion repository.

The Road Ahead

There’s still plenty to do. While I’ve gone through the whole self-publishing process before, I had a lot of help last time, since I was editing and “producing” an anthology. Now, I am doing it pretty much all by myself, and the game has changed a lot in the ten years since I last played. Every week there’s some bit of news or the other that makes me stop and think briefly about my publishing strategy. When should I release a Kindle version? How do I get on the iPad? Is it worth targeting retail outlets at all? Should I even look for a returns-based distribution channel given how that model is collapsing all around?

Those are the fascinating publishing strategy questions. There’s also a whole lot of tedious process grunt-work involved in publishing a book today, but fortunately, much of it is behind me.

So I am very hopeful that this time I will stick to my planned release schedule: February 2011. No more delays. Famous last words.

And again, don’t forget to sign up for the announcement list, if you haven’t already.

If you want to help me do more pre-publicity for the book, please grab the linked image on my sidebar, and put it on your blog.

And thank you all for your patience and continued interest. It is gratifying to be periodically asked how the project is coming along.

Allow me to introduce you to Seb Paquet, an expert Socratic fisherman on Lake Quora.  He is particularly adept at baiting the hook just right to catch fish of the species Wannabis Oracularis, to which I belong. He is entirely to blame for getting me addicted to Quora in the last month or so (you can follow me here). For those who haven’t yet heard of it, Quora is a booming Q&A site. It just might be the next big social media site to cross the chasm and go mainstream. It is certainly booming right now, and is the darling of tech watchers. But unlike other recent Valley favorites like FourSquare (narrow appeal) and Groupon (for shopaholics), Quora might well become as fundamental to the Web as Google, Facebook or Twitter. Everybody asks and answers questions after all.

If you think the Q&A market is a tired and played-out ancillary market (lazy schoolkids looking for help cheating on homework on Yahoo Answers, tedious transactional Q&A on LinkedIn, let-me-Google-that-for-you sites), you’d be wrong. Quora has demonstrated that Q&A is a viable fundamental market, not a bolt-on ancillary to other markets like social networking or asymmetric messaging. Hang Zhang first helped me appreciate the very subtle social design lurking underneath the apparently simple architecture of Quora, and Seb Paquet, through his baiting, has provided me, over the last month or so, with a crash course in the dynamics of Q&A. Initially, I thought Quora was a fad, that owed its initial meteoric growth to the pedigree of its founders and early backers. I even unfairly labeled it in my head as “Valley mutual admiration society,” but I have now become a convert.

The DNA of Q&A

Think about Q&A for a moment. Until Quora came along and forced me to think about it, I’d assumed that it was a derivative type of transaction, rather than a fundamental kind. Q&A, I figured, was just a conversational dynamic within any community, and didn’t need specialized treatment. Unlike general socializing (parties, games) and quick bump-into conversations (watercoolers), Q&A is not specially associated with any particular part of our physical social world. So unlike Facebook or Twitter, there are no obvious source metaphors to draw from. Q&A happens everywhere, and nowhere in particular.

Support for Q&A, I figured, was a commodity “cost of doing business” feature on any community site where status is based in part on authority, and there is a conversational dynamic underlying the system.

To understand the Quora phenomenon, you have to think about the phenomenology of Q&A. Consider the different types we encounter in everyday life:

1. Commodity factual questions (“let me Google that for you” or LMGTFY)
2. Test/Interview questions
3. Learning questions
4. Socratic questions
5. Procedural questions (“do we vote now or later?”)
6. Transactional questions (such as asking for directions, or asking about the price of an item in a shop)
7. Privileged data questions (such as asking about the details of an insider deal)
8. Permission questions (Can I/May I?)
9. Ascriptive authority questions (based on formal signs of authoritative knowledge, such as degrees: “Doctor, what’s this rash on my arm?”)
10. Situational authority questions (a news anchor asking an on-site reporter a question)
11. Opinion questions
12. Solicitous questions (“Are you comfy?”)
13. Diagnostic questions
14. Trick questions
15. Entrapment questions
16. Questions people wish others would ask them (faux-FAQs on personal blogs)
17. Insight questions that motivate research (I consider myself pretty good at this kind and have blogged about it)
18. Debate questions that trigger a conversation between people with comparable, but non-identical relevant knowledge bases
19. Flattery questions that are designed to give the person being asked an opportunity to show off
20. Insult questions that are designed to put down the person being asked

I am sure there are many more varieties, but here’s one interesting feature: in Q&A, the relative status of questioner and answerer depends on the type of question, and the answer and response that actually follow. Very interesting sorts of status conflict can emerge when the two parties make different status assumptions, as in the classic kind of bad-faith entrapment question:

Jane Doe: So, what do you think about the situation in Burma?

John Doe: Well… [launches into a 1-minute, speculative mini-lecture based on what he knows]

Jane Doe: You have trouble admitting when you don’t know something, don’t you?

(here, Jane wasn’t really interested in Burma; she was interested in tempting John into a moment of mini-pedantry so she could make fun of it; both assumed status superiority to start, but Jane won the contest.)

Q&A and Status Fluidity

This fluidity of relative status in Q&A is what makes it a fundamental kind of messaging transaction. Status shifts strongly, but locally, during the process of asking and answering questions. Which makes sense, since status shifts generally occur in response to truly new information being injected, and Q&A models are optimized to draw new information in.

By contrast, neither Facebook or Twitter is designed for that.  Your status on those services is dominated by the accumulated, average status of your recent past, and no one action can move that much.  Your current status just gets reinforced over time. It takes a tweet or wall post of extreme stupidity to damage your credibility on Twitter or Facebook. And you cannot build Q&A effectively into either because the accumulated status would be eroded by the acid effects of unbridled Q&A, unless moderated deliberately.

You can see this deliberate moderation (achieved via self-policing) on LinkedIn, where Q&A is muted, since everybody is a little afraid of having a stain on what is effectively their resume (resumes are the main professional status object).

New information can and does enter these systems, but it is strongly filtered by a confirmation bias, either by individuals or groups.  Your LinkedIn resume-profile is going to be self-serving. Your Facebook world is going to socially filter for “friendly.” Your Twitter world lets in new information, but cannot really process it with any depth, only pass it along as shortened URLs.

Which means that Q&A is a fundamental interaction, marked by high, but localized status volatility, and the dominance of current, situational status over aggregate, accumulated status.

Just as Twitter carved out a fundamental role for itself by making one sort of asymmetry explicit (relationship asymmetry), Q&A formats make another type of asymmetry explicit (situational status asymmetry based on questioner/answerer roles). In Q&A, unlike in general conversations, you are your situational role.

In Q&A therefore, each question induces a local universe of status on its own. If you have a lot of credibility from past answers, and a huge following, it still won’t excuse a stupid answer or prevent a smarter upstart from winning the “best answer” game. Q&A is fundamentally a more temporally and socially-localized game than other conversation types. The followers/following ratio, which is so  important on Twitter, does not matter as much on Quora (and for that reason, I predict that celebrities will have much less influence, apparent or real, on Quora than on Twitter, when they discover it).

The Genealogy of the Conversational Web

Q&A is one of the oldest time-suck activities on the Web. It predates the Web and even the Internet actually. It was a core part of the dial-up BBS (bulletin board) systems that used to exist independently of the ‘Net in the eighties. Usenet was the form the idea took on the Internet initially (Usenet has since wound its way, via a tortuous path through Deja, into Google Groups). Q&A was a big part of the conversational activity in those old media.

The overall undifferentiated idea of “bulletin board” has morphed today into an entire universe of multiple conversation models:

• Blog-comment models
• Reblogging and backtracking models (blog-to-blog relays and conversations)
• Comment aggregation (Disqus)
• Email groups
• Wall-to-wall chatter on Facebook
• Hashtag based transient conversations on Twitter
• Web forums (generally bolted on to a community that does other things as well)
• News aggregation/discussion sites.
• General announcement lists/boards

The basic Web forum is easiest to recognize as the lineal descendant of the original world of BBSes, but the BBS is actually the ancestor of nearly all conversation models on the Web.

The one conversation model that was clearly not present in the BBS world was wikis. Wikis are, in my opinion, the only fundamentally new conversation model on the Internet that arose independently of the BBS/Usenet family tree. The basic metaphor for wikis is the blackboard as a collective, harmonized, canonical, erasable, editable, conversational memory. By contrast the bulletin board is more of a cacophony, where the only source of privileged status for a piece of content is recency of creation. The two metaphors are conceptually distinct, and one cannot be derived from the other, which is why wikis had to be invented separately.

Conversation models are fundamental to the Web, and are more fundamental than either publishing or relationship models (the bases of Web 1.0 and Web 2.0 respectively), since specific types of conversations, in specific venues, are what produce content and help you acquire and retain relationships. The taxonomy/genealogy of Web technology is at heart a taxonomy/genealogy of conversation types, with blackboards and bulletin boards as the two original ancestors (perhaps there are others; I can’t think of any, unless you include the basic payment transaction as a financial “conversation”).

You could think of the history of the conversational Web as a series of failed and successful attempts to take a particular conversational dynamic that existed in an undifferentiated medium like Usenet, making it explicit, and adding features relevant to that dynamic. So relationship maintenance conversations were made explicit by social networks. Speech-on-a-stage+discussion session models became blogs. Watercooler conversations turned into Twitter and news aggregation. All were nascent in the original BBS models.

But somehow, nobody got Q&A truly right until now. Sure, there were specialized ones like Stack Overflow, but really, Quora is the first serious candidate. What makes it work?

RTFM, LMGTFY, FAQ

Think back to the single most annoying element of the BBS family tree: newbies joining and asking stupid questions or attempting to restart discussions that old-timers have beaten to death multiple times.

The real-time-evolving BBS model was and remains just plain bad at handling this. And all its descendants have this flaw as well.  Think of a physical cork bulletin board: all the space gradually gets used up, and rude people start to tack new notices on top of old ones. At some point, somebody cleans up by ripping out old notices. Reverse chronology and archiving/trashing is implied in the very physical structure.

And nobody likes to go hunt in the archives, no matter how good the search, or how well organized and categorized the content. If you see a system where there live people and dead (or “archived”) information, your instinct is to ask the live people rather than go digging in the graveyard. In fact, I’ll propose that as a law:

The First Law of Information Discovery: In a social information system containing both identifiable experts and well-organized and searchable information archives, newcomers will always choose to ask the experts over searching the archives, unless the cost of appearing stupid/lazy/rude is too high.

And it is the right instinct, since seasoned old-timers will not only be able to point you to the right piece of dead content faster than search, but will also be most able to answer questions that haven’t been asked, or interpret distant relatives of existing questions.  The “cost of appearing stupid/lazy/rude” is much higher in a tightly-knit community where everybody knows each other and you want to build up a reputation over time. If you are just asking a drive-by question, as a visitor in a community where you don’t intend to stay, you’ll have much less concern about appearing stupid/lazy/rude (security forums are full of people asking stupid questions when they discover a virus problem, and those communities are among the nicest in how they handle the newbies; they don’t tolerate lazy/rude though).

Of course, another reason people ask stupid/lazy/rude questions is because they are too stupid/lazy/rude, relative to the community. More on that later.

The main reaction to the First Law is the well-known RTFM phenomenon: impatient and crotchety old timers telling annoying newbies to go RTFM (“read the friendly manual” is the usual polite expansion of the acronym). FAQs (bottom-up manuals, really) were invented as a band-aid for situations where there was no canonical manual anchoring the discussion theme. LMGTFY is a cousin of RTFM: discovery help delivered with a dose of sarcasm.

So how do you solve this problem?

The Blackboard and the Bulletin Board Have a Baby

Here’s my theory of why it took so long for somebody to get Q&A right (or right enough to merit serious discussion): Q&A fundamentally needs a blend of wiki and BBS elements. And so far, with the exception of Google Wave, the two families have not intermarried effectively. There are other pieces to the puzzle, but this is the main one. Two relatively isolated gene pools had to mix.

This is also why there is a basic problem with creating a “true FAQ” system. People ask the same or similar questions in different ways, so how do you count “frequency?”

There are two obvious elements to solving this problem: redirection and question editing. You should be able to redirect a new “Anyone know the capital of Alaska?” to an old “What’s the capital of Alaska?” and keep editing old questions to reflect the most frequent form/search phrase, thereby giving it its final canonical form.

Any given question is actually what is technically known as a small world in the overall “question graph.” At the core of each question’s small world are basically interchangeable questions, with more peripheral members having subtly different emphases that make them true variants. Each question’s small world is connected to related questions via weak links.

And no, this isn’t 20/20 hindsight on my part. I personally, and several people I know, have had this insight independently. Any decent engineer with a basic understanding of data structures and graph theory will think of this within a few hours of playing around with a BBS system or creating a faux-FAQ for a product.

The trick is executing on that insight right, and getting the key psychological pieces right, and finishing the solution to the point that it solves the RTFM/LMGTFY/FAQ problem that BBS descendants struggle with.

In terms of social psychology, this means that questions need to be owned by the community, but answers by the individual. Quora gets this dynamic almost perfectly right.  The names of questioners are not highlighted in any way. Questions can be redirected or edited without the permission of the original person asking the question, but edits to answers come to you as “suggestions.”

They’ve even managed to have their cake and eat it too: individual answers are owned by their authors, but there is a room for an “answer summary” that may or may not get used depending on the level of dissent in the answer set, and the structural friendliness of the answers to easy synthesis. List generation questions such as “best books of 2010?” can obviously be summarized in true wisdom-of-the-crowd ways, while ideological questions are harder.

You also need a mechanism to correct for the First Law and send newbies to existing questions without annoying old timers, and Quora gets this almost perfectly right as well (and this is an idea I hadn’t thought of or heard of before). Rather than search as the starting point for the experience, they stuck with asking a question as the starting point (i.e., they decided not to fight the First Law directly). But they drive users to search anyway, via a pretty robust autocomplete mechanism, and an integrated cue that takes you to explicit search. So they head off newbies asking stupid repeat questions as early as technically and psychologically possible. I’ll be surprised if this can be simplified further:

There are many more interesting design features to Quora (I could do an entire post on the voting and following models), but this is merely good engineering. The fundamental innovation lies in the marriage of wiki and BBS models and working through the implications correctly, and in a social-psychologically sound way.

Can Quora Grow Past the Good Old Days?

About 10 years ago, I was heavily involved in an early Web forum at Sulekha.com. I was there on Day 1, became one of the opinion leaders, and eventually joined the company and managed the forums for a while. It was on the huge, noisy fishmarket end of the spectrum, as far as these things go. I’ll admit it was fun playing dictator for a while.

There I noticed a dynamic which has since become a widely-recognized characteristic of any conversation-based system: the early opinion leaders get increasingly dismayed by the quality of new recruits and new conversations (the stupid/lazy/rude threshold getting gradually lowered). At some point, after a bunch of hand-wringing “Good Old Days” conversations among themselves, they leave. The phenomenon has a name. This is the evaporative cooling effect that Hang Zhang wrote about. The main symptom is the “Good Old Days” conversational trend. Social systems may or may not survive the mass defections. Some communities happily adopt a lower-status identity, while others are hollowed out and die.

Hang also proposed that one of the main ways to combat evaporative cooling is to use an architecture he labeled “warren” as opposed to the alternative “plaza” model. I riffed on the Warren-Plaza theme a few weeks ago, and threw in one of my own favorite memes, legibility, and in particular, status legibility.

If you don’t have time to read all those posts, here is the 30-second version. Warrens are like rabbit warrens: low global visibility, highly personalized corners where small groups can aggregate/self-segregate. Plazas are like grand public town squares with lots of global visibility. Warrens are maze-like: it is hard to break out of your current corner, and it is easy to get lost. You couldn’t get lost in a plaza even if you wanted to.

In other words, the metaphoric-spatial structure of a warren is (deliberately designed to be) somewhat illegible, and maintains that calibrated level of legibility as it grows, thereby retaining a “small town” feel even as it grows into a metropolis. The illegibility also applies to social status. You can easily see relative, global status when everybody is crowding into a big public plaza, trying to get a glimpse of the King. Much harder when the entire group is broken up into multiple small groups.

“Warrenizing” a site undeniably slows evaporative cooling, by allowing groups to self-segregate. On Sulekha, we had a warren of individual topic boards, and a firehose plaza that consolidated the entire discussion stream at the landing page. When flame wars got too hot, we used to simply turn the plaza stream off, so people were forced to go directly to individual corners. Things would immediately get quieter. Groups would get sequestered, and therefore less likely to leave because of unpleasant activity elsewhere that they didn’t like.

But now that I have used Quora for a while, I am not sure the warren architecture and sufficient status illegibility will be enough. Apparently, I am a latecomer. There is already a lot of Good Old Days chatter on Quora.

In fact, I am leaning right now towards the conclusion that if they don’t make some basic changes,  Quora will die, rather than go downmarket, through evaporative cooling.  One reason is that though the architecture is a warren architecture, one corner is heavily overpopulated (the techie/entrepreneurial crowd).  Unfortunately, the theme that this crowd loves is, I believe, at its limit of “warrenizability.” You cannot subdivide the topics within it in more fine-grained ways, to sequester the crowds more finely. Every user’s Q&A interests have a resolution limit. I am interested in “lean startups.” I don’t want to further confine myself to “East Coast lean startups for consumer content businesses.” Topic-based (interest graph) warrenization is at its limit on Quora. Social-network-based warrenization is also at its limit (it’s just inherited from Facebook and overloaded).

To use an ontogenic metaphor, Quora is a victim of unbalanced growth. It’s like a baby with a fully-developed, brawny and hairy right arm. It is flexing that bicep, and intimidating the rest of the body so much, growth is being arrested. You can’t amputate the arm. You can’t ignore it. So what can you do?

Warrenizing Time

Quora is faced with two problems: calming the limit-of-differentiation entrepreneurial corner, and getting other corners to catch up. The trick to both, I think, is tuning the blackboard-bulletin-board knob more firmly towards “blackboard.”

The baby needs more wiki genes.

The problem here is that the core experience architecture of Quora is built around what is known as a stream: a real-time scrolling reverse chronological flow (pioneered in its “live” form by Facebook and Twitter, but dating back to BBSes). Highly personalized and customized no doubt, but still basically a ticker, zoomed in strongly on “now.” This creates a strong bias towards the bulletin-board end of the architecture (by emphasizing “now”). By contrast, wiki models are typically timeless. You get no time cues when you land on a Wikipedia page, usually.

My colleagues at PARC have built a tool called WikiDashboard that allows wikis to bolt on a more explicit temporal dimension. Quora needs the reverse technology: something that mutes the excessive temporality (to be precise, excessively real-time temporal structure) and emphasizes longer time scales and timelessness.

One way to think of this is that Quora has successfully warrenized with respect to the interest graph and the social graph. But temporally, you still have a plaza. One single, universal time scrolling back at a fixed rate to the dawn of time.

Think about time in relation to questions.

• There are timeless questions, and there are current questions, and there are questions with life-cycles of every size in between.
• There are questions that pop up frequently, infrequently, or clustered around a particular date.
• Questions may be related to each other very strongly, but be too far apart in time (wouldn’t you want to automatically situate discussions of the 2nd Iraq war near discussions of the 1st Iraq war based on time?)
• Some questions evolve to their canonical form very quickly, because lots of people ask it all the time, while others take much longer.
• Some questions are large “small worlds” (lots of variants), while others are tiny worlds (few variants). Question-and-Search works better on the latter. Question-and-Search only works well on the former after the community has invested significant effort in building redirects, and editing existing questions.

Here, I suspect the Quora team has merely taken the path of least resistance for a first iteration, since the basic stream is the most popular architecture right now. There are many obvious attacks on the “warrenize time” problem (an all-time-great questions tab, a rewind button that allows newbies to start consuming a Quora theme from when it gained momentum, rather than jumping into its current state, a way to easily find “most answered” or “most variants” corners of the question graph, graphics that show the lifecycle of a question or set of questions, a logarithmic receding horizon as opposed to linear). Again, all of these ideas are sort of obvious to anyone who’s thought even briefly about temporal organization (okay, maybe I’ve thought about it a more than most, since I am writing a book about it).

But I bet there are social psychological conundrums (like the First Law) lurking here on the time dimension too. It won’t be an easy path.

But I hope Quora makes it. In a way, this is the fundamental social technology that is a most natural fit for me personally. I am not a natural Facebook or Twitter fish. I might be a natural Quora fish.

Over the Thanksgiving weekend, I tried, and failed, to relax. I am sure I am not alone, and that many of you had the same experience. But I failed in a very revealing way, that led me a very interesting definition of work.

What happened was this:

I was reading a book to relax (Robert D. Kaplan’s excellent Monsoon: The Indian Ocean and the future of American power). It was pure relaxation in the sense that the subject has nothing to do with either my work or subjects I normally blog about (my other “job”). But a few chapters in, something very interesting happened: I suddenly decided I might want to blog about the book. And just as suddenly, a relaxing experience turned into “work,” and within a half-hour, I felt I needed a “relaxation break.”  So what happened?

Consider what hard work used to mean. If you worked on an assembly line 40 hours a week, cranking widgets, you were working. If you put in 20 hours of overtime to make it 60 hours, you were working hard. Get up to 80 hours, and you were killing yourself.

The point is, crank-widget work is easy to define and measure in objective terms. Information work is different. Drucker offered the correct, but mostly useless idea that part of information work is defining the work to begin with, which makes it very ambiguous. Call it define-and-do work.

Crank-widget jobs do exist in information work too. Making sales calls for products with a proven sales script, or routine types of trading on Wall Street are crank-widget jobs because there are measurable, tightly self-contained units of work (calls/trades) that have already been defined. And those professions tend to have a “work hard/play hard” culture because the boundary between work and non-work is clear. You can actually decide to “stop work for the day.” There may be no punch-clocks, but it is punch-clock work. And this can be true of even very complex types of information work, so long as it is formulaic (which is a synonym for “pre-defined” if you think about what “formulaic” actually means). The formula may take a PhD and lots of specialized knowledge to execute, but if there is a formula, it’s crank-widget work.

But when you are doing define-and-do Druckerian work, it is basically impossible to decouple definition from execution a priori in useful ways (“my job is to define my job and do it” — sounds like a GNU recursive acronym, doesn’t it?). “Work” is whatever the hell you need to do to “get the job done.” A non-constructive definition.

But even if you can’t define this kind of define-and-do  information work, you can detect when you are doing it.  Here’s how you detect it:

It is work if there is a customer other than yourself.

Or to put it more explicitly: It is work if it will impact something that will be evaluated by others, and if their reactions will have consequences for you that you care about (If you don’t care how they react, they are by definition not a customer).

That’s why I experienced the sudden attitude shift while reading the book. Suddenly, I went from not caring how sloppily or carefully I read the book or what ideas I picked up, to thinking in terms of the needs of a blog post that others would evaluate.

It has nothing to do with whether or not you enjoy the work (as I do blogging). People who claim they love their work so much “it doesn’t feel like work” are deluding themselves. They’ll still be work-tired and need to “unwind” when they quit.

This definition can lead to some very surprising conclusions. George Lucas said, a few years ago, that he’d made so much money now that he could “afford to make bad movies for the rest of my life.” This illustrates one surprising conclusion. Finishing the Star Wars franchise was “work” for him even if he didn’t need the money, because the evaluation of the audience mattered, since in a sense the original trilogy was part of publicly-owned social capital, and if they rejected the prequels he was attempting to tack on, he’d have felt a personal sense of failure. But now that’s he done, he really can afford to ignore the audience.

By the same standard, you sometimes need a  vacation to recover from a vacation if the original vacation involved meeting family expectations. I am sure that’s especially true for many Americans who are returning after stressful family weekends. That’s a “work” vacation because between the turkey and the pumpkin pie, you may have had to justify your career/life to your Dad, and the reaction mattered. If this is true for you, then for better or worse, your Dad is your customer for a product you are creating called “my life.”

The level of apparent hard work and enjoyment is irrelevant. I once went on a hard 10 day hiking trip, where we’d cover miles of mountain trails every day, for 6-7 hours at a stretch. I’d be exhausted, but when we camped for the night, I felt relaxed. I felt unwound, rather than feeling a need to unwind.

There have been times when I’ve written some simulation code just for the hell of it, to have some fun for myself. For example, I once wrote a little simulation of a sky-writing airplane that would fly around and write any text string I gave it on the screen; thing took hours, and I never properly finished it, but it was relaxing. And I could abandon the project halfway precisely because there was no external party in the loop who cared.

And on the other end of the spectrum, I often have entire days now when I do absolutely nothing except sit around/walk around and think. No observable output. Sometimes not even working notes or scribbles. Just thinking. And at the end of the day, I am utterly  exhausted, because I’ve been thinking about work-related problems where any decisions I take as a result of my thinking will have consequences to be evaluated by others.

The definition “others will evaluate the consequences, and their reaction will have consequences for you” basically says it is work if there is a feedback loop that matters, that goes through an external party.

What causes the stress that makes it “work” is a combination of two factors. First, since you define what to do, there are no natural limits. You can define your work to be as simple or complicated as you like. Second, since the reaction of an unpredictable external party is going to determine how you feel about the work, there is a reason for you to endlessly work to try and lower the anxiety about the unpredictability. In other words, there is an infinitely-stretchable component to the work (the definition), and a fundamentally unknowable component to its consequences (the external reaction).

When crank-widget work fails to satisfy the customer, you can say “not in my job description” and pass the buck.

When define-and-do work fails to satisfy the customer, that’s not possible.

This definition gives us the easiest way to measure how hard you are working. Take 24 hours, subtract sleep time, subtract the time you are focused on doing something where there is no customer. The rest is work. It’s far easier to spot the negative space than to spot the positive space.

And by that standard, many of us are working 80-90 hour weeks, even if it doesn’t feel like it.

This also gives us the best way to relax: shorten all feedback loops as much as possible, and stop work when a round of feedback is in, and before you begin the next define-and-do iteration. If you routinely work in 3 month loops, you’ll never sleep.

Unfortunately, for truly complex work, there is a limit to how much you can shorten the customer loops for the things you are doing, no matter how much agile iteration you build in.

“Good is the enemy of great” is an insight that a lot of people have stumbled upon, though I can’t trace the origin of the phrase.  It might be Jim Collins’ Good to Great, but I am not sure. A hint about the dynamics are in that book (again, an insight I’ve heard elsewhere): good people with a bad process will always beat incompetent people working with a good process.

The clue is in the word process. Process is how good becomes the enemy of great. And I mean process in its most general form, not just the rigid bureaucratic stereotype. So a specific portfolio analysis technique for picking stocks to maximize some risk/returns function, or any sort of “methodology” is a process. A 12-step program is a process. A “Maximize Your Creativity” book that deals in colorful balls and right-brained art exercises is still a process. “Be agile and improvise” is also a process. If it can be defined and written down as a prescription, with any kind of promise attached, it is a process.

Here’s why this happens. Processes (and systems) of any sort first emerge when a spectacular and undisciplined success occurs. Like a startup — XYZ Corp. say, getting wildly successful. Or the PQR basketball team racking up a string of victories. Or an actor making it big in Hollywood. First, there’s a success that attracts imitative greed. Then something very predictable happens. A “great” story is retold in ways that only capture the “good” part.

“Great” success breeds inefficiency and waste, and somebody brings order into the chaos by imposing thoughtful systems and processes based on on 20/20 hindsight. These systems and processes, if they are well-designed, codify the knowledge that is chaotically distributed in the successful non-system/growth trajectory. They eliminate certain risks and redundancies, and build in some insurance. So you get ideas and intelligence moving from the heads of creative people and into rules and procedures. If you’ve been following this blog recently, this should remind you of the idea of legibility.

This is where the problem starts. Two additional things happen (this is going to sound slightly similar to Gladwell’s Outliers and like Nicholas Nassim Taleb’s critique of The Millionaire Next Door, but I am making a somewhat different argument):

1. The people behind the original success discount the amount of luck, special conditions and randomness involved in the success, and tell stories (unconsciously or consciously) designed to minimize factors besides their own contribution. You get “How I Did It” books from the CEO of XYZ Corp., and the coach of the PQR team.
2. The codified processes undergo a further evolution and become ritualized, and you get process books like “The XYZ Way” or “The PQR Formula for Winning!” But this book, it is important to note, is not a “How to evolve systems and processes like ours” book. It is an “if I could do it again” book. It has to be, otherwise you are reduced to telling people: “to be successful, be me.” So rather than getting the unreconstructed messy story of how chaos turned into order, you get an apparently logical backward extrapolation of the final state. We get the speculative assertion that the end-state process, with its checks and protections could have emerged more efficiently with a given hypothetical storyline. To take a simple example, if you grew a successful blog, and adding an email list in year 3 helped you really get to the next level, you say something like, “if I had to do it again, I’d start growing an email list from Day 1.”

The first kind of manufactured history creates a false sense of predictability, destiny and inevitability. You get the “origin myth” type stories. The second kind of “how to” cookbook offers an improved, more efficient, repeatable formula based on a messy template. Often you’ll find both in the same book. Both types of narratives take this equation:

Great = Good + Luck

And turn it into this one, by dropping “luck.”

Great = Good

In other words, they describe “good” but call it “great.” Because it isn’t fun to admit that the difference is often just luck, and that truly “great” is actually quite rare (I don’t think, for instance, that Einstein was just a lucky Poincare). But this is where the naive can get caught in a bait-and-switch. They think they are buying a formula for “great” and end up executing in best faith and getting trapped in the merely “good.”

If you must, you could say “Openness to Luck” instead of “Luck,” and that was the point of an article I wrote a couple of years ago, The Fine Art of Opportunism. I’ve since gotten a little sadder and more realistic about the world. I still stand by that article, and the model it proposes about how “luck” operates, but I believe that not everybody is lucky enough to be in places where it is possible to manufacture your own luck by being opportunistic.

Censoring Path Depedency and Initial Condition Dependency

Here’s where it gets interesting: the two elements conspire. The self-serving history hides the special conditions (if your success can be explained by a time/place/opportunity, in the sense of Gladwell’s Outliers, there’s less autobiography to write, isn’t there? Those story elements, if they are offered, are offered as “human interest” elements rather than causal elements).  The cookbook on the other hand, creates a conservative formula that ostensibly is about “learning by example.” By promising a less messy (“learn from our mistakes”) version of XYZ’s path, without the “inefficient” twists and turns, and by promising the beautiful and efficient end state that currently exists, a path-dependent history culminating in a desirable steady state gets retold as a path-independent story. Worse, you get the suggestion that it is the end-state that contains the value, rather than the truth: most of the value was banked along the way. The end-state merely milks already-won assets efficiently.

That last point bears repeating: the end-state is not where the value is. The end-state is a hard-won and defensible value-adding position. The value was banked along the way.  The first person who climbs a mountain the hard way gets famous. If he later suggests a road building route and others can now drive to the top, they are going to get far less value. Most of the value the original climber banked was due to the path he took. There are many roads to the top of a mountain, but the view is not always the same.

See what happened here? The manufactured history removes dependency on initial conditions. The “cleaned up” process-evolution prescription removes path dependency. Value-attribution is shifted from process states to final outcome state.

A story that is neither necessary, nor sufficient to explain wild success has become both necessary and sufficient. “If you do it this way, you WILL succeed, and this is the only way you CAN succeed.” Few authors are quite that naively deterministic, but you get almost that promise.

That’s actually the definition of a process: a manufactured, self-serving history justifying a conservative, “cleaned up” necessary-and-sufficient “do-over” evolution path that ends in a promised high-value-adding state that isn’t.

Quite a nice piece of sleight of hand, isn’t it?

What actually happened: luck, special conditions, and talents conspired to create a messy story that was neither necessary nor sufficient, and led to a high-value position, with most of the value already added, and a state that efficiently milks that position for a while.

But why is this an explanation of “Good is the Enemy of Great?”

Fooling Some of the People Some of the Time

Why do even smart people get taken in by a fake necessary-and-sufficient argument?

Here’s why: even though the initial conditions and path-dependency have been hidden, simply through how imitation works, most attempts to repeat the formula will share some (but crucially, not all) of the undocumented enabling conditions. If the manufactured story and cookbook emerge out of a “real and messy” story that played out with a Chinese hero in Taiwan, well, other Chinese wannabe-greats in Taiwan are the most likely mimics. So some of the enabling conditions are repeated even if nobody acknowledges that they matter. That may be enough for many people to get to “good.”

So if you follow the “great” formula faithfully you will accidentally build walls that prevent your own bits of luck and serendipity from getting through. It is only by creating your own bloody mess that you will be responding to your own unique local conditions and environments, and the lucky breaks your unique initial conditions and unique path offer.

Because you are following the straight-and-narrow version of a tortuous original, and with blinkers on, you don’t see your opportunities. Yes, you are enjoying the benefits of starting your email list on Day 1, but so is every other imitator. But what you lose is the things you might have spotted if you’d chosen to be a little messier. A little more focused on your situation as opposed to somebody else’s playbook.

So you get “good” instead of “great” to the extent that undocumented local conditions are shared. Eventually the local conditions get saturated with formula repeaters, and people start to apply the formula too far afield, and you get failure instead of even “good.” The fad is over.

Those who drive to the top of the mountain also get a good view, though they don’t get the great view at the end of the first climb, or the fame. And of course, if too many people are driving to the top of a given mountain, the overcrowding itself ruins the view.

There’s one more twist to the tale. The probability and information theorists among you will want to dismiss this analysis as a complicated way of saying “high risk/high returns gets converted to low risks/low returns, followed by diminishing returns, as information spreads through a system.

But there’s a subtlety. “Great” stories are not merely high risk/high returns. They seem to display certain asymmetries. They are stories that are more open to good luck than bad luck compared to random stories.

The asymmetry is created by the path dependence and initial conditions. You get information out of your environment and that allows you to swing things in favorable ways. But some bad luck is blocked out by a formula. Your unique formula created by rule-breaking within an older formula.

Selective Rule Breaking

I’ve been criticizing formulas and praising “messy.” To a certain extent, I am advocating methodological anarchy.

But this does not mean “be stupid and random.”

The real secret to getting from “good” to “great” is selective rule breaking. And this “secret” isn’t a formula (that would be ironic, wouldn’t it?) because I am offering a completely non-constructive observation. I have no idea what rules you’ll need to break and how or why. I couldn’t write a “how to break rules” book. Here’s my non-formula:

1. Look for the nearest “good” formula and skeptically deconstruct and reconstruct to recover the hidden initial conditions and path dependencies
2. Check if you are close enough; if yes that’s the formula you are going to break.
3. Now selectively break the formula using every bit of special-local-conditions information you have access to.

That’s how you get the rewarding asymmetric openness to luck. It is a very rude and disrespectful process. You read the formula-books and self-serving biographies like a psychoanalyst, teasing out the special conditions and the real story that might not make the hero look so good. You map yourself to those conditions. Then you look at your exclusive information advantage and decide which rules in the formula the information allows you to ignore.

This by the way, seems to be a common characteristic of “great” as opposed to “good” imitators. “Good” imitators either try and achieve modest success, or fail, by applying formulas religiously. But the “greats” find “good” formulas to break. I haven’t completely clarified the connection to imitation theory to my satisfaction though. Perhaps the way to say this is “good is the parent of great.” Both parent and enemy.

There is no guarantee that this non-formula will work, but if you despise “good” and aspire to “great” and can accept “failure,” at least it won’t fail in the “good is the enemy of great” mode of mediocrity. You don’t know whether you can make it rich in the stock market, but you do know that keeping your savings in cash won’t make you rich.

And if there’s no formula worth breaking in your neighborhood, welcome to pioneer country. All bets are off.

This is an abstract and advanced (i.e. “overcomplicated”) version of an idea I am trying to simplify and illustrate with better examples in my book. Sign up for the book release announcement list if you haven’t already, though I think most regulars already have.

I don’t like reinventing the wheel, so for months now, I’ve been trying to reconcile everything I know about traditional business (think Peter Drucker and the Harvard Business Review) with all the seductive ideas I’ve been learning from the Lean Startup movement (and I’ll admit I am simultaneously attracted to, and wary of, those ideas). Some instinct led me to focus on a single word: positioning.

It seemed to be the key word, and I think my instincts were correct. I’ve concluded that positioning, defined in a 7-dimensional way, is the single most important word in business. So what is positioning? To paraphrase Marc Andreessen, it is the only thing that matters. It is the controlled, but not deterministic, crossing of a threshold beyond which the business suddenly seems to come alive and “work.” The emotion changes from depressed to excited. The energy changes from languid to explosive. The rhythms change from weak and uncertain to harmonious, vigorous and steady. Positioning happens when a business has an “Aha!” moment, and discovers identity, profitability and sustainability.  The business has found its groove and tempo (the business word for tempo is clockspeed) Positioning involves throwing seven firing switches from “Off” to “On” position and all 7 cylinders firing steadily enough that anyone in the business can take a real vacation without everything going to hell.

Seven is not an arbitrary number. I looked hard and that’s all I could find. I’ll tell you about two that didn’t make the cut later. Each of the 7 switches, if it causes successful firing, induces an S-curve (if not, you get a peak and collapse).

If the S-curves are clustered close together in time, you get one big Aha! Otherwise you get a series of smaller Ahas! All 7 must be switched on. Otherwise you’ll get a change in emotion and energy, but not a true business positioning. The characteristic sign is that you get a frenzied, high-anxiety, manic energy tempo instead of a harmonious, vigorous and steady tempo. I call the former the “fire alarm” situation, and it will collapse if it isn’t corrected. Steady rhythms are a sign that you are in a predictable place. So let’s explore the seven dimensions of positioning and see if there’s anything useful to be found.

1. Marketing: Positioning as Hole-in-the-Head

Positioning in marketing is Al Ries’ classic theory of marketing. People’s heads are overstuffed. The only way to get in is to associate yourself with what’s already in there. Avis, We’re No. 2, so we try harder, and The Uncola are examples. These either fill a “hole in the head” (creneau to use Ries’ French term), or reposition an incumbent to create a space for yourself. Nyquil created a position against strong incumbent cold remedies by turning their 24-hour nature against them. There was a creneau that could be created for a night-time cold remedy.

What happens when you get this right? Simple. An anemic demand-driven business turns into an overbooked supply-limited business. This is what Drucker meant when he said the job of marketing is to make sales superfluous.  One killer positioning concept smoked up by some Mad Men can bring the business to you, so you don’t have to pound the pavements.  Your selling costs shrink spectacularly. (Aside to readers who’ve been demanding I watch and write about Mad Men: I  finally caved and watched the whole series to date on DVD over the last few months; thank you all for a great recommendation, and stay tuned.)

Marketing positioning is not the same thing as finding a “repeatable sales road map” in the sense of customer development inside lean startups. Yes, you still have to be agile, and pivot, and get outside the building. But you don’t use the customer development model, which is optimized for sales-led discovery. You focus on typical marketing things like finding good names and taglines. If you talk to potential customers at all, you do so in different ways, to find a creneau. You look for inspiration in pop-culture trends. If it works really well, you may not have to do the sales pavement-pounding and hypothesis testing at all. At the risk of losing half of you, here’s the football metaphor. Customer development is a rush offense. One yard at a time. One problem or product presentation to a customer/group at a time. Effective marketing positioning is a Hail-Mary passing offense. Touchdown in one pass if you are lucky and skillful.

For ribbonfarm, I did no customer development, hypothesis testing or anything of the sort. I just wrote whatever the hell I liked. Then the Gervais Principle happened. Now ribbonfarm is positioned as a blog selling a certain darkly-humorous, realist, dystopian view of life, the universe and everything. Marketing positioning and luck, not customer development.

Getting marketing positioning right is at once liberating and confining. In the case of ribbonfarm, it is liberating because the blog is operating cash-flow positive (not counting my time, which I view as ongoing in-kind capital infusion). It is also enough of a believable insurance policy that I think I could make a living off it if I had to. Constraining because “ribbonfarm” now means very specific things to readers. Now if I want to experiment outside this core, I’ll need a different blog and brand. But within this core, my marketing costs are near zero. The Be Slightly Evil list is a natural line extension, but not a brand extension. With near zero additional marketing, and ONE email to a few readers counting as “customer discovery,” I was able to launch it. And in less than 3 months it is already getting close to 500 subscribers. But I could not have done this if I’d wanted to build a non sequitur or dissonant brand off the ribbonfarm base, like a blog about inspiring quotes or great shopping deals.

2. Operations: Positioning as Rapidly Improving Margins

Chronologically, this notion of positioning came first, with BCG’s pioneering role in the strategy industry (a long story I’ve told elsewhere), and focus on the fact that market leaders grow rapidly, learn and drive down cost curves, setting a pace that followers cannot keep up.  At the heart of it is an accelerating trajectory of increasing margins, generating growth money, leading to more revenues at better margins, a virtuous cycle that leaves competitors far behind until you are the entrenched low-cost leader. Only true disruption (item 7, wait for it) can displace you. Until then, others can fight over your table scraps at the margins. This is the growth curve you get to after Crossing the Chasm. This is a positioning problem start-ups rarely have to solve, since principals often exit before they are forced to solve it. You can get roaring rivers of revenue and still bleed margins for a long time.

This is also not the free-cash-flow positive threshold. It is the accelerating margin improvements threshold. You can limp along with razor-thin margins for quite a while and call yourself cash-flow positive, but until you hit this phase transformation, your position is very shaky indeed. Specific things happen to trigger this phase transformation. Startup types think of it as mechanical “introduce big company systems and processes” but there’s a lot more. You have to find the artistically right kind of systems and processes that can put you on the accelerating margins trajectory. For Zappos, for instance, it appears to have been the decision to move away from drop shipping. So it is not a matter of just hiring a few bureaucrats to create some tedious forms. Big companies know all about this transition. I’ve done work on this dimension, but unfortunately it isn’t work I can talk about publicly.

The fully-refined version of this gets you the classic  positioning model of Michael Porter (the five forces model). Practitioners like to call it “strategy” but it doesn’t deserve that lofty term. It’s operations they are talking about. Very useful nevertheless.

In BCG Growth Share Matrix language, the switch gets thrown when an uncertain “wildcat” (or “question mark”) business suddenly turns into a “Star” (moving from the top-right to the top-left quadrant). From here you can drive down costs faster than competitors can, and move the business into a relatively unassailable high-margin cash cow position.

3. Sales: Positioning as a Pain Point Relief

If you plow through the Lean Startup material, you’ll find that the entire customer development process hinges on one crucial decision: you only go after a small subset of early customers who a) have a problem you can solve, b) are aware that they have a problem c) are actively shopping for a solution d) are actually improvising temporary solutions.

This is a customer “in pain” as it were. Product-Market Fit (PMF) in this narrow sense “relieves” a pain for someone. Focusing on customers “in pain” is a very specific way to find a market.

In an earlier Drucker-inspired article, I defined a customer as a “novel pattern of human behavior” based on Drucker’s notion of “customer creation.” Creation is expensive, but it can be done. But in CD-driven businesses, you don’t create this novel pattern so much as you recognize it in the wild and then offer a less painful substitute. This is significantly cheaper, which is why it is so popular in the startup world.

It is a slightly worrying metaphor, but I like it: in customer development, you domesticate a wild customer.

Here is my example. I was the first employee at Sulekha.com, after the two founders, 10 years ago. Today, it is sort of the Craigslist-plus-Facebook-plus-Fandango of India. I witnessed (and, in modest ways, contributed to) the PMF phase change, when we found our first strong revenue model (online ticket sales). And yes, the script ran exactly as the lean startup people describe it, with pivots and everything. We just used different language to talk about what was happening.

4. Engineering: Positioning as Killer App

Everybody hates us engineers when it comes to the business side of things. Even engineers themselves, when they move over to the dark side, have a tendency to speak disparagingly about the narrow mindset they’ve left behind.  I’ve done the leap, but I don’t do the disparagement. For positioning to work you also need an engineering switch to fire: from platform concept to killer-app.

Visicalc is everyone’s favorite example of a killer app. Killer App is primarily an engineering dimensions of positioning. Engineers, like mathematicians, are lazy. They like to generalize and come up with powerful solutions that can do lots of things. This generality is what ultimately creates value, otherwise we’d be living in a flood of what Alton Brown (in the context of kitchen equipment) calls “unitasker” products. But a journey of a thousand apps must still begin with a first app.

The story repeats itself all over the place. Walk through this trail of killer apps to see more examples (Atari and Pong, Nintendo and Mario Brothers, Gutenberg’s Press and the Bible, and many more).

Brad Feld has labeled “platform” the annoying word of 2010. He correctly notes that you cannot build a platform, anymore than you can make a viral video. The best you can do is build a platform-intent product or service, or a viral-intent video. But platform-intent thinking is crucial. Otherwise if your first and only application idea fails, well, you’re screwed. Nor will a generic “multi-tasking” minimum-viable product do the trick. That gets you a Swiss Army knife. That still has only one shot at success. You don’t just want a multi-tasker product. You want multiple cheap shots at making an application catch on.

Once you ask the question minimum viable product that does WHAT? you’ll see why “Killer App” is a useful separate term. It is that last 20% of the engineering that brings in 80% of the value. First you build a minimum-viable platform, and then you start doing several 20% stabs to find your first killer app. Each stab is a minimum-viable product hypothesis, but each stab is not necessarily a full repositioning or pivot. Think of a startup as a new PC that and each MVP stab as a half-assed app like Microsoft Works.  If you find that a lot of people are using Microsoft Works, well, go ahead and build and sell Office. That’s your killer app. But if it doesn’t work, you shouldn’t have to retool 100%. Only 20%.

Most high-value engineering products turn out to be platforms with applications. So platform-intent is the right strategy. Unitaskers, such as combs or toothbrushes, are rarely enough to build a business (unitaskers are usually made by companies that maintain portfolios based on similarities in manufacturing or service delivery processes).

But don’t let the word “platform” intimidate you. A platform does not have to be as complex as an operating system or a new fighter plane. A knife is a very simple instrument, but it is a “platform” in the kitchen because it can do so many things. The killer app turned out to be “chopping,” but it can still do some mean squashing, stirring, serving and spatula-ing. Some caveman or cavewoman probably started the search for a business model with a stick, and figured out that sharpening one edge created the first “killer” app. Pun intended.

Note: there are two engineering styles which I call “vertical first” (the first app comes before the minimum-viable platform) and “horizontal first” (the other way around). I think both can work, but the risk-benefit tradeoff does favor at least some platform work upfront, in my opinion. Pure vertical-first too easily leads to a series of narrow visions, none of which is worth much.

5. Public Relations: Positioning as Brand Socialization

While a “pure marketing” brand can exist just as a service or product, entrenched and strong brands also become part of the society within which they live. Levi’s is not just a famous (and now trashed through mismanagement) brand. It is part of the story of the American West. Ford stars in the story of American ingenuity, with its role in the growth of the assembly line. The Tatas are the story of industrialization in India. The East India Company is the story of 17th Century Britain.

PR is the difference between a strong marketing position for an unsocialized brand and a socialized brand with a role in the grand narrative of its host society. The story doesn’t just happen. But it can’t be created in controlled ways like advertising either. You have to scan for sparks of genuine social “integration” in the environment and pour fuel on them. Volkswagen’s ongoing “punch-dub” series of commercials is an attempt to do exactly this: talk up something to do with VW customer culture. I am not sure if it will work though, because this is a case of trying to make marketing do PRs job. PR is essentially a hidden and delicate backstage influencer activity. You are trying to co-opt a story that’s already “out there,” in service of your brand. Many people have a stake in that story, so at best you can influence the story, not “tell it.” VW may regret its punch-dub series of commercials. It may have killed the golden goose. Now I bet people who play the game might want to stop. If, on the other hand, VW had spent its money on a grassroots word-of-mouth campaign around the punch-dub game, a lot more could have happened. Groundswell has several great examples. I could be totally wrong on this one. Only time will tell.

Aside: this is why the new continent of social media has primarily been colonized by PR people. The marketing and sales people are talking a lot about the potential, but it is PR people who are making the medium work for them. Good marketing talks more than it listens. Good sales listens more than it talks. Good PR strikes a conversational balance. Social media is fundamentally friendlier to PR than either sales or marketing. In the past companies had to have either marketing or sales cultures. You could not lead with PR. Today you can.This is especially true because rank-and-file employees can be turned into a PR army. To use them in marketing means cheesy employee photos in brochures. Using them in sales means sales people bringing customers in for “insider visits.” Though Word-of-Mouth can work for sales (forwarding discount coupons/referral/lead generation schemes), marketing (contests, “viral” videos) or PR, it works best for PR.

This is where the classic reading of the Google origin myth gets it wrong. The story goes that Brin and Page, when told they had to “choose” between a marketing or a sales culture,  (and this is engineering braggadocio pure and simple)  “chose” to create an “engineering” culture instead. This is wrong on two levels. First, it is a three-way fork today, not two way, and Google is a company built on effective PR. “Don’t be Evil” and stories about great buffets (and ironically, the story of Brin and Page “choosing” an engineering culture) are basically the core of a PR socialization narrative (how many people know Google’s marketing tagline of “organizing the world’s information?” or have encountered its AdSense/AdWords sales face?). Second, culture isn’t yours to choose. Your business model completely determines it, and it will always be a culture driven by a customer-facing function. More on that later.

6. Finance: Positioning as Pricing Sweet Spot

You didn’t think the bean counters would have nothing to say, do you? Pricing confuses a lot of people because they think it is some sort of objective, if inexact science. The most naive people think: “if only I had perfect information and could construct my demand/supply curves, identify my substitutes and measure elasticity, I could price this thing perfectly to maximize earnings.”

Wrong. Economics constrains, but does not determine, pricing design. Economics will make you crash and burn if you get it wrong, but it won’t tell you how to get it right. It’ll just create a canvas. Getting the pricing model right is a positioning switch in its own right.

Creative finance people know that pricing is a positioning art. There are many famous products that made it via the right pricing strategy. Gillette (cheap razors, expensive blades), Xerox (originally, “lease the copier, sell the toner”) and Netflix (no late fees) are examples. And of course the whole world of $0.99, $19.99, “introductory price,” artificial scarcity “limited editions,” and and the like are all pricing design ideas. The entire cloud computing sector is driven by a pricing idea: pay-by-the-sip $0.10 offerings for enterprises that are used to paying by the million. To innovate in the cellphone market, pricing should be your top concern.

I recently tried myfooddiary.com (a great calorie counting tool) for a couple of weeks. They advertise $0.29 a day. Not the equivalent $8.70 a month. Why? Monthly subscriptions are better, right? No. This has to do with the psychology, calibration points and money metaphors at work in the prospect’s mind. See my Fools and their Money Metaphors article. Calorie counting is a daily activity for dieters. Health and fitness run on “daily” tempo mental models. The most effective pricing models are likely to be “daily.” That way you can compare it to other daily health/nutrition expenses like food purchases. Gyms would do well to shift to a daily price advertising model. A $90/month gym membership is a $3/day membership. So I know that it costs me about as much to ruin my healthy day with a slice of pizza as it is to redeem it with a workout. Why would you want me to think about my gym membership with a mental model that contains things like rent checks and phone bills? If some gym uses this daily price advertising idea, I demand a royalty!

Money metaphors are complex beasts. Entrepreneurs think with the entrepreneurship (capitalist) metaphor. But to sell stuff, you must think and talk within the customer’s active metaphor. Get it right, and the pricing cylinder fires.

7. Innovation: Positioning as Disruptive Breakthrough

Disruption theory is the most fundamental explanation of differentiation. It is an innovation model, and while it can seem very close to engineering, it isn’t. Innovation can come from a platform-creating scientific breakthrough, but it can just as easily be an enabling breakthrough along any of the other 6 positioning dimensions. It may be technically major or trivial (or to use the correct terms, radical or incremental), but you won’t know what it enables until after it has happened.

Three conditions have to be met for disruptive breakthrough. First, an innovation is “disruptive” because the place it is born is not the place it can grow. So it needs to be transplanted into a new business unit run by a logic within which the idea is sustaining. Second, you need a grow-in-peace peripheral position next to a major disruptee market, where you are too small to pay attention to, but too big for the incumbent to kill once you gain traction. If you don’t do the first, the business is stillborn. If you transplant, but there is no big disruptee market, you create a small niche business. But if you do both, you can get “breakthrough.”

The theory of disruption is highly evolved, and the relevant phase change happens when your adoption S-curve crosses an older one. Read my primer if you are not sure about what disruption means (and most people who use the term without having read Clayton Christensen’s book don’t know what it means, but think they do).

Is every new business disruptive? Is this an optional switch? I’ll leave that for later.

What Really is Business Strategy?

The 7-dimensions model allows you to view the essence of business in a very simple way. It is a matter of turning 7 switches to the “On” position, and hoping the corresponding cylinder fires. If you’re lucky, you’ll start out with one or more of the cylinders already firing. If not, you’ll have to keep trying each switch till all cylinders are firing.

Are there more than 7 switches? I thought about this really hard, especially about two very attractive candidates for an eighth switch: the “culture” switch (going from an inchoate culture of random types of people to a distinctive one) and an “ecosystem” fit (where the corporation is socialized into a supply chain).

After much thought, I gave up on culture. A distinctive culture is an outcome, not a control variable. How you throw the 7 basic switches determines what a corporate culture looks like. Equally, when a culture seems to be going wrong or toxic, it is almost certain that one of the 7 basic cylinders is misfiring, and the switch has been reset to “Off.”  I think if you try direct cultural design rather than hiring against your 7-switch needs, you are asking for trouble. And once culture has emerged, naming, codifying or ritualizing it is a very dangerous game. All you can do is try subtle things to not screw up a working culture, and to protect it from too much toxic disruption. At the same time, you shouldn’t protect it too much, otherwise the culture will ossify, and when the business environment makes a particular cylinder misfire, the culture will lack the ability to adapt.

The last candidate is ecosystem fit. Normally, this would be part of operational fit (strong, effective and mutually beneficial supplier and distributor relationships are a big part of switching from Wildcat to Cash Cow). But there is a difference between “inside the corporation” fit as processes stabilize and fit into a jigsaw puzzle, and “outside the corporation” fit as a vertical or horizontal integration structure emerges in a sector. But overall, I don’t think this is a meaningfully separate distinction with separate legal control variables. Antitrust laws see to that. When these laws can be bent or broken without consequence, or the government gets involved, then you’ve got an eighth switch. Ecosystem fit design is therefore just a part of organization design. Where you draw the boundary of the “organization” is a somewhat arbitrary legal issue.

That’s it for now. This is Part I of a two-part article (the whole thing was starting to weigh in at over 6000 words, which I’ve decided is too much even for me, so I decided to separate this idea into two parts). I’ll finish and post Part II if people like this one. Call this the MVP of a potential series.

I am rather surprised that the game of cricket has never gained popularity as a comprehensive metaphor for work, life and business. I don’t mean localized figurative metaphors like “on a sticky wicket” (tricky situation) or “bowled over” (fell in love/was caught by surprise). I mean a broad, coherent conceptual metaphor. The way American football is sometimes seen as a metaphor for industrial organization, soccer as a metaphor for reactive and opportunistic “network” styles of decision-making, and basketball as a metaphor for an artistic, Zen-like approach to life.  I think I know why this has been the case, and why it might change in the near future.

What Distinguishes Cricket

If you are of the Robin Williams “cricket is baseball on Valium” school of thought, and are not interested in changing your mind, then don’t bother reading further. This post really is not about that.

If you are willing to learn, take a quick look at the basic rules of cricket (don’t worry, we won’t need all the details). Curiously, despite the general contempt for the game in America, the first ever international game of cricket was actually played between the United States and Canada in 1844. There is apparently an ongoing exhibit at Lord’s, a collaboration between the MCC and the Baseball Hall of Fame, on the shared history.

Here’s all you really need to know about the cricket-baseball difference, as far as understanding this post goes.  The cricket-baseball difference is actually the cricket-everything-else difference. Despite superficial similarities, baseball is more like most other team field sports than it is like cricket. Four differences almost completely explain the difference in character between the two games.

1. In cricket, you don’t have to play every legal ball. You can let it go. No ‘strike’
2. In cricket, if you hit the ball, you don’t have to run. You don’t have to risk a run-out. You can stay put.
3. In the classic form of the game, (test cricket), in principle there is no limit to how long a batsman can keep playing. He’s in till he’s out, or decides to ‘retire’ due to fatigue or injury.
4. The ball is allowed to “pitch” on the ground once before reaching the batsman. This means its trajectory can be a lot more varied than in baseball, since both in-air and bouncing dynamics are involved. The ball and pitch also “age” through the game, due to wear, giving the later stages a very idiosyncratic character, and allowing the home team a decisive advantage in “framing” the game by curating the pitch in specific ways, and helping the ball age in specific  ways (there are legal and illegal ways of doing this). Australians, for instance, like to create hard, bouncy pitches to favor their fast bowlers, while Indians like to create low-bounce pitches that favor spinners in the middle of the game.

Though modern innovations such as limited-overs cricket and Twenty20 constrain the third feature (each side gets 50 or 20 overs at-bat respectively; an over is six “balls” or pitches), dominate the game today, test cricket is still the holy-grail form of the game, and the form which gets you into the serious record books.

The point of these four features of the game is that the game can get very long. Five days is the official full length of a test match, and if there isn’t an outcome by then, it is a draw. Five days is long enough to exhaust most of the creative possibilities of the rules of the game (though games like the original Native American version of lacrosse seem to have had enough richness to go on for weeks, on fields that were miles long).

If you want an analogy, cricket is to baseball as Go is to chess. A lot of the comparisons between those two board games hold for baseball vs. cricket.

The game can also get very uneventful. Due to the first two features, a good (and patient) defensive batsman can basically stay on the field for a long time, with very little happening. And cricket aficionados will still like the contest, a grim battle of attrition between increasingly weary bowlers and batsmen. Cricket is the only game where a non-event comes in for high praise: “well left!” is a compliment when a batsman judiciously decides not to take a swipe at a tempting ball, due to a high risk of a mishit leading to a catch. “Well left” is the negative space move in cricket that separates it from almost every other game I know of.

But with the good comes the bad. The huge temporal canvas, with its ability to hold negative space (uneventful periods), and presence of true “personality creating” elements, such as artistically curated pitches and balls that “age” over 5 days, allow something very unique to happen: cricket games can turn into real stories.

That’s where the conceptual-metaphoric potential of cricket lies, in its ability to sustain truly rich narratives.

Cricket as Narrative

To summarize my claim about the unique feature of cricket: it allows enough time, negative space, and “personality” into the structure of the game that it can sustain narratives that are richer than those of any other game. This means, for any kind of story, you can find a metaphoric equivalent in cricket. There have been matches that read like the Lord of the Rings, and others which read like The Great Gatsby.

So my candidate for the metaphoric “meaning” of a game of cricket (and I mean test cricket, for the rest of this post) is “cricket is narrative.” If American Football is about top-down military-industrial planning, and soccer is about free-form reactive/opportunistic improvisation, cricket is about narrative thinking and storytelling.

Every game of cricket has the potential to become a story of truly literary proportions. Of course, other games have narratives too, but they are more like genre fiction, limited to a set of fairly formulaic scripts. The excitement is more due to immediate stimulation, visual spectacle, and the like. This is not because the rules are not rich enough to create complex stories. It is because those games impose a few constraints (mostly due to television economics) that prevent the full richness of the rules from being expressed. This delivers a more reliable level of spectacle and entertainment.

The general criticism of cricket is true: it can deliver games at a level of stupefying boredom that simply cannot be achieved in other sports. A cricket game can languish in unbearably dull low-tempo doldrums. But that is also the strength of cricket. It can deliver games of sublime narrative richness that other sports cannot hope to achieve. Cricket can produce anything from trash to Shakespeare. Other sports have a much narrower quality range.

Cricket as narrative also means that games are often far more interesting to read about than actually watch, which is why cricket writers can get a lot more literary than writers in other sports. In fact, you could argue that a game of cricket is really just a preliminary phase. It’s purpose is to create narrative fodder, raw material for literary analysis. Other sports have strong post-game analysis cultures, but in cricket, it is practically the main act.

Which is fine by me. In fact, when I follow cricket these days (rare; I’ve lost touch with the personalities of major players — “characters” or dramatis personae — since moving to the States), I usually do so by reading post-game summaries. Unlike in many other sports, the outcome matters less than the story.  Draws can have more interesting stories than wins. Games that are exciting to watch can be dull as hell when written down, while dull-to-watch games, in the hands of a good storyteller who really understands the psychology and personal and interpersonal histories involved, can turn into epics. In fact the declared literary winner by the best writers (and statisticians) are often more admired than the technical “points” winners.

In most other games, there is not enough time for rich narratives to emerge, and there are artificial rules imposed to keep driving the action along as a visual and crowd-participatory spectacle. Even when non-action (artistically necessary for rich narratives) might be more appropriate. The only sort of non-action in games besides cricket is the occasional use of “run out the clock” tactics (frustrating to live watchers, great fodder for writers).

Removing negative space potential might make for exciting TV and good fuel for drunken rioting, but it limits artistic storytelling potential. To be fair though, cricket fans have been known to riot as well.

Some Examples

The players in a game of cricket must sometimes make decisions that are practically literary. For example, when a team has strong, entrenched batsmen who have run up a huge total, and the other side is unable to get them out, the captain of the batting team must make a curious decision that has no parallel in any other game I know of. He must decide if and when to “declare” the innings closed, and give the other team a chance to bat.

Why is this a literary decision? Yes, at one level, it is merely a decision about risk,  like deciding to punt in American football. The timing of a declaration can either lock in a draw or keep the game open for an outright win/loss outcome. But this is about more than just risk, for two reasons:

1. Unlike similar decisions in other games, where particular points or tactical decisions may assume situational strategic importance, “declaration” is fundamentally a purely strategic decision. There is no tactical skill involved. It is a pure “thinking” decision. So it isn’t like a “crucial” point, which would be a tactical decision elevated to strategic importance.
2. It is a point of narrative control. There is no way to really compute the risks quantitatively. There is too much action to go, too much uncertainty. So it is mainly a “let’s make this interesting” decision. Or not. Captains can get booed for declaration decisions that basically make the game uninteresting from a narrative perspective. Playing for a safe draw as opposed to playing for a story. The rest of the gameplay becomes a listless working out of the nearly inevitable. Oddly enough, though narrative interest is lost, this can lead to some of the most spectacular viewing events, since it becomes a “nothing to lose” situation, batsmen and bowlers alike can try risky things that make for good watching. But fans of the game are rarely in it for pure spectacle, so it is a weak pleasure at best. If it were chess, the players would shake hands, agree to a draw (or one would concede) and walk away.

The point I am making is that cricket is full of such narrative decisions (for those who know the lingo, the follow-on and shuffling the batting or bowling order are all narrative decisions).

This sort of dynamic leads to the possibility of every game having a unique “fingerprint.” There is enough room on the canvas for every game to take a unique trajectory. Cricket strategies, unlike strategies in other games, don’t read like “plays” in Football or doctrines like “full court press” in basketball. They read like stories. Cricket teams have invented entire narrative “styles.” “Team personalities” are often finely drawn in literary ways, rather than just being labeled as “industrial style” or “artistic style” as in some sports. Here are just two examples.

The first example is the infamous Bodyline series between England and Australia.  This was a narrative designed by one peculiar character, the vicious Douglas Jardine, based on two other peculiar characters: the fast bowler Harold Larwood and the Australian star Don Bradman (still regarded as the finest player of the game). Until then, cricket had been a game known for its very English “Gentleman’s Game” style of play, based on gracious decision-making. So it is rather ironic that it was the English themselves who broke with tradition, and created a vicious style of play based on attempting to intimidate and injure the opposing team (this was before the advent of protective gear in the game). It was motivated by the need to contain one player, Don Bradman, who was so good that the English team could think of no way to hold him and the rest of the Australians back. It was enabled by another player, Harold Larwood, who could bowl to break bones.

The English won by the letter of the law, but the Australians won by the spirit of the game.

A second, more recent example involves the cricket team of Sri Lanka, in the limited overs (50 a side, still enough to allow for more richness than most other sports) form of the game. These games tended to have a certain fixed narrative which held that 250 runs was a respectable total to defend, and that the side batting first should aim for a cautious “50 in 15″ start, followed by a steady, higher scoring tempo in the next 20 overs, and indiscriminate “slogging” in the last 10. Teams would lead with their relatively cautious and hard-to-intimidate defensive batsmen, who could survive the intimidation of the fast new-ball bowlers, clearing the way for the middle order of the batting lineup to score faster against the less intimidating old-ball slow bowlers (who rely on spin rather than speed).

Sanath Jayasurya of Sri Lanka changed all that. Along with his team mate Romesh Kaluwitharana, they introduced a narrative that involved attempting to hammer away at the opening bowlers. Rather than meeting intimidation with caution (opening bowlers tend to be intimidating fast bowlers), they turned the story into a high-risk intimidation-vs.-intimidation premise. If it worked, huge, unprecedented totals could be piled up.

Some saw this innovation as merely a case of bringing to cricket a sense of spectacle it normally lacks, but that wasn’t it (that would be done by Twenty20 nearly two decades later). The Sri Lankan offensive model created thriller narratives with real depth.

The Valley of Darkness

In classic narrative models, such as Joseph Campbell’s monomyth, there is often a “Valley of Darkness” phase when the hero must struggle alone through a scary set of trials. The emotion and tempo in these narrative phases is one of gritty persistence, and the ability to press on through the darkness, driven only by faith and a sense of bloody-minded relentlessness.

The characteristic feature of such narrative phases is the length. You can’t have a story where you claim there’s a gritty “valley of darkness” phase and it is wrapped up in five minutes. If Frodo, Samwise and Gollum had finished their journey from the border of Mordor to Mount Doom in five minutes, you’d have felt cheated. It is the sheer length of the journey, that sense of hopelessness that descends, that makes it such a critical phase in the “Hero’s Journey”  narrative template.

Cricket is the only game I know of that can produce this phase. Where Frodo trudges, burdened by despair, one step at a time, in cricket there’s a phrase used to describe what bowlers must do: “line and length.” For hours bowlers must attack with a very steady, disciplined control, not allowing the ball to stray. The batsmen must remain equally disciplined, resisting the temptation to take unnecessary risks, and milking runs from every loose ball.

There’s plenty of anxious negative space with nothing happening.

Sounds a lot like life itself, doesn’t it?

This is one of many rabbit-trail pieces that I wrote while exploring ideas for my book project, Tempo. This essay distracted me for a few pleasant hours from my chapter titled “Narrative Rationality.” I have to regretfully inform cricket fans that there isn’t much reference to cricket in the book. I am sure non-cricket fans are relieved. Sign up for the book release announcement mailing list.

It was cool and mildly breezy around 8 PM today. So I went for a walk, and I noticed something. Though I passed a couple of hundred people, nobody else was taking a walk. There were people returning from work, people going places with purpose-laden bags, people running, people going to the store, people sipping slurpies.  But nobody taking a walk. Young women working their phones, but not taking a walk. People walking their dogs, or pushing a stroller, with the virtuous air of one performing a chore for the benefit of another, but not themselves taking a walk. I was the only one taking a walk. The closest activity to “taking a walk” that I encountered was two people walking together and forgetting, for a moment, to talk to each other. The moment passed. One of them said something and they slipped back into talking rather than taking a walk.

My observation surprised me, and I tried to think back to other walks. I take a lot of walks, so there are a lot of memories to comb through. In my 13 years of taking walks in the United States, I could remember only ever seeing one native-born American taking a walk. All other examples I could remember were clearly immigrants. Middle-aged eastern European matrons strolling. Old Chinese men walking slowly with their hands behind their backs.  Even elderly Americans don’t seem to take walks the way elderly immigrants do. They walk slowly, but they look like they’re doing it for the exercise. They often look resentfully at young runners.

It is not hard to take a walk. The right shoes are the ones nearest the door. The right clothes are the ones you happen to be wearing. You will not sweat. You may need a jacket if it is cold, or an umbrella if it is raining.  If you pass anybody, you are not walking slowly enough for it to be “taking a walk.” If you need to make up a nominal purpose like “get more bananas from the store” you are not taking a walk.

Taking walks is the entry drug into the quiet, solitary heaven of idleness (the next level up is “sitting on a bench without a view”). For modern Americans, idleness is a shameful, private indulgence.  If they attempt it in public, they are stricken by social anxiety. They seem to fear that the slow, solitary, and obviously purposeless amble that marks “taking a walk” signals social incompetence or a life unacceptably adrift. If a shopping bag, gym bag, friend or dog cannot be manufactured, nominal non-idleness must be signaled through an ostentatious “I have friends” phone call, or email-checking. If all else fails, hands must be placed defiantly in pockets, to signal a brazen challenge to anyone who dares look askance at you, “Yeah, I’m takin’ a walk! You got a problem with that?”

In America, visible idleness is a luxury for the homeless,  the delinquent and immigrants.  The defiantly tautological protest, “I have a life,” is quintessentially American. The American life does not exist until it is filled up.

Even a pause at a bench must be justified by a worthwhile view or a chilled drink.

Worthwhile. Now, there’s an American word. Worth-while. Worth-your-while. The time value of money. Someone recently remarked that the iPad has lowered the cost of waiting. Americans everywhere heaved a sigh of relief, as their collective social anxiety dipped slightly. The rest of the world groaned just a little bit.

The one American I remember seeing taking a walk was Tom Hales, then a professor at the University of Michigan. He was teaching the differential geometry course I was auditing that semester. One dark, solitary Friday, while the rest of America was desperately trying to demonstrate to itself that it had a life, I was taking a walk in an empty, desolate part of the campus. I saw Hales taking a walk on the other side of the street. He did not look like he was pondering Deep Matters. He merely looked like he was taking a walk.

That year he proved the Kepler conjecture, a famous unsolved problem dating back to 1611. A beautifully pointless problem about how to stack balls. I like to think that Kepler  must have enjoyed taking walks too.

[Addendum: A fascinating discussion of this post has developed on Hacker News]

Update: the book is published, cover and all. Adam Hogan did the cover.

I am at that dangerous stage with my first book project, Tempo, where I am going around telling people the manuscript is “95% done,” but with the last 5% threatening to take 50% of the time by the time the it is actually done. But still, with cautious optimism, I can report that I really do think I’ll get the book out by November, as I’ve promised. Which brings me to the reason for this post: I need a cover design. If you are a book cover designer and want to take a shot at it, read on. If you are not, but happen to know good book cover designers, please help me out by emailing along a link to this post, reblogging it, and so forth. Designers with no book-cover experience, you can still bid, but I’ll probably favor people with experience unless they ALL price themselves out of my budget. All bids due by August 10, 2010

9 Simple Rules

1. My maximum budget is $1000. And I’d rather spend MUCH less upfront. I intend this book to recover my cash investment and start making money as soon as possible.
2. You can bid for the job by emailing me your bid with links to samples of your work. Do mention how you found out about this job. Read the rest of this list first though.
3. MY WEIRD OFFER: You can choose to bid for some mix of a dollar amount under $1000, and a per-copy profit share up to $1 per copy, on copies sold in the first year, up to a maximum of 3000 copies. So if I sell 500 copies in Year 1, and you bid $500+$1/copy, you’ll make $1000. If it becomes a runaway hit,  you make a maximum of $3500.
4. Calibration: I have approximately 500 people signed up for the book release announcement/beta lists already. And this blog has 2000+ RSS subscribers, growing steadily. You decide what that means.
5. I would prefer bids from the United States to keep the logistics and communication simple, but will consider bids from other countries.
6. If you have never done book design before, send links to samples of your most relevant work. Adjust your bid downwards accordingly
7. This is just an informal, non-binding, request for quotes (RFQ). If I pick your bid, we’ll try to figure out a deal and a mutually acceptable creative brief. If we can’t, I’ll move on to my second choice. And so on.
8. I know many readers of this blog are designers. If you choose to bid, please don’t be offended if I don’t end up picking you. I appreciate your loyalty to this site as a reader, but my priority is to get a great design.
9. Even if you are a big fan of ribbonfarm, please don’t offer to do it for free (I’ve received such offers before, but I can’t accept free work when I make money myself)

So if you’re interested, please email me your bid (dollar amount plus profit-share proposal) and samples to your work. Mention how you found out about the job.

All bids due by August 10, 2010.

If I don’t get enough good bids through this post, I’ll end up looking at the normal channels.

James C. Scott’s fascinating and seminal book, Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed, examines how, across dozens of domains, ranging from agriculture and forestry, to urban planning and census-taking, a very predictable failure pattern keeps recurring.  The pictures below, from the book (used with permission from the author) graphically and literally illustrate the central concept in this failure pattern, an idea called “legibility.”

States and large organizations exhibit this pattern of behavior most dramatically, but individuals frequently exhibit it in their private lives as well.

Along with books like Gareth Morgan’s Images of Organization, Lakoff and Johnson’s Metaphors we Live By, William Whyte’s The Organization Man and Keith Johnstone’s Impro, this book is one of the anchor texts for this blog. If I ever teach a course on ‘Ribbonfarmesque Thinking,’ all these books would be required reading. Continuing my series on complex and dense books that I cite often, but are too difficult to review or summarize, here is a quick introduction to the main idea.

The Authoritarian High-Modernist Recipe for Failure

Scott calls the thinking style behind the failure mode “authoritarian high modernism,” but as we’ll see, the failure mode is not limited to the brief intellectual reign of high modernism (roughly, the first half of the twentieth century).

Here is the recipe:

• Look at a complex and confusing reality, such as the social dynamics of an old city
• Fail to understand all the subtleties of how the complex reality works
• Attribute that failure to the irrationality of what you are looking at, rather than your own limitations
• Come up with an idealized blank-slate vision of what that reality ought to look like
• Argue that the relative simplicity and platonic orderliness of the vision represents rationality
• Use authoritarian power to impose that vision, by demolishing the old reality if necessary
• Watch your rational Utopia fail horribly

The big mistake in this pattern of failure is projecting your subjective lack of comprehension onto the object you are looking at, as “irrationality.” We make this mistake because we are tempted by a desire for legibility.

Legibility and Control

Central to Scott’s thesis is the idea of legibility. He explains how he stumbled across the idea while researching efforts by nation states to settle or “sedentarize” nomads, pastoralists, gypsies and other peoples living non-mainstream lives:

The more I examined these efforts at sedentarization, the more I came to see them as a state’s attempt to make a society legible, to arrange the population in ways that simplified the classic state functions of taxation, conscription, and prevention of rebellion.  Having begun to think in these terms, I began to see legibility as a central problem in statecraft. The pre-modern state was, in many crucial respects, particularly blind; it knew precious little about its subjects, their wealth, their landholdings and yields, their location, their very identity. It lacked anything like a detailed “map” of its terrain and its people.

The book is about the 2-3 century long process by which modern states reorganized the societies they governed, to make them more legible to the apparatus of governance. The state is not actually interested in the rich functional structure and complex behavior of the very organic entities that it governs (and indeed, is part of, rather than “above”). It merely views them as resources that must be organized in order to yield optimal returns according to a centralized, narrow, and strictly utilitarian logic. The attempt to maximize returns need not arise from the grasping greed of a predatory state. In fact, the dynamic is most often driven by a genuine desire to improve the lot of the people, on the part of governments with a popular, left-of-center mandate. Hence the subtitle (don’t jump to the conclusion that this is a simplistic anti-big-government conservative/libertarian view though; this failure mode is ideology-neutral, since it arises from a flawed pattern of reasoning rather than values).

The book begins with an early example, “scientific” forestry (illustrated in the picture above). The early modern state, Germany in this case, was only interested in maximizing tax revenues from forestry. This meant that the acreage, yield and market value of a forest had to be measured, and only these obviously relevant variables were comprehended by the statist mental model. Traditional wild and unruly forests were literally illegible to the state surveyor’s eyes, and this gave birth to “scientific” forestry: the gradual transformation of forests with a rich diversity of species growing wildly and randomly into orderly stands of the highest-yielding varieties. The resulting catastrophes — better recognized these days as the problems of monoculture — were inevitable.

The picture is not an exception, and the word “legibility” is not a metaphor; the actual visual/textual sense of the word (as in “readability”) is what is meant. The book is full of thought-provoking pictures like this: farmland neatly divided up into squares versus farmland that is confusing to the eye, but conforms to the constraints of local topography, soil quality, and hydrological patterns; rational and unlivable grid-cities like Brasilia, versus chaotic and alive cities like Sao Paolo. This might explain, by the way, why I resonated so strongly with the book.  The name “ribbonfarm” is inspired by the history of the geography of Detroit and its roots in “ribbon farms” (see my About page and the historic picture of Detroit ribbon farms below).

High-modernist (think Bauhaus and Le Corbusier) aesthetics necessarily lead to simplification, since a reality that serves many purposes presents itself as illegible to a vision informed by a singular purpose. Any elements that are non-functional with respect to the singular purpose tend to confuse, and are therefore eliminated during the attempt to “rationalize.” The deep failure in thinking lies is the mistaken assumption that thriving, successful and functional realities must necessarily be legible. Or at least more legible to the all-seeing statist eye in the sky (many of the pictures in the book are literally aerial views) than to the local, embedded, eye on the ground.

Complex realities turn this logic on its head; it is easier to comprehend the whole by walking among the trees, absorbing the gestalt, and becoming a holographic/fractal part of the forest, than by hovering above it.

This  imposed simplification, in service of legibility to the state’s eye, makes the rich reality brittle, and failure  follows. The imagined improvements are not realized. The metaphors of killing the golden goose, and the Procrustean bed come to mind.

The Psychology of Legibility

I suspect that what tempts us into this failure is that legibility quells the anxieties evoked by apparent chaos. There is more than mere stupidity at work.

In Mind Wide Open, Steven Johnson’s entertaining story of his experiences subjecting himself to all sorts of medical scanning technologies, he describes his experience with getting an fMRI scan. Johnson tells the researcher that perhaps they should start by examining his brain’s baseline reaction to meaningless stimuli. He naively suggests a white-noise pattern as the right starter image. The researcher patiently informs him that subjects’ brains tend to go crazy when a white noise (high Shannon entropy) pattern is presented. The brain goes nuts trying to find order in the chaos. Instead, the researcher says, they usually start with something like a black-and-white checkerboard pattern.

If my conjecture is correct, then the High Modernist failure-through-legibility-seeking formula is a large scale effect of the rationalization of the fear of (apparent) chaos.

[Techie aside: Complex realities look like Shannon white noise, but in terms of deeper structure, their Kolmogorov-Chaitin complexity is low relative to their Shannon entropy; they are like pseudo-random numbers or π, rather than real random numbers; I wrote a two-part series on this long ago, that I meant to continue, but never did].

The Fertility of the Idea

The idea may seem simple (though it is surprisingly hard to find words to express it succinctly), but it is an extraordinarily fertile one, and helps explain all sorts of things. One of my favorite unexpected examples from the book is the “rationalization” of people names in the Philippines under Spanish rule (I won’t spoil it for you; read the book). In general, any aspect of a complex folkway, in the sense of David Hackett Fischer’s Albion’s Seed, can be made a victim of the high-modernist authoritarian failure formula.

The process doesn’t always lead to unmitigated disaster. In some of the more redeeming examples, there is merely a shift in a balance of power between more global and more local interests. For example, we owe to this high-modernist formula the creation of a systematic, global scheme for measuring time, with sensible time zones. The bewilderingly illegible geography of time in the 18th century, while it served a lot of local purposes very well (and much better than even the best atomic clocks of today), would have made modern global infrastructure, ranging from the railroads (the original driver for temporal discipline in the United States) to airlines and the Internet, impossible. The Napoleanic era saw the spread of the metric system; again an idea that is highly rational from a centralized bird’s eye view, but often stupid with respect to the subtle local adaptions of  the systems it displaced. Again this displaced a good deal of local power and value, and created many injustices and local irrationalities, but the shift brought with it the benefits of improved communication and wide-area commerce.

In all these cases, you could argue that the formula merely replaced a set of locally optimal modes of social organization with a globally optimal one. But that would be missing the point. The reason the formula is generally dangerous, and a  formula for failure, is that it does not operate by a thoughtful consideration of local/global tradeoffs, but through the imposition of a singular view as “best for all” in a pseudo-scientific sense. The high-modernist reformer does not acknowledge (and often genuinely does not understand) that he/she is engineering a shift in optima and power, with costs as well as benefits. Instead, the process is driven by a naive “best for everybody” paternalism, that genuinely intends to improve the lives of the people it affects. The high-modernist reformer is driven by a naive-scientific Utopian vision that does not tolerate dissent, because it believes it is dealing in scientific truths.

The failure pattern is perhaps most evident in urban planning, a domain which seems to attract the worst of these reformers. A generation of planners, inspired by the crazed visions of Le Corbusier, created unlivable urban infrastructure around the world, from Braslia to Chandigarh. These cities end up with deserted empty centers populated only by the government workers forced to live there in misery (there is even a condition known as “Brasilitis” apparently), with slums and shanty towns emerging on the periphery of the planned center; ad hoc, bottom-up, re-humanizing damage control as it were. The book summarizes a very elegant critique of this approach to urban planning, and the true richness of what it displaces, due to Jane Jacobs.

Applying the Idea

Going beyond the book’s own examples, the ideas shed a whole new light on other stories/ideas. Two examples from my own reading should suffice.

The first is a book I read several years back, by Nicholas Dirks, Castes of Mind: Colonialism and the Making of Modern India, which made the argument (originally proposed by the orientalist Bernard Cohn), that caste in the sense of the highly rigid and oppressive, 4-varna scheme was the result of the British failing to understand a complex social reality, and imposing on it their own simplistic understanding of it (the British Raj is sometimes called the “anthropological state” due to the obsessive care it took to document, codify and re-impose as a simplified, rigidified, Procrustean prescription, the social structure of pre-colonial India).  The argument of the book — obviously one that appeals to Indians (we like to blame the British or Islam when we can) — is that the original reality was a complex, functional social scheme, which the British turned into a rigid and oppressive machine by attempting to make it legible and governable. While I still don’t know whether the argument is justified, and whether the caste system before the British was as benevolent as the most ardent champions of this view make it out to be, the point here is that if it is true, Scott’s failure model would describe it perfectly.

The second example is Gibbon’s Decline and Fall of the Roman Empire, which I am slowly reading right now (I think it is going to be my personal Mount Everest; I expect to summit in 2013). Perhaps no other civilization, either in antiquity or today, was so fond of legible and governable social realities.  I haven’t yet made up my mind, but reading the history through the lens of Scott’s ideas, I think there is  strong case to be made that the fall of the Roman empire was a large-scale instance of the legibility-failure pattern. Like the British 1700 years later, the Romans did try to understand the illegible societies they encountered, but their failure in this effort ultimately led to the fall of the empire.

Aside: if you decide to attempt Mount Everest along with me, take some time to explore the different editions of Gibbon available; I am reading a $0.99 19th century edition on my Kindle — all six volumes with annotations and comments from a decidedly pious — and critical — Christian editor. Sometimes I don’t know why I commit these acts of large-scale intellectual masochism.  The link is to a modern, abridged Penguin edition.

Is the Model Relevant Today?

The phrase “high-modernist authoritarianism” might suggest that the views in this book only apply to those laughably optimistic, high-on-science-and-engineering high modernists of the 1930s. Surely we don’t fail in these dumb ways in our enlightened postmodern times?

Sadly, we do, for four reasons:

1. There is a decades-long time lag between the intellectual high-watermark of an ideology and the last of its effects
2. There are large parts of the world, China in particular, where authoritarian high-modernism gets a visa, but postmodernism does not
3. Perhaps most important: though this failure mode is easiest to describe in terms of high-modernist ideology, it is actually a basic failure mode for human thought that is time and ideology neutral. If it is true that the Romans and British managed to fail in these ways, so can the most postmodern Obama types. The language will be different, that’s all.
4. And no, the currently popular “pave the cowpaths” and behavioral-economic “choice architecture” design philosophies do not provide immunity against these failure modes. In fact paving the cowpaths in naive ways is an instance of this failure mode (the way to avoid it would be to choose to not pave certain cowpaths). Choice architecture (described as “Libertarian Paternalism” by its advocates) seems to merely dress up authoritarian high-modernism with a thin coat of caution and empirical experimentation. The basic and dangerous “I am more scientific/rational than thou” paternalism is still the central dogma.

[Another Techie aside: For the technologists among you, a quick (and very crude) calibration point should help: we are talking about the big brother of waterfall planning here. The psychology is very similar to the urge to throw legacy software away. In fact Joel Spolsky's post on the subject Things You Should Never Do, Part I, reads like a narrower version of Scott's arguments. But Scott's model is much deeper, more robust, more subtly argued, and more broadly applicable.  I haven't yet thought it through, but I don't think lean/agile software development can actually mitigate this failure mode anymore than choice architecture can mitigate it in public policy]

So do yourself a favor and read the book, even if it takes you months to get through. You will elevate your thinking about big questions.

High-Modernist Authoritarianism in Corporate and Personal Life

The application of these ideas in the personal/corporate domains actually interests me the most. Though Scott’s book is set within the context of public policy and governance, you can find exactly the same pattern in individual and corporate behavior. Individuals lacking the capacity for rich introspection apply dumb 12-step formulas to their lives and fail. Corporations: well, read the Gervais Principle series and Images of Organization. As a point of historical interest, Scott notes that the Soviet planning model, responsible for many spectacular legibility-failures, was derived from corporate Taylorist precedents, which Lenin initially criticized, but later modified and embraced.

Final postscript: these ideas have strongly influenced my book project, and apparently, I’ve been thinking about them for a long time without realizing it. A very early post on this blog (I think only a handful of you were around when I posted it), on the Harry Potter series and its relation to my own work in robotics, contains some of these ideas. If I’d read this book before, that post would have been much better.

One reason I have delayed posting the next part in the Gervais Principle series is that as expectations have grown, I have gotten more wary about shooting from the hip. Especially because the remaining ideas in the hopper (there’s enough for two more posts before I call the main series complete) will likely be even more controversial than the first two. So one of the things I have been doing is testing the foundations laid in the first two posts more rigorously. So here goes, a (very pictorial) survey of the ancestry of the MacLeod hierarchy and the Gervais Principle. This is not Part III. It is another side trip. Not many new ideas here, but genealogy should prove interesting for at least some of you. A sense of history is a necessary (though unfortunately not sufficient) requirement for  effective sociopathy. For those who came in late, this post will make no sense to you. Read The Gervais Principle and The Gervais Principle II before you tackle this one.

The Complete MacLeod Hierarchy

First, I need to draw a more detailed version of the original cartoon by Hugh MacLeod, one that captures the Gervais Principle itself, which I only described with words before. Here are two views of the complete picture. The first, the dynamics view, shows the 3 behaviors: checking out, overperformance and Machiavellian scheming, that animate the hierarchy and create its people flows. It pained me to have to mess with the elegant simplicity of Hugh’s original, but too many people were getting confused about what exactly I meant, and we need this detailed view to do the genealogy.

This dynamics/behavioral view does not capture the full complexity of what’s going on, so you need a complementary structural view to capture that.  This diagram is even uglier. Here it is:

This is way too mechanistic for my tastes, but is the best I could do for now (I was trying to draw it using a volcano metaphor, but got nowhere). The important structural elements to note are the three ways out of the entry lobby at the lowest level, of which two have filters/gatekeeper mechanisms. Machiavellian scheming takes you straight up the pipe, and if you sweet-talk the bouncers and get past the velvet rope, you are in.  The second filter is the Gate of “Come hang out with us.” Aspirants to checked-out loserdom must actually get past the social gatekeepers that control access to the checked-out loser groups that make life at the bottom bearable. In a very explicit example, Andy is kept out of the “finer things club” run by Pam, Oscar and Toby (an adult version of a new kid in a school trying to join the ‘cool kids’ table, then the ‘geek kids’ table, and being rejected by both, having to eat alone). An everyday example is the New Guy being invited to join the regular evening beer session.

The only path that does not contain a gatekeeper is the Employee of the Month funnel. Mere record-setting overperformance — something within your own control to a large extent — gets you there. That’s why it’s actually lonelier in the middle than at the top. The loneliness of leadership (the top) is not as extreme as the loneliness of clueless middle-management life, rejected by the sociable huddle below you, and by the predatory players above you. And remember, the Sociopaths actually like solitude, while the Clueless yearn for connections that they are denied. So they suffer more.

Okay, now that we’ve gotten that clearly reviewed, let’s look at the history.

Drucker Hierarchy (1954)

Curiously, despite the severe handicap of being, apparently, a nice guy, Drucker actually got this stuff. In Management by Objectives and Self-Control (included in The Essential Drucker) he wrote:

A favorite story at management meetings is that of the three stonecutters who were asked what they were doing. The first replied, “I am making a living.” The second kept on hammering while he said, “I am doing the best job of stonecutting in the entire country.” The third looked up with a visionary gleam in his eyes and said, “I am building a cathedral.” The third man is, of course, the true “manager” [we are more likely to call this person 'leader' in 2010 - vgr] The first man knows what he wants to get out of the work and manages to do so. He is likely to give a “fair day’s work for a fair day’s pay.”… It is the second man who is a problem… there is always a danger that the true workman, the true professional, will believe that he is accomplishing something when in effect he is just polishing stones or collecting footnotes.

That, in short, is the source of blinded-by-craftsmanship guildism that is particularly characteristic of the Clueless. Think the varied passions of Dwight. Though Drucker offers some screwed-up views on occasion, he has sufficiently many gems like this scattered through his (volumnious) writing that he earns redemption, and entry into the Whyte school of management. Here’s the hierarchy capturing Drucker’s anecdote:

Whyte Hierarchy(1956)

I’ve talked enough about William Whyte, that I can just offer you his ideas in triangular form without further comment. In my Organization Man trail, I haven’t yet gotten to the part in his book where he talks about the “Non-well rounded man,” but I will. Note that Whyte doesn’t talk about the lowest level much because he isn’t interested enough in the Losers to explore their lives in detail (he is actually most interested in the after-work life of the organization man). It is a pity this version of the hierarchy looks so impoverished, because Whyte has some of the deepest, richest things to say about all this.

Hammerstein-Equord Hierarchy (1933)

This one was new to me, but I was delighted to find it, thanks to @amitseshan. Here is the relevant quote from General Kurt von Hammerstein-Equord, from the German military, 1933:

I divide my officers into four classes; the clever, the lazy, the industrious, and the stupid. Most often two of these qualities come together. The officers who are clever and industrious are fitted for the highest staff appointments. Those who are stupid and lazy make up around 90% of every army in the world, and they can be used for routine work. The man who is clever and lazy however is for the very highest command; he has the temperament and nerves to deal with all situations. But whoever is stupid and industrious is a menace and must be removed immediately!

So let’s put him on the family tree as well.  I have been wary about applying these ideas to the military because you don’t want to mess with even ironic-offensive pigeonholing of the people protecting the democracy we are having fun in. But if the military themselves arrive at the idea through introspection, who am I to object? (besides this quote, there are moments in Band of Brothers, Men Who Stare At Goats and of course, Catch 22, that really get at the military version of all this). The good general though, apparently hadn’t heard of 2×2 diagrams, so his language is a little confused. He actually has 2 axes and 4 quadrants, each defined by a pair.

Veblen Hierarchy (1899)

Now we get to what is probably the granddaddy of this line of thinking. I bought Veblen’s Theory of the Leisure Class several years ago, but for some reason (now I know: the anxiety of influence), didn’t get beyond Chapter 2 even though I was deeply attracted to the ideas. After I wrote GP I and GP II, the selfsame @amitseshan (he’s been among the people prodding me the hardest to continue this series) urged me to finish the damn book. I did and was blown away. Expect a review soon. This version of the hierarchy is a pretty good summary of Veblen’s (very densely-argued) book:

Veblen not only gets that the Sociopaths are in many ways very similar to the checked-out Losers (“lower delinquent class” in his vocabulary) he actually explores the dynamics of exodus (beyond checking out: dropping out) that I haven’t included in my model yet. The problem with dealing with the phenomenology of exit/exodus is that it takes you into full-blown social psychology and sociology within society at large, and I won’t have time to take that on anytime soon. Maybe when I retire.

Of all the precursors of the MacLeod hierarchy, Veblen’s is the most complete. He even anticipates evolutionary psychology arguments that can plausibly explain this outcome (he has a fairly robust social-Darwinist model that outlines a decent speculative theory of how the modern leisure class evolved out of “higher barbarian” and “barbarian” classes in earlier societies).

Shakespeare “Julius Caesar” Hierarchy (1623)

You should expect that Shakespeare had something to say about all this. He did:

Let me have men about me that are fat;
Sleek-headed men and such as sleep o’ nights:
Yond Cassius has a lean and hungry look;
He thinks too much: such men are dangerous.

…
Would he were fatter! But I fear him not:
Yet if my name were liable to fear,
I do not know the man I should avoid
So soon as that spare Cassius. He reads much;
He is a great observer and he looks
Quite through the deeds of men: he loves no plays,
As thou dost, Antony; he hears no music;
Seldom he smiles, and smiles in such a sort
As if he mock’d himself and scorn’d his spirit
That could be moved to smile at any thing.
Such men as he be never at heart’s ease
Whiles they behold a greater than themselves,
And therefore are they very dangerous.

Julius Caesar, Act 1, Scene 2

No comments necessary. The diagram:

Which brings us to the most basic version of the hierarchy of all: the Darwinist one.

Red Queen Hierarchy (2006/20,000 BC, depending on how you measure the age)

The Red Queen is yet another @amitseshan recommendation, this being one that I hadn’t heard of. I may or may not review this (it would be flame-bait due to its treatment of sexual selection, which I find unproblematic, but will likely make many readers go postal). Though I didn’t get a whole lot out of the book that I hadn’t already gotten out of The Selfish Gene, one theme that made a major impression on me revolves around the huge impact parasites have had in human evolutionary history. We tend to think of evolution primarily in predator-prey terms, but it turns out that parasitism might have been an evolutionary force equal to, or greater than, predation. This version of the triangle is the one I trust the least, since a lot of the arguments are very technical-biological ones, but here goes nothing:

I won’t attempt to seriously argue that this hierarchy is basically correct, because of the biological details, but three points to note are:

1. Unlike the Social-Darwinist versions of the hierarchy, the genetic-Darwinist analogue does not permit movement among layers (for obvious reasons)
2. Predators are sudden and great threats to prey, but parasites are low-level weakening agents that only kill you if their activity goes above a threshold.
3. A potentially hazardous analogy: parasites weaken prey enough for predators to prey on. The clueless make the management of the losers easier for the sociopaths.

Other Potential Versions

There are other family tree candidates that I have not verified. Greek mythology (especially the idea of Titans battling the Olympians, and the stories of Icarus and Prometheus) seems to contain several of these themes, but I am not yet ready to venture a mapping. Indian mythology (which I am more familiar with), also maps, but again, not with the clarity I would like. If you can think of any other potential candidates, do post a comment. Curiously, The Lord of the Rings seems to map as well.

And to conclude, a connection that surprised me: an earlier version of the triangle in the backyard of my own thinking.

The Tempo Drummers and Dancers Hierarchy (2007)

My book-in-progress, about decision-making, Tempo, was initially meant to be a comprehensive treatment of all kinds of decision-making. That turned out to be too much to put into one book, so early on, I partitioned my material into individual and collective decision-making buckets, with the idea that I would put the collective stuff into a sequel. Keeping with the theme of “tempo,” I labeled my folder of notes for the sequel Drummers and Dancers. That has turned into a sort of working title. I know, what am I thinking, right? I haven’t even shipped the first book and I am already planning the sequel. Oh well.

Drummers and Dancers was a right-brained working-title choice, and I hadn’t thought deeply about the significance of the phrase. It only struck me about a month ago, reviewing my material, that the organizing theme of Drummers and Dancers is basically another variant of the MacLeod hierarchy.

I hope the symbolism is obvious.  If not, wait for the book. After I finish Tempo, and if I have the energy to work on the sequel, that will be the book version of this series. I may keep Drummers and Dancers as the title. Sign up for the Tempo mailing list if you haven’t already.

I recently bought a classic, cherry-finish  River City hourglass. It was the first time I deliberately bought something to serve as a social object, which I’ll define as any tangible entity that can catalyze a characteristic social chemistry. In this case, the hourglass helped me tweak the ambiance of a writers meetup I run in the Washington, DC area.

I’ve wondered for years about how people connect over particular elements of their environment, ranging from water coolers and YouTube videos to parrots. We are currently in the thick of social object season:  turkeys, Christmas trees, mistletoe.

Social objects are a complex idea. We need a theory that can provide a conceptual framework and vocabulary, suggest conjectures that might become laws, and distinguish between social objects and related but distinct creatures such as memes, social signals, brands and ritual objects. A good theory should also shed light on specific questions, such as “why have so many hip young American women taken up knitting in recent years?”

I am finally beginning to see the outlines of such a general theory. The first useful inference I have been able to derive is this: when communities digitize, social objects replace walls. I call this the first law of social objects. Let’s work our way up to that. (before more people yell at me… yes, this is an early beta stab at a new theme, so apologies for the length and looseness of editing).

The Universe of Social Objects

I’ve been wondering about how people connect for a while, but I only learned about Jyri Engstrom and Hugh Macleod’s ideas on social objects a few months ago, and realized that I’d found a productive attack on the topic. Unlike Engstrom and MacLeod though, I am primarily interested in physical social objects to begin with, and in their intrinsic nature rather than aspects like sharing or how they induce the structure of social networks. Let’s start with a list of the usual suspects; candidate social objects old and new.

1. Village well
2. Office watercooler
3. Cognac among African Americans
4. Louis Vuitton purses
5. Knitting among hip young American women
6. A street musician
7. A pet or baby in a public space
8. That YouTube video of a wedding.
9. A controversial WikiPedia page
10. The #iranelection hashtag on Twitter
11. The public sculptures of Larry Morris,
12. A coffee table book

I have a growing branch on my social objects trail that has other examples (including people-as-social-objects), but this dozen should be enough. Take a minute to scan the list and mentally pick what you think are the best and worst examples of social objects. Yes, it is a fuzzy set, with degrees of membership, but there’s a point to this exercise.

Done? Let me throw a couple of extreme examples at you, unusual suspects, so you get a sense of the outer limits of the design space we are working with here.

First, consider a delayed flight. You’ve boarded and left the gate, and then the captain announces that you’re going to sit on the tarmac for an hour, thanks to a weather-related ground stop. If you’ve ever been in this situation, you know what happens next: a good subset of people start talking to each other. Here the social object – a delay – is borderline intangible. Almost a metaphysical abstraction. It is also about as simple and attribute-free as a social object can get, since it is just an interval of time.

The second example is from the other end of the spectrum of complexity. It is the traditional  Balinese cockfight, which inspired an influential essay by anthropologist Clifford Geertz (the title of this post is an homage to that essay, “Deep Play: Notes on the Balinese Cockfight”). The spectacle, as Geertz describes and analyzes it, is about as complex as a social object can get:

But the cockfight is – or more exactly, deliberately is made to be – a simulation of the social matrix, the involved system of crosscutting, overlapping, highly corporate groups –villages, kingroups, irrigation societies, temple congregations, “castes” – in which its devotees live.

Geertz’ article repays careful reading many times over, and illuminates many themes, but the one we are interested in here, is how social objects highlight and reinforce the key elements of the social matrix within which they are embedded. It is through that effect that social objects catalyze connections.

So we’ve seen a few examples, and seen some limit points – nearly featureless and almost intangible social objects at one extreme, and highly elaborate social enactments on the other.

This brings me to the question that interests me. What exactly is a social object? What can we say beyond “catalyzes a specific social chemistry?”

Social Objects and Group Dynamics

An episode of Two and a Half Men highlights some of the subtleties involved in thinking about social objects.

In the episode in question, Alan, the clueless younger brother, is trying to get invited to a regular scotch-and-cigars group of which Charlie, the player older brother, is part. The other characters (correctly) recognize that the group is really a support group for men facing the problems of  early, easy success – too much money, too many women – and (incorrectly) dismiss the scotch and cigars.

Alan clearly does not meet the criteria. Still, the group finally takes pity on him, and overriding Charlie’s objections, lets him join. Alan’s first gauche moves instantly reveal that he does not belong: with a great show of propriety, he declares that he doesn’t want to disrupt the group, and asks whether there are “any rituals I should know about, like a talking stick?”

As you might expect, things get worse, and the episode ends with Alan being humiliated and thrown out of the group.

The interesting thing here is the interplay of two social objects. Alan thinks of the group as a support group, projects his own (clearly feel-good hippie) idea of support groups, and asks about “talking sticks.” He is oblivious to the fact that the real social object of the evening is scotch-and-cigars. A more appropriate opening question would have been, “What are we drinking?” or “Are these Cubans?”

What can we learn about social objects from this little vignette? Plenty. Notice that cigars and scotch act as an exceptionally good filter in picking out the target subculture: manly macho men. Not effete, ineffectual ones like Alan. It sets the tone for the gathering. It is unstructured and mellow, but tough. It is confessional, but in a no-whining-no-bullshit way. There is an element of ritual, but not an overwhelming display. Scotch and cigars signal a certain grown-up-ness about the proceedings. The very idea of a “talking stick” on the other hand, suggests highly structured and ritually procedural settings, governed by parental figures such as therapists (or Native American chiefs).

Social Containers and Object Fields

Is Alan’s talking stick merely an inappropriate social object for the situation, or just a bad social object in an absolute sense? We need some general ideas before we can make sense of this sort of question.

The single most important characteristic of a social object is that it divides the world into an in-group and an out-group. It shares this property with a complementary construct I’ll call a social container. A social container is a boundary-signifier  that isolates an in-group, by explicitly blocking outsiders, who are defined in terms of membership signifiers. Guest lists, passports and actual physical walls are examples.

A social object on the other hand, includes/excludes members by creating an object field. Like a magnetic field, or a gravitational field. Or a catchment area. Fields are defined by a gradually fading “region of attraction.” Two social objects near each other — say a stimulating painting on the wall facing a fireplace in a large room where a party is being held — create invisible boundaries between them where the influence and attraction of the two objects is about equal (like a watershed dividing two valleys). Acting on social graphs, social objects induce geometry where previously there was only topology (if that math reference is too obscure for you, ignore it).

If you suspect that the distinction is about fuzziness of the boundary you are mistaken. Containers can have fuzzy boundaries (race is an example), while object field boundaries can be pretty sharply defined ( Barack Obama either recognizes you, or he does not).

The real distinction is  that social containers merely classify and sort people. Social objects animate them, by stimulating behavioral responses. The tenor of those responses determines membership in the induced group, not a checklist of attributes. Social objects are a behaviorist model of inclusion, while social containers are structuralist models.

The two, however, can interact. A fireplace at a party is a social object that operates within a social container, the home of the host, containing guests the host lets in.

This leads us to first of several important properties of social objects: boundedness.

Ten Properties of Social Objects

Every social object draws from a catchment area defined by its tightest social container. This is a trite observation. A neighborhood knitting group is only accessible to those who live in or near that neighborhood, so a fuzzy geographic (or virtual-geographic) container boundary determines basic access. Beyond that, we get more subtle properties.

1. Boundedness determines whether or not an object field is contained within, or larger than, its immediate container. An object whose field is larger than its tightest leak-proof container is bound (because the shape of its container, rather than the shape of its field, determines its extent). Otherwise it is unbound.
2. Fertility: A social object must induce a rich web of associations, and trigger a rich variety of behaviors. This is because it is the pattern of responses in the candidate member that matters. An overall gestalt of behavior. If you clumsily drop a fork once, you won’t necessarily raise eyebrows in high society. On the other hand, a pattern of unpolished table manners marks you out clearly as an outsider. A person around a social object is like filings around a magnet. Some patterns mark you out as iron, others do not. The social object in question here, by the way, is silverware (or chopsticks in China, or your fingers, in India). Quality silverware induces rich associations of high society, and defines a behavioral universe. Note however, that we are talking about fertility of shared associations and common behaviors.  Proust’s famous madeline, that evoked memories of his entire life, does not qualify. You and I would just eat it.
3. Capacity: Only so many people can stand around a fireplace or watercooler. Only so many people can feel a sense of belongingness before the idea of a nation fragments. Every social object has a capacity. This means that objects with apparently large capacities, such as national flags, actually fragment into aspect-meanings. Blue and Red states in America do not relate to the American flag in the same way (the object is burnable as a socially legitimate act of protest at the extremes of one group, but not in the other).
4. Tempo: A powerful social object affects emotions, energy levels and our perception of time. A fireplace makes you more mellow, calms you, encourages perspective over immediacy, and slows down the experience of time.
5. Tribalism: A newspaper is a social object that reinforces an abstract imagined community (a nation) of people who mostly do not know each other. A neighborhood café with a Clover coffee-maker on the other hand, is a social object that reinforces a much smaller community with tribal dynamics. A social object that creates a field that can hold less than 150 (the Dunbar number) people before it gets saturated, is a tribal object. One that can hold more is an imagined-community object.
6. Context-Sensitivity: Whiskey and cigars were social objects in Charlie’s party, but in other contexts, they might just be whiskey and cigars.
7. Moderation: Curiously, one of the ways in which social objects catalyze social chemistry is by moderating aspects of interactions that otherwise cause so much discomfort that people avoid contact altogether, or slowing down interactions that would otherwise happen too quickly to allow engagement. An example is an intriguing public sculpture. This idea has been used in these innovative and modern examples of social object design.
8. Tangibility: Social objects must be tangible; anchored in physical reality. They can be nearly abstract like an airplane delay, but not completely so. Digital social objects too, are physical, because information is physical (techie aside: this is an idea known as Landeur’s principle, and used to prove that Turing machines obey the laws of thermodynamics).
9. Recognizability: Like brands, social objects must be recognizable, and as we said, must evoke a unique web of shared meanings and associations in members of the social group it defines. There may be many clones (in which case each is locally a social object, as well as an imagined-community object), but it cannot be mistaken for something else.

But perhaps the most important property is property 10, Indirect Salience

Property 10: Indirect Salience

Effective social objects must have a recognizable connection to a theme that, if highlighted, can connect two or more people. This can either be an effect of design or selection. If the social object is to actively influence behavior, it must lend meaning to the behaviors in question. Magnets attract iron. There is a ferromagnetic theme.

But curiously, indirect salience is far more effective than direct salience. A web of connotations with no denotation. The most important kind of indirect salience is metonymy. In language, metonymy is a figure of speech, in which “a thing or concept is not called by its own name, but by the name of something intimately associated with that thing or concept.” Social objects achieve a sort of non-linguistic material metonymy. A club for surrealists might be called “Damp Clocks” for example, by reference to the famous Dali painting. The resultant object field would be more powerful than the “Surrealism Interest Group” container. This indirect salience is a feature social objects share with conceptual metaphor. By not being too in-your-face with one idea, they encourage peripheral vision, gently amplifying some connections, and weakening or destroying others.

I used to be really into amateur astronomy as a kid, and I still remember how I learned to “see” dim objects like the Andromeda galaxy: you look a little to the side, so the light hits a more sensitive part of your retina. Look at it head-on, and it will vanish. A more complex version of this is going on. Social objects are the instruments through which you see society.

Memes, Signals, Brands and Ritual Objects

For quite a while, I was unable to see the distinction between memes and social objects. Why, I wondered, did we need a new term? A meme, if you recall Richard Dawkin’s original model, is merely a fragment of information (physically embodied of course) that successfully replicates and propagates through what we would call, today, the social graph.

The  distinction isn’t sharp, but it is useful nevertheless. Memes are fragments of information that propagate on the social graph. Not all memes are social objects, and not all social objects are memes.

Consider that YouTube video of the wedding. Yeah, that one. The object in question is the hip, dancing entrance made by the wedding party at the start of the ceremony. Was this spectacle a social object or meme? That’s tricky.  It was a creative, tribal social object among the actual attendees. It was a reinforcing social object among groups that merely passed it along and used it to spark conversations (for example, the marketing community, which saw it as an instance of remarkable). Since it was canonized in pop-culture through a parody on The Office, it evolved into a social object at the level of an imagined community (the audience for The Office). And finally, among those who saw it and didn’t connect with anyone over it, it was merely a meme.

Consider social signals next. Say two men in conservative suits run into each other at a technology conference full of laid-back, casually-dressed engineers. The two men might recognize each other as (potentially) Wall Street types. They might even attempt to connect. But the suit is merely a social container (as an abstraction, don’t confuse that with the physical containment that is its utilitarian function) that enables recognition. It does not anchor the discourse. On the other hand, business cards can be social objects, most famously in that terrific scene in American Psycho where a bunch of Wall Street types ooh and aah over pretty much indistinguishable business cards (must read: The Onion: Why Can’t Anyone Tell I’m Wearing This Suit Ironically?).

Simpler examples are wedding rings, old school ties, secret handshakes and the like. All are signal objects, not genuine social objects.

What about brands? The idea of social objects, after all, first took root in the context of “viral” videos and the whole marketing side of social media.

One of Al Ries’ immutable laws of branding holds that you ought to pick proper nouns over common or category names for brands. Amazon.com has more potential than books.com.

Brand names though, need not mean anything. Kodak, famously, is a completely made-up word, chosen for the pure memetic stability of the bookend k’s. Social objects on the other hand, must exhibit the property of indirect salience. Brands are not social objects (though products can be), but can make social objects more powerful by making them more unique (as in the case of the iPod). Is Windows 7 a social object? Read on to find out. That’s one of my mini case-studies.

Ritual objects gave me the most difficulty. Some secularized religious objects with layers of ironic meaning (such as Christmas trees) are clearly social objects because they obey the “richness” attribute. They suggest many associations (Santa, snow, caroling) and spark many behaviors (gift wrapping, singing, “Hannukah bush” jokes and Cartman’s songs on South Park). Unreconstructed ritual objects though, tend to strongly constrain behavior. You must behave in certain constrained (and therefore contained) ways around them, or trouble finds you. The Muhammad cartoons incidents in Danish newspapers illustrated that.

The First Law of Social Objects

Let’s use the elements of the ideas we’ve assembled so far to do a bit of inference.

What does digitization of culture do to social objects? “It digitizes them” is the wrong answer. That’s a minor effect (yes, YouTube videos join fireplaces, but that’s minor).

Here’s what happens. Our social graph neighborhoods grow ever bigger and more distended and unpredictable, to the point where none of the usual social container, such as village or town walls, can contain them. Even cities and countries cannot contain them. The only meaningful container is the surface of the planet. Hence, globalization.

But this does not mean that a “global village” is formed in any meaningful sense. McLuhan used that term in a special sense (“retribalization”). What happens is that all meaningful social objects become unbound, because their tightest container – the world – is much larger than their fields. But their fields do not include the entire planet (except very rarely, as in the case of the images of 9/11).

So the first law is this:

When communities digitize, social objects replace walls.

I mean replace them socially. Walls cease to be meaningful social containers, but watershed lines between overlapping social object fields still matter. Even in those ancient cities where people still live inside old walled parts, the kids inside are busy texting to a social graph of mobile friends that seeps effortlessly through those walls. All traditional containment has been breached. The world becomes a massive tangled web of virtually local, geographically global, social-object fields of various sizes. Some call that virtual, cultural, balkanization.

Applications

Let’s do some applications. Mini case-studies.

Windows 7 Launch Parties

By now, it has become clear that the Windows 7 launch party idea has been an embarrassment. (I can’t find that post with a Googler making fun of the kit he/she received — help?).

What happened? Windows 7 is a brand, has impoverished associations of the wrong sort, and while salient, is directly salient. That doesn’t work. One association overwhelms everything else.

A pity, since Hugh MacLeod is a Microsoft sympathizer and actually came up with a great Microsoft social object: the blue monster. “Blue Monster” parties would have worked much better.

Facebook Networks and Twitter Lists

Facebook shut down regional networks, while Twitter launched lists. One Web behemoth was admitting the increasing irrelevance of the most powerful less-than-global contained imagined communities we’ve seen, while another was riding the increased relevance of leaky networks that respect no social container boundaries at all. I haven’t created any Twitter lists myself, but none of the lists I’ve been put on seem to map neatly to any traditional social container. Nobody has put me on lists named “India” or “University of Michigan” or “Xerox.”

This does not mean though, that geography is irrelevant. As communities fragment and “default” containers (both the workplace and the neighborhood are weakening as containers) vanish, other geographically anchored communities, built within object  fields of things like football team mascots or hourglasses, are taking shape. Face-to-face contact is still central to social interaction, and therefore geography matters in the design of social objects. Which brings me to meetup.com.

1000 Words a Day

The writing meetup group I mentioned before is an example of a small social engineering experiment of my own. I was unhappy with the writers groups I found in my area because (to use my new vocabulary) they were container-based, not object-based. I found that I was having a mixed reaction to the member lists of existing groups, and unhappy with the typical focus on critiquing and mutual support. It was all (forgive me) a little too talking-stick for me.

So, with my usual sociopathic approach, I started my own. After some thought, I decided the social object would be the Microsoft Word wordcount box, that one thing all serious writers, who are actually writing rather than making excuses, identify with. “1000 Words a Day” seemed like a nice rhetorical/realistic objective statement, one that would weed out self-absorbed whiners who wanted to talk more about writing than actually write.

After a few sessions, it became clear that things weren’t quite right. The “1000 Words a Day” word-count social object was lending too procedural an atmosphere to the sessions, and was also not indirect enough or tangible enough. I wanted to soften and defocus things.

And then inspiration hit. I had been meaning to buy a stopwatch to time us anyway, but had been worried by (with hindsight) the fact that it would make the proceedings even more left-brained. On the other hand, I did want to preserve the sensitivity to the idea of time passing, and the sense of urgency required to get people to write.

Voila: I added the hourglass. A poetic and philosophical, but still utilitarian, symbol of what we wanted to do. It worked, and nicely complemented the wordcount motif. The members liked it. I take it to every meeting now. Okay, it is a little talking-stick-ish.

Knitting Groups

Finally, let’s do the title case study. I first noticed the knitting trend almost 6 years ago. Hip-looking young girls were sitting around engaged in that fuddy-duddy grandmotherly occupation, knitting.

I was finishing my PhD at the time, and just starting to feel truly disconnected from high schoolers and undergraduates. Fortunately, a hip young fresh(wo)man acquaintance explained that it was a new trend among alternate kids.

Those alternate kids are now grown up. These days, I frequently see a regular group of young, polished and sophisticated looking early-career knitters at my local Starbucks. Some are in professional looking suits. Others are in casual clothes.

I’ve heard very local, phenomenological explanations of what’s going on from women. It all has to do, apparently, with post-feminist bonding among women. Kinda like the Roller Derby, which inspired that recent Ellen Page/Drew Barrymore movie, Whip It (social object: the roller derby event)

That explanation doesn’t interest me. My question is why knitting? Why not weaving, pottery or a million other things? Here’s why:

Knitting is a calm, low-tempo activity that can occur in public spaces. It is not demanding enough that conversation must stop, but not easy enough to allow unrestrained and excited talk. In its expected context, it signals traditional gender roles. In its hip context though, it is neither a non sequitur, nor a deeply significant reconstructed act. Since you will see young women dressed mostly in somewhat alternate clothes, or professional clothes,  knitting,  you have to read the whole scene, not just the knitting. The scene suggests both being comfortable in the mainstream and apart from it, maintaining a distinct identity. But since knitting is normally a traditional-gender-roles social object, it filters out the old-school feminists and neo-hippies. But it continues to effectively exclude men.

Contrast that with, for example, weaving (a common artsy occupation among older New Agey liberal women). Ostensibly a similar activity, but a vastly different social object.

A Social Object for Social Object-ers

I couldn’t resist. The Klein bottle is a great candidate social object for people interested in social objects. It is has no inside or outside. You can interpret that figuratively as an object that contains the whole universe. You can buy neat glass Klein bottles from Acme. If someone wants to start a “Klein Bottle Club” meetup group in the DC area, I’ll join, and donate a bottle.

Remember Kramer’s coffee-table book about coffee tables, that could become a coffee table (fold-out legs on the back cover?).

Where to Now?

I seem to keep getting hooked by these very rich veins of exploration that first excite, and then dismay me. You find one promising attack, and suddenly you are up nights thinking about how the angle sheds new lights on all sorts of questions. So my usual teaser: if you are interested in fuller analyses, post comments with your thoughts and letting me know if you want me to continue. Some potential follow-up topics are: how social objects work with introversion, how social objects undermine the idea of global citizenship, digital social objects, social objects in marketing…

One topic that I’ve had a good deal of success analyzing this way is systems and processes in decision-making. Rather than look at them as documented, codified, embedded and rational workflows, I view systems and processes as carefully designed patterns of overlapping social-object fields and containers. I devote a full chapter to this view in my upcoming book, Tempo. It was working on that chapter that inspired this article.  Think of it as whole-brained, emotionally intelligent system-and-process design.

And here is a link to the map view of the Social Objects  trail I’ve been referencing throughout this post. I am starting to really get the hang of using trails as a medium of writing. If you can think of other great examples of social objects, post ‘em here and I’ll add them to the trail.