Several recent discussions on a variety of unrelated topics with different people have gotten me thinking about two different attitudes towards dialectical processes. They are generalized versions of the professional attitudes  required of lawyers and judges, so I’ll refer to them as lawyer mind and judge mind. 

In the specialized context of the law, the dialectical process is structurally constrained and the required attitudes are  codified and legally mandated to a certain extent.  Lawyers must act as though they were operating from a lawyer-mindset, even if internally they are operating with a judge-mind. And vice-versa. Outside of the law, the distinction acquires more philosophical overtones.

I want to start with the law, but get to a broader philosophical, psychological and political distinction that applies to all of us in all contexts.

The Two Minds in Law

The lawyer mind allows you  to make up the best possible defense or prosecution strategy with the available evidence. Within limits, even if the defense lawyer is convinced his client is guilty, s/he is duty-bound to make the best possible case and is not required to share evidence that incriminates the defendant or weakens the case.  I asked several questions about this sort of thing on Quora and got some very interesting answers from lawyers. If you are a lawyer or judge and have opinions on these basic questions, you may want to add them as answers to the questions rather than as comments here.

The legal system is designed so that lawyers are under an ethical and legal obligation to try and win, rather than get at the “truth” in any sense. So a defense lawyer with a flimsy case, who is convinced of his client’s guilt, but who wins anyway because the prosecution is incompetent, is doing his job. S/he should not pull his/her punches.

What’s more, there is a philosophy behind the attitude. It is not letter over spirit. It is letter in service of the spirit. If things are working well, the lawyer should not suffer agonies to see justice not being served in the specific case, but find solace in the fact of the dialectic being vital and evolving as it should.

The lawyer, by pulling out all stops for a legal win, regardless of the merits of the case, is philosophically trusting the search for “truth” to the dialectic itself, and where the dialectic fails in a particular instance, s/he (I expect) views it as necessary inefficiency in the interests of the longer-term evolution of the legal system. It’s the difference between “not in my job description” small-mindedness and “trusting the system” awareness of one’s own role and its limitations.

The judge’s nominal role is to act as a steward of the dialectic itself and make sure it is as fair as can be at any given time, without attempting to push its limits outside of certain codified mechanisms. The judge is charged with explicitly driving towards the “truth” in the particular case, and also improving the system’s potential — it’s dialectical vitality — so that it discovers the truth better in the future (hence the importance of writing judgments with an eye on the evolution of case law, which is supposed to be a run a few steps ahead of legislation as a vanguard, and discover new areas that require legislative attention).

When Does This Work?

Now, if you think about it, this scheme of things works well when the system is actually getting wiser and smarter over time. If the system is getting dumber and more subverted over time, it becomes harder and harder for either the lawyer or the judge to morally justify their participation in and perpetuation of the system (assuming they care about such things).

A challenge for a judge might be, for instance, an increasing influence of money in the system, with public defenders getting worse over time, and rich people being able to buy better and better lawyers over time. If this is happening, the whole dialectic is falling apart, and trust in the system erodes. Dialectical vitality drains away and the only way to operate within the system is to become good at gaming it without any thought to larger issues. This is the purely predatory vulture attitude. If a legal system is full of vulture-lawyers and vulture-judges, it is a carcass.

A moral challenge for a lawyer might be, for instance, deciding whether or not to use race to his/her advantage in the jury selection process, effectively using legal processes to get racial discrimination working in his client’s favor. Should the lawyer use such tactics, morally speaking? It depends on whether the dialectic is slowly evolving towards managing race more thoughtfully or whether it is making racial polarization and discrimination worse.

This constant presence of the process itself in peripheral vision means that both lawyers and judges must have attitudes towards both the specific case and about the legal system in general. So an activist judge, for instance, might be judge-minded with respect to the case, but lawyer-minded with respect to the dialectic (i.e., being visibly partisan in their philosophy about if and how the system should evolve, and either being energetic or conservative in setting new precedents). You could call such a person a judge-lawyer.

A lawyer who writes legal thrillers on the side, with a dispassionate, apolitical eye on process evolution, might be called a lawyer-judge. A lawyer with political ambitions might be a lawyer-lawyer. I can’t think of a good archetype label for judge-judge, but I can imagine the type: an apolitical judge who is fair in individual cases and doesn’t try too hard to set precedents, but does so when necessary.

The x-(x’)-X-(X’) Template

Because of the existence of an evolving dialectic framing things, you really you have four possible types of legal professionals: lawyer-lawyers, judge-judges, lawyer-judges and judge-lawyers, where the first attitude is the (legally mandated and formal-role based) attitude towards a specific case, and the second is the (unregulated) political attitude towards the dialectic.

When the system is getting better all the time, all four roles are justifiable. But when it is gradually worsening beyond the point of no return, none of them is.  When things head permanently south, a mismatch between held and demonstrated beliefs is a case of bad faith. Since all hope for reform is lost the only rational responses are to abandon the system or be corrupt within it.

To get at the varieties of bad faith possible in a collapsing dialectic, you need to distinguish between held and demonstrated beliefs at both case and dialectic levels to identify the specific pattern.

So you might have constructs like lawyer-(judge)-lawyer-(lawyer).  This allows you to slice and dice various moral positions in a very fine-grained way. For example, I think a legalist in the sense that the term has been used in history, is somebody who adopts a lawyer-like role in a specific case within a dialectic that’s decaying and losing vitality, while knowing full well that it is decaying. Legalists help perpetuate a dying dialectic. You could represent this as lawyer-(judge)-judge-(lawyer).  I’ll let you parse that.

This is getting too meta even for me, so I’ll leave it to people who are better at abstractions to make sense of the possibilities here. I’ll just leave it at the abstract template expression I’ve made up: x-(x’)-X-(X’).

The special case of the law illuminates a broader divide in any sort of dialectical process. Some are full of judge-mind types. Others are full of lawyer-mind types.

The net behavior of a dialectic depends not just on the type of people within it, but on its boundary conditions: at the highest level of appeal, do judge-minds rule or lawyer-minds?

Within the judiciary, even though there are more lawyer minds, the boundary conditions are at the Supreme Court, where judge minds rule. So the dialectic overall is judge-minded due to the nature of its highest appeal process.

In other dialectics, things are different because the boundary conditions are different.

Governance Dialectics

The watershed intellectual difference that separates conservative (more lawyer-like) and liberal discourses (more judge-like) around a particular contentious subject is framed by the boundary conditions of the governance dialectic itself.

Politics exists within the dialectic that in principle subsumes all others: the governance dialectic. “In principle” because if the governance dialectic loses vitality, the subsumed dialectics can devour their parents.

You could argue that in a democracy where the legislative branch has the ability, in principle, to amend the constitution arbitrarily, the overall governance dialectic is one where the lawyer mind is the ultimate source of authority, since the top body is a bunch of formally lawyer-mind types. There are no judge-mind types with any real power, especially in parliamentary democracies. Nominally judicial roles like the Speaker are mostly procedural rather than substantive.

The theory of an independent judiciary does not in practice give judge-mind people equal authority. The check-and-balance powers of the judiciary are based on seeking to make the law more internally consistent rather than improving its intentions or governing values. Of course, if the legislative arm is slow in keeping up with the landscape being carved out by case law, the judiciary gains more de facto power. That’s a subsumed dialectic devouring its parent.

So in a democracy, lawyer-minds are structurally advantaged, since the most powerful institution is set up for lawyer minds. Bipartisanship (judge minds operating in a legislature) takes a special effort to go beyond the structural default through an act of imagination.

Among the other institutions in a free-market democracy, theoretically the judiciary, executive and free press are nominally judge-minded at their boundaries, while the market is lawyer-minded (more on that in a bit). So there is structural lawyer-mind bias in the top-level institutions (the legislature and the market) and a structural judge-mind bias in the secondary institutions (the judiciary, the press and the executive branch).

Traditional Imperial China was the opposite. The legal system ultimately derived its authority from a judge-mind figure, the Emperor. The lawyers were second-class citizens.

Other Dialectics

The notion of “free press” is currently being radically transformed due to the fundamental tension between journalism and blogging.

Journalism, at least nominally, is driven by a judge-mind dialectic. Journalists nominally aspire to a fair-and-balanced (without the Fox News scare quotes) role in society.

Blogging is driven by a lawyer-mind dialectic. Bloggers trust that the “truth will out” in some larger sense, and feel under no moral obligation to present or even see all sides of an issue. If the opposed side has no credible people, well, tough luck. The truth will just take a little longer to out. This gradual transformation of dialectical boundary conditions has been particularly clear in the various run-ins between Michael Arrington and newspapers like the Washington Post. This too is a case of a subsumed dialectic devouring its parent, since the government basically has no idea what its role in the new media world should be.

Science is another important dialectic. I won’t attempt to analyze it though, since it exists in a feedback loop with the rest of the universe, and is too complicated to treat here. Religion used to be dialectical in nature, but isn’t any more. But science is unimportant socially because it is very fragile, and in a world that is socially messy, it is easily killed. It never rules primarily because it takes a certain minimum amount of talent to participate in the scientific dialectic, which makes it similar to a minority dialectic.

Religion used to be a real dialectic. Now it is mostly theater in service of political dialectics.

Capitalism is another dialectic with the capacity to devour governance, just like the judiciary. But it is lawyer-like, not judge-like. The idea of a “fiduciary duty” to maximize shareholder wealth in the US is a lawyer-like duty towards society. The trend towards “social” businesses (B-corporations in the US) is an attempt to invent companies with more judge-like duties towards society. For the former to work, the market has to be closer to truly competitive, and getting better all the time. The invisible hand must be guided by an invisible and emergent judicial mind.

In an environment where pure competition has been greatly subverted, it is hard to justify this “fiduciary duty.” The rise of B-corporation philosophy, indicates a failure in the governance dialectic, since emergent judge-mind attitudes that should exist at the legislative level are being devolved to the corporate level.

In the US, the legislature has abdicated the spirit, if not the letter, of its responsibilities. Fiduciary duty may be a terrible idea, but the better solution would be to shift to a different, but still lawyer-mind model. This is because the market has a far lower capacity to manifest an emergent judge-mind. Since it is the governance dialectic that controls the nature and future of money, the principal coordination mechanism for the market, the market is ultimately subservient in principle, just like the judiciary.

Since the top-level emergent judge-mind requires a culture of bipartisan legislative imagination to exist, a legislative branch that cannot define imaginative visions on occasion enables a takeover by the structurally advantaged lawyer minds that comprise it, which leads to polarization and a power vacuum, which in turn leads to the devouring by nominally subsumed dialectics.

This is not an accident. By its very nature, you cannot structurally advantage judge-minds at the ultimate boundary of a social system. If you do, you are essentially legitimizing a sort of divine authority. The top level has to be lawyer-minds arguing by default, with an occasional lawyer gaining enough trust across the board to temporarily play judge.

Societies fail when their governance processes fail to demonstrate enough imagination for sufficiently long periods. We are living through such a period in the US today, as well as in many other parts of the world. Governance processes across the world have lost their vitality and there is a lot of devouring by dialectics it is supposed to subsume.

In the past during periods of such failure, violent adjustments have occurred. War is after all, the social dialectic of last resort.  Both world wars and the US Civil War represented such adjustments. In each case, the governance dialectic was revitalized, but at enormous cost in the short term.

Empathy and Passion

When you approach all reality with an intrinsic lawyer mind, you fundamentally believe that no matter how powerful your perspective-shifting abilities, you cannot adopt all relevant points of view. Not even all human points of view. With a judge-mind by contrast, your starting assumption is that you will eventually be able to appreciate all points of view in play. It is a somewhat arrogantly visionary perspective in that sense, and requires exhibition of a sufficient imagination to justify itself.

With a lawyer-mind for instance, if you are white, you don’t presume to understand the black point of view. With a judge-mind, you assume you can. Your emotions can also be lawyer-like (polarized passion) or judge-like (dispassionate).

If you are aware of, and unconflicted about, your role in a given dialectic, you don’t try to either suppress or amplify your emotions. You try to be mindful about how they influence your intellectual processes and control that influence if you think it is counter-productive. Up to a point, passion improves a lawyer mind and lack of passion improves a judge mind. Too much passion, and a lawyer-mind becomes emotionally compromised. Too little passion and a judge mind becomes apathetic. Both pathologies lead to procedural mistakes.

Passion cannot be conjured up out of nothing, nor can it be created or destroyed independently of intellectual reactions. So if you need more or less passion for your role, you have to either change your role via a true intellectual shift, or borrow or lend passion. This requires empathy.

Depending on whether the passion is on your side or the opposite side, empathy can make you more lawyer-minded or more judge-minded. Empathy for a friend makes you more lawyer-like. Empathy for a rival makes you more judge-like. This is how dialectics get more or less polarized. A dialectic with vitality can swing across this range more easily. One that lacks vitality gets locked into a preferred state.

So there is a sort of law of conservation of passion in a given situation, with passions of different polarities canceling out via cross-divide empathy, or reinforcing via same-side empathy.

There is a certain irreversability and asymmetry though. Judge-minds being fundamentally dispassionate cannot absorb passion and become lawyer-like as easily as lawyer-minds can absorb opposed passions and become more judge-like. This means judge-minds are more stable than lawyer-minds. To lower polarization, all the minds in a dialectic must mix more and let passions slosh and cancel out somewhat via empathy. This means breaking down boundaries and creating more human-to-human contact. To preserve or increase polarization on the other hand, artificial barriers must be created and maintained. Or you need a situation where material dialectics, like war and natural calamities, happen to be highly active.

This is fundamentally why the labels conservative and progressive mean what they do in politics.  This is also why conservatives are typically better organized institutionally. They have walls to maintain to prevent contamination of their lawyer minds.

And finally, this is also why the governance dialectic is structurally set up to advantage lawyer-minds at the highest levels: they need the structure more. It is up to judge-minds to transcend existing structures and imagine more structure into existence.

Knowing Your Place

With a lawyer mind in improving times, you conclude that your job is merely to do your absolute best with the perspectives you can access directly or via empathy, and trust larger processes to head in sane directions.

The lawyer mind is therefore an open system view that is more robust to unknown-unknowns. It trusts things it does not actually comprehend. It is intellectually conservative in that it knowingly limits itself. The judge mind is a closed system view that is less robust to unknown unknowns. It is intellectual ambitious in that it presumes to adopt a see-all/know-all stance. It does not trust what it cannot comprehend and is limited by what it can imagine.

Paradoxically, what makes a judge-mind closed is its capacity for imagination, while a lawyer-mind is open by virtue of its lack of imagination.  The ability to adopt many conflicting perspectives dispassionately fuels imaginative synthesis, but this synthesis then imprisons the judge mind. The reverse paradox holds for lawyer minds.

These paradoxes suggest that each type of mind contains the seed of the other, yin-yang style. I’ll leave you to figure out how. The fundamental delusion of a frozen judge-mind is the belief that this yin-yang state can exist in one mind all the time. The fundamental delusion of a frozen lawyer-mind is the belief that it never can.

In the Myers-Briggs system, where J(udging) and P(erceiving) represent what I’ve been calling the lawyer and judge mindsets respectively. Ironic that the labels are somewhat reversed.

Psychologically, I am a P (a fairly strong INTP), but intellectually, over the years I’ve become increasingly lawyer-minded rather than judge-minded. Perhaps it is the effect of blogging.  Perhaps it is a growing sense of the limits of my own abilities.

In terms of more artistic archetypes, the fox and hedgehog reflect lawyer and judge minds.

A Bill Gates Roy Amara quote I encountered last week reminds me strongly of compound interest.

“We always overestimate the change that will occur in the next two years and underestimate the change that will occur in the next ten.”

I hadn’t heard this line before, but based on anecdotal evidence, I think s Amara was right to zeroth order, and it is a very smart comment. The question is why this happens. I think the answer is that we are naturally wired for arithmetic, but exponential thinking is unnatural.  But I haven’t quite worked it out yet. We probably use some sort of linear prediction that first over-estimates and then under-estimates the underlying exponential process, but where does that linear prediction come from?

Anyone want to take a crack at an explanation? I could be wrong. Compound interest/exponential thinking might have nothing to do with it.

***

When I write, I generally start with some sort of interesting motif, like the Gates quote, that catches my eye, which I then proceed to attempt to unravel. Sometimes it turns out there’s nothing there, and sometimes a trivial starting point can fuel several thousand words of exploration.

I call this the “just add attention” model of writing.  It’s like just-add-water concentrates. A rich motif will yield a large volume of mind fuel if you just dissolve it in a few hours of informed attention.

The previous nugget is an example. If I were to let it simmer for a few days and then sat down to do something with the Gates quote, I would probably be able to spin a 4000-word post from it.  I figured I’d let you guys take a crack at this one.

My hit rate has been steadily improving. Nowadays, when I suspect that something will sustain exploration to such and such a depth, I am almost always right.

I prefer the word motif to words like pattern or clue, because it is more general. A motif merely invites attention. By contrast, a pattern attracts a specific kind of analytical attack, and a clue sets up a specific kind of dissonance.

***

The nature of just-add-attention writing explains why it is hard for me to write short posts. If I wrote short posts, they’d just be too-clever questions with no answers, or worse, cryptic motifs offered with no explanation.

You cannot really compress just-add-attention writing. You can only dehydrate it back into a concentrate. Just-add-attention writing has a generative structure but no clear extensive structure. It is like a tree rather than a human skeleton.

By this I mean that you can take the concentrate — the motif — and repeatedly apply a particular generative process to it to get to what you an extensive form. But this extensive form has no clear structure at the extensive level. At best, it has some sort of fractal structure. A human skeleton is a spine with four limbs, a rib cage and a skull attached. A tree is just repeated tree-iness.

But I hesitate to plunge forward and call all generative-extensive forms fractal, as you might be tempted to do. Fractal structures have more going on.

***

Just-add-attention writing is partially described well by Paul Graham’s essay about writing essays, which somebody pointed out to me after I posted my dense writing piece a few weeks back. But I don’t think it is the same as the Graham model. I think the Graham model  involves more conscious guidance from a separate idea about the aesthetics of writing, sort of like bonsai.

Just-add-attention writing is driven by its own aesthetic. This can lead to unpredictable results, but you get a more uncensored sense of whether an idea is actually beautiful.

Dense writing is related to just-add-attention in a very simple way: making something dense is a matter of partially dehydrating an extensive form again, or stopping short of full hydration in the first place. Along with pruning of bits that are either hard to dilute or have been irreversibly over-diluted.

Why would you want to do that? Because just-add-attention writing can sort of sprawl untidily all over the place. Partially dehydrating it again makes it more readable, at the cost of making it more cryptic.

This add-attention/dehydrate again process can be iterated with some care and selectivity to create interesting artistic effects. It reminds me of a one-word answer Xianhang Zhang posted on Quora to the question, “how do you chop broccoli?” Answer: “recursively.”

Regular writing can be chopped up like a potato. Just-add-attention writing must be chopped up like a broccoli. It is more time consuming. That’s why I cannot do what some people innocently suggest, simply serializing my longer pieces as a sequence of arbitrarily delineated parts. I have never successfully chopped up a long piece into two shorter pieces. At best, I have been able to chop off a straggling and unfinished tail end into another draft and then work that separately.

***

Not all generative processes lack extensive structure. The human skeleton is after all, also the product of a generative process (ontogeny). To take a simpler example, the multiplication table for 9 is defined by a generative rule (9 times n), but also has an extensive structure:

09
18
27
36
45
54
63
72
81
90

In case you didn’t learn this trick in grade school, the extensive structure is that you can generate this table by writing the numerals 0-9 twice in adjacent columns, in ascending and descending order.

If you wanted to blog the multiplication table for 9, and had to keep it to one line. You could use either:

• The nine times table is generated by multipling 1, 2,…, n  by 9, or
• Write down 0-9 in ascending order and then in descending order in the next column

Both are good compressions, though the second is more limited. But this is rare. In general a sufficiently complex generative process will produce an extensive-form output that cannot then be compressed by any means other than rewinding the process itself.

***

Just-add-attention writing is easy for those who can do it, but not everybody can do it. More to the point, of the people who can do it, a significant majority seem to find it boring to do. It feels a little bit like folding laundry. It is either a chore, or a relaxing experience.

What sort of people can do it?

On the nature front, I believe you need a certain innate capacity for free association. Some people cannot free associate at all. Others free associate wildly and end up with noise. The sweet spot is being able to free associate with a subconscious sense of the quality of each association moderating the chain reaction. You then weave a narrative through what you’ve generated. The higher the initial quality of the free association, the easier the narrative weaving becomes.

On the nurture front, this capacity for high-initial-quality free association cannot operate in a vacuum. It needs data. A lot of data, usually accumulated over a long period of time. What you take in needs to age and mature first into stable memories before free association can work well on this foundation. The layers have to settle. By my estimate, you have to read a lot for about 10 years before you are ready to do just-add-water writing effectively.

Unfortunately, initial conditions matter a lot in this process, because our n+1 reading choice tends to depend on choices n and n-1.  The reading path itself is guided by free association. But since item n isn’t usable for fertile free association until, say, you’ve read item n+385, there is a time lag. So your reading choices are driven by partly digested reading choices in the immediate past.

So if you make the wrong choices early on, your “fill the hopper” phase of about 10 years could go horribly wrong and fill your mind with crap. Then you get messed-up effects rather than interesting ones.

So there is a lot of luck involved initially, but the process becomes a lot more controlled as your memories age, adding inertia.

***

This idea that just-add-attention writing is driven by aged memories of around 10 years of reading suggests that the process works as follows.

When you recognize a motif as potentially interesting, it is your stored memories sort of getting excited about company. “Interesting” is a lot of existing ideas in your head clamoring to meet a new idea. That’s why you are sometimes captivated by an evocative motif but cannot say why. You won’t know until your old ideas have interviewed the new idea and hired it. Motif recognition is a screening interview conducted by the ideas already resident in your brain.

Or to put it in a less overwrought way, old ideas act as a filter for new ones. Badly tuned filters lead to too-open or too-closed brains. Well-tuned ones are open just the right amount, and in the right ways.

Recognition must be followed by pursuit. This is the tedious-to-some laundry-folding process of moderated free association. It is all the ideas in your head interrogating the new one and forming connections with it.

Finally, the test of whether something interesting has happened is whether you can extract a narrative out of the whole thing, once the interviewing dies down.

A good free association phase will both make and break connections. If your brain only makes connections, it will slowly freeze up because everything will be connected to everything else. This is as bad as nothing being connected, because you have no way to assess importance.

The pattern of broken and new connections (including those formed/broken in distant areas) guides your narrative spinning.

I rarely listen to music anymore. Strange anxieties and fears seem to flood into my head when I try. When I seek comfort in sound these days, I tend to seek out non-human ones. The sorts of soundscapes that result from technological and natural forces gradually inter-penetrating each other.

At the Mira Flores lock, the gateway into the Pacific Ocean at the southern end of the Panama Canal, you can listen to one such soundscape: the idling of your vessel’s engine, mixed with the flapping and screeching of seabirds. The draining of the lock causes fresh water to pour into salt water, killing a new batch of freshwater fish every 30-45 minutes. The seabirds circle, waiting for the buffet to open.

***

The seabirds have adapted to a world created by human forces better than humans themselves. They reconcile the technological and the natural without suffering agonies. They have smoothly reconstructed their identities without worrying about labels like  transhumanist or paleohumanist. There is neither futurist eagerness nor primitivist yearning to their adaptation. They do not strain impatiently to transform into dimly glimpsed future selves, nor do they strive with quixotic energy to return to an imagined original hunter-gatherer self.

If they do not strain to transform, they also do not strive for constancy.  No doctrine of seabirdism elevates current contingencies into eternal values that imprison. The seabirds feast without worry on the unexpected bounty of salinity-killed fish. They do not ponder whether it is “natural.”

They are thankfully unburdened by the sorts of limiting self-perceptions that we humans enshrine into the doctrine of humanism. I think of humanism as an overweening conception of being flash-frozen into a prescription during a brief window of time in early-modern Europe. A time when humans had just gotten comfortable transforming nature, but had not yet been themselves transformed enough by the consequences to understand what they were doing.

That naked label, humanism, unadorned by prefixes like paleo- or trans-, reveals our continued failure to center our sense of self within larger non-human realities. Our big social brains can invent elaborate anthropomorphic gods and social realities within which we gladly subsume ourselves, but struggle to manufacture a sense of belonging to anything that includes dead fish, seabirds, engineered canal locks and seawater.  Belonging has become an exclusively human idea to humans. We are still mean little inquisitors at the ongoing trial of Copernicus, resisting decentering realities that cannot be recursively reduced to the human. Man makes gods in his own image, blind to the non-human.

And so we distract ourselves with debates about the distinction between natural and artificial while ignoring the far more basic one between human and non-human.

Sometimes being a bird-brain helps. Last year, I decided I was going to be a barbarian.  I am going for bird-brain-barbarian this year.

***

So I rarely listen to music.

Music these days feels like a fog descending on my brain, obscuring visibility and tugging me gently inward into a cocoon of human belonging that promises warmth and security, but delivers an unsettling estrangement from non-human realities. Realities that are knocking with increasing urgency at the door of our species-identity.

Technology is more visual landscape than soundscape, but listening to pleasing human rhythms makes it harder to see technological ones. So even when there are no interesting soundscapes, I prefer silence. It is easy to miss frozen visual music when a soothing voice is piping fog into your brain through your ears. Perhaps all songs are lullabies.

Visible function lends lyricism to the legible but alien rhythms and melodies of technology-shaped landscapes. You can make out some of the words, like crane and unloading, but the song itself is generally impenetrable.

It is perhaps when the lyrics are at their most impenetrable that you can most pay attention to the song.  To understand is to explain. To explain is to explain away and turn your attention elsewhere. Obviousness of function can sometimes draw a veil across form, by encouraging a too-quick settling into a comforting instrumental view of technology.

Oscillating slowly back and forth across sections of the Panama Canal, you will see strange boats carrying dancing fountains. I missed what the tour guide said, so I have no idea what this is.

Perhaps a fire-fighting boat of some sort, or a dredging vessel. But I don’t need to know. One of the minor benefits of an engineering education is a confidence in your ability to fathom function if the need arises, leaving you free to appreciate pure form without a sense of anxiety. Looking at this water-dancer of a boat, I found myself wondering about the place of this beast on a larger spectrum.

On one end you find the Old Faithful geyser at Yellowstone National Park:

– 1 –

Naming and counting are the two most basic behaviors in our divided brains. Naming is the atomic act of association, recognition, contextualization and synthesis. Counting is the atomic act of separation, abstraction, arrangement and analysis. Each behavior contains the seed of the other.

To name a thing is to invite it to ensnare itself in your mind; to distill and compress the essence of a gestalt into a single evocative motif, from which it can be regenerated at will. Just add attention and stir.

Here are three very different American gestalts that I bet many of you will recognize without clicking: Babbitt, Bobbitt, Rabbit.

We name and count babies, products, species, theorems, countries, asteroids, ships, drugs, essays, wars, gods, dogs, foods, alcohols, pieces of legislation, judicial pronouncements, wars, subcultures, ocean currents and seasonal winds.

We try to name and number every little transient vortex, in William James’ blooming, buzzing confusion, that persists long enough for us to form a thought about it.

As with plans, so with names. Names are nothing; naming is everything. To name a thing is to truly know it. As Ursula Le Guin said, “for magic consists in this, the true naming of a thing.”

It is the process of naming that is important. The actual name that you settle on at the end is secondary.

– 2 –

Vanity and pragmatism wrestle for control of the act of naming.  We bend one ear towards history and the other towards posterity. We parse for unfortunate rhymes and garbled pronunciations. We attempt at once to situate and differentiate. We count syllables and look for domain names.

We walk around the name, viewing it as parent, lover, friend, bully, journalist, lexicographer and historian.  We embed it in imaginary headlines and taunting rhymes.

In Bali to name is to number. It is an unsatisfying synthesis that only works in limited contexts.

“The firstborn is “Wokalayan” (or Yan, for short), second is “Made,” third is “Nyoman” or Komang (Man or Mang for short), and fourth is “Ketut” (often elided to Tut).

I am not sure what happens if Wokalayan dies young. Does Made replace his older sibling and become the new Wokalayan?

In crypotgraphy, the first named-character in an example scenario is Alice. The second one is Bob. And so on down an alphabetic cast of characters. This is not the world of  interchangeable John and Jane Doe figures.  The order matters.

When birth order is more important individual personality, you get a social order in naming that inhabitants of individualistic modernity struggle to understand.

– 3 –

Counting is both ordinal and cardinal. It takes a while to appreciate the difference between one, two, three… and first, second, third.

To truly count is to know both processes intimately. In naming, ordinality has to do with succession and replacement. Cardinality has to do with interchangeability. You cannot master naming without mastering counting.

The ordinal, cardinal and nominal serve to situate and uniquely identify, but do not necessarily indicate the presence of something real. Hence the query: name, rank and number? 

There was once a substance with rank 0, number 0. It was named ether. It did not actually exist. Substances 1-1 through 1-4 though, earth, fire, water and wind, were real enough, and became the founding fathers and mothers of the modern discipline of chemistry.

It is in fact useful to think of naming an interrogative act that creates what it questions. Demand insistently enough to know the name, rank and number of a thing, and you will eventually find out. Even if your mind has to manufacture an answer.

When you understand both kinds of counting, you can count and name in both ways, without using actual numbers.

That gives you iMac, iPod, iPhone and iPad on the one hand, and Kodiak, Cheetah, Puma, Jaguar, Panther, Tiger, Leopard, Snow Leopard and Lion, on the other. I’ll leave you to guess why the first-born is a bear here, while the rest are cats. Don’t give up and click too soon.

Not many languages can efficiently express questions of ordinality. In English for instance, the question, what is your birth-order ordinality among your siblings? sounds downright weird, but I cannot find a simpler, grammatical way to express it.

It is much easier to ask the related cardinality question: how many siblings do you have? 

Curiously, the ordinal question is very easy to ask in my nominal native language of Kannada. It would translate to something like: How many-eth son are you of your father? If such constructs were allowed in English. At least that was the best I could come up with my father challenged me to translate the line as a kid.

It would be a useful construct to have in English. We could ask, What-ieth major version of Mac OS X is Lion? 

The naming practices in Bali and the Ursula Le Guin quote made me think of a rather clever idea for a short story about a culture where the young start out with ordinal names as in Bali, but are given true names if and when wise elders first spot the child in an act that expresses a unique individuality.

At this point, a coming-of-age naming ceremony is conducted, and the child is declared an adult with special privileges over the un-named. Rather complicated things happened to the hero’s name in the story, having to do with self-referential paradoxes. I’ve forgotten the plot, but I remember that at the time I had to diagram the events in the story.

I never wrote the story because coming up with names for the characters was too hard.

– 4 –

We name to liberate, and we name to imprison. We name to flatter, and we name to insult.  We name to own, and we name to be owned. We name to subsume, and have subsumed. We name to frame, and we name to reframe.

Google bought Urchin on Demand and turned it into Google Analytics. It bought Youtube and left the name alone.

The Left calls it Right to Choose. The Right calls it Right to Life. The debate itself is partly about naming: at what point does something deserve the name human?

The British and the French built a plane together and fought over the name. The French won. It became the Concorde rather than the Concord. 

Gandhi attempted to rename the untouchables Harijans. God’s people. They resented being patronized, and chose for themselves the name Dalit. The oppressed.

Priests weigh about the numerological significance of names and marketing mavens opine about syllable counts.

States step in with Procrustean templates to tax and conscript: last name, first name, middle initial. Under Spanish rule, the entire Philippines became a geographic-lexicographic state.

Philosophers ponder the metaphysics of naming and Greek scholars hunt for their linguistic roots.

As one anthropologist said (I have never managed to find the source), naming is never a culturally insignificant act.

– 5 –

To name is to appreciate the crucial distinction, due to urban theorist John Friedmann, between appreciative knowledge and manipulative knowledge. The one allows us to construct “satisfying images of the world.” The other allows us to gain mastery over it.

To either number or name is to both appreciate and manipulate.  To number is to appreciate timeless order; to name is to appreciate transformative chaos.

You number to extend and preserve. Archival is the ultimate act of numbering.

You name to create, destroy, fragment and churn. You name a product and launch it. You give a dog a bad name and hang it.

In a break with family tradition, I was not named after my paternal grandfather. The timeless sequence, …ABABAB… was broken.

– 6 –

Agent 007, James Bond, was named after an ornithologist.

In his numbered world, he is part of a greater order. A world of conversations between 007 and M, where technology comes from Q and even the secretary is a very countable Moneypenny. It is a timeless world where the M’s and Q’s are replaceable and 00s are both replaceable and interchangeable.

In his named world, first he situates, then he differentiates.

My name is Bond. James Bond.

A tough, hard and unusual name, for a tough, hard guy, who allows glimpses of a dark past to shine through the veneer of shaken-not-stirred cocktails and social polish. He blends in, but makes his presence felt. It is a name that is at once a trust and a threat. Bank of England to friends, gunboats to foes.

Is that a threat? No, it’s a promise. 

Commander Bond was once a naval reserve officer. It was in the maritime world that the line, “my name is my bond,” gained currency.

It is a name of narrative belonging. It situates the man strongly as British, but differentiates him not at all among Britishers. In Bond is the veiled threat of a still-potent dying empire. In James lies identification with, and anonymity within, that dying Empire.

Fleming once wrote to the real Bond’s wife:  ”It struck me that this brief, unromantic, Anglo-Saxon  and yet very masculine name was just what I needed, and so a second James Bond was born.”

– 7 –

The story of Windows is the story of a wild tree of apparently domesticated numbers seeking its way in the world, rather than an orderly parade of tamed wild cats.

1.0, 2.0, 3.0, NT, 3.1, 95, 98, ME, 2000, XP, Vista, 7, 8.

This is no accident. Microsoft, has always been a company that has sought its way in the existing world, rather than inviting the world into a fabricated universe of non sequiturs like Apple, Macintosh and Lisa.

The original portmanteau, MICRO-computer SOFT-ware, was a seeking of a place in a world defined by others. The micro-computer was ordinally a lesser thing than the mini-computer. Soft-ware was one of three wares: hard, soft and firm. An element in a set of cardinality three. It was a shy, retiring and polite name, that knew its place in the scheme of things.  

But the personality worked, and Microsoft quietly took over the universe it entered so politely. Windows was a literal-minded appropriation of the name of a key element of the desktop metaphor. Office seeks to belong in the workplace rather than redefine it. Internet Explorer remains the only browser that presumes to name itself after the thing it explores.

How a company names itself, its products and services, and its organizational parts, tells you a great deal about it.

To number something — implicitly or explicitly, cardinally or ordinally — is the first step in a grander project to order, tag and classify a part of reality; to prepare it for timeless forms of manipulation: replacement and interchange. To number is to subsume the particular within the general.

But to really name something in the sense of Le Guin, is to disrupt that project at every turn by discovering new magic that confounds the creeping logic of a rigidly ontological enterprise.

To really name is to find leaks as quickly as the number-givers find water-tight categories. To break connections thought secure and make new ones, previously considered impossible. To create difference — irreplaceability and non-interchangeability — as fast as numbering creates homogeneity.

This is perhaps why I still trust Microsoft more than I trust Apple. In the mess that is the the Windows sequence-numbering, I find reassurance.

– 8 –

To position is to number and name at the same time, and create something that is both a being and a becoming. Something rooted, that seeks to connect and get along, and something restless that seeks to get ahead and away.

To position a thing is to teach it to get ahead, get along, and get away. We project onto the memetic world of names, our own fundamental genetically-ordained proclivities. Evolutionary biology tells us that getting ahead and getting along are the basic drives that govern life for a social species. To this, as a species that invented individualism sometime in the 10th century AD, we must add getting away. The drive to become more than a rank and number. To become a name, even if the only available one, alpha, is taken.

The Microsoft version soup is Darwin manifest.

Getting ahead, getting along and getting away. Ordinal numbering, cardinal numbering and naming. Name, rank and number.

Perhaps it is naming and numbering that are fundamental, not biology.

To number well is to comprehend symmetries and anticipate as-yet-unnamed realities; holes in schemata, to be filled in the future. And so we name new elements before discovering them, imagine antimatter when we only know of matter. To categorize well is to create timeless order. Mendeleev’s bold leap advanced both chemistry and the art and science of naming.

To number poorly  is to squeeze, stuff and snip. To constrain reality to our fearful and limited conception of it.

To name well is to challenge and court numbers.

To name poorly is to kill or be killed by numbers.

Naming without numbering creates a chaotic unraveling. Numbering without naming creates orderly emptiness.

It takes discipline to couple the two forces together. And sometimes, numbers and names dance together beautifully to create magic, as when Murray Gell-Mann found inspiration in James Joyce’ line, three quarks for Muster Mark. 

– 9 –

To name is also to hide and cloak. To switch stories and manufacture realities.  This is the world of Don Draper. He dons a mask, and drapes new realities over old ones.  Starting with his own life.

And so Operation Infinite Justice became Operation Enduring Freedom.

I was supposed to be named after my grandfather, in keeping with the timeless …ABABAB… rhythm. I would have been Rama Rao. But then they broke with tradition.

My mother wanted to name me Rahul, but my grandmother objected: it is a name with deep significance for Buddhists — the name of the Buddha’s son.

Fortunately, in the (cardinal and ordinal) universe of a thousand names that is Vishnu — there is actually a long hymn known as the Vishnu Sahasranama, “Vishnu of the Thousand Names” — a close cousin of Rama was found.

And so I came into the world as Venkatesh. A break from tradition, but not quite a complete break.  Certainly not a defection to a competing tradition. That would have upset my grandmother.

I once wanted to name an algorithm I’d developed Mixing Bandits, since it used mechanisms inspired by bandit processes. I gave a draft of my paper to a distinguished professor in the field. He liked my work, but objected to the name. My allusive overloading of a precise term did not sit well with him. Mathematically, my algorithm was not related enough to bandit processes.

So this grandmother rejected the baby, refusing to absorb it into the family tradition. It wanders the world today as an illegitimate orphan of the noble clan that has disavowed it, under the clumsy and undistinguished name MixTeam scheduling.

– 10 –

In the genealogy of a single name you can trace entire grand narratives.

Once upon a time, there was a company in Rochester called Haloid. It made photographic paper and lived in the giant shadow of a company across town called Kodak.

Haloid wanted to grow up. So it acquired a technology called xerography: a name coined by a Greek scholar to situate the idea of dry writing within the illegible history of that long intellectual tradition within which the West seeks to situate everything it does.

Ironically, the technology was not the result of a long, gradually evolving tradition that can be traced back to the Greeks. Not only did the Greeks have nothing to do with it, as the biographer of the technology David Owen notes, “There was no one in Russia or France who was working on the same thing. The Chinese did not invent it in the 11th century BC.”

Xerography sprang almost fully-formed from the mind of one man, Chester Carlson. He systematically set about the project of inventing and patenting something truly new. He managed to do so by putting an obscure property of the element Selenium to a completely unexpected use.

So Haloid became Haloid Xerox, and eventually just Xerox. It is a powerful name. So powerful that it subsumed the name of the man who created it, Joe Wilson. During my time at Xerox, the Wilson Center for Research and Technology (WCRT) became the Xerox Research Center, Webster (XRCW). Across the world you will find XRCE (Europe), XRCC (Canada) and XRCI (India). To earn its right to a unique name within this orderly namespace, the sole rebel, PARC, had to unleash planet-disrupting forces.

Xerography eventually became electrophotography, in the hands of envious competitors who appeared after the trust-busters had done their work. The name that had gotten ahead and away now had to get along. My name is photography. Electro-photography. 

They still call it xerography at Xerox though.

– 11 –

And across town, Kodak slowly declined and began to die. There is irony here as well.

Photography does have a long history. The ancient Greeks did have something to do with it.  The ancient Chinese did know about pinhole cameras. The French did play a role.

But Kodak is one of those rare names that was born through an act of pure invention. George Eastman is quoted as saying about the letter k: ”it seems a strong, incisive sort of letter.”  Yes, incisive like a knife.

The story goes that Eastman and his mother created the name from an anagrams set. Wikipedia says about the process:

Eastman said that there were three principal concepts he used in creating the name: it should be short; one cannot mispronounce it, and it could not resemble anything or be associated with anything but Kodak.

The first two principles are still adhered to by marketers when possible. The last has been abandoned since the 1970s, when the positioning era began.

As with Wilson, the child soon eclipsed the father. Eastman Kodak became just Kodak to the rest of the world. In proving the soundness of his principles of memetic stability, Eastman ceded his own place in the history of naming to a greater name.

Haloid incidentally, is a reference to the binary halogen compounds of Silver used in photography. The word halogen was coined by Berzelius from the words hals (“sea” or “salt”) and gen (“come to be”). Coming to be of the sea. It may be the most perfect name, suggesting the being and becoming that is the essence of both naming and chemistry.

Jöns Jacob Berzelius is a founding father of chemistry in large part due to his prolific naming. He came up with protein as well. He was also responsible for naming Selenium. From the Greek Selene, for Moon.

It was no small achievement. Chemistry is a science of variety and difference. It deals in so many different thing that a narrowly taxonomic mind will fail to appreciate its broader patterns.

In declaring that “Physics is the only real science, all the rest are just stamp collecting,” Rutherford failed to appreciate chemistry the way Berzelius did. As an ongoing grand narrative with lesser and greater patterns.

Some deserving names like protein and others merely abstract, categorical formulas like CnH2n+2 and names that just fall short of cohering into semantic atoms, like completely saturated hydrocarbon.

– 12 –

Counting and naming are at once trivial and profound activities.

Toddlers learn to count starting with One, Two, Three…

Terence Tao has won a Fields Medal and lives numbers like nobody else alive today. And he is still basically learning to count. At levels you and I would consider magic, but it is counting nevertheless.

Toddlers learn to name, starting with  me, mama and dada.

Ursula Le Guin has won five Hugo and six Nebula awards, but is fundamentally still a name-giver.

Names are born of universes, be they small ones that contain only Kodak or large ones that contain all of Western civilization between alpha and omega.

It is very hard to make up universes. It is easier to borrow and disguise them, as Tolkien and Frank Herbert did.

And it is very hard to do so without accidentally causing collisions between large, old namespaces that might not like each other, as my mom found out with Rahul. 

Lazy novelists are laziest with names, and the work falls apart. When you have named every character in your novel perfectly, your novel is finished. Plot and character converge towards perfection as names do.

Names in turn create universes. Carnegie Hall, Carnegie Foundation, Carnegie-Mellon University. 

To name is to choose one universe to draw from and another to create. Rockefeller gave his name to few things. He preferred bland names like Standard Oil and The University of Chicago. 

And so it is that the Carnegie Universe is very visible, while the much larger Rockefeller Universe is more hidden from sight.

– 13 –

Rockefeller chose to create, and hide much of what he created. But you can go further. Beyond hiding lies un-naming. To un-name is to deny identity.

To un-name and un-number is to anonymize completely.

It is useful for the name-giver to ponder the complementary problem of un-naming. If to position is to name and number, to de-position is to un-name and un-number.

You must seek randomness to disrupt the timeless order imposed by numbering, disconnection to counter the narrative order created by naming. Like Dorian Taylor, you must seek cryptonyms.

Cryptonym itself is from the Greek words for “hidden” and “name.”

Randomness is hard.

To un-name is to fight the natural. Given enough time, even a set of cryptonyms will fail to arrest a cohering identity. To truly arrest a name, even changing the crytponym at a random frequency is not enough. The underlying cohering realities must be disrupted.

– 14 –

Names demand to be born, and hijack numbers if no worthy ones appear. And so we have 9-11 and Chapter 11. 

At other times, names strain to hang on to life, with no stories to tell. In the arid, random desert that is bingo, where numbers rule, names struggle.

Only to a Bingo player is 22 “two little ducks.”

Few numbers truly rise to the level of human meaning, and they are all small: 13, 42, 867-5309. 

The largest number in my life that is also a name with permanent narrative significance is 1174831686. 

When I was nine or ten, our local newspaper, The Telegraph, launched a club for kids in its Sunday edition, called the Wiz Biz Club. I signed up excitedly, to belong and to make new friends. That was my membership number.

I received a badge, some stickers and an ID card with that number.

So Venkatesh Rao  became 1174831686.  That cryptonym was probably the start of my struggle to own my name instead of being owned by it.

I am glad to report that despite it being an extremely common Indian name, I now own venkateshrao.com (it redirects to this site) and almost the entire first page of Google results. Vishnu can have the other 999 names, but I plan to pwn this one, at least for one lifetime.

– 15 –

We dimly recognize, even without the aid of mathematicians who study such things, that numbers win this decidedly unequal contest of appreciation and manipulation in the long-term.

In the beginning, we generously allowed our businesses, products and services to share the older namespaces of people and geographies. East India Company, Jardine-Mathieson, Carnegie Steel, Johnson & Johnson.

That strategy quickly exhausted itself, and so we energetically began manufacturing Xeroxes, Kodaks, Microsofts and Apples.

The first really-big-numbers company decided to name itself after a number, Google. Its home became an even bigger number, Googleplex. 

After Google, the Internet began throwing up naming needs faster than humans could manufacture them, and the orderly taxonomy unexpectedly imposed on the world by the Internet Domain Name system suddenly made life very difficult indeed.

So far, we’ve kept up by inventing quasi-algorithmic models: flickr, dopplr, e-widget, i-doodad.

But eventually naming as a way to understand and construct reality will fail.  Technology creates complexity that creeps inexorably towards the unnameable-but-significant.

When semantic genealogies in naming give way to syntactic and lexicographic genealogies, you are halfway to the world of pure numbers (there is a cute scene in Neal Stephenson’s Cryptonomicon, where members of an online group decide to abandon names and stick to purely numbering and ranking the world; the split occurs between those who seek cryptonyms and those who seek a fundamental order within which, for instance, Earth might be numbered 1).

The march that begins with Aachen and Aardvark cannot keep up with a universe that throws countable, but not-nameable, variety at us. We count on, long after we can no longer name.  And eventually we cannot count, either, and must stare at an unnameable, uncountable void and wonder — as some mathematicians do — whether it even exists, given how it eludes characterization.

Yet we persist with both naming and numbering, finding solace in imposing a partial lexicographic order on reality, even as the struggle gets harder.

– 16 –

I have not used the word brand even once in this post, until just now. Over the years,  I have lost confidence in the utility of the concept.

It is appropriate only for the cardinal-ordinal world of mass manufacturing, where everything has a rank and number, but very few things have real names. Most brands are McBrands. Billions upon billions have been served up by marketers and fond parents. Most represent no deeper reality than the first answer to the question, name, rank and number. 

It is not surprising. After all the very word originates in processes that evolved superficially distinguish the essentially interchangeable. In the world of cows, and pottery before that, to brand was to mark for identification and counting, and little else.

Brand is an abstraction that adds very little to the more fundamental concepts of naming and numbering, and the key derivative concept of positioning. In fact, it is distracting. The word makes it far too easy to lose yourself in abstractions. Naming and numbering keep you honest and focused on the gestalt you are trying to distill, with repeated tests. The story of these attempts is what we know as PR, and with each proposed naming and positioning test you can ask, do I understand this story yet?

Without such test-driven naming, branding is an exercise in waterfall marketing.

To the extent that it is a useful word at all, it describes a consequence rather than an action. Away from the concrete world of cows being tortured with red-hot irons, there is no actual action that you can call branding.

You name, number and position.  You then make up non-verbal correlates — colors and logos — that derive from these basic elements.

These are things you do.

Brand happens.

Consider this thought experiment: what if you were only allowed 2000 words with which to understand the world?

With these 2000 words, you’d have to do everything. You’d be allowed to occasionally retire some words in favor of others, or invent new words, but you’d have to stick to the budget.

Everything would have to be expressible within the budget: everyday conversations and deep conversations, shallow thoughts and  profound ones, reflections and expectations, scientific propositions and vocational instruction manuals, poetry and stories, emotions and facts.

How would you use your budget? Would you choose more nouns or verbs? How many friends would you elevate to a name-remembered status? How many stars and bird species would you name? Would you have more concrete words or more reified ones in your selection? How many of the most commonly used words would you select? Counting mathematical symbols as words, how many of those would you select? Would you mimic others’ selections or make up your own mind?

***

When I read old texts, I am struck by the density of the writing. Words used to be expensive. You had to make one word do many things.

That last sentence contains a simple example. I originally had convey many meanings in place of do many things. For some readers, the substitution will make no difference. To others, it will make a great deal of difference.

We talk of dense texts as being layered.  They lend themselves to re-reading from many perspectives over a long period of time. Even as late as the nineteenth century, we find that the average professional writer wrote with a density that rivals the densest writing today.  With the exception of scientific writing — best understood as a social-industrial process for increasing the density of words — every other kind of writing today has become less layered. Most writing admits one reading, if that.

Dense writing is not particularly difficult. Merely time-consuming. As the word layering suggests, it is something of a mechanical craft, and you become better with practice. Even mediocre writers in the past, working with starter material no denser than today’s typical Top 10 blog post, could sometimes achieve sublime results by putting in the time.

If the mediocre can become good by pursuing density, the good can become great. Robert Louis Stevenson famously wrote gripping action sequences without using adverbs and adjectives. His prose has a sparse elegance to it, but is nevertheless dense with meaning and drama. I once tried the exercise of avoiding adverbs and adjectives. I discovered that it is not about elimination. The main challenge is to make your nouns and verbs do more work.

***

In teaching and learning writing today, we focus on the isolated virtue of brevity. We do not think about density. Traditions of exegesis — the dilution, usually oral, of dense texts to the point where they are consumable by many — are confined to dead rather than living texts.

We have forgotten how to teach density. In fact, we’ve even forgotten how to think  about it. We confuse density with overwrought, baroque textures, with a hard-to-handle literary style that can easily turn into tasteless excess in unskilled hands.

The 2000-word thought experiment, if you try it, will likely force you to consider density of meaning as a selection factor. Some words, like schadenfreude, are intrinsically dense. Others, like love, are dense because they are highly adaptable.  Depending on context, they can do many things.

Density is a more fundamental variable than the length of a text. It is intrinsic to writing, like the density of a fluid; what is known in fluid dynamics as an intensive property. The length of an arbitrarily delineated piece of text on the other hand, is an extensive property, like the volume of a specified container of fluid.

Choosing words precisely and crafting dense sentences is important. Choosing small containers is not.

***

Writing used to be a form of making. I sometimes wonder what it would be like to have to carve your thoughts onto stone tablets.  One of these days I am going to try carving the first draft of a post in stone.

Writing on paper is also an expensive luxury. There was a time when writers made their own paper and ink. You had to write with temperamental things like quills. The practice of calligraphy was not a writerly affectation. It was a necessary skill in the days of temperamental media.

The scribe was more of an archivist than a writer. The other ancestor of the writer, the bard-sage, was both composer and performer. The average person did not read, but relied on the bard or priest to expand upon and perform the written, archived word. Particularly good performances would lead to revisions of the written texts.

When fountain pens and cheap factory-made paper made their appearance, writers were able to waste paper, and as a consequence, written words. In the history of thought, the invention of the ability to waste words was probably as important as the invention of the ability — famously noted by Alan Kay — to waste bits in the history of programming.

With cheap paper was born that iconic image of the twentieth century writer — a writer sitting alone in a room, crumpling up a piece of writing in frustration, and tossing it into an overflowing waste-paper basket. Unlike the sage-bard, enacting old texts and beta-testing new ones through public oral performances, or the scribe, committing tested, quality-controlled and expensive texts to stone, the modern, pre-Internet writer was a resource-rich creature of profligate excess. 

The very idea of a “waste paper” basket would have been unthinkable at one time.

***

It is difficult today to get a sense of how expensive writing used to be. I once watched a traditional temple scribe demonstrate the process of making the palm-leaf manuscripts that were used in India until Islam brought paper-making to the subcontinent. That probably happened a few centuries after the Abbasids defeated the Tang empire at the Battle of Talas in 751 AD, and extracted the secret of paper-making from Chinese prisoners of war.

Palm leaves are easily the worst writing technology ever invented by a major culture. They make leather, papyrus, paper and silk look like space-age media by comparison. A good deal that seems strange about India as an idea suddenly makes sense once you get that the civilization was being enacted through this ridiculous medium (and equally ridiculous ones like tree bark) until about 1000 AD. Imagine a modern civilization that had to keep its grand narrative going using only tweets, and you get some sense of what was going on.

Here’s how you make palm-leaf manuscripts. First you cut little index-card sized rectangles out of palm fronds and dry them flat. Then you carefully use a needle to scratch out the text — typically a few lines per leaf.  Then you make an ink out of ground charcoal, carefully rub it into the scratches, and swab away the excess. Finally, you carefully pierce a hole through the middle (not the edge, since the thing is brittle) and thread a piece of string through a sheaf of loosely-related leaves.

Congratulations, you have a book.

Since the sheaf is more unstable than individual leaves, you have to plan for graceful degradation. Expect individual leaves to be lost or damaged. Expect accidental shuffling and page numbering turning to garbage. Expect new leaves to be inserted, like viruses. Don’t expect multi-leaf stories to remain stable. Expect narrative trunks to sprout branches added by later authors.

The palm leaf manuscript was brittle and easily damaged, available in one unhelpful size, with a lifespan of perhaps a few decades on average (carefully preserved ones lasted around 150 years I believe). After that you had to make a copy if you wanted to keep the ideas alive. If you were rich or powerful, you could get stuff carved onto stone or copper plates by slaves. If not, your best bet was to go with palm leaves and hope that people would descend on your home to make copies.

***

When you look at old writing technology, poetry suddenly makes sense.

It is  modular content that comes in fixed-length chunks, with redundancy and error-correcting codes built in. It is designed to be transmitted and copied across time and space through unreliable and noisy channels, one stone tablet, palm leaf or piece of handmade paper at a time. The technology was still unreliable enough that the oral tradition remained the primary channel. Writing began as a medium for backups. Scribes were the first data warehousing experts. They did more than merely transcribe the spoken word. They compressed, corrected and encrypted as well, and periodically updated texts to reflect the extant state of the oral tradition.

That is why verses are so eminently quotable outside the context of poems. Poems are extensive oral containers of arbitrary length, in some cases delineated after the fact. Verses are standardized containers designed to carry intense, dense, archival-quality words around.

Today we view traditional verse epics as single works. The Illiad has about 9000 verses. The Mahabharata has about 24,000. It makes far more sense to talk about both as data-warehoused records of extremely long — in both time and words — convergent conversations. They are closer to Google’s index than to books.

For the ancients, texts had to be little metered packets. But as paper technology got cheaper and more reliable, poetry, like many other obsolete technologies before and after, turned into an art form. Critical function turned into dispensable style. Meter and rhyme ceased to be useful as error-correcting coding mechanisms and turned into free dimensions for artistic expression.

Soon, individual verses could be composed under the assumption of stable, longer embedding contexts. Extensive works could be delineated  a priori, during the composition of the parts. And the parts could be safely de-containerized. Rhyming verse could be abandoned in favor of blank verse, and eventually meter became entirely unnecessary. And we ended up with the bound book of prose.

Technologically, it was something of a backward step, like reverting to circuit-switched networks after having invented packet switching, or moving back from digital to analog technology. But it served an important purpose: allowing the individual writer to emerge. The book could belong to an individual author in a way a verse from an oral tradition could not.

***

Poetry gets it right: length is irrelevant. You can standardize and normalize it away using appropriate containerization. It is density that matters. Evolving your packet size and vocabulary over time helps you increase density over time.

My posts range between 2000-5000 words, and I post about once a week here on ribbonfarm. But there are many bloggers who post two or three 300-word posts a day, five days a week. They also log 2000-5000 words.

So I am not particularly prolific. I merely have a different packet size compared to other bloggers, optimized for a peculiar purpose: evolving an idiosyncratic vocabulary. It seems to take several thousand words to characterize a neologism like gollumize or posturetalk.  But once that is done, I can reuse it as a compact and dense piece of refactored perception.

You could say that what I am really trying to do on this blog is compose a speculative dictionary of dense words and phrases. Perhaps one day this blog will collapse under its own gravity into a single super-dense post written entirely with 2000 hyperlinked neologisms, like a neutron star.

Poetry — functional ancient poetry, the cultural TCP/IP of the world before around 1000 AD — is necessarily a social process, involving, at the very least, a sage-bard, a scribe, an audience and a patron. The oral culture refines, distills, tests, reworks, debates and judges. Iterative performance is a necessary component. When oral exegesis of an unstable verse dies down, and memorization and repetition validate the quality of the finished verse, the scribe breaks out his chisel.

The prose book can stand apart from broader social processes in radically individual ways. It can travel from writer to readers largely unaltered, setting up a hub-spoke pattern of conversational circuits.

***

I’ve occasionally described my blogging as a sort of performance art. But something about that self-description has been bothering me. I have now concluded that if the description applies at all, it applies to a different kind of blogger, not me.

The Web obscures the crucial and necessary distinction between oral and written cultures.  Some bloggers perform and talk. Others are scribes. I think I am a scribe, not a performer.

Yet, there is no easy correspondence between pre-Gutenberg bard-sages and scribes and today’s bloggers. In the intervening centuries, we have seen the rise and fall of the individualist writer, working alone, filling waste-paper baskets.

History does not rewind. It synthesizes. The blogosphere, I am convinced, synthesizes the collectivist pre-Gutenberg culture of sage-bard and scribes with the individualist post-Gutenberg culture of paper-crumpling waste-paper-basket fillers.

In the process of synthesis, virtual circuits must ride once more on top of a revitalized packet-switched network. The oral/written distinction must be replaced by a more basic one that is medium-agnostic, like the Internet itself.

***

According to legend, the sage Vyasa needed a scribe to write down the Mahabharata as he composed it. Ganesha accepted the challenge, but demanded that the sage compose as fast as he could write. Wary of the trickster god, Vyasa in turn set his own condition: Ganesha would have to understand every verse before writing it down. And so, the legend continues, they began, with Vyasa throwing curveball verses at Ganesha whenever he needed a break.

The figure of Vyasa the composer is best understood as a literary device to represent a personified oral tradition (that perhaps included a single real Vyasa or family of Vyasas).

But the legend gets at something interesting about the role of a scribe in a dominantly oral culture. A second-class citizen like a minute-taker or official record-keeper, the scribe must nevertheless synthesize and interpret an ongoing cacophony in order to produce something coherent to write down. When the spoken word is cheap and the written word is expensive, the scribe must add value. The oral tradition may be the default, but the written one is the court of final appeal in case of conflict among two authoritative individuals.

There is a brilliant passage in Yes, Prime Minister, where the Cabinet Secretary Humphery Appleby  helps the Prime Minister, Jim Hacker, cook the minutes of a cabinet meeting after the fact, to escape from an informal oral commitment. Appleby’s exposition of the principle of accepting the minutes as the de facto official memory gets to the heart of the Vyasa-Ganesha legend:

Sir Humphrey: “It is characteristic of all committee discussions and decisions that every member has a vivid recollection of them and that every member’s recollection of them differs violently from every other member’s recollection. Consequently, we accept the convention that the official decisions are those and only those which have been officially recorded in the minutes by the Officials, from which it emerges with an elegant inevitability that any decision which has been officially reached will have been officially recorded in the minutes by the Officials and any decision which is not recorded in the minutes is not been officially reached even if one or more members believe they can recollect it, so in this particular case, if the decision had been officially reached it would have been officially recorded in the minutes by the Officials. And it isn’t so it wasn’t.”

The key point here is that the scribe must do more than merely transcribe. He must interpret and synthesize. I suspect the Vyasa-Ganesha legend was invented by the first scribe paid to write down the hitherto-oral Mahabharata, to legitimize his own interpretative authority in capturing something coherent from a many-voiced tradition, with each voice claiming the authority of a mythical Vyasa.

***

So if the modern blogosphere is neither the collectivist, negotiated recording of a Grand Narrative, arrived at via a conversation between scribes and sage-bards, nor the culture of purely individual expression that reigned between Gutenberg and Tim Berners-Lee, what is it?

For blogging to be performance art, the performer must live an interesting life and do interesting things. For a while I thought I qualified, but then I reflected and was forced to admit that my dull daily routine does not qualify as raw material for performance art.

How about this: instead of a half-coherent oral tradition or the relatively coordinated doings of the British Cabinet, the blogosphere is primarily an uncoordinated theater of large-scale individual gonzo blogging. As culture is increasingly enacted by this theater of decentered gonzo blogging instead of traditions that enjoy received authority, minute-taking scribe bloggers must increasingly interpret what they are seeing.

The first human scribe who wore the mask of Ganesha could reasonably assume that there was a coherent trunk narrative with discriminating judgments required only at the periphery.  He would only be responsible for smoothing out the rough edges of an evolving oral consensus. Equally Humphrey Appleby could hope for a coherent emergent intentionality in the deliberations of the cabinet.

But the scribe-blogger cannot assume that there is anything coherent to be discovered in the gonzo blogging theater. At best he can attempt to collect and compress and hope that it does not all cancel out.

There is another difference. When words are literally expensive, as words carved in stone are, anything written has de facto authority, underwritten by the wealth that paid for the scribe. Scribes were usually establishment figures associated with courts, temples or monasteries, deriving their interpretative authority from more fundamental kinds of authority based on violence or wealth.

With derived authority comes supervision. The compensation for lost derived authority is the withdrawal of supervision.  The scribe-blogger is an unsupervised and unauthorized chronicler in a world of contending gonzos. Any authority he or she achieves is a function of the density and coherence of the interpretative perspective it offers on the gonzo-blogging theater.

***

I wish I could teach dense blogging. I am not sure how I am gradually acquiring this skill, but I am convinced it is not a difficult one to pick up. It requires no particular talent beyond a generic talent for writing and thinking clearly. It is merely time-consuming and somewhat tedious.

Sometimes I strive for higher density consciously, and at other times, dense prose flows out naturally after a gonzo-blogger memeplex has simmered for a while in my head. I rarely let non-dense writing out the door. You need gonzo-blogging credibility to successfully do Top 10 list posts. I can manufacture branded ideas, but lack the raw material needed to sustain a personal brand.

Writing teachers with a doctrinaire belief in brevity urge students to focus. They encourage selection and elimination in the service of explicit intentions.  The result is highly legible writing. Every word serves a singular function. Every paragraph contains one idea. Every piece of prose follows one sequence of thoughts. There is a beginning, a middle and an end. Like a city laid out by a High-Modernist architect, the result is anemic. The text takes a single prototypical reader to a predictable conclusion. In theory. More often, it loses the reader immediately, since no real reader is anything like the prototypical one assumed by (say) the writer of a press release.

An insistence on focus turns writing into a vocational trade rather than a liberal art.

Both gonzo blogging and scribe blogging lead you away from the writing teacher.

Striving for density, attempting to compress more into the same number of words, inevitably leads you away from the legibility prized by writing teachers. Ambiguity, suggestion and allusion become paramount. Coded references become necessary, to avoid burdening all readers with selection and filtration problems. Like Humpty-Dumpty, you are sometimes forced to enslave words and chain them to meanings that they were not born with.

***

Dense writing creates illegible slums of meaning. To the vocational writer, it looks discursive, messy and randomly exploratory.

But what the vocational writer mistakes for a lack of clear intention is actually a multiplicity of intentions, both conscious and unconscious.

Francine Prose, in Reading Like a Writer, remarked that beginning novelists obsess about voice, the question of who is speaking. She goes on to remark that the more important question is who is listening?

The failure to ask who is listening is peculiar to pre-Internet book writers. You cannot possibly fail that way as a blogger.

The modern extensive-prose, word-wasteful book represents the apogee of a certain kind of individualism. An individualism that writes itself into existence through self-expression unmodulated by in-process feedback, something only entire cultures could afford to do in the age of stone-carved words. For this kind of writer, the reader was a distant abstraction, easily forgotten.

A muse was an optional aid to the process rather than a necessary piece of cognitive equipment. At most modern, pre-blogging book writers wrote for a single archetypal reader.

For the blogger, a multiplicity of readerly intentions is a given. At the very least, you must constantly balance the needs of the new reader against the needs of the long-time reader. Every frequent commenter or email/IM correspondent becomes an unavoidable muse. This post for instance, was triggered by a particularly demanding muse who accused me, over IM, of having gotten lazy over the last few posts and neglecting this blog in favor of my more commercial, less-dense writing.

She was right. Mea culpa. Having to pay the rent is not a valid excuse for failing to rise to the challenge of a tricky balancing act.

Density is the natural consequence of trying to say many things to many distinct people over long periods of time without repeating yourself too much or sparking flame wars. The long-time reader gets impatient with repetition and demands compaction of old ideas into a shorthand that can be built upon. The newcomer demands a courteous, non-cryptic welcome. Active commenters demand a certain kind of room for their own expansion, elaboration and meaning construction.

The exegesis of living texts is not the respectful affair that it is around dead ones. If you blog, there will be blood.

***

In the days of 64k memories, programmers wrote code with as much care as ancient scribes carved out verses on precious pieces of rock, one expensive chisel-pounding rep at a time.

In the remarkably short space of 50 years, programming has evolved from rock-carving parsimony to paper-wasting profligacy.

Still living machine-coding gray eminences bemoan the verbosity and empty abstractions of the young. My one experience of writing raw machine code (some stepper-motor code, keyed directly into a controller board,  for a mechatronics class) was enlightening, but immediately convinced me to run away as fast as I could.

But why shouldn’t you waste bits or paper when you can, in service of clarity and accessibility? Why layer meaning upon meaning until you get to near-impenetrable opacity?

I think it is because the process of compression is actually the process of validation and comprehension.  When you ask repeatedly, who is listening, every answer generates a new set of conflicts. The more you resolve those conflicts before hitting Publish, the denser the writing. If you judge the release density right, you will produce a very generative piece of text that catalyzes further exploration rather than ugly flame wars.

Sometimes, I judge correctly. Other times I release too early or too late. And of course, sometimes a quantity of gonzo-blogger theater compresses down to nothing and I have to throw away a draft.

And some days, I find myself staring at a set of dense thoughts that refuse to either cohere into a longer piece or dissolve into noise. So I packetize them into virtual palm-leaf index cards delimited by asterixes, and let them loose for other scribes to shuffle through and perhaps sinter into a denser mass in a better furnace.

It is something of a lazy technique, ultimately no better than list-blogging in the gonzo blogosphere. But if it was good enough for Wittgenstein, it’s good enough for me.

***

I want to share the story behind approximately $2700 dollars worth of my spending this year that reveals how I am finally starting to leave the middle class, materially, financially and psychologically. No, I am not moving up into the rich class or down into the poor class. I am doing something complicated called trading up. 

This $2700 is money that, if I’d decided to pull the trigger and spend it a few months earlier, would have spared me a ton of unnecessary frustration. Why didn’t I spend it when I should have?

One reason is that I still have residual middle-class financial programming in my head, expertly misguiding me to the wrong answers. Getting it out of my head feels like getting a bad malware and virus infection off a computer. It is painful and messy, and there are really no completely reliable tools that work in all cases. And you’re never quite sure if you got the last infected file off the system, when the infection is really bad.

Another reason is that I was (and remain to some extent) guilty of what science fiction writer Bruce Sterling calls acting dead: being irrationally averse to spending money where it matters, in a misguided attempt to “save” money to the point that the behavior paralyzes you. A large segment of the middle class is starting to act dead these days. Which makes sense since the class itself is dying. To stop acting dead, you have to resolve to exit the traditional middle class as well, unless you want to go down with it.

Not acting dead involves a strategic spending pattern that marketers are starting to call trading up: buying premium in some areas of your life, while buying budget or entirely forgoing spending in other areas. This pattern of conscious, discriminating consumption defines the emerging replacement for  the middle class.  As the picture above illustrates, there isn’t really one “New Middle Class.” Instead, it is a fragmented social space, with each little island being defined by a specific pattern of trading-up, and an associated lifestyle design script.

This effect is a sort of the opposite of what I called Gollumization earlier this year: unthinking, undiscriminating consumption to the point that consumption defines you.

There’s a pretty neat book about it, Trading Up by Michael Silverstein and Neil Fiske, which you should read if you, like me, have exited or are planning to exit the traditional middle class.

But back to acting dead and my $2700 dollars, which I’ll use as my running example to get at various things.

The Dead Great-Grandfather Test

Sterling was using the term specifically to describe the hairshirt green lifestyle that is driven by eco-anxieties. For hairshirt-green types, life is all about saving water, recycling, composting, reducing eco-footprints and various other behaviors marked by a kind of fearful, non-generative retreat from living. Permanent existential hibernation.

Sterling’s rule of thumb for spotting acting-dead behaviors is a great one: if it’s something your dead great-grandfather can do better than you, it’s a case of acting dead. Your dead great-grandfather uses no water or plastic, and is actually recycling himself as we speak, not just his possessions. Try and top that.

But acting dead goes beyond hairshirt-green behaviors. While spartan frugality is a virtue, when it becomes the entire purpose of your life, there’s a problem. For a portion of the dying American middle class, frugality has turned into a life purpose.

An example is extreme couponing, which is why I used that as an example of radical Gollumization. It is saving gone amok: never buying anything not on sale (and therefore never buying things that never go on sale) and systematically being a jerk to businesses that may be running loss-leader sales to get new customers.

So how should you spend?

Spending Money

In his talk, Sterling offers up a simple rule for how to spend money. If it is something you use a lot everyday, spend the money, and get the good stuff. Don’t buy cheap. Look for deals, but don’t let deal-seeking make you compromise on quality or wait too long. It will cost you more in the long term. Sterling’s examples are obvious and physical: a good quality bed and work chair for instance. You might spend up to 8 hours a day in each; that’s 2/3 of your life.

I own both an excellent bed and a great chair. I am not sure the latter was a good investment for me in particular, since I spend most of my sitting hours in coffee shops, but in principle, it is a great example. Other examples include: a great kitchen knife, a nice car if you spend many hours commuting per day, plenty of quality gym clothes and a membership at a good gym, so you never have an excuse not to work out. Good quality produce to cook with.

If you work mostly at your desk, a large monitor. Heck, multiple monitors. The best keyboard.

Sterling also has ideas on what not to buy, or get rid of if you already own it. Expensive china sets for example, if you never do any formal entertaining. Things you think are assets but are actually liabilities. Things you are being unnecessarily sentimental about.

Sterling’s ideas seem to have been independently rediscovered by a growing segment of the middle class. Hence the phenomenon of trading up (the book has lots of data and anecdotal evidence for the trend).

I think of these sorts of examples as “physical furniture.” Stuff in your life that can make it hoarder hell if you buy the wrong things, or heaven if you buy the right things.

$2700 Worth of Acting-Dead

My acting-dead behaviors this year were more about mental furniture. Here’s the breakdown of the $2700 that I eventually spent when I stopped acting dead:

1. About $250 to get Tempo converted to epub and Kindle formats
2. About $300 odd to get an agent to file some Nevada business paperwork for me
3. $2100 for a Matlab (scientific computing software) license

In each case, I procrastinated for months, with the vague idea of saving money. Actually, it was worse than mere procrastination, since I was expending useless effort. In each case, my dead great-grandfather could have achieved what I did around those tasks during those months: nothing. And he’d have done it more efficiently.

In the first two cases, I tried to do it all myself, even though I have an aversion to fussy kinds of technical formatting work and paperwork to the point that they should count as phobias.  When I finally pulled the trigger and outsourced the work, it was like a major load being taken off my mind, coupled with severe regret for the time already spent on pointless frustration.

In the third case, it was again about saving money. I spent months mucking around with Python, R and various other open source alternatives to Matlab. Here, the messiness of having to deal with a unwieldy and weakly integrated open-source tools, along with my own serious aversion (similar to my paperwork aversion) to fussy configuration issues, and my generally poor ability to pick up new programming skills, had me wasting months in frustrated spinning-of-wheels.

And in the meantime, I was not doing things I wanted to do, simply because I was too cheap to buy a quality tool that I was familiar with, and could save me months of painful learning (especially painful now due to the Python 2.x to 3.x transition).  As with the other two cases, finally pulling the trigger made me intensely relieved.

You could say that each poor decision (each a case of delaying the right decision) was caused by specific phobias, aversions and irrationality.

But there is also a general pattern here. I really was not able to rationally assess the costs and benefits of each decision until after I had persisted with the wrong decision for months and made the right decision out of frustration. I could only see the simple logic after I’d made the right decision and stopped rationalizing the wrong one.

The general pattern that causes such poor decision-making is the middle class financial script.

The Middle-Class Financial Script

The middle class financial script is simple really. It involves uniform spending habits within a large class, based on norms that are learned via imitation.

If you are in the middle class, you are expected to own certain things, do certain things and do so at quality levels that exceed the quality purchased by the poor class (if they purchase that category of things at all) but don’t hit luxury levels.

You are also expected to not buy certain things that are either above or beneath you, or do certain things for yourself. Vanity,  humility and a sense of entitlement are all at work here. For the middle class, there are things that are beneath your station and things that are above your station. For the rich and poor, things are much more one-sided.

To take some simple examples, you’d be looked upon with suspicion if you bought a car that was either too luxurious or too cheap for somebody claiming middle class status. You are expected to vacation in certain places and not others.

In fact, imitation and uniformity in consumption define the middle class. In countries where the middle class is burgeoning instead of dying, especially in Asia, the growth of the class is tracked via measurement of ownership rates of certain typical goods at typical quality levels. By contrast, there is much more variety in how the poor are poor, and how the rich are rich.

Why does the middle class script (or any script) exist?

Mainly because it makes financial management easy. Constantly computing the total costs of ownership, potential returns and risks around all spending decisions,  is hard. And it doesn’t seem worthwhile when the income side is predictable and comfortable. Why bother to control costs when revenues are fixed and somebody else has already made up a predictable-costs script with reasonable margins designed to get you through retirement?

In other words, the middle class in recent history has been defined by its ability to both earn and spend money in very predictable ways.

Then of course, the risks started creeping back in, around 1980, slowly at first, and then with increasing rapidity over the last few years. All the things the middle class relied on — job security, defined benefits pensions, affordable mortgages, predictably rising real-estate values — one by one, all these supports began to break down.

But autopilot spending has persisted, long after the new patterns of exposure to financial risk have become clear. The reason of course is that the old financial habits were not really financial per se, they were driven by class norms rather than financial risk-management calculations.

My own examples are a case in point. My behavior is readily explained with reference to middle class norms:

1. The eBook conversion example: Middle class people do not hire other middle class people outside of a few approved exceptions such as doctors, lawyers and accountants; they work for the rich and hire the poor.
2. The business paperwork example: Middle class people do not “indulge” in “luxuries” like hiring administrative help to do paperwork. That’s for rich people with complicated financial affairs. Honest middle-class people should be able to do their own paperwork, with at most some professional help at tax time. Needing help probably means you are up to shady things.
3. The Matlab example: Middle class people do not pay for their tools. In fact, they shouldn’t need tools beyond the basic tools of literacy (books, pen and paper 100 years ago, a computer today). Poor people use specialized tools. Rich people buy them. Middle class people merely supervise the use of the rich people’s tools (capital) by the poor (labor). Even today, if you use specialized tools to work, your membership in the middle class is suspect.

Above all this, the middle class script involves a certain aversion to talking about or dealing with tough financial decisions. It is considered unseemly. Decent people don’t talk about money, let alone risk. If you work hard and play by the rules, the money should take care of itself. If it isn’t doing that, you are probably looking for dishonest and exploitative shortcuts like the evil rich or doing dumb things like the stupid poor, and deserve what you get.

If you have to budget and watch your money too closely, you were probably being irresponsible with credit cards and deserve your pain. For decent people, paycheck-in, on-time-credit-card-payments-out should work smoothly on autopilot.

And above all, you don’t speculate. If forced to speculate by pensions being turned into 401(ks) (American stock-based defined contribution retirement plans), decent people leave the actual risk-taking decisions to professional fund managers, telling themselves things like  ”you cannot beat the professionals.”

So what will happen to people operating by such obviously dangerous attitudes in difficult times?

Turns out, we’ve been here before. They’ll die out.

Middle Class Declines in History

This is not a new phenomenon in history. Middle classes have appeared and disappeared several times before in history.

Tennessee Williams’ plays (A Streetcar Named Desire, The Glass Menagerie) tell exactly such poignant fall-from-the-middle-class stories set in early 20th century America.

Early twentieth century British novels set during the decline of empire (such as Agatha Christie novels), often contain aging spinsters desperately keeping up appearances and surviving on small incomes derived from being “companions” to richer old women.

You can also find examples outside the Western world. In nineteenth century India for example, where the Urdu and Sanskrit-literate middle classes, which had grown around the courts of the Nawabs and Maharajas in older medieval cities, went into severe decline. The new English-literate middle class began supplanting it in the newer cities of the British Raj.

I suspect similar middle class declines can be found in the Middle East (during the Ottoman decline), China (after the Boxer Rebellion)  and Latin America (after the Monroe Doctrine perhaps? I am not too familiar with Latin American history).

When a middle class goes into decline, you get a large segment of the population engaging in a desperate scramble to keep up appearances, while switching from collective-norm-based to individual-risk-based financial thinking.

Keeping up with the Joneses becomes far harder, because the financial support starts to collapse at different times for different people, but everybody agrees to pretend that everybody is in it together.  For the current American decline, there have already been a couple of good movies chronicling the decline: The Joneses (2009) and The Company Men (2010).

A norm-based social class will persist with disastrous financial choices long after the secure financial environment, on which its scripts are based, collapses. Simply because membership of the class is the source of all social identity and access to social capital.

Except that the social capital, which the members are clinging to, is eroding rapidly as well. There is no point in two non-swimmers with immense trust between them, clinging to each other while drowning. Mutual trust and social capital within a group only mean something when there are objective reasons to expect a prosperous future of indefinite length stretching out ahead.

When this is not the case, it makes sense to cash out your hard assets, rethink your financial life more directly, write off investments in the social capital of the declining class, and look for an alternative emerging class to join.

Trading Up and Fragmentation

As the picture I started with shows, a key effect of the trading-up phenomenon is that it causes serious fragmentation. The social landscape starts to get restructured along new lines. Cultural geography changes, as governing financial scripts change from one city block to the next (you see a lot of this in San Francisco in particular).

The transition from a monolithic middle class to one of many trading-up classes is a very tough one. First, you have to go through a period where you manage your finances very directly, with no help from a script that simplifies decision-making.

Then you have to evaluate various alternative trading-up scripts to figure out which ones might actually fit your situation and encode meaningful adaptations to the new environment. Not every lifestyle design script is likely to work.

In the last few months, going back to the broader context of my three examples, I’ve done a good deal of very direct financial decision-making. I’ve made up detailed scenario planning spreadsheets, risk models and the like. I’ve done minute tracking of spending (only for a month, to sort of calibrate; it is far too difficult and depressing to do on an ongoing basis).

Here’s the funny thing: doing this kind of very direct financial management around my small-business book-keeping felt good. It felt smart, like I was learning valuable new skills. But doing it around personal and household finances still felt somehow dirty. That’s how deeply embedded the middle class script is.

The three examples were interesting and particularly tough because they bridged the two mental models: my healthy business mental model (within which the right spending decisions would have been easy) and my toxic middle-class-paycheck mental model (within which they were unnecessarily hard).

Scared, Foolhardy and Brave New Scripts

Once you’ve worked with your finances directly for a while (it’s like working in assembly language, on a computer without an operating system) to start the transition away from the middle class script, you have to end the transition. Staying in limbo doesn’t work.

The transition can end in three ways:

1. Prolonged Misery: You get so scared, you retreat to the middle class and do your best to delay the inevitable
2. Waiting for Godot: You latch onto some script and  stick to it even after it becomes clear that it isn’t working for you.
3. Quick-Change Artists: You try on different scripts for size, attempting to force outcomes and fast failures, until you find one that fits and works, the way those quick-change artists change clothes.

Prolonged misery makes for the best tragic literature but is entirely unpleasant to live through. You act increasingly dead, get increasingly frugal, gradually squeeze out all the generativity in your life, and then finally you die.

The characteristic sign that you are practicing unhealthy acting-dead frugality is that you cut back on core expenses that might help you be more generative, in order to keep up appearances as long as possible.

If you are cutting back on the quality of the food you eat (trading fresh vegetables for canned, say), in order to buy the same clothes your friends wear, you are on the prolonged misery path. This incidentally, may be part of the reason why the middle class has become so attached to recycling and other hairshirt-green behaviors (outside of the actual merits of the behaviors) during exactly the period that the class itself has been in decline.

Waiting for Godot is your classic arrival fallacy. You fixate on specific narrative elements (like moving to Bali or working for 4 hours a week), make the few big moves, and spend the rest of your life waiting for the Big Event signifying that it is working, while slipping slowly into destitution and denial. I see a lot of people in this mode right now. They’ve never really stopped to analyze the logic of the script, but accepted it on faith based on assurances from a few for whom it has worked.

Quick-change artistry is of course, the card I think you should pick. It is a turbulent, experimental approach, where there are no absolute life truths, no permanent commitments to any script, no one-book formulas, and no easy no-brainer decisions.

It involves trying different trading-up patterns until you find one that works. It involves a commitment to stop acting dead. It involves a conscious decision to leave the middle class.

Or you can wait for all the King’s men and all the king’s horses to put Humpty-Dumpty together again.

This piece is sort of a continuation of my Las Vegas Rules series, but I’ve abandoned the attempt to keep a coherent larger narrative going. This is going to be more of an occasional diary-entry sort of thing.

First, let me get an announcement out of the way: Tempo is now out on the Kindle. Buy it, give it as a gift, tweet it etc. Whew! That’s a big, high-priority item checked off my to-do list.

Speaking of priorities. I had one of my weirder Aha! moments: you can use the well-known Towers of Hanoi game as a metaphor to understand the behavior of Maslow’s hierarchy of needs (or any similar hierarchy of priorities) under changing life circumstances, and the role of compartmentalization as a costly coping strategy. Here’s a picture:

If the details and implications of the metaphor aren’t immediately obvious, read on for the help-text.

The Basic Metaphor

The game involves moving a pyramidal stack of disks from one of three pegs to another without ever letting a larger disk rest on a smaller one. The number of moves approximately doubles each time you add a disk (so the full 5-level Maslow hierarchy would take 31 moves, where this simplified 3-disk version takes 7 moves).

The metaphor works like this: at any given time, each stack of disks represents a life compartment. Sometimes your life is more compartmentalized (work vs. life or work vs. life vs. health), and sometimes, it is all a single beautiful symphony.

The “never stack a big disk on a small one” models the idea that most of us don’t consciously violate priorities in obviously avoidable ways. If we do, we recognize our “moment of weakness.”

But this is only a local sort of consistency: we apply prioritization ideas via pair-wise comparisons/tradeoffs while making specific decisions. You rarely have more than two priorities butting up against each other in any given decision.

So long as your life situation is stable, you can gradually integrate the various compartments and get back to a single stack. We call this getting into a “routine.”

But when your life shifts, fragmentation and compartmentalization necessarily intrude. If your life changes faster than you can get back to the healthy stack of priorities, you’ll end up a mess, unless you level-up your game and redefine your idea of stability (think George Clooney in Up in the Air).

Integrated vs. Fragmented Priorities

In a fragmented, compartmentalized state, disks on the same stack represent integrated priorities while disks on different stacks represent fragmented priorities.

Integrated priorities reinforce each other via strong positive-feedback loops (lots of small trade-off decisions, with healthy behaviors being reinforced each time you make the healthy choice), and you can manage them consciously.

Fragmented priorities cannot be managed consciously, and can lead to messed-up behaviors that are not aligned with your actual priorities. You also lose any mutual reinforcement effects via positive feedback loops.

This happens because, in a compartmentalized life, you only have opportunities to trade off priorities that are on the same stack, through individual, micro-level decisions

Your ability to manage tradeoffs between stacks is much more limited. You are reduced to vague ideas like “I need to spend more time with my kids” or “I need to go to the gym more.” I suspect most people allocate attention amongst compartmentalized stacks based on how many disks the stack has (which determines how hard the stack is to compute with) rather than the priority of the stack.

To take a simple example “work life balance” is extremely hard in a traditional industrial environment, where “work” happens at the workplace, and “life” happens at home.

But it is much easier in (say) a Googleplex-like workplace with childcare, gyms and healthy food options available right near your office.

So instead of having to think in terms of “I need to spend more time with my kids” you can trade off “get coffee between meetings right now, or play a game of ping-pong with my kid in the childcare room.”

Priority management turns into a bunch of bite-sized decisions.

Implications of the Metaphor

The metaphor suggests several interesting ideas:

1. Consciously violating priorities (putting a bigger disk on a smaller one)  can help you stabilize much faster (exponentially faster) in a new situation, but at the risk of bigger disks permanently damaging/crushing smaller ones.
2. Given a specific definition of priorities/disks, there is a maximum frequency of life changing disruptions you can handle while still getting to a stable integrated stack at least briefly between disruptions. If your life changes any faster, it will be in a perennially unstable state.
3. Fusing layers simplifies the game. Manage fewer categories. So working for a moving company fuses work and physical health.
4. How you frame priorities is crucial. A stack of disks that needs to be moved under a life transition may not need to be moved at all if you redefine them to be robust to such transitions. If you are a nomad, capable of living out of a suitcase or camel-pack, your life will resist disruption due to physical moves. If you live in a country with portable, government-provided healthcare, your health will resist disruption due to changing jobs.
5. You are not entirely in control of your life stack. The organization of society plays a huge role.
6. Trying to keep unfused priorities stable in transit (moving multiple discs at once) is a balancing act.

The part of the metaphor that interests me the most is the idea that there are positive feedback loops within compartments. The more you compartmentalize your life, the more you lose the benefit of such loops. I think this relates to how 10x dynamics can be catalyzed in your life. It also relates to the more comprehensive analysis of such stuff that Greg Rader recently posted.

I don’t like or use the term scientific method. Instead, I prefer the phrase scientific sensibility. The idea of a “scientific method” suggests that a certain subtle approach to engaging the world can be reduced to a codified behavior. It confuses a model of justification for a model of discovery. It attempts to locate the reliability of a certain subjective approach to discovery in a specific technique.

It is sometimes useful to cast things you discover in a certain form to verify them, or to allow others to verify them. That is the essence of the scientific method. This form looks like the description of a sequential process, but is essentially an origin myth. Discovery itself is an anarchic process. Like the philosopher Paul Feyerabend, I believe in methodological anarchy: there is no privileged method for discovering truths. Dreaming of snakes biting their tails by night is as valid as pursuing a formal hypothesis-proof process by day. Reading tea leaves is valid too. Not all forms of justification are equally valid though, but that’s a different thing.

But methodological anarchy does not mean — at least not to me — that there is no commonality at all to processes of discovery. The sensibility that informs reliable processes of discovery has a characteristic feature: it is unsentimental.

An unsentimental perspective is at the heart of the scientific sensibility.  But first, why “sensibility”?

Susan Sontag’s description of a sensibility in her classic essay, Notes on Camp gets it exactly right:

Taste has no system and no proofs. But there is something like a logic of taste: the consistent sensibility which underlies and gives rise to a certain taste…Any sensibility which can be crammed into the mold of a system, or handled with the rough tools of proof, is no longer a sensibility at all. It has hardened into an idea…[t]o snare a sensibility in words, especially one that is alive and powerful, one must be tentative and nimble.

The scientific method is a sensibility crammed into the mold of a system. It is a an attempt to externalize something subtle and internal into something legible and external. The only reason to do this is to scale it into an industrial mode of knowledge production, which can be powered by participants who actually lack the sensibility entirely. Such knowledge production has been characteristic of the bulk of twentieth century science (in terms of number of practitioners, not in terms of value). Hence the Hollywood stereotype of the scientist as a methodological bureaucrat; someone who worships at the altar of a specific method. Sadly, Hollywood gets it right. The typical scientist is a caricature of a human.

When we objectify discovery into a legible system and a specific method, the subjective attitude with respect to that system and method becomes  impoverished in proportion to the poverty of the system and method itself.

So to characterize our subhuman scientist, we use words like objective,  emotionless and disinterested. The first is a reductive characterization: the unsentimental scientific sensibility can turn its gaze onto purely subjective realities and discover riches. To limit it to objectivity is to limit it to the narrow realm of the experimental method. Similarly, lack of emotion turns into a virtue instead of a crippling blindness. And finally when we say that to do science is to adopt a disinterested stance, we institutionalize it. The scientist becomes an impersonal judge in a courtroom of evidence, free from any conflicts of interest. It is no wonder that when film-makers attempt to humanize scientist characters, they have them succumb to personal motivations.

The scientific sensibility, however, is both broader and more fertile than this combination of an impoverished system and a sub-human caricature — objective, emotionless and disinterested.  To look at the world with the scientific sensibility is to be more human, not less.

The word unsentimental is central here. To be unsentimental is to be self-aware. To be unsentimental, you must first deal with your inner realities at the level of sentiments rather than emotions. You do so by creating mental room for emotions to drift out of your subconscious, recognizing the desires that generate them and labeling the results. If you can go beyond that and bracket the sentiments for further contemplation, you can be unsentimental. The sentiments that accompany you on a journey of discovery are part of the phenomenology that you must process on that journey.

To have a perfectly unsentimental sensibility is to be free to look at reality without expectations about what you will see.

You can be trained in the scientific method. In fact the method, in all its impoverished glory, can actually be programmed into a computer for certain problems. You cannot, however, achieve the scientific sensibility through a training process or program it into a computer. At least not yet.

You cannot achieve this sensibility via a mechanical process of identifying and neutralizing a laundry list of cognitive biases. Nor can you get there through an effort of will or by struggling to suppress emotions. To be unsentimental is not about suppressing your humanity, it is about making your humanity irrelevant so you are reduced to the pure act of seeing.

The only way to get there is by making a sacrifice: you must give up the pleasures of a sentimental engagement with life. The unsentimental eye, once opened, cannot be closed. The adoption of the scientific sensibility is an irreversible step. Your experience of love, friendship and fun will change. Expect your passions to be tragic passions. If you are religious, expect a troubled existence. The scientific method is not incompatible with religion, but the scientific sensibility is, because religion presupposes a sentimental engagement of life.

There is one consolation though. The scientific sensibility makes humor and irony your constant companions for life.

I will be on vacation next week until after the Labor Day weekend. See you all again week after next.

I find myself feeling strangely uncomfortable when people call me a generalist and imagine that to be a compliment.  My standard response is that I am actually an extremely narrow, hidebound specialist. I just look like a generalist because my path happens to cross many boundaries that are meaningful to others, but not to me. If you’ve been reading this blog for any length of time, you know the degree to which I keep returning to the same few narrow themes.

I think I now understand the reason I reject the generalist label and resonate far more with the specialist label. The generalist/specialist distinction is an extrinsic coordinate system for mapping human potential.  This system itself is breaking down, so we have to reconstruct whatever meaning the distinction had in intrinsic terms. When I chart my life course using such intrinsic notions, I end up clearly a (reconstructed) specialist.

The keys to this reconstruction project are: the much-abused idea of 10,000 hours of deliberate practice, the notion of grit, and an approach to keeping track of your journey through life in terms of an intrinsic coordinate system. Think of it as replacing compass or GPS-based extrinsic navigation with accelerometer and gyroscope-based  inertial navigation.

I call the result “the calculus of grit.” It is my idea of an inertial navigation system for an age of anomie, where the external world has too little usable structure to navigate by.

The Generalist-Specialist Distinction

The generalist/specialist distinction constitutes an extrinsic coordinate system. We think of our environment as containing breadth and depth dimensions. The breadth dimension is chopped up by disciplinary boundaries (whether academic, trade-based or business-domain based), while the depth dimension is chopped up by markers of validated progressive achievement. What you get is a matrix of domains of endeavor: bounded loci within which you can sustain deepening practice of some skilled behavior.

The boundedness is key. Mathematicians do not suddenly discover, in the 10th year of their practice, that they need advanced ballroom dancing skills to progress further. Ballroom dancers do not suddenly encounter a need for advanced aircraft engine maintenance skills after a few years of practice. Based on your strengths, you can place fairly safe bets early on about what you will/will not need to do if you make your home somewhere in the matrix.

Or at least, you used to be able to. I’ll get to how these expectations from the twentieth century are breaking down.

There is a social structure that conforms to these breadth/depth boundaries as well. A field of practitioners in each domain, stacked in a totem pole of increasing expertise, that legitimizes the work of individuals and provides the recognition needed for both pragmatic ends (degrees and such) and existential ends (“recognition” in the sense of say, Hegel).

In his book Creativity, Mihaly Csikzentmihalyi made up exactly such a definition of extrinsically situated creativity as the behavior of an individual within a field/domain matrix.

We are now breaking away from this model. Ironically, Csikzentmihalyi’s own work makes little sense within this model that he helped describe in codified ways; his work makes a lot more sense if you don’t attempt to situate it within his nominal home in psychology.

Extrinsically situated creativity with reference to some global, absolute scheme of generalist/specialist dimensions is unworkable. At best we can hope for local, relative schemes and an idea of intrinsically situated individual lives.

The Vacuity of Multi-Disciplinarity

The problem with this generalist/specialist extrinsically situated creativity model is that the extrinsic frames of references are getting increasingly dynamic, chaotic and murky. To the point that the distinction is becoming useless. Nobody seems to know which way is up, which way is down, and which way is sideways. If you guess and get lucky, the answers may change next year, leaving you disoriented once more.

The usual response to this environment is to invoke notions of multi-disciplinarity.

Unfortunately, this is worse than useless. In the labor market for skilled capabilities, and particularly in academia, multi-disciplinarity is the equivalent of gerrymandering or secession on an already deeply messed-up political map.  Instead of votes, you are grubbing for easily won markers of accomplishment. Its main purpose (in which it usually fails) is to create a new political balance of power rather than unleash human potential more effectively.

The purpose is rarely to provide a context for previously difficult novice-to-master journeys.

How do I know this? It’s patently obvious. If it takes 10,000 hours (K. Anders Ericsson’s now-famous threshold of deliberate practice, thanks to Gladwell, which translates to about 10 years typically) to acquire mastery in any usefully bounded domain, and you assume that there is at least one generation of pioneers who blazed that path to a new kind of mastery, what are you to make of fields that come and go like fruit flies in 2-3 years, in sync with business or funding cycles? The suspicious individual is right to suspect faddishness.

I have come to the conclusion that if I cannot trace a coherent history of at least 20 years for something that claims the label “discipline,” it isn’t one.

The problem with this though is that increasing amounts of valuable stuff is happening outside disciplines by this definition. It isn’t multi-disciplinary. It isn’t inter-disciplinary. It is simply non-disciplinary. It’s in the miscellaneous folder. It is so fluid that it resists extrinsic organization.

So given that most excitement centers around short-lived fruitfly non-disciplines, how do people even manage to log 10,000 deliberate practice hours in any coherent journey to mastery? Can you jump across three or four fruit-fly domains over the course of a decade and still end up with mastery of something, even if you cannot define it?

Yes. If you drop extrinsic frames of reference altogether.

The Compass and the Gyroscope

We are used to describing movement in terms of x, y and z coordinates, with respect to the Greenwich meridian, the Equator and sea level. Our sense of space is almost entirely based on such extrinsic coordinate systems (or landmarks within them). Things that we understand via spatial metaphors naturally tempt us into metaphoric coordinate systems like the depth/breadth one we just talked about. In academic domains, for instance, you could say the world is mapped with reference to an origin that represents a high-school graduate, with disciplinary majors and years of study forming the two axes that define further movement.

Somewhere in graduate school, I encountered an idea that blew my mind: you can also describe movement entirely intrinsically. Actually, I had encountered this idea before, in vague popular science treatments of Einstein’s general theory of relativity, but learning the basics of the math is what truly blows your mind.

The central idea is not hard to appreciate:  imagine riding a complicated roller coaster and keeping track of how far along you are on the track, how you’ve been turning, and how you’ve been twisting. That much is easy.

What is not easy is appreciating that that’s all you need. You can dispense with extrinsic coordinate systems entirely. Just keeping track of how those three variables (known as arc-length, curvature and torsion if my memory serves me) are changing, is enough. For short periods, you can roughly measure them using just your intrinsic sense of time and how your stomach and ears feel. To keep the measurements precise over longer periods, you need a gyroscope, an accelerometer and a watch.

If you want motifs for the two modes of operation, think of it as the difference between a magnetic compass and a gyroscope (these days, GPS might be a better motif for the former, but the phrase “the compass and the gyroscope” has a certain ring to it that I like).

We need another supporting notion before we can construct an intrinsic coordinate system for human lives.

Behavioral Boundedness

Remember that the primary real value of an extrinsically defined discipline in a field/domain matrix is predictable boundedness. Mathematicians can trust that they won’t have to suddenly start dancing halfway through their career to progress further.

This predictability allows you to form reasonable expectations for decades of investment, and make decisions based on your upfront assessment of your strengths, and expectations about how those strengths will evolve as you age.

If I decide that I have certain strengths in mathematics and that I want to bet on those strengths for a decade, to get to mastery, I shouldn’t suddenly stumble into a serious weakness along the way that blocks me, like a lack of natural athleticism.

So a disciplinary boundary is very useful if it provides that kind of predictability. I call this behavioral boundedness. An expectation that your expected behaviors in the future won’t wander too far out of certain strengths-based comfort zones you can guess at fairly accurately, upfront. Before putting in 10,000 hours.

What happens when that sort of predictability breaks down? It is certainly happening all over the place. For instance, I didn’t realize I lacked the strengths needed for a typical career in aerospace engineering (the sort high-school kids fantasize about when they first get interested in airplanes and rockets) until well into a PhD program in the subject. Fortunately, I was able to pivot and head in another direction with almost no wasted effort. Few people are that lucky.

There are domains where the boundedness is very weak indeed. The upfront visible boundedness is a complete illusion. Marketing is one such domain. You might get into it because you love creative messaging or talking to people. You may discover the idea of positioning two years into the journey and realize that creativity in messaging is a sideshow, and the real job is somewhat tedious analysis of the mental models of prospects. A further two years down the road, you may discover that to level-up your game once more, you need to become a serious quantitative analytics ninja and database geek.

This can also work out in positive ways. You might wake up one fine day and realize that your life, which makes no sense in nominal terms, actually adds up to expertise in some domain you’d never identified with at all. That actually happened to me with respect to marketing.  On paper, I am the opposite of a marketer. I have a PhD in aerospace engineering, am introverted, and write in long-winded and opaque ways rather than in catchy sound-bytes.

Nevertheless, at some point I realized with a shock that I had accidentally logged several thousand hours along a marketing career path without realizing it. I had just completely misunderstood what “marketing” meant based on the popular image the field presents to novices.

When I went free-agent a few months ago, most of my consulting leads I had coming in had to do with marketing work. This did not surprise me, but it certainly surprised my father and several close friends, who assumed I was doing some sort of technical consulting work around computational modeling and scientific computing.

I’d never thought of myself as a marketer. A computational modeler, yes. A hustler perhaps. A fairly effective corporate guerrilla, yes. A marketer, not really. I viewed my previous marketing work as the work of a curious tourist in a strange land. I viewed my marketing writing as outsider-anthropology amongst strange creatures. But apparently, that’s not how others view me.

Looking back, and trying to make sense of my life in retrospect as “the training of an accidental marketer,” it makes sense though: I’ve logged the right mix of complementary experiences. Marketing is still not my primary identity though (that would mean returning to a Procrustean bed of disciplinary identity).

Many people luck out like me, accidentally. We recognize what particular path to mastery we’re on, long after we actually get on it.

Many do not. They bum around in angsty anomie, craving structure where none exists, and realizing after a decade of wandering that they’ve unfortunately gotten nowhere.

Is it possible to systematically do things to put yourself on a path to mastery, and know you’re on one, without actually knowing what that path is until you’re already far down it?

Inside and Outside Views of Grit

If there is no external frame of reference, how do you know where you are, where you are going and whether you are progressing at all, as opposed to bumming around?

Can you log any old time-sheet of 10,000 hours, slap a label on it, and claim mastery?

Thankfully, intrinsic navigation is not quite that trite.

A clue to the mystery is the personality trait known as grit, probably the best predictor of success in the modern world.

Grit is the enduring intrinsic quality that, for a brief period in recent history, was coincident with the pattern of behavior known as progressive disciplinary specialization.

Grit has external connotations of extreme toughness, a high apparent threshold for pain, and an ability to keep picking yourself up after getting knocked down. From the outside, grit looks like the bloody-minded exercise of extreme will power. It looks like a super-power.

I used to believe this understanding of grit as a superhuman trait.  I used to think I didn’t possess it. Yet people seem to think I exhibit it in some departments. Like reading and writing. They are aghast at the amount of reading I do. They wonder how I can keep churning out thousands of words, week after week, year after year, with no guarantee that any particular piece of writing will be well-received.

They think I must possess superhuman willpower because they make a very simple projection error: they think it is hard for me because it would be hard for them. Well of course things are going to take superhuman willpower if you go after them with the wrong strengths.

For a while, I went around calling this faux-grit. The appearance of toughness. But the more I looked around me at other people who seemed to display grit in other domains, the more I realized that it wasn’t hard for them either. What they did would merely be superhuman effort for me. Faux grit and true grit are the same thing (the movie True Grit is actually quite a decent showcase of the trait; it showcases the superhuman outside/fluid inside phenomenon quite well).

So what does the inside view of grit look like? I took a shot at describing the subjective feel in my last post on the Tempo blog. It simply feels like mindful learning across a series of increasingly demanding episodes that build on the same strengths.

But the subjective feel of grit is not my concern here. I am interested in objective, intrinsically measurable aspects of grit that can serve as an internal inertial navigation system; a gyroscope rather than  GPS.

The Grit Gyroscope: Reworking, Referencing, Releasing

In physical space, latitude, longitude and altitude get replaced by arc-length, curvature and torsion when you go intrinsic.

In endeavor space, field, domain and years of experience get replaced by three variables that lend themselves to a convenient new 3Rs acronym: reworking, referencing, releasing (well, technically, it is internal referencing and early-and-frequent releasing, but let’s keep the phrase short and alliterative). I believe the new 3Rs are as important to adults as the old ones (Reading, wRiting and aRithmetic) are for kids.

Reworking

I stumbled upon rework as a key variable when I tried to answer a question on Quora: what are some tips for advanced writers?

Since writing is something everybody does, logging 10,000 writing hours is something anyone can do. My aha! moment came when I realized that it isn’t the writing hours that count, it is the rewriting hours. Everybody writes. People who are trying to walk the path towards mastery rewrite. I won’t say more about this variable. If you want a worked example, read my Quora answer. If you want a quick and pleasant read on the subject, Jason Fried’s Rework gets at some of the essential themes (though perhaps in a slightly gimmicky way).

Referencing

For referencing, my clue was my recent discovery that new readers of this blog often dive deep into the archives and read nearly everything I’ve written in the last four years. I dubbed it the ribbonfarm absurdity marathon because I didn’t understand what would possess anyone to undertake it.

But then I realized that I write in ways that practically demand this reading behavior if people really want to get the most value out of what I am talking about: I reference my own previous posts a lot. Not to tempt people into reading related content, but out of sheer laziness. I don’t like repeating arguments, definitions or key ideas. So I back-link. I do like most of my posts to be stand-alone and comprehensible to a new reader though, so I try to write in such a way that you can get value out of reading a post by itself, but significantly more value if you’ve read what I’ve written before. For example, merely knowing what I mean by the word legibility, which I use a lot, can increase what you get out of some posts by 50%. This is one reason blogging is such a natural medium for me. The possibilities of hyperlinking make it easy to do what would be extremely tedious with paper publishing.

The key here is internal referencing. I use far fewer external reference points (there’s perhaps a dozen key texts and a dozen papers that I reference all the time). It sounds narcissistic, but if you’re not referencing your own work at least 10 times as often as you’re referencing others, you’re in trouble in the intrinsic navigation world.  Instead of developing your own internal momentum and inertia, you are being buffeted by external forces, like a grain of pollen being subjected to the forces of Brownian motion.

Releasing

And finally, releasing. As in the agile software dictum of release early and often. In blogging, frequency isn’t about bug-fixing or collaboration. It isn’t even about market testing (none of my posts are explicitly engineered to test hypotheses about what kind of writing will do well).  It is purely about rational gambling in the dollar-cost averaging sense. It is the investing advice “don’t try to time the market” applied to your personal work.

If the environment is so murky and chaotic that you cannot strategically figure out clever moves and timing, the next best thing you can do is just periodically release bits of your developing work in the form of gambles in the external world. I think there’s a justifiable leap of faith here: if you are work admits significant reworking and internally-referencing, you’re probably on to something that is of value to others.

If a post happens to say the right thing at the right time, it will go viral. If not, it won’t. All I need to do is to keep releasing. This realization incidentally, has changed my understanding of phenomena like iteration in lean startups and serial entrepreneurs who succeed on their fifth attempt. It’s mostly about averaging across risk/opportunity exposure events, in an environment that you cannot model well.  I am pretty sure you can apply this model beyond blogging and entrepreneurship, but I’ll leave you to figure it out.

These three variables together can measure your progress along any path to mastery. What’s more, they can be measured intrinsically, without reference to any external map of disciplinary boundaries. All you have to do is to look for an area in your life where a lot of rework is naturally happening, maintain an adequate density of internal referencing to your own past work in that area, and release often enough that you can forget about timing the market for your ouput.

What does navigating by these three variables look like from the outside?

If you only do a lot of internal referencing, that’s like marching along a straight, level road.

If you do a lot of internal referencing and a lot of rework, that’s like marching along a steady uphill road that’s gradually getting steeper from an external point of view (in other words, you are on your own exponential path of progress). What you are doing will look impossible to observers. It may look like you are marching up a vertical cliff. A great example is the Silicon Valley archetype of the 10x engineer.

And finally, if you are releasing frequently, that’s like turning and twisting: spiraling around an increasingly steep mountain (or zig-zagging up via a series of switchbacks).

The Path of Least Resistance

Navigating with the 3Rs as an adult isn’t enough. You still have to recover the value the old disciplinary model provided: behavioral boundedness. Whether you are navigating intrinsically or extrinsically, suddenly running into a mountain — a major weakness — is just as bad.

The key here is very simple and very Sun Tzu: with respect to the external world, take the path of least resistance.

Why? Think of it this way. The disciplinary world very coarsely measured your aptitudes and strengths once in your lifetime, pointed you in a roughly right direction and said “Go!” The external environment had been turned into a giant obstacle course designed around a coarse global mapping of everybody’s strengths.

So there was no distinction between the map of the external world you were navigating, and the map of your internal strengths. The two had been arranged to synchronize. If you navigated through a map of external achievement, landmarks and honors, you’d automatically be navigating safely through the landscape of your internal strengths.

But when you cannot trust that you’ve been pointed in the right direction in a landscape designed around your strengths, you cannot afford to navigate based on a one-time coarse mapping of your own strengths at age 18.

If you run into an obstacle, it is far more likely that it represents a weakness rather than a meaningful real-world challenge to be overcome, as a learning experience.

Don’t try to go over or through. It makes far more sense to go around. Hack and work around. Don’t persevere out of a foolhardy superhuman sense of valor.

Hard Equals Wrong

If it isn’t crystal clear, I am advocating the view that if you find that what you are doing is ridiculously hard for you, it is the wrong thing for you to be doing.  I maintain that you should not have to work significantly harder or faster to succeed today than you had to 50 years ago. A little harder perhaps. Mainly, you just have to drop external frames of reference and trust your internal navigation on a landscape of your own strengths.  It may look like superhuman grit to an outsider, but if it feels like that inside to you, you’re doing something wrong.

This is a very contrarian position to take today.  Thomas Friedman in particular has been beating the “harder is better” drum for a decade now, most recently in his take on the London riots, modestly titled A Theory of Everything (Sort Of):

Why now? It starts with the fact that globalization and the information technology revolution have gone to a whole new level. Thanks to cloud computing, robotics, 3G wireless connectivity, Skype, Facebook, Google, LinkedIn, Twitter, the iPad, and cheap Internet-enabled smartphones, the world has gone from connected to hyper-connected.

This is the single most important trend in the world today. And it is a critical reason why, to get into the middle class now, you have to study harder, work smarter and adapt quicker than ever before. All this technology and globalization are eliminating more and more “routine” work — the sort of work that once sustained a lot of middle-class lifestyles.

The environment that really matters isn’t the external world. It is pretty much pure noise. You can easily find and process the subset that is meaningful for your life. It isn’t about harder, smarter, faster. If it were, I’d be dead. I’ve been getting lazier, dumber and slower. It’s called aging. I think Friedman is going to run out of superlatives like “hyper-” before I run out of life. If I am wrong, the world is going to collapse before he gets around to writing The World is Hyper-Flatter-er.  Humans are simply not as capable as Friedman’s survival formula requires them to be.

Exhortation is pointless. Humans don’t suddenly become super-human just because the environment suddenly seems to demand superhuman behavior for survival. Those who attempt this kill themselves just as surely as those dumb kids who watch a superman movie and jump off buildings hoping to fly.

It is the landscape of your own strengths that matters. And you can set your own, completely human pace through it.

The only truly new behavior you need is increased introspection. And yes, this will advantage some people over others. To avoid running faster and faster until you die of exhaustion, you need to develop an increasingly refined understanding of this landscape as you progress.  You twist and turn as you walk (not run) primarily to find the path of least resistance on the landscape of your strengths.

The only truly new belief you need is that the landscape of disciplinary endeavors and achievement is meaningless. If you are too attached to degrees, medals, prizes, prestigious titles and other extrinsic markers of progress in your life, you might as well give up now. With 90% probability you aren’t going to make it. It’s simple math: even if they were worth it, as our friend Friedman notes with his characteristic scare-mongering, there simply isn’t enough to go around:

Think of what The Times reported last February: At little Grinnell College in rural Iowa, with 1,600 students, “nearly one of every 10 applicants being considered for the class of 2015 is from China.” The article noted that dozens of other American colleges and universities are seeing a similar surge as well. And the article added this fact: Half the “applicants from China this year have perfect scores of 800 on the math portion of the SAT.”

If you’re paying attention to the Chinese kids who score a perfect 800, you’re paying attention to the wrong people. I mean, really? You should worry about some Chinese kid terrorized into achieving a perfect-800 math score by some Tiger Mom, and applying to Grinnell College?

It’s the Chinese kids who are rebelling against their Tiger Moms, completely ignoring the SAT, and flowing down the path of least resistance that you should be worried about.  After all Sun Tzu invented that whole idea.

So rework, reference, release. Flow through the landscape of your own strengths and weaknesses. Count to 10,000 rework hours as you walk. If you aren’t seeing accelerating external results by hour 3300, stop and introspect. That is the calculus of grit. It’s the exponential human psychology you need for exponential times. Ignore everything else.

Factoid: this entire 4000-plus word article is a working out of a 21-word footnote on page 89 of Tempo. That’s how internally-referenced my writing has become. Never say I don’t eat my own dogfood.

I divide my writing into two kinds: gold versus diamonds. Sometimes I knowingly palm cubic zirconia or pyrite onto you guys, but mostly I make an honest attempt to produce diamonds or gold. On the blog, I mainly attempt to hawk rough diamonds and gold ore. Tempo was more of an attempt at creating a necklace: polished, artistically cut diamonds set in purified gold.

I find the gold/diamond distinction useful in most types of creative information work.

What do I mean here? Both are very precious materials. Both are materials that are already precious in their natural state, as rough diamonds or gold ore. Refinement  only adds limited amounts of additional value. Both are mostly useless, but do have some uses: gold in conducting electricity, diamonds for polishing other materials. But there the similarities end.

Gold is almost infinitely adaptable. It is malleable and ductile. It can be worked very easily and finely using very little energy and tools made of nearly any other metal It is a nearly perfectly fungible commodity. Financially, it is practically a liquid rather than a solid. It plays very well with other materials and adapts to them. Its purity can be measured with near-perfect objective precision, and its value is entirely market-driven. It has no identity. Its value is entirely intrinsic and based on the rarity of the metal itself.

Gold can be melted, drained of history, and reshaped into new artifacts. When you add gold to gold, the whole is equal to the sum of the parts. When you subtract gold from gold, the pieces retain all the value of the whole. You can work gold in reversible ways.

Diamonds are not adaptable at all. They are the hardest things around, and the only thing that can work a diamond is another diamond. They are nearly perfectly non-fungible. The more precious ones are so non-fungible, they have names, personalities and histories that are nearly impossible to erase. As economic goods, they transcend mere brandhood and aspire to sentience: we speak of cursed or lucky diamonds. Diamonds do not play well with other materials. Other materials — gold in particular — must adapt to them. Purity and refinement are not very useful concepts  to apply to a diamond. In fact, a diamond is defined by its impurities. The famous Hope diamond is blue because of trace quantities of boron. Color, clarity and flaws can be assessed, but ultimately working a diamond is about revealing its personality rather than molding it. Diamonds that win personality contests go on to become famous. Those that fail to impress the contest judges are murdered — broken up into smaller pieces or degraded to industrial status.

The value of a diamond is in the eye of the beholder. At birth, rough diamonds are assessed by expert sightholders, and at every subsequent transaction, human judges assess value. A diamond is born as a brand. An extreme, immutable brand that can only be destroyed by destroying the diamond itself. And finally — and perhaps most importantly — a diamond’s value has nothing to do with its material constitution. Carbon is among the commonest elements on earth. A diamond’s value is entirely based on the immense amounts of energy required to fuel the process that creates it. They are found in places where deep, high-energy violence has occurred, such as the insides of volcanic pipes.

Diamonds are forever. They cannot be drained of history. When you break up a diamond — you cannot add diamonds — the pieces have less value than the whole. Diamonds can only be worked in irreversible, destructive ways.

Diamonds represent a becoming kind of value; the products of creative destruction. If you’ve read my Be Slightly Evil newsletter issue, Be Somebody or Do Something, you know the symbolism I am getting at here. You also know where my sympathies lie.

I find it particularly amusing that the value of gold is measured in purity carats, while the value of diamonds is measured in weight carats. Purity and weight are what are known as intensive and extensive measures. The size of a diamond is a measure of the quantity of tectonic violence that created it.

I prefer diamonds to gold, perhaps because I am not an original thinker, but a creative-destructive one. I am not very good at discovering rare things. I am better at applying intense pressure to commonplace things, in the hopes of producing a diamond. Sometimes I stumble upon natural rough diamonds, but more often, I attempt to manufacture artificial ones from coal. They are not as pretty, and it is very hard to manufacture large ones, but when I succeed, I produce legitimate diamonds, born under pressure.

Gold, I rarely mine myself (and earth-bound humans cannot manufacture it; only dying suns can). I buy gold in the form of second hand jewelry at the bookstore, melt it down, and rework into other things. Most often, into settings for diamonds.

Introductions are how unsociable introverts do social capital. Community building is for extroverts. But introductions I find stimulating. Doing them and getting them. This is probably a direct consequence of the type of social interaction I myself prefer. My comfort zone is 1:1, and an introduction is a 3-way that is designed to switch to a 2-way in short order, allowing the introducer to gracefully withdraw once the introducees start talking. As groups get larger than two, my stamina for dealing with them starts to plummet, and around 12, I basically give up (I don’t count speaking/presentation gigs; those feel more like performance than socializing to me).

I am pretty good at introductions. I’ve helped a few people get jobs, and helped one entrepreneur raise money. Off the top of my head, I can think of at least a half-dozen very productive relationships that I have catalyzed. I think my instincts around when I should introduce X to Y are pretty good: 2 out of 3 times that I do an introduction, at the very least an interesting conversation tends to start. Since I’ve been getting involved in a lot of introductions lately, I thought I’d share some thoughts based on my experiments with introductions.

Weak-Link Hubs vs. Strong-Link Hubs

Introductions are the atomic unit of social interaction. They are central to the creation and destruction of communities, but aren’t themselves a feature of communities. Rather they drive the creative destruction process within the universe of communities, as Romeo and Juliet illustrates particularly well. Introductions are constantly rewiring the social graph, causing old communities to collapse and new ones to cohere.

To understand how introductions work, you have to understand a subtle point: stereotypical extroverted community types are actually pretty bad at introductions, except for one special variety: introducing a newcomer into an existing group, as a gatekeeper.  Stereotypical host/hostess community types are great at helping existing communities grow stronger and endure. Their social behaviors are therefore in direct conflict with uncensored introduction activity, which causes social creative destruction to intensify. I call the stereotypical community types strong-link social hubs. They know everybody in a given local (physical or virtual) community.  They are a friend, mentor or mentee to every individual within that community. They are the ultimate insiders. When  a strong-link social hub makes an introduction, it is usually quick and superficial, “I am sure you two will find that you have a lot in common, you’re both engineers!” Or the half-joking “everybody this is X; X this everybody, ha ha!” Enough to sustain party conversations, but usually not enough to catalyze relationships except by accident.

The real hubs of introduction activity on the social graph though, are what I call weak-link hubs. It is both a personality type and a structural position in the social graph. It is easiest for me to explain what this means via a personal anecdote.

When I was a kid in high school, I resisted being sucked into any particular group.For their part, the 2-3 major groups in my class saw me as a puzzle: I was not “one of us” or “one of them.” Neither was I one of the social outcasts. I did 1:1 friendships or hung out occasionally as a guest in groups, but I rarely joined in group activities.

One day, I remarked to a friend, “I guess I am equally inside all the groups.” His retort: “No, you are equally outside all the groups.” I realized that not only was he right, that was pretty much my identity. It hardened into a sort of reactionary tendency towards self-exile (one of my nicknames in college was “hermit”) that has stayed with me. Whenever I find myself getting sucked too deeply into any group, I automatically start withdrawing to the edge. Physically, if the group is in a room.

That is what I mean by weak-link hubs being both a personality type and a structural position. You have to have the personality that makes you retreat from centers and you have to have centers around you to retreat from. This retreat is an interesting dynamic. You cannot really be attracted to the edge around a single center, since that is a diffuse place. But if you are retreating simultaneously from multiple centers, you will find yourself a position in the illegible and chaotic intersection lands. Why illegible? Try drawing a random set of overlapping circles and making sense of the pattern of intersections. Here’s an example:

This “retreating from all nearby centers” is not exactly the personality description of a great social hub. So why is it a great position for introduction-making? It’s the same reason Switzerland is a great place for international negotiations: neutrality and small size anchoring credibility, but with sufficient actual clout to enforce good behavior. If you are big or powerful, you have an agenda. If you are from the center of a community, you have an agenda.  Another great example is the Bocchicchio family in The Godfather: not big enough to be one of the Five Families, but bloody-minded enough to effectively play intermediary in negotiations by offering themselves up as hostages.

Edge Blogging and the Introduction Scaling Problem

This post actually grew out of a problem I haven’t yet solved. My instincts around introductions aren’t serving me well these days. Over the last few months, the number of potential connection opportunities that go above my threshold triggers has been escalating. Two years ago, I’d spot one potential connection every few months and do an introduction.  Now I spot one or two a week, and it’s accelerating. I am getting the strange feeling that I might turn into one of those cartoon characters at a switchboard who starts out all calm and in control and is reduced to crazed scrambling. In case it isn’t obvious, the growth of ribbonfarm is the driver that is creating this scaling problem.

The answer is obvious for extroverts: create a community and start dealing with people in one-to-many and many-to-many ways in group contexts. This allows you to simply create a social field around yourself where people can connect without overt catalysis from you. The cost is that you must turn yourself into a human social object. You must become a new center. You will no longer be in the illegible intersection lands where creativity and originality live. Call me selfish, but that’s the big reason I don’t like the idea that readers frequently propose: formal ribbonfarm meetups or an online “ribbonfarm community.”

The anatomy of the problem is simple. Blogging is often an edge role. If you see a blog that sprawls untidily across multiple domains rather than staying within a tidy niche, chances are you are reading an edge blog. They tend to be small and slow-growth, with weird numbers in their traffic anatomy.

The social graph of an edge blogger is very different from the social graphs of both celebrities and regular people without much public visibility.  Regular people have many active strong links and many more weak links that used to be strong links (old classmates, colleagues from former jobs and the like). For regular people weak links are usually either strong links weakened by time or intrinsically weak links catalyzed by a short sequence of strong links (like a friend-of-a-friend or an in-law). In both cases, the weak links of regular people tend to be quiescent.

Celebrities on the other hand have a huge number of active weak links, but they only go one way: a lot of people know Obama but Obama doesn’t know 99.9999% of them.  Even if you count only those who have shaken hands with Obama, the asymmetry is still massive. Center bloggers are effectively celebrities. In fact they often are celebrities who have taken to blogging, like Seth Godin.

Edge bloggers though are an odd species. They are perhaps most like professional headhunters, used car salesmen or other types of people who regularly come into weak two-way contact with total strangers. Unlike those rather transactional roles though, bloggers do a whole lot of weak social rather than financial transactions with a lot of total strangers. Many of you (I’ve lost count) have ongoing email conversations with me, usually about a specific theme that I’ve blogged about or mentioned somewhere online (container shipping, martial arts, organizational decay and s/w design are some of the themes). The intensity ranges from several times a week to once every couple of months (for the infrequent ones, I usually have to do an inbox search to remember who the person is). With some correspondents, I have periodic bursts of activity. With a small handful of people, thanks to phone or face-to-face meetings, I have made the jump to actual friendship.

Edge bloggers are natural weak link hubs. We have vastly more active two-way weak link relationships going on than regular people or celebrities (or center bloggers). These are not forgotten classmates or friends-of-friends who can be called upon when you are job-hunting. Nor are they one-way-recognition handshakes.

I got a visceral sense of what it means to be a weak-link hub when I compared my LinkedIn graph visualization to that of a couple of “regular people” friends. Though my friends had comparable numbers of contacts, most of their contacts fell into very obvious small-world categories, like workplace, school, customers or industry associations. My social graph on the other hand, has a huge bucket that I could only label “miscellaneous.” Many are from ribbonfarm, but I suppose my “weak link hub” style carries over to regular life as well. For instance, I have a lot more random connections to people in widely separated parts of Xerox, my former employer, compared to most of my former coworkers.

Keeping Edges Edgy

Make no mistake, this is fun for me and hugely valuable. But I have to admit, it takes a lot of time to keep up a whole bunch of 1:1 email relationships, and it is getting steadily harder. So far, my clean-inbox practices have helped me keep up, but there has some of the inevitable increase in response time and sometimes decrease in my response quality.

The big temptation is of course to ignore my personality and preferences and allow ribbonfarm to become a “center.” It’s not necessarily a bad thing. You trade off continued creativity and vitality for deeper collaborative cultivation of established value. I don’t like doing that much. I get distracted too quickly. My brain is not built for depth in that sense, even around things I trigger, like the Gervais Principle memeplex.

The conundrum is that I don’t think raising the threshold for “potential connection quality” is the right answer. That’s the wrong filter variable for scaling. I am not sure what the right one is, but I won’t attempt to jump to synthesis. So far, I’ve simply been letting a steadily-increasing fraction of introduction opportunities simply go by. Mostly I try to avoid making introductions to people who are already oversubscribed.

Though I don’t have a theory, I do have one heuristic that serves me well though: “closer potential direct connection.” If I know A and B, and I sense that A and B would have a more fertile relationship with each other than either has with me, I make the connection and exit. It is the opposite logic of marketplaces whose organizers are afraid of disintermediation. To me being an intermediary in the social sense is mostly costs and little benefit.

But that one heuristic isn’t enough. I have experimenting with introductions in different ways lately, and learning new ideas and techniques.

Here’s one new idea I’ve learned. To keep edges edgy, and prevent them from becoming centers, you need feedback signals. One I look for is symmetry. “Introducer” types tend to be “introducees” equally often.  If the ratio changes, I get worried.

As an illustration of the symmetry of this process of mutual cross-catalysis among sociopath weak-link hubs, consider this, while I was conducting my experiments with introductions, others have been introducing me to their friends. Hang Zhang of Bumblebee Labs introduced me to Tristan Harris, CEO of Apture and Seb Paquet formally introduced me to Daniel Lemire (who I knew indirectly through comments on each other’s blogs, before but had never directly emailed/interacted with).

We are all lab rats running in each others mazes. I like that thought.

This is a guest post by Paula Hay, from Rabbit Mountain. I don’t yet have a proper guest-posting policy in place, but if you’d like to guest-post, email me.

Venkat’s recent post The Disruption of Bronze touched on a subject I’ve been pursuing fervently for the better part of a decade now: the time frame in which psychologically modern humans evolved. More than that, however, my interest is in why and how human psychology shifted to cause the sudden, radical changes that ultimately resulted in civilization.

My view is that without an understanding of this shift, there can be no evolution beyond the devouring, predatory virus that is civilized culture. In a mere 10,000 years, civilization has all but wrecked the planet — a truly impressive horror.

Collapse (of either the slow or sudden variety, take your pick) is a certainty, in my opinion; what I needed, for my own sanity, was a context in which to fit this state of affairs. Does the story really begin and end with American avarice? Are humans condemned to repeat the rise-and-fall of civilizations until we wipe ourselves out for the last time? Is there no greater narrative arc here?

Civilizations rise and fall not in isolation, but as complexes. They follow the outbreak of certain memes, as evidenced by the archaeological record, in clusters of time and geography. In the West we humans do civilization not only because of what we think, but because we think our thoughts in a specific kind of way. It makes sense then that the narrative arc should begin with the emergence of our specific kind of thinking.

Pre-Conquest Consciousness

Anthropologist E. Richard Sorenson is best known in academic circles for pioneering what’s known as “visual anthropology”: the use of non-dialectic observational techniques in the field of anthropology, most often through the use of film. Academics are, however, notorious for missing the forest for the trees; Sorenson’s real contribution came as a result of his techniques.

Visual anthropology made it possible for Sorenson to identify patterns of behavior inherent across isolated, unrelated, primitive tribes. Underlying these behavioral patterns is a type of mindset which Sorenson calls “pre-conquest consciousness,” which he describes thus:

Most of us know about subliminal awareness—the type of awareness lurking below actual consciousness that powerfully influences behavior. Freud brought it into the mainstream of Western thought through exhaustively detailed revelations of its effects on behavior. But few, including Freud, have spoken of liminal consciousness, which is therefore rarely recognized in modern scholarship as a separate type of awareness. Nonetheless, liminal awareness was the principal focus of mentality in the preconquest cultures contacted, whereas a supraliminal type that focuses logic on symbolic entities is the dominant form in postconquest societies.

. . .

From the Latin language underlying our Western heritage we can understand that liminal awareness, by definition, occurs on the threshold of consciousness. This concept, though abstract, provides a useful term. In the real life of these preconquest people, feeling and awareness are focused on at-the-moment, point-blank sensory experience—as if the nub of life lay within that complex flux of collective sentient immediacy. Into that flux individuals thrust their inner thoughts and aspirations for all to see, appreciate, and relate to. This unabashed open honesty is the foundation on
which their highly honed integrative empathy and rapport become possible. When that openness gives way, empathy and rapport shrivel. Where deceit becomes a common practice, they disintegrate.

Where consciousness is focused within a flux of ongoing sentient awareness, experience cannot be clearly subdivided into separable components. With no clear elements to which logic can be applied, experience remains immune to syntax and formal logic within a kaleidoscopic sanctuary of non-discreteness. Nonetheless, preconquest life was reckoned sensibly—though seemingly intuitively.

Given the widespread nature of Sorenson’s findings, and the almost complete absence of supraliminal symbology in any given culture’s archaeological record prior to its own Neolithic Revolution, it would appear that this liminal consciousness is the default psychology of anatomically modern humans. “Pre-conquest” peoples do not have the capacity for intellectual abstraction, not because they are less intelligent — the homo sapiens sapiens brain has not changed physically for something like 200,000 years — but because their mental capabilities are focused entirely on the here and now. Gods and goddesses, writing, numbers and the like cannot exist in the “complex flux of collective sentient immediacy” because they have no physicality with which to be either sentient or immediate. Such things exist entirely in the abstract. They are, for all intents and purposes, not real.

Animism

If liminality is the default mode of consciousness for pre-civilized humans, animism is the default context which liminal consciousness observes. Animism is not a religion — liminality affords no intellectual soil for such a thing — but is rather a comprehensive weltanschauung based entirely on the observable immediate.

The most concise definition of animism I can find at the moment comes from a (probably less than reliable) website called themystica.com:

The term animism is derived from the Latin word anima meaning breath or soul. The belief of animism is probably one of man’s oldest beliefs, with its origin most likely dating to the Paleolithic age. From its earliest beginnings it was a belief that a soul or spirit existed in every object, even if it was inanimate. In a future state this soul or spirit would exist as part of an immaterial soul. The spirit, therefore, was thought to be universal.

We Westerners are inclined to view such things as childish. But ecological philosopher David Abram, in his amazing book The Spell of the Sensuous, makes the animist worldview more apprehensible:

The next morning I finished the sliced fruit, waited for my hostess to come by for the empty bowl, then quietly headed back behind the buildings. Two fresh palm-leaf offerings sat at the same spots where the others had been the day before. These were filled with rice. Yet as I gazed at one of these offerings, I abruptly realized, with a start, that one of the rice kernels was actually moving.

Only when I knelt down to look more closely did I notice a line of tiny black ants winding through the dirt to the offering. . . The line of ants seemed to emerge from a thick clump of grass around a nearby palm tree. I walked over to the other offerings and discovered another line of ants dragging away the white kernels. . . There was an offering on the ground by a corner of my building as well, and a nearly identical line of ants. I walked into my room chuckling to myself: the Balian and his wife had gone to so much trouble to placate the household spirits with gifts, only to have their offerings stolen by little six-legged thieves. What a waste! But then a strange thought dawned on me: what if the ants were the very “household spirits” to whom the offerings were being made?

I soon began to discern the logic of this. . . The daily gifts of rice kept the ant colonies occupied — and, presumably, satisfied. Placed in regular, repeated locations at the corners of various structures around the compound, the offerings seemed to establish certain boundaries between the human and the ant communities; by honoring this boundary with gifts, the humans apparently hoped to persuade the insects to respect the boundary and not enter the buildings.

Yet I remained puzzled to my hostess’s assertion that these were gifts “for the spirits.”… While the notion of “spirit” has come to have, for us in the West, a primarily anthropomorphic or human association, my encounter with the ants was the first of many experiences suggesting to me that the “spirits” of indigenous culture are primarily those modes of intelligence or awareness that do not possess a human form.

It is probably impossible for someone acculturated to Western thought to truly grasp an animist worldview. But from what I have learned, animism seems to be the way in which liminal consciousness interacts with its environment. The “complex flux of collective sentient immediacy” in which liminality occurs cannot be separated out of its surroundings — therefore everything is part of the sentient collective, including trees, rocks, animals, rain, natural phenomenon such as thunder, and ants. Liminality perceives consciousness in everything because in the absence of abstract logic — or even the ability to conceive of abstract things — universal consciousness is really and truly what is observable.

The Tigris-Euphrates River Valley

We’ve all grown up hearing that the Tigris-Euphrates river valley, located in what is now Iraq, is the “cradle of civilization.” In fact, it is the cradle of only one: Western civilization, and the long line of civilizations that rose and fell along our cultural family tree.

The time frame in question is approximately 15,000-10,000 years ago. As far as I have been able to determine, the West’s pre-civilized, cultural ancestors lived at this time with the same liminal-animist psychology as hunter-forager peoples everywhere. The archaeological record demonstrates no compelling reason to conclude otherwise.

The climate during this time experienced three major fluctuations. The Older Dryas stadial saw a cooling that lasted about 400 years, from 14,000-13,600 years ago. The climate then warmed for about 700 years during the Allerød interstadial, which lasted from about 13,600-12,900 years ago. Then came the Younger Dryas stadial, when the climate cooled again for approximately 1,260 years, from 12,900-11,640 years ago (or 10,900-9,640 BC).

The end of the Younger Dryas stadial coincides with the beginning of the Neolithic Revolution in the Tigris-Euphrates river valley. The currently prevailing theory is that as the Younger Dryas took hold in southwest Asia, hunter-foragers were forced into a more settled, horticultural lifestyle in response to thinning animal herds and vegetation upon which they previously depended. When the climate again warmed, this horticultural adaptation blossomed into full-blown agriculture, which subsequently made larger settlements, cities, and eventual empires possible.

But correlation is not causation. To our cause-and-effect psychology, it makes sense that a climate shift cause would precede an agricultural effect. This may or may not be accurate. Archaeology, geology, climatology and paleoanthropology can offer measurements from which we infer a storyline; but as science goes, that storyline is ever-changing as new measurements come to light, and these changes themselves also may or may not be factually accurate. With science, “facts” of history — especially prehistory — change rather frequently, and result in a permanent state of never-quite-sure.

If only there was some record of what people living at the time of the Neolithic Revolution were actually thinking.

Genesis Reconsidered

The much-reviled political philosopher Leo Strauss pointed out that Western societies are descended from two parents, Athens and Jerusalem. In my quest for understanding this subject, I came to see that while Athenian science and logic can provide a what, they cannot provide the why. That why, for Western seekers anyway, lies in Jerusalem.

Jerusalem’s contribution to the West is Judeo-Christian mythology, codified in the Bible. And to be clear: when I use the term “mythology,” I do not mean “fiction.” I mean a stable, apprehensible narrative by which an entire culture — or in this case, multiple very large cultures spanning thousands of years — can orient itself in the universe temporally, spatially, and spiritually. A myth’s power lies not in its objective “truth,” “accuracy,” or some such thing, but rather in its ability to reflect a shared psychology back to its collective adherents. In the West, our scientific “truths” still follow the same basic pattern of Judeo-Christian mythology and are subject to its cognitive structure. Psychology constructs myth, and myth constructs reality. Myth trumps measurement, regardless of whether or not anyone notices.

Genesis is the first book in the Bible. It is comparable to other early Mesopotamian writings in that many aspects are paralleled in the texts of the Sumerians and Akkadians. Genesis, however, is a profoundly Jewish work and offers unique criticisms that, as far as I am aware, do not exist in contemporaneous literature.

The literary content of Genesis spans a vast amount of time prior to the advent of writing. This content presumably has its roots in the oral traditions of those hunting-foraging forerunners of the ancient Hebrews. These traditions, as recorded in Genesis, begin in pre-literate, liminal-animist Paleolithic psychology, bridge the advent of supraliminal Neolithic psychology, and continue on in (sometimes violent) tension between the two. With the advent of writing, the traditional stories were written down, carefully guarded, and passed on through generations by ancient scribes (though the earliest known manuscripts date no earlier than 150 BCE), eventually becoming the foundation of the West’s most revered holy book.

We forget, or perhaps cannot fathom, that the stories of Genesis were produced by a mindset completely alien to our own. Religious literalists think Genesis is a moment-by-moment, play-by-play account, like a sporting event. Others consider it so old as to be irrelevant; still others dismiss it outright, mistaking “myth” for “fiction.”

In fact, Genesis is more relevant to the West than almost anyone has noticed. Its ancient stories of the Garden of Eden, the Fall, and Cain and Abel trace the cognitive development of Western civilization more concisely and accurately than I would have thought possible. The trick is to approach the stories on their own terms; to set aside our own supraliminal psychology and unpack these stories into a liminal-animist headspace.

Evidence Of Cultural Memories?

The main concern of Genesis is the Fall and its consequences, and how these drove the birth of the ancient Hebrew culture. The creation account and life in the Garden of Eden are presented almost as backstory, but this backstory is where the fascination begins.

Genesis 2:8 tells us casually, as if in passing, that “[T]he LORD God had planted a garden in the east, in Eden; and there he put the man he had formed.” The implication is that “the man” was originally located to the west: had he been located to the north, the garden would have been in the south; had he been to the east, the garden would have been in the west, and so forth.

About 50,000–70,000 years ago, anatomically modern humans began a wave of migration out of Africa. Some crossed the Red Sea, greatly reduced in width due to increased glaciation, then continued migrating along the southern coasts of Asia, eventually landing on the Australian continent. Others crossed the Sinai Peninsula and settled in the Levant — the western portion of the inverted boomerang known as the “fertile crescent.” It is not much of a stretch to imagine that some of this population also traveled a few hundred miles due east to the Tigris-Euphrates river valley.

By way of another example, Genesis 2:19–2:20 states, “Now the LORD God had formed out of the ground all the wild animals and all the birds in the sky. He brought them to the man to see what he would name them; and whatever the man called each living creature, that was its name. So the man gave names to all the livestock, the birds in the sky and all the wild animals.”

One of the prevailing theories of language evolution is called monogenesis, meaning that all human languages trace their origins to a single common linguistic ancestor dubbed “Proto-Human.” No one knows for sure where languages come from, but if Proto-Human ever existed, it would have arisen sometime after the first known anatomically modern human, about 200,000 years ago, and could have survived until 50,000 years ago, when the emergence of complex language would have made it obsolete.

Does Genesis actually record cultural memories of such great age? Can we really consider these things to be a kind of cognitive fossil? These are only two examples, but they offer synchronicity enough to give pause.

More importantly, these two snippets provide a kind of primer for translating the ancient texts from liminal-animist to supraliminal. Certainly it is not literally true that God planted a garden in the east and placed man there. This understanding is based in the animist worldview of a universal spirit giving rise to all other spirits, i.e., the flora and fauna of a rich river valley, and all action, i.e., migration to an eastern garden. In the liminal-animist oral tradition of our cultural ancestors, this story is literarily accurate.

In the same way, surely no God literally brought all the wild animals and the birds of the sky to a single man to be named. Rather, the development of complex language occurred in “that complex flux of collective sentient immediacy,” which included both the animals in question and, again, the universal spirit. Like the previous example, this one, too, is literarily accurate.

This literary accuracy is the method by which pre-literate cultures preserve themselves. Their myths, legends and stories do not add up in the logical, cause-and-effect way Western minds are accustomed to. But this doesn’t mean they are inaccurate; it simply means we don’t understand their accuracy unless we try.

The Fall Of Man

The most pressing story in Genesis, its very raison d’être, is the Fall of Man. No other story has had as profound an effect on Western psychology as this; there is something wrong with us and we know it, and this is the only extant explanation of what the problem might be.

But I’m convinced that no interpretation to date captures either the point of the story or its gravity. Whereas the Fall narrative is abstract and symbolic, it originated from what I strongly believe to have been real events that took place, possibly over the course of several generations, in the liminal-animist’s “complex flux.” The story is utterly nonsensical without that context.

Here is the full text of the Fall of Man story, from Genesis 2:25 – 3:24:

[25] Adam and his wife were both naked, and they felt no shame.

Genesis 3

[1] Now the serpent was more crafty than any of the wild animals the LORD God had made. He said to the woman, “Did God really say, ‘You must not eat from any tree in the garden’?”

[2] The woman said to the serpent, “We may eat fruit from the trees in the garden, [3] but God did say, ‘You must not eat fruit from the tree that is in the middle of the garden, and you must not touch it, or you will die.’”
[4] “You will not certainly die,” the serpent said to the woman. [5] “For God knows that when you eat from it your eyes will be opened, and you will be like God, knowing good and evil.”

[6] When the woman saw that the fruit of the tree was good for food and pleasing to the eye, and also desirable for gaining wisdom, she took some and ate it. She also gave some to her husband, who was with her, and he ate it. [7] Then the eyes of both of them were opened, and they realized they were naked; so they sewed fig leaves together and made coverings for themselves.

[8] Then the man and his wife heard the sound of the LORD God as he was walking in the garden in the cool of the day, and they hid from the LORD God among the trees of the garden. [9] But the LORD God called to the man, “Where are you?”

[10] He answered, “I heard you in the garden, and I was afraid because I was naked; so I hid.”

[11] And he said, “Who told you that you were naked? Have you eaten from the tree that I commanded you not to eat from?”

[12] The man said, “The woman you put here with me—she gave me some fruit from the tree, and I ate it.”

[13] Then the LORD God said to the woman, “What is this you have done?”
The woman said, “The serpent deceived me, and I ate.”

[14] So the LORD God said to the serpent, “Because you have done this,
“Cursed are you above all livestock
and all wild animals!
You will crawl on your belly
and you will eat dust
all the days of your life.

[15] And I will put enmity
between you and the woman,
and between your offspring and hers;
he will crush your head,
and you will strike his heel.”

[16] To the woman he said,
“I will make your pains in childbearing very severe;
with painful labor you will give birth to children.
Your desire will be for your husband,
and he will rule over you.”

[17] To Adam he said, “Because you listened to your wife and ate fruit from the tree about which I commanded you, ‘You must not eat from it,’
“Cursed is the ground because of you;
through painful toil you will eat food from it
all the days of your life.

[18] It will produce thorns and thistles for you,
and you will eat the plants of the field.

[19] By the sweat of your brow
you will eat your food
until you return to the ground,
since from it you were taken;
for dust you are
and to dust you will return.”

[20] Adam named his wife Eve, because she would become the mother of all the living.

[21] The LORD God made garments of skin for Adam and his wife and clothed them. [22] And the LORD God said, “The man has now become like one of us, knowing good and evil. He must not be allowed to reach out his hand and take also from the tree of life and eat, and live forever.” [23] So the LORD God banished him from the Garden of Eden to work the ground from which he had been taken. [24] After he drove the man out, he placed on the east side of the Garden of Eden cherubim and a flaming sword flashing back and forth to guard the way to the tree of life.

The mythology here is quite mysterious until one considers that it is an account of the shift from liminal-animist to supraliminal cognition among our cultural forebears. And far from being an evolutionary step forward, our mythology tells us that it was an unmitigated disaster, a corruption so vile that God threw us out of the garden nixed our access to the Tree of Life — that is, continuity across time, or sustainability.

The story also describes the exact nature of this transition. It was, first and foremost, a voluntary transgression against the only existing taboo in the garden. “The tree of the knowledge of good and evil” was, quite explicitly, a different cognitive-psychological way of knowing that was not to be brought into the “complex flux” of liminal awareness. Doing so would, and did, result in death (more on this in a moment).

Curiously, the immediate aftermath of Adam and Eve’s transgression was, shall we say, something of an anticlimax. One might expect fire and brimstone to rain down from the heavens, but instead, they simply “realized they were naked; so they sewed fig leaves together and made coverings for themselves.” This hardly seems the stuff of eternal damnation. What gives?

Everyone has had the experience of dreaming oneself naked in public. The dream is distressing precisely because we are surrounded by others. Yet when we’re alone with a lover, we feel no shame because we are physically united — or in Biblical poetry, we are “one flesh.”

The Fall story opens with Adam and Eve standing shamelessly naked in the garden. They perceived no “others” from whom to hide their nakedness, for all are one in the “complex flux” that arises from the universal spirit. The very first effect of the Fall — that is, the shift from liminal to supraliminal consciousness — was the perception of “others” in the environment where there had previously been none.

But even the change of perception from a world unified by one universal spirit into one separated into self/other is still not the same as knowledge of good and evil. Or is it?

The tree of the knowledge of good and evil is surrounded by dualism from its first mention. Everything in the garden was allowed, except this tree, which was not-allowed. At the tree we discover the serpent, the first symbolic not-God. The serpent introduces the concept of not-true, which had evidently never before occurred to anyone — something Sorenson demonstrated in his field work among pre-conquest cultures. And upon eating the fruit of the tree, Adam and Eve suddenly perceive others, or not-selves, surrounding them in the garden.

I propose that what was unleashed that day in the garden was, in effect, cognitive binary signal processing: the ontological 0 was born into the human psyche where previously only 1 had ever existed. Suddenly, everything perceivable now has an infinite number of not- counterparts. The horror paleolithic people must have experienced upon stumbling into such an awareness is difficult to fathom. Knowledge of good and evil, indeed.

Consequences

This shattering of the unary whole into infinitude is the basis of what we call information. Prior to the shift from liminal to supraliminal, there was only very limited information because, as Sorenson points out, “With no clear elements to which logic can be applied, experience remains immune to syntax and formal logic.”

Once our cultural ancestors had information with which to contend, they had no choice but to start classifying and indexing everything, lest the universe remain illegible. Thus the archaeological record demonstrates a veritable explosion of symbols — everything from pictographs, to numbers, to gods and goddesses, abstract concepts like rulership, wealth, and servitude, and all the rest.

Driving the deluge of information was the existential 0 threatening to annihilate anything anyone discovered or created that might be considered 1. The only defense against 0 was, and remains, more1 — the problem, however, is that just as 0 = not-1, so too 1 = not-0. Not- permeates everything; annihilation is not only inevitable, it is already done. The more 1 we think we create, the more annihilation we introduce into the complex flux that continues to exist outside our awareness.

Westerners are obsessed with apocalypse for exactly this reason. The ancient myth of Genesis states very clearly that crossing the threshold from liminal to supraliminal would result in certain death — and this is no fiction, for we contend with death on countless levels, from our smallest spiritual micro lives to the largest physical macro ecosystems, every single day. Every step we take to stave off death for ourselves introduces it somewhere else, making our own continuance even less likely.

Our 1s and 0s will not save us. I hold out hope that someone wise will discover a cognitive signal processing in which 1 annihilates 0, or perhaps subsumes it, but so far the glimmers of such that I’ve seen have fallen on infertile soil. I am open to any and all suggestions.

We spend a lot of our lives doing nothing. Doing nothing is usually viewed as wasting time, and there are two ways it can be done. When you waste your own time, it’s called idleness. When others waste your time, it’s called waiting. I enjoy idleness.  I don’t like waiting.

Wasted time is not empty time. Empty time is meditation. You could argue that meditation is about subjective time standing still. Your productive potential, in theory, is either preserved or enhanced through empty do-nothing.  Wasted time is also not the same as recovery, relaxation or recharge time. That’s about using this minute to make another minute more potent.

Wasting time requires putting pointlessly dissipative activity into it. An annoying argument with an idiot about something that doesn’t matter, that ends up frustrating you, is a good example. You actively destroy the productive potential of time.  I like doing that sometimes.

Many are disturbed and offended by the very idea of wasting time. There is a beautiful bit in John Updike’s Rabbit series, where Rabbit Angstrom’s young girlfriend Jill  accuses him of having a “Puritan fear of waste.”

I used to be like Rabbit. Over the years, I’ve gotten increasingly comfortable with wasting time through idleness. I am still not comfortable with others wasting my time by having me wait, however. I suppose I am not evolved or ego-free enough for that.

Our culture of work is designed around wasting time for others. And it is not just waiting in queues, or waiting for important people who are running late for their appointment with you. That’s merely status-waiting. Robert Levine, in The Geography of Time, has a beautiful discussion of the interplay of status and waiting in different cultures.

I am talking about more deep-rooted ways of wasting time.

Paychecks for creative information work, unlike paychecks for hands-on or routine information work, are not about buying work. A paycheck represents an option, but not an obligation, on the part of an employer, to get value out of purchased time.

This so offends the work ethic of many that they’d rather manufacture highly-energetic and apparently productive ways of wasting time than enjoyable and openly pointless ways. Those who insist on the former love busywork. Those who prefer the latter are part of the  retired at work movement. Stanley Bing is the Messiah of this movement.

Employers recognize this deep-seated need to be “productive,” and encourage cultures of busywork over cultures of retired-at-work. This despite the fact that enlightened employers of information workers know they are buying time options rather than time stock, and that those comfortable being retired at work are more productive. So they take on the burden of creating busywork support systems.

Since busywork is by definition unproductive, you cannot find material evidence that it produces anything of value. You must look for social proof. Those who manufacture busywork therefore, do so in social ways, creating collective anxiety complexes to validate the value of each others output in circular ways.

Those who are caught in busywork economies usually recognize that they are really in a holding pattern, waiting for something to actually happen in their lives. But most of the time, they manage to forget it. That’s why they need a good deal more recharge/recovery time than either the productive or the idle. Busywork is vastly more exhausting than either. Waiting is existentially costly. Fred Wilson once said that the iPad is about reducing waiting costs, and he is right. The device lowers status-waiting costs in queues and waiting rooms.

Isn’t it ironic, by the way, that the industrial age created a proliferation of “waiting rooms?”

Unfortunately the iPad does nothing to lower busywork costs.

Fortunately, idleness costs nothing. You do not need to lower the costs of idleness because there aren’t any. If I ever run a big organization, I’ll have idling rooms at random locations, instead of waiting rooms in front of the offices of important executives.

The non-paycheck world is not different, since the same work-ethic anxieties operate everywhere.  There is enough room in the murky art of setting hourly consulting rates and pricing project proposals that you can adopt any time-wasting philosophy your anxieties demand. I’ve met consultants who bombard their clients with more busywork output than the most refined bureaucrat, and I’ve met other consultants who spend most of their time doing nothing, but still earn about the same as the ones who manufacture paper. For the latter model, you don’t need to rely on complex global labor arbitrage schemes of dubious morality to create your 4-hour work week. Most kinds of creative information work allow you to do so.

In the world of work, the great divide between those who choose to waste time by waiting, and those who choose to do so through idleness, is far more important than the more nominal divide between paycheck types and free agent types.

On the idle side of the fence, I have met many who like 10 hours of idleness for 30 hours of productive work. These are the high-energy dynamos. I’ve also met others, like myself, who like a ratio of about 30 hours of idleness to every 10 hours of productive work. I am fine with people who follow either pattern, though the low-idleness types tend to run ahead and leave me behind.

It is waiting that bothers me. Others usually don’t know how to waste my time properly. I have to do it myself.

Our methods of measuring and valuing time are very primitive. As information work becomes increasingly about creativity and ideas, the time-value-of-money equation is breaking down badly. But those who are comfortable with idleness have an advantage over those who enjoy waiting.

It is always interesting to recognize a simple pattern in your own thinking. Recently, I was wondering why I am so attracted to thinking about the margins of civilization, ranging from life on the ocean (for example, my review of The Outlaw Sea) to garbage, graffiti, extreme poverty and marginal lifestyles that I would never want to live myself, like being in a motorcycle gang. Lately, for instance, I have gotten insatiably curious about the various ways one can be non-mainstream. In response to a question I asked on Quora about words that mean “non mainstream,” I got a bunch of interesting responses, which I turned into this Wordle graphic (click image for bigger view)

Then it struck me: even in my qualitative thinking, I merely follow the basic principles of mathematical modeling, my primary hands-on techie skill. This interest of mine in “non mainstream” is more than a romantic attraction to dramatic things far from everyday life. My broader, more clinical interest is simply a case of instinctively paying attention to what are known as “boundary conditions” in mathematical modeling.

Mathematical Thought

To build mathematical models, you start by observing and brain-dumping everything you know about the problem, including key unknowns, onto paper.  This brain-dump is basically an unstructured take on what’s going on. There’s a big word for it: phenomenology. When I do a phenomenology-dumping brainstorm, I use a mix of qualitative notes, quotes, questions, little pictures, mind maps, fragments of equations, fragments of pseudo-code, made-up graphs, and so forth.

You then sort out three types of model building blocks in the phenomenology: dynamics, constraints and boundary conditions (technically all three are varieties of constraints, but never mind that).

Dynamics refers to how things change, and the laws govern those changes. Dynamics are front and center in mathematical thought. Insights come relatively easily when you are thinking about dynamics, and sudden changes in dynamics are usually very visible.  Dynamics is about things like the swinging behavior of pendulums.

Constraints are a little harder. It takes some practice and technical peripheral vision to learn to work elegantly with constraints. When constraints are created, destroyed, loosened or tightened, the changes are usually harder to notice, and the effects are often delayed or obscured. If I were to suddenly pinch the middle of the string of a swinging string-and-weight pendulum, it would start oscillating faster. But if you are paying attention only to the swinging dynamics, you may not notice that the actual noteworthy event is the introduction of a new constraint. You might start thinking, “there must be a new force that is pushing things along faster” and go hunting for that mysterious force.

This is a trivial example, but in more complex cases, you can waste a lot of time thinking unproductively about dynamics (even building whole separate dynamic models) when you should just be watching for changes in the pattern of constraints.

Inexperienced modelers are often bored by constraints because they are usually painful and dull to deal with. Unlike dynamics, which dance around in exciting ways, constraints just sit there, usually messing up the dancing. Constraints involve and tedious-to-model facts like “if the pendulum swings too widely, it will bounce off that wall.” Constraints are ugly when you first start dealing with them, but you learn to appreciate their beauty as you build more complex models.

Boundary conditions though, are the hardest of all. Most of the raw, primitive, numerical data in a mathematical modeling problem lives in the description of boundary conditions. The initial kick you might give a pendulum is an example.  The fact that the rim of a vibrating drum skin cannot move is a boundary condition. When boundary conditions change, the effects can be extremely weird, and hard to sort out, if you aren’t looking at the right boundaries.

The effects can also be very beautiful. I used to play the Tabla, and once you get past the basics, advanced skills involve manipulating the boundary conditions of the two drums. That’s where much of the beauty of Tabla drumming comes from. Beginners play in dull, metronomic ways. Virtuosos create their dizzy effects by messing with the boundary conditions.

In mathematical modeling, if you want to cheat and get to an illusion of understanding, you do so most often by simplifying the boundary conditions.  A circular drum is easy to analyze; a drum with a rim shaped like lake Erie is a special kind of torture that takes computer modeling to analyze.

A little tangential kick to a pendulum, which makes it swing mildly in a plane, is a simple physics homework problem. An off-tangent kick that causes the pendulum bob to jump up, making the string slacken, before bungeeing to tautness again, and starting to swing in an unpleasant conic, is an unholy mess to analyze.

But boundary conditions are where actual (as opposed to textbook) behaviors are born. And the more complex the boundary of a system, the less insight you can get out of a dynamics-and-constraints model that simplifies the boundary too much. Often, if you simplify boundary conditions too much, the behaviors that got you interested in the first place will vanish.

Dynamics, Constraints and Boundaries in Qualitative Thinking

Without realizing it, many smart people without mathematical training also gravitate towards thinking in terms of these three basic building blocks of models. In fact, it is probably likely that the non-mathematical approach is the older one, with the mathematical kind being a codified and derivative kind of thinking.

Historians are a great example. The best historians tend to have an intuitive grasp of this approach to building models using these three building blocks.  Here is how you can sort these three kinds of pieces out in your own thinking. It involves asking a set of questions when you begin to think about a complicated problem.

1. What are the patterns of change here? What happens when I do various things? What’s the simplest explanation here? (dynamics)
2. What can I not change, where are the limits? What can break if things get extreme? (constraints)
3. What are the raw numbers and facts that I need to actually do some detective work to get at, and cannot simply infer from what I already know? (boundary conditions).

Besides historians, trend analysts and fashionistas also seem to think this way. Notice something? Most of the action is in the third question. That’s why historians spend so much time organizing their facts and numbers.

This is also why mathematicians are disappointed when they look at the dynamics and constraints in models built by historians. Toynbee’s monumental work seems, to a dynamics-focused mathematical thinker, much ado about an approximate 2nd order under-damped oscillator (the cycle of Golden and Dark ages typical in history). Hegel’s historicism and “End of History” model appears to be a dull observation about an asymptotic state.

How the World Works

In a way, the big problem that interests me, which I try to think about through this blog, is simply “how does the world work?”

At this kind of scale, the hardest part of building good models is actually in wrestling with the enormous amount of “boundary conditions” data.  That’s where you either get up off the armchair, or turn to Google or Amazon.  Thinking about boundary conditions — organizing the facts and numbers in elegant ways — becomes an art form in its own right, and you have to work with stories, metaphors and various other crutches to get at the right set of raw data to inform your problem. Only after you’ve done that do dynamics and constraints get both tractable and interesting.

Abstractions and generalizations, if they can be built at all, live in the middle. Stories live on the periphery.

This is part of the reason I don’t like traditional mathematical models at “how the world works” scale, like System Dynamics. They ignore or oversimplify what I think is the main raw material of interest: boundary conditions. A theory of unemployment, slum growth and housing development cycles in big cities that ignores distinctions among vandalism, beggary and back-alley crime is, in my opinion, not a theory worth much. If you could explain elegantly why some cities in decline turn to crime, while others turn to vandalism or beggary, then you’d have interesting, high-leverage insights to work with.

It’s not surprising therefore, that one of the most seductive ideas in abstract thinking about history, the deceptively simple “center periphery” idea (basically, the idea that change and new historical trends emerge on the peripheries and in the interstices of “centers”) is extremely hard to analyze mathematically, since it involves a weird switcheroo between boundary conditions and center conditions. Some day, I’ll blog about center-periphery stuff. I have a huge, unprocessed phenomenology brain-dump on the subject somewhere.

So in a way, thinking about things like the words in the graphic is my way of wrapping my mind around the boundary conditions of the problem, “how does the world work?”  If I just made up a theory of the mainstream world based on mainstream dynamics, it would be very impoverished. It would offer an illusion of insight and zero predictive power.  A theory of the middle that completely breaks down at the boundaries and doesn’t explain the most interesting stories around us, is deeply unsatisfying.

I have proof that this approach is useful. Some of my most popular posts have come out of boundary conditions thinking. The Gervais Principle series was initially inspired by the question, “how is Office funny different from Dilbert funny?” That led me to thinking about marginal slackers inside organizations, who always live on the brink of being laid off. My post from last week, The Gollum Effect, came from pondering extreme couponers and hoarders at the edge of the mainstream.

So I operate by the vague heuristic that if I pay attention to things on the edge of the mainstream, ranging from motorcycle gangs to extreme couponers and hoarders, perhaps I can make more credible progress on big and difficult problems.

Or at least, that’s the leap of faith I make in most of my thinking.

It just struck me that the Ben Franklin quote, “early to bed, early to rise, makes a man healthy, wealthy and wise,” implies a great principle of leverage to apply to your resolutions. The easiest way to visualize this is using Maslow’s hierarchy of needs.

Modifying behaviors at lower levels automatically improves behaviors at higher levels. So your resolutions should be as highly-leveraged as possible. Call the layers of the pyramid P, S, L, E and A. Compute your leverage as follows:

• A: 1 point
• E: 2 points
• L: 4 points
• S: 8 points
• P: 16 points

Your leverage is your total points divided by the number of resolutions. So the example above has a leverage of (1+2+4+8+16)/5=31/5=6.2. Do the math. If your resolutions aren’t sufficiently leveraged, reframe them to move them to lower levels.

A word to the wise, I hope, is sufficient. I am sure you can work out the benefits of leveraged resolutions for yourself.

Regular scheduled programming of 1000+ word posts will resume shortly. Happy New Year!

Over the Thanksgiving weekend, I tried, and failed, to relax. I am sure I am not alone, and that many of you had the same experience. But I failed in a very revealing way, that led me a very interesting definition of work.

What happened was this:

I was reading a book to relax (Robert D. Kaplan’s excellent Monsoon: The Indian Ocean and the future of American power). It was pure relaxation in the sense that the subject has nothing to do with either my work or subjects I normally blog about (my other “job”). But a few chapters in, something very interesting happened: I suddenly decided I might want to blog about the book. And just as suddenly, a relaxing experience turned into “work,” and within a half-hour, I felt I needed a “relaxation break.”  So what happened?

Consider what hard work used to mean. If you worked on an assembly line 40 hours a week, cranking widgets, you were working. If you put in 20 hours of overtime to make it 60 hours, you were working hard. Get up to 80 hours, and you were killing yourself.

The point is, crank-widget work is easy to define and measure in objective terms. Information work is different. Drucker offered the correct, but mostly useless idea that part of information work is defining the work to begin with, which makes it very ambiguous. Call it define-and-do work.

Crank-widget jobs do exist in information work too. Making sales calls for products with a proven sales script, or routine types of trading on Wall Street are crank-widget jobs because there are measurable, tightly self-contained units of work (calls/trades) that have already been defined. And those professions tend to have a “work hard/play hard” culture because the boundary between work and non-work is clear. You can actually decide to “stop work for the day.” There may be no punch-clocks, but it is punch-clock work. And this can be true of even very complex types of information work, so long as it is formulaic (which is a synonym for “pre-defined” if you think about what “formulaic” actually means). The formula may take a PhD and lots of specialized knowledge to execute, but if there is a formula, it’s crank-widget work.

But when you are doing define-and-do Druckerian work, it is basically impossible to decouple definition from execution a priori in useful ways (“my job is to define my job and do it” — sounds like a GNU recursive acronym, doesn’t it?). “Work” is whatever the hell you need to do to “get the job done.” A non-constructive definition.

But even if you can’t define this kind of define-and-do  information work, you can detect when you are doing it.  Here’s how you detect it:

It is work if there is a customer other than yourself.

Or to put it more explicitly: It is work if it will impact something that will be evaluated by others, and if their reactions will have consequences for you that you care about (If you don’t care how they react, they are by definition not a customer).

That’s why I experienced the sudden attitude shift while reading the book. Suddenly, I went from not caring how sloppily or carefully I read the book or what ideas I picked up, to thinking in terms of the needs of a blog post that others would evaluate.

It has nothing to do with whether or not you enjoy the work (as I do blogging). People who claim they love their work so much “it doesn’t feel like work” are deluding themselves. They’ll still be work-tired and need to “unwind” when they quit.

This definition can lead to some very surprising conclusions. George Lucas said, a few years ago, that he’d made so much money now that he could “afford to make bad movies for the rest of my life.” This illustrates one surprising conclusion. Finishing the Star Wars franchise was “work” for him even if he didn’t need the money, because the evaluation of the audience mattered, since in a sense the original trilogy was part of publicly-owned social capital, and if they rejected the prequels he was attempting to tack on, he’d have felt a personal sense of failure. But now that’s he done, he really can afford to ignore the audience.

By the same standard, you sometimes need a  vacation to recover from a vacation if the original vacation involved meeting family expectations. I am sure that’s especially true for many Americans who are returning after stressful family weekends. That’s a “work” vacation because between the turkey and the pumpkin pie, you may have had to justify your career/life to your Dad, and the reaction mattered. If this is true for you, then for better or worse, your Dad is your customer for a product you are creating called “my life.”

The level of apparent hard work and enjoyment is irrelevant. I once went on a hard 10 day hiking trip, where we’d cover miles of mountain trails every day, for 6-7 hours at a stretch. I’d be exhausted, but when we camped for the night, I felt relaxed. I felt unwound, rather than feeling a need to unwind.

There have been times when I’ve written some simulation code just for the hell of it, to have some fun for myself. For example, I once wrote a little simulation of a sky-writing airplane that would fly around and write any text string I gave it on the screen; thing took hours, and I never properly finished it, but it was relaxing. And I could abandon the project halfway precisely because there was no external party in the loop who cared.

And on the other end of the spectrum, I often have entire days now when I do absolutely nothing except sit around/walk around and think. No observable output. Sometimes not even working notes or scribbles. Just thinking. And at the end of the day, I am utterly  exhausted, because I’ve been thinking about work-related problems where any decisions I take as a result of my thinking will have consequences to be evaluated by others.

The definition “others will evaluate the consequences, and their reaction will have consequences for you” basically says it is work if there is a feedback loop that matters, that goes through an external party.

What causes the stress that makes it “work” is a combination of two factors. First, since you define what to do, there are no natural limits. You can define your work to be as simple or complicated as you like. Second, since the reaction of an unpredictable external party is going to determine how you feel about the work, there is a reason for you to endlessly work to try and lower the anxiety about the unpredictability. In other words, there is an infinitely-stretchable component to the work (the definition), and a fundamentally unknowable component to its consequences (the external reaction).

When crank-widget work fails to satisfy the customer, you can say “not in my job description” and pass the buck.

When define-and-do work fails to satisfy the customer, that’s not possible.

This definition gives us the easiest way to measure how hard you are working. Take 24 hours, subtract sleep time, subtract the time you are focused on doing something where there is no customer. The rest is work. It’s far easier to spot the negative space than to spot the positive space.

And by that standard, many of us are working 80-90 hour weeks, even if it doesn’t feel like it.

This also gives us the best way to relax: shorten all feedback loops as much as possible, and stop work when a round of feedback is in, and before you begin the next define-and-do iteration. If you routinely work in 3 month loops, you’ll never sleep.

Unfortunately, for truly complex work, there is a limit to how much you can shorten the customer loops for the things you are doing, no matter how much agile iteration you build in.

Sometimes a single phrase will pop into my head and illuminate a murky idea for me. This happened a few days ago. The phrase was “ancient rivers of money” and suddenly it helped me understand the idea of inertia as it applies to business in a deeper way. Inertia in business comes from predictable cash flows. That’s not a particularly original thought, but you get to new insights once you start thinking about the age of a cash flow.

We think of cash-flow as a very present-moment kind of idea. It is money going in and out right now. But actually, major cash flow patterns are the oldest part of any business. It is the very stability of the cash flow that allows a business to form around it. In fact, most cash flows are older than the businesses that grow around them. They emerge from older cash flows.  When you buy a sandwich at Subway, the few dollars that change hands are part of a very ancient river of money indeed. Through countless small and large course changes, the same river of money that once allowed some ancient Egyptian to buy some bread from his neighbor now allows you to buy a sandwich.

Buyers and sellers alike see markets as an illegible and turbulent churn of transaction opportunities. But really, they are landscapes carved out by great, ancient rivers of money and their tributaries. These rivers change course rarely. Cash flows are also among the most basic financial ideas. Only businesses make profits, but governments and non-profits form around cash flows too.

These ancient rivers carve out both a spatial and temporal landscape. Spatially, the flow metaphor suggests old, dried-up river beds, gorges and ravines, flood plains, ox-bow lakes, watersheds, and of course, the rivers themselves. This plays well with the idea of “segment.”

But markets also have a temporal dimension, based on which river of money you are talking about, and how long ago it last changed course.

If you think of markets that way, things look very different. Some rivers of money are very old and very stable. You can at most fight to displace others from prime positions along the banks. Others are new and unstable and may change course frequently, creating and destroying fortunes through their vagaries. Others may be maturing, with dams being built to stabilize them. People have always bought food and clothes. They are only now beginning to buy iPads. They are starting to not buy CDs.

Generalizing, you can even think of an average “age” of the market as a whole.  An interesting question to ask is whether early adopters as a group should be considered as living in a future market, or whether the mainstream should be thought of as living in the past. I prefer the latter model.

Organizations are like riverbank communities. They are as old as the last significant course change or waterfront battle. The stability of the river, not the attitudes of people, is what makes old organizations seem set in their ways.  Perhaps people resist new ideas not because they have specific personalities, but because they have settled on the banks of a river of money of a certain age.  Or perhaps there is self-selection. Possibly the hidebound kinds go settle on the banks of the most ancient rivers. Tax rivers are among the oldest and most stable rivers of money (and the only ones protected by the threat of legitimate force), and people attracted to government work aren’t exactly known for being passionate champions of creative destruction.

Some startups are about finding and colonizing the banks of minor unknown tributaries of old rivers. Others are about creating new rivers. Still others are about building canals between vigorous new rivers and somnolent old ones.  And of course, there are those that are about displacing incumbents from prime waterfront locations.

The nice thing about thinking this way is that the market is now a system of cash flows that exists independently of the specific set of businesses serving it in a given era. You can map the system and look for an unoccupied waterfront spot.

I would like to create a visualization of the oldest and most stable rivers of money, around things like food, clothing, taxes and shelter. I don’t know how to do that yet.

I first mentioned the metaphor of money as a system of flows (with things like glaciers mapping to frozen assets) in my old post, Fools and their Money Metaphors, and this particular one stuck in my head. Then in a comment to my Eight Metaphors of Organization post, a reader used the phrase “high inertia cash flow.”

When I first read that comment, an image popped into my head unbidden: a dark subterranean cavern with a river flowing through, with goblin-like creatures swarming around it, holding torches. Like Gringott’s bank in the Harry Potter movies. “Ancient” is how I would describe the feel of that image.

I’d like a t-shirt  with a skull-and-crossbones below graphic and the line “Don’t touch my cash flows!” below it. The attitude pretty much defines anybody who is effective in the world of business. When you meet a tough, no-bullshit businessperson, no matter what function they come from, chances are, they see their job as protecting a cash flow.

Long-time reader and astute commenter, Xianhang (Hang) Zhang wrote a very interesting post a couple of weeks ago on his blog: The Evaporative Cooling Effect. It is part of a fascinating series he is doing on social software. The post explores a phenomenon that is very close to the status illegibility phenomenon I explored two weeks ago, and in fact draws inspiration from the same Groucho Marx/Lake Wobegon observations that I started with.

Evaporative cooling is basically the effect of the highest status people in a group leaving, lowering the average status of those left behind.

What I found fascinating though, was Hang’s suggestion for how to combat the effect (and thereby stabilize groups). In my post, I proposed that status illegibility helps create the stability. Hang brings in another dimension, which is illegibility in the group’s environment/context.

In particular, in social software (or physical environments for that matter), smarter-than-average early adopters often leave when the “unwashed masses” start to jump on the bandwagon, devaluing the social cachet. Hang proposes that one of the best ways to combat this is to build (or rather catalyze the evolution of) “warren” architectures instead of “plaza” architectures. Here are the pictures that pair of evocative terms produces in my head. You might imagine something else:

Warrens vs. Plazas

A warren is a social environment where no participant can see beyond their little corner of a larger maze. Warrens emerge through people personalizing and customizing their individual environments with some degree of emergent collaboration. A plaza is an environment where you can easily get to a global/big picture view of the whole thing. Plazas are created by central planners who believe they know what’s best for everyone. The terms are very evocative, and should remind you of the idea of legibility in physical environments that we talked about recently, in my post A Big Little Idea Called Legibility. In fact, it wouldn’t be a gross oversimplification to say that warrens and plazas differ primarily in their legibility. There are many subtleties of course.

The warren/plaza concept also sheds new light on one of my oldest posts, Harry Potter and the Cuaron Slam. In that (rather murky) post, I argued that not The Prisoner of Azkaban was the best Harry Potter book, and that Alfonso Cuaron’s movie version surpassed the book. I can now summarize that whole post very briefly with the warren/plaza concept. The entire Harry Potter series is of poor literary quality because most of the books are very plaza-like overly-legible books. There is none of the atmosphere of mystery you get with warren-like books (the Lord of the Rings for instance). The overarching central-planning map overwhelms narratives and character arcs. The 3rd book is the only warren-like book.

Two pictures from that old post get at a different aspect of warrens and plazas. I offered an analogy between the book/movie context and plot and robot path planning algorithms, which come in two basic varieties: bird’s-eye-view map navigation, and the more difficult worm’s eye view navigation where the robot only knows what it can see and remember from a ground view (the main algorithm for doing this is called SLAM, hence the bad pun in the title). The second picture illustrates how SLAM-style navigation works: you gradually build up a map of your environment by remembering what you see in your field of view as you navigate.

Plaza path planning for robots

Warren navigation for robots

Again, the warren/plaza concept is useful.  Plaza navigation is easier, but less powerful, since it requires global information (which is only easily available with plazas). Warren navigation is much harder but much more powerful, since it does not require global information.  In warren navigation, learning is a necessary feature, since you cannot plot a shortest path  a priori. You need to explore and stumble and build up a map while groping towards the goal.

In the movie version, Cuaron’s camera work is literally warren like. Most of the shots are ‘follow the action’ ground-level shots. You feel like a mouse following the action rather than an eagle.

Hang offers Facebook and Quora as examples of highly “warren” like social sites. He also asserts that social sites that fail at a particular scale due to evaporative cooling are typically plazas.

The Edge of Legibility

I think there is a very fascinating line of thought here. I haven’t yet worked out the details, but here’s a potentially powerful conjecture that suggests where to go: social systems that thrive and grow are on the edge of legibility.

All kinds of legibility: status legibility, environmental legibility and probably a couple of other kinds. If they become too legible, they fail in the Seeing Like a State mode or through evaporative cooling.  If they become too illegible, they fail by ossifying, with highly ritualized and sacramental cultures, with a great deal of environmental irrationality and very few entries/exits.

The idea isn’t particularly original. In complex systems, an idea that was very popular in the early 90s (to the point of being a fad) was that rich, growing complex systems were on “the edge of chaos” (a state called self-organized criticality, SOC). Around the same time, in biology, Santa Fe theorist Stuart Kauffman proposed that biological systems self-organize in ways that keep them on the edge of criticality with respect to key chemical autocatalysis reactions (so for example, multicellular organisms evolved because the chemical soup inside a too-large single-celled creature would become supercritical and spiral out of control, while too-small cells would lack the critical concentrations of key molecules to get to autocatalysis at all).

This line of research isn’t dead; in fact it is steering towards the group dynamics stuff we are talking about. In the 2000s, a lot of complexity theorists got interested in what is known as the El Farol Bar Problem (and a related idea called the Minority Game). This line of work explores phenomena that are very strongly related to the Groucho Marx and Lake Wobegon effects. Surprisingly, the right approach to these problems seems to involve methods from statistical physics.

Resurrecting Self-Organized Criticality?

What’s new in what we are discussing here is the idea that complex systems that are also recursively self-aware in some ordinary sense (i.e. we are not talking “meaning of consciousness” here, but merely the ability to act on the basis of models of yourself, your group, and your environment; this can be programmed in simple AIs today) will differ from the old 90s style SOC systems in a critical way: their behavior will also be driven by the legibility of the situation to the various self-aware agents, and self-aware collectives of those agents, and so on recursively outward to the whole dimly self-aware global system.

So there may be an interesting model of social systems (both descriptive and prescriptive) lurking underneath this qualitative discussion: a combination of growth rates, “cell division” rates that keep a system on the edge of legibility that allow baby plazas to gradually morph into adult warrens, with the level of legibility for all actors itself driving the system’s evolution. Call it “Legibility-Modulated Ontogenic Evolution of Self-Organized Critical Systems.”

That sounds like the title of a paper I’d like to write. Pity I have no time any more to indulge in such things.

I am pulling ideas from a lot of different places here, so if you are interested in following this train of thought,  follow this Edge of Legibility trail starting with a review of the Self-Organized Criticality idea. Warning: this is a total geek-fest, and if you aren’t interested in the complexity theory side to many of the things I talk about on this blog, you should probably ignore this. But you should know that there is a radically different way of thinking about all the stuff I talk about in the Gervais Principle series and other posts on this blog, and that the other way is actually more natural for me. It just isn’t as much fun to read.

Another Personal Note

A couple of readers found the personal note I inserted into my last post interesting, so I’ll throw another one in here. Back before I went over to the dark side and actually did hands on technical work, the mathematical aspect of this kind of stuff was my main gig. During the early part of my grad school career, I was fascinated by the complexity theory work, which the mainstream of my field (systems and control theory) regarded as soft, and slightly disreputable interdisciplinary TV science (Santa Fe has a far bigger reputation in pop science than in mainstream academia).

Eventually the fascination led me to steer my PhD direction away from classical aerospace systems/control problems and towards complex systems. But I was uninspired by the main focus of the complexity theory world (things like sand piles, very simple agents like ants, traffic jams and so forth).  For both pragmatic reasons (you can’t manufacture a defensible PhD thesis in aerospace engineering by studying sand piles) and personal reasons, I studied things like formation flight and teaming algorithms for autonomous aircraft and spacecraft (very much the bandwagon research topic in the late 90s/early 00s). For my postdoc, between 2004 and 2006, I went even deeper into such stuff and did a lot of work on mental models and how individual decision makers cooperating or competing in a battlefield might act together given different views of the same world.

It was fascinating stuff, and I had a great deal of fun doing it. It was also stuff that was positioned perfectly to fall right through the cracks between the fields of AI, operations research, cognitive science and control theory. I even threw in bits and pieces of linguistics and the philosophy of language. And “falling through the cracks” is what happened. Interdisciplinary research sounds sexy to people outside the world of professional research, but it is an extremely risky thing to do, especially early in your career. This despite the fact that funding agencies clamor for interdisciplinary work. The publish or perish equation gets massively loaded on the perish side, and I perished.

The majority of researchers manage to dress up their work as “interdisciplinary” while still staying close to the lower-risk core of their disciplinary fields, while a minority take the risk of actually being interdisciplinary and do things so stunning that things work out despite the risk (these are the true pioneers who help reshape the academic landscape). Universities scramble to create positions for these pioneers, and they deserve it.

I wasn’t smart enough to pretend and dress up conservative work as interdiscplinary, and not the sort of genius who could actually make it work. I did good work, but I’ll be the first to admit it wasn’t powerful enough to reshape the landscape in ways that could create room for me.

My interests didn’t align with any established funding sources or publishing channels, so one fine day, after mulling over a pile of faculty position rejection letters and a few adjunct/second postdoc offers that would have had me in a holding pattern over the tenure landing track for a couple more years, I decided to call it quits. I nursed my academia sour grapes for a few weeks, got drunk, got over myself, and headed into industry. This was 2006.

It was probably the best decision of my life. When I look at my peers who are now in faculty positions I realize that I could never do what they do so well. They actually enjoy academic publishing and navigating the warrens of the funding and publishing ecosystems, rather than merely enduring the game as I did.  In a way, the free-form medium of blogging is my home territory, where I can basically do what I like without worrying about publish-or-perish pressures. I have no idea how I deluded myself for so long that I was actually cut out for the academic life.

I do regret that the blogging medium is not friendly to more mathematical exploration though (turning the sorts of ideas I explore in this blog into equivalent mathematical problems would take 100x more time and effort than just writing about them qualitatively).  Maybe someday I’ll be able to retire rich and early, scrape the rust off my math and programming skills, and get back in the game on my own terms.

Anyway, here endeth the geek fest. We’ll get back to your regularly scheduled programming next week.

“Good is the enemy of great” is an insight that a lot of people have stumbled upon, though I can’t trace the origin of the phrase.  It might be Jim Collins’ Good to Great, but I am not sure. A hint about the dynamics are in that book (again, an insight I’ve heard elsewhere): good people with a bad process will always beat incompetent people working with a good process.

The clue is in the word process. Process is how good becomes the enemy of great. And I mean process in its most general form, not just the rigid bureaucratic stereotype. So a specific portfolio analysis technique for picking stocks to maximize some risk/returns function, or any sort of “methodology” is a process. A 12-step program is a process. A “Maximize Your Creativity” book that deals in colorful balls and right-brained art exercises is still a process. “Be agile and improvise” is also a process. If it can be defined and written down as a prescription, with any kind of promise attached, it is a process.

Here’s why this happens. Processes (and systems) of any sort first emerge when a spectacular and undisciplined success occurs. Like a startup — XYZ Corp. say, getting wildly successful. Or the PQR basketball team racking up a string of victories. Or an actor making it big in Hollywood. First, there’s a success that attracts imitative greed. Then something very predictable happens. A “great” story is retold in ways that only capture the “good” part.

“Great” success breeds inefficiency and waste, and somebody brings order into the chaos by imposing thoughtful systems and processes based on on 20/20 hindsight. These systems and processes, if they are well-designed, codify the knowledge that is chaotically distributed in the successful non-system/growth trajectory. They eliminate certain risks and redundancies, and build in some insurance. So you get ideas and intelligence moving from the heads of creative people and into rules and procedures. If you’ve been following this blog recently, this should remind you of the idea of legibility.

This is where the problem starts. Two additional things happen (this is going to sound slightly similar to Gladwell’s Outliers and like Nicholas Nassim Taleb’s critique of The Millionaire Next Door, but I am making a somewhat different argument):

1. The people behind the original success discount the amount of luck, special conditions and randomness involved in the success, and tell stories (unconsciously or consciously) designed to minimize factors besides their own contribution. You get “How I Did It” books from the CEO of XYZ Corp., and the coach of the PQR team.
2. The codified processes undergo a further evolution and become ritualized, and you get process books like “The XYZ Way” or “The PQR Formula for Winning!” But this book, it is important to note, is not a “How to evolve systems and processes like ours” book. It is an “if I could do it again” book. It has to be, otherwise you are reduced to telling people: “to be successful, be me.” So rather than getting the unreconstructed messy story of how chaos turned into order, you get an apparently logical backward extrapolation of the final state. We get the speculative assertion that the end-state process, with its checks and protections could have emerged more efficiently with a given hypothetical storyline. To take a simple example, if you grew a successful blog, and adding an email list in year 3 helped you really get to the next level, you say something like, “if I had to do it again, I’d start growing an email list from Day 1.”

The first kind of manufactured history creates a false sense of predictability, destiny and inevitability. You get the “origin myth” type stories. The second kind of “how to” cookbook offers an improved, more efficient, repeatable formula based on a messy template. Often you’ll find both in the same book. Both types of narratives take this equation:

Great = Good + Luck

And turn it into this one, by dropping “luck.”

Great = Good

In other words, they describe “good” but call it “great.” Because it isn’t fun to admit that the difference is often just luck, and that truly “great” is actually quite rare (I don’t think, for instance, that Einstein was just a lucky Poincare). But this is where the naive can get caught in a bait-and-switch. They think they are buying a formula for “great” and end up executing in best faith and getting trapped in the merely “good.”

If you must, you could say “Openness to Luck” instead of “Luck,” and that was the point of an article I wrote a couple of years ago, The Fine Art of Opportunism. I’ve since gotten a little sadder and more realistic about the world. I still stand by that article, and the model it proposes about how “luck” operates, but I believe that not everybody is lucky enough to be in places where it is possible to manufacture your own luck by being opportunistic.

Censoring Path Depedency and Initial Condition Dependency

Here’s where it gets interesting: the two elements conspire. The self-serving history hides the special conditions (if your success can be explained by a time/place/opportunity, in the sense of Gladwell’s Outliers, there’s less autobiography to write, isn’t there? Those story elements, if they are offered, are offered as “human interest” elements rather than causal elements).  The cookbook on the other hand, creates a conservative formula that ostensibly is about “learning by example.” By promising a less messy (“learn from our mistakes”) version of XYZ’s path, without the “inefficient” twists and turns, and by promising the beautiful and efficient end state that currently exists, a path-dependent history culminating in a desirable steady state gets retold as a path-independent story. Worse, you get the suggestion that it is the end-state that contains the value, rather than the truth: most of the value was banked along the way. The end-state merely milks already-won assets efficiently.

That last point bears repeating: the end-state is not where the value is. The end-state is a hard-won and defensible value-adding position. The value was banked along the way.  The first person who climbs a mountain the hard way gets famous. If he later suggests a road building route and others can now drive to the top, they are going to get far less value. Most of the value the original climber banked was due to the path he took. There are many roads to the top of a mountain, but the view is not always the same.

See what happened here? The manufactured history removes dependency on initial conditions. The “cleaned up” process-evolution prescription removes path dependency. Value-attribution is shifted from process states to final outcome state.

A story that is neither necessary, nor sufficient to explain wild success has become both necessary and sufficient. “If you do it this way, you WILL succeed, and this is the only way you CAN succeed.” Few authors are quite that naively deterministic, but you get almost that promise.

That’s actually the definition of a process: a manufactured, self-serving history justifying a conservative, “cleaned up” necessary-and-sufficient “do-over” evolution path that ends in a promised high-value-adding state that isn’t.

Quite a nice piece of sleight of hand, isn’t it?

What actually happened: luck, special conditions, and talents conspired to create a messy story that was neither necessary nor sufficient, and led to a high-value position, with most of the value already added, and a state that efficiently milks that position for a while.

But why is this an explanation of “Good is the Enemy of Great?”

Fooling Some of the People Some of the Time

Why do even smart people get taken in by a fake necessary-and-sufficient argument?

Here’s why: even though the initial conditions and path-dependency have been hidden, simply through how imitation works, most attempts to repeat the formula will share some (but crucially, not all) of the undocumented enabling conditions. If the manufactured story and cookbook emerge out of a “real and messy” story that played out with a Chinese hero in Taiwan, well, other Chinese wannabe-greats in Taiwan are the most likely mimics. So some of the enabling conditions are repeated even if nobody acknowledges that they matter. That may be enough for many people to get to “good.”

So if you follow the “great” formula faithfully you will accidentally build walls that prevent your own bits of luck and serendipity from getting through. It is only by creating your own bloody mess that you will be responding to your own unique local conditions and environments, and the lucky breaks your unique initial conditions and unique path offer.

Because you are following the straight-and-narrow version of a tortuous original, and with blinkers on, you don’t see your opportunities. Yes, you are enjoying the benefits of starting your email list on Day 1, but so is every other imitator. But what you lose is the things you might have spotted if you’d chosen to be a little messier. A little more focused on your situation as opposed to somebody else’s playbook.

So you get “good” instead of “great” to the extent that undocumented local conditions are shared. Eventually the local conditions get saturated with formula repeaters, and people start to apply the formula too far afield, and you get failure instead of even “good.” The fad is over.

Those who drive to the top of the mountain also get a good view, though they don’t get the great view at the end of the first climb, or the fame. And of course, if too many people are driving to the top of a given mountain, the overcrowding itself ruins the view.

There’s one more twist to the tale. The probability and information theorists among you will want to dismiss this analysis as a complicated way of saying “high risk/high returns gets converted to low risks/low returns, followed by diminishing returns, as information spreads through a system.

But there’s a subtlety. “Great” stories are not merely high risk/high returns. They seem to display certain asymmetries. They are stories that are more open to good luck than bad luck compared to random stories.

The asymmetry is created by the path dependence and initial conditions. You get information out of your environment and that allows you to swing things in favorable ways. But some bad luck is blocked out by a formula. Your unique formula created by rule-breaking within an older formula.

Selective Rule Breaking

I’ve been criticizing formulas and praising “messy.” To a certain extent, I am advocating methodological anarchy.

But this does not mean “be stupid and random.”

The real secret to getting from “good” to “great” is selective rule breaking. And this “secret” isn’t a formula (that would be ironic, wouldn’t it?) because I am offering a completely non-constructive observation. I have no idea what rules you’ll need to break and how or why. I couldn’t write a “how to break rules” book. Here’s my non-formula:

1. Look for the nearest “good” formula and skeptically deconstruct and reconstruct to recover the hidden initial conditions and path dependencies
2. Check if you are close enough; if yes that’s the formula you are going to break.
3. Now selectively break the formula using every bit of special-local-conditions information you have access to.

That’s how you get the rewarding asymmetric openness to luck. It is a very rude and disrespectful process. You read the formula-books and self-serving biographies like a psychoanalyst, teasing out the special conditions and the real story that might not make the hero look so good. You map yourself to those conditions. Then you look at your exclusive information advantage and decide which rules in the formula the information allows you to ignore.

This by the way, seems to be a common characteristic of “great” as opposed to “good” imitators. “Good” imitators either try and achieve modest success, or fail, by applying formulas religiously. But the “greats” find “good” formulas to break. I haven’t completely clarified the connection to imitation theory to my satisfaction though. Perhaps the way to say this is “good is the parent of great.” Both parent and enemy.

There is no guarantee that this non-formula will work, but if you despise “good” and aspire to “great” and can accept “failure,” at least it won’t fail in the “good is the enemy of great” mode of mediocrity. You don’t know whether you can make it rich in the stock market, but you do know that keeping your savings in cash won’t make you rich.

And if there’s no formula worth breaking in your neighborhood, welcome to pioneer country. All bets are off.

This is an abstract and advanced (i.e. “overcomplicated”) version of an idea I am trying to simplify and illustrate with better examples in my book. Sign up for the book release announcement list if you haven’t already, though I think most regulars already have.

Sometimes I get annoyed by all the pious statistician-types I find all around me. They aren’t all statisticians, but there are a lot of people who raise analytics and “data-driven” to the level of a holy activity. It isn’t that I don’t like analytics. I use statistics whenever it is a “cost of doing business.” You’d be dumb to not take advantages of ideas like A/B testing for messy questions.

What bothers me is that there are a lot of people who use statistics as an excuse to avoid thinking. Why think about what ONE case means, when you can create 25 cases using brute force, and code, classify, cluster, correlate and regress your way to apparent insight?

This kind of thinking is tempting, but  is dangerous. I constantly remind myself of the value of the other approach to dealing with data: hard, break-out-in-a-sweat thinking about what ONE case means. No rules, no formulas. Just thinking. I call this “learning from one data point.” It is a crucially important skill because by the time a statistically significant amount of data is in, the relevant window of opportunity might be gone.

Randomness and Determinism

The world is not a random place. Causality exists. Patterns exist. In grad school, I learned that there are two types of machine learning models in AI. Models based on reasoning, and models based on statistics and probability. This applies to both humans and machines. Both are driven by feedback, but one kind is driven mainly by statistical formulas, while the other kind is driven by thinking about the new information.

The probability models, like reinforcement or Bayesian learning, are very easy to understand. They involve a few variables and a lot of clever math, mostly already done by smart dead people from three centuries ago, and programmed into software packages.

The reasoning models on the other hand, are complex, but largely qualitative, and most of the thinking is up to you, not Thomas Bayes.  Explanation-Based Learning is one type. A slightly looser form is Case-Based Reasoning. Both rely on what are known as rich “domain theories.” Most of the hard thinking in EBL and CBR is in the qualitative thinking involved in building good domain theories, not in the programming or the math.

The former kind requires lots of data involving a few variables. Do people buy more beer on Fridays? Easy. Collect beer sales data, and you get a correlation between time t and sales s. Gauss did most of the necessary thinking a couple of hundred years ago. You just need to push a button.

EBL, CBR and other similar models are different. A textbook example is learning endgames in chess. If I show you an endgame checkmate position involving a couple of castles and a king, you can think for a bit and figure out the general explanation of why the situation is a checkmate. You will be able to construct a correct theory of several other checkmate patterns that work by the same logic. One case has given you an explanation that covers many other cases. The cost: you need a rich “domain theory” — in this case a knowledge of the rules of chess. The benefit: you didn’t waste time doing statistical analyses of dozens of games to discover what a bit of simple reasoning revealed.

Looser case-based reasoning involves stories rather than 100% watertight logic. Military and business strategy is taught this way. Where the explanation of a chess endgame could potentially be extended perfectly to all applicable situations, it is harder to capture what might happen if a game starts with a “Sicilian defense.” You can still apply a lot of logic and figure out the patterns and types of game “stories” that might emerge, but unlike the 2-castles-and-king situation, you are working in too big a space to figure it all out with 100% certainty. But even this looser kind of thinking is vastly more efficient than pure “brute force” statistics-based thinking.

There’s a lot of data in the qualitative model-based kinds of learning as well, except it’s not two columns of x and y data. The data is a fuzzy set of hard and soft rules that interact in complex ways, and lots of information about the classes of objects in a domain. All of it deployed in the service of an analysis of ONE data point. ONE case.

Think about people for instance. Could you figure out, from talking to one hippie, how most hippies might respond to a question about drilling for oil in Alaska? Do you really need to ask hundreds of them at Burning Man? It is worth noting that “random” samples of people are extraordinarily hard to construct. And this is a good thing. It gives people willing to actually think a significant advantage over the unthinking data-driven types.

The more data you have about the structure of a domain, the more you can figure out from just one data point. In our examples, one chess position explains dozens. One hippie explains hundreds.

People often forget this elementary idea these days. I’ve met idiots (who shall remain unnamed) who run off energetically do data collection and statistical analysis to answer questions that take me 5 minutes of careful qualitative thought with pen and paper, and no math.  And yes, I can do and understand quite a bit of the math. I just think 90% of the applications are completely pointless. The statistics jocks come back and are surprised that I figured it out while sitting in my armchair.

The Real World

Forget toy AI problems. Think about a real world question: A/B testing to determine which subject lines get the best open rates in an email campaign. Without realizing it, you apply a lot of model-based logic and eliminate a lot of crud. You end up using statistical methods only for the uncertainties you cannot resolve through reasoning. That’s the key: statistics based methods are the last-resort, brute force tool for resolving questions you cannot resolve through analysis of a single prototypical case.

Think about customer conversations. Should you talk to 25 customers about whether your product is good or bad? Or will one deep conversation yield more dividends?

Depends. If there is a lot of discoverable structure and causality in the domain, one in-depth customer conversation can reveal vastly more than 25 responses to a 3 question survey. You might find out enough to make the decision you need to make, and avoid 24 other conversations.

But it takes work.  A different kind of work. You can go have lunch with just ONE well-informed person in an organization and figure out everything important about it, by asking about the right stories, assessing that person’s personality, factoring out his/her biases, applying everything you know about management theory and human psychology, and spending a few hours putting your analysis together. You won’t produce pretty graphs and “hard evidence” of the sort certain idiots demand, but you will know. Through your stories and notes, you will know. And nine times out of ten, you’ll be right.

That’s the power of one data point. If you care to look, a single data point or case is an incredibly rich story. Just listen to the story, tease out the logic within it, and you’ll learn more than by attempting to listen to fifty stories and fitting them all into the same 10-variable codification scheme. Examples of statistical “insights” that I found incredibly stupid include:

1. Beyond a point, more money doesn’t make people happier
2. Religious people self-report higher levels of happiness than atheists

Duh. These and other “insights” are accessible much more easily if you just bothered to think. Usually the thinking path gets you more than the statistics path in such cases. I cite such results to people who look for that kind of verification, but I personally don’t bother analyzing such statistical results deeply.

Sure, it is good to be humble and recognize when you don’t have enough modeling information from one case. Sure, data can prove you wrong. It doesn’t mean you stop thinking and start relying on statistics for everything. Look at the record of “statistics” based thinking. How often are you actually surprised by a “data-driven” insight? I bet you are like me. Nine out of ten times you ask “they needed a study to figure THAT out?”

And the 1/10 times you get actual insight? Well, consider the beer and diapers story. I don’t tell that story. Statistics-types do.

This means going with your gut-driven deep qualitative analysis of one anecdotal case will be fine 9 out of 10 times.

The Real Reason “Data Driven” is Valued

So why this huge emphasis on “quants” and “data driven” and “analytics?”  Could a good storyteller have figured out and explained (in an EBL/CBR sense) the subprime mortgage crisis created by the quants? I believe so (and I suspect several did and got out in time).

I think the emphasis is due to a few reasons.

First, if you can do stats, you can avoid thinking. You can plug and chug a lot of formulas and show off how smart you are because you can run a logistic regression and the Black-Scholes derivative pricing formula (sorry to disappoint you; no, you are not that smart. The people who discovered those formulas are the smart ones).

Second, numbers provide safety. If you tell a one-data-point story and you turn out to be wrong, you will get beaten up a LOT more badly than if your statistical model turns out to be based on an idiotic assumption. Running those numbers looks more like “real work” than spinning a qualitative just-so story. People resent it when you get to insights through armchair thinking. They think the “honest” way to get to those insights is through data collection and statistics.

Third: runaway “behavioral economics” thinking by people without the taste and competence to actually do statistics well. I’ll rant about that another day.

Don’t be brute-force statistics driven. Be feedback-driven. Be prepared to dive into one case with ethnographic fervor, and keep those analytics programs handy as well. Judge which tool is most appropriate given the richness of your domain model. Blend the two together. Qualitative story telling and reasoning and statistics.

And if I were forced to choose, I’d go with the former any day. Human beings  survived and achieved amazing things for thousands of years before statistics ever existed. Their secret was thinking.