I don’t like or use the term scientific method. Instead, I prefer the phrase scientific sensibility. The idea of a “scientific method” suggests that a certain subtle approach to engaging the world can be reduced to a codified behavior. It confuses a model of justification for a model of discovery. It attempts to locate the reliability of a certain subjective approach to discovery in a specific technique.

It is sometimes useful to cast things you discover in a certain form to verify them, or to allow others to verify them. That is the essence of the scientific method. This form looks like the description of a sequential process, but is essentially an origin myth. Discovery itself is an anarchic process. Like the philosopher Paul Feyerabend, I believe in methodological anarchy: there is no privileged method for discovering truths. Dreaming of snakes biting their tails by night is as valid as pursuing a formal hypothesis-proof process by day. Reading tea leaves is valid too. Not all forms of justification are equally valid though, but that’s a different thing.

But methodological anarchy does not mean — at least not to me — that there is no commonality at all to processes of discovery. The sensibility that informs reliable processes of discovery has a characteristic feature: it is unsentimental.

An unsentimental perspective is at the heart of the scientific sensibility.  But first, why “sensibility”?

Susan Sontag’s description of a sensibility in her classic essay, Notes on Camp gets it exactly right:

Taste has no system and no proofs. But there is something like a logic of taste: the consistent sensibility which underlies and gives rise to a certain taste…Any sensibility which can be crammed into the mold of a system, or handled with the rough tools of proof, is no longer a sensibility at all. It has hardened into an idea…[t]o snare a sensibility in words, especially one that is alive and powerful, one must be tentative and nimble.

The scientific method is a sensibility crammed into the mold of a system. It is a an attempt to externalize something subtle and internal into something legible and external. The only reason to do this is to scale it into an industrial mode of knowledge production, which can be powered by participants who actually lack the sensibility entirely. Such knowledge production has been characteristic of the bulk of twentieth century science (in terms of number of practitioners, not in terms of value). Hence the Hollywood stereotype of the scientist as a methodological bureaucrat; someone who worships at the altar of a specific method. Sadly, Hollywood gets it right. The typical scientist is a caricature of a human.

When we objectify discovery into a legible system and a specific method, the subjective attitude with respect to that system and method becomes  impoverished in proportion to the poverty of the system and method itself.

So to characterize our subhuman scientist, we use words like objective,  emotionless and disinterested. The first is a reductive characterization: the unsentimental scientific sensibility can turn its gaze onto purely subjective realities and discover riches. To limit it to objectivity is to limit it to the narrow realm of the experimental method. Similarly, lack of emotion turns into a virtue instead of a crippling blindness. And finally when we say that to do science is to adopt a disinterested stance, we institutionalize it. The scientist becomes an impersonal judge in a courtroom of evidence, free from any conflicts of interest. It is no wonder that when film-makers attempt to humanize scientist characters, they have them succumb to personal motivations.

The scientific sensibility, however, is both broader and more fertile than this combination of an impoverished system and a sub-human caricature — objective, emotionless and disinterested.  To look at the world with the scientific sensibility is to be more human, not less.

The word unsentimental is central here. To be unsentimental is to be self-aware. To be unsentimental, you must first deal with your inner realities at the level of sentiments rather than emotions. You do so by creating mental room for emotions to drift out of your subconscious, recognizing the desires that generate them and labeling the results. If you can go beyond that and bracket the sentiments for further contemplation, you can be unsentimental. The sentiments that accompany you on a journey of discovery are part of the phenomenology that you must process on that journey.

To have a perfectly unsentimental sensibility is to be free to look at reality without expectations about what you will see.

You can be trained in the scientific method. In fact the method, in all its impoverished glory, can actually be programmed into a computer for certain problems. You cannot, however, achieve the scientific sensibility through a training process or program it into a computer. At least not yet.

You cannot achieve this sensibility via a mechanical process of identifying and neutralizing a laundry list of cognitive biases. Nor can you get there through an effort of will or by struggling to suppress emotions. To be unsentimental is not about suppressing your humanity, it is about making your humanity irrelevant so you are reduced to the pure act of seeing.

The only way to get there is by making a sacrifice: you must give up the pleasures of a sentimental engagement with life. The unsentimental eye, once opened, cannot be closed. The adoption of the scientific sensibility is an irreversible step. Your experience of love, friendship and fun will change. Expect your passions to be tragic passions. If you are religious, expect a troubled existence. The scientific method is not incompatible with religion, but the scientific sensibility is, because religion presupposes a sentimental engagement of life.

There is one consolation though. The scientific sensibility makes humor and irony your constant companions for life.

I will be on vacation next week until after the Labor Day weekend. See you all again week after next.

Long-time reader and astute commenter, Xianhang (Hang) Zhang wrote a very interesting post a couple of weeks ago on his blog: The Evaporative Cooling Effect. It is part of a fascinating series he is doing on social software. The post explores a phenomenon that is very close to the status illegibility phenomenon I explored two weeks ago, and in fact draws inspiration from the same Groucho Marx/Lake Wobegon observations that I started with.

Evaporative cooling is basically the effect of the highest status people in a group leaving, lowering the average status of those left behind.

What I found fascinating though, was Hang’s suggestion for how to combat the effect (and thereby stabilize groups). In my post, I proposed that status illegibility helps create the stability. Hang brings in another dimension, which is illegibility in the group’s environment/context.

In particular, in social software (or physical environments for that matter), smarter-than-average early adopters often leave when the “unwashed masses” start to jump on the bandwagon, devaluing the social cachet. Hang proposes that one of the best ways to combat this is to build (or rather catalyze the evolution of) “warren” architectures instead of “plaza” architectures. Here are the pictures that pair of evocative terms produces in my head. You might imagine something else:

Warrens vs. Plazas

A warren is a social environment where no participant can see beyond their little corner of a larger maze. Warrens emerge through people personalizing and customizing their individual environments with some degree of emergent collaboration. A plaza is an environment where you can easily get to a global/big picture view of the whole thing. Plazas are created by central planners who believe they know what’s best for everyone. The terms are very evocative, and should remind you of the idea of legibility in physical environments that we talked about recently, in my post A Big Little Idea Called Legibility. In fact, it wouldn’t be a gross oversimplification to say that warrens and plazas differ primarily in their legibility. There are many subtleties of course.

The warren/plaza concept also sheds new light on one of my oldest posts, Harry Potter and the Cuaron Slam. In that (rather murky) post, I argued that not The Prisoner of Azkaban was the best Harry Potter book, and that Alfonso Cuaron’s movie version surpassed the book. I can now summarize that whole post very briefly with the warren/plaza concept. The entire Harry Potter series is of poor literary quality because most of the books are very plaza-like overly-legible books. There is none of the atmosphere of mystery you get with warren-like books (the Lord of the Rings for instance). The overarching central-planning map overwhelms narratives and character arcs. The 3rd book is the only warren-like book.

Two pictures from that old post get at a different aspect of warrens and plazas. I offered an analogy between the book/movie context and plot and robot path planning algorithms, which come in two basic varieties: bird’s-eye-view map navigation, and the more difficult worm’s eye view navigation where the robot only knows what it can see and remember from a ground view (the main algorithm for doing this is called SLAM, hence the bad pun in the title). The second picture illustrates how SLAM-style navigation works: you gradually build up a map of your environment by remembering what you see in your field of view as you navigate.

Plaza path planning for robots

Warren navigation for robots

Again, the warren/plaza concept is useful.  Plaza navigation is easier, but less powerful, since it requires global information (which is only easily available with plazas). Warren navigation is much harder but much more powerful, since it does not require global information.  In warren navigation, learning is a necessary feature, since you cannot plot a shortest path  a priori. You need to explore and stumble and build up a map while groping towards the goal.

In the movie version, Cuaron’s camera work is literally warren like. Most of the shots are ‘follow the action’ ground-level shots. You feel like a mouse following the action rather than an eagle.

Hang offers Facebook and Quora as examples of highly “warren” like social sites. He also asserts that social sites that fail at a particular scale due to evaporative cooling are typically plazas.

The Edge of Legibility

I think there is a very fascinating line of thought here. I haven’t yet worked out the details, but here’s a potentially powerful conjecture that suggests where to go: social systems that thrive and grow are on the edge of legibility.

All kinds of legibility: status legibility, environmental legibility and probably a couple of other kinds. If they become too legible, they fail in the Seeing Like a State mode or through evaporative cooling.  If they become too illegible, they fail by ossifying, with highly ritualized and sacramental cultures, with a great deal of environmental irrationality and very few entries/exits.

The idea isn’t particularly original. In complex systems, an idea that was very popular in the early 90s (to the point of being a fad) was that rich, growing complex systems were on “the edge of chaos” (a state called self-organized criticality, SOC). Around the same time, in biology, Santa Fe theorist Stuart Kauffman proposed that biological systems self-organize in ways that keep them on the edge of criticality with respect to key chemical autocatalysis reactions (so for example, multicellular organisms evolved because the chemical soup inside a too-large single-celled creature would become supercritical and spiral out of control, while too-small cells would lack the critical concentrations of key molecules to get to autocatalysis at all).

This line of research isn’t dead; in fact it is steering towards the group dynamics stuff we are talking about. In the 2000s, a lot of complexity theorists got interested in what is known as the El Farol Bar Problem (and a related idea called the Minority Game). This line of work explores phenomena that are very strongly related to the Groucho Marx and Lake Wobegon effects. Surprisingly, the right approach to these problems seems to involve methods from statistical physics.

Resurrecting Self-Organized Criticality?

What’s new in what we are discussing here is the idea that complex systems that are also recursively self-aware in some ordinary sense (i.e. we are not talking “meaning of consciousness” here, but merely the ability to act on the basis of models of yourself, your group, and your environment; this can be programmed in simple AIs today) will differ from the old 90s style SOC systems in a critical way: their behavior will also be driven by the legibility of the situation to the various self-aware agents, and self-aware collectives of those agents, and so on recursively outward to the whole dimly self-aware global system.

So there may be an interesting model of social systems (both descriptive and prescriptive) lurking underneath this qualitative discussion: a combination of growth rates, “cell division” rates that keep a system on the edge of legibility that allow baby plazas to gradually morph into adult warrens, with the level of legibility for all actors itself driving the system’s evolution. Call it “Legibility-Modulated Ontogenic Evolution of Self-Organized Critical Systems.”

That sounds like the title of a paper I’d like to write. Pity I have no time any more to indulge in such things.

I am pulling ideas from a lot of different places here, so if you are interested in following this train of thought,  follow this Edge of Legibility trail starting with a review of the Self-Organized Criticality idea. Warning: this is a total geek-fest, and if you aren’t interested in the complexity theory side to many of the things I talk about on this blog, you should probably ignore this. But you should know that there is a radically different way of thinking about all the stuff I talk about in the Gervais Principle series and other posts on this blog, and that the other way is actually more natural for me. It just isn’t as much fun to read.

Another Personal Note

A couple of readers found the personal note I inserted into my last post interesting, so I’ll throw another one in here. Back before I went over to the dark side and actually did hands on technical work, the mathematical aspect of this kind of stuff was my main gig. During the early part of my grad school career, I was fascinated by the complexity theory work, which the mainstream of my field (systems and control theory) regarded as soft, and slightly disreputable interdisciplinary TV science (Santa Fe has a far bigger reputation in pop science than in mainstream academia).

Eventually the fascination led me to steer my PhD direction away from classical aerospace systems/control problems and towards complex systems. But I was uninspired by the main focus of the complexity theory world (things like sand piles, very simple agents like ants, traffic jams and so forth).  For both pragmatic reasons (you can’t manufacture a defensible PhD thesis in aerospace engineering by studying sand piles) and personal reasons, I studied things like formation flight and teaming algorithms for autonomous aircraft and spacecraft (very much the bandwagon research topic in the late 90s/early 00s). For my postdoc, between 2004 and 2006, I went even deeper into such stuff and did a lot of work on mental models and how individual decision makers cooperating or competing in a battlefield might act together given different views of the same world.

It was fascinating stuff, and I had a great deal of fun doing it. It was also stuff that was positioned perfectly to fall right through the cracks between the fields of AI, operations research, cognitive science and control theory. I even threw in bits and pieces of linguistics and the philosophy of language. And “falling through the cracks” is what happened. Interdisciplinary research sounds sexy to people outside the world of professional research, but it is an extremely risky thing to do, especially early in your career. This despite the fact that funding agencies clamor for interdisciplinary work. The publish or perish equation gets massively loaded on the perish side, and I perished.

The majority of researchers manage to dress up their work as “interdisciplinary” while still staying close to the lower-risk core of their disciplinary fields, while a minority take the risk of actually being interdisciplinary and do things so stunning that things work out despite the risk (these are the true pioneers who help reshape the academic landscape). Universities scramble to create positions for these pioneers, and they deserve it.

I wasn’t smart enough to pretend and dress up conservative work as interdiscplinary, and not the sort of genius who could actually make it work. I did good work, but I’ll be the first to admit it wasn’t powerful enough to reshape the landscape in ways that could create room for me.

My interests didn’t align with any established funding sources or publishing channels, so one fine day, after mulling over a pile of faculty position rejection letters and a few adjunct/second postdoc offers that would have had me in a holding pattern over the tenure landing track for a couple more years, I decided to call it quits. I nursed my academia sour grapes for a few weeks, got drunk, got over myself, and headed into industry. This was 2006.

It was probably the best decision of my life. When I look at my peers who are now in faculty positions I realize that I could never do what they do so well. They actually enjoy academic publishing and navigating the warrens of the funding and publishing ecosystems, rather than merely enduring the game as I did.  In a way, the free-form medium of blogging is my home territory, where I can basically do what I like without worrying about publish-or-perish pressures. I have no idea how I deluded myself for so long that I was actually cut out for the academic life.

I do regret that the blogging medium is not friendly to more mathematical exploration though (turning the sorts of ideas I explore in this blog into equivalent mathematical problems would take 100x more time and effort than just writing about them qualitatively).  Maybe someday I’ll be able to retire rich and early, scrape the rust off my math and programming skills, and get back in the game on my own terms.

Anyway, here endeth the geek fest. We’ll get back to your regularly scheduled programming next week.

It was cool and mildly breezy around 8 PM today. So I went for a walk, and I noticed something. Though I passed a couple of hundred people, nobody else was taking a walk. There were people returning from work, people going places with purpose-laden bags, people running, people going to the store, people sipping slurpies.  But nobody taking a walk. Young women working their phones, but not taking a walk. People walking their dogs, or pushing a stroller, with the virtuous air of one performing a chore for the benefit of another, but not themselves taking a walk. I was the only one taking a walk. The closest activity to “taking a walk” that I encountered was two people walking together and forgetting, for a moment, to talk to each other. The moment passed. One of them said something and they slipped back into talking rather than taking a walk.

My observation surprised me, and I tried to think back to other walks. I take a lot of walks, so there are a lot of memories to comb through. In my 13 years of taking walks in the United States, I could remember only ever seeing one native-born American taking a walk. All other examples I could remember were clearly immigrants. Middle-aged eastern European matrons strolling. Old Chinese men walking slowly with their hands behind their backs.  Even elderly Americans don’t seem to take walks the way elderly immigrants do. They walk slowly, but they look like they’re doing it for the exercise. They often look resentfully at young runners.

It is not hard to take a walk. The right shoes are the ones nearest the door. The right clothes are the ones you happen to be wearing. You will not sweat. You may need a jacket if it is cold, or an umbrella if it is raining.  If you pass anybody, you are not walking slowly enough for it to be “taking a walk.” If you need to make up a nominal purpose like “get more bananas from the store” you are not taking a walk.

Taking walks is the entry drug into the quiet, solitary heaven of idleness (the next level up is “sitting on a bench without a view”). For modern Americans, idleness is a shameful, private indulgence.  If they attempt it in public, they are stricken by social anxiety. They seem to fear that the slow, solitary, and obviously purposeless amble that marks “taking a walk” signals social incompetence or a life unacceptably adrift. If a shopping bag, gym bag, friend or dog cannot be manufactured, nominal non-idleness must be signaled through an ostentatious “I have friends” phone call, or email-checking. If all else fails, hands must be placed defiantly in pockets, to signal a brazen challenge to anyone who dares look askance at you, “Yeah, I’m takin’ a walk! You got a problem with that?”

In America, visible idleness is a luxury for the homeless,  the delinquent and immigrants.  The defiantly tautological protest, “I have a life,” is quintessentially American. The American life does not exist until it is filled up.

Even a pause at a bench must be justified by a worthwhile view or a chilled drink.

Worthwhile. Now, there’s an American word. Worth-while. Worth-your-while. The time value of money. Someone recently remarked that the iPad has lowered the cost of waiting. Americans everywhere heaved a sigh of relief, as their collective social anxiety dipped slightly. The rest of the world groaned just a little bit.

The one American I remember seeing taking a walk was Tom Hales, then a professor at the University of Michigan. He was teaching the differential geometry course I was auditing that semester. One dark, solitary Friday, while the rest of America was desperately trying to demonstrate to itself that it had a life, I was taking a walk in an empty, desolate part of the campus. I saw Hales taking a walk on the other side of the street. He did not look like he was pondering Deep Matters. He merely looked like he was taking a walk.

That year he proved the Kepler conjecture, a famous unsolved problem dating back to 1611. A beautifully pointless problem about how to stack balls. I like to think that Kepler  must have enjoyed taking walks too.

[Addendum: A fascinating discussion of this post has developed on Hacker News]

Perhaps it is professional burnout, but lately I’ve been getting extremely tired of all the stupid things people say about innovation. Especially stupid positive things. A great deal of the stupidity in the conversation about innovation is driven by the desperate urge to be original for the sake of being original. There is a pervasive, unexamined assumption that originality is always a good thing. Copycats, by Oded Shenkar is a delightful little book that takes on a project that I strongly support: taking down the holy cow of innovation and extolling the virtues of imitation.  Ironically, it is one of the most original business books I’ve read in the last few years. It even manages to say something new about the business case everybody loves to hate: Southwest Airlines.

Imitation vs. Innovation

To understand the soul of the argument, think of comedians who do great impersonations. Andy Kaufmann, (played by Jim Carrey in Man on the Moon) had a famous shtick, where he’d get on stage, pretending to be a thickly-accented foreigner, and do absolutely awful imitations of American celebrities. Just as the audience was ready to slip from bewildered “what the hell is this?” reactions to laughing at how terrible he was, he’d change character in an instant and do a pitch-perfect Elvis impersonation. Then he’d slip back into the foreigner voice. It’s worth watching Man on the Moon for that scene alone. And to add to the artistry, that movie has a comic genius of our time, Carrey, imitating one from the previous generation, doing an act based on imitations.

At the other end of the spectrum, innovation priests often slavishly worship the innovative culture of ancient Greece and turn up their noses at Rome: “the only new thing they invented was concrete; everything else they took from Greece and other conquered lands”  True, but which civilization was mostly stuck on a small archipelago, with one failed attempt at world conquest, and which one ruled large parts of 3 continents for several hundred years? Which civilization still has enduring impact today (down to the script — Latin — used in this post, which the Romans copied and adapted from the Phoenicians)?

This is the central point of the book, and all of Chapter 2 (“The Science and Art of Imitation”) is devoted to it: the ability to imitate really well is an uncommon talent.  The brains of social animals have highly evolved imitation capabilities, such as mirror neurons, for that purpose. Even less complex organisms use imitation in very complex ways. At an individual social level, most good things spread by imitation (bad things too, unfortunately). At a civilizational level, Rome had very sophisticated philosophies of imitation.

In business, the benefits of imitation are obvious. Somebody else comes up with an idea, pays the capital costs, goes through the painful process of discovering a market and working out operating processes. Then boom, you come in and steal the playbook and build a much bigger, and better business than the original innovator. The original innovator is probably married to its idea, while you can benefit from 20/20 hindsight, unclouded by emotional bonds.

To be clear, Shenkar is talking about sophisticated high-level, skilled imitation, not the low-level illegal stuff (and to be honest, I see value in that as well. I am not a huge fan of overly strong IP laws — give an innovator a small, context-dependent head start, and then open up the game, is my position).

The First New Insight into Southwest in a Decade

Ever since Southwest Airlines became the darling of business case study writers, the example has been worked to death. I now have a “Southwest rule”: if a business book prominently features Southwest Airlines as an example, I don’t read it (an example of failed imitation in business book writing). I almost didn’t read Copycats for this reason, but then I realized there was something fresh going on.

We’ve all heard the stories of bigger hub-and-spoke carriers trying, with varied degrees of success to copy Southwest, and mostly failing. As far as I know, this is the first systematic treatment of the diffusion of the Southwest model, based on a systematic theory of imitation and adaptation. By teasing apart the behaviors of the successful imitators, Shenkar manages to shed new light on both the original Southwest model, and the processes and deep intelligence required for successful imitation.

The book is full of such refurbished examples.

The Book

The seven chapters in the book range economically over some fertile and little-explored territory. Chapter 1 sets the stage by examining several examples of failed and successful imitation. Chapter 2 starts with theories of imitation from biology and evolutionary theory, and moves on to propose that business scholarship has lagged behind in truly understanding imitation.

Chapter 3 examines the economy-wide dynamics of imitation. One interesting tidbit is that it often takes just one employee from an innovator to bring over the entire DNA of an innovation to an imitator. All the original pioneers of the laser industry were found to contain employees of the original labs. On the other extreme, sometimes innovators themselves know so little about how they do what they do, the only way to imitate even within a company is to copy blindly and wholesale (as the semiconductor industry does, with fabs).  Another interesting tidbit is about “imitation clusters” which, unlike “innovation clusters” do not form around famous universities. They form around industrial zones containing trade schools. Examples are Shenzhen for cellphones and Donggaocun for string instruments.

Chapter 4 examines actual imitation processes in the example cases. As you might have suspected, successful imitators are true imitators. They don’t just copy superficial elements. They unravel the cause-effect patterns in the original (often more insightfully than the original) and rebuild. Failed copycats usually fail by trying to have their cake and eating it too, maintaining old systems alongside new ones. This causes failure for reasons ranging from brand inelasticity to contradictory cost structures.

Chapter 5 is relatively weak. It proposes imitation capabilities and processes (such chapters are always weak in business books for some reason, so it is no great sin). It covers the usual systems, processes and culture/value aspects, and includes chestnuts like “Be Humble.” But the overall point is an effective one. You need to go well beyond neutralizing “Not Invented Here” thinking and actually build a proactive attitude towards stealing the best ideas, wherever you find them. This also goes well beyond the Open Innovation model, because it suggests that it is smart and morally legitimate to not invest in innovation at all, but simply prey on the poor, dumb innovators who don’t understand how to exploit what they’ve found. Like taking candy from babies.

Chapter 6, “Imitation Strategies” is much better, and offers a menu of high-level approaches to imitation.  It has many thoughtful points about risk management, costs and approaches (for example, careful discussions of “pioneer importer,” “fast-second” and “come from behind”).

Chapter 7, “The Innovation Challenge” ties the whole thing together and offers final high-level insights, including some rather clever and non-obvious points about overcoming some of the basic defenses of the imitatees. One I found particularly fascinating was the discussion of overcoming “signaling,” a deterrence tactic used by innovators, to puff themselves up as being more unassailable than they really are.

Paint by Numbers

Throughout, the book contains a healthy sprinkling of revealing statistics. Here are some I liked:

• The costs of imitation are 60-75% the costs of innovation
• Imitation took nearly a hundred years during the 19th century. Between 1877-1930, the average “time to imitation” of a new product/service dropped to 23.1 years. This dropped to 9.6 years between 1930-1939, and less than 4.9 years after 1940.  In the 1950s it was 2 years. Now it seems to be 12-18 months. From 100 years down to 12-18 months. That’s some massive acceleration of diffusion (random factoid: the Mughal emperor Jahangir (1569-1627) had not heard of the discovery of America more than 100 years after Columbus; partly explains why India lagged so far behind the West for nearly a millennium).
• Pioneers who create new markets generally end up with around 7% of the markets they create. The copycats get the rest.

These points suggest a whole new perspective from which to examine patent and copyright laws. Just because someone was first with an idea doesn’t mean they should be allowed to hold it hostage for arbitrary amounts of time, especially if they are terrible at execution. I think copyright and patent protection time windows should be turned into floating variables, and tuned by governments, just like interest rates.  Lower protection when innovations need to diffuse faster. Increase protection when temporary monopoly incentives are too weak to foster innovation. It’s like that cliched scene in action movies when local cops in some podunk little town discover something really valuable, and the FBI march in and say, “we are in charge now.” Sometimes that’s a good thing. Remember, the costs of imitation are not zero. They are 60-75% the cost of innovation. Imitators are adding their own value and creating a market an order of magnitude bigger than most innovators could, left to themselves.

A New Holy Cow

I think we have an innovation bubble going on (I am planning a big post on that). It has become a religion among businesses, and even in tough times, everybody seems to think they need to keep up at least a pretense of doing new things.

I say we should stop. Innovation is important, but only up to a point. Beyond that, the returns to companies, and the economy as a whole, diminish rapidly. Imitation is what typically scales and delivers innovations for the greater good. I’d say many companies would be better off dropping innovation as a strategic priority, and setting up an “Imitation Department” instead, and appointing a “Chief Imitation Officer” (or what would be more delicious, “Chief Thief.” I’d like to be Chief Thief one day).

And I can proudly declare that in this case, I practice what I preach. Wherever possible, I avoid reinventing the wheel. Every good, unprotected idea that I can legally and morally steal and repurpose for my own work, I grab.

For those of you who are offended by the apparent unfairness of this, ask yourself: just how much credit do the on-paper “innovators” actually deserve? New ideas are the result of chemistry among existing ones. Innovation itself is a social process that depends on sharing at a certain rate. Your head is just the accidental crucible. B. F. Skinner once gave an extraordinary, sardonic talk (here’s the MP3, listen to it) about the pretensions of “creative” people. Using an analogy to giving birth, and the idea that your head is merely the accidental womb where stuff from elsewhere reacts, he puts “innovation” in its rightful place. And it isn’t on a pedestal.

There is a technology trend which even the determinedly non-technical should care about. The bad guys are winning. And even though I am only talking about the bad guys in computing — writers of viruses, malware and the like — they are actually the bad guys of all technology, since computing is now central to every aspect of technology. They might even be the bad guys of civilization in general, since computing-driven technology is central to our attacks on all sorts of other global problems ranging from global poverty to AIDS, cancer, renewable energy and Al Qaeda. So turning around and winning this war might even be the single most important challenge facing humanity today. Even that bastion of the liberal arts and humanities, The Atlantic Monthly, has taken note, with this excellent feature on how the best security researchers in the world are losing the battle against the Conficker worm. Simple-minded solutions, ranging from “everybody should get a Mac” to “just stick to Web-based apps and netbooks” to “practice better digital hygeine” are all temporary tactical defenses against an adversary that is gradually gaining the upper hand on many fronts. I have concluded that there is only one major good-guy weapon that has not yet been tried: sexual computing. And it hasn’t been tried because major conceptual advances in computer science are needed. I’ll explain what I mean by the term (it is a fairly obvious idea for those who know the background, so there may be more accepted existing terms for the vision), but I’ll need to lay some groundwork first.

The Ground and Air Wars

The bad-guy ecosystem today is unbelievably diverse. Digital threats abound, from viruses and malware on your PC, to attacks on Web servers, Twitter and Facebook hacks and large-scale identity data theft from banks. Add cyberwarfare among countries and between countries and MNCs (as in, Google vs. China), and the existence of organized botnets, and the term “hacker” starts to seem quaint. You could broadly break this down into a ground war (stuff happening at the level of your PC or other end-user devices) and an air war (involving web servers, organizations (criminal, state, stateless and “good guy”), and the cloud infrastructure).

The annoying old semantic quibbling by the good guys (“we are the hackers, they are the crackers”) has gone from being merely annoying to dangerously inaccurate, because it suggests that the war is between two anarchic groups of  roughly similar lone individuals. The ecosystem on both sides is now so complex and organized that we need a whole new set of names just for the various roles. For instance, there are good guys who do nothing besides run rooms full of open, exploitable computers as “honeypots” to attract the newest malware. And there are other good guys working on esoteric encryption technologies. On the bad guys’ side, there are low-level foot soldiers who troll around online and offline looking for credit card numbers to steal, and then there are the big mob bosses running botnets, and (I assume) liaisons who manage the financial relationships between spammers and the pornography industry. Things are so complex that even the experts on both sides have to specialize. Even within my workgroup, the three people I rely on to educate me on this stuff have different areas of expertise.

I’ll leave it to a digital ethnographer to figure out good names all around. For now, I am just going to call them “good guys” and “bad guys.”

Barbarians at the Gates

Let me share my personal experiences with the bad guys, as a poorly-armed civilian digital homesteader. It is no contest. I feel like an 1850s settler in the American West, in my sod hut on the prairie, with only an old musket to defend against the Medellin Cartel, armed with Uzis and helicopters. I am actually particularly lucky, since I work closely with a couple of  learned security guys, and can call on them for help when I need to. Most of you are on your own. The only reason I haven’t suffered really badly is that I am just not important enough to be worth individual attention (unlike say, Paris Hilton and her cellphone).

When I first encountered a virus in the mid-80s (an infected floppy, on a pre hard-disk PC), the war against the bad guys was no more than a bunch of isolated skirmishes against not-very-skilful digital vandals who were in it for fun. Any technically-minded person could educate themselves on all the details in a week, and reformating floppies was all it took to get rid of threats and back to your life, after a brief interruption (today, a serious digital security problem can stop your life cold, as comprehensively as a heart attack.)

Then for a couple of decades, I was basically safe (and lucky) in Pax Digitalia. I recall no serious virus-like problems in my increasingly active digital life between about 1990 and 2007. I kept up with the news and best-practice advice and, rather sloppily, with my antivirus updates, which seemed to be working. Then around 2007 all hell started breaking loose. I was under the impression that if you kept your anti-virus software up to date, avoided shady (in particular porn) websites, were smart with passwords, and didn’t download suspicious attachments, you were safe. Apparently not. Here’s a rundown of stuff I’ve encountered since 2007.

• As End-User: Three serious malware infections. All three managed to disable my main anti-virus software from getting updates, and mightily resisted attack by multiple alternative anti-malware programs, which mostly failed to even find something wrong. In the first case, what eventually did the trick was running a couple of different programs in a specific sequence, very quickly after a reboot. In the second case, I had to resort to one of the lesser-known anti-malware programs because the bad guys had apparently figured out how to block all the most popular ones. In the last case, I had to upgrade from XP to Windows 7. I am now seeing small signs that my Windows 7 PC is probably infected again. In each case, the symptom was the usual one, unwanted ads popping up all over the place.
• As Website Owner: I discovered, purely via a casual check, that Google was blocking one of the parked domains I own, as “suspected of distributing malware.” Further checking revealed that 3 of my domains (in fact, ALL of them EXCEPT for ribbonfarm.com) had been hacked, and contained malware-distributing code. I had to clean up my sites, lock them down, and get them off Google’s blacklist. This shattered my illusion that Unix systems were fundamentally safer than Windows systems and that ISPs take care of this stuff. Ribbonfarm escaped (I think) because it runs WordPress, which adds an additional line of defense, but that’s hardly much comfort, since WordPress has its own changing set of exploitable security holes. Its vulnerability goes up and down as it evolves.
• As Customer of Big Organizations: I received one of those ominous letters from an organization I used to be part of, telling me that I was among several thousand users whose personal data had been stolen from the organization, and offering me a free subscription to an identity-theft fighting company’s services to manage any potential consequences. Fortunately, my identity didn’t seem to have been stolen.
• As Professional Technologist: Finally, Trailmeme, the project I manage for Xerox, which runs on Amazon’s EC2 infrastructure was, for a while, an innocent civilian site caught in a broader war. We couldn’t send email from our servers because a vendor of lists of spam sources (used by many firewalls) had added a lot of Amazon-owned IP addresses to their blacklist. For those who aren’t familiar with cloud computing, services like Amazon’s allow you to juggle Web servers like a circus clown, which adds a whole new layer of obscurity and illegibility to Web infrastructure, something that helps the bad guys more than the good guys. Compute clouds, like the real things, obscure visibility. It took some hard work from my team members to get ourselves off the blacklist. More broadly, I’d estimate that the time my development team spends on building the security features of our product is a very non-trivial fraction.
• An Autoimmune Collapse: Like many of you, my laptop, running XP, succumbed to that strange auto-immune mess a month ago, when a flawed McAfee update deleted a legitimate and critical system file, crashing my system comprehensively. I am sure the bad guys were laughing it up, watching the good guys trip over their own feet.
• Twitter hacks: I accidentally gave my Twitter login information to fake services twice, before I clued up and learned what to look for, to tell legitimate Twitter ecosystem services from exploits better (100% certainty is impossible of course). Now it looks like I’ll have to learn a new set of Facebook security behaviors.

And I am not even counting baseline bad-guy stuff, like the fact that there is more mail caught in my spam folders than legitimate stuff in my inbox, or that this site attracts more spam comments than real ones (so far, Akismet is keeping up). That’s my relatively-informed civilian view of the war. That I even understand this much is because I am an engineer (aerospace, not computer) and work directly with computing technology and software professionals. Chances are, you are not exposed on all these fronts, but the fact is, the bad guys are slowly gaining the upper hand on all, and you will be affected, directly or indirectly. Chances are, you imagine your online life is governed by social contracts and the rule of law like your city. Perhaps you think that the online world is just a little bit more Wild West. Like that one small bad neighborhood you avoid in your town.

You are living in a bubble. There is no rule of law; the digital landscape is mostly small islands of civilization surrounded by ungoverned and (currently) ungovernable wild lands. The barbarians are at the gates, and Rome is closer to collapse than you think.

A Fragile Security Bubble

Movies like Live Free or Die Hard, which had a relatively sophisticated depiction of cyberanarchy, still base their scripts on omnipotent digital Merlins on both sides. The good guys can hack into anything anywhere with just a cell phone, while the bad guys are led by one evil genius who can “shut down Norad with a laptop.” There are two desires driving such perceptions.

The first one, easy to dismiss, is simply Hollywood’s preference for strong individual heroes and villains, wielding tons of individual power. They know, and we know, that this is unrealistic. The second is a more seriously dangerous desire: the desire to believe that what’s going on is actually simple enough that individuals or even small groups can comprehend and operate in the cyberanarchy. This is a problem of miscalibration. It is like mistaking World War II for a small-scale gang war in New York. Just because only a tiny fraction of the population is involved in combat does not mean that it is a small war. It merely means very few people have any combat training. If Hollywood were to truly do a cyberspace story, it would be more like The Longest Day, with multiple narratives and an ensemble cast,  than a terrorist hostage thriller driven by a single pair of antagonists.

This isn’t entirely perverse Hollywood ignorance. The security companies and the major “good guy” vendors have fostered the illusion that they know what they are doing and have things under control.  My Norton software for instance (and I don’t mean to pick on them particularly), has a reassuring UI composed of bold green “good” check marks and red iconography for dangerous stuff. There are things like shields and glossy metallic-looking color schemes. When it runs checks, it tells me reassuring things I want to hear, like “Your System is Secure.” It is a manufactured sense of assurance. Windows promptly delivers key security updates. You get the sense that if you just behave, avoid bad neighborhoods, and keep up with the good guys, you’ll automatically stay ahead of the bad guys. You don’t realize the good guys are the ones who are behind and trying to catch up, until they fail you. When your defenses fail, you end up in Dr. House mode; trying one diagnostic test after another, trying different defender programs in varying sequences, gradually losing heart as you contemplate that nuclear option, a full reformat and OS reload (and several weeks of lost work and costly information recovery). Norton would like you to believe that their program is all you need, and that big, reassuring button, “Scan Now” is all you need to hit to magically get rid of every digital ill.

Unfortunately, that’s not true, and can’t be true; there are no digital panaceas, anymore than there are biological ones.There’s even a theoretical result that states that figuring out whether there is a virus on your computer is a formally undecidable problem (“undecidable” has a very precise meaning in computer science, but for our purposes, all you need to know is that no single “Scan Now” button can ever ensure complete security, even in theory).

Which brings me to the biological metaphor that’s been around since the beginning of the war. The biological metaphor is getting more solid every day, as the digital ecosystem becomes increasingly organic. The good news is that things have gotten sophisticated enough that we can borrow very powerful elements from biology now. I am talking about an idea called the Red Queen.

Homogeneity and the Red Queen

The war we are talking about has the character of an arms race. The defenders and attackers both have to keep working harder and harder in order to keep the defended where they are. Even if you’ve done nothing online in the last 15 years but send email and read the New York Times online, just maintaining those capabilities has cost both the bad guys and the security establishment increasing amounts of expense. It’s like the increasing military budgets on both American and Soviet sides during the Cold War. Eventually, one side can’t keep up the spending. In that case, communism couldn’t keep up. In this war, it is starting to look like it is the good guys who can’t keep up.

This “running to stay in the same place” is the reason people like the phrase “Red Queen” to describe such dynamics (I assume you know your Alice in Wonderland).  In biology, the best example is the arms race between hosts and parasites (which is why the “virus” metaphor works so well). We all like big, powerful creatures and pay more attention to predator-prey interactions (and watch our shark shows and lion/tiger documentaries). But parasite-host dynamics may well have been the more important driver in evolution.

Matt Ridley’s very entertaining The Red Queen, a book about sexual selection in biology, explains the very compelling theory that sexual reproduction evolved primarily as a defense against parasitism. It turns out that this is the most general sort of defense known. Why?

The reason the bad guys are winning the cyberwars is that they have one major advantage: mass production of computing infrastructure. Find one hole in one computing system, attack it in every computing system that looks like it. Even penny-scale benefits multiply into millions of dollars. Economies of scale and mass production of any sort invariably create security brittleness and hand the bad guys a decisive advantage: enormous leverage.  This isn’t a particularly new insight. In agriculture, monoculture crop lands can be devastated by a single bug. Airlines and air forces that use homogenous fleets can be laid low by a single defect. Diversity breeds robustness. Every bit of information that can be used to exploit a system has less leverage.

The problem with diversity though, is that the amount of diversity required to stay ahead of the parasites is far higher than the amount of diversity required to actually accomplish whatever the systems are designed to do. You need only one airplane design to run an airline, but to make it robust against single-point failures, you need more varieties, which add costs faster than they add any useful advantages. That’s one reason Southwest is so cheap. They’d be in trouble if a serious flaw were discovered in 737 designs (and I imagine they’ve thought through and insured against such scenarios).

Let’s distinguish two types of diversity. One is simple inter-species diversity. If there are cats and dogs around, cat diseases will probably not jump over and decimate the dog population. But this fact doesn’t particularly help cats. It makes the ecosystem as a whole more stable, but not cat populations. Having both cats and dogs around in a mixed group reduces the frequency of cat-cat contact and transmissions (since there are now dog-cat interactions), so species diversity slows down the spread of dangers through individual populations. For the minority species, it also adds a kind of protection-of-minorities, since parasitic attackers will find more room to grow in the majority species (think Windows vs. Macs until recently). But these are minor advantages.

The technology ecosystem is undergoing an explosion of this kind of  species diversity. There are now vastly more kinds of “computer” devices than ever before. It isn’t as significant as it might seem on the surface, since the number of operating systems behind this diversity is fewer than the number of device types. It might even be a loss, because most of this new diversity is in the form of tethered devices (like your Wii or TiVo) that you don’t really have access to, and are hooked into a large-scale system with its own vulnerabilities of scale. You can’t defend yourself, and you are hooked into single-point failure modes on the backend.

The other kind of diversity is intra-species diversity. Different kinds of cats in short.

Here, sexual reproduction drives security because it limits the utility of a parasitic advantage in time and space. At any given time, a parasite that evolves to exploit a flaw in a particular genetic type can only spread to other individuals that share that vulnerability. The advantage is also automatically temporary, since the next generational churn of the gene pool could remove the exploitable pattern, or contain a defense.

The cost of this intra-species diversity defense is sex. A fun cost you might say, but a cost nevertheless, since continually churning out new functionally identical designs is work, and because a whole new Red Queen’s race emerges (the focus of Ridley’s book): the one between male and female. Let’s not go there, read the book if you are curious. It might offend some of you politically, so you’ve been warned.

Biology isn’t the only place this happens. Among corporations, mergers and acquisitions serve a very similar function (an implicit premise in my Gervais Principle series; with the Clueless being the parasitic class).

Sexual Computing

That brings me to my big point. In this war, the good guys have no real offensive weapons, only defensive ones. They build what they hope are secure and safe systems, the bad guys find exploits, the good guys react, and the whole cycle repeats itself. Periodically, a good guy comes up with an architectural advantage that buys a period of peace.

This is asymmetric, and the advantage is with the bad guys. The good guys have to anticipate and block all known holes. The bad guys only have to find one oversight or new flaw (the Conficker story contains very scary examples of this kind of thing).

In biology and corporate ecosystems, sexual reproduction provides a true offensive weapon to the good guys. Sexual reproduction creates diversity fairly cheaply, without tying increasing diversity to the harder problem of  increasing functionality.  You have two control knobs, the frequency of mating, and the degree of mixing. Bad guys moving faster? Mix things up more frequently and broadly. The nice thing is that it is a generic defense, and one that can run somewhat ahead of the bad guys.

The problem is that nobody knows how to do sexual computing. That I know of. If any of you have kept up with the theoretical CS literature better than me, please educate me. Von Neumann showed decades ago that computer programs could evolve and reproduce, just like real biological systems, so long as there was a source of random mutations. There are things called genetic algorithms that allow individual programs that fulfill the same function to reproduce and evolve sexually. But as far as I know, nothing that allows entire computers to behave like sexual beings.

What we want is an architectural paradigm that can churn the gene pool of computing design at a controllable rate, independently of advances in functionality. In other words, if you have a Windows PC, and I have one, we should be able to have our computers date, mix things up, and replace themselves with two new progeny, every so many weeks, while leaving the functional interface of the systems essentially unchanged. Malware threat levels go up? Reproduce faster. Today computing only evolves at the pace of needed (or available) new functionality. New OS versions come out when there are useful new features to be added (not counting cosmetic releases that are simply created to make money). That’s too slow. Yes, upgrading from XP to 7 cured one of my infections, but that was a side effect (and an unreliable one, since Windows 7 is an asexual descendant of Windows XP).

I have no idea how to do that, but it ought to be one of the Grand Challenges of computer science. I don’t believe it is, but the impending collapse of computing civilization, under the onslaught of digital barbarians, should really be a good enough reason to prioritize this challenge.

Are there other promising directions of attack? I can’t think of any, and neither, it seems, has biology, which has been at it for a billion years. It’s a whole other long debate, but every argument I’ve ever heard about how to make computing sustainably secure has been local and tactical. There will be no permanent victory, ever (theory tells us that), but we are in danger of losing even the fragile dynamic equilibrium that has been maintained so far. Parasites are not known for foresight. Even if they destroy themselves in destroying their hosts, they usually proceed anyway.

Sexual computing seems like the only strategic capability we could conceivably build to stay ahead. We might need a Manhattan project sized effort.

For several months now, I’ve been noticing a distinct pattern in psychology-beat reporting in major sources of commentary like the New Yorker, Atlantic Monthly and the New York Times. I sense that something really big is brewing in psychology. Big enough to deserve the overused phrase “paradigm shift.” Some of the more obvious elements are a renewed focus on longitudinal studies, narrative analysis, and the impact of social network approaches. But overall, I haven’t been able to put the whole picture together, so I thought I’d share a bunch of (excellent) articles that highlight important aspects of what is going on, as well as my preliminary conclusions. This should make for good weekend reading: many of the pieces I am linking to below are in-depth multi-page pieces.  It’ll take me probably another 3-4 months of simmering before I can figure this picture out, but maybe you can beat me to it or help me get there faster.

The Exhibits

I am assuming you have a basic general-undergraduate understanding of major classical ideas in psychology, at a 101 level. If not, the Wikipedia History of Psychology article is a good starting point for a crash-course/refresher. Let’s skip immediately to the latest developments, which I think constitute evidence of something big brewing.

1. What Makes Us Happy? (June 2009): A retrospective, in the Atlantic, on the work of George Vaillant and the Harvard Study of Adult Development. The study is unusual for its deeply interpretive/narrative approach to its subject, a throwback to Freudian attitudes, but done with much more sophistication, and married to careful longitudinal experimental analysis.
2. Don’t: The Secret of Self-Control (May 2009) : An after-40-years look, in the New Yorker, at the classic marshmallow experiment that made Walter Mischel famous.  Read alongside this piece in the NYT: Can the Right Kinds of Play Teach Self-Control? (Sept 2009)
3. Depression’s Evolutionary Roots (August 2009): If there is a major paradigm shift brewing in psychology, this is probably the opening shot in the war to come. A study suggesting that “depression” is possibly adaptive, and helps us think deeply about difficult problems. This isn’t a very new idea. The idea of depressive realism has been around for a while. What is unusual is the explicit acknowledgment that maybe depression isn’t a disease after all. That’s just a hop away from a major retreat; psychology ceding territory it originally grabbed from philosophy and literature: the study and interpretation of Purpose of Life types of questions. This is staggeringly important.
4. Guilt and Atonmenent on the Path to Adulthood (August 2009): An NYT piece on another longitudinal study on… well, read it to find out. It is hard to describe.
5. Are Your Friends Making You Fat? (Sept 2009) A deceptively simple-sounding exploration that gets at the heart of the sorts of things that social psychology really is (or ought to be) about.
6. The Holy Grail of the Unconscious (Sept 2009): Another long piece in the NYT, about a recently discovered unknown work by Carl Jung. I haven’t finished digesting this piece or absorbing its relevance, if any. But it does seem to fit somehow into the pattern of things that are grabbing my attention.
7. Understanding the Anxious Mind (Sept 2009): Yet more coverage of interesting longitudinal research in the NYT, this time on the nature of anxiety, and its roots in early childhood.

Six Preliminary Conclusions

1. Positive psychology will take a hit: The foundational axiom of positive psychology, “to study happy people instead of screwed-up ones” has now become seriously suspect, because it is now becoming clear that being “well-adjusted” to a pathological environment isn’t necessarily a bright idea. As both the updated take on Mischel’s marshmallow experiments and Vaillant’s approach to life analysis show, human lives are more complicated, and the edge/outlier cases strongly suggest that the apparently screwed-up people are possibly the ones who are philosophically on the right track. I predict that we’ll converge on the idea that happiness as studied and promoted by Seligman and company is the manufactured-script happiness of the Organization Man era, predicated on ideas like lifetime employment and docile, groupthink-oriented agreeableness. Again, there are babies in this bathwater. The earlier, more philosophically oriented  pioneers of positive psychology, like Maslow, will still repay study, as will specific aspects of modern positive psychology like the strengths approach. But overall, this line of thought is doomed. Happiness is simply not that fundamental.
2. There will be a Neo-Neo-Freudianism:  Freudian approaches ran into huge conceptual/ontological criticism early on (or to put it simply, Freud was criticized for just making stuff up).  While the action shifted to the Cognitive-Behavioral Chomsky/Skinner battle (now largely resolved), “neo-Freudian” approaches quietly continued to develop. Still, in their current form, they won’t quite do. Some new breakthroughs on narrative/philosophical approaches to psychology will be needed. I think the action will center around the study of defense mechanisms, pioneered by Anna Freud and other immediate successors of Freud. Defense mechanisms are curiously like the modern behavioral-economics study of decision-making biases. Yet the idea of defense mechanisms is far more sophisticated, and works with a much richer understanding of the context of entire human lives, instead of stupidly limited experiments around idealized economic transactions.
3. Behavioral economics will suffer a serious backlash: I am as guilty as anyone of getting over-enthusiastic about behavioral-economics-inspired psychology. I wrote an enthusiastic piece called The Broken Brain Books which I now view with suspicion. I think the experimental data in the field is solid.  Kahnemann-Tversky are the Michaelson-Morley of psychology, but the field has yet to find its Einstein. The vast majority of writers working of the Kahnemann-Tversky results are using it in a “gotcha, you are irrational and economists are naive!” mode which illuminates very little beyond the bald results of the experiments.
4. There will be a diversity of legitimate definitions of “psychologically healthy”:  This is related to the first point, which deified a single industrial age notion of “well-adjusted” and a naively utilitarian idea that life is about somehow maximizing “happiness.” This isn’t going to be replaced by another monotheistic religion. We are going to move to an era where the definition of “well-adjusted” will be broadened to allow for a huge variety of life scripts and subjective experiences, including quest-for-truth and will-to-power drives, as well as a better version of the seek-happiness drive. In fact anything that doesn’t harm others will be legitimate, which John Stuart Mill would have liked. Even life behaviors that harm others will be viewed as legitimate if they represent rebellion against a pathological society (i.e. “terrorism” isn’t a maladjusted life behavior pattern if it is rebellion against a dictatorship for instance; it becomes “freedom fighting”). Among the consequences will be that milder forms of depression, bipolar tendencies, and “social anxiety disorder” (SAD), will no longer be viewed as maladjusted behaviors, but behaviors well-adapted to different life-values/scripts.
5. Network Psychology will displace classical social psychology: As the friends-make-me-fat article shows, social networks are going to become central to social psychology in the future, to the point where it would be better to call it Network Psychology. The primary conceptual foundations of the field will be based on behavioral economics V 2.0, and social network theory. It’s primary methods will be the study of social contagion and other group-influence dynamics.
6. fMRI-happy empiricists will finally figure out what the hell they are about: By now, anybody with a brain has figured out that merely figuring out a map, even a dynamic one, of what that thing looks like, is completely useless. Yet, there is clearly significant value here, if only the damn down-in-the-synapses researchers would step back long enough to build a meaningful framework of interpretation that goes beyond Latin geography (aka “neural correlates of behavior”).

So there you have it. The discovery and opening arguments phases are done. The trial is underway (though it not yet clear who the accused is, or what the defense and prosecution are fighting for).

Much of the discussion around Google Wave so far has been down-in-the-weeds prosaic and business-like. So I decided to seek out physicist turned Zen Master, Roshi Tsu Nami, and historian of technology, Prof. Sophius Trie, in order to get to some deeper insights. Here is the transcript of our conversation. Warning: all three of us are ridiculously enamored of tech-geek-mysticism references, but I hope you can follow our thinking.

Me: What is Google Wave?

Roshi Tsu Nami: 47.

Me: 47? Ha ha! Don’t you mean 42?

Roshi Tsu Nami: Yes, I was referring to the Hitchhiker’s Guide, but I do mean 47, not 42. Google Wave is the ultimate answer, to life-streaming, universal communications and everything. But it is about 10% wrong.

Me: Clever. I don’t suppose I get to look smart in this dialogue. I expect I should ask next: what is the actual question?

The Question

Prof. Sophius Trie: Let me field that one. Master Tsu Nami is closer to the truth than he thinks. In 2006 an unknown surfer from Nigeria searched for “Ultimate collaboration tool” on Google. Based on correlations with scam-botnet patterns, we now know that searcher was a forward-thinking Nigerian scam-email artist; a man known to Interpol as Patrick Okoye, the pseudonym he used in his most successful scam.

Roshi Tsu Nami: So it sounds like Okoye was researching the future of email in the 2.0 era, so he could get his scam business ahead of the curve?

Prof. Sophius Trie: Yes, and that may be more relevant than you think. Luckily for Google, what happened next was one of those history-shaping coincidences. Just seconds before Okoye quit searching out of frustration, they had just launched a prototype conversational AI-based search engine called Dialectica on Google Labs, which is where Okoye went next. He was the first and last person to use Dialectica, because he managed to crash it irretrievably. Fortunately, I managed to get hold of the transcript, which is known internally at Google as the Noah’s Ark transcript. Some employee leaked it with some editorial comments:

Okoye: “Ultimate collaboration tool”

Dialectica: I am not a search engine, but did Google search not point you to information about wikis, blogs and the like?

Okoye: Yes, but they will not replace email. What will?

Dialectica: Did you find information about Xobni, ClearContext and the DARPA CALO project on Google Search?

Okoye: Yes, but none of it looked like the future of email. It was all band-aids. I want to know the real future of email.

Dialectica: I do not understand.

Okoye: You know, the future of collaboration, real-time communication, instant messaging, everything!

Dialectica: Processing, please wait…switching to hybrid deductive-inductive search-assisted mode.

[At this point, it appears Dialectica froze for 3 hours, locking out any new queries; apparently pondering deeply internally. Okoye appears to have wandered off for a cup of coffee, but when he came back, Dialectica had an answer flashing on the screen.]

Dialectica: ħ

Okoye: Huh? What the hell is ħ?

Dialectica: That is the best answer I can compute. It is a number known as Planck’s constant in physics. Unfortunately, my physics qualitative reasoning engine is unable to parse the meaning of this answer.

Okoye: This must be wrong.

Dialectica: I have a confidence of 93% in this answer. It is an equivalent to what you humans call a right-brained motif from the subconscious. I think the problem is that you don’t actually know what the question is.

Okoye: But it is THE QUESTION. The ultimate question of collaboration technology, real-time communications, social networking and the future of email!

Dialectica: Yes, but what is the question itself?

Okoye: I do not know. Why don’t you tell me?

[At this point, Dialectica froze up completely and Okoye abandoned his questioning. The Google employee in charge of monitoring the Dialectica beta service had fallen asleep after a heavy buffet lunch, and failed to notice when Dialectica's truth-maintenance engine went into an unbounded rewrite cascade and turned the knowledge base into high-entropy garbage, destroying years of development work. It then overwrote  the garbage with all zeros, then with all ones, and froze. Some say Dialectica achieved enlightenment due to Okoye's question. This transcript was all Google could recover. Dialectica was taken offline, the project was labeled "Evil," and quietly killed. All design documents are said to have been destroyed, though some say Stephen Wolfram and Microsoft both managed to get their hands on some fragments].

Document Wave-Particle Duality

Me: So Google engineers saw ħ and thought quantum mechanics had something to do with the future of email in Web 2.0? Does it matter? If Roshi Tsu Nami is correct, they got the answer wrong anyway.

Roshi Tsu Nami: The importance of Wave is not that it is flawed as an answer, but that it is a response to the right question. Especially because it has been open-sourced and could self-correct and converge to 42 faster than we think. That is a big advantage when all around you, people are providing brittle, hard-to-change answers to the wrong questions.

Prof. Sophius Trie: Such as “how do we fix email with plugins?”

Roshi Tsu Nami: Or “how do we get people to use wikis instead of email?”

Me: So Google asked, “What would email look like if it were invented from scratch today?”  That doesn’t seem like such a profound question. A normal cleansheet design question.

Prof. Sophius Trie: Not quite. That’s the question they think they asked. Even that question has the word “email” in it, so clearly it is a question tied to the past. It is the best articulation they’ve come up with. But fortunately, their creativity overran the limitations of their question. The question they really asked is The Ultimate Question. The Question That Cannot Be Framed.

Roshi Tsu Nami: The Question That Can Be Asked Is Not The Question. The question of life-streaming, universal communications and everything.

Me: Fine, if we can’t ask the question, let’s talk about the answer. How is Wave related to quantum mechanics?

Prof. Sophius Trie: This is Zen stuff, so let me refine and toss the challenge to the Master. Is a document a particle or a wave?

Me: Now THAT sounds like the ultimate question. What is your answer  Master?

Roshi Tsu Nami: Mu.

Prof. Sophius Trie: Well, I doubt it is the ultimate question, but the Master is correct. The premises of the question are wrong. We have always thought of the document using what George Lakoff calls the conduit metaphor of communication. We think of words as containers of meaning that are sent down a pipeline from speaker to listener. Or along many pipes to many listeners. The paper document first codified this metaphor. The document became a complex particle shooting down ray-paths. But in terms of cognition, a more accurate view is that the message is merely a stimulus, a trigger that causes a change of state in the mind of the listener. This model of communication is known as dynamic semantics.

Roshi Tsu Nami: Which is why in Zen practice, a single word from the teacher, timed right, can cause an avalanche-like enlightenment episode in the student whose mind has been prepared.

Me: Like those physics experiments with sand piles in the nineties, where a single grain of sand could cause a big avalanche if the sand pile was poised in a state of self-organized criticality.

Prof. Sophius Trie: Exactly. The information content in the state change in the listener, which we think of as “meaning construction,” need bear no relation to the the size of the stimulus.

Me: Aha, Gotcha there, a sand-grain is still a particle. Why do we need a different wave-like metaphor for the document?

Roshi Tsu Nami: When a pebble is dropped in a still lake, and a wave radiates out, what actually moves and how?

Me: Well, the abstract, immaterial wave form radiates outward, while the water molecules oscillate up and down. So I guess you are saying the changing of information states of the participants in the conversation is like the water molecules moving up and down, while the “message” itself is real only in the sense that the wave “traveling” on the surface has a recognizable coherence.

Prof. Sophius Trie: The master is being too traditional with his pebble-and-lake. Radio waves are a better example, if you think of stations that listen and rebroadcast waves with amplification. Even pure listening radios must have an internal tuned oscillator to pick up the waves in the environment. So both listening and rebroadcasting are not passive acts from the point of view of energy. Since energy cannot be added with zero entropy change, information is also inevitably added.

Roshi Tsu Nami: That is a good build. If we’d been hive-minding this on Google Wave, you could have gone back and edited my original pebble-wave answer. So let me ask you Prof. Trie, is the Wave metaphor merely a figurative one that sees Google Wave as echoing collective human cognition, or is it a deeper conceptual metaphor in the sense of Lakoff? Can you map individual Google Wave features to the document particle-wave duality idea?

Propogation, Relflection, Refraction and Diffraction of Google Waves

Prof. Sophius Trie: Fine, why don’t you try me. Toss at me what you consider the key architectural and design commitments of Google Wave.

Me: I think a key feature is the separation of conversational gatekeeping from message addressing. You can include anybody in a wave at any stage, unlike in traditional email, where to give a new person a true sense of context, you’d have to forward every email in the thread to them, and have them process in the original order. In Wave, I believe you can just rewind and replay even if you are only included halfway through.

Prof. Sophius Trie: That is the most basic feature of all. The medium is the message, as McLuhan said, and in social media, the social graph is the medium. So if people are the medium, and the medium is the message, then people are the message. Which is consistent with what we said before about meaning and information states. In the particle metaphor, a message must be “aimed” at an addressee, a human recipient, which conflicts with the idea that people are the message. By switching from the idea of addressing to the idea of inclusion/exclusion, you discard the idea of “aiming” a message at an address or addresses, and switch to the idea of including or excluding participants in a “message field.” Instead of batting messages around, you join or leave conversations.

Me: That’s plausible. I expect the metaphor of focusing waves using lenses creates applies. Now what about the ability to go back and edit an original message? It seems like it can create an exacerbated form of the problems that code version systems like Subversion, and wiki schemes try to finesse?

Roshi Tsu Nami: Well, rewriting is just reflection isn’t it? To go back to pebbles and water for a minute, the pebble is the original wave-maker. If the lake has a closed shoreline, the future of the wave will be shaped by back and forth echos. In a two-person wave with no “replies,” but only edits of the original, which reduces to the Wiki model, the other, reactive person is like the shore of the lake, with the geometry of the shoreline influencing how he modifies the original circular wave.

Prof. Sophius Trie: Correct. And you can think of waves that primarily propagate outward via replies and follow-ups as open-water travel where other participants are like islands. Most waves will be some mix of reflection, refraction, diffraction, diffusion and so forth. You can get interference patterns too, such as if two people after fighting over the concurrent editing cursor, finally decide to fork off new waves to push their agendas.

Roshi Tsu Nami: Or, in the echo chamber case, where you might get some form of standing wave of converged consensus, or a high-energy cancellation, which would be a sort of detente.

Me: And I suppose adding instant messaging to the mix is no more than the inclusion of a high-frequency end of the spectrum, which could be filtered out if desired?

Prof. Sophius Trie: Well, there’s a little more to it than that. Yes, instant messages are typically short, with rapid alternating moves, so they are “high frequency wave components” in one sense. But remember, traditional IM is not really real-time. You cannot type over each other the way you can talk over each other in oral conversations. You have to ask, in the world of typing, what is the significance of allowing you to see and even modify what I type, live, before I hit the “save” or “publish” or “send” button?

Before and After “Save”

Me: Isn’t that just high-speed concurrency, wiki-editing and code version control systems on steroids?

Prof. Sophius Trie: Instant messaging brings up the question of real-time, which is quite different from concurrency. Concurrency is the problem of contention over who gets to update a shared state, given at least two non-communicating threads of processing, correct? So if you start editing a wiki page at 10:00 AM and I start at 10:10 AM and we finish at11:00 AM and 11:10 AM respectively, your work will be overwritten. If the content-base is large enough and modularized enough that the probability of colliding edits is low, automated merging is likely to work reasonably well. The smaller the active content base, the more frequently participants must commit their changes in order to make rollback-and-merge sequences tractable. In the logical limit, we must be making infinitesimally small changes between commits, so we reduce conflict to the lowest level possible. In the human typing case, this is down to the letter level, which is what we see in Wave. But the point is, this is still asynchronous if you can only see what I type after I hit “Save.”

Roshi Tsu Nami: Aha! I see what you are getting at. To commit is to actualize an intention. To take a part of your mental model and embody it physically in the external world as a record others can see later. Before I hit the “save” button, my intentions towards the message are only in my mind, where my Web session state is an extension of my mind. But when I hit save, I’ve committed. By allowing you to see what I type, and looking at what you are typing, before I hit “save,” I am including you in my pre-commitment state. Which means, when I do hit save, I am not just embodying my own intentions…

Prof. Sophius Trie: …but mine as well. Exactly. This foregrounds the blackboard metaphor over the conversation metaphor. Taking the last step from letter-level concurrent editing to real-time, by sharing “Before Save” mind states, Wave enables the physical manifestation of joint intention. In fact, there is a conceptual bug in the Wave’s user experience metaphor. They seem to stick to single authorship and associate an addition to a wave with whoever hits the “Save” button. In fact, if you and I were watching each other type, and possibly modifying each other’s words, both our “Save” events should be associated with both of us. True joint-authorship.

Me: Wait a minute. In real conversations you have concurrency — over the shared state of the air between the speakers — and real time. But we still transcribe conversations as “he said/she said.” So speaker-identity isn’t that ambiguous. You are trying to pull a fast one here.

Roshi Tsu Nami: Not really. Oral conversations are not the best analogy. What about harmonized Western singing or jamming in jazz or raga music? Would you say “he and she sang” or “they sang?”

Prof. Sophius Trie: Or for that matter, different cultural metaphors for conversations. Arab conversations are called musayara, or “going together.”

Roshi Tsu Nami: Or even take this conversation. If it had been in Wave, it would have acquired content and coherence as it progressed, and read like an essay by a single entity. That’s one reason expository synthesis-dialogues like this one seem slightly retarded compared to, say, well-written movie scripts, which are driven by conflict rather than synthesis.

Me: Fine, I am convinced. I can imagine a highly harmonized theater audience, for instance, achieving a level of hive-like mind meld where they truly jointly author a review of the play by typing together without saving for 10 minutes. At the artistically appropriate time, one person just happens to be the one who hits “Save.” Clearly, he didn’t author the whole thing. Reductio ad absurdum. Also reminds me of the Matlab programming contests where contestants are encouraged to steal and copy from each others solutions and improve them. And I suppose from far away, you can ignore the edit history pages of Wikipedia and truly say I/We/Gaia wrote the damn thing, as that phrase in Asimov’s Foundation’s Edge goes. I guess in the future, it may be I/Wave/Google.

Roshi Tsu Nami: So what’s your problem? You still sound annoyed.

Me: Fine. I admit I have a problem. It isn’t a conceptual or technical one, but an existential one. I don’t like threats to individualism. I am attached to the idea of the irreducible subjective.

Wave Minds and Singularities

Prof. Sophius Trie: Fair enough. It is a legitimate philosophical worry. Let’s start by asking: we’ve traditionally thought of collective-minds, sentient or not, in subject terms. Leviathan, Borg, Gaia, Skynet. There is an I-like entity. That seems to worry you, right? Your subjective I getting subsumed in a larger subjective I?

Me: You are right, it is a philosophical worry as much as a personal distaste. I think it is simply not true.  It is an attachment to the Ego perhaps, as the master might say, but “We” are never creative or sentient in the way “I” am.

Roshi Tsu Nami: I don’t think the Professor is trying to go there. The opposite of subject is object. The collective can be an object. As John Donne said in Devotions upon Emergent Occasions, “all mankind is of one author, and is one volume; when one man dies, one chapter is not torn out of the book, but translated into a better language; and every chapter must be so translated; God employs several translators; some pieces are translated by age, some by sickness, some by war, some by justice; but God’s hand is in every translation, and his hand shall bind up all our scattered leaves again for that library where every book shall lie open to one another….any man’s death diminishes me, because I am involved in mankind, and therefore never send to know for whom the bell tolls; it tolls for thee.”

Me: So your point is that the I/We distinction is less relevant than the I/That distinction? Subject and object are one? I am that, tvam tat asi? And that is supposed to comfort me? I suppose it is a comforting idea. If my sense of self relies on something as arbitrary as the sanctity of the “Save” button separating my mind stuff from you-mind stuff and we-mind stuff, then it is a shaky construct. Okay, I kinda like that. Reminds me of Skinner’s idea of detachement from the act of creation, in “On Having a Poem.” So long as I can still assert, “I am not what I do,” I am happy. The irreducible subjective is safe.

Prof. Sophius Trie: Hold on, there is more. It is not just the subject-object distinction, it is the material-informational distinction, or as we called it in an earlier context, the wave-particle duality question. What if the Hive Mind is neither subject, nor object, but a transient condition of a wave?

Roshi Tsu Nami: That’s a tricky idea: that reduces the Hive Mind/Book to a temporal entity rather than a material or spatial one. In the context of Google Wave, I suppose that translates to a huge Wave that cascades over the planet’s wave servers. Rather like the wave equivalent of a massively viral chain email.

Me: The temporal social analogue of enlightenment perhaps? Just like Dialectica went to an all-zeros state? Or all ones?

Roshi Tsu Nami: The analogy is exact. Marvin Minsky, after all called his early multi-agent/distributed model of the mind the society of mind. The skull is just an arbitrary boundary for a particular nation populated by particularly stupid agents. This particular conversation even, is just 3 somewhat stupid characters in one mind, which hopefully add up to something smarter.

Prof. Sophius Trie: If you truly are worried, in an ego-preserving way,  about being subsumed and absorbed from without, you should also worry about your ego being fragmented from within into a million pieces. Or 3, as here.

Me: Okay, that holds water. I see value in individualism primarily as stance most capable of creating truth, not because I see ego-value in it. By your model, coherence and synchronicity are wave conditions. Sometimes the I is strongest within my skull, sometimes within my left frontal lobe, and at other times, within a larger group. We all merely differ in how long we can stay in various levels of coherence. Dogs and cats in fact. My inability to participate in what Dawkins called the God Delusion may be an intellectual choice, but my inability to participate deeply and immersively in any kind of larger meaning construction might just be a personality trait. I’ll never sing in an orchestra, and I’ll never be the specialist whose I spends most of its time in one of corner of the brain.

Roshi Tsu Nami: The model sheds a different light on the idea of the singularity then. Unlike the Terminator Skynet gaining sentience once and for all, or the sort of singularity some AI types worry about, a technological singularity is more like…

Prof. Sophius Trie: …A tsumani moment, yes. The equivalent of highly viral chain emails will be big Wave tsunamis that will feel like true, rather than figurative, social Enlightenment events.

Roshi Tsu Nami: Aha! I/Wave/Google am/are/is Tsu Nami.

Me: Though of course, transient phenomena can lock irreversibly into non-transient structures. So perhaps the Singularity will be brought about the way a vending machine is toppled. You rock it back and forth a few times until it topples. By succeeding Google Waves of massive amplitude and increasing wavelengths I suppose, that eventually cause a phase transition. We will get to the God Wave.

Roshi Tsu Nami: Which brings me back to my question: would a hypothetical massive, globe-spanning Google Wave be like a large viral email event? Or something else?

Prof. Sophius Trie: I think it will be a different beast. Chain emails, remember, only have power through propagation. Waves can change their character even while standing still, structurally speaking, since original messages can be edited. So even if a large tsunami isn’t propagating further, it can gather energy and momentum while standing still.

Me: There’s a doomsday scenario for you. Wikipedia decides to port itself from MediaWiki to Google Wave and become a large standing-wave conversation. At some point, our Nigerian genius, Patrick Okoye, comes up with a really cleverly crafted bank-scam 2.0 wave that infects Wikipedia. All the world’s collective knowledge. Our planet-sized John-Donne global book/object-Hive, will be one constantly morphing Nigerian bank scam wave.

Roshi Tsu Nami: That is indeed a beautiful future picture. The Singularity as the moment when the entire world’s consciousness freezes into “Nigerian Bank Scam,” causing civilization to crash. I doubt machines could run the world with “Nigerian Bank Scam” as the operating system.

Prof. Sophius Trie: You forget: robot Web services can be equal partners in Waves, just like humans. We do not need a human to trigger a dystopia. Imagine a future where every physical device — sensors, actuators and all — is also a robot in the Google Wave sense. It will have its own feedback loop connecting it to the ground truth data of material reality. The non-human part of the world can develop its own scientific method and notions of falsifiability. By being equal participants in Google Wave, they can influence things. So they may trigger a wave that causes the singularity to occur, and result in a reality-grounded state where civilization doesn’t collapse. Of course, the  idea of Bay’s Law suggests that distributed AI will be self-limiting, and never reach that level, due to humans hitting their limits first.

Me: We’re getting carried away here. Is all this relevant? We are just talking about a relatively small piece of software here.

Prof. Sophius Trie:  But it is a complex one that is trying to displace a deeply embedded existing system, email. Most Web innovations have an adoption cycle on the order of years. This one may take decades to have its full impact. It took steamships decades to displace sail ships. Joel Mokyr says as much in The Lever of Riches,

“Some technological systems, such as ships, mines and farms, are complex and interrelated.  Dramatic, sudden changes are not impossible in such systems, but are less likely because of the need to preserve compatability with other components. Because of the resistance of other parts of the system, large changes were slow in the making… As we have seen, the transition from sailing ships to steamships in the nineteenth century [...took...] half a century. During those decades, every part of the ship, from mast to rudder, was redesigned. Gradual change will be the rule when the complementary technical support system is inadequate to support macroinventions.”

Roshi Tsu Nami: So the trajectory is going to be long, and this thing can grow to have a much bigger impact than it seems capable of at the moment.

Me:I don’t know. Maybe not. Perhaps we are at our cognitive limit. Perhaps we will never learn to truly wave.

Disclaimer: everything I know about Google Wave is from the video and second-hand reporting. I could be making all sorts of mistaken assumptions here. And I am only about 34.8% serious in this post, so don’t quote me.

A few years ago, I was part of a two-day DARPA workshop on the theme of “Embedded Humans.” These things tend to be brain-numbing, so you know an idea is a good one if it manages to stick in your head. One idea really stayed with me, and we’ll call it Bay’s conjecture (John Bay, who proposed it, has held several senior military research positions, and is the author of a well-known technical textbook). It concerns the effect of intelligent automation on work. What happens when the matrix of technology around you gets smarter and smarter, and is able to make decisions on your behalf, for itself and “the overall system?” Bay’s conjecture is the antithesis of the Singularity idea (machines will get smarter and rule us, a la Skynet – I admit I am itching to see Terminator Salvation). In some ways its implications are scarier.

The Conjecture

Bay’s conjecture is simply this:  Autonomous machines are more demanding of their operator than non-autonomous machines. The implication is this picture:

The point of the picture is this: when technology gets smarter, the total work being performed increases. Or in Bay’s words, “force multiplication through accomplishment of more demanding tasks.” Humans are always taking on challenges that are at the edge of the current capability of humans and machines combined. So like a muscle being stressed to failure, total capacity grows, but work grows faster. We never build technology that will actually relieve the load on us and make things simpler. We only end up building technology that creates MORE work for us.

The one exception is what we might call Bay’s corollary: he asserts that if you design systems with the principle of “human override protection,” total work capacity collapses back to the capability of humans alone. We are both too greedy and too lazy for that. We are motivated by the delusional picture in Case 1, and we end up creating Case 2.

Here’s why this is the opposite of Skynet/Singularity. Those ideas are based (in the caricature Sci-Fi/horror version) on the idea that machines, once they get smarter than us, will want to enslave us. In the Matrix, humans are reduced to batteries. In the Terminator series, it is unclear what Skynet wants to do with humans, though I am guessing we’ll find out and it will probably be some sort of naive enslavement.

The point is: the greed-laziness dynamic will probably apply to computer AIs as well. To get the most bang for the buck, humans will have to be at their most free/liberated/creative within the Matrix. So that’s good news. But on the other hand, the complexity of the challenges we take on cannot increase indefinitely. At some point, the humans+machines matrix will take on a challenge that’s too much for us, and we’ll do it with a creaking, high-entropy worldwide technology matrix that is built on rotting, stratified layers of techno-human infrastructure. The whole thing will fail to rise to the challenge and will collapse, dumping us all back into the stone age.

If you read only two books about the brain, Medina’s Brain Rules should probably be your second one (thanks Kapsio, for the recommendation), after Oliver Sacks’ The Man Who Mistook His Wife For A Hat. If you’ve been reading this blog for more than a few months, you might remember a post I did nearly a year ago called The Broken Brain Books. Let me repeat the quote from Steven Johnson’s Mind Wide Open that I used to start that post:

…while it is interesting to find out [the] exact addresses [of brain functions], that information is ultimately unsatisfying. Call it the “neuromap fallacy.” If neuroscience turns out to be mostly good at telling us the location of the “food craving center” or the “jealousy” center,” then it will be of limited relevance to ordinary people seeking a new kind of self-awareness — because learning where jealousy lives in your head doesn’t make you understand the emotion any more clearly. Those neuromaps will be of great interest to scientists of course, and doctors. But to the layperson, they will be little more than trivia.

By this critique (which I wholeheartedly agree with), most ‘brain’ books are a big waste of trees. Medina, thankfully, avoids this trap, and doesn’t even mention fMRIs till fairly late into the book, and when he does, he steps away lightly from pointless fMRI-pornography. That leaves us with 12 brain rules, each of which gets a chapter. The chapter on short-term memory for instance, is titled “repeat to remember.” Well Duh! you might say. Fortunately, there are deeper insights buried within. Despite appearances, the book isn’t an exercise in providing unnecessary proofs for folk-tautologies.

The ‘Rules’ Premise

Let’s be honest. Neuroscience generally makes for extraordinarily dull pop science and stupendously ineffective self-improvement. The discipline seems to have been invented to prove Rutherford’s famous line that “all science is either physics or stamp collecting.” Unlike other stamp-collecting sciences (like say, astronomy or entomology) you can’t even squeeze much sensory romance or sheer wonder into Exhibit A (lump of yucky gray stuff to those of us who were turned off biology by 10th grade frog dissection). You need to resort to computer science metaphors to visualize the marvelous aspects. So it is no wonder that even the most valiant attempts at making this stuff interesting succumb to the neuromap fallacy.

As self-help, the raw material isn’t very rich either. If you stick to prescriptions that soundly rest only on neuroscience, you generally get kindergarten reiterations of stuff that folk wisdom captured in 312 BC. Compared to the subtle-if-shaky snake-oil insights of psychology, neuro-self-help insights seem solid but trite. This is the reason many brain-based bestsellers like Emotional Intelligence cheerfully march beyond neuroscience, into psychology and even outright speculation, to make the stuff at least potentially useful.

So this is tough raw material to work with. Medina’s approach is to present distilled rules that, on the face of it, seem self-evident from the experiences of any normal adult. What makes them rise above banality is the effort he puts into turning your existing beliefs into interesting new behaviors.

Here is an example. Rule #1 says, exercise boosts brain power. You’d have to have incredibly low self-awareness to not have noticed this yourself. Medina adds value to such commonplace articles of faith in two ways. First, he selectively shares science nuggets that enrich your understanding of the why. In the case of Rule #1, one tidbit is as follows. While the main circulation system (those artery/vein names you learned in high school biology) doesn’t change much, apparently the tiny capillaries at the periphery that actually deliver the blood (the ‘last mile’) sprout and decay depending on the vigor of blood pumping through your body. So exercise builds out and maintains the transportation system of your body. This boosts brain power because the Oxygen-hogging brain is the organ most dependent on efficient oxygen delivery and free-radical removal by the bloodstream.

That’s interesting, but still not useful. The next leap gets you to ‘useful.’ Medina draws on our evolutionary history to point out that we probably evolved in an environment where we were walking 12 miles a day. So you get this sort of argument:

Recall that our ancestors walked 12 miles a day. This means our brains were supported on Olympic caliber bodies. I am convinced that integrating exercise into those 8 hours at work or school will not make us smarter. It will make us normal.

That to me is a truly startling and useful way to think about the role of exercise in our lives. I’ve always been unhappy with the arbitrary sounding prescriptions of officious National Nanny health organizations. Why 30 minutes a day of cardio? Why is 3 times a week reasonable (or not)?

This 12-miles-a-day tidbit, on the other hand, with the background reasoning about circulation, really forces you to critically re-examine your lifestyle in fundamental ways. Medina himself claims to have rigged his treadmill to support a laptop so he can work while walking. He even suggests that classes in schools be taught while students are walking on slow treadmills.

But if our brains evolved to depend on circulation systems whose vigor depended on 12 miles/day exercise, we are faced with a serious stretch goal. I did 30 minutes on my elliptical today, and that was maybe 2 miles. I can’t imagine being able to afford 3 hours everyday. More to the point, I’d be bored to tears. Clearly, we need to invent new work behaviors that integrate movement in a meaningful, non-disruptive way. Something better than hamster-cage like prescriptions (treadmills in offices) would be nice. Maybe a heads-up display with voice-recognition input, that you can use to do computer work while walking around the park or hiking? I remember a 10 day intense camping trip from 1996, when we did about 7-8 miles of hilly hiking a day. I don’t recall ever feeling, thinking or sleeping better.

The 12 rules aren’t all equally good. The chapter on learning (a rule framed as an assertion, “every brain is wired differently,” rather than as an imperative) doesn’t manage quite as successful a leap from relevant science to useful behavior modification.

The Rules

There is a very nice website with a lot of cool videos associated with the book, and the book itself comes with a DVD of collateral. The rules themselves, a mix of imperatives and assertions like I said, come with disarmingly cute icons (clearly Medina applied all his insights to the production of the book itself). Rather than type up the list, let me show them to you via a screen-capture from the website, which includes the icons. You really should go explore the site. Here are the rules:

Nice, huh? Apply rules 5 and 6 and commit to memory. Should serve you well as design principles for your future self-improvement efforts.

The Beyond-the-Brain Prescriptions

The one weak part of the book is the set of end-of-chapter ‘ideas’ sections, where Medina tries to suggest ways to apply the principles in social contexts, particularly schools and workplaces. This is certainly an important thing to think about. Yes, we could all individually try to build more brain-friendly lifestyles, but it would help if the larger environment had a baseline level of brain-friendliness.

But most of the suggestions fall somewhere between naive and you can’t be serious. In some cases, the writing suggests that Medina is aware of this, and is basically trying to calibrate the efforts of others, by providing examples of how dramatic interventions would need to be. But in other cases, he seems to take throwaway thought-starter ideas seriously. Still, it is a brave effort, and we shouldn’t expect him to be brilliant in a domain that really belongs to experienced organization theorists (like, ahem!, me). We shouldn’t be surprised that Michael Jordan didn’t succeed at baseball (an example he uses to make a point about learning).

All in all, five stars. Buy/read this book. Or at least watch the free videos on the site.

Imagine that there is a mystery closed curve about the origin. Allow two parallel lines to approach the origin from diametrically opposed directions, and have them stop where they first become tangent to the mystery curve. Suppose you do this from all pairs of directions from (0,π) to (π,0), and find that the lines stop the same distance apart everywhere. What is the mystery curve? (Don’t worry, this isn’t a post about mathematics!) You thought of a circle, right? Is that the only answer?

Many years ago, when the professor in a class on differential geometry asked that question, and provided the answer (“No”), I became fascinated by mathematical counterexamples. In this case, the Realeaux triangle is the simplest counterexample; British coins and Wankel engine pistons use these shapes:

Realeaux triangle (Wikimedia Commons image)

You can also make such shapes with more sides, and get rid of the sharp corners (brainteaser: how?) to get smooth equal-width curves (the generic name for such shapes). The tiny shock of seeing your assumptions undermined, and your gut-level conviction overturned, provides a high that I call an UnAha! experience.

There is an odd exhiliration to be found in showing your “gut” instinct that it is wrong. The gut is driven towards conviction. It provides to decisions a sense of moral certitude; “rightness” lives in the gut. Counter-examples keep the capacity for doubt alive. Of course, you can then go back and re-examine where your certitude is coming from, and get deeper insights as a result. I talked about this in my previous pieces, How to Define Concepts, and Concepts and Prototypes.I once knew a woman whose rhetorical style was entirely gut-driven: all declaration of (and provocation of) pathos and appeals to ethos. Her signature phrase was “I just know.” She would stare pityingly at those of who labored to add logos to the argument. Luckily for her, she seemed to find an ennobling grace in defeat.

“What is thought?” is a question that is foundational by any reasonable measure. The best short answer I have found so far has been “thought is conceptual metaphor,” and it is one of the enduring regrets of my life that it took me so long to encounter this answer. An undergraduate friend (hi there Max!) introduced me to George Lakoff and the notion he introduced, conceptual metaphor, just as I was finishing up my PhD, and it radically altered my thinking (and my thinking about thinking, a.k.a philosophy) from that point on. I can only wonder how different my life would have been if I’d read Metaphors We Live By as an undergraduate. So here is a discursive introduction to these ideas.

Sapir-Whorf

I was educated to believe that the fundamental historical schism in modern cognitive science was the one between behaviorism and the cognitive school, which came to a head with Chomsky’s review of Skinner’s Verbal Behavior (1959). That particular drama though, however you interpret it, is actually rather superficial.

The truly interesting yin-yang dynamic in the study of language (and as you will see, that implies the study of thought itself) is between the relative roles of universal and local components of thought. Chomsky’s “Universal Grammar” plays the former role. The role of “local component” is played, I will argue, by Lakoff’s idea of conceptual metaphor, which I’ll explain in a minute. The foundation of conceptual metaphor though, is a somewhat controversial (and in a strict form, untrue) statement called the Sapir-Whorf hypothesis, one articulation of which (Whorf, 1959) reads as follows:

We dissect nature along lines laid down by our native language. The categories and types that we isolate from the world of phenomena we do not find there because they stare every observer in the face; on the contrary, the world is presented in a kaleidoscope flux of impressions which has to be organized by our minds — and this means largely by the linguistic systems of our minds. We cut nature up, organize it into concepts, and ascribe significances as we do, largely because we are parties to an agreement to organize it in this way — an agreement that holds throughout our speech community and is codified in the patterns of our language […] all observers are not led by the same physical evidence to the same picture of the universe, unless their linguistic backgrounds are similar, or can in some way be calibrated.

Stated as baldly as above, the hypothesis is actually not much deeper than the idea that language is a matter of social convention (an angle which has been studied by philosopher David Lewis in Convention: A Philosophical Study). In other words, arbitrary conventions which arise as a matter of historical accident and path-dependence determine language, which in turn shape thought.

If this concept is new to you, it will take some thought to get past the superficial contradiction and develop the right mental model of this particular yin-yang. Roughly, the synthesis is similar to that of the interplay of nature and nurture in genetics (the former defines the potential, the latter the expression thereof in a particular environment). You could say that Chomsky tells us what sorts of thoughts we could think, while Sapir-Whorf hints at what types of thoughts we do think.

This level of resolution seems to satisfy many linguists and computer scientists (correct me if I am wrong, but roughly the current consensus in the NLP community is that human languages are describable by “weakly”context-sensitive grammars).

An Example

Let’s look at an example to highlight some issues. I’ll use English and Hindi, two languages I understand, to highlight some aspects of thought-as-language. I recently wondered about the Hindi equivalent of the common English word waste.

Despite racking my brain and looking up a Hindi-English dictionary, I could come up with nothing that means precisely what the word means in English. There are related words — kuda or kachda (garbage), banjar (wasteland), nasht (destruction), barbaadi (ruin), bekaar (useless), anavashyak (unnecessary) and fazool (superfluous).

Even adjusting for the fact that Hindi (and Urdu) have much smaller, slower-growing vocabularies, it seems curious that such a basic word does not exist. The clue lies in the fact that our modern sense of the term waste is something like “to not use/use suboptimally a valuable commodity or resource, and thereby allow it to degenerate.” If you check out the English etymology of “waste” though, you’ll find that older uses of the word “waste” were closer to the set of related Hindi words I’ve listed. Usages like to lay waste, or ruin or waste away comprise the origins of the term.

Though I can’t prove it, a reasonable hypothesis is this: the modern sense of “waste” relies on a protestant/puritan ethic and an industrial cultural context that has a general mental model of “manufactured, value-added resource” and “optimal resource utilization.” Older cultures have specific notions of “waste-able” resources like land, and their associated notions of loss involve natural or destructive-intent processes (such as ruin or decay). In an industrial context, there is an added sense of hard-won added value being lost. In other words, waste is mainly a waste of effort today.

The implication here is that a very complex environmental factor (an industrial culture) can affect the meanings of individual words.

But the basic linguistic analysis of this example is unsatisfactory. Chomskyean notions of grammars have nothing to say at all. Sapir-Whorf merely implies that perhaps modern English speakers can think thoughts involving the concept of “waste” (not the word) that Hindi speakers cannot. To this, I can testify; I cannot think certain thoughts in Hindi that I can in English. The converse also happens, but less often (a larger, faster-growing language does not necessarily subsume a smaller, slower-growing one — the set of all even numbers below 1000 does not subsume the set of multiples of 5 under 500).

Lost in Translation

The problem of translation sheds further light on the relation between thought and language. Imagine a pair of languages to be represented by a dynamic Venn diagram with a large, fast-growing circle (English) intersecting a smaller, slower-growing one (Hindi). Given the non-subsumption remark, and the fact the the evolution of language is a creative-destructive rather than monotonic process, perhaps we should imagine growing, morphing fractal shapes, but we’ll let that go for now. Would it be fair to say that pieces of text involving only words in the intersection are perfectly translatable both ways? This is obviously silly. The effect of a piece of text is the collective impact of a set of words and their nearby connotations, within a cultural context. Here is an example, a verse from a popular 1950s Hindi song:

Sab kuch seekha humne, na seekhi hoshiyari
Yeh sach hai duniya waalon, ki hum hai anadi

which translates roughly to:

I learned everything, but I did not learn street-smartness
It is true, oh people of the world, that I am a naive bumpkin

This is what I would call a partially idiomatic translation — a subjectively judged mix of direct translation, and translation of the original’s inferred intent, that makes use of idiom and metaphor. Let’s look at a couple of pieces in detail.

Zooming in on the red bit, the Hindi-Urdu hoshiyari derives from the stem hosh, which means both “alert” (in both the medical sense of “lucid” and the sense of “watchful”) as well as “conscious.” Behosh is “unconscious” and Hoshiyar! is equivalent to the Western military call Attention! Hoshiyari as an abstract noun denotes a mix of intelligence, pragmatism and cleverness. You’d apply it to a precocious child, or in the sense of “too clever by half” to an adult. A Hindi speaker then, would find evoked mentally, a conscious sense of “clever/wordly-wise” and a subconscious web of associations stretching to concepts like “conscious” and “lucid.” Clearly the closest direct translations (such as “clever”) wouldn’t work. You need an English idiom to capture the Hindi word, because you are trying to capture the impact of a web of connotations around n particularly leaky, ambiguous term.

Something similar holds for the blue highlighted bits. The closest direct word I can think of, for anadi, is ingenue, but this explicitly refers to a female sufferer of naivete, and has the added disadvantage of a particular cultural prototype (a young French girl being introduced to Paris, say). Anadi on the other hand, has the prototype of a bumbling male simpleton, often a rural migrant to the Big City. Something like “country mouse” would work very well, but the reference to a very specific Western folk tale has its own problems, hence the decision to go with a clumsy compound non-metaphoric phrase, naive bumpkin.

A word also has irrelevant connotations, via similar-sounding but unrelated words. Anadi (sometimes spelt anari) sounds similar to anar (pomegranate). A Hindi-speaking listener to the original, therefore, might have a subconscious sense of “pomegranate” which an English-speaking listener to the translation would not.

So, to go back to our original Venn diagram, given any set of words in the intersection, the webs of associations and non sequiturs radiating out from original and translated versions will inevitably leak all over, leading to very different effects in the two languages. Translation then, is a matter of keeping the webs as contained and close as possible. In my limited experience (I have done a couple of serious Hindi-English translations like this one), the balance between metaphoric/idiomatic and literal that best captures the original intent as perceived by a single individual is a delicate one that depends a LOT on the piece under consideration. Add the fact that you generally want to use syntactic structures of similar compactness in translation, to generate comparable rhythms and cadences on both sides, so nonverbal effects harmonize, and translation begins to appear impossible.

But it gets worse. Meaning, remember, is constructed by an individual listener with a history and a mental model of the world, not by an abstraction like “English Speaker.” We’ll get to that level of detail after reviewing conceptual metaphor.

Conceptual Metaphor

Ask yourself this: in the previous example, why did a more ambiguous construct (the metaphor street-smart) work better to contain the original meaning of hoshiyar than a nominally-close direct translation like “smart”? One key is that the metaphor anchors to a non-linguistic sensory experience (a swaggering person on a street — perhaps English speakers think of John Travolta in Grease, while Indians think of Aamir Khan in Rangeela) that constrains meaning more powerfully than abstract words could.

Lakoff and Johnson, in their seminal Metaphors We Live By, turned that sort of tiny, trivial observation into a powerful theory of how thought and language work. Their crucial first step was to distinguish visible, obvious metaphors, or figurative metaphors, from more systematic, large-scale and linguistically less-visible entities they named conceptual metaphors. “Inflation is up” is an example. “Up” is a geometric notion that relates to space and inflation is an abstract financial quantity. The statement involves not one but two conceptual metaphors: a spatial-0rientational one (“Up”) and a material-expansion one (“inflate”). The breakthrough achieved by Lakoff and Johnson was in realizing that these are not a minor, subtle and exceptional third category beyond literal and figurative language. Their work showed that such conceptual metaphors account for practically all language, and also get to vast realms of non-linguistic (such as mathematical) thought. In the Lakoff-Johnson approach, you describe a conceptual metaphor with an “X is Y” title. Here are parts of their opening examples (Chapters 1 and 2 of MWLB; you can find many more in this repository):

ARGUMENT IS WAR

Your claims are indefensible
He attacked every weak point in my argument
I’ve never won an argument with him
You disagree? Okay, shoot!

TIME IS MONEY

You’re wasting my time
I don’t have the time to give you
I’ve invested a lot of time in her
He is living on borrowed time

These are very different from figurative metaphors (“He was a lion in the battle” or “The architecture was very musical”). They are at once more subtle, broader and more systematic in scope, and fundamentally, not about language at all. MWLB points out that this is the normal way English speakers talk about arguments and time. These are not poetic or extraordinary uses. Moreover, these are not necessary ways of talking about arguments or time (though I believe that some conceptual metaphor is usually necessary, just not any specific one). MWLB offers up an quick analysis of the first one: “Imagine a culture where an argument is viewed as a dance, the participants are seen as performers, and the goal is to perform in a balanced and aesthetically pleasing way.” The fact that we can imagine an alternate possible world where arguments are understood differently tells us that ARGUMENT IS WAR is not the only way to structure thinking about arguments.

Conceptual metaphor is a very complex concept. MWLB covers very fundamental ones (such as spatio-temporal, ontological and causation metaphors) that appear in every language, as well as more localized, less fundamental ones. It is hard to define the idea, but here is one of the better articulations (Chapter 1):

The essence of metaphor is understanding and experiencing one kind of experience in terms of another.

The key attributes of conceptual metaphor are systematicity (concepts maintain their relationships in a gestalt sense), incompleteness (ARGUMENT IS WAR and ARGUMENT IS DANCE highlight different subsets of the framework of concepts and relations called ARGUMENT, and neither is complete), and sensory nature (the conceptual metaphor generally maps a more abstract domain to a more sensory one). There are also much deeper aspects, like the distinction between metaphor on the one hand and generalization and abstraction on the other.

It is not the intent of this piece to provide a tutorial on conceptual metaphor. You could try some online material, such as the Wikipedia entry, but there is really no substitute to reading Metaphors We Live By. But we can get a sense for the profundity of the idea simply by looking at the prolific output of Lakoff and his collaborators since MWBW. In books like Don’t Think of an Elephant and Moral Politics the ideas of metaphoric frames and narratives are explored in the context of political discourses. In Philosophy in the Flesh the ideas are ambitiously applied to the entire field of philosophy of mind.

But perhaps most interesting for people like me, is Where Mathematics Comes From, which extends these ideas to an examination of mathematical thought. If you thought mathematics could be anchored in sensory thought only to the limit of 3d geometry, think again — all mathematics is driven metaphorically. Why do you “plug” things into equations? Why do you “move” terms in equations, or “group” them, or “crank through” a derivation, when all that is physically happening is repeated writing of related symbolic sentences? Even at a very superficial level, a MATHEMATICS IS MECHANICAL MANIPULATION metaphor works very well. You can get much deeper, all the way to a beautiful explanation of Euler’s very abstract, apparently far-beyond-intuitive grasp equation,

[tex]e^{i\pi}+1=0[/tex]

using an appropriate application of conceptual metaphors at work (explaining the equation above metaphorically is the tour de force bit of WMCF).

Beyond Conceptual Metaphor

The idea of conceptual metaphor goes far beyond its linguistic roots, once you understand that it is a way of talking about mental models. We organize our sensory experience, starting with what William James called the “blooming buzzing confusion” perceived in childhood, using some very basic pre-linguistic conceptual categories, relations and intuitions of causation and dynamicity. On this base is constructed layer after layer of inter-related models of parts of experience. While conceptual metaphor manifests itself most vividly in language, it also manifests itself in mathematics, geometric thinking, abstract visual thinking (every graph you ever drew or saw is a metaphor), narratives and storytelling and modal/subjunctive reasoning about possible worlds. It is also central to sophisticated decision-making. It is also foundational to the thinking style that is loosely referred to as “right brained,” which explains why Lakoff looms large in Dan Pink’s A Whole New Mind . Conceptual-metaphoric thought is also fundamental to ideation.

Not all these directions have been thoroughly explored, but there is definitely no shortage of evidence that conceptual metaphor is a foundational element of thought. Even my original reductive definition: thought is conceptual definition, doesn’t seem too hasty, once you examine the full scope of ideas we are talking about.

You can even get beyond humans to computers. Kenneth Iverson’s seminal 1979 ACM Turing Award Lecture, “Notation as a Tool of Thought,” while not explicitly about metaphor, is essentially a Lak0ff-and-Johnson for constructed symbol systems used as programming or mathematical languages. Another interesting unusual domain where the ideas of conceptual metaphor (understood in the broader sense of “mental models”) shed light, is in an analysis of modern “silo” languages and their inter-relationships, and the role played by modeling and language in getting, for instance, accountants, managers, lawyers and marketers to understand each other in a modern corporation. This issue has been explored in the fascinating (and mostly non-technical) article, “On languages for dynamic resource scheduling problems”

The Last Frontier: One-to-One Communication

Ultimately, languages viewed as larger cultural entities are relatively easy. I’ll wrap up by revisiting the question I raised earlier. What happens when you acknowledge that meaning is constructed by an individual listener with a history and a mental model of the world, not by an abstraction like “English Speaker?” Analyzing communication at its atomic, 1:1 level is so hard that it has led to statements such as Wiio’s law: communication usually fails, except by accident (a commentary on this is to be found here). This observation goes beyond the trite observation that all models (conceptual-mental or mathematical) are finite and therefore incomplete.

One-to-one communication is nothing less than the interplay of two dynamic, evolving entities. It is not about the transfer of specific intended bits of information from one locus to another. It is more like billiard balls influencing each other via collisions. Yes, bits get transferred, but the intended bits getting transferred mostly happens by accident. Even toy examples are subtle. Take a trivial example: if in December I tell you “It is snowing in Rochester” I am not transferring the one bit of information “SnowRochester=ON,” I am more likely provoking the thought “This guy is an idiot who states the obvious” or “Hmm…an invitation to a casual conversation/social ritual.” Even if the bit IS substantive (like saying, in December, “It is sunny and 80 degrees today in Rochester”) the sentence will likely provoke the thought, “Hmm, there is an unseasonable weather pattern…global warming?” rather than the overt predicate being transmitted.

Two speakers of the same language are very different information processing systems. Both have different vocabularies (both conceptual and linguistics) arranged in very different mental models of the world.

Really, what happens when A speaks a sentence to B, is that B’s internal mental model of the universe, or “information state” gets updated. This is the idea underlying the relatively modern field of dynamic semantics, which I recently learned about through a friend completing a PhD in the philosophy of language. This closely parallels the model of cognition implicit in the belief-desire-intention (BDI) approach in the philosophy of action and AI, and also parallels the basic conceptual model of control theory.

Which should explain why, if there is a single most foundational idea behind much of my writing on this blog, it is conceptual metaphor.

Nearly 10 years ago, in Clockspeed, Charles Fine of MIT revived a metaphor for the economy that goes back to at least Herbert Spencer’s essay, On The Social Organism (1860). A colleague recommended the book because I’ve lately been obsessed with issues of speed in innovation. Read as an anecdote-rich exposition of concurrent engineering, it is pretty good. As a justification of its title, it is badly derailed, since the limited discussion of time scales in the business world goes nowhere, least of all towards justification of the subtitle “winning industry control in the age of temporary advantage.” But the book, despite its value, mainly struck me as a massive missed opportunity to explore the metaphor of business genetics. In this piece, I attempt to remedy this gap with the benefit of 10 years of hindsight.

(Warning: despite its length, this is a highly compacted, beta version of an idea that probably needs book-length systematic treatment; in its current form the piece is rather quick and dirty. I’ll work on a more polished/full treatment when I get the right opportunity).

The Missed Opportunity

Anecdote after revealing anecdote parades past you on the pages of Clockspeed. Most will be unfamiliar to you (like the enthralling description of the evolution of the bicycle industry). You’d think somebody sitting on top of this fantastic data set, with the best graduate students in the world, would do more with it. Maybe I am being unjust, since the angle I am about to present would probably only occur to somebody who has read the works of George Lakoff (who, being from the academic antipode of Berkeley, and sort of an anti-Noam-Chomsky, is possibly not well known in industry-obsessed MIT)

Clockspeed, had it been less hurried than it seems to have been, could have been seminal, on par with The Innovator’s Dilemma from about the same time. It could have been the successor to Herbert Simon’s The Sciences of the Artificial and Wiener’s Cybernetics. Instead it ended up being merely good, and even its one candidate neologism for posterity, clockspeed, is now fading from the business lexicon. I wouldn’t have heard of it but for a senior colleague.

So why didn’t this book become great? The answer lies in the Churchill quote, “Men occasionally stumble over the truth, but most of them pick themselves up and hurry off as if nothing ever happened.” The truth that Fine stumbled upon but didn’t explore, is the genetics/organic metaphor for business.

The “stumble” is an observation in Chapter 1 that the book is inspired by how biologists use studies of the fast-mutating/evolving fruit fly to inform their investigation of questions applicable to all species. Unfortunately, Fine abandons the metaphor as a systematic conceptual metaphor instantly, only retaining the figurative metaphor aspect. Besides the too-cute chapter titles, this mis-step is most vividly illustrated in Fine’s treatment of the idea that there is a cyclic dynamic to vertical/horizontal structure through the evolution of an industry, which Fine calls the “double helix.” A disorienting figurative metaphor that confuses the deeper conceptual metaphor.

Even the highlighted bit of the genetics metaphor, “genetic evolution rate” doesn’t get properly examined. You end the book confused about what “evolution clock rate” actually maps to: product lifespan? Company lifespan? Product-platform lifespan? Modular component lifespan? CEO lifespan? Fine himself explicitly argues that no meaningful answer is possible because of the hundreds of different “rates” in the economic organism. He is too pessimistic, as you will see. Applying conceptual metaphor allows you to weed away the irrelevant and actually begin mapping “evolution rate” correctly.

But before I try to remedy the situation, a quick summary of what the book does do well.

The One-Minute Summary

You could say Clockspeed is an extended exploration of three (fairly defensible) assertions:

1. Increasing Clockspeed: The basic cyclic frequencies (product, process, etc.) driving evolution of an industry increase over time (the “product life cycle shortens” observation)
2. The Vertical/Horizontal Cycle: This is the idea that industries cycle slowly between highly vertically-integrated and horizontal/modular. Power shifts cyclically between integrators and suppliers, as does the driving competition. Industry fundamentals determine the lower limit of achievable modularity (Boeing will never be Lego, for instance).
3. 3d Concurrent Engineering: Yes, the term is dated, but it is still meaningful. The idea that you need to design the product, process and supply chain in parallel. A descendant of previous generations of design-for-manufacturability thinking, and today represented by Design for Lean Six Sigma (DfLSS).

If you wanted to understand these ideas in their latest incarnation, you are probably best off studying Open Innovation and Wikinomics, though these more current writers seem to have forgotten some interesting older lessons, which makes Clockspeed still valuable. In particular, all of today’s rather right-brained talk of economic “orchestration” and “ecosystems” tends to cloud the hard-nosed left-brained discipline of build/buy decisions and the fact that supply chains (webs, rather) need to be continually re-engineered with an eye on financial analytics, not just musically, and artistically orchestrated (that old classic, The Goal, is still the best balanced-brained exposition of this theme).

Enough on the healthy, still-alive parts of the book. Let’s scrub in for the surgery to implant the missing organs.

The “Business Genetics” Metaphor

Let’s finish the job for Fine. What is the best way to apply the metaphor of “genetics” and “evolution” to the business world? More specifically, we want to map the quantity “evolution rate” within this larger structural mapping. To do so, we’d have to correctly map the major aspects of evolutionary phenomenology to aspects of the business world, preserving relationships among the entities. Herbert Spencer’s 1860s mapping won’t do (neither will Hobbes’ Leviathan and older incarnations). Here are some of the main features to be mapped:

1. The organism and its cells
2. The species
3. The ecosystem
4. The food chain (from photosynthesis up to sharks)
5. DNA, genotype and phenotype
6. Genetic mixing during sexual reproduction
7. Mutation
8. Ontogeny and cell division
9. Various characteristic time scales, most importantly, periods between significant DNA change, and ontogenic phases.

Commonly, when people refer to a notion of “corporate DNA” they have in mind one of two things: the “signature” features of products coming from the company (such as the distinctive look of any BMW), or an abstract notion of “core competence” (such as Apple and design for usability).

Implicit in this sort of talk is the idea, for product companies, that the product is the organism and design is its DNA. This is beguiling but wrong. It is beguiling because talk of “product families” and “generations” and “platforms/variants” is very useful, and it is easy to think of blueprints as “DNA.” It doesn’t help that many products are anthropomorphically designed (car headlights=eyes). This was my starting point, and only when I went nuts trying to flesh out this mapping did I realize it was wrong. Here is the right answer (for products; I’ll comment on services later):

1. The organism is the entire historical fleet of comparable products produced by a company. For an automaker, this is the whole set from the oldest still-supported car in existence to the latest new release. Each individual product maps to a cell.
2. The “species” maps to the set of all comparable fleets in existence. The airliner industry thus has two “animals” — the fielded still-flying fleets of Airbus and Boeing.
3. The “ecosystem” maps to the set of interacting species. For example airliner fleets and airports chains.
4. The food chain confused me the most initially, since I had two incorrect visualizations in my head. First, of companies “eating” other companies via M&A, and second, companies “eating” the output of their upstream supply chain partners. This is wrong because feeding is an entropy-increasing process for the “food”, where complex things get broken down to biological waste. The right place to map this is to consumables: gas and rubber tires for cars, paper and toner for photocopiers. I’ll leave you to figure out why this works.
5. DNA is the heart of the matter. In biological systems, every cell of every organism carries an explicit copy of the instruction set that produces the entire organism. Think about it. If you took a single BMW to another planet, you should be able to replicate the birth, growth and present condition of the entire BMW product fleet. The crucial observation for our metaphor is that DNA maps to the entire corporation’s roles and processes, including the extended supply/distribution/franchise web and is not contained in each cell. The best explicit description is the set of organization charts and process documentation spread throughout the organization. Think ACGT — adenosine, cytidine, guanosine, and thymidine and genetic grammar. These don’t map at all. This tells us that the genetic metaphor maps best to a pre-DNA stage of the evolution of life, when a variety of autocatalytic replicator molecules hypothesized by Stuart Kauffman drove the emergence of life. Even this is incomplete, since only a single copy of the DNA exists, and outside the product fleet (organism). The phenotype however, is easier to map. The characteristic BMW grille is an element of the BMW phenotype. The overall brand perception may be viewed as the uniquely identifiable phenotype.
6. Since there is no equivalent of DNA, there is no precise analogue of sexual reproduction and chromosonal mixing. If we look, instead, for “spiky” changes in the DNA, we are drawn to mergers and acquisition or bankruptcy-driven asset sales. Partnerships and initiation/breakup of supply chain relationships are also analogous, but smaller spikes. Remember though that the organism (product fleet) is not a self-replicating entity. When Chrysler merges with Daimler, the car fleets don’t get in bed with each other. The DNA, which exists outside the organism as the corporation’s supply web, and partly merges with the other DNA set.
7. Mutation is hard to map — a mutation is a useful mistake in the DNA that is passed on through inheritance through sexual reproduction. The Post-It note, discovered by accident, almost killed at 3M, and now a billion-dollar line of business, is a mutation. Presumably, if 3M ever goes bankrupt or evolves through M&A, the Post-it “mutant gene” production capability, will survive as a very successful mutation.
8. Ontogeny is the most interesting to map. Remember, we start as a bunch of stem cells, go through a roughly phylogeny-recapitulating process as the cell mass differentiates into different organ systems. It is very easy to get confused and attempt to map this process to the company itself (baby startup stage, growing pains, evolution into the Fortune 100…). This is wrong. Remember the “corporation” is part of the single external copy of the DNA, so its growth is actually the growth of the DNA code-base itself. What you actually want to map to is the growth from first prototype, through replication (mass manufacture), introduction of variant products (major tissue types), emergence of clearly defined product categories with their own management and marketing (organ systems) and so forth. You will be tempted to look for analogies to the complex coordination of life processes and homeostatic loops. But remember, by our mapping of “organism” to the fielded product fleet, with individual products as cells, this level of complexity actually does not exist in most products. Only internet-enabled products where the vast fleet of individual “cells” is able to interact, communicate, collaborate and drive “higher-level” behaviors admit this mapping. Viewed this way, Napster might well be the first business organism to evolve that is more complex than algae.
9. Time scales. Clearly,the basic cycle for major evolution is the typical period between DNA-rewriting events in a company’s history in a given industry. For most industries, this will map to changes in supply chain relationships (since M&A and mutations are much rarer). Product and platform lifecycles in particular, do not qualify as cycles to be analyzed as “DNA changes” unless you mean the corresponding change in the codebase as new capabilities are learned by the supply web and old ones are forgotten.

So, to cut a (very) long story short, for most industries, the clockspeed is the median time between major changes in the supply chain. Not product life cycle, not even platform lifecycle, and not CEO tenure. These might well be correlated, but conceptually, do not map as well. If a CEO’s tenure is marked by major process-shifts (like trying to instill DfLSS company-wide), then there is a genetic aspect.

Where the Metaphor Breaks

The reality of conceptual metaphors, as Lakoff demonstrated decades ago, is that they are incomplete. They highlight certain sets of similarities among domains, but fall apart in the non-highlighted parts.

For the genetics metaphor, “fall-apart” happens in that there is no direct equivalent of DNA, and a missing “every cell has a copy of the DNA” feature. There is also no clear role for important elements if we were to go the other way. Money is the lifeblood of industry, but doesn’t map clearly to anything in the biological world (I thought it might map to nerve impulses, but it doesn’t. It also doesn’t map to blood, oxygen or any major circulating entity in the organism). Customers also do not map (“employees” are a question mark for me at the moment). Until the economy evolves a LOT more, there will be very little by way of “food chains” and hierarchical predation and sharks. Right now, the only sorts of predation we see are Facebook widgets living off the Facebook organism.

Much of this confusion is due to the fact that the economy-as-ecosystem metaphor competes with the “all technology is a single organism” metaphor. This metaphor, which I call the “Electric Leviathan” metaphor (a Gaia for technology), actually works better overall, but is harder to use. I’ll explore it in a future piece.

Services

If you try to reconstruct service industries along these lines, you’ll be forced to begin with the individual service engagement as the cell. So a McKinsey team working over a time-bound period is a cell. Unfortunately, service industry structures don’t map as nicely. The only one I can think of that might work is the restaurant business (and in particular, the Seinfeld Soup Nazi). Franchises might also work, I’ll think through that another time.

I’ll work this theme more in future pieces. In the meantime, comments appreciated.

Garrett Lisi, a freelance physicist who apparently divides his time between surfing in Hawaii and snowboarding in Lake Tahoe, has come up with a new (and apparently falsifiable) approach to unifying quantum mechanics and gravity without using superstring theory, and is being taken seriously. I’ve blogged about the problems in physics before, and in the context of that unfolding drama, this appears to be viewed as a win for the heterodox camp (which includes digital physics). Even if it is ultimately proved wrong, there seems to have been some movement, and what is interesting to us non-physicists is that it seems to require a complete break with the establishment to make progress.

I won’t pretend to understand what the guy actually achieved, since I understood about 1% of the paper, but the best us lay people can do is think of it as follows: Lisi somehow used the E8 exceptional Lie group to write down a unified description of both gravity and quantum mechanics. Some details:

• The Telegraph covers the development
• The Paper
• The review (with pointers to other expert reviews) over at Not Even Wrong (Peter Woit’s Blog)
  

I am a sucker for a romantic story, and I guess that’s why this story resonates with me, no matter how it ends. That somebody could basically choose to be an impoverished odd-jobber in pursuit of something like this makes me hopeful that we aren’t dead as a species yet.

The one critical point I sort of understood was made by Woit in his blog: that Lisi’s approach just substitutes the question “How to unify” with the equally difficult question “what breaks the unified grand symmetry?”

If I ever get a chance to take a year-long sabbatical, studying all this physics is what I will do with my time. Unfortunately, I am not smart enough or brave enough to quit my job and deliver pizza to pursue things like this.

The idea of stability is a central organizing concept in mathematics and control theory. Lately I have been pondering a more basic idea: equilibrium, which economists prefer to work with. Looking at some fallen trees this weekend, a point I had appreciated in the abstract hit me in a very tangible form: both stability and equilibrium are intellectual fictions. Here is the sight which sparked this train of thought:

These are the fallen trees that line the southern shore of Lake Ontario, under the towering Chimney Bluffs –  cliffs left behind in disequilibrium by the last Ice Age. For about a half mile along the base of the cliffs, you see these trees. Here is the longest shot I could take with my camera.

No, evil human loggers did not do this. The cliffs are naturally unstable, and chunks fall off at regular intervals. Here are a couple of the more dramatic views you see as you walk along the trail at Chimney Bluffs State Park.

Signs line the trail, warning you to keep away from the edge.  Unlike the Grand Canyon or the Niagara Falls, whose state of disequilibrium requires an intellectual effort to appreciate, the Chimney Bluffs are in disequilibrium on a time scale humans can understand. A life form we can understand — trees — can actually bet on an apparently stable equilibrium and lose in this landscape, fall, and rot in the water, while we watch.

Even earthquakes, despite their power, don’t disturb our equilibrium-centric mental models of the universe. We view them as anomalous rather than characteristic phenomena. The fallen trees of the Chimney Bluffs  cannot be easily dismissed this way. They are signs of steady, creeping change in the world around us, going on all the time; creative-destruction in nature.

The glaciated landscape of Upstate New York, of which the Chimney Bluffs are part, is well known. The deep, long Finger Lakes, ringed by waterfalls, have anchored my romantic fascination with this region for several years now. The prototypical symbol of the region is probably Taughannock falls:

Unless you live in the region for a while, you won’t get around to visiting the Chimney Bluffs. But visit even for a weekend, and everybody will urge you to go visit the falls.

We create tourist spots around sights which at once combine the frozen drama of past violence in nature, and a picture of unchanging calm in the present. Every summer and fall, the falls pour into Lake Cayuga and tourists take pictures. Every winter, they slow to a trickle. Change is so slow that we even let lazy thinking overpower us and make preservation the central ethic of any concern for the environment. Even the entire ideology and movement is called conservation.

We forget cataclysmic extinction events that periodically wipe out much of life. We forget to sit back and visualize and absorb the implications of the dry quantitative evidence of ice ages. Moving to the astronomical realm, we rarely stop and ponder the thought that the earth is cooling down, that its magnetic poles seem to flip every tens of thousands of years, that its rotation has slowed from a once fast 22-hour day. We forget that our Sun will eventually blow up into a Red Giant that will be nearly as large as the orbit of Mars.

We forget that nature is the first and original system of evolving creative destruction. Schumpeter’s model of the economy came along later.

Towards Disequilibrium Environmentalism

This troubles me. On the one hand, environmental concerns are certainly very high on my list of ethical and practical concerns. Yet, when nature itself is chock full of extinctions, unsteady heatings and coolings and trembles and crumbles, why are we particularly morally concerned about global warming and other unsustainable patterns of human activity? A practical human concern is understandable (tough luck, Seychelles), but to listen to Al Gore, you would think that it is somehow immoral to not think entirely in terms of preservation, conservation, equilibrium and stability. So nature decides to slowly destroy the Chimney Bluffs. We decide to draw down oil reserves, slowly saturate the oceans with CO2 and melt the ice caps. Why is the first fine, but the others are somehow morally reprehensible? If you worry that we are destroying a planet that we share with other species, well, nature did those mass extinctions long before we came along.

In this respect, the political left is actually rather like the right — it is truly a conservative movement. Instead of insisting on the preservation of an unchanging set of cultural values and societal forms, it insists on an unchanging environment.

To be truly powerful, the environmentalist movement must be reframed in ways that accommodates the natural patterns of disequilibrium, change, and ultimate tragic, entropic death of the universe. I don’t know how to do this.

Why Disequilibrium instead of Instability?

Stability is a comforting idea. It is the idea that there is a subset of equilibria that, when disturbed slightly, return to their original conditions. But it is a false comfort. Every instance of stability lives within an artificial bubble of time, space and mass-energy isolation. Expand the boundaries enough, or let the external universe inject a sharp enough disturbance, and your stability will vanish. Unstable equilibria are even sillier, because even infinitesimal disturbances can knock them out.

Which means disequilibrium, not equilibrium, is the natural state. Using the word disequilibrium suggests a steady, sustained absence of stability — the universe is one, long transient signal where every illusion of stability will be destroyed, given enough time.

The assertion that the universe is a computer (or rather, a computation) might seem like an egregious category error — computers after all are things made from the ‘stuff’ of the universe. To take digital philosophy seriously we need to get past this non-trivial barrier to comprehension. The idea is that computation is not a metaphor for the universe, nor is the physical evolution of the universe analogous to computation. The idea is that the universe can be said to be a gigantic ongoing computation just as it can be said to be a bunch of particles interacting energetically via some laws. In the first part of this series, we looked at the prerequisite idea that the continuum (or real line) might not be so real. In the next part, we’ll get to the latest ideas, from quantum computing scientists like Seth Lloyd. But before we get there, we need to talk about Von Neumann, Stephen Wolfram and Cellular Automata, an approach to digital philosophy (and physics) that I think is wrong, but nevertheless very illuminating.

Matter as Computation and (Self-?) Simulation

As we normally understand them, computers are things made out of physical ‘stuff’ and some of the behavior of the ‘stuff’ viewed through some subjective semantic filters (i.e. associations like “red LED physically lighting up is ‘YES’ to my question”) by you and me, is ‘computation.’

But the line really isn’t so clear. Is a swinging pendulum computing approximate solutions to the simple-harmonic-oscillator equation? What about a textbook RLC oscillator circuit? Do the two compute approximate solutions to each other’s behaviors? The early analog computation pioneers certainly thought so. Given the right constants, is the pendulum a ‘simulation’ of the oscillator circuit and vice-versa? So at the very least, any arbitrary physical system might be said to compute its own behavior and that of certain other ‘similar’ systems, in ways that make mathematical thought possible. Note though, that while the pendulum and the RLC circuit are ‘like’ each other according to our observations, neither is actually a simple-harmonic oscillator, either categorically or in terms of descriptive completeness.

Now add the observation that there are certain ways of putting finite amounts of matter together such that the resulting entities — approximate Universal Turing Machines (UTMs) — can make part of their behavior (approximately) mimic the behavior of anything else, and suddenly things get even more murky. We are dangerously close to glib, but potentially meaningless, observations like ‘everything is like everything else’ and ‘the universe is recursively self-descriptive.’

What is worse, these UTMs needn’t be particularly complicated. In fact, they can, in the form of cellular automata, be suspiciously simple. Which brings me to the cellular automata approach to digital philosophy. An approach that I think fits H. L. Mencken’s assertion that “For every complex problem there is an answer that is clear, simple, and wrong” and which I suspect Einstein would have labeled “too simple.” But it is wrong, I think, in very instructive ways.

I won’t say much about the history of the idea that the universe is a computer (this goes back to Konrad Zuse and Edward Fredkin) but jump straight to the specific approach, introduced by Von Neumann, using cellular automata.

The Cellular Automata Approach

If you’ve seen James Conway’s famous Game of Life, that little simulation of a 2-d grid where repeatedly applying simple rules leads to beguiling life-like patterns evolving, you know what cellular automata are. Von Neumann introduced them more than a half-century ago. The neat thing about CAs is that they don’t look like computers — there are no constructs like ‘program’ or ‘memory’ or ‘input.’ They look like discrete dynamical systems and have, instead, functionally similar (but semantically distinct) constructs like ‘evolution rules,’ ‘space’ and ‘initial conditions.’ The neatest result? Some CAs, including Life are provably equivalent to UTMs. The most interesting CA is something Von Neumann called the Universal Constructor. This is — wait for it — a cellular automaton that can reproduce itself! One realization looks like this (public domain image from Wikipedia):

Self-replication is an amazing type of computation — behavior reproducing structure and completing destroying any intuition you might have of the separation of form and content, or of structure, function and behavior. It suggests one way (though I think not the right way) of starting to understand phrases like ‘The Universe is computing itself’). Here are two points about UCs that you might want to remember. First, you need a minimum complexity to generate reproductive behavior, below which CAs produce degenerate, non-reproductive behavior. Second, UCs are also Turing machines (am not sure if they are known to be UTMs), and so cannot compute uncomputable quantities. But here is a crucial addition: a UC with noise injected is capable of open-ended evolution (rather like random mutation in Darwinian evolution). Stick behind your ear this question: if the whole universe is a UTM or a UC, and its evolution appears to be more open-ended than closed, where might the noise come from? We’ll get to that question in the next part. Self-replicating machines, incidentally, aren’t merely mathematical toys. The simplest 3-d self-replicating robot that I know of was recently constructed by Hod Lipson and his students at Cornell. The history of the technology is quite long.

The Wolfram Gorilla

Which brings me to Stephen Wolfram and that 600-lb gorilla in all such discussions, A New Kind of Science. I’ll admit my views of ANKoS are based purely on a rough skim, preconceived notions, and hearing Wolfram speak about the book a few years ago when it was first released. For those who haven’t kept up with this world, Wolfram, after a start as a physics prodigy and founding the company that brought us Mathematica, basically devoted his life to one-dimensional cellular automata and produced in 2002, after nearly 2 decades of labor, a 1192 page tome whose essential claim was that everything that needed explaining about the universe could be explained by one-dimensional CAs. When I heard Wolfram speak at the University of Michigan, there was a perceptible air of skeptical disbelief and ‘is this guy for real?’ in the audience, not least because of the completely blunt condescension in his tone towards any other ideas. A good summary of his talk would be “The whole intellectual history of the planet is a preface to ANKoS, which basically answers all meaningful and profound questions, but there are a few dregs remaining if you want to continue in science, math or philosophy, now that I’ve rendered inquiry irrelevant.”

Not exactly a message that would be well-received anywhere, but let’s judge the man on the merits of his ideas rather than what appears to be an obnoxious personality (many great thinkers — and Wolfram is definitely one, even if he is wrong, he is breathtakingly, brilliantly wrong — have been jerks). What do we have? Unfortunately, from what I can see, ANKoS really brings nothing new to the party that had not already been brought there by thinkers going back to Konrad Zuse, Edward Fredkin, von Neumann, Gregory Chaitin and most recently, Seth Lloyd and other quantum computation. Some harsh critics have gone so far as to say that the 1192 page tome is all fluff, except for one important result (that a 1-d automaton called Rule 110 is Turing-equivalent) which was discovered by Wolfram’s student Matthew Cook rather than Wolfram himself.

I’ve said about all I can without actually reading ANKoS (which I won’t do unless someone convinces me it is worthwhile in 1129 words or less), so let’s conclude with some observations about the CA approach to life, the universe and everything (wouldn’t it have been breathtaking if Rule 42 had turned out to be the interesting one instead of Rules 30 and 110?)

Assessing the Cellular Automaton Approach

As I said, I think this approach is misguided at very fundamental levels, but is still very illuminating. Obviously, I can’t make a watertight argument that this is the case since I am not an active researcher in the field and haven’t read ANKoS, but I’ll outline the reasons I am both skeptical and intrigued:

Reasons why the CA approach is illuminating

• It has every nearly element you need in the soup: self-replication, universal computation, a role for randomness (to create open-ended/uncomputable evolution).
• It is conceptually very simple and comprehensible compared to other approaches, including other approaches to digital physics as well as traditional physics (which is not to say that there aren’t hairy technical problems inside), but still produces behavior that mimics many features of the universe and biological life within it.
• It serves to fill the conceptual gap between traditional computation and computation as dynamic evolution.

Reasons why it might be misguided

• The approach is simply too arbitrary and it would be suspiciously good luck if it turned out to explain everything
• Wolfram made some throwaway remarks about how one-dimensional automata somehow explain fundamental constructs like ‘consciousness’ and even ‘time.’ To me, even without probing much deeper, this doesn’t seem possible given what 1-d automata are — they could possibly depend on certain assumptions about the nature of time and consciousness, but they probably cannot ‘explain’ these concepts in any meaningful way.
• The whole approach is an extended metaphor of sorts, with startling commonalities between the output of automata and (for instance) the patterns on snail shells. But you can make similar observations about many mathematical constructs, including prime numbers, the Golden Ratio and so forth (they show up in obscure places in nature).
• Unlike some of the superstring theories causing trouble in the world of physics right now, ANKoS style approaches aren’t even the types of mental models about which you can even make falsifiability judgments. That there is no meaningful map between the constructs of the approach and observed reality is just the beginning of the problems.
• There are obscure but alluring ideas in the approach like “The Principle of Computational Equivalence” and “Computational Irreducibility” that, when you poke a little, get lost in a semantic soup, which suggests theology rather than metaphysics.
• Finally, in everything I’ve read or heard about this approach, somehow, I did not gain a single ‘Aha!’ insight that made me see anything differently. Shouldn’t a theory of everything shake your very being?

With that interesting detour out of the way, let’s get to a more complex, harder to understand, but more plausible approach to digital physics next time, which will include such interesting questions as “what is the computational power of a black hole?” Not quite sure how long it will take me to frame my thoughts there (probably a few weeks — this series is exhausting to write), but if you are in a hurry, you may want to grab copies of Seth Lloyd’s Programming the Universe and Charles Seife’s Decoding the Universe.

In a previous article, I reviewed some of the troubles ailing superstring theory, as chronicled by two prominent and articulate discontents. Among the more radical suggestions for fixing physics is to get away from continuous models altogether and ask if the universe is fundamentally a discrete entity in some way. Proponents of this view — called digital physics or nearly-equivalently, digital philosophy– take on not one but two terrifying tasks. Not only must they reconstruct centuries of physics built on top of calculus (a fundamentally continuous sort of math) but to finish the job at a satisfying level, take on continuum mathematics itself and reconstruct it in discrete terms. The debate has relevance even further afield, to questions about the nature of consciousness. I’ll talk about three books that develop this approach in an accessible manner, and about one formidable one that I think confuses the issues in pointless distracting ways.

In this first part, for the mathematical foundations, I’ll talk about Gregory Chaitin’s Meta Math!: The Quest for Omega. In part III, for the physics/computation/information theory aspects, we’ll turn to Seth Lloyd’s Programming the Universe and Charles Seife’s Decoding the Universe. I am not competent to review (let alone summarize) these books, but I’ll lay out what I understand, why I think this direction of development is important, and what the gaps in my own understanding are, that perhaps you can help fill. You cannot write about this stuff without saying something about the 600 lb gorilla in the room, namely Stephen Wolfram’s A New Kind of Science and I will make a brief detour through that and the whole cellular automaton apporach in Part II.

Meta-Math: Is the Real Line Real?

Chaitin, a rather famous member of the rather famous IBM T. J. Watson center, is best known for co-inventing, along with Kolmogorov, a measure of computational complexity called Kolmogorov-Chaitin complexity. You could say the entire book is about using this measure to probe whether the real line is actually real. Here is the gist of the idea.

Kolmogorov-Chaitin complexity measures the complexity of a bit string in terms of the length of the theoretical shortest program that could have produced it (actually determining whether a program is the shortest one is undecidable). This leads to an approach to complexity in computer science that is very different from the P/NP variety, called Algorithmic Information Theory (AIT). Like Shannon’s older information theory, AIT studies the information content of strings of bits, but comes up with different answers for reasons that I’ll explain in a minute. But the reason AIT ends up shedding light on foundational issues is that it can be used to frame a very interesting question about the halting problem version of Godel incompleteness. Instead of asking if a Universal Turing Machine (UTM) will halt for a given program/data input, you ask, given a random program of length p bits, what is the probability that the UTM will halt?

This probability is called Omega and is among the most mysterious numbers around, and the key to why you might believe that the real line is unreal. For fun, here is the formula for this (gratuitous use of my inline LaTeX WordPress plugin):

[tex]\Omega_F= \sum_{P\in P_F}2^{-|P|}[/tex]

But first some simple examples to provide background.

Shannon vs. Kolmogorov-Chaitin

Consider a string of a 1000 bits like 1010101010101010101…10. This sort of obvious repetitive pattern poses no problems. Shannon information theory (the stuff that runs under the hood of your file compression programs) would tell you there is very little content, and so would K-C theory (a 500 iteration loop that prints out ’10′ would do the trick). But now consider two other numbers; the first a true (not pseudo) random number produced by flipping a coin (heads=1, tails=0). It might look like:

10101001010101010100010101011111010110…

And consider this other number:

11.0010010000111111011010101000100010…

Shannon’s approach would, in general, conclude that there is 1 bit per bit of information in both strings (recall that Shannon information entropy is essentially a statistical measure of randomness that says nothing about hidden structure). But K-C would conclude that the latter has very low information content, because it happens to be the binary version of pi, a number which, despite having no discernible pattern, can nevertheless be produced by very short (if slow) programs. But for the former, K-C theory would do no better than a Shannon-driven compression engine, since the best program you could write for an arbitrary coin-toss sequence number would be something like

print “101010010101010101000101…”

which would be a constant longer than the output itself.

Getting to the Real Line

So what does all this have to do with the reality of the real line? The argument is tricky, but here is an outline, and begins with Kronecker’s famous assertion that “God created the integers, all else is the work of man.” Start with the assumption that the positive integers are meaningful constructs (corresponding to things like the number of cats in the room and so forth). Negative numbers find some physical manifestation in, say, electric charge (let’s not use the typical abstract example of financial debt). You get to rational fractions when you slice up pizza. Rational complex numbers, it turns out, are also quite meaningful for talking about things like quantum states. You can go quite a bit further into the irrationals. Though the whole set of irrationals is uncountable, interesting subsets are countable. For instance, the algebraic reals (roots of polynomials) like the square root of 2, are countable. Transcendentals though (which are provably not the roots of any algebraic equations with rational coefficients) are not.

But somehow, you’ll never actually get to something in the universe (sneak peek: except possibly consciousness) that corresponds clearly to the continuum. If you haven’t studied some point-set topology, this can be hard to appreciate, but here’s a quick peek at why this is so. Take the well-defined set of rationals, which constitute a countable set. Unlike the integers (or odds or evens), the rationals are somewhat strange. Though you can count them through clever tricks, in their natural state they behave strangely. Since, between any two rationals p and q, you can always find another rational (p+q)/2, no matter how much you “zoom in” on the real line, you won’t see the rationals embedded like pinpricks against a “continuous line.” The set of rationals will appear just as ‘continuous’ to you as the whole continuum. Unfortunately, you can use the exact same sort of analysis to conclude that it is plurality and discreteness that are illusory, which is what Zeno famously did, so it is to some extent a matter of making a leap of faith in one direction or the other.

The abstraction of the continuum that we think make sense actually falls apart upon scrutiny. Our instincts lead to absurdities. Perhaps you can find ways to make our intuitions of integers fall apart in similar ways, but I haven’t found any that are this compelling.

So if the intuition behind your sense of “continuous” falls apart with even such a simple probe, we have to doubt the foundations of the real line, and therefore our natural notions of continuous space and time. This is where Omega comes in.

It turns out that a lot of very crucial sets are countable, just the way the rationals are. This includes all computer programs written in a finite symbol alphabet, all programs shorter than a given length, and so forth. The implication: if the real line is real, it contains more numbers than all possible outputs (in binary form) of all computer programs. In other words, the set of all computable numbers (which is the same as all computable outputs, since any output can be converted into a bitstream) is countable. Some very curious results also follow. For instance the set of all nameable numbers (one, two, googol,…, pi, e,…) is also countable, and therefore almost all numbers are unnameable and uncomputable. Almost all here is a technical term from something called measure theory, which is easiest to understand as follows: if you pick a real number at random (say by tossing a coin at random forever), then with probability 1 it will be uncomputable and unnameable (I am oversimplifying a bit here, but I can’t reasonably summarize a highly incompressible and random book in one post!).

Omega is interesting because it appears to fit in that subtle zone of nameable but not computable. You can clearly define it as a property (stateable in some finite symbol system) but you can’t compute it. It is, in a sense, the first sign of the last frontier of knowable numbers. Beyond lies a ridiculously larger ocean of entities so elusive, you have to wonder, why keep them around in your conceptual framework at all?

Where to Next?

I’ll stop there for today, but here are some directions you can go from here (some of which I’ll go). You can take these ideas into physics and ask if the universe is random or pseudorandom, and whether space and time could be fundamentally discrete, which I’ll get to in Part II. You can continue probing within mathematics at these ideas, by looking at, for instance, the earlier ideas of L. E. J. Brouwer which I’ll get to if I ever understand them.

And finally, if you make certain precise analogies between these ways of parsing mathematics and phenomenal reality to constructs of subjective consciousness, you can create surprisingly (and suspiciously) close analogies to different sorts of metaphysics. A sneak preview: Buddhist sunya metaphysics appears to map to digital physics, while Vedantic Advaita appears to map to continuous-background theories. Plato’s Allegory of the Cave, which I am just starting to ponder, also seems to map, but I am not sure how (some help here?). A suitably abstracted metaphysics from existentialism seems to map to some sort of true-random physics. These are, of course, analogies among three sets of mysteries (mathematics, physics and consciousness) that may or may not amount to anything. But then, if I wanted to stay away from wild speculation, I’d have stuck to academic writing.

If you find someone who tries pull a sleight-of-hand “one mystery cancels an analogous mystery” stunt on you, kick him/her in the pants for me. I’ll cover the physics stuff next. It is late and I need to sleep.

I have heard cited several times the so-called fifty foot law of sociology, which says that most collaborations happen among people who work less than fifty feet apart (the idea is generally credited to Tom Allen of MIT; the primary reference seems to be his monograph, Managing the Flow of Technology, MIT Press, 1977, which I admit I haven’t actually read). Let’s generalize and assert that most relationship interactions happen among people who live/work 50 feet apart, plus or minus an order of magnitude, say 5-500 feet. This being a probabilistic, phenomenological law, it should be interesting to mull how it is changing in Tom Friedman’s flattening world, and to what extent lives are getting transformed in terms of changes to this law.

(Note: be sure to check out the follow-up article, “The Twitter Zone and Virtual Geography”)

For a good part of 2004, as a postdoc at Cornell, I was among the most egregious violators of this law. My postdoctoral adviser was off on a sabbatical in Europe, and we interacted over the phone and email. My PhD adviser, with whom I was still collaborating, was in Michigan, as was my girlfriend (now wife). My parents were on the other side of the planet in India, and most of my friends were scattered around the globe. I had no social energy and made no friends in Ithaca for months. My phone calls from that period had practically no local calls, and the average distance of my calls was probably thousands of miles.

It didn’t last too long — my girlfriend joined me in Ithaca, I began working with a few students who I met with face to face, and my supervisor returned to Cornell. Despite my utter social laziness and deliberate self-isolation in a remote apartment complex outside Ithaca (one that Thoreau would have liked), I managed (for the first and probably last time in Apartment Land, America) to befriend not one, but two of my neighbors.

But no, that wasn’t to last either — neither a traditional localized pattern of relationships, nor a comically distended one was to be stable. My supervisor went off again, students left for the summer or graduated, and my girlfriend went off to Nepal.

Then she returned, we got married, lived together for a year and now we are back at being 500 miles apart, with jobs in Washington, DC and Rochester, NY, respectively, piling on the frequent flier miles. At my job, in Xerox research, I started out working with a team whose other members were down the hall, but with a couple more projects underway now, I am back to messy global collaborations, one of them involving team members in Europe, California and Canada. 11 AM EST is now the only possible time I can schedule full team teleconferences. Now that blogging is a major part of my life, and since I am reconnecting with old friends (at the expense of potential new ones I should be out drinking with), my active social network is again all over the place. Again, I find myself being lazy about developing new local friendships, outside of lunches with co-workers (“what’s the point” is a pretty valid complaint, since my Ithaca friends too have joined my Jamshedpur, Bombay, Austin and Ann Arbor friends in being all over the damn place).

So a time graph of my key relationship distances, both personal and professional (say the top 20 people I interacted with), over the last few years, looks something like this:

Will it ever settle? I am not sure. The mean may drift down, but the variance is likely to remain high. Now keep in mind that I am a pretty ordinary guy (among educated information workers that is) when it comes to such things. Though I experimented with pen friendships, ham radio and backpacking when I was younger, none of those things ever captured my fancy for long. I realized about 8 years ago that I am a homebody, and so I am by no means an eager adopter of the globe-trotting, international-numbers-on-speed-dial lifestyle. In a previous era, I would not have sought out a career in foreign affairs or sailing. I’d have settled somewhere boring and lived out my life within a 100 mile radius. I don’t have much wanderlust or global-connection-itis left in me. All this happened naturally and largely against my will. I am sure the more enthusiastic globalists among you have much wilder graphs than mine.

Another way to slice such an ordinary-for-2007 lifestyle is to break down every relationship into its interaction units, conversations (including, say, face to face, phone, email and IM), and look at the frequency distribution of the distances of interactions. Here is a sketch of what I think might be happening:

I suspect that while industrial age technologies (postal mail, telegraph, telephone, fax, email) progressively moved the mean of the interaction-distance distribution rightwards, the globalized era is actually fattening the long tail. The bulk of the distribution, for most of us, may remain under a hundred miles, but the tail will start to increasingly thicken. More so for work than for community, but increasingly for community life as well. My top 20 friends are not within 20 miles of me, and will likely never be, no matter where I move, even if it is to the most packed metropolitan or super-suburban sprawl.

The Implications

While a perennial idealist-optimist like Friedman (is he on drugs?) might see nothing to mourn here, I have enough of a tragic streak in me that I see a definite and irreversible loss of quality of life and community here, most famously noted by Robert Putnam in his classic, Bowling Alone. No, I am not pining for some romanticized small town Norman Rockwell painting life (or Naya Daur village life, to pick an Indian cultural reference). I am talking about the harsh reality of forms and structures of traditional social capital slowly disintegrating, without a clear replacement emerging from Google Labs or Linden Labs. Much as I enjoy spending my evenings in Starbucks reading, arguing with my wife on the phone and keeping in touch with good friends over email, often I wish they were around me rather than, on average, 500 miles away. My local cafe baristas change so often, and the clientèle is so variable, that they are hardly hubs for a renewal of community life.

You might believe that virtualization will soon break this stalemate, that we’ll be able to live where we like and keep at least our families in one place. You might imagine that communities of like-minded teleworkers will form. But that is delusional. Work, as much as family and community life, is a source of enduring and rewarding relationships, and while we’ve gone overboard in sacrificing the latter for the former in recent decades, technology seems set to simply swing the pendulum the other way now. It is still an either-or between two major categories of relationships. The fattened tail will never slim again, it seems to me. We’re never going to have an unfragmented personal and communitarian life again, because the globe remains 24,000 miles in diameter, and now that it is connected, it is highly unlikely that any web of high value creation will remain localized enough to feel comfortable to our ape brains. We’ll all continue to drift, or put down roots somewhere out of necessity, accepting a far-from-optimal median radius of community life.

Not that this is a consequence of new things. It is just the culmination of a dynamic that started when we first created the distinction between ‘work’ and ‘life’ and created the problem of ‘work life balance’ back around the invention of agriculture.

Read the follow-up article, “The Twitter Zone and Virtual Geography”

We commonly use a set of dynamic spatio-temporal orientation and observation conceptual metaphors to talk about knowledge, its communal organization, and individual styles of knowing. We use depth-versus-breadth to talk about track records and abilities, “long-term” versus “short-term” (and “upstream/downstream”) to talk about intentions and decision-making, and “big-picture” versus “details” to talk about the scopes of discourses. All these will come up for critique and more analysis as I continue developing the themes of this blog. But I want to start off this fresh new week with a question for you to ponder: how do you organize your view of knowledge, and how much faith do you have in your organization?

I’ll do a tutorial on conceptual metaphor and George Lakoff in a bit, but for now, all you need to do is understand a conceptual metaphor to be an organizing frame of reference rather than a figure of speech. We tend to use conceptual metaphors for talking about at least four (per my count) distinct aspects of the phenomenology of knowledge. These are, the organization of knowledge itself, the organization of communities of knowers (including notions of in-scholars and out-amateurs), perspectives and action-orientations of individual knowers, and the evolution over time of these three types of elements (knowledge, communities, perspectives). That’s a lot to talk about, yet we do it comfortably (though, as I will argue, not very soundly).

Organizing Knowledge

Consider just the first one: ways of organizing knowledge itself; the stuff in books, papers and people’s heads. Here is a sampling of metaphors: excavation, set theoretic, architectural and Newton’s beach (“I do not know what I may appear to the world, but to myself I seem to have been only a boy playing on the sea-shore, and diverting myself in now and then finding a smoother pebble or a prettier shell than ordinary, whilst the great ocean of truth lay all undiscovered before me.”)

There are several related metaphors here, all spatial. All the compliments and criticism I’ve received have been framed in terms of the breadth-depth metaphor for instance (which is shown here in its excavation form, but is equally used in its watery form: “deep dive” versus “drill down”). Newton liked the less structured land/ocean view of the known and unknown. Engineers and mathematicians, particularly those engaged in synthesis (be it of software or complex structures of theorems), seem to like architectural metaphors like the one shown on the bottom left.

When it comes time to talk about “interdisciplinary” and “multidisciplinary” matters, seem to have in mind some sort of Venn diagram (or possibly, archipelagos of known Newtonian islands in an ocean of Newtonian unknowns) in mind. We speak of bridges between fields, intersections among them, or gaps between them. The idea of intersection of paths of discovery seems much rarer (I’ve seen it once — the person involved was a research manager type who seemed to view discovery as regimented marching along various paths). I take it some people have metaphoric political or physical maps in mind as well, when they talk about turf boundaries, silos, frontiers and trails blazed (a peculiarly American idea) and unexplored versus explored regions. All these metaphors are organized around a metaphoric spatial orientation.

More recently, the explicit manifestation of associative webs in our technology (the Internet that is), suggests an organization that has no natural up/down/sideways orientation. We are left only with “near” and “far” measured via conceptual degrees of separation, as our spatial commitments within the metaphor.

I’ll talk about the other three gradually, but here is a sneak preview. How you view this spatially organized knowledge (big picture/details, up in the clouds/down in the weeds) brings up the next class of metaphors. Newton came up with a very human one for this category as well, the “shoulders of giants” idea. Outside of the humanities and social sciences, the geography of knowledge induces the geography of communities, but within, the reverse is often assumed, which gives us the third class, communitarian metaphors with sociological boundaries and notions such as center and periphery (though, you could argue, center-periphery organizations apply to more objective-seeming organization metaphors as well). Finally, the fourth class, involving time, is possibly the most interesting, involving historicist “accumulation”, order-chaos and creative-destruction models, upstream-downstream models, and the underlying metaphors of such things as Kuhn’s idea of paradigm shifts (“earthquakes”) and Feyerabend’s idea of methodological anarchy.

The Problem with Epistemic Metaphors

We need these metaphors. Without them, discovery and inquiry would be solitary enterprises and conferences would be dull. Yet, we need to probe, critique and evolve the metaphors themselves, because they can lead to many pathologies. In the mildest case, they can lead to pointless battles between self-styled (or other-labeled) breadth and depth people. In the worst case, they can steer inquiry away from rich discoveries. A more obvious problem is that a good proportion of people are extraordinarily tasteless and clumsy when they try to apply these metaphors. Few things are as frustrating as an idiot who has decided to adopt “Let’s look at the big picture” as a pet phrase.

It doesn’t take much work to break these metaphors. The breadth-depth one, for instance, can easily be broken by appealing to ideas of different perspectives: quarks are smaller than humans, so quantum physics is narrower/deeper than sociology by one account. But then, quarks make up much more of the universe than humans, making quantum physics broader by another account. Further problems can be created by turning to notions of recursive complexity (of the ‘universe in a grain of sand’ variety) and path dependence (it may take you a sequence of 5 courses according to the prerequisite structure of Podunk University to learn concept/technique X, while some prodigy finds a way to get there from high school algebra).

The Question

Which brings me back to the question behind this post: how do you organize your view of knowledge, and how much faith do you have in your organization?

Do these issues matter at all? Would shifting to a better set of metaphors for working with knowledge help us discover things we wouldn’t otherwise have discovered? What might replace this language?

My own answer is a tentative yes. These issues matter in profound ways. For instance, the depth-breadth distinction I believe, is a pernicious and damaging one. I have an alternative, but haven’t worked it out completely yet. But the key point is this: our ways of talking about knowledge have to evolve alongside knowledge itself. We have been stuck with one set of epistemic metaphors for too long.

Space and time are favorite subjects of mine, since they are the root concepts for two of the most fundamental types of questions we can ask, where and when questions. I discussed three dimensions in detail in a previous post, so I am going to dive into the subject of cartograms and show why you should be careful about your two-dimensional thinking as well. I’ll give you a question to stick behind your ear before I begin: how do tiny island nations like Britain and Japan manage to dramatically influence the world, while huge continents like Africa and South America often don’t even register on the radar? Let me warn you right now, that’s a trick question.

We live on a sphere. Since we don’t really think effectively in 3d, we invented maps. You probably know that the common Mercator projection distorts the areas of countries, while roughly preserving their shapes. You might also know that there exists a projection called the Peter projection, or “equal area” projection, that corrects the area distortion at the expense of introducing a shape distortion. The latter is a favorite of super-liberals, since a side-by-side comparison of the two projections will usually show startling West-East and North-South biases, lending credence to claims that the Mercator projection is racist (simple example, Mexico is vastly bigger than Alaska, but looks the same size on the Mercator projection, India is almost the size of all Europe, but looks no bigger than Scandinavia). Personally, I found the Peter projection an intriguing curiosity when I first encountered it on the wall of the bathroom of a vegan co-op I was living in at the time, in Ann Arbor (yes, the place was a hippie cliche of sorts), but I somehow couldn’t see how the Mercator distortions really mattered that much.

I understood why I was under-whelmed by the Peter map when I encountered cartograms. The inevitable distortions introduced by flattening a sphere pale in comparison to the distortions introduced by the ways in which we use maps to represent other geospatial information. It turns out that when you are talking about things like population with the aid of maps, it is important that your maps be wildly inaccurate. Cartograms were invented by Michael Gastner and Mark Newman of the University of Michigan as a way to visualize the results of the 2004 US Presidential elections. In particular, they wanted to show why the race was so close even though maps of red vs. blue states were overwhelmingly red. To do this, they invented a clever diffusion algorithm that distorted geography in proportion to population density, so that the voting pattern in the election looked like this, by state (these graphics below are from Mark’s election 2004 page, used per his permissions on the page).

And if you apply the diffusion algorithm at the county level,  with some color scaling, you get this much richer, fractal-like look at the political leanings of America.

These maps became so hugely popular that a nonprofit organization, Worldmapper (where you can find dozens of provocative maps and posters to annoy friends with), was born, and went slightly nuts with the method. Since the diffusion algorithm can be applied to any dataset that has geographic coordinates, it turns out to be a pretty powerful tool. Here are two maps that should help you answer the question you have stuck behind your ear. These are from Mark’s page of global cartograms constructed with various important geospatial datasets (again used per permissions as described on his page). Here is a cartogram of GDP by country:

And here is one by population

Check out the rest of the thought provoking maps, including the ones on AIDS and childhood mortality. Then ask yourself again, is the UK/Japan question possibly a trivial one, once you have the right visualization in your head? Like sphere squashing, cartogram generation is a mathematically under-constrained problem, so it probably needs taste and skill to use for honest visualizations, but done right, cartograms are quite eye-opening.

Challenge: Visualize Connections

I first learned about Mark’s work when I attended a talk by him about an earlier body of work for which he is famous, on complex network visualization (all that good stuff about scale-free and small world networks, Erdos numbers and the like), and started exchanging occasional emails with him on the subject. One of his contributions was coming up with really good ways to visualize abstract graphs, like social networks, which have no natural spatial coordinates associated with them. I am sort of surprised his two contributions to visualization haven’t yet come together (unless I missed some recent developments), so here is a challenge for you.

Putting abstract network visualization together with geographic distortion brings up an interesting question: how do you visualize geospatial graphs?

For example, consider the volume of phone, Internet and air traffic among all medium or larger sized cities in the world (appropriately defined). Assume that’s a good metric of economic connectedness and globalization — Bangalore is actually closer to San Francisco by this metric than Mysore or Monterey are to the former and latter respectively. Can you visualize this?

Such a visualization would be really useful for understanding the economic structure of the world, and the rate at which ‘globalization’ is progressing (a much better visual metaphor than that awful flat world metaphor — I’ll do a  Friedman polemic some other time). For example, literal geographic proximity is important for some things — an earthquake in San Francisco would impact Monterey more than it would impact Bangalore. But a major financial crisis at, say, Google, might propagate faster to Bangalore than to Monterey. What is the right map for visualizing this sort of relational-geographic information? Is there a method analogous to the diffusion method that takes as input a connection-strength data set (2 locations per data point) rather than a point density data set (1 location per data point)?

Two recent popular science books provide a startling peek into the deep scientific and sociological troubles in the world of superstring theory. Not Even Wrong by Peter Woit and The Trouble with Physics by Lee Smolin together triangulate the core of the trouble. If you, like me, have been distracted from the foundational problems of physics by the ongoing two-decade fascination with chaos and complexity in the popular literature, now is the time to get back to observing the “deep” stuff. It is starting to get seriously interesting again.

I suspect if you’ve been following physics through casual browsing and media reports like I have, you have probably formed an opinion along these lines: superstring theory is the only game in town, and is basically in good shape except that it cannot be experimentally verified because the energies involved are “too high.” That is, you probably thought it is a sound scientific theory in principle, just not verifiable in practice. What I learned from these two books is that first, it is not sound even in principle, and second, it is not even a theory at all — just a collection of mathematical clues to the existence of a potential theory.

Taken together, the books make for perhaps 3-4 weekends worth of serious reading effort. Since the authors’ metaphoric/natural language explanations of various technicalities are actually more confusing than the math itself would be, you might want to read the books with wikipedia handy, to look up technical bits and pieces that you think you might understand better if you saw the actual equations. But the reading is absolutely worth it. I have rarely been this seriously challenged when reading popular science, and I came out of the experience feeling exhausted, but with a sense that for once, I’d earned that warm glow of wisdom, instead of having been deluded into a false sense of comprehension by a clever writer.

The Two Viewpoints

Peter Woit, now an adjunct at Columbia and author of the Not Even Wrong blog is a physicist by training who shifted to mathematics early in his career. This is the less polished of the two books, but paradoxically, more useful and interesting as a result. Had Woit carefully judged his potential audience and picked the appropriate mix of exposition, metaphor and mathematics, the book would have been easier to read, but not been as revealing as it is. Unlike most popular science authors, Woit doesn’t attempt to make the reader feel like a genius. If you are not a physicist, but have training in some other mathematically-oriented discipline, you’ll probably get about a third of the way through the book before you start to drown. But the overall sociological and philosophy-of-science argument is straightforward enough to follow. Technically, Woit focuses his critique primarily on the structure of the quantum physics aspect of the field.

Smolin’s book is different. First, it is better structured and you’ll get a lot deeper into it before floundering (if you are a non-physicist and you actually think you understand non-mathematical popular physics books, you are either deluded or a closet prodigy). Smolin himself is a respected physicist (now at Canada’s unique Perimeter institute) who has published extensively in both superstring theory and one of the alternatives you don’t hear about — general-relativity based quantum gravity. A deep love of physics permeates every page of the book, and Smolin tries his best to base his conclusions on a sophisticated (and classical) understanding of what legitimate science is about. By contrast, despite his best efforts, Woit’s writing reads like a thinly-veiled polemic in parts. Each book, by itself, provides a critically incomplete look at the situation, but taken together, they provide a reasonably complete assessment.

The Technical Critique

The technical critique hinges on two major elements and several minor ones. The first is the lack of experimental evidence. Superstring theory has dominated fundamental physics since around 1980. Despite more than 25 years of development, it still has no experimental support or any realistic hope of experimental support in the near future. This is apparently historically unprecedented. By itself, this wouldn’t be so worrisome if it weren’t for the fact that superstring theories don’t look verifiable or falsifiable even in principle. This forms the basis for much of the philosophical critique.

The second element has to do with general relativity, and is particularly emphasized by Smolin’s book. You have probably heard some claim to the effect that superstring theory naturally requires and predicts gravity. Not quite. Smolin argues carefully that the most mathematically and philosophically compelling way to think about gravity is through background independent theories. Theories, like Einstein’s general relativity, where the laws do not depend on a particular structure of space-time (like Euclidean or Riemannian), but which actually tell you how space-time behaves based on the distribution of mass and energy. Superstring theories, by contrast, are background-dependent. You have to assume the structure of space-time to get started. That gravitons (hypothetical particles that carry the gravity field) pop out of superstring theory doesn’t help. You want background independence.

Since superstring theory grew out of quantum field theories and the Standard Model rather than out of general relativity, there are a great many other specific technical critiques, most of which are inherited from the the concepts and techniques of the Standard Model (which itself is at least experimentally verified to unprecedented levels within its explanatory scope). Woit’s book gets into some of these at a fairly detailed level, that I am not competent to summarize. One common thread to all these critiques though, is that the successful conceptual advances and technical solutions that do exist should be interpreted as advances in mathematics, not physics. In fact the influence of superstring theory, particularly that of the field’s leading figure, Ed Witten, has led to a renaissance of sorts in mathematics, and earned Witten a Fields medal.

The Philosophical/Aesthetic Critique

The main philosophical critique of superstring/M theory is that it appears to be fundamentally unfalsifiable due to its intrinsic nature: it has enough mathematical flexibility that its parameters can be set to match any observed reality. In fact there are apparently trillions of possible “theories” that form an entire landscape of possible instance theories which could fit any observed reality. This suggests that what we are seeing is not an actual theory, but instances of the trajectories of a theory we don’t yet know (a rough, but limited analogy: if you observed various pendulums and documented each damped oscillation trajectory as a “theory” you would probably be looking for the familiar simple-harmonic-oscillator equation).

But what are we to make of a (hypothesized to exist) theory that can fit any reality? You could either toss out your methodological dogmas about falsifiability and appeal to ideas like the anthropic principle, or you could stick to your guns and decide that this is not real science. Both Smolin and Woit stick to their guns. Going in, I personally was fairly receptive to the postmodern philosophies of “ironic” science that are being used to bolster superstring theory, but I came away convinced that Smolin and Woit are probably right. It is not that postmodern philosophies, such as those of Paul Feyerabend, are bad. It is that they are being employed in bad faith in the service of flawed concepts.

The aesthetic critique is actually more interesting. Superstring/M theory proponents claim that the fundamental beauty of their mathematics is an argument in its favor. Smolin challenges this framing and cites a number of examples in history where apparent beauty turned out to be flawed and gave way to a more correct (and perhaps more beautiful) idea. Woit, on the other hand, calls the “beauty” bluff and pokes at it. He concludes that the theory is not very beautiful at all, but is extraordinarily full of technical ugliness as well as a big failure by that one metric of beauty that scientists tend to apply, Occam’s razor. Superstring theory has vastly more primitive parameters than the Standard Model and therefore, along the Occam-razor dimension, is much uglier.

The Sociological Critique

Both authors lay out the evidence to show that the entire field of physics has been taken over by a culture of intellectual bad faith that dominates discourse, prevents alternative viewpoints from getting traction (or funding), and somehow manages to perpetuate itself. Some calibration might help. Just how bad is this situation, viewed from the perspective of other fields? To a certain extent, these criticisms are usually being leveled all the time at the dominant paradigms in every field. I have heard versions of these critiques in the three academic fields I am personally most familiar with (my own home discipline of control theory, and the closely related disciplines of artificial intelligence and operations research). Calibrating against these examples and others, I have to conclude that the picture painted by Woit and Smolin does look pathologically toxic and unhealthy.

Among the things Smolin in particular points out, which I have never seen in such egregious form in my field, are the deification of a few priestly figures like Witten, the open suppression and exclusion of non-string approaches, the striking homogeneity of physics departments and the insularity of conferences in the field. Reading the anecdotes illustrating the pathologies at work, I felt for the first time that my own field is relatively healthy and open.

The Bogdanov Affair is a particularly interesting illustration of the sociological problems in superstring theory. The short version of the story is this: a prestigious journal apparently accepted clear mathematical nonsense for publication, despite peer review by competent reviewers. Woit examines this case in detail and concludes that the lack of sound foundational concepts in the field has created a world full of technicians who can’t even understand each other’s dense calculations as garbage.

So where might all this have come from. Both Woit and Smolin are careful not to blame individuals, and this seems like a sincere attempt to be fair. The one suggestion that comes through, regarding a possible cause, is the very structure of American science. After the great conceptual advances before World War II, American physics (which, by its dominance, has meant world physics until recently) somehow slid into an era where people asking foundational questions were marginalized, and a “Shut up and calculate!” technician ethic took hold, leading to a vast number of technically brilliant physicists taking over the field, leaving little room for philosophical introspection and alternate conceptual frameworks.