Mastering the Hype Cycle by Fenn and Raskino

by Venkat on October 16, 2008

You know you’ve got an interesting idea on your hands if it helps you build a fairly compelling case that the Amish are more sophisticated meta-innovators than Fortune 100 CEOs. Mastering the Hype Cycle by Jackie Fenn and Mark Raskino of Gartner, manages to do that about a third of the way through. Now, the Hype Cycle (the Wikipedia entry is pretty decent) is one of those intuitive and obvious-seeming ideas that makes you wonder (with 20-20 hindsight), did that actually have to be invented? (answer: yes it did). It’s been Powerpoint fodder for a few years now, and I’ve used the thing myself, to justify proposals. You can think of it as the older, bigger, badder brother of Godin’s Dip. If you aren’t familiar with it, it is a data-validated curve that looks like the picture below, and captures the typical pattern of hype associated with an innovation as it diffuses through the economy:

Hype Cycle (from Wikipedia, Creative Commons license)

Hype Cycle (from Wikipedia, Creative Commons license)

To be legitimate, the curve needs to be constructed through reasonable analysis of a good proxy measure of “hype,” such as media reports, modulated by human or automated sentiment analysis. I of course, as a trust-my-gut guy with far too much confidence in my own zeitgeist spotting ability, often just make up my own hype cycles when I am thinking through the implications of a trend. I hereby coin and claim credit for the term hype guesscycle: a hype cycle you just make up on a paper napkin.

As I started reading this book, I thought I’d check out Google Trends and try common-seeming negative/positive search term pairs (for example, “Web 2.0 sucks” and “Web 2.0 rocks”) to see if I could find quick-and-dirty evidence, but that turned out to be hard; major terms like “Web 2.0″ register on Google Trends, but qualified phrases don’t have the critical mass to be tracked.

Interestingly though, just plain search terms often showed curves roughly of the shape above (try “Second Life”). In retrospect this makes sense: if the trough of the curve represents negative (new and old) media attention, presumably the subjects would be coy about publicizing trough-worthy news, and even without subtracting negative from positive, you’ll still get the same rough shape (visibility unadjusted for sentiment is, I conjecture, a weighted root-mean-square type quantity, where the lower weight on negative hype will reflect its relatively lower frequency/visibility, since people hide failures).

I expect Google hotshots are busy building the next-generation Trends feature that will produce hype cycles on demand and put Gartner out of business.

The Book

The book is about the meta-innovation problem of if and when to adopt an innovation that is diffusing through the economy, with an emphasis on when. Hence the relevance of the Amish (whose communal answer tends to be “probably not” and “two centuries after others”).

Part I

Part I of the book starts with a fairly detailed exposition of every twist and turn of the curve, from technology trigger to the peak of inflated expectations, through the trough and up the plateau. As you would expect from a book based on a pretty novel perspective, there are a lot of fresh examples in the book, something I always look for. These include:

  1. The diffusion of grocery store smartcard loyalty programs in the UK
  2. GPS-based auto insurance policies
  3. The effect of Walmart demanding that its top 100 suppliers adopt RFID
  4. An odd mobile banking case study from Canada
  5. CRM systems

The freshness of the examples alone make the book worth a read. The next person who uses Southwest Airlines as a case study for anything ought to be shot. Interestingly, the book offers up some commentary on over-used case studies in terms of the Hype Cycle (basically, “lazy reporters recycling the same few early success stories”).

There is also a discussion of the rarely-seen extended Hype Cycle, which extends the plateau of productivity through a swamp of diminishing returns to a gentle-decline cliff of obsolescence, which the authors describe, in rather poetic terms, as “a crumbling escarpment where erstwhile innovations begin the often long and drawn-out, and almost irreversible, slide into oblivion.”

Besides the examples, the first part of the book offers up a quasi-mathematical explanation for the curve (which I didn’t quite buy, see my critique later), and a list of traps, opportunities and challenges involved in adopting innovations at various stages of the cycle. Among the less obvious observations is one about talent: that by adopting too early you might not find people with the skills to work with the innovation. I learned that personally, when I was won over by the (well-deserved) hype around the Ruby programming language and found myself faced with floods of Java and PHP resumes when I went looking to hire a Ruby-skilled team member.

The final element of Part I is a somewhat useless classification of companies as types A, B and C: risk-takers, moderates, conservatives. The application of that rather arbitrary and content-free classification, as you might expect, yields only rather obvious insights (such as the idea that watching true first-adopters make mistakes is a great tactic if you want to be a high-speed-overtaker type second adopter). But I guess some sort of classification was necessary, and I can’t think of a more substantive one myself, so A, B and C it is.

I hereby coin another term (I am on a roll today): OIOO, Obvious-In-Obvious-Out. The slightly smarter brother of GIGO. The Hype Cycle itself is not OIOO though.

Part II

Part II begins with the Amish (more on that in a minute), and goes to present (aaargh!!!) a systematic adoption process with (what else) a pronounceable-acronym workflow, STREET. You have:

  • S is for Scope: decide what’s valuable to you, and develop systems to scan for innovations in that space
  • T is for Track: develop a radar ]that helps you watch your targets evolve through the Hype Cycle
  • R is for Rank: given your risk profile and your current bandwidth for absorbing innovations, rank your targets, since you’ll have to choose the most worthy ones
  • E is for Evaluate: the Duh! step
  • E is for Evangelize: the other Duh! step
  • T is for Transfer: the step after you’ve driven through an adoption decision

One relatively nice thing the authors do, which other acronym-process coiners tend not to, is acknowledge and diagram the loopy, iterative nature of following such a process. So instead of a nice 6-step forward-flow process, you get some spaghetti, but at least they keep themselves honest. Game-changers, for instance, are described in terms of a T-to-S back loop.

I won’t say much about the rest of the book, since by the time you hit the acronym part of a management book, you are at the part where most people can just make up their own execution sequence and get it roughly right. But there are two non-trivial (and problematic) points that I’ll highlight:

  • The idea that before you even evaluate candidates as part of innovation adoption decisions, you need to define and clarify your own innovation needs. This is conveyed neatly by means of a travel metaphor. If you are new to San Francisco, obviously buying a travel guide is better than just wandering out of your hotel hoping to hit some good sights by accident. But even before you buy a travel guide, you probably ought to first decide whether you like nature stuff or museum stuff.
  • The idea that the all-crucial S and T parts of the process need to be systematically managed. Here the authors lay out an obvious spectrum: centralized, decentralized to the level of functional units, and decentralized to the level of individuals. They make the rather obvious recommendation that you need elements of all three. But this idea needs further probing. I’ll offer my build.

The Amish as Model Meta-Innovators

I always look out for the surprising and evocative high-concept of any book I tackle, and this book had a whopper: the idea that the Amish actually are great models of technology meta-innovators. The analysis is as follows.

The Amish live according to their reading of the Bible, which suggests a way of life that emphasizes pacific separatism and guarding against changes that encourage individuals to stand apart from the community. Unlike the other common example of anti-technologists, the Luddites, the Amish aren’t anti-technology per se; they just have a very strong filter based on their values, and very little makes it through. I was surprised to learn how sophisticated their overall consensual adoption model is. Apparently, they do not operate via blanket bans on technology. They’ve got their own enthusiastic early adopter types who try stuff out, before their elders step in to legislate (and generally, limit), use.

So, for instance, some sects of the Amish view the telephone as a useful innovation, but one that disrupts family bonds by making what I’ve called virtual geography stronger than physical geography. So they don’t just reject the phone; apparently several families share a phone, which is sequestered in a sort of out-house, for emergency use.

Now that’s way smarter than most of us are about technology. Certainly not an “Ooh, now they have fuschia-colored iPod nanos!” approach to adoption.

The Analysis: The Top Four Unanswered Questions

Mastering the Hype Cycle is definitely worth reading, absorbing and pondering. It is among the more novel perspectives on the subject, and it definitely deserves its book-length treatment. That said, like Hercule Poirot used to say, the book gave me “furiously to think.” I didn’t come up with any major critiques per se; the book doesn’t have any of the more obvious flaws and failings of business books. It is well-written and cogently argued (if in a rather empiricist mode).

So what I have I have instead, is a bunch of follow-up thoughts. Not many books spark such a cascade of thoughts for me, so that suggests this is fertile territory. I’ll formulate my analysis in the form of a list of unanswered questions:

  1. What is the real model? The authors present a suspiciously quick and dirty sketch of the possible mathematics underlying the curve: the sum of a normally-distributed curve, and a delayed technology-adoption S-curve (which, I vaguely recall from somewhere, is an integrated normal distribution: a cumulative distribution function). This has two problems. First, even for a phenomenological model, the causal reasoning is too sketchy. I was left wondering why the authors didn’t dig deeper. Second, it bothers me that the authors seem to have jumped so quickly to a non-explanatory sum-of-distributions type model. There’s a whole bunch of first-principles models that might plausibly apply: oscillators, predator-prey models like the logistic or Lotka-Volterra, diffusion models (both classical and social-network), system dynamics models and so forth. I was sort of surprised that the last one didn’t hit the radar for the authors. Then I figured, it being the calling card of rival consulting shop Forrester might explain things. Anyway, there’s enough to keep a graduate student happily occupied for a few years. Let me hasten to add, these are top-of-my-mind thoughts. Some might be lousy attacks. I don’t know much about what seems to be a significant literature on the hype cycle, so some of these might have already been tried. But the curve is characteristic enough that it deserves some first-principles investigation.
  2. Is the Amish Model sound? A major premise of the book is that you can and should choose to not adopt some innovations. I wonder how true that is. You may have some control over timing, but successful technologies tend to have a sort of inevitability to them, and roll on everywhere they apply with the grandeur of entropy. It takes seriously mulish resistance and high cost to resist for too long. In the case of the Amish, their resistance is bought at the cost of inbreeding and genetic diseases.
  3. Is value-filtered Scan/Track safe? This idea, that you need to scan and track with a motivated bias towards your own needs has one problem. Some of the things that are the most important to track are potentially disruptive developments (both threats and opportunities) well outside your front-and-center vision. Even peripheral vision will not do. You need a certain amount of 360 degree scanning and adoption. You might be blindsided by black swans too frequently otherwise.
  4. Is middle ground the best place to manage Scan/Track? The authors suggest that central+departmental+individual in a reasonable mix might be the way to go. This screams “lazy compromise” to me. There is a fairly compelling case to be made that the Scan/Track mechanism should be bottom heavy, using the standard arguments from crowdsourcing and wisdom-of-the-crowds (prediction markets, say) and open source (the many-eyeballs argument). You need smart and lightweight central coordination mechanisms, but if the technology is workable, I’d bet on crowd-heavy over central-heavy any day. Even the US intelligence agencies are betting on an Intellipedia for their scan/track needs. The biggest piece of evidence? Our entire global economy runs on a largely crowd-sourced Scan/Track collective radar, the international system of stock markets.

Surfing on the Hype Waves

I’ll conclude with one random thought I had. I could pretty much describe my own career in terms of hype cycles. I started as an undergraduate researcher in 1997, in robust control, which was just hitting its slope of enlightenment. Then I started graduate school working on problems in adaptive control, which (though I didn’t know it at the time) had peaked in 1983 and was hitting its cliff of obsolescence. I switched halfway through, and for my thesis, took on formation flight in multiagent systems, a topic that was just peaking. For my postdoc, I switched to semi-autonomous systems, which was just past the academic equivalent of a technology trigger (“major funding interest”).

Today, I rather consciously tend to try and find and work on stuff that’s just hitting a peak, or just coming off the peak. The valley of darkness presents happy hunting grounds for those who dislike crowds. Most of what I do today is 2.0-themed, stuff that (I would guess) is currently starting up the slope of enlightenment. What I absolutely cannot stand is the plateau of productivity, where money-making takes off. I’ll never be rich.

I call this behavior “surfing on the hype waves.”

Tom K October 16, 2008 at 6:39 pm

Your digest of this book reminds me a problem that has bothered me for some time that stems from the general shape of hype cycles or other such similar things. The thesis has always been that there are predictable patterns behind trends that proceed in a more or less standard way through the growing pains of adoption. But are these learnings not wrought with survivor bias? For any trend that survives many more crash and burn never to follow that path. Only in a retrospective way can you trace that path. Many trends in the end appear as short lived fads. Google trends analysis can bring that out, but we don’t yet have enough data to generate a statistical argument. Does this book shed some light on predicting which emerging trends will die outright and which will ride the hype roller coaster?

Tom K

Bobby October 16, 2008 at 8:11 pm

Really great, thanks.

I wonder about the curve applying generally to many web 2.0 services that evolve via a beta model… would you say that each release has it’s own curve? I mean, many of the 2.0 applications out there are launching, developing, and (hopefully) improving, at a rate which creates a volume of information that is fairly unmanageable for individuals… and so the channels of distribution tend to be more non-traditional. I wonder how the curve will hold up as the rate at which technologies develop becomes increasingly exponential and the noise that occurs when information starts to exceed capacity begins to make it even more difficult to keep up with new developments…

I think a good example would be flickr. I wouldn’t say that flickr is still considered the darling of early adopters, my parents use it. I would argue that they have steadily grown the user base by focusing on quietly improving from the get go, and relying on the web’s “grapevine” to spread the word. I really don’t recall a real flickr hype, personally (other than the Yahoo purchase, but is that the same kind of hype?), just running across a friends stream and thinking “oh, neat.” I certainly never saw flickr advertised anywhere… and I’m willing to bet their adoption rate has been pretty steady.

agreeing that it’s not the best way to gauge:

http://www.google.com/trends?q=flickr%2C+second+life&ctab=0&geo=all&date=all&sort=0

Bobby October 16, 2008 at 8:36 pm

dug around some more… would you call that a diffusion model?

Venkat October 17, 2008 at 4:46 am

Bobby:

Yes, what you are describing for Flickr is a classic diffusion process. Somewhere in the book the authors describe the characteristics of quick adoption, and visible use is one of them.

The cycle is a ‘typical’ example of a family, not the only pattern. The book lists some variants, like a two-peak curve (with a second small peak at the start of the slope of enlightenment) and ‘high-flier’ vs ‘low-flier’ curves for higher/lower plateaus etc.

I think the curve is likely to be a sort of fractal curve in reality, the somehow folds in lots of smaller cycles of peaks and troughs, but the way Gartner tracks it is based primarily on media reports I believe, not on every bug and every release version of s/w. That, as you say, would be a very noisy signal.

Tom:

Good point about survivorship bias. The book does suggest that lots of hype cycles never make it past the trough, such as in the case of the more vacuous management fads. They do not propose any leading indicators at the trigger/peak stage (such as “if all early adopters have green beards, the thing will fail in the trough.”).

Overall, I don’t view the hype cycle as a particularly good predictive tool. Though the authors have bits of the book devoted to “spotting” the peaks and troughs, they aren’t very convincing. I think it is best to treat the hype cycle as a conceptual and retrospective-analytical tool, whose evolution can only be observed with a short lag.

Predicting the peak/trough and especially the onset of the slope, without fundamental analysis of what the curve is tracking, is obviously going to be as difficult as timing the stock market. But with technological analysis, it should be possible, since developments will actually mean something.

I think the best way to understand the value of the cycle is to compare somebody, say A, who thinks with the HC model vs. somebody, say B, who believes a more naive model (say monotonic exponential or worse, a ramp or step function hype signature).

At the very least, A sees and can take advantage of one opportunity B cannot: the equivalent of shorting a stock, by investing in a technology via fire sales after a peak, getting stuff dirt cheap and then persevering through the trough.

Venkat

Venkat October 17, 2008 at 4:49 am

Okay, that’s not quite so much shorting as it is bargain-hunting. Wonder if there IS an equivalent of shorting in an adoption tactic.

Bobby October 17, 2008 at 9:23 am

Yeah, neat- I like the fractal idea for sure. Also, I would like to think that my analysis would reflect actual adoption rates, and I wonder what kind of patterns would emerge as a sort of z axis to the visibility/time thing?

Jackie Fenn October 24, 2008 at 8:26 am

Venkat: Thanks for the thorough and fair review – I found it funny and insightful, always a great combination. You also raise some interesting topics for further development, including the potential first-principles nature of the hype curve. Some kind of damped oscillation is certainly close and makes sense intuitively in terms of expectations converging on reality. However we haven’t yet managed to find a single equation for the shape of the curve – neither of us is a mathematician or physicist, so we have to rely on the kindness of friends and strangers to try and nail this one. If anyone out there is up for the challenge, we would love to see if a single equation exists, and if so, what it tells about the theoretical nature of the curve and what else it can be compared to.

Comments on this entry are closed.