The Bees and the Trees – Part II

This is part two of the Bees and the Trees. Part one contains the basic models and I assume that you have read it. This part contains supporting evidence and observations.

Additional Background Information

Pentaho is the third start-up that I have co-founded, the other two both being BI start-ups founded with the same group of co-founders: Richard Daley, Doug Moran, Marc Batchelor, and Adrian Marshall. The first start-up was called Appsource with a product called Wired for OLAP that is now Hyperion Analyzer (shortly to become Oracle Analyzer). The second start-up was called Keyola and its products are now the foundation of the BI suite of Lawson Software. For both these start-up we used, as our ‘bible’, a book called ‘Crossing the Chasm’ by Geoffrey Moore. Chasm Theory describes how technology start-ups can overcome the hurdles to mainstream adoption via a specific marketing and partnering strategy. Our business strategy for both start-ups was based on Moore’s Chasm Theory. Many other successful technology start-ups have followed the same path.

When it came to starting Pentaho we left Moore’s trusty book on the shelf. We were stepping into the new territory of professional open source where the relevance or applicability of Chasm Theory was not proven.

At that time, reaching around for reading material about open source, two things became apparent. Firstly there were no texts about professional open source models. Secondly, when it came to regular open source Eric Raymond’s ‘The Cathedral and the Bazaar’ (CatB) is a highly recommended book. I have read Raymond’s book several times and it deserves its recommendations. In it, in a series of essays, Raymond explains the roots and history of open source and looks at open source and its participants from an anthropological and motivational perspective. In one of the essays, ‘The Magic Cauldron’, Raymond summarizes various possible open source business models. He does not go into much detail on how those models work in practice, probably because Raymond, very commendably, sticks largely to his own experience and did not have any in-depth experience with professional open source models at the time. My intention here is not to plagiarize Raymond’s works in any way but to provide some details from my own personal experience of professional open source. I hope these pages are taken in that spirit.

The ultimate question I want to answer here is: Do all the principles, effects, and disruptive nature of open source apply to professional open source or does its commercial bias fatally pollute its own foundation?

I will describe how, to my mind, professional open source works in practice by looking at the differences I have experienced between working in proprietary software companies and working in a professional open source company. I will contrast and compare proprietary, open source and POSSs from many perspectives and point out interesting comparisons as they come up.

My intention is that you do not need to read either ‘Crossing the Chasm’ or ‘The Cathedral and the Bazaar’ in order to understand this paper although they are both great books that are well worth reading. If you do read them both you will come to appreciate how hard they are to compare to each other. They describe very different worlds.

Crossing the Chasm

Moore contends that the early adoption of a new technology involves two groups of people: technology enthusiasts (techies) who review and ‘bless’ the soundness of the technology and visionaries who see the potential of the technology and want to use it. The visionaries trust the opinions of the techies and are willing to take the risks of using the technology on a larger scale. So far so good. The next phase in the adoption involves the ‘early majority’ or ‘pragmatists’ of the market. This group of people are much more conservative than the visionaries. They want a ‘whole product’ that is proven and endorsed by people they trust, is packaged in an easy-to-consume and maintain form, that comes with guarantees and warranties, and has specialists that they can engage to provide services, support, and training if needed. The ‘Chasm’ described by Moore exists between these two populations of adopters as the ‘early majority’ often view the visionaries as maverick risk-takers who are not to be trusted. Moore suggests marketing and partnering strategies that can be used by a technology organization to get across this ‘Chasm’.

The history of MP3 music players includes an example of ‘Chasm Theory’ in action. In 1998 the Recording Industry of America Association (RIAA) took Diamond Multimedia to court in an attempt to block the sale of their Rio MP3 player. The Court of the Ninth Circuit eventually rejected RIAA’s case. After their defeat RIAA made the statement that it really didn’t matter because it was too hard to download music and transfer it to the Rio for it to gain mass market adoption. They were predicting that the Rio could not cross the Chasm.

At that time Moore was in agreement that MP3 players were pre-Chasm. In his description of the techies he said ‘At the moment I am writing this sentence they are on the Internet at an MP3 site downloading songs to play on a Diamond Rio playback machine.’

In 2007, 9 years later, we could hardly use ‘owning an MP3 player’ to identify the techies. Even the Pope had an iPod by then.

In the end the MP3 player that crossed the Chasm was Apple Inc’s iPod, not the Rio. Why? The iPod came in multiple styles and colors, with a desktop interface (iTunes) to make it easy to transfer songs, an on-line store for buying songs (iTunes Store), and a very effective mass-marketing media campaign. Also, very significantly, Apple had support from the music industry because the iPod includes encryption to protect their rights. The iPod was a ‘whole product’ designed for the mainstream market. Credit is due to Moore as he specifically identified Apple Inc’s Steve Jobs as being a prime example of a visionary.

Chasm Theory, as defined by Moore, only applies to discontinuous innovation (one requiring a change in behavior by the consumer). Some people jump up and down and wave their arms in frustration when people refer to it in the context of smaller (not discontinuous) innovations. Some say that open source does not represent a discontinuous innovation. I say, from experience, that Chasm Theory can be applied very successfully to all kinds of innovation. I also will show that either way open source represents a discontinuous innovation because it does involve a (positive) change in behavior.

Open source has a large number of technology enthusiasts that bless it and there are many visionaries companies that are using it in production. But the barriers listed above are classic ‘Chasm’ ones. What Ray Lane was saying was that open source is not a ‘whole product’ and by Moore’s definition open source software is a pre-Chasm technology.

In my career I have been involved with creating ‘whole product’ for 8 different software packages. In my experience this is not an easy process. There is a lot more to creating a product than the 0’s and 1’s of the software. Making it work is easy, making it easy is work – hard work. The open source model is great at producing software, but it does not produce ‘product’. Thus the barriers to adoption of open source are real.

There are purists in the open source community who feel that open source exists in a realm that should not be open to commercial organizations. The opportunists and pragmatists see the removal of these barriers and the potential of mainstream adoption of open source as a huge opportunity. This second group contains the founders of the POSS Companies.

Professional open source companies exists to create a ‘whole product’ around an open source project (software+community) and deliver it to a mainstream market. They must get across the Chasm on their own.

If you take a good student of business or economics, gave them only ‘Crossing the Chasm’ and an understanding of open source, they should be able to predict the existence of professional open source companies.

Operational Observations

The proprietary software development approach is what Eric Raymond likens to building a cathedral. It is undertaken by a closed sect of people who work in isolation until the edifice is complete. The open source approach Raymond likens more to a bazaar where there is much interaction between consumers and producers and dynamic competition for space and participation. Hence the title of his ground-breaking work ‘The Cathedral and the Bazaar’.

Raymond approached his comparisons from an anthropological perspective. I will reach many of the same conclusions by making comparisons from operational, procedural, and practical perspectives.

In the following sections I will discuss how the factors that affect COSS, proprietary, and open source models.