Here are the results of a new survey by the BeyeNETWORK

http://www.pentaho.com/open_source_solutions_report/

As part of Pentaho’s Agile BI initiative:

This recorded presentation is based on the Agile BI one I did at TDWI last week – http://wiki.pentaho.com/display/AGILEBI/Introduction+to+Agile+BI

We also have a wiki space for Agile BI – http://wiki.pentaho.com/display/AGILEBI/Welcome+to+Agile+Business+Intelligence

I will be talking about Pentaho’s development focus for the near future and our Agile BI initiative. Agile BI presents a fast, interactive, and iterative approach to building and deploying BI applications. We are planning some exciting new capabilities and features around Kettle, Mondrian and Metadata.

I’ll will also be showing off some related prototype/storyboard code.

This is a technical event for the Pentaho community – click here for information about the event

Bradley M. Kuhn posted recently that “Open Core” Is the New Shareware

In it he trots out all of the usual misconceptions that the free software advocates frequently do about the open core model. But I give him credit for being direct about his Free Software allegiance (not purporting an open source one), and for describing what he imagines to be an ideal Free Software company.

The first move we have to make is simply give up the idea that the best technology companies are created by VC money. This may be true if your goal is to create proprietary companies, but the best Free Software companies are the small ones, 5-10 employees, that do consulting work and license all their improvements back to a shared codebase. From low-level technology like Linux and GCC to higher-level technology like Joomla all show that this project structure yields popular and vibrant codebases.

This identifies something that, to me, is the biggest flaw in the ideology of the Free Software movement – an end game that is unsustainable and unattainable.

As I have written before (in this post Misunderstanding open source #3: applying ‘Free Software’ religion to open source business models). I think it is important to imagine a future where the majority of software is free/open source software (FOSS). If that is something you think is desirable (and there are a lot of people who don’t), I think it is important to also imagine how we get there.

I assume that if Kuhn thinks that 5-10 person companies are the ideal ones to represent free software, then his ‘future’ for free software is a collection of 5-10 person companies stewarding all of the planet’s software. This implies little or no involvement from large software vendors or services companies like IBM, Oracle/Sun, and Microsoft. Not even medium sized or small companies can participate, only microscopic companies. Ask yourself this: in this future, can the software and support needs of most organizations and governments of the world be handled by this group of 5-10 person companies? Clearly not, this is ridiculous. Large organizations and companies require the products and support of much larger software providers. They require things like pre-sales support, RFPs, and service level agreements. They require 24×7 support worldwide. In this version of the future the open source vendors that exist cannot meet the needs of the mainstream markets. As a result the large proprietary software vendors survive happily, because the open source vendors do not provide a viable alternative. This is clearly at odds with the desires of the free software movement. They want to eliminate proprietary software and proprietary software companies, yet their behaviour ensures the survival of those things.

Aha! (you might exclaim) what about the example of Linux that Kuhn provides. Ok, lets examine that. Firstly you cannot use the original creator of Linux, Linus Torvalds, as a shining example of the virtue of free software because he frequently is critical of the free software ideology, he is not a free software advocate. Certainly you can point out that Linux is steered by a foundation and not by a corporation. But look at the board of directors of the Linux Foundation. As of today they are Larry Augustin CEO of SugarCRM (an dubious open core company, according to some), James Bottomley (Novell), Alan Clark (Novell), Wim Coekaerts (VP Engineering at Oracle), Masahiro Date (general manager of Fujitsu), Frank Fanzilli, Doug Fisher (VP Intel), Dan Frye (VP IBM), Bdale Garbee (Chief Technologist Hewlett-Packard), Tim Golden (Bank of America), Hisashi Hashimoto (Hitachi), Brian Pawlowski (NetApp), Chris Schlaeger (AMD), Tsugikazu Shibata (NEC), Eric Thomas (Texas Instruments), Christy Wyatt (VP Motorola). The mighty and free Linux is steered by a big software/hardware cartel – Novell (Microsoft’s Linux partner), Oracle, IBM, HP, Hitachi – and a chipset cartel – Intel/AM/TI/Motorola. I am not saying that this is a bad thing, in fact it is a good thing. What I’m saying is that using Linux as a ‘free software’ example is a big stretch.

The open core companies are trying to disrupt entrenched proprietary software markets. The free software movement dislikes proprietary software. They have a common enemy in proprietary software. The open core companies would welcome the free software movement as an ally, however the free software advocates choose open core as an enemy. What they don’t realize is that many potential open source companies are choosing between proprietary and open core – 100% open source is not an option. By inciting community members, as Kuhn does, to fork open core projects, he is creating an anti-open core environment that might persuade new start-ups to go proprietary rather than open core. The open core model is preferable to proprietary, but listening to the opinions of the free software advocates you don’t hear that message at all.

The path from a proprietary world to an open source one is a long one. It started over 20 years ago, and it will take another 20 years to get to the end. Maybe at the end the open core model will be a historical footnote. But along the road the open core model is a useful stepping stone that allows otherwise proprietary software companies to become at least partially open, on the way to full openness.

Brian Prentice at Gartner wrote a post titled Open Source’s Dying Narrative in which he talks about his perception that the ‘narrative’ behind open source is fading away. He says:

Well, actually, it’s interesting only to the extent you still believe the romantic narrative that commonly circulates around Open Source. That story involves bands of fiercely independent geek-heroes. Armed only with an Eclipse IDE, a weekend’s supply of Jolt Cola for energy and a poster of Jean-Luc Picard for inspiration, they set out to usurp the big software companies in their attempt to control the software universe.

This ‘romantic narrative’ that he is talking about is the ‘Free Software’ story, not the ‘Open Source’ story. As such Brian is falling into this trap – Misunderstanding open source #4: not knowing your own alignment

In my opinion open source has its own narrative – that open source, due to its inherent freedoms, is a better way to develop software, and that using and participating in open source makes sense from many angles. It is a pragmatic narrative not a romantic or ideological one.

But Brian’s point is interesting. I think he is correct. Certainly there are some topics, such as the on-going ‘open core business model’ debate, where the Free Software fans are very vocal. But in general it is natural and good that Brian has this perception – here is why:

Open source is slowly but surely being understood, accepted, and used by the mainstream software and IT groups. You only have to look at what has happened at Oracle and Microsoft in the last few months so see evidence of this. As this growth happens the new participants will talk about it and the press and the bloggers will chime in. Due to the nature of their involvement with open source, these new participants (established software vendors, systems integrators, IT departments, software startups, VCs etc) will be mainly aligned with the open source narrative, not the free software narrative. So as open source is adopted by the mainstream markets there will be much more open source narrative, but only slightly more free software narrative. The ratio of new news about open source vs free software will shift further in favour of open source. Since we can only consume so much information each day, our perception will be that the amount of free software narrative is decreasing. In reality it is not, the amount of romantic free software narrative is growing, but the volume of pragmatic open source narrative is drowning it.

Seth Grimes at Intelligent Enterprise posted an article today – Open Source Decision Time for Pentaho BI. He raises some interesting points, although the article is a couple of years too late. Grimes seems to think that the addition of Lucidera’s ClearView into the Pentaho Enterprise Edition is, or indicates, a change in direction for Pentaho. This is not the case. Pentaho has had enterprise features for several years.

In the article, prompted by Pentaho’s recent acquisition of some of Lucidera’s IP, he writes:

That this centerpiece Enterprise Edition component was not and is not open source invites a question. Is Pentaho, founded as a “commercial open source” BI vendor, still an open-source company? Pentaho itself seems unsure.

It is good to question the actions of any vendor, open source or not, but this question makes no sense to me. If Pentaho was an “open source” BI vendor before this acquisition, and since all the open source software is still available, how can its status have changed?

Grimes also states that:

Analyst Merv Adrian characterizes Pentaho as “open core,” which seems like a very apt description. The Pentaho BI Suite’s basic components are open source, and non-open source elements are based on open standards. Neither reliance on an open core nor past exclusive use of open source components, however, is not sufficient for Pentaho to continue to call itself, at this juncture, a “commercial open source” company.

Pentaho, the company, seems itself of two minds about its status.

Personally I don’t like the term ‘open core’, but of all of the currently used terms, it is the closest to Pentaho’s business model. My objection to it, in Pentaho’s case, is that we offer much more than a ‘core’ in open source.

Grimes also seems, through the use of double or triple negatives, to imply that maybe Pentaho cannot call itself a “commercial open source” company any more, or is getting close to some boundary. The problem here is that there is no formal definition of the term “commercial open source”, and Grimes does not provide his definition. Some people have an inclusive definition, some people (ironically often the ‘free software’ advocates) have a very narrow and exclusive definition.

Since he raises the issue, we are not in two minds at all. Pentaho and it communities provides, in open source:

• An OLAP engine (Mondrian) with web-based slice-dice (JPivot)
• An ETL engine (Kettle)
• A report engine (formerly JFreeReport) with web-based ad-hoc reporting
• A BI platform with out-of-the-box web server and web-app deployments
• A Dashboard framework – CDF
• A metadata layer

This is more open source functionality than any other commercial open source BI vendor provides, and more than any open source community has managed to create. There are many thousands of implementations of these open source packages running around the world.

Pentaho Corporation provides to its customers:

• Additional features, functionality, and services that mainly provide a lower cost-of-ownership and faster development cycles.

For many open source vendors messaging in press releases and on websites is in continual evolution. The business models themselves are being refined – Alfresco over the past few years has switched from an open-core model to a services-only model, and then back to open-core again.

Still, I suppose those people who desire absolute messaging clarity from their software vendors, rather than good software with an exceptional value proposition, have every right to object.

Many people are confused about the free software vs open source debate because they don’t know (or they hide) their own alignment. Take this quiz to help you identify where your inner compass lies:

Poll #1

Poll #2

Poll #3

Poll #4

Poll #5

You Are…

Mostly ‘1’s: You are a free software junkie. Own it, celebrate it, enjoy it. Just don’t pretend you are an open source advocate, you are not.

Mostly ‘2’s: You are a true open source hacker. As long as the source code is available (should you ever need it), and the license is OSI approved, you are all about getting the job done and moving on to the next problem.

Mostly ‘3’s: You are still trying to convince yourself that this open source thing is a temporary fad. You do this because it is much easier to do this than accept the truth – which is that your line of work, your company, your business model, and/or your past vendor choices are starting to seem badly wrong.

* Yes, actually.

Applying the philosophies of the Free Software movement to the world of open source business models is a pointless exercise.

To the participants in the open source movement, open source is a great way of creating software – the transparency, openness, and participation/contribution mechanisms enable better software to be created more quickly. This is a practical, pragmatic viewpoint. This is the viewpoint of the commercial open source companies. Open source advocates want a future where great, customizable software is available for every task or domain.

The participants in the Free Software movement see things differently. They have a purist viewpoint of  intellectual property rights, and apply their viewpoint religiously. Free Software advocates want a future where intellectual property and intellectual property laws are a thing of the past.

When Free Software advocates give opinions about commercial open source companies and business models, its like sending vegans to a  steak-house – there isn’t much on the menu to their liking.

Open Core Business Model

A particularly contentious issue for Free Software advocates is the ‘open core’ business model. In this model a company creates an open source code-base in a particular domain (business intelligence in our case), and provides additional features, support, services as commercial offerings. The idea is that the community around the open source code will help lower the development costs so that the commercial offerings can be at a price point that is disruptive in that domain.

The open core model only works if the open source software is full-featured and valuable. If it is not there will be no community, and no community contributions. Without this the development costs will be the same as with a proprietary development model and the company will fail. In most cases the ratio of installations of the open source software vs the commercial version is 99:1 or higher.

The Free Software advocates, however, claim that the ‘open core’ software is hobbled, restrictive, or so feature depleted, that everyone needs the commercial version. This is utter nonsense. Take Pentaho, we offer in open source:

• An Extract/Transform/Load (ETL) tool – Kettle.
• A metadata editor and OLAP schema editor
• A report designer
• A Business Intelligence server complete with ad-hoc reporting, pivoting slice/dice UI, dashboard framework, and scheduling.

I see the open core model as vital in this phase of the growth of open source. Imagine a future where the majority (or all) of the software used by individuals and businesses is in open source. How do we get there? By creating an environment where innovators and investors are discouraged from creating any open source? That does not help. I see the open core model as something that provides new software start-ups and established software vendors a viable option to use an open source model. If there is any significant up-front development or investment needed, a support/services only model is rarely seen as a viable model. For an ‘open source future’ the usage of the open core model is much more preferable to the proprietary option.

When it comes to start-ups the options considered viable (by most investors) are
1) produce 80-90% open source code with an open core model
or
2) produce 100% closed code under a proprietary model

Given these choices the first option is clearly preferable to open source advocates. To Free Software advocates, using an ‘exclusively 100% free’ viewpoint, the open core model is flawed and so they argue and petition against it – regardless of the practical consequences, which is more proprietary software.

If you reject the open core model you are, in effect, asking start-ups to stick to proprietary development. This is a step backwards and delays the creation and adoption of open source software over the long term

A while back Red Hat did a great video called ‘Truth Happens’ based on this Mohandas Gandhi quote

First they ignore you…

Then they laugh at you…

Then they fight you…

Then you win…

The video is here: http://www.youtube.com/watch?v=wjSDAUykkzQ

MicroStrategy

Earlier this year MicroStrategy released a ‘free’ reporting edition. At the time BI commentators such as Cindi Howson did not see this as a strategy against open source – despite the fact that in the database and middle-ware space Microsoft, IBM, and Oracle all used this ploy. Now, it seems, MicroStrategy is contacting community members and bloggers in the open source BI space to ‘push’ their free offering.

http://twitter.com/josvandongen/statuses/4313121049

This indicates to me that their ‘free’ edition is, at least in part, an anti-open source gambit. This puts MicroStrategy in the ‘then they fight you…’ stage. Only one more stage to go MicroStrategy. We’re cheering for you…

Panorama Software

Further behind Microstrategy in the progression is Panorama. They seem to be moving from the ‘First they ignore you…’ to the ‘then they laugh at you…’ stage – judging from this post on their site:

http://www.panorama.com/blog/2009/09/will-saas-lead-open-source-to-extinction/

I particularly like this quote:

Well, advocates for political correctness will be happy to hear that cloud is chasing open source out the window, especially in the BI space.

This is amusing because, to my knowledge, the only BI companies to go under this year have been, wait for it…, SaaS BI providers – not open source BI companies or on-premise BI companies. First Lucidera and, this week, Blink Logic. To quote Captain Barbossa from Pirates of the Caribbean:

Think you can outrun the world? You know the problem with being the last of anything, by and by there be none left at all.

So Panorama seems to be firmly in the second stage, with two more to go. Keep going Panorama, maybe you can catch up with MicroStrategy. We’re cheering for you…

A beta version (known as ‘Release Candidate 1′) of Pentaho BI Suite Enterprise Edition V3.5 is now available for customers to try.

If you are not a customer and want to evaluate the Pentaho BI Suite Enterprise Edition you can either try the online demo or download a full version of the suite and try it out for 30 days with support.