James Dixon’s Blog

As part of our Agile BI initiative I am gathering data about the current use of agile methodologies for BI projects. What has been your usage or thoughts of agile?

Question 1

Question 2

There are different kinds of agile methodologies. In the simple agile approach every iteration has the same structure and produces the same kinds of deliverable (reports and cubes that users can access). In an Agile Unified Process the iterations come in phases – requirements/inception, design/elaboration, build/construction, deploy/transition – in each phase the iterations produce different deliverables.

Question 3

Agile teams are small and cross-functional. Who do you have on your agile BI teams?

Thanks for participating. For more information on our agile BI initiative:

Pentaho Agile BI Wiki

Introduction to Pentaho Agile BI

As part of Pentaho’s Agile BI initiative:

This recorded presentation is based on the Agile BI one I did at TDWI last week – http://wiki.pentaho.com/display/AGILEBI/Introduction+to+Agile+BI

We also have a wiki space for Agile BI – http://wiki.pentaho.com/display/AGILEBI/Welcome+to+Agile+Business+Intelligence

Seth Grimes at Intelligent Enterprise posted an article today – Open Source Decision Time for Pentaho BI. He raises some interesting points, although the article is a couple of years too late. Grimes seems to think that the addition of Lucidera’s ClearView into the Pentaho Enterprise Edition is, or indicates, a change in direction for Pentaho. This is not the case. Pentaho has had enterprise features for several years.

In the article, prompted by Pentaho’s recent acquisition of some of Lucidera’s IP, he writes:

That this centerpiece Enterprise Edition component was not and is not open source invites a question. Is Pentaho, founded as a “commercial open source” BI vendor, still an open-source company? Pentaho itself seems unsure.

It is good to question the actions of any vendor, open source or not, but this question makes no sense to me. If Pentaho was an “open source” BI vendor before this acquisition, and since all the open source software is still available, how can its status have changed?

Grimes also states that:

Analyst Merv Adrian characterizes Pentaho as “open core,” which seems like a very apt description. The Pentaho BI Suite’s basic components are open source, and non-open source elements are based on open standards. Neither reliance on an open core nor past exclusive use of open source components, however, is not sufficient for Pentaho to continue to call itself, at this juncture, a “commercial open source” company.

Pentaho, the company, seems itself of two minds about its status.

Personally I don’t like the term ‘open core’, but of all of the currently used terms, it is the closest to Pentaho’s business model. My objection to it, in Pentaho’s case, is that we offer much more than a ‘core’ in open source.

Grimes also seems, through the use of double or triple negatives, to imply that maybe Pentaho cannot call itself a “commercial open source” company any more, or is getting close to some boundary. The problem here is that there is no formal definition of the term “commercial open source”, and Grimes does not provide his definition. Some people have an inclusive definition, some people (ironically often the ‘free software’ advocates) have a very narrow and exclusive definition.

Since he raises the issue, we are not in two minds at all. Pentaho and it communities provides, in open source:

• An OLAP engine (Mondrian) with web-based slice-dice (JPivot)
• An ETL engine (Kettle)
• A report engine (formerly JFreeReport) with web-based ad-hoc reporting
• A BI platform with out-of-the-box web server and web-app deployments
• A Dashboard framework – CDF
• A metadata layer

This is more open source functionality than any other commercial open source BI vendor provides, and more than any open source community has managed to create. There are many thousands of implementations of these open source packages running around the world.

Pentaho Corporation provides to its customers:

• Additional features, functionality, and services that mainly provide a lower cost-of-ownership and faster development cycles.

For many open source vendors messaging in press releases and on websites is in continual evolution. The business models themselves are being refined – Alfresco over the past few years has switched from an open-core model to a services-only model, and then back to open-core again.

Still, I suppose those people who desire absolute messaging clarity from their software vendors, rather than good software with an exceptional value proposition, have every right to object.

A while back Red Hat did a great video called ‘Truth Happens’ based on this Mohandas Gandhi quote

First they ignore you…

Then they laugh at you…

Then they fight you…

Then you win…

The video is here: http://www.youtube.com/watch?v=wjSDAUykkzQ

MicroStrategy

Earlier this year MicroStrategy released a ‘free’ reporting edition. At the time BI commentators such as Cindi Howson did not see this as a strategy against open source – despite the fact that in the database and middle-ware space Microsoft, IBM, and Oracle all used this ploy. Now, it seems, MicroStrategy is contacting community members and bloggers in the open source BI space to ‘push’ their free offering.

http://twitter.com/josvandongen/statuses/4313121049

This indicates to me that their ‘free’ edition is, at least in part, an anti-open source gambit. This puts MicroStrategy in the ‘then they fight you…’ stage. Only one more stage to go MicroStrategy. We’re cheering for you…

Panorama Software

Further behind Microstrategy in the progression is Panorama. They seem to be moving from the ‘First they ignore you…’ to the ‘then they laugh at you…’ stage – judging from this post on their site:

http://www.panorama.com/blog/2009/09/will-saas-lead-open-source-to-extinction/

I particularly like this quote:

Well, advocates for political correctness will be happy to hear that cloud is chasing open source out the window, especially in the BI space.

This is amusing because, to my knowledge, the only BI companies to go under this year have been, wait for it…, SaaS BI providers – not open source BI companies or on-premise BI companies. First Lucidera and, this week, Blink Logic. To quote Captain Barbossa from Pirates of the Caribbean:

Think you can outrun the world? You know the problem with being the last of anything, by and by there be none left at all.

So Panorama seems to be firmly in the second stage, with two more to go. Keep going Panorama, maybe you can catch up with MicroStrategy. We’re cheering for you…

A beta version (known as ‘Release Candidate 1′) of Pentaho BI Suite Enterprise Edition V3.5 is now available for customers to try.

If you are not a customer and want to evaluate the Pentaho BI Suite Enterprise Edition you can either try the online demo or download a full version of the suite and try it out for 30 days with support.

This coming Friday I will be on the Frugal Friday Show – http://www.frugalfridayshow.com talking about open source-related stuff.

The show is hosted by Ken Hess (an author and consultant and Linux blogger on DaniWeb) and Jason Perlow who is a blogger on ZDNet

The show starts at 6:30 Eastern and I’ll be on at 7pm. I am looking forward to it, should be fun.

You can listen online – http://www.blogtalkradio.com/frugalfriday

Over the past few months I have been mentoring two students, Zueber Juma and Tej Amin, from Lake Highland Preparatory School. They have been working on a science fair project related to OpenMRS and the PEPFAR aids relief program using Pentaho’s technology. They set out to prove that medical record data from OpenMRS can be transformed using open source software into data structures that can be used to create PEPFAR reports. This system would allow AIDS treatment centers to create their own reports, instead of sending their data to the USA for report preparation.

They placed second at the country and state level and are now in Reno for the Intel International Science and Engineering Fair.

Good luck guys.

I have just added a few more ideas to the open projects page in the Pentaho community wiki – http://wiki.pentaho.com/display/COM/Open+Project+Ideas

If you are interested in trying any of them just let us know.

Sourceforge has opened the nominations for their 2009 Community Choice Awards. Pentaho qualifies in three categories:

• Best Project for the Enterprise
• Best Commercial Open Source Project
• Best Project

Please vote by clicking on this icon:

Thanks…

There are many advantages to embedding business intelligence into applications.

I just added a short movie to youtube about some of them: http://www.youtube.com/watch?v=uMyR-In5nKE

The advantages of embedding BI include:

• You get to re-use the security, context, and business rules of the application.
• The user gets a more seamless and integrated experience
• Easier installation and upgrade paths (for packaged applications)

The Pentaho BI platform was designed from the start to be embeddable. This entails architecting the entire platform so that it can be consumed as a set of libraries. This is not easy to achieve and is almost impossible to retro-fit into an architecture that was not designed to support it.

In a few lines of Java (10-20 lines) you can initialize the platform and execute reports and  ETL processes etc. The advantage of doing BI at this level is no request gets to the BI libraries, and no content gets back to users, without coming through the application’s security, context, and business rules etc.

Another advantage of embedding BI is that it enables a level of integration that is not possible any other way. For example when providing the data that is used to populate a report you have many options:

• The application calls the BI platform to execute a report, and the data source and the query are defined in the report itself.
• The application creates a query and passes it to the report before execution.
• The application creates and executes the query, and passes a live database cursor to the BI platform for the report engine to use.

It is this last option that is impossible to implement using any kind of web service or RPC-level integration between an application and a BI server. Only a fully-embedded BI solution can provide this level of integration.