James Dixon’s Blog

James Dixon’s thoughts on commercial open source and open source business intelligence

Forking Protocol: Why, When, and How to Fork an Open Source Project

with 8 comments

The right to fork open source projects is an important part of the freedoms that open source provides. This right is not theoretical it is real. It is seldom user for several reasons. As Eric Raymond says in ‘Homesteading the Noosphere’ in ‘The Cathedral & the Bazaar”:

There is a strong social pressure against forking projects. It does not happen except under plea of dire necessity, with  much public self-justification, and requires re-naming.

The recent fork of Nagois by Icinga is the most recent example of forking. I have been involved in several forking or potential forking incidents. The Why, When, and How presented below are based on that experience, on observations of other forking incidents, and on the writings and blogs of Eric Raymond and others. I have searched the internet for a step-by-step guide or protocol for forking and could not find one, if anyone knows of one please let me know, and I will reference it.

Why Fork?

Answer – Because you cannot get the software to meet your needs any other way.

There are different reasons that this might be the case. If the core developers, who have write-access to the source code, don’t accept your features or patches. If the core developers reject use cases or a direction that community members want. If the core developers don’t agree between themselves on what direction to go in.

Just because you don’t like, or don’t trust, the core developers or custodians of the project is not a good reason to fork the project. Being worried that the core developers or custodians, at some time in the future, will do something you don’t like, is not a good reason to fork a project.

Any preemptive fork of Java or MySQL, purely because of Oracle’s acquisition of Sun, is bad. Until Oracle does something bad and repeatedly fails to make it right (despite rationally-communicated requests from the community) there is no need for a fork of Java or MySQL.

There are things that you, as a project owner, can do to reduce the potential of a fork. First and foremost is to listen to, and interact with, your community. A lack of communication was one of the reasons for the Adempiere fork of Compiere.

There are also technical things that you can do.

Avoid Forks Via Plug-Ins

Support plug-ins. Plug-ins make it easy for developers to add features to the software. By using a plug-in architecture developers can create new features by writing only what is necessary for the functionality they need. The core software will automatically recognize the plug-in and make the feature available to the end user. Transformations and filters in graphics programs are a good example of this – no changes to the core program are needed to create a new filter. Plug-ins are a good way to allow ‘me-too’ or ‘same-category’ features to be added. For example a graphics program might allow plug-ins that provide certain types of functionality: filters, transformations, brushes, import/export.

The limitation of plug-ins is that they do not typically enable low-level or fundamental modifications.

Avoid Forks Via Inversion of Control

Inversion of control uses object factories or injection to provide software that is being executed with the objects that it needs. The objects used by the system can now be defined in configuration files instead of being fixed in code.

As it relates to forking inversion of control is very powerful. When inversion of control is implemented fully it enables community developers to extend or modify any part of the software, not just plug-in features.

For example in the Pentaho BI platform we use a combination of Spring xml files (injection) and object factories to enable all of the main platform objects to be defined in configuration. We use this ourselves as we have  multiple implementations of some of the major objects, for example the repository layer. The benefit to the community is that they can change the behavior of any part of the system very easily:

  • Sub-class the object that closest matches your needs.
  • Override the methods needed to introduce the functionality you need.
  • Change the configuration to use your object instead of the original one.

The advantage of this approach is that different segments of the community can create their own extensions by writing the smallest amount of code necessary, and their is no need or incentive to fork the core project.

When To Fork?

Answer – When you have exhausted all other options.

Take a look a:

The publicly stated reasons for the Icinga fork of Nagios – Why A Fork?

The debate about whether to fork Compiere – Debate – Has Compiere Become Closed? Do We Fork? How?

The letter from the Mambo developers before the Joomla fork – Mambo Developers Letter To The Community

You can see from these pages that the decision to fork was not made lightly. It is clear there was much discussion and involvement happened before the idea of forking was considered. The reasons for the fork are communicated clearly, often with regret that a fork was necessary.

How To Fork?

Answer: Carefully and responsibly

Here are my step-by-step suggestions.

Step 1: Exhaust All Other Options

Communicate your frustrations clearly. Doing this in a private email will help reduce the chance of the core developers/custodians getting defensive.

  • Find out how other community members feel. If no-one else agrees with you, there is a good chance your frustrations are misplaced.
  • Describe what you feel would be the bast-case outcome, and how it would be a good result for everyone (not just you).
  • Do not threaten to fork the project at this point.
  • Give the core developers/custodians 2 to 4 weeks to react.

If you get no reply or a negative one, proceed to step 2.

Step 2 – Communicate Intent Privately

If step 1 fails, in a private email:

  • Communicate your disappointment that a mutually beneficial resolution has not been reached
  • Communicate that you have no option but to announce an ‘intent to fork’.
  • Again give the core developers/custodians 2 to 4 weeks to react.

If again you get no reply or a negative one, proceed to step 3.

Step 3 – Publicly Announce Intent to Fork

Publicly announce your intent to fork the project. Communicate that you have exhausted all other options. Specify the date at which the fork will occur. This date should again be 2-4 weeks into the future.

Create forum threads so that other community members can voice their opinions of the proposed fork. Participate in any discussions that starts up.

In most cases the community members who have pledged to follow the fork will spend the time between the public announcement and the fork date working on materials for the new project, such as the home page.

Step 4 – Fork

If nothing changes before the fork date arrives – fork.

When creating your new project choose a name that is different from the original project.

On your project web site make it clear that this is a fork, and why the fork occurred.

Do not remove the names of developers or copyright notices from any of the source files or documentation. Choose the same license type as the original project.

Make it clear which version or revision or tagged branch was forked.

Many Forks Are Avoided

By following the steps above forks can be avoided. I have been involved in situations that made it to step 2 in this process. In these cases, when the threat of a fork was seen as credible, the custodians of the project started to engage and a mutually beneficial agreement was reached. I am not naming these projects, that was part of the agreement we reached.

Bad Example of a Fork

Here is a bad case: In July 2007 someone tried to fork Pentaho Data Integration. The project was called ‘No Pentaho Pentaho Data Integration’. The reasoning given for this fork was:

No Pentaho Pentaho Data Integration is intended to take an early version 3.x of Pentaho Data Integration, remove most of the features Pentaho added and as such starting a new more stable version of PDI.

The apparent motivation for this attempted fork was that the trunk code for the next major release was, at the time, unstable. This was due to a significant re-write of the core engine, and the fork happened at a time then the instability was expected. None of the core developers or active community members supported the fork. To date this project has produced no downloads and has one anonymous developer.

This was clearly an unsuccessful fork that did not follow the steps above. The Pentaho Data Integration developers received no communication that anyone was frustrated with the running of the project. The developers had no opportunity to resolve any of the issues the member had. There was no privately or publicly communicated intent to fork. The fork just happened. Since then has produced nothing.

Feedback Welcome

If you have any feedback to this or links to related content please comment on this page: http://jamesdixon.wordpress.com/forking-protocol-why-when-and-how-to-fork-an-open-source-project


About these ads

Written by James

May 7, 2009 at 5:27 am

8 Responses

Subscribe to comments with RSS.

  1. Hi James,

    Thanks for picking up your blogging pace. You certainly have interesting things to say.

    There is indeed a strong prejudice against forking, and I certainly don’t want to contradict Eric Raymond, but really, a fork isn’t always a bad thing. One example is the Compiz/Beryl fork. The Beryl team forked Compiz and they both created different architectures for 3D desktop effects on Linux. When they were done experimenting with their respective architectures, and done arguing, they re-joined forces again, took the best parts of both projects. (Compiz Fusion) So the fork was good for all involved. These days, Compiz is still a monument in Linux desktop land.

    http://wiki.compiz-fusion.org/AboutCompizFusion

    So maybe we should see the right to fork as one important part of the open source ecosystem, where there is a struggle very much like a survival of the fittest. However, open source has on occasion (CompizFusion) the advantage of reasoning and enlightened spirits.

    I think slowly, but more and more, open source developers are starting to realize it’s not just about the code, it’s about the users. Licenses like GPL try to balance the rights of users and developers alike. A fork in that respect is rarely bad for the users. There are many examples of forks where users got the better end of the deal.

    However, from the perspective of a commercial open source company, any situation where users get the better end of the deal, is particularly hostile so I guess it makes sense to try and prevent it as good as possible.

    The idea of “Inversion of Control” (IoC) sounds really great in that respect and as mentioned elsewhere I agree to a very large extend. However, there is a very high associated cost with it. Anyone that ever tried to debug a Spring configuration file knows what I’m talking about. Spending days figuring out a small typo in a set of XML documents makes nobody happy. more than 4 years after that horrible project experience I had with it, I’m still traumatized. :-) There’s got to be a better way.

    So, my gut feeling is that IoC is great for major architectural components but not a magical solution for everything given the high re-design, development, configuration and maintenance cost associated with it. And I don’t think it can or should prevent forks if there are sufficient different goals, needs or (heaven forbid) trust issues. IMHO, no amount of code or re-architecture can prevent a fork for those reasons.

    Still, again, despite my comments, I still agree with you in the broader sense.

    All the best,

    Matt

    Matt Casters

    May 7, 2009 at 8:26 am

    • Matt,

      I think your comment about forks being done by the user community is a good one. The core developers have no need to fork, they have all the access they need – unless they can’t agree between themselves.

      James

      James

      May 7, 2009 at 1:48 pm

  2. Hi Matt,

    Thanks for the points.

    Inversion of control does not have to involve lots of XML files – I too have had really bad experiences trying to tweak those. There are other ways to do it.

    James

    James

    May 7, 2009 at 1:14 pm

  3. Another huge recent example: X.Org vs XFree86. The core developers failing to meet users’ and distributions’ needs for a long time, combined with strong personal rancor (and a public “expelling” of a key contributor who’d been at the centre of X development since 1990) led to more or less everyone immediately moving to the fork.

    David Gerard

    May 7, 2009 at 8:11 pm

  4. Good example David.

    James

    May 7, 2009 at 8:39 pm

  5. [...] Dixon recently blogged about the Forking Protocol: Why, When, and How to Fork an Open Source Project. I have for some time been a follower of implementing the forking protocol, as eloquently described [...]

  6. Interesting application by Smari – forking nations

    James

    May 7, 2009 at 9:07 pm

  7. Great post. All your post very important. I like content about open source, i need it. Thanks.

    Open Source Release

    May 8, 2009 at 1:46 pm


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 751 other followers

%d bloggers like this: