Sunday, September 25, 2005

Surf's up

Earlier this year there was a small uprising in the open source world over Apple's commitment to giving back to the community. At the core of the dispute was Apple's WebCore project. Open source advocates were upset that Apple wasn't giving more back to the KHTML project considering what had been gained for free. The argument really heated up shortly after Dave Hyatt posted to his original Surfin Safari blog that WebCore, and hence Apple's Safari browser, now passed the Acid 2 CSS test.

Because of the publicity Apple has given to the close relation between the WebCore project and the KHTML project the members of the KHTML project were being asked how soon it would be until KHTML would pass the Acid 2 test. This led members of the KHTML project to post a few open letters to the community expressing their discontent over how Apple failed to collaborate effectively with the open source project.

Hyatt soon posted a reply to his blog asking anyone with suggestions about how they thought the WebCore project could change to improve the situation. Since the situation was already heated many of the comments fell into what I would consider non-productive chatter. I wouldn't say that it evolved to a flame war but there was much blame being placed on either party and a definite lack of concrete solution proposals.

I care about the success of Safari and the WebCore project just as I care about seeing an open source project like KHTML succeed. So I decided that enough was enough and that I should write a detailed response proposing a handful of manageable changes that would allow both teams to benefit even more from their partnership.

The WebCore project is now being managed differently and there is a new Surfin Safari blog. Although I have no way of knowing for sure, some of the ideas I had suggested are very similar to the changes that have actually been put in place. Now it would be extremely vain of me to think that there are no other bright people at Apple that could have come up with similar ideas. However, it is somewhat comforting that I was partly responsible for getting things rolling in what seems to be a productive direction.

Here was my response:

First let me give some background about myself so you can assess my level of bias. I am a computer science student at a Canadian university and I have an Apple ADC Student membership. I am also an Apple Campus Rep and many of the students in my program are very pro open source (as am I) so responding to issues like this one is something I must deal with often. Our university has a co-op program and while I am only beginning my 3rd year I effectively have 1 full year experience working in a commercial setting, particularly using open source frameworks to produce a commercial web apps. I know this isn't a great amount of work experience but it's better than none. My minor is in business with a focus on management sciences.

With that said here are a few things I think could be done better based on the information I have, which I admit is incomplete much like most people who have posted.

1- From what I gather communication between Apple and the KDE group has been only through email or informally through blogs. One of my co-op jobs was with Research In Motion (makers of the blackberry) so if anyone I can appreciate the benefits of email. However, email and blog postings fall very far from being a rich enough communication medium to coordinate development efforts. In my opinion many developers fall prey to over-relying on email because of it's ease and speed. Both companies and open source communities use it because it's cheap and widespread and I think that companies and open source projects run into problems when they rely on email too much. Email can't capture facial expressions or vocal intonations and is often miss-interpreted and the miss-interpretation can't be corrected on the fly like you can when you are talking face to face.

Soln: Organize a monthly or by monthly conference call with the lead developers of both parties (I know Apple has the resources for this since Campus reps have regular calls). Invite some of the KHTML devs to WWDC and send some WebKit devs to the KDE conference. I think this alone would change the outlook on the situation and knowing that there are people on the other end.

2- Apple is very closed about the bugs it is working to fix. This is a company policy and it is completely up to Apple if it wants to leave it as such. However, I think it not only helps external relations (members of the community wanting to contribute to an open source project) but also Apple (internally) if bugs logged against all of it's open source projects are also open.

Soln: There is no doubt already a team responsible for the initial bug triage that comes into radar (be it crash reports, people clicking on the report a bug button in safari, or even people logging bugs on their own through bugreporter.apple.com). When these bugs are initially reviewed and dispatched if they are dispatched to an open source project (WebKit, Darwin, GCC, Bonjour etc.) they should be sent to a separate, open bug tracking system. For example if a safari bug it logged but it ends up being a bug in the client then don't put it in the open source tracking system but if the bug originates in WebKit then go ahead and open source the bug. I have no idea what Apple uses for Issue tracking software but I have experience with 3 of the most popular issue tracking products and they provided some way of synchronizing the status of bugs between 2 databases. Better yet the radar database schema could be changed to incorporate an Open source flag. Whatever happens Apple needs to let other people interested in one of the open source projects what is currently being worked on so that work is not duplicated and it is well coordinated. What is the point of open source otherwise.

3- There were both suggestions and criticisms towards adopting a common code base. A point to keep in mind, KHTML and WebKit in their current state are 2 separate albeit open source projects. I don't think Apple has done any false publicity here since they were fairly straight forward when they said that WebKit was a fork. Apple in my opinion has also done a good job of helping the KHTML project both in terms of added functionality and publicity. KHTML has come a long ways since Safari has been released (I'm happy about this since KDE is my platform of choice when I run Linux) and although Apple's contribution seems to have slowed down lately compared to the initial benefits that were seen I think this is mainly a communication problem. If it takes more time to integrate a fix from one project into the other than it does to actually write the patch because of differences in both projects (i.e.: KHTML using Apple patches and vice versa) then it only makes sense that a developer would rather spend the time fixing it and feeling some sense of accomplishment rather than spend the time sorting through some other developer's work. To address the issue of what Apple has to gain from going through the trouble of moving to a common code base, well lots. Better PR than what it has been getting lately, increased awareness and use of a common web engine, a more robust rendering engine since any abstractions introduced separate the engine functionality from native UI toolkit functionality that may have crept through undocumented, and in the end, less work since the community can be fixing some of the bugs. Notice that these are potential benefits for KDE as well in my opinion.

Soln: I think that fundamentally both teams still have many of the same core values and direction for the project. Both teams still care very much about making a light-weight, fast, and common rendering engine. In my opinion I think that the problems here stem more from what I would call a superficial project management issue than a core divergence in project direction. Simply put, I think it would be a shame for a combined effort with so much potential to cease existing. Most of the complaints I can gather from the KDE camp are frustrations about how hard it is to reap the benefits of the combined effort; how hard it is to actually do more of the work the love doing rather than code base administrivia. Mr. Hyatt's offer about simply using web core (which may seem drastic at this point in the game), his continued posting of patches, and the very fact that he posed the question "what can Apple do?" is very clear indication to me that at least he, if not more people on the WebKit team and higher up at Apple, are more than willing to try to improve the situation. If Apple truly didn't care then why would they even bother with this sort of thing. The proposition to just go ahead an use WebKit as a common code base may seem like an effort from Apple to simply supplant KHTML but, although I am just guessing here, I think it shows just how much Mr. Hyatt wants to improve the situation and maybe even how desperate he might be and the lengths he is will to go to help resolve this issue.

Just so that this post doesn't suffer from any recency effects and people finish reading this thinking that I am putting most of the pressure on KDE devs, I think that both teams have to work equally if they hope to gain.