August 22, 2006

In support of complex tools

I think the calls for Java EE's demise way, way premature, and seem to be more about laziness (avoiding a learning curve), hubris ("I could do better!"), and generating notoriety on behalf of up-and-coming analysts & authors than being arguments of real substance.

I really like simple tools, and expect people to use them to the extent possible. But many problems require sophisticated tools. Yes, there is a tendency for people to over-complicate things, or "puff up" the problem to make it sound harder than it really is. But there are many hard problems, and there aren't always easy or "reasonable" solutions to these problems in many environments.

In my experience, the way to economically solve hard problems in an IT shop (whether an outsourcer, internal, whatever) is to use tools with multiple robust layers of abstraction, which can be peeled back to solve harder problems. This similar to what many Java EE, Microsoft .NET, CICS, IMS, Tuxedo, Oracle, or many OS-specific C++ environments have.

The way NOT to solve hard problems is to re-invent your own language and framework from the ground up, or rely on a new language with incomplete frameworks or niche target environments -- unless you're willing to invest in building or contributing to an open source community around the language or framework for the long haul, have the appropriate expertise & flexibility on tap, and are willing to deal with schedule risk.

The big problem with this latter approach is the following: when you want to peel back a layer of abstraction in such environments, you're left with either an small feature set or unstable abstraction, or a big gaping hole in the layers, requiring you to drop down to very low levels to solve the problem (with C libraries, for example). This is approach not clearly condusive to business value (it might be, but it's rare).

So, when looking for an alternative to Java EE, when dealing with hard problems, I think it's wrong to suggest, as some analysts do, that one can turn to a community-supported language & framework environment. The companies that do this must have tremendous expertise and flexibility in working at low levels to work with unstable abstractions or write their own versions of incomplete features (which likely would be re-contributed into OSS). For unpopular, difficult, or frustrating features, it's difficult to incent communities to build robust versions of such functionaltiy, and one likely has to pay for it themselves.

More plausible alternatives include Microsoft .NET, though I don't believe Microsoft .NET is really simpler to Java EE. It's clearly getting more complex at the language level, with C# 2.0's support for generics, partial classes, and C# 3.0's support for LINQ, extension methods, and lambda expressions. This isn't necessarily a bad thing, as one person's complexity is another person's "richness".

In any framework, as more people use it, they will require more variety and breadth, they will follow a similar path as the Java libraries have, with one exception: Java's API's have always been built with swappability in mind, most other library language bindings are one-offs. Building a language binding for a C library is one thing, building a canonical language binding for ANY library is quite another.

My point is this: while sometimes you can hit the "reset button" on a language to clean up unnecessary complexity (Java has a lot, to be sure), you can't magically whisk away natural complexity that is extrinsic to the language and framework. Database access, transactions, concurrency, fault tolerance, reliable distributed communication, cryptography and PKI, data transformation & binding, search & indexing, web frameworks, O/R frameworks etc. are all hard, complex problems to create a robust API standard for. Sun and the JCP, the Apache Jakarta project, and the Eclipse community have done tremendous work here and it would be a shame push it aside just because it has a learning curve & may have some design flaws.

Posted by stu at 10:15 AM