March 17, 2005


I caught this cello-based metal/rock/classical band last night in Toronto. One of the more entertaining live acts I've ever wtinessed, I can't remember grinning ear-to-ear as much. They played a mix of original songs and covers, most notably a bunch of old Metallica tunes, and a Sepultura track. They teamed up with Slayer's Dave Lombardo for select drum tracks on their last two albums, though he's not touring with them. I've been following these guys since 1996, when they released their first Metallica covers, and they've certainly risen past the "gimmick" stage, they're the Real Deal now.

Anyhow, for Americans, they're playing in Austin TX and California later in the week, I would suggest you check them out if you like heavy and fast music with a classical twinge.

Posted by stu at 11:54 AM

March 10, 2005

Interop with SOAP and REST

Carlos Perez has a series of articles about why REST is apparently better than SOAP. This whole thing is quite confusing to me, as I wasn't aware they were in conflict -- REST-like architecture is doable in SOAP, as it is in XML+HTTP. Chris Ferris has pointed out a lot of the problems with this series.

It really seems to be an argument that XML+HTTP is sufficient for web services , while SOAP and WS-* are unnecessary and complex. Secondly, it seems to be an emotional rant against an invisible body of "SOAP proponents" that are seeking to destroy interoperability in their wake.

He starts out with the following:

object.method( arg1, arg2, arg3 );

A collection of these methods is the typical starting point of a SOAP implementation.

Whoa, whoa, WHOA!? Perhaps in 4 or 5 years ago, this was true. SOAP and WSDL unfortunately had a lot of wrong turns in its early days, but they've been largely fixed through SOAP 1.2 and WS-I. So, I haven't seen this approach in a long while. The starting point of a SOAP implementation is to figure out what your XML looks like. Your basic invocation is more like:


Because WS-I Basic Profile lists document/literal as the preferred style of communication. RPC/literal is also supported but I don't really know of any vendors or users that use it.

Now, a modern SOAP framework will dispatch to a method based on the document's root element. And it will allow you to take an incoming XML document and divvy it up into arguments. WebLogic Workshop does this with XQuery maps. At their most simple, we just apply an XPath expression to point to the section of the document that maps to a method argument. But we could transform any inbound document into whatever method signature and data binding you want. This certainly helps interoperability.

How do SOAP and REST differ? Assuming HTTP as the transport, REST has the intent of the document transfer associated with the HTTP method, effectively layering a uniform interface on top of your document. Why is this a good thing? To quote Roy Fielding's thesis...

By applying the software engineering principle of generality to the component interface, the overall system architecture is simplified and the visibility of interactions is improved. Implementations are decoupled from the services they provide, which encourages independent evolvability.

Sounds like a good plan. Now, with WS-I SOAP+WSDL (irrespective of transport), the document itself indicates the intent. You figure out what to do with it based on the document type and/or contents. Thus, it's tailored to whatever the application's specific needs are. Let's continue that quote from Roy:

The trade-off, though, is that a uniform interface degrades efficiency, since information is transferred in a standardized form rather than one which is specific to an application's needs. The REST interface is designed to be efficient for large-grain hypermedia data transfer, optimizing for the common case of the Web, but resulting in an interface that is not optimal for other forms of architectural interaction.

And here we come to the problem. Many people are trying to use SOAP and WS-* as a general suite of protocols, one that's applicable to many different kinds of architectural interactions. XML+HTTP "REST" style approaches tend to come from large web-site companies, because that's their business - large grain hypermedia data transfer. Not all systems have that pattern. They can, and probably should, create their own uniform interface, but it should be in whatever approach makes sense for THAT application.

It's becoming extremely tiresome listening to SOAP proponents continually shift the argument. I need to emphasize again, the only 3 valid reasons are "Interoperability, Interoperability, Interoperability".

Accusations of "shifting the argument" usually indicate that the author has no respect or understanding for the other party's perspective. Other quotes: "SOAP proponents are full of disdain for REST" (really?), "We all know that its all broke, so stop with the farce and reboot.", and "Sure you guys listened, but it was with contempt. Just as you continue to write in a contemptuous manner."

I think Carlos is mistaking contempt and disdain for REST with contempt for his line of argument. The tone and intent of this series of blog entries is not of education, or insight, or information, it's pure hubris -- he's trying to prove that he holds THE ANSWER. Hammers and nails.

In all of these pseudo-REST arguments, where WS-* is apparently jettisoned, I haven't seen any indication of how to meet requirements about security (including identity), intermediaries, routing, callbacks, integrity, etc. other than "you don't really need those features". Tell that to our clients. They are saying something very differently -- "yes, we do need that". Misguided souls, or enlightened veterens?

Like CORBA and COM, I think SOAP and WS-* will have their successes. As will XML+HTTP. Perhaps the latter will be more prevalent -- I would even HOPE so. But it's silly to turn this into some sort of religious war about SOAP. There are numerous SOAP successes today that are invisible to the blogosphere, because they're inside corporations. Millions, if not billions of dollars of transactions run through SOAP at this very moment. I've helped to build some of these systems. And everything I see , talking with CIOs and enterprise architects, suggests that more will come. Live and let live....

Posted by stu at 11:10 AM

March 06, 2005

Expert One on One 10g edition sample chapters

Tom Kyte's Expert One on One: Oracle book is one of the best practical tech books available, in any topic. It was a real career changer for me. He's updating it for 10g, and the beta versions of chapter 1 and chapter 2 are now available. The final book is due by the end of the year. I'm giddy.

Posted by stu at 10:38 AM

March 01, 2005

building the new database, pt 2

In my last entry on this topic, I discussed Bosworth's blog entry back in December calling for the "new database". In my opinion, the "new database" is perhaps a combination of three trends:

a. Emphasizing probability over logical certainty. This means fewer "queries" and more "searches" with ranking-based approaches. This, by and large, seems to be the fundamental shift underway to deal with infoglut, and it's the hardest one. It completely changes the notion of what a database is. It no longer primarily is a fact-base or 'oracle' (ahem). It becomes (mostly) a predictor, or statistician.

b. Convergence of search operations, logical set-operations, tagged data, and common programming languages. It's very difficult" to truly create good abstractions, and even then they still leak. In terms of data, I think this requires a fundamental change of language, though certainly we've tried and failed in this task many times. The closest I've seen to a truly elegant data/language unification is was with Gemstone + Smalltalk -- and I think it can be done again, better.

c. A separation of logical from physical data structures. Schemas change a lot, they're much more dynamic than the late 1980's. This means database vendors actually need to implement the relational theory as intended - where one can compose a physical data structure that does not necessarily map 1:1 to its logical structure, as almost all databases continue to do today.

I reject claims that XML databases will be the ascendent to the "new database" for many reasons that one can find elsewhere.

The above three trends are ideals that may take years to solve. But It's my belief that the "new database", in some respects, is already here, but culturally I don't believe most developers are capable of understanding it. I'm going to explore why I think this is, along with how today's databases solve the three general problems of a) dynamic schemas, b) massive scalability & data volume, c) better physical/logical separation. Each of these will be in a seperate part... let me lead off with a couple of comments on why we're in this predicament.

"If the database vendors ARE solving these problems, then they aren't doing a good job of telling the rest of us."

I think the database vendors are trying quite hard to solve these problems and communicate this. Browse through Oracle's marketing material.

"The customers I talk to who are using the traditional databases are esentially using them as very dumb row stores and trying very hard to move all the logic and searching out into arrays of machines with in memory caches."

And this is completely due to what I would consider closed-minded cultural / historical reasons. It has a seed in reality, back around, say, 1997, when databases weren't as clusterable or built for web access. And it is exacerbated by legions of database developers that haven't unlearned their bad habits from 1995. But it's unnecessary.

My observation is that all of the three major programming camps - .NET, LAMPW (Linux,Apache,MySQL,PHP/Perl/Python,Whatever), or Java - especially the latter two, seem to have an allergic reaction to relational databases, in all aspects. Relational theory & design is loathed, physical design issues are glossed over, and generally the attitude consists of covering one's eyes and yelling "la la la la la la" loudly whenever someone suggests that it might actually be useful to really learn this stuff in-depth.

Each camp seems to have its own neurotic view of the world -- whether being wedded to the one "true" database (SQL Server, MySQL, PostgreSQL, Oracle, or DB2) , or being an (object|XML) bigot and turning one's nose up at 30 years of database theory, or believing that databases altogether are a stupid idea.

There's a mix of Not-Invented-Here, hubris, fear, confusion, embarrassment, and a general lack of memory about the debates that took place through the 1980's and 1990's on this stuff. Gurus of yesteryear - Kent, Codd, Date, Darwen, Pascal, etc. - are relics to today's generation of Java and .NET programmers. People complain constantly about how difficult it is to understand XYZ database because it's so different from ABC database, when it would be clear why this is if they read the FINE and FREELY AVAILABLE online documentation for all of the major databases - Oracle, DB2, SQL Server, MySQL.

Oracle has been a FREE download for over 6 years, and people still don't experiment with it, they just fear it.

Databases are perhaps the most complicated piece of software in use, after an operating system. People spend time to learn operating systems in-depth at college. They don't usually get the same in-depth exposure to databases. Perhaps that's one of the problems? Or is there just a religious fervour in the air?

All of the arguments paraded around Wikis and mailing lists and blogs on the "right" data format and "relations vs. objects vs. XML" was hashed over 20 years ago, and sadly it seems the intellectual results of that debate are widely scattered in journals, books, and out-of-print articles. All that's left is the observation that relational databases (or "SQL databases") triumphed in the marketplace, while objects triumphed in the minds of developers. Network/object databases are a small niche, and a tremendously entrenched number of hierarchical / flat databases on mainframes continue to demonstrate the incredible power of IT managers who just. don't. care.

But these worlds - programming, data management, and data interchange - don't need to be in opposition to one another. They can be complementary. And hopefully someone will figure out to find their appropriate strengths and unify them into the next great programming environment.

Posted by stu at 02:21 AM