February 03, 2005

IBM vs Oracle in the TPC

The latest TPC-C benchmarks are an interesting war between Oracle's RAC vs. IBM DB2 on an SMP machine. A recent Usenet skirmish broke out on this, one that I feel the urge to echo here.

Benchmarks are rather useless for real-world comparison purposes, as all the machines are apples/oranges, but it makes for good entertainment. I liken it to a combination of horse racing (who will be fastest this time?), car stereo contests (stereos with bass response and wattage that could kill small animals), and fantasy football (what would have happened if XYZ conjectural system were used instead)?

I'm going to take the fantasy football route, briefly, because I'm a believer in clustering and am very interested in Oracle RAC.

IBM's p5 595 benchmark results: 3,210,540 tpmC, $5.19/tpmC.
Total 3-year system cost: $16.7m
Oracle/HP's Integrity rx5670 benchmark results: 1,184,893 tpmC, $5.52/tpmC
Total 3-year system cost: $6.5m

Even though both solutions are 64 processors, this can be deceiving. The HP benchmark is underpowered vs. the IBM benchmark in terms of cores, GHz, memory, cache, and spindles.

Let's even things up a bit before I get into differences. IBM's hardware is wonderful, IMHO, and the POWER5 rocks the Itanium2. So, let's put Oracle on an IBM POWER5 solution, say the p5 570.

A 4-way pSeries p5 570 is $94k USD on IBM's website.
The only TPC benchmark for the p5 570 lists ram at $108k for 32gb.

16 x 4-way p5 570's with 8gb RAM and 72mb L3 cache each = $1.5m
64 x 32gb memory boards = $6.9m (2 terabytes RAM)
Subtotal: $8.4m
Less 47% Discount: $3.95m
Total: $4.45m for the CPU + Memory + Chassis.
Add another ~$500k for extras (Fibre channel HBAs, Interconnect, etc.)
Grand Total: $5m

That's $1.5m less than the $6.5m (discounted) for the p595.

So, assuming the same storage & client configuration as IBM's p595 benchmark, and maintenance on the p570 and p595 is a wash, all that's left is software -- Oracle+Redhat is $1.6m discounted, vs. AIX+DB2's $800k. So add $800k to the total.

That's still $800k cheaper than the p595 solution, with 128 GB more ram (2176 gb total) and double the L3 cache. The only open question is if the cluster would bring in similar TPCC figures as the SMP box. Hopefully the following analysis will convince you about HOW superior IBM's hardware is to HP/Intel's.

Now, let's look at the differences between the two ORIGINAL setups:
- Both had 64 processors. 1.9GHz POWER5 vs. 1.5 Ghz Itanium2.
- The POWER5 is dual core and hyper-threaded, so can execute 4 threads simultaneously.
- The Itanium is 1 core and not hyperthreaded.
- The POWER5 has 96kb L1 cache and 1.92MB L2 cache. The Itanium2 has 32kb L1 cache and 256kb L2 cache. That's 3x L1, and 87% more L2.
- IBM had OVER THREE TIMES the number of disk spindles - 6,400 x 36 gig Ultra320 hard drives vs. 2,000 (mix of 18gb and 36gb) in the HP solution. That's 240 terabytes (well, 120, assuming half are RAID 1 spares), folks, for a 24 terabyte database.
- IBM had 83% more L3 cache (36 mb/proc vs. 6 mb/proc). That's 2.3 GB of L3 cache, if you were counting.
- IBM had 2 TB of RAM, which is 62.5% higher than HP's 768 GB.

Now, my p570 solution is close to even with the above (which is, remember, $800k less), but with the following improvements:
- 128 GB more ram (2176 GB)
- Double the L3 cache (76 MB / proc), which is around 4.8 GB of L3.

Given all of this, especially that the POWER5's are dual core HT, I think there's a reasonable chance Oracle RAC would sing. But this is all back-of-the-blog calculations, so, YMMV.

Of course these numbers all belong in "green giant" land. L3 cache in the GB and memory in the TB, and while we're at it, disk arrays with thousands of spindles, only 10% utilized, that can aggregately transfer (conservatively) around 10 GB/sec (more if you had more FC cards & switches)? Or how about a cluster with 8 GigE interconnects, giving an ability to transfer around 1 GB/sec across the cluster nodes? Monster truck madness!

Posted by stu at February 3, 2005 11:29 PM