g r o t t o 1 1

Peeve Farm
Breeding peeves for show, not just to keep as pets
  Blog \Blôg\, n. [Jrg, fr. Jrg. "Web-log".
     See {Blogger, BlogSpot, LiveJournal}.]
     A stream-of-consciousness Web journal, containing
     links, commentary, and pointless drivel.


On My Blog Menu:

InstaPundit
USS Clueless
James Lileks
Little Green Footballs
As the Apple Turns
Entropicana
Cold Fury
Capitalist Lion
Red Letter Day
Eric S. Raymond
Tal G in Jerusalem
Secular Islam
Aziz Poonawalla
Corsair the Rational Pirate
.clue

« ? Blogging Brians # »





Book Plug:

Buy it and I get
money. I think.
BSD Mall




 10/6/2003 -  10/8/2003
 9/29/2003 -  10/5/2003
 9/22/2003 -  9/28/2003
 9/15/2003 -  9/21/2003
  9/8/2003 -  9/14/2003
  9/1/2003 -   9/7/2003
 8/25/2003 -  8/31/2003
 8/18/2003 -  8/24/2003
 8/11/2003 -  8/17/2003
  8/4/2003 -  8/10/2003
 7/28/2003 -   8/3/2003
 7/21/2003 -  7/27/2003
 7/14/2003 -  7/20/2003
  7/7/2003 -  7/13/2003
 6/30/2003 -   7/6/2003
 6/23/2003 -  6/29/2003
 6/16/2003 -  6/22/2003
  6/9/2003 -  6/15/2003
  6/2/2003 -   6/8/2003
 5/26/2003 -   6/1/2003
 5/19/2003 -  5/25/2003
 5/12/2003 -  5/18/2003
  5/5/2003 -  5/11/2003
 4/28/2003 -   5/4/2003
 4/21/2003 -  4/27/2003
 4/14/2003 -  4/20/2003
  4/7/2003 -  4/13/2003
 3/31/2003 -   4/6/2003
 3/24/2003 -  3/30/2003
 3/17/2003 -  3/23/2003
 3/10/2003 -  3/16/2003
  3/3/2003 -   3/9/2003
 2/24/2003 -   3/2/2003
 2/17/2003 -  2/23/2003
 2/10/2003 -  2/16/2003
  2/3/2003 -   2/9/2003
 1/27/2003 -   2/2/2003
 1/20/2003 -  1/26/2003
 1/13/2003 -  1/19/2003
  1/6/2003 -  1/12/2003
12/30/2002 -   1/5/2003
12/23/2002 - 12/29/2002
12/16/2002 - 12/22/2002
 12/9/2002 - 12/15/2002
 12/2/2002 -  12/8/2002
11/25/2002 -  12/1/2002
11/18/2002 - 11/24/2002
11/11/2002 - 11/17/2002
 11/4/2002 - 11/10/2002
10/28/2002 -  11/3/2002
10/21/2002 - 10/27/2002
10/14/2002 - 10/20/2002
 10/7/2002 - 10/13/2002
 9/30/2002 -  10/6/2002
 9/23/2002 -  9/29/2002
 9/16/2002 -  9/22/2002
  9/9/2002 -  9/15/2002
  9/2/2002 -   9/8/2002
 8/26/2002 -   9/1/2002
 8/19/2002 -  8/25/2002
 8/12/2002 -  8/18/2002
  8/5/2002 -  8/11/2002
 7/29/2002 -   8/4/2002
 7/22/2002 -  7/28/2002
 7/15/2002 -  7/21/2002
  7/8/2002 -  7/14/2002
  7/1/2002 -   7/7/2002
 6/24/2002 -  6/30/2002
 6/17/2002 -  6/23/2002
 6/10/2002 -  6/16/2002
  6/3/2002 -   6/9/2002
 5/27/2002 -   6/2/2002
 5/20/2002 -  5/26/2002
 5/13/2002 -  5/19/2002
  5/6/2002 -  5/12/2002
 4/29/2002 -   5/5/2002
 4/22/2002 -  4/28/2002
 4/15/2002 -  4/21/2002
  4/8/2002 -  4/14/2002
  4/1/2002 -   4/7/2002
 3/25/2002 -  3/31/2002
 3/18/2002 -  3/24/2002
 3/11/2002 -  3/17/2002
  3/4/2002 -  3/10/2002
 2/25/2002 -   3/3/2002
 2/18/2002 -  2/24/2002
 2/11/2002 -  2/17/2002
  2/4/2002 -  2/10/2002
 1/28/2002 -   2/3/2002
 1/21/2002 -  1/27/2002
 1/14/2002 -  1/20/2002
  1/7/2002 -  1/13/2002
12/31/2001 -   1/6/2002
12/24/2001 - 12/30/2001
12/17/2001 - 12/23/2001
Monday, March 11, 2002
16:25 - Ahh, that's better.
http://www.appleturns.com/scene/?id=3619

(top) link
Yeah, call it "damage control" if you want. But while As the Apple Turns primarily bills itself as a lighthearted comedy news-type site about as seriously to be taken as "Wait, Wait, Don't Tell Me", the fact is that they've built up a pretty respectable history over the years of accuracy and reliability. Their rumormongering is limited but almost always correct (they were about the only ones to correctly predict the announcements at MWNY last year). Their links and commentary are always on-target. And their readership always seems to have useful information with which AtAT can fire back against dubious benchmarking results that seem to fly in the face not only of years' worth of core marketing specs, but of independent analysis and objective results in the commercial field:

First and foremost, c't itself admitted that the G4 should have mopped the floor with "the x86 FPU with its antiquated stack structure and eight registers only"-- so why, when the G4 was shown to be half as fast as the Pentium III, did the magazine just say "gee, we guess the G4's no supercomputer" and then saunter away, hands in pockets, whistling a jolly tune? Doesn't anyone think it's strange that they failed to mention that the SPEC2000 test, as compiled, utterly ignores the G4's Velocity Engine registers, which are what gives that chip its supercomputer-class, greater-than-gigaflop floating point performance? What c't did is tantamount to forcing you to write with your toes and then telling you that your handwriting sucks.

What's more, while the industry just loves SPEC benchmarks, faithful viewer Mark Davis reminds us that they've always been biased towards Intel processors, in part because the SPEC code just floods the chip with a constant stream of perfect instructions and let it work at peak efficiency, which is nothing like how real software is processed. As you may recall from Jon Rubinstein's "Megahertz Myth" spiel, Intel's recent chips take a speed hit from the recurring need to clear and refill those extra-long pipelines due to incorrect predictive branching-- it's that whole "pipeline tax" thing. With the SPEC test, there are no data dependency bubbles, and therefore no pipeline tax, so Intel's chips perform better than they would in actual battle conditions.

Ah yes, I knew there was something fishy. Of course, you have to dig down a level into the facts to get this interpretation, and there are still more mitigating factors (like the one about C't's compilers not being SIMD-enhanced either), but the thing about the SPEC benchmark being tuned for a completely non-real-world-like environment is really the big corker. Just another result of the PPC architecture being the underdog in rudimentary tests (like judging it purely by clock speed, or running tests that never introduce a branch misprediction bubble into the Pentium's long pipeline). I deal with this kind of thing at work all the time; network benchmarking companies use tools like SmartBits and Chariot to test the limits of networking equipment; unfortunately, the traffic they send is either so uniform and well-behaved as to hide all the benefits of our product's real-world adaptivity, or so bogus in its construction that our software can't deal with it with anything like the same efficiency as we handle realistic TCP/IP traffic in a real-world network environment. So the benchmarks favor the more simplistic technology, while it takes a good deal more delving and research to find out how those second-order conditions affect the results-- and a lot more explanation and spin to get people to accept it.

After all, those RC5 benchmarks that I linked to last week were not by any internationally respected testing authority, but they did certainly describe a heartwarming speed advantage in the G4's corner on tasks like RC5 keypair checking (a task which, I daresay, involves a lot of branch misprediction bubbles-- which is why the P4 fared even more poorly than the P3 in that test).

And besides, there have been documented a number of design oddities in the Intel architecture that are there solely to make the chip perform better in the SPEC benchmarks-- they provide no benefit in the real world, but they do kick up those numbers.

This is why I'm so looking forward to the G5-vs-Itanium days. Motorola's chips will be running in the 2GHz range while Intel's struggle to break 1GHz-- and that without 32-bit backwards compatibility in the latter case. At least there will be that one presumption working in our favor, one less thing that has to be refuted and debunked.

It just really sucks when everything looks bad until you peer deeper to find the real story, as opposed to the real-world truth being worse than the idealized lab tests. Though from a scientific and technophilic standpoint, the former certainly feels more elegant and satisfying.

Back to Top


© Brian Tiemann