It was just 30 something years ago that I measured performance of an Intel
4004 @ 108 KHz. This year I plan to measure performance of an Intel P4 @
1.5 GHz. As the technology is skirting the limits of Moore's law and
limitless appetite for performance continues to grow, so does the need for
a fair yardstick. And vigilance is needed to assure that the performance
yardstick remains fair.
From the very beginning, architects, chips, systems, hardware, software,
application designers, and users have worked hard to measure and improve
performance. And the benchmark results show incredible improvement.
If all of the above is true, why do I have to waste a significant portion
of my day looking at the hourglass icon? Are the computers not getting
faster? Or are the performance pundits lying to us? Are the measurement
yardsticks, fashioned by pundits, fooling both architects and designers?
So are the benchmarks helping us or hurting us? I plan to discuss some of
these issues from the perspectives of users, engineers, marketeers,
lawyers, decision makers, and standards organizations. I will discuss
SPEC's structure, vision and future plans.
I would also like to solicit help from this august body in search and
sponsorhip of "better benchmarks", and ideas for the next suite.
Bio
Kaivalya M. Dixit is Program Director, Server Benchmarking at IBM in
Austin, TX, He is also the Chief Cook and Bottle-washer (aka President) for
the Standard Performance Evaluation Corporation (SPEC) and has held this
position since 1990. Kaivalya is a self confessed benchmarkoholic and
performance zealot who, in his copious free time (:--, runs and analyzes
benchmarks.
He has spent more than 30 years in the computer industry. During that time
he has worked for Ford Aerospace, Sun Microsystems, and IBM, where
he managed their RS6000 Architecture and Performance organization.
Kaivalya received his MS (EE) from Illinois Institute of Technology in
1965, when computers were as big as a building and students used log
tables and slide rules for computing. He claims that, while slide rules
may be slower on most computation, they are less subject to power
failures. He always keeps one on his desk.
