People write toy Java technology benchmarks all the time. Nearly always they "get it wrong" -- wrong in the sense that the code they write doesn't measure what they think it does. Oh, it measures something all right -- just not what they want. This session presents some common Java technology benchmarking pitfalls, demonstrating pieces of real, bad (and usually really bad) benchmarks, such as the following: SpecJVM98 209_db isn't a DB test; it's a bad string-sort test and indirectly a measure of the size of your TLBs and caches. SpecJAppServer2004 is a test of your DB and network speed, not your JVM™ machine. SpecJBB2000 isn't a middleware test; it's a perfect young-gen-only garbage collection test. The session is for any programmer who has tried to benchmark anything. It provides specific advice on how to benchmark, stumbling blocks to look out for, and real-world examples of how well-known benchmarks fail to actually measure what they intended to measure.