GPU-Assisted
Password Cracking

Who may need
Password Recovery?
 Ordinary users (own passwords)
 IT Departments (employee’s passwords)
 Security auditors, consultants and
penetration testers
 Law enforcement & government
 Hackers usually don’t!

Why speed counts?
Users and IT Departments:
«We needed those passwords
yesterday»
Auditors, consultants
and pentesters:
«Time is Money»

How to increase speed?
Traditional way is to network together
many computers to form a cluster
• Communication
overhead
• Difficult to manage
• Not power-efficient

Any other options?

Yes!
For many HPC applications GPUs
are many times faster
than CPUs
But they’re not only
faster, they are
greener!

Why?

CPUs are designed to
be efficient at serial
computing…
…while GPU’s
main concern is
parallel computing

Intel® Core™ i7-965
“The highest performing
desktop processor on the
planet.”
4 cores
3,2 GHz
731 million transistors
263 mm2

Memory Controller
IO
IO
Q
Core Core Core Core
u
e
u
e
QPI
QPI
L3 cache
8 Мb
>384 million transistors

Out-of-
Memory Controller
Order
Execution Units Scheduling
&
Retirement
IO
IO
Q
Ordering &
Execution
u Instruction
Core Core Core Core
Memory
L1
e Decode &
Data
Microcode
u
e
Branch Prediction
Inst Fetch & L1
QPI
QPI
Paging
L3 cache
L2 8 Мb
>384 million transistors

Memory Controller
IO
IO
Q
Core Core Core Core
u
e
u
e
QPI
QPI
L3 cache
8 Мb
>384 million transistors

CPU dedicates only
about 10% to the
execution units!
1/10

CPU dedicates only
about 10% to the
execution units!

NVIDIA®
GeForce® GTX 285
240 cores
1.476 GHz
1.4 billion transistors
470 mm2

PCIe &
TPC TPC TPC Memory TPC TPC
Controller
Thread
ROP Setup ROP
Dispatch
Memory
TPC TPC TPC TPC TPC
Controller

PCIe &
Multiprocessor
Multiprocessor
Multiprocessor
TPC TPC TPC Memory TPC TPC
Controller
Thread
ROP Setup ROP
Dispatch
Texture Memory
Fetch &
TPC TPC TPC TPC TPC
Controller
Other

PCIe &
TPC TPC TPC Memory TPC TPC
Controller
Thread
ROP Setup ROP
Dispatch
Memory
TPC TPC TPC TPC TPC
Controller

GPU dedicates about
30% to the execution
units!
1/3

GPU dedicates 6 times as many
resources to the execution units
as CPU!
183 Watts 6x130=780 Watts
full load full load

Performance
680
250
LM
195
32 S1070
GTX 295
2 600
1 330
NTLM GTX 285
795
87
Q6600
1 920
920
MD5
570
70
0 1 000 2 000 3 000
Millions passwords per second

Performance per $
85
521
LM
557
S1070
178
GTX 295
325
2 771
NTLM GTX 285
2 271
483
Q6600
240
1 917
MD5
1 629
389
0 500 1 000 1 500 2 000 2 500 3 000
Thousands passwords per $ per second

Performance per Watt
850
865
LM
1 066
305
S1070
GTX 295
3 250
4 602
NTLM
GTX 285
4 344
829
Q6600
2 400
3 183
MD5
3 115
667
0 1 000 2 000 3 000 4 000 5 000
Thousands passwords per watt per second

Bad News:
Not every algorithm is worth
offloading to GPU
MD4 / MD5 
GPU is good at computing SHA-1 / SHA-2 
RIPEMD 

but MD2
AES 
DES 
GPU is bad at accessing RC4 
random memory locations

Good News:
Humans love
repetition

WPA-PSK
Tesla S1070 52400
31500
HD4870x2
21700
GTX 295
15750
HD4870
12500
GTX 285
11800
GTX 280
3100
Core 2 Quad Q6600
0 10000 20000 30000 40000 50000 60000

Other Accelerators?
Based on FPGA (Xilinx)
FireWire
Proprietary SDK
US $3’995

Single Unit Performance
45000
40000
40000
35000
30000
27000
25000
TACC1441
20000
GTX 285
16000
15000 13500
10000
5050
5000 3050
0
PGPdisk 128 PGPdisk 256 Office 2007

US $3’995

US $3’995

Performance for $4K
500000
440000
450000
400000
350000
300000
250000 TACC1441
11x GTX 285
200000 176000
150000
100000
55550
50000 27000
13500 3050
0
PGPdisk 128 PGPdisk 256 Office 2007

Greener Computing
• Consider a cluster of 25
dual-CPU quad-core
computers
• 400 watts full load each
• 10’000 watts total

Greener Computing
• Two Tesla S1070 provide
same performance
• 800 watts full load each
• One computer for
management
• 2’000 watts total

Greener Computing
• 8’000 watts saved
• 49’090 kWh a year (at 70% utilization)
• € 5’890 savings on electricity a year
(at 0.12€ per kWh average rate)
• Prevents 27’500 kg CO2 emission
• Takes 5 cars off the roads
• Saves 2’300 trees/year

Thank You!
Andrey Belenko
a.belenko@elcomsoft.com