This is an old revision of the document!


John the Ripper GPU support

John the Ripper 1.7.9-jumbo-6 supported GPUs for the following hashes and non-hashes:

Reasonably efficient:

phpass sha256crypt sha512crypt md5crypt bcrypt MSCash2 WPA-PSK RAR Password Safe
CUDA Yes Yes Yes Yes Yes Yes Yes
OpenCL Yes Yes Yes Yes Yes Yes Yes Yes Yes

Notes:

  • Latest release include more formats and latest git trees include even more.
  • CUDA significantly benefits from compile-time tuning of parameters for a given GPU, see doc/README-CUDA
  • sha512crypt is currently faster on NVIDIA than on AMD/ATI, and moreover the OpenCL code is currently faster than CUDA (unusual)
  • bcrypt is sort of supposed to be inefficient on GPU, so it achieves at best CPU-like performance currently (yet you do get the option to put your many GPUs to at least some reasonable use if you only have bcrypt hashes to attack)
  • WPA-PSK CUDA uses a little bit of CPU code as well, and benefits from OpenMP (so you'll get slightly faster speed if you enable OpenMP in the JtR build, although you'll load multiple CPU cores then). The OpenCL version has post-processing on GPU too so doesn't use much CPU resources and will not benefit from OpenMP.
  • RAR is a mix of CPU and GPU code, with the ratio varying widely between different RAR archives (type, size). Later versions has mature “early reject” code that minimizes this.

Currently inefficient because the candidate passwords are generated on CPU only:

NT raw-MD4 raw-MD5 raw-SHA1 raw-SHA256 raw-SHA224 raw-SHA512 XSHA512 MSCash NSLDAP MySQL 4.1
CUDA Yes Yes Soon Yes Yes Yes
OpenCL Yes Soon Yes Yes Yes Yes Yes Yes Yes

Notes:

  • XSHA512 is actually pretty efficient in the “many salts” case and with few hashes per salt (normally there's just one hash per salt), achieving close to 70M c/s on GTX 580, which is reasonable for SHA-512 as it is not very GPU-friendly
  • raw-MD4 is inadvertently missing its high-level *_fmt.c file in 1.7.9-jumbo-6, although the OpenCL code is included. From Jumbo-7 on, this is corrected.

Previously, some of these things were available as patches, but starting with 1.7.9-jumbo-6 they're integrated into the jumbo tree.

Usage

AMD

  1. Install GPU driver (e.g., amd-driver-installer-12-4-x86.x86_64.run) and AMD APP SDK (e.g., AMD-APP-SDK-v2.6-lnx64.tgz)
  2. Build John
    • make linux-x86-64-opencl for 64-bit systems
    • make linux-x86-opencl for 32-bit systems
      • Replace the -opencl suffix with -gpu to build with CUDA support as well (if you have NVIDIA's CUDA SDK installed as well)
  3. Test John
    • Run ../run/john –test to make sure everything is OK
      • Actually, some tests will likely fail because there are currently no “format destructors”; you'd need to test the individual formats on their own, not all at once, to really make sure that everything is OK (or to find out if not, for real). So run commands like e.g. ”../run/john –test –format=wpapsk-opencl”.
    • Optionally download john opencl tester, extract to run directory and run opencl-test.sh (note: it tests a subset of OpenCL-using formats only)
  4. Now you are ready to go

NVIDIA

Add OpenCL and CUDA support.

  For 64 bits

Install GPU driver (e.g., NVIDIA-Linux-x86_64-340.32.run) and CUDA SDK (e.g., cuda-repo-7.0-28_amd64.deb)

 http://www.yourownlinux.com/2014/12/how-to-install-nvidia-340-65-stable-graphics-drivers-in-linux.html For Nvidia Driver Install
 http://docs.nvidia.com/cuda/cuda-getting-started-guide-for-linux/ For Cuda Install

Exports variables to terminal or add to ~/.bashrc (If you add to bashrc file remember to reload terminal –> source ~/.bashrc)

 export PATH=$PATH:/usr/local/cuda-7.0/bin
 export LD_LIBRARY_PATH=:/usr/local/cuda-7.0/lib64
 export PATH=$PATH:/usr/local/cuda-7.0/targets/x86_64-linux/include
 export LD_LIBRARY_PATH=:/usr/local/cuda-7.0/targets/x86_64-linux/lib

Build John: 1.7.9-jumbo-7

 make linux-x86-64-cuda for 64-bit systems
 make linux-x86-cuda for 32-bit systems

Build John: 1.8.0-jumbo-1

 ./configure && make clean && make -sj4 That should detect automatically Nvidia Opencl and CUDA

Check the output in the terminal, you should see:

 ....
 Opencl .... yes
 CUDA ...... yes
 ....

Check formats supported:

 ../run/john --list=formats format=cuda for CUDA
 ../run/john --list=formats format=opencl for Opencl

Test John

 Run ../run/john --test to make sure everything is OK
 Actually, some tests will likely fail because there are currently no "format destructors".
 you'd need to test the individual formats on their own, not all at once, to really make sure that everything is OK (or to find out if not, for real).  
 So run commands like e.g. "../run/john --test --format=wpapsk-opencl".
 Optionally download {{john:john_test_data:john-opencl-tester.tar.gz|john opencl tester}}, extract to run directory and run opencl-test.sh (note: it tests a subset of OpenCL-using formats only)

Now you are ready to go

  For 32 bits

Do the same as 64 bits but choose the correct files.

GPU selection

List available GPUs

../run/john --list=opencl-devices

Select specific GPU (in this example device #3)

../run/john --devices=3

Select multiple GPUs (set fork to the number of GPUs selected)

../run/john --devices=0,1,4 --fork=3

Benchmarks

MSCash2

As of 1.7.9-jumbo-6:

OpenCL version of MSCash2 supports cracking on multiple GPUs. For more detail see common_opencl_pbkdf2.h.

      Benchmarks: AMD Radeon HD 7970: 92.5K c/s real 
                  ATI Radeon HD 4890: 19.9K c/s real
                  Nvidia GTX 570    : 26.7K c/s real
                  AMD Radeon HD 7970 + Nvidia GTX 570: 113.5K c/s real

Old content

The old content of this page has been moved to GPU development.

john/GPU.1507943173.txt · Last modified: 2017/10/14 03:06 by rofl
 
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
Recent changes RSS feed Donate to DokuWiki Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki Powered by OpenVZ Powered by Openwall GNU/*/Linux