Jack and Coke? How about John and CUDA (w/ Rocky 8.7 Live)

In previous writeups such as xmrig with cuda for Rocky Linux 8.5 and nVidia CUDA with the wrong video card I’ve navigated Rocky Linux and Cuda. It’s now time to see if we can get John the Ripper CUDA’s components running on a Rocky 8.7 Live Workstation USB install.

Personally, I love projects like this. I started this on 1 8GB USB stick and quickly realized that not only the space required wasn’t enough but I’d need more to do what I needed. I ended up getting 3 SanDisk 32GB Ultra USB 3.0 Flash Drives from Amazon for $16.96.

The biggest help with the Live USB install is using balenaEtcher to get the 2.1GB ISO to an 32GB USB stick. Once that’s done we can boot directly to the Live OS and start our installs.

I did have some derps with balenaEtcher failing to burn the ISO due to a failure of diskpart not returning a positive result to the clean operation. To resolve this I had to use PowerISO to clean the USB volume before windows would properly do it’s clean operation. Minor note to PowerISO is that it contains bloatware during the install and a wrongly-clicked click can give you headaches.

Live Stuff

  • Booted up Rocky 8.7 Workstation Live Workstation from a USB to install Rocky 8.7 Workstation on a separate USB stick.
  • Root with password, user with password
  • rebooted into USB bootable
  • #win

Now onto the necessities to get to our final goal

Downloads and Installs

Fresh install and first boot leaves up with a bit of free space, albeit many of the nVidia and CUDA installs are hefty we might have to do some space-shifting to make sure this all works.

[user@johncuda ~]$ df -h
Filesystem                           Size  Used Avail Use% Mounted on
devtmpfs                             3.9G     0  3.9G   0% /dev
tmpfs                                3.9G     0  3.9G   0% /dev/shm
tmpfs                                3.9G  9.5M  3.9G   1% /run
tmpfs                                3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/mapper/rl_localhost--live-root   25G  5.4G   20G  22% /
tmpfs                                3.9G   60K  3.9G   1% /tmp
/dev/sdb1                           1014M  264M  751M  26% /boot
tmpfs                                789M   24K  789M   1% /run/user/1000

There’s a ton of typical set up for something like this, such as updating everything. On an Intel(R) Core(TM)2 Quad CPU Q9650 @ 3.00GHz system running off a USB stick this takes quite a period of time.

$ sudo yum install epel-release
$ sudo dnf config-manager --set-enabled powertools
$ sudo /usr/bin/crb enable
$ sudo time dnf install kernel-devel-$(uname -r)
34.76user 11.92system 1:24.75elapsed 55%CPU (0avgtext+0avgdata 191272maxresident)k
24992inputs+578976outputs (5major+1733808minor)pagefaults 0swaps

Installing CUDA Toolkit

Downloading the nVidia CUDA Tooklit for Rocky 8 was a breeze. Installing it, however, took it’s sweet time

$ time sudo rpm -i cuda-repo-rhel8-12-1-local-12.1.0_530.30.02-1.x86_64.rpm
warning: cuda-repo-rhel8-12-1-local-12.1.0_530.30.02-1.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 1dcc03d4: NOKEY

real	46m55.862s
user	0m48.051s
sys	0m7.276s

Ran all the commands, rebooted, and cuda was installed but it seems that nvidia-driver:latest-dkms didn’t complete. Testing nvidia-smi resulted in the following:

[user@johncuda ~]$ nvidia-smi
NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

nVidia Driver

Per nVidia CUDA with the wrong video card we get to install the latest nVidia driver. Instead of downloading the 470.103.01 (Jan 31, 2022) driver from the previous post I’m gonna get the latest 470.182.03 (Mar 30, 2023) driver for our Geforce GT630.

The installer needed specific steps, such as “Continue” and “No” before it began to build new Kernel Modules. Clicked an awesome “No” for compatibility library installation. Received an error about libglvnd, which I don’t need. Finished with the nvidia-xconfig run and things finalized without an issue. Rebooted one more time to make sure I’m in the green and it completely failed.

As a minor note it’s best to install all this stuff while X is not running. Quickly going to runlevel 4 and rerunning the driver install made it much easier and quicker things to finally work once kicking back on X.

Prior to figuring all this out there were more weird steps. Rocky 8.7 workstation comes with the nouveau driver, an open-sourced nVidia driver, and this installation required that it be disabled.

$ nvidia-smi
Tue Apr 18 11:59:12 2023       
| NVIDIA-SMI 470.182.03   Driver Version: 470.182.03   CUDA Version: 11.4     |
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 N/A |                  N/A |
| N/A   35C    P8    N/A /  N/A |    192MiB /   981MiB |     N/A      Default |
|                               |                      |                  N/A |
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|  No running processes found                                                 |

John & CUDA

A quick link out to John the Ripper we get 1.9.0 jumbo downloaded.

./configure led us to needing openssl-dev and opencl-headers. Compiling was quick and easy. Checking our built executable and the necessary support:

[user@johncuda run]$ ./john --list=opencl-devices
Platform #0 name: NVIDIA CUDA, version: OpenCL 3.0 CUDA 11.4.364
    Device #0 (1) name:     NVIDIA GeForce GT 630
    Device vendor:          NVIDIA Corporation
    Device type:            GPU (LE)
    Device version:         OpenCL 3.0 CUDA
    Driver version:         470.182.03 [recommended]
    Native vector widths:   char 1, short 1, int 1, long 1
    Preferred vector width: char 1, short 1, int 1, long 1
    Global Memory:          981 MB
    Global Memory Cache:    32 KB
    Local Memory:           48 KB (Local)
    Constant Buffer size:   64 KB
    Max memory alloc. size: 245 MB
    Max clock (MHz):        901
    Profiling timer res.:   1000 ns
    Max Work Group Size:    1024
    Parallel compute cores: 2
    CUDA cores:             384  (2 x 192)
    Speed index:            345984
    Warp size:              32
    Max. GPRs/work-group:   65536
    Compute capability:     3.5 (sm_35)
    Kernel exec. timeout:   yes
    NVML id:                0
    PCI device topology:    01:00.0
    Temperature:            38°C
    Utilization:            n/a

Now we’re good to go!

Leave a Reply