Made a video walkthrough on how to run the CUDNN version from scratch

[ChrisDryden]

https://www.youtube.com/watch?v=eowhH4Nsx4I

[dagelf]

(I think you can embed it https://stackoverflow.com/questions/4279611/how-to-embed-a-video-into-github-readme-md)

Good video, TLDR;

CUDNN is about 15% faster on an A100 - 21000 tok/s (28ms/iteration vs 33ms/iteration)

Steps on clean Ubuntu 22.04:

git clone https://github.com/karpathy/llm.c
cd llm.c
sudo apt update
sudo apt install python3-pip
pip3 install -r requirements.txt
sudo apt install nvidia-cuda-toolkit # this links nvcc to 11.5 which is an old version
sudo apt -y install cuda-toolkit-12-4
PATH=/usr/local/cuda-12.4/bin:$PATH # this puts the newer version earlier in the path so nvcc --version shows 12.4
echo 'PATH=/usr/local/cuda-12.4/bin:$PATH' | tee -a ~/.bashrc
python prepro_tinyshakespeare.py
python3 train_gpt2.py
# make # will work here, but without CUDNN
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install libcudnn9-dev-cuda-12
git clone https://github.com/NVIDIA/cuddn-frontend ..
make USE_CUDNN=1
./train_gpt2cu

LOL you already had it typed up in the comments 🤦
Sadly you didn't share a key, or kept the host up 😋