wafer_ai@wafer_ai1d ago

we launched the most comprehensive ai performance engineering repo in the world now we'll be posting every single resource this is Wafer's ai performance engineering series save this to keep up with the series. links in thread 🧵 part 3: "Intro to CUDA C++" from NVIDIA's CUDA Programming Guide. NVIDIA covers the execution model, memory movement, and correctness checks behind CUDA programs: - kernel launches, grid dimensions, and the organization of threads into blocks. - thread indexing and work assignment, including bounds checks for inputs that aren't multiples of the block size. - unified memory and explicit memory management, including control over data placement and transfers between CPU and GPU. - asynchronous kernel execution and synchronization before the CPU uses GPU results. - shared memory and block-level synchronization for threads that need to exchange data and coordinate their work. - runtime initialization and the setup costs that can affect measurements of the first runtime calls. - error handling for kernel launches and execution, including failures that surface in later API calls. - checking GPU results against a CPU implementation with a floating-point tolerance. the worked examples connect these concepts in a complete vector addition program, showing how to divide the work, manage its memory, and check the results before moving on to more complex kernels. figure from An Even Easier Introduction to CUDA (Updated)