[1] Bjarne Stroustrup. The C++ Programming Language, 4th Edition. Addison-Wesley Professional, 4th edition, 5 2013.
[2] Graham Sellers, Richard S. Wright, and Nicholas Haemel. OpenGL SuperBible: Comprehensive Tutorial and Reference (6th Edition). Addison-Wesley Professional, 6th edition, 7 2013.
[3] John Cheng, Max Grossman, and Ty McKercher. Professional CUDA C Programming. Wrox, 1st edition, 9 2014.
[4] Soyata, Tolga. GPU parallel program development using CUDA. CRC Press, 2018.
GPU Parallel Computing
| Type of study | Follow-up Master |
|---|---|
| Language of instruction | Czech |
| Code | 460-4181/01 |
| Abbreviation | GPU |
| Course title | GPU Parallel Computing |
| Credits | 4 |
| Coordinating department | Department of Computer Science |
| Course coordinator | doc. Ing. Petr Gajdoš, Ph.D. |
Subject syllabus
Mastering the following topics will serve as an entry point for the exercises. They set themselves the objective of introducing students into the problematics and outlining the subsequent work on the exercises.
Discussed Topics:
- Introduction to parallel programming on GPU, a brief history, CUDA
- CUDA architecture and its integration within standard C++ project
- Threads and kernel functions
- CUDA memories, patterns and usage
- Memory bank conflicts
- Program execution control, distribution of an algorithm
- Algorithm performance with respect to its parallelization on GPU
- Optimization on the data level, effective data structures.
- Optimization of programs with respect to the maximum GPU performance
- Support library CUBLAS
- The Case study
Practical exercises will follow on the above mentioned topics, which will expand on the practical level of programming.
- The first application in CUDA
- Data transfers to/from GPU
- Threads hierarchy, basic thread life cycle, limits, calling of kernel functions, parameters and restrictions
- CUDA memories, patterns and usage
- Memory bank conflicts, access optimization, suitable data structures
- Streams, parallel calling of kernel functions, synchronization on several levels
- The case study, experiment with more variants of the same program
- Vectors and matrices, the case study, large data processing, parallel reduction
- Introduction to several support libraries for linear algebra
- The case study, image manipulation, double buffering, optimization at the level of blocks, registers, etc.
- The case study, Interesting research topics, outline of possible Solutions, experiments
- Program tuning, debugging with nVidia nSight
Discussed Topics:
- Introduction to parallel programming on GPU, a brief history, CUDA
- CUDA architecture and its integration within standard C++ project
- Threads and kernel functions
- CUDA memories, patterns and usage
- Memory bank conflicts
- Program execution control, distribution of an algorithm
- Algorithm performance with respect to its parallelization on GPU
- Optimization on the data level, effective data structures.
- Optimization of programs with respect to the maximum GPU performance
- Support library CUBLAS
- The Case study
Practical exercises will follow on the above mentioned topics, which will expand on the practical level of programming.
- The first application in CUDA
- Data transfers to/from GPU
- Threads hierarchy, basic thread life cycle, limits, calling of kernel functions, parameters and restrictions
- CUDA memories, patterns and usage
- Memory bank conflicts, access optimization, suitable data structures
- Streams, parallel calling of kernel functions, synchronization on several levels
- The case study, experiment with more variants of the same program
- Vectors and matrices, the case study, large data processing, parallel reduction
- Introduction to several support libraries for linear algebra
- The case study, image manipulation, double buffering, optimization at the level of blocks, registers, etc.
- The case study, Interesting research topics, outline of possible Solutions, experiments
- Program tuning, debugging with nVidia nSight
Literature
Advised literature
[1] Bjarne Stroustrup. The C++ Programming Language, 4th Edition. Addison-Wesley Professional, 4th edition, 5 2013.
[2] John Cheng, Max Grossman, and Ty McKercher. Professional CUDA C Programming. Wrox, 1st edition, 9 2014.
[3] Tuomanen, Brian. Hands-On GPU Programming with Python and CUDA: Explore high-performance parallel computing with CUDA. Packt Publishing Ltd, 2018.
[4] Volodymyr Kindratenko, editor. Numerical Computations with GPUs. Springer, 2014 edition, 7 2014.
[5] Vaidya, Bhaumik. Hands-On GPU-Accelerated Computer Vision with OpenCV and CUDA: Effective techniques for processing complex image data in real time using GPUs. Packt Publishing Ltd, 2018.
[6] Jung W. Suh and Youngmin Kim. Accelerating MATLAB with GPU Computing: A Primer with Examples. Morgan Kaufmann, 1st edition, 12 2013.
[2] John Cheng, Max Grossman, and Ty McKercher. Professional CUDA C Programming. Wrox, 1st edition, 9 2014.
[3] Tuomanen, Brian. Hands-On GPU Programming with Python and CUDA: Explore high-performance parallel computing with CUDA. Packt Publishing Ltd, 2018.
[4] Volodymyr Kindratenko, editor. Numerical Computations with GPUs. Springer, 2014 edition, 7 2014.
[5] Vaidya, Bhaumik. Hands-On GPU-Accelerated Computer Vision with OpenCV and CUDA: Effective techniques for processing complex image data in real time using GPUs. Packt Publishing Ltd, 2018.
[6] Jung W. Suh and Youngmin Kim. Accelerating MATLAB with GPU Computing: A Primer with Examples. Morgan Kaufmann, 1st edition, 12 2013.