Heterogeneous Parallel Programming

All computingsystems, from mobile to supercomputers, are becoming heterogeneous, massivelyparallel computers for higher power efficiency and computationthroughput. While the computing community is racing to build tools andlibraries to ease the use of these systems, effective and confidentuse of these systems will always require knowledge about low-levelprogramming in these systems. This course is designed for students tolearn the essence of low-level programming interfaces and how to use theseinterfaces to achieve application goals. CUDA C, with its good balance betweenuser control and verboseness, will serve as the teaching vehicle for the firsthalf of the course. Students will then extend their learning into closelyrelated programming interfaces such as OpenCL, OpenACC, and C++AMP.

The course is unique in that it is application oriented and only introduces thenecessary underlying computer science and computer engineering knowledge forunderstanding. It covers the concept of data parallel execution models,memory models for managing locality, tiling techniques for reducing bandwidthconsumption, parallel algorithm patterns, overlapping computation withcommunication, and a variety ofheterogeneous parallel programming interfaces. The concepts learned in thiscourse form a strong foundation for learning other types of parallelprogramming systems.