Lectures:
=========
1. Introduction to parallel programming. Processes and threads. Processes and threads from the operating system perspective.
2. Sequential vs. parallel programming. Parallel programming caveats. Deadlock (definition, properties, conditions, detection, elimination).
3. Parallel vs. distributed applications. Classification of parallel systems. Shared memory systems and distributed memory systems. Flynn's taxonomy.
4. Shared memory systems programming. Programming with threads. The pthreads library, Threads in Java and C#. Synchronization and exclusion, deadlock.
5. The OpenMP interface. OpenMP support in modern compilers. OpenMP directives and functions. Reduction in OpenMP.
6. Fork-join model, Cilk and Cilk++ languages. Parallel Matlab. Parallel programming in Python, the NumPy library.
7. Distributed memory systems programming. Message passing. Posix queues, sockets. The MPI interface, fundamental MPI functions. MPI libraries.
8. Partitioned Global Address Space (PGAS) model. The Unified Parallel C (UPC) language.
9. Batch (non-interactive) task execution. Schedulers PBSPro and Torque.
10. Grid and cloud programming. Web services and distributed applications using web services. Map-reduce paradigm and Hadoop framework.
11. Introduction to accelerator programming. GPGPU architecture (program organization, memory organization). Data parallelism. CUDA platform and CUDA-C language.
12. Other interfaces for accelerator programming: OpenCL, OpenACC, OpenMP 4.
13. Intel MIC architecture. Programming the MIC and Intel Xeon Phi. Emerging platforms.
14. Hybrid applications. MPI-OpenMP programming, hybrid CPU-GPU architectures.
Tutorials:
==========
The topics and tasks dealt with at the tutorials will correspond to the topics presented at the lectures.
1. Overview of environments for parallel programming. SIMD instructions.
2. Implementation of a simple multithreaded program.
3. Implementation of a simple multiprocess program. Inter process communication with the help of sockets and queues.
4. Debugging a profiling of parallel programs.
5. Implementation of a simple OpenMP program. Parallelization of a sequential program by OpenMP.
6. Implementation of a simple program in Cilk++. A demonstration of parallel programming in Matlab.
7. Implementation of a simple MPI program.
8. Implementation of a simple program in UPC.
9. The use of PBS, PBSPro, and Torque schedulers; qsub command and QSUB directives.
10. Implementation of a simple map-reduce application under the Hadoop framework.
11. Implementation of a simple data-parallel program in CUDA-C.
12. A demonstration of OpenCL, OpenACC, and OpenMP 4.
13. Intel MIC programming. Implementation of a simple Intel MIC program.
14. Implementation of a simple hybrid MPI - OpenMP program.
=========
1. Introduction to parallel programming. Processes and threads. Processes and threads from the operating system perspective.
2. Sequential vs. parallel programming. Parallel programming caveats. Deadlock (definition, properties, conditions, detection, elimination).
3. Parallel vs. distributed applications. Classification of parallel systems. Shared memory systems and distributed memory systems. Flynn's taxonomy.
4. Shared memory systems programming. Programming with threads. The pthreads library, Threads in Java and C#. Synchronization and exclusion, deadlock.
5. The OpenMP interface. OpenMP support in modern compilers. OpenMP directives and functions. Reduction in OpenMP.
6. Fork-join model, Cilk and Cilk++ languages. Parallel Matlab. Parallel programming in Python, the NumPy library.
7. Distributed memory systems programming. Message passing. Posix queues, sockets. The MPI interface, fundamental MPI functions. MPI libraries.
8. Partitioned Global Address Space (PGAS) model. The Unified Parallel C (UPC) language.
9. Batch (non-interactive) task execution. Schedulers PBSPro and Torque.
10. Grid and cloud programming. Web services and distributed applications using web services. Map-reduce paradigm and Hadoop framework.
11. Introduction to accelerator programming. GPGPU architecture (program organization, memory organization). Data parallelism. CUDA platform and CUDA-C language.
12. Other interfaces for accelerator programming: OpenCL, OpenACC, OpenMP 4.
13. Intel MIC architecture. Programming the MIC and Intel Xeon Phi. Emerging platforms.
14. Hybrid applications. MPI-OpenMP programming, hybrid CPU-GPU architectures.
Tutorials:
==========
The topics and tasks dealt with at the tutorials will correspond to the topics presented at the lectures.
1. Overview of environments for parallel programming. SIMD instructions.
2. Implementation of a simple multithreaded program.
3. Implementation of a simple multiprocess program. Inter process communication with the help of sockets and queues.
4. Debugging a profiling of parallel programs.
5. Implementation of a simple OpenMP program. Parallelization of a sequential program by OpenMP.
6. Implementation of a simple program in Cilk++. A demonstration of parallel programming in Matlab.
7. Implementation of a simple MPI program.
8. Implementation of a simple program in UPC.
9. The use of PBS, PBSPro, and Torque schedulers; qsub command and QSUB directives.
10. Implementation of a simple map-reduce application under the Hadoop framework.
11. Implementation of a simple data-parallel program in CUDA-C.
12. A demonstration of OpenCL, OpenACC, and OpenMP 4.
13. Intel MIC programming. Implementation of a simple Intel MIC program.
14. Implementation of a simple hybrid MPI - OpenMP program.