1. Introduction.
Parallel processing and High Performance Computing (HPC). Parallel versus distributed processing. Parallel architectures, classification and impacts on parallel processing.
2. Introduction to Programming Parallel applications.
Parallel Programming models: Message passing, Shared memory, Data-parallel access. Examples of Application, advantages, and disadvantages. Overview of implementation environment and programming tools.
3. Message Passing Model.
Principles and characteristics of Message passing model. Types of Communication, basic terms. Implementation pre-requisites. Message Passing Systems as the model realization. Message Passing Interface (MPI).
4. Methods of Parallel Application Development.
Foster Methods. Data and Functional Decomposition. Communication Analysis, Perfectly Parallel problems. Agglomeration and Granulity, Processor Mapping.
5. Analysis of Parallel Algorithms.
Performance Models and Metrics. Parallel Codes Execution Time. Speed-up, Efficiency, and Costs. Superlinear Speed-up and its Causes. Amdahl’s and Gustafson’s Law.
6. Analysis of Parallel Algorithms (continuation).
Scaleability, Rigid and variable size of a problem. Asymptotic analysis. Experiments, Benchmarking.
7. Perfectly-parallel computations.
Ideal Parallelization. Monte Carlo Methods. Further Examples.
8. Decomposition and a Method called “Divide and Rule”
Decomposition Strategy. Numerical Integration. Further Examples.
9. Pipelining.
Principle of Pipelining. Prime Numbers Generation. Further Examples
10. Synchronized Computations.
Synchronized Constructs. Data-parallel Computations. Heat Transfer Problem. Further Examples.
11. Load Balancing and Termination Detection.
Centralized and Decentralized schemes. Mapping for Load Balancing. Shortest-Way Problem. Further Examples.
12. Shared Memory Programming.
Shared Variables Model. Technical Requirements. Threads and low-level Synchronization Primitives. Standard OpenMP. Automatic Parallelization.
Parallel processing and High Performance Computing (HPC). Parallel versus distributed processing. Parallel architectures, classification and impacts on parallel processing.
2. Introduction to Programming Parallel applications.
Parallel Programming models: Message passing, Shared memory, Data-parallel access. Examples of Application, advantages, and disadvantages. Overview of implementation environment and programming tools.
3. Message Passing Model.
Principles and characteristics of Message passing model. Types of Communication, basic terms. Implementation pre-requisites. Message Passing Systems as the model realization. Message Passing Interface (MPI).
4. Methods of Parallel Application Development.
Foster Methods. Data and Functional Decomposition. Communication Analysis, Perfectly Parallel problems. Agglomeration and Granulity, Processor Mapping.
5. Analysis of Parallel Algorithms.
Performance Models and Metrics. Parallel Codes Execution Time. Speed-up, Efficiency, and Costs. Superlinear Speed-up and its Causes. Amdahl’s and Gustafson’s Law.
6. Analysis of Parallel Algorithms (continuation).
Scaleability, Rigid and variable size of a problem. Asymptotic analysis. Experiments, Benchmarking.
7. Perfectly-parallel computations.
Ideal Parallelization. Monte Carlo Methods. Further Examples.
8. Decomposition and a Method called “Divide and Rule”
Decomposition Strategy. Numerical Integration. Further Examples.
9. Pipelining.
Principle of Pipelining. Prime Numbers Generation. Further Examples
10. Synchronized Computations.
Synchronized Constructs. Data-parallel Computations. Heat Transfer Problem. Further Examples.
11. Load Balancing and Termination Detection.
Centralized and Decentralized schemes. Mapping for Load Balancing. Shortest-Way Problem. Further Examples.
12. Shared Memory Programming.
Shared Variables Model. Technical Requirements. Threads and low-level Synchronization Primitives. Standard OpenMP. Automatic Parallelization.