Message Passing Interface (MPI) is a standardized and widely used communication protocol for parallel computing. It enables efficient communication and coordination between multiple processes running simultaneously on different processors or computers within a distributed computing environment.
Parallel computing: It allows tasks to be divided among multiple processors, improving computational speed and efficiency.
Scalability: It scales effectively for both small and large-scale parallel applications.
Portability: Being a standardized interface, MPI implementations are available across different platforms, ensuring code compatibility and portability.
The working of MPI is as follows:
Message passing: It processes communication by sending and receiving messages explicitly using MPI functions.
Point-to-point communication: It sends and receives messages between specific processes.
Collective communication: It allows communication involving multiple processes simultaneously, such as broadcast, scatter, gather, and reduce operations.
Synchronization: It provides mechanisms for synchronization among processes.
MPI implementation is as follows:
Library: MPI is implemented as a library of functions that can be called from various programming languages, such as C, C++, and Fortran.
MPI routines: It contains a set of routines for various communication operations, such as sending messages, receiving messages, and synchronizing processes.
MPI standards: It follows a set of standards defined by the MPI Forum, ensuring compatibility and uniformity across different implementations.
The below coding example demonstrates basic MPI functionalities by initializing the MPI environment, querying the total number of processes and each process's unique rank, retrieving the processor's name, and outputting a "Hello World" message that includes the rank, total process count, and processor name for each process.
#include <mpi.h>
#include <iostream>
#include <sstream>
#include <iomanip>
#include <vector>
int main(int argc, char *argv[]) {
MPI_Init(&argc, &argv);
int world_size, world_rank;
MPI_Comm_size(MPI_COMM_WORLD, &world_size);
MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);
char processor_name[MPI_MAX_PROCESSOR_NAME];
int name_len;
MPI_Get_processor_name(processor_name, &name_len);
// Create a single "Hello world" message
std::stringstream ss;
ss << "Hello world from processor " << processor_name
<< ", rank " << world_rank << " out of " << world_size << " processors";
std::string message = ss.str();
message.resize(100, ' '); // Ensure each message is 100 characters long
// Gather buffer only on rank 0
std::vector<char> gather_buffer;
if (world_rank == 0) {
gather_buffer.resize(100 * world_size);
}
// Gather messages from all ranks to rank 0
MPI_Gather(message.c_str(), 100, MPI_CHAR, gather_buffer.data(), 100, MPI_CHAR, 0, MPI_COMM_WORLD);
// Rank 0 displays each gathered message exactly once
if (world_rank == 0) {
for (int i = 0; i < world_size; ++i) {
std::cout << std::string(&gather_buffer[i * 100], 100) << std::endl;
}
}
MPI_Finalize();
return 0;
}In the code above:
Lines 1-4: The necessary MPI and C++ headers are included. This includes <mpi.h> for MPI functions and <iostream>, <sstream>, <iomanip>, and <vector> for input/output operations and data handling.
Line 6: The main function begins with parameters argc and argv, which are standard for C++ programs to accept command-line arguments.
Line 7: MPI_Init initializes the MPI environment, allowing the program to use MPI functions.
Lines 9-11: The MPI_Comm_size and MPI_Comm_rank functions retrieve the total number of processes (world_size) and the unique rank of the current process (world_rank), respectively.
Line 13: An array processor_name is declared to hold the name of the processor, and name_len will store the length of the processor name.
Line 14: MPI_Get_processor_name retrieves the name of the processor where the current process is executing, storing it in processor_name.
Lines 17-21: A std::stringstream is used to format a "Hello world" message, incorporating the processor name, rank, and total number of processors. The resulting message is then stored in a string message.
Line 22: The message is resized to 100 characters, padding with spaces if necessary, ensuring uniformity in message length for gathering.
Lines 25-28: A gather_buffer vector is initialized to store the messages. This buffer is only allocated on rank 0, ensuring that it holds enough space for all gathered messages.
Line 31: The MPI_Gather function is called to collect messages from all processes. Each process sends its message to the root process (rank 0), which receives them into gather_buffer.
Lines 34-39: Only rank 0 processes the gathered messages. A loop iterates through the gather_buffer, printing each message to the console, ensuring that each message is displayed exactly once.
Line 41: MPI_Finalize cleans up the MPI environment before the program exits, ensuring proper termination of MPI processes.
High-performance computing: It is used extensively in scientific simulations, numerical analysis, and complex computations.
Big data processing: It enables faster processing of large datasets by distributing tasks across multiple nodes.
Research and academia: It is widely utilized in research environments for solving computationally intensive problems.
MPI facilitates efficient parallel computing by enabling communication and synchronization among multiple processes. Its standardized approach, scalability, and portability make it a fundamental tool in various computational domains.
Free Resources