First MPI Program

First MPI Program: Hello World

The usual first program is a "hello world" one, which in C takes the stripped down form:

#include 
#include 
int main(int argc, char **argv) {
    printf("Hello world from process %d\n", myrank);
    return 0;}

Of course life cannot be so simple in the parallel programming world, so here is the first (but wrong) multiprocessor version:

#include <stdio.h>
#include <stdlib.h>
#include "mpi.h"
int main(int argc, char **argv) {
    int myrank, p;
    MPI_Init(&argc, &argv);
    MPI_Comm_rank(MPI_COMM_WORLD, &myrank);
    MPI_Comm_size(MPI_COMM_WORLD, &p);
    printf("Hello world from process %d\n", myrank);
    MPI_Finalize();
    return 0;}

First of all, the program has to be compiled with all the MPI libraries - a makefile will be provided for that. Secondly, it cannot be run simply by typing the executable name. For example, here is can happen when that is done:

[scrap04.cs.indiana.edu:84] ./howdy
[0]  Aborting program ! Could not create p4 procgroup.  Possible missing file
or program started without mpirun.

Instead, it must be run through a script, typically named mpirun which starts up a certain number of processes, all running the same program. For example, to run 4 processes of the above program, you would execute

   mpirun -np 4 ./howdy

When the job runs, what happens is:

Four processes are started, and some kind of communication method is established between them. For NOWs this can be opening up sockets (TCP/IP) between the processes, while for a shared memory machine it involves setting aside some part of shared memory for buffering messages.
Every process has its own data. So each of the four processes has its own copy of myrank and p.
Everybody gets their own "myrank", which will be between 0 and 3.

Side note: MPI has a standard start-up mechanism, mpi-exec. Unfortunately mpirun is embedded in the folklore, and mpi-exec can seem arcane in comparison. So most people continue to provide/use mpirun which is a simplified wrapper for mpi-exec.

Typical output from the program looks like:

[scrap04.cs.indiana.edu:102] mpirun -np 4 howdy
Hello world from process 0
Hello world from process 2
Hello world from process 3
Hello world from process 1

Note that the output is not necessarily in rank order - but why should we think that the one with rank 1 will print before the one with rank 2? The above program is actually wrong in that there is no guarantee of what printf() does with multiple processes. Does each get connected to your terminal? Do they each get their own file descriptor? In is possible that one output line will appear in the middle of another. Following is a safer way to do the howdy program:

Second version of Hello World

#include <stdio.h>
#include <stdlib.h>
#include "mpi.h"
int main(int argc, char **argv) {
    int myrank;
    int src, dest, p;
    int tag = 1;
    char msg[256];
    MPI_Status status;

    MPI_Init(&argc, &argv);
    MPI_Comm_rank(MPI_COMM_WORLD, &ampmyrank);
    MPI_Comm_size(MPI_COMM_WORLD, &ampp);
 
    if (myrank != 0) {
      sprintf(msg, "Hello world from process %d\n", myrank);
      dest = 0;
      MPI_Send(msg, strlen(msg)+1, MPI_CHAR, dest, tag, MPI_COMM_WORLD);
    }
    else {
      printf("Hello world from process %d\n", myrank);
      for (src = 1; src < p; src++) {
        MPI_Recv(msg, 256, MPI_CHAR, src, tag, MPI_COMM_WORLD, &status);
        printf("%s", msg);}
    }
    MPI_Finalize();
    return 0;}

What this program does is

Each process gets its rank as before
Every process except the one with rank 0 copies its greeting into a character array buffer called msg. Keep in mind that there are p different arrays called msg - one for each process.
Each process other than the one with rank 0 sends a message to the one with rank 0 (by specifying dest = 0).
The process with rank 0 does not bundle up such a message. Instead, it prints its own greeting, then one-by-one receives the messages from the other processes and prints them out. It receives them into its character array called msg.

This shows the basics of how to get a MIMD code from a SPMD model: run the exact same program, but there is a a branch depending on the process's rank. Now exactly one process is doing I/O, and we are guaranteed that the messages will be printed in rank order.

Last Modified: Thu 08 Feb 2018, 07:15 AM