MPI is a standard for communication among a group of distributed (or local) processes. It includes routines to send and receive data, communicate collectively, and other more complex tasks.

The standard provides an API for C and Fortran, but bindings to various other languages also exist.

Collectives operations are MPI calls designed communicate the processes pointed out by a communicator in a single operation or to perform a synchronization among them. These are often used to calculate one or more values based on data contributed by other processes or to distribute or collect data from all other processes.

Note that all the processes in the communicator should invoke the same collective operations in order, otherwise the application would block.

MPI is a standard, not a programming library. There are many implementations of the standard. The most common open source ones are MPICH and Open MPI. There are many derivatives of these two libraries that are either open source or commercial (or both).

It's important to know which implementation you have because the way you compile or run your program might change subtly.

MPI needs to add extra libraries and include directories to your compilation line when compiling your program. Rather than tracking all of them yourself, you can usually use one of the compiler wrappers.