Parallel Programming with MPI and OpenMP (Sep 02 09)

Last week, I attended the course "Multi-Core Faculty Training" given by Intel at the Department of Computer Science of the University of the Philippines - Diliman. Our trainors were Rama Malladi and Vipin Kumar of Intel-India. The participants were computer science faculty of various universities in the Philippines. The training was about programming threads on Intel multi-processor computers, called Intel multi-core computers, without the need to use the POSIX threads application programming interface (API) [1]. In particular, we used the Intel Threading Building Blocks [2], which is actually an implementation of the OpenMP [3] standard. The laboratory exercises were done on Intel Core2 Quad (four-processor) computers with one MB ram, running 64-bit Windows XP Professional. We used Microsoft Visual Studio with Intel C/C++ compilers, Intel VTune Performance Analyzer, Intel Thread Checker, and Intel Thread Profiler. It was a pretty interesting course, made slightly less interesting by the fact that we used Windows when professional scientific practice on multi-threading almost always happens on Linux scientific clusters. This is a slight oversight on the part of Intel, which is, after all, a closed-source commercial software vendor like Microsoft.

After the course, I decided to find out what similar tools are available for Linux, and I found that all recent versions of the GNU compiler collection [4] already support OpenMP, including the most recent gcc-4.4 C/C++ compiler. This means that all the friendly tools like Eclipse and similar IDEs automatically support OpenMP. I also found a beautiful book, "Parallel Programming in C with MPI and OpenMP" by Michael Quinn [5]. MPI stands for message passing interface [6], and it is a tool for dividing up a big program into smaller pieces for execution in several separate computers connected by fast network switches. OpenMP, on the other hand, is a tool for executing a program as several threads in one multi-core computer, without the programmer worrying about API calls like pthread_create() and pthread_join(). So MPI and OpenMP are the tools that we need for programming clusters of Intel Core2 Duo/Quad/etc computers.

Today, even the poor computing science researcher can construct using commodity products, compute clusters of cheap Intel multi-core computers, and pretend that they are gods computing with very expensive Cray super computers. Isn't life beautiful?

End notes:

[1] On a Linux box, type the command "man pthreads" to get an idea of the POSIX threads API.

[2] The Intel TBB documentation can be found here: http://software.intel.com/en-us/multi-core/

[3] The official website of OpenMP is http://www.openMP.org

[4] The GNU compiler collection is supplied as standard on most Linux distributions. Its documentation is available froma the website http://directory.fsf.org/project/gcc/

[5] Michael Quinn, "Parallel Programming in C with MPI and OpenMP", McGraw-Hill, NY, 2004, xiv + 529 pages

[6] Documentation for MPI is available from http://www.mpi-forum.org/docs/