Parallel Programs

CRMDA facilitates usage of the advanced computing facility (ACF) cluster computing system. We consult with research projects and prepare training exercises and workshops.  We have worked on parallel (multi-computer) programs that are written in R, Mplus, and OpenBUGS. In the process of learning how to write research computing programs, we usually develop smaller "working examples," some of which are later used in training sessions.

The majority of our parallel research programs are written with R (http://www.r-project.org), a software framework for which there has been steady development of new packages. Researchers around the world have proposed various strategies to distribute jobs among compute nodes. Because R is an open source project, it is possible to learn how these new programs work and experiment with variations.


A Collection of Working Examples

The high performance computing examples are hosted at:

https://gitlab.crmda.ku.edu/crmda/hpcexample​

The Web server allows visitors a chance to browse the contents of each project.

We suggest that new users should "clone" a copy of the repository in their accounts on the cluster. The examples we provide do run without revision. Examples include README files as well as example output that we have obtained.  After testing our working examples, then users can make changes to implement their own projects. Computing novices are especially strongly urged to clone our repository and run the examples.  Running our examples, and inspecting the output, is the best way to learn how the moving parts fit together.

There are instructions for downloading the repository on the main README file of hpcexample. We receive quite a few panicked emails from users who don't read the instructions.  When they try to create a clone using the Web-based protocol "HTTPS", the following error occurs:

$ git clone https://gitlab.crmda.ku.edu/crmda/hpcexample.git
Cloning into 'hpcexample'...
fatal: unable to access 'https://gitlab.crmda.ku.edu/crmda/hpcexample.git/': 
server certificate verification failed. 
CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none

The following "work around" is needed on some computers:

$ GIT_SSL_NO_VERIFY=true git clone https://gitlab.crmda.ku.edu/crmda/hpcexample.git
Cloning into 'hpcexample'...
remote: Counting objects: 1248, done.
remote: Compressing objects: 100% (26/26), done.
remote: Total 1248 (delta 11), reused 0 (delta 0)
Receiving objects: 100% (1248/1248), 2.25 MiB | 0 bytes/s, done.
Resolving deltas: 100% (699/699), done.
Checking connectivity... done.

That creates directory called "hpcexample" that has a complete snapshot of the Git repository.

That command can be used on any networked system that is inside the KU network, including the login nodes on the Advanced Computing Facility.

The clone that is created by HTTPS can be edited and used in the user's account.  The revisions within those directories, however, can never be contributed back to the hpcexample repository.  For developers who intend to contribute to the examples, it will be necessary to use the SSH protocol.  That is described in our notes on using Gitlab.


Additional Cluste"R" Readings: Links and Documents

Click Here! We are cultivating a small garden of additional especially useful R documentation


CRMDA Calendar

Like us on Facebook
One of 34 U.S. public institutions in the prestigious Association of American Universities
Nearly $290 million in financial aid annually
44 nationally ranked graduate programs.
—U.S. News & World Report
Top 50 nationwide for size of library collection.
—ALA
23rd nationwide for service to veterans —"Best for Vets," Military Times
Equity & Diversity Calendar

KU Today