Guide to Mount Moran

Getting Started on Mount Moran for first-time users:

First of all: Mount Moran is free to use.
Please see the Getting Connected documentation for information on getting an account.
Students and non-tenured researchers need to have a faculty member sponsoring their research. Non-UW collaborators will also need to request a Special Account.

Logging in
Once your account has been set up, an automated email with a temporary password will be sent to your UWYO account. You can connect to the cluster from a Unix system by typing the following command into a terminal window. Windows users will first need to install Puty or similar software.

ssh <username>@mtmoran.uwyo.edu

When you successfully log in, you will land in your $HOME directory.

Should you need to reset your password, please see the instructions here: Reset Password


Computing Environment
The default setup gives each project a project directory and each user a home directory within the project directory. The project directory is shared among the members of that project. Local scratch (950 Gb) is provided on each node.

Environment Variables
The default shell is /bin/bash. Users can find additional pre-set environment variables by using the env command. These variables can be overridden using dot files (hidden files whose titles begin with a "."). To display a listing of these hidden dot files, use the -a flag on the ls command as follows:

ls -a

.bash_profile: a start-up shell script for Bourne Again shell (bash) users. Users may modify this file to add/override any environment variables or conventions that were established by the system. If you need to add something to your PATH, do it here.

.bashrc: a start-up shell script for bash users that is executed each time a new bash is invoked. Users may modify this file to establish variables and aliases.

Software Modules
Mount Moran uses the Lmod Modules package to provide greater flexibility and reduce environment maintenance. Modules supplies a unified command structure that allows you to dynamically modify your user environment by adding, changing, or removing an application’s environmental information.
The hierarchy format helps when multiple compilers with multiple versions and multiple packages with multiple version.

Lmod ARCC setup

There are three general categories that ARCC systems depend on. The first level is “System” modules, these modules include the job scheduler, ARCC provided utilities, and other system commands such as software configuration packages (GNU autoconf, Cmake, etc.). The second category is compiler independent packages, meaning that they can be used with any compiler chain or without one. Usually these modules contain packages compiled with the system compiler, packages which are pre-compiled binaries or that ARCC has compiled the software such that a compiler tool chain is not required when using the application. You will often find packages such as Octave, NCL, Python, and Java in these modules. The third category is the compiler dependant modules. Generally, a user can only have one compiler module loaded at one time. This is for user environment protection so that libraries compiled with different compilers are not necessarily used together. An example would be attempting to use a BLAS library compiled with gfortran with a fortran file attempting to be compiled with PGI fortran. Therefore loading a compiler first will then allow the loading of additional modules that are dependent on the compiler such as an MPI stack or HDF5 package.

Lmod Commands

Lmod breaks things up into a set of commands, not a single command with options. However, all Lmod commands are preceded by the ‘module’ command. For example, to list all the currently loaded modules, one would execute the following command:

$ module list

The above command shows a list of all the currently loaded modules (including deactivated modules). Example output would look like the following:
Currently Loaded Modules:
1) moab/7.2.6 2) torque/4.2.6 3) arcc/1.0.0
Getting module command help is easy when using the 'help' option. Executing the following command will provide a help section on utilizing the modules command:

$ module help

To view which modules are immediately available for loading, execute the avail command:

$ module avail

The output will show which modules are directly available to load, without loading any additional modules. However, if a desired module is not listed here, then a broader search of the modules must be done. Therefore, using the 'spider' command will allow for a full search of the system-wide available modules.

$ module spider

More gratifying searches can be performed using additional parameters. The module commands that support regular expressions include: list, avail, spider, and keyword. Thus, determining which hdf5 packages on the system could be done with the following commands.
system-wide:

$ module spider hdf5

or

$ module keyword hdf5

To check for specific module in the default heirarchy, the spider command is useful:

 module spider <package>

To test if a particular package is immediately available:

module avail <package>

If no package is given, it will list all available modules. This command is useful to see which versions of particular software are installed.

Adding modules to the environment is easy. On ARCC systems, a compiler is usually required to be loaded before any compiling can be accomplished. Therefore, loading the GNU Compiler Collection version 4.8.2 can be done with the following command:

$ module load gnu/4.8.2

If the command completes successfully, then there will be no messages printed to STDOUT or STDERR.
Multiple modules can be loaded with a single command such as:

$ module load gnu/4.8.2 hdf5 netcdf

Modules can be unloaded from the environment just as easily:

$ module unload gnu/4.8.2

All modules can be removed from the environment with a single command:

$ module purge

To reload all currently loaded modules:

$ module update

Often times, a user will need to swap between compilers. This is effectively done with the 'swap' command. Consider that I want to swap from GNU compilers to Intel compilers and have the MKL module loaded. Issuing the following command should swap the compiler sets, and reload the MKL module, unless the module is not found, in which case it will be deactivated (stripped from the environment)

$ module swap gnu/4.8.2 intel/14.0.0

Example output would be the following:

Due to MODULEPATH changes the following have been reloaded:
1) mkl/11.1

LMOD Advanced Usage
Lmod has some advanced features over other module implementations that have been researched. One of those features is the ability to save a collection of modules. This is accomplished by using the save command. The method of doing this is to load all the modules that are desired and then executing the following command:

$ module save

This will save the current set of modules to the 'default' name. If multiple collections are required, a name can be given to the list as a unique identifier. Consider that a user is running several codes which have been compiled with different compilers and software packages. The first package may require gnu/4.8.2, hdf5, and openblas. Therefore, loading the modules:

$ module load gnu/4.8.2 hdf5 openblas

then saving it as a collection named “pkgA”:

$ module save pkgA

Then to create the second collection named “pkgB”, which requires intel/14.0.0, impi, hdf5-mpi

$ module unload gnu/4.8.2 hdf5 openblas
$ module load intel/14.0.0 impi hdf5-mpi
$ module save pkgB

Then when scheduling jobs to run, a collection of modules can be loaded with a more simplistic command:

$ module restore pkgA

to load the packages required in pkgA.
If several collections have been saved, then a list of these collections can be viewed by executing the savelist command:

$ module savelist

If at any point the system default modules are needed, the module restore system command can be used:

$ module restore system

To swap package B for package A. This is useful to change PE modules to switch compilers and between versions of other modules.

module swap <package A> <package B>

To check for specific module in the default heirarchy, the spider command is useful:

 module spider <package>

To see details about a particular package

module show <package>

This gives you the information concerning the installed software. One will see the setenv commands that will modify your environment if you decide to load that module. This is useful for two major reasons. First, you can make sure what executable you like to run- there might be a small difference of the executable name on Nautilus versus another machine. Here, you can perform a ls on the outputted bin directory. Secondly, some environment variables could be introduced. For instance, the FFTW module will provide an environment variable that points to the library and include directories- simply include this variable into your makefile versus including the full path.


Software:
Software on ARCC supported high performance computing systems includes vendor supplied operating systems and programming environments. In addition, we offer a number of libraries and applications.

Applications:
Software applications installed on ARCC supported HPC systems. Read More »
Compilers:
ARCC supports the GCC, Intel, and PGI compilers. Read More »
Programming Libraries:
Various programming libraries are available on ARCC supported HPC system. Read More »
Debugging and Profiling:
Debugging and profiling tools available to ARCC users. Read More »

Running jobs:
(For general information about how jobs are scheduled, please refer to Mount Moran's Job Scheduling Policy.)

In this section you will learn how to submit a job to the queue, use the 'module' command, and monitor your job in the queue. Also find information on OpenMP and MPI runtime settings as well as general application tuning options.

To run a job on Mount Moran, you will need the following:

  • to have an Account that is associated with a Project
  • to have an active SSH connection to the cluster
  • to be aware of QOS reservations and how to use them

ARCC resources all utilize job or batch scheduler software to monitor and run code, ideally achieving optimum utilization of that resource, including processors, memory, and disk space. Mount Moran currently uses SLURM. Please see the Running Jobs guide for examples on how to submit a job using SLURM.

Status:
View
real-time status information on Mount Moran.