The University of Arizona
    For questions, please open a UAService ticket and assign to the Tools Team.
Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Column
width40%

Image Removed

Column
width60%

Table of Contents

Policies


Page Banner
imagehttps://public.confluence.arizona.edu/download/attachments/86409265/CD.jpeg?api=v2
actionTitleRequest Software Install
actionUrlhttps://uarizona.service-now.com/sp?id=sc_cat_item&sys_id=102d93a71ba720107947edf1604bcb55&sysparm_category=84d3d1acdbc8f4109627d90d6896191f
titleSoftware



Panel
borderColor#9c9fb5
bgColor#fafafe
borderStylesolid

Overview

The policies regarding the installation of software

is

are on this page.  In general, scientific software is installed as requested with the caveats noted in that section.

CentOS6
Ocelote is built on CentOS 7 which is the foundation for all compilers, libraries, and applications.  Some workloads need CentOS 7 for its newer kernel.  A number of GPU nodes have been reprovisioned with a CentOS7 image. To access one of these nodes, modify the PBS select statement to include os7=True. Note the upper case T
Code Block
select=1:ncpus=28:mem=224gb:np100s=1:os7=True

Installed software

There are over 100 software applications installed as modules so you should look there before submitting an installation request. As an alternative, you are always welcome to install your own software or other software in your file space.

Puma, Ocelote, and ElGato are built with CentOS 7 along with the system libraries, compilers and utilities that are needed for HPC operations. 



Panel
borderColor#9c9fb5
bgColor#fafafe
titleColor#fcfcfc
titleBGColor#021D61
borderStylesolid
titleContents

Table of Contents
maxLevel2



Panel
borderColor#9c9fb5
bgColor#fafafe
borderStylesolid

Available Software

Software packages are available as modules and are accessible from the compute nodes of any of our three clusters. They are not available on the login nodes so you will need to be in an interactive session to access them. A list of installed software is kept

at .

Modules

Many popular software packages are installed and available as modules.  There may be several versions of a package available

, but may not be as current as using the module avail command as shown below.

Module Commands

Tip

If multiple versions of software are available on the system, the newest is made the default. This means loading a module without specifying the version will select the most recent. We strongly recommend including version information in your module statements. This ensures that you maintain a consistent environment for your analyses in the event of a software upgrade.

To see, access, and get information on individual software packages available on the system, use the module commands detailed below. If you have trouble accessing the module command, see our FAQ section.

Module CommandDescription
module

$ module avail

 Display all the software and versions installed on the system
 module

$ module avail module_name

Display all installed versions of the software "module_name" 

$ module list

Display the software you have loaded in your environment
 module load modulename

$ module what-is module_name

Displays some descriptive information about a specific module

$ module load module_name

Load a software module in your environment
 module purge Unload all the software modules Unload a specific software package
$ module unload module_nameUnload a specific software package from your environment
 module unload modulename

$ module purge

 Unload all the software modules from your environment
 module

$ module help

 Display a help menu for the module command

Compilers

Puma, Ocelote, and El Gato all run CentOS7 and have the following compilers available:

CompilerVersionModule Command
Intel 2020.1$ module load intel/2020.1
Intel2020.4$ module load intel/2020.4
gcc5.4.0

$ module load gnu/5.4.0

gcc7.3.0$ module load gnu7/7.3.0
gcc8.3.0$ module load gnu8/8.3.0
This one is loaded by default




Panel
borderColor#9c9fb5
bgColor#fafafe
borderStylesolid

Common Software Packages

Children Display
styleh4
sorttitle
reversetrue
excerptTypesimple



Panel
borderColor#9c9fb5
bgColor#fafafe
borderStylesolid

Installing additional software

To submit a request to have software installed on the

UA

UArizona HPC systems, use

the https://it.arizona.edu/hpc-software-install-requestThere

. There is no expected

timeframe

time frame for how long it takes to install software

.  There

, there are many variables that determine this. If you haven't heard back in a week, it is reasonable for you to follow up

with hpc-consult@list.arizona.edu

with a support ticket

You

can

may also install software packages into

your home directories with

the space that is allocated to you with your HPC account.  However, you cannot install software that requires root permission

,

or use a method like "yum install" that accesses system paths.

Follow this  link for detailed

For information on

how to install your own software

Using and Customizing Perl

Follow this link for more information on using Perl.

Using and Customizing Python

Follow this link for more information on using Python.

Using and Customizing R Packages

Follow this link for more information on using R.

Using Matlab

Follow this link for more information on using Matlab

There are three ways to run Matlab

  1. The command line version using modules.  This is the most common as you will typically submit a job using PBS.
  2. Graphical mode using the Ocelote Desktop of OnDemand
  3. Using the Matlab integration with Python in a Jupyter notebook. 

Here are the details for method 1.

MATLAB performs its own hardware discovery and it might try to access all the cores and the memory of the node even if the full node wasn't allocated. That will result in scheduler killing the job. To prevent that the full Ocelote node of 28 cores and 168GB of memory should be allocated to run a MATLAB job.

Like any other application, MATLAB has to be loaded as a module before you can use it. To see all the installed versions of the MATLAB use command module avail matlab.

The typical procedure for performing calculations on UA HPC systems is to run your program non-interactively on compute nodes. The easiest way to run MATLAB non-interactively is to use input/output redirection. This method uses Linux operators < and > to point MATLAB to the input file and tell where to write the output (see the example script). The other method is to invoke MATLAB from the PBS script and execute specified statement using -r option. For details please refer to the manual page of matlab command:

https://www.mathworks.com/help/matlab/ref/matlablinux.html

Code Block
#!/bin/bash
#PBS -N job_name
#PBS -W group_list=group_name
#PBS -q standard
#PBS -l select=1:ncpus=28:mem=168gb:pcmem=6gb
#PBS -l walltime=01:00:00
#PBS -l cput=28:00:00

cd $PBS_O_WORKDIR

module load matlab

matlab -nodisplay -nosplash < script_name.m > output.txt

The options -nodisplay and -nosplash in the example prevent MATLAB from opening elements of GUI. To view the full list of options for matlab command load the MATLAB module and type matlab -h in Linux prompt, or use the link above to the manual page on MathWorks website."

Using Anaconda

Anaconda is available as a module and comes with Glueviz, JupyterLab, Jupyter Notebook, Spyder, Orange and RStudio.  It also supports the conda commands for building a personal environment.

More details are at this page.

Spark

Apache Spark is a fast and general-purpose cluster computing system.  However it has not been installed for support in a multi-node environment as yet.  That functionality is planned for the future.  It provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution graphs. It also supports a rich set of higher-level tools including Spark SQL for SQL and structured data processing, MLlib for machine learning, GraphX for graph processing, and Spark Streaming.

There are some simple tests here.

Compilers

  • El Gato  

    ElGato is now running CentOS 7.  The compilers available are:
    • Intel/2019.3: "module load intel"

    • GCC 5.4.0: "module load gnu"
    • GCC 8.3.0: "module load gnu8"
    • LLVM 5.0.1" "module load llvm5" 
Ocelote
  • GCC is available without loading a module.  gcc --version shows that it is 6.1.0.  If you also need the GNU Scientific Libraries (gsl), that is available using module load gsl which will get you version 2.1
  • The Intel Compiler suite 2016 is available in both 32 and 64 bit versions.  The math kernel libraries (mkl) are provided as separate modules, also in 32 and 64 bit versions.
  • The Intel Compiler 2019 is available and is no longer separated like 2016.  So there are not separate modules for mkl, mpi, etc
  • PGI compilers 2018 (Portland Group) are installed as a module.  We have provided these in particular for their OpenACC support. We do not build software with the PGI compilers.
  • MPI.  There are more choices now so pay attention.
    • There are standard Red Hat versions of mpich, mvapich, mvapich2 and openmpi.  Some extra options are invoked when you load one - use module avail to see the specific name.
    • The same four MPI are available with more detailed options for gcc, intel and open64.  Again use module avail for the appropriate choices.
  • AVX2  Ocelote has Intel V3 Haswell processors.  A key feature of these is AVX2. Read this Intel document.

    For AVX2 support, compile with the -xHOST option. Note that -xHOST alone does not enable aggressive optimization, so compilation with -O3 is also suggested. The -fast flag invokes -xHOST, but should be avoided since it also turns on interprocedural optimization (-ipo), which may cause problems in some instances.

    For GNU compilers, AVX support is only available in version 4.6 or later.  For AVX support, compile with -mavx

    Image Removed

    Image Removed

    Image Removed

    Image Removed

    Image Removed

    Image Removed

    Image Removed

    installing software locally, see our online guide for an example.