Software

Finding Available Software

OIT manages a set of supported software for use on the Easley Cluster. Software binaries and libraries installed globally by OIT administrators are most often found in the /tools directory, each accompanied with a corresponding Environment Module.

The best way to get started using a piece of software is to take a look at the corresponding module files.

To see what software is currently available on Easley, you can look in the /tools directory or view available software modules …

ls -al /tools

module avail

Each module contains detailed information about the software including a short description, where to obtain documentation, when and how it was installed, and any corresponding changes that are made to your environment when loaded

module show <software_name> [version]

New Software

Any unlicensed open-source software of use to the general user community can be installed by OIT provided that the Easley Cluster meets the software requirements. Please contact OIT support staff or submit a software request at https://aub.ie/hpcsw.

Licensed software must be legally licensed before it can be installed. Since there are many different types of licenses and different vendor definitions of these licenses, any multi-user licensed software installation needs to be coordinated with OIT support staff. Single-user licensed software should be installed in the user’s /home directory.

Custom software should be built in the user’s /home directory. Users are encouraged to maintain and publish their own local module files to configure the environment for their software.

OIT support staff: hpcadmin@auburn.edu Software request: https://aub.ie/hpcsw

Set the Environment

Using Environment Modules

Different programs require that OS environment variables be correctly defined in order to use them. For example, the commonly used $PATH and $LD_LIBRARY_PATH environment variables assign the locations of available software binaries and the existence of shared libraries, respectively. Most software can be properly executed with the correct manipulation of these variables. More complex software also requires the assignment of additional, custom environment variables.

These variables can be set manually or in the user’s profile through the traditional shell methods…

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/my/library

However, this can be cumbersome when a user wants to use different versions of an application, employ a different compiler, or build against different libraries. A convenient solution is to use environment modules.

Environment modules provide the user a way to easily and dynamically change their environment. With simple commands, modules define the environment variables of your shell environment, so that the correct versions of executables are in the path and compiler toolchains can find the correct libraries.

OIT creates modules for globally installed software in the tools directory. Users must find and load the module(s) needed by their application.

Common Module Commands

Module Command

Description

module avail [module]

List all modules available to be loaded

module whatis <module>

Display info about a module

module list

List modules currently loaded

module load <module>

Load a module

module unload <module>

Unload a module

module swap <existing> <new>

Unload a module

module purge

Remove all modules

Creating your Own Modules

Easley uses LMOD as its environment module platform. LMOD modules can be written in either TCL or LUA scripting languages.

For more control and/or customization of your research software environments, you can use LMOD features to create your own collection of modulefiles in your home directory or a shared filesystem location (e.g. shared lab files) to which you have write access.

Example: Create and Enable Personal Modulefiles

  1. Create a directory to hold your custom LUA or TCL modulefiles

mkdir -p ~/.auhpc/modules

  1. Develop and/or copy TCL or LUA modulefiles

# borrow a modulefile from Easley's system modules
cp /cm/shared/modulefiles/.auhpc/auhpc.lmod.ref.lua ~/.auhpc/modules/template.lua

# example customization, point the cc command to a custom location with a shell alias
echo 'set_alias("cc", base .. "/bin/gcc")' >> ~/.auhpc/modules/template.lua

  1. Change the LMOD search path to enable module commands for a custom environment modules.

 # tell LMOD to where to look for modulefiles
 module use ~/.auhpc/modules

 # module use really just mods an environment variable under the hood, good to know ...
 echo $MODULEPATH

 # query LMOD for our custom module by name
 module av template

 --------- /home/<username>/.auhpc/modules ---------
template

 # module show is a quick way validate your module (i.e. to check for errors)
 module show template

  1. The resulting output should dynamically reflect locations in your home directory, which were derived using LMOD/LUA/Shell functions/calls

# <<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>>

Home Directory: /home/<username>
Home Modules: /cm/shared/modulefiles
Home Software: /home/<username>/template
Module Name: template
Module Full Name: template
Category: template
Description:

This is a placeholder for an informative description of the software that this module enables
Change to describe your custom software. NOTE: .. is used to concatenate strings and slash n is a newline.

URL: http://hpc.auburn.edu
Keyword: cluster, system, environment, modules, software, shell
Installed by: hpcadmin@auburn.edu
Install date: 08.08.08

Build notes:

we recommend adding detailed notes/steps used to compile andor install the software
it is helpful to know what modules were loaded at build time: module load gcc/4.8.5
the flags used for the build steps, key environment variables, etc.
./configure --prefix=~/gcc-5.3.0 --with-gmp=/tools/hpclib/gmp-6.2.0
--with-mpfr=/tools/hpclib/mpfr-4.1.0 --with-mpc=/tools/hpclib/mpc-1.2.0

# <<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>><<>>

If module show returns an error, rather than output similar to that above, the error details typically will point you to the exact line and character in your script.

To change the software base path (or prefix, or trunk) to point to a more likely compiler install location, modify the values to include any missing pieces.

Review module show output for any errors or invalid paths and correct them with your preferred method as needed.

Create a custom module structure

Suppose you installed a gcc version 8.8.8 with blazing fast optimizations tailored specifically for your workload in your home directory, and specified /software/compilers/gcc/8.8.8 as the software prefix path.

The software is installed, but to enable and run it, our shell environment will need to be set with any runtime requirements.

To automate and manage these environment settings for a set of software, create a corresponding modulefile for each software installation.

A directory structure can be defined to group software versions by software name, and optionally an additional directory to categorize software types

# the compilers subdirectory allows us to organize our compilers by family, here we just have gcc
mkdir -p ~/.auhpc/modules/compilers/gcc

Now that we have staged a directory structure to logically organize our expected compiler software, let’s go ahead and create modulefiles for the version(s) we expect to use. In addition to gcc 8.8.8, we are reproducing some experiments that have a specific compiler requirement of gcc 7.6.5. We can use the generic “compiler” module we created earlier as a template for these specific versions.

cp ~/.auhpc/modules/template.lua ~/.auhpc/modules/compilers/gcc/8.8.8.lua
cp ~/.auhpc/modules/template.lua ~/.auhpc/modules/compilers/gcc/7.6.5.lua

The resulting structure should look something like ….

tree

/home/<username>/.auhpc/modules/compilers
└── gcc
    ├── 7.6.5.lua
    └── 8.8.8.lua

We should now have compiler modules that correlate to two different versions of gcc. Let’s see if LMOD is aware of them …

module av gcc
------------------------------------------ /home/<username>/.auhpc/modules -----------------------------------
compilers/gcc/7.6.5    compilers/gcc/8.4.3    compilers/gcc/8.8.8 (D)

Our structure and naming convention were recognized by LMOD.

Check each new module with module show and correct any errors or invalidities.

More on LMOD modules

Existing system modules in /cm/shared/modulefiles/ are readable and can be reviewed as additional examples.

For more advanced/in-depth module development, take a look at these LMOD docs …

Modules homepage: https://lmod.readthedocs.io/en/latest/ Modules manual: https://lmod.readthedocs.io/en/latest/010_user.html Repository: https://github.com/TACC/Lmod

Compile Software

You are free to compile and run any custom software needed for your research within your home directory, provided that it is from a reputable, trustworthy source and adheres to the acceptable use policies described above.

Easley provides a variety of development tools that you can use to build software, including several versions of gcc, make, make, and openmpi.

Your software will most likely provide a README or INSTALL file in the source distribution that should provide hints on what tools you will need and what procedure to follow. Alternatively, check the software’s web site or other Internet sources if you encounter problems.

For parallelized software, you will want to compile with your software’s recommended version of openmpi, i.e…

module avail openmpi
module load openmpi/1.8.3
./configure --prefix=/home/software/build
make
make install