An Incomplete Guide to ICS-ACI for PCHES Users

Systems and Services

Centralized cyberinfrastructure resources for PCHES are hosted on Penn State’s ICS-ACI comprise a computing allocation of 60 “standard-memory” cores and a 100 TB storage allocation, as codified in a 3-year service level agreement (SLA) that ends on 30 April 2020 (additional resources will be available in the near future). Services available to PCHES researchers under this agreement include:

  • ICS-ACI 3.1 batch system (cluster) aci-b:
    • replacement for now-retired ACI 1.0 and legacy Lion-X systems
    • shell access via SSH: ssh -Y username@aci-b.aci.ics.psu.edu
    • time-averaged usage of up to 60 cores (calculated over a 90-day moving window) with a guaranteed response time of one hour or less on the kaf26_a_g_sc_default queue
    • instantaneous (a.k.a. “burst”) usage of up to 240 cores
    • unlimited use of the open queue up to 100 cores (total, not per job) with a maximum wall clock time of 48 hours and a maximum of 100 queued jobs (these jobs may be preempted at any time)
  • ICS-ACI 3.1 interactive system aci-i:
    • for interactive sessions only; similar to the old Hammer system
    • remote desktop via Exceed onDemand (tutorial, download client software)
    • shell access via SSH: ssh -Y username@aci-i.aci.ics.psu.edu
  • RStudio Server:
  • storage:
    • up to 10 GB in your home directory, /storage/home/<username>
    • up to 128 GB in your work directory, /storage/work/<username>
    • up to 100 TB in our group storage pool, /gpfs/group/kaf26/default, shared with all other PCHES users
    • up to 1 million files in your scratch directory, /gpfs/scratch/<username>; this storage resource is intended for temporary files and is not backed up; files residing here for more than 30 days will be automatically deleted
    • all filesystems are accessible from both aci-b and aci-i
    • move files to and from these storage pools via special file transfer node datamgr.aci.ics.psu.edu using scp from the command line or a graphical SFTP client

Getting an Account

The above resources are available to all PCHES researchers, including non-PSU PCHES participants. Contact PCHES Managing Director Robert Nicholas (ren10@psu.edu) for details on how to gain access.

Submitting Jobs on the Batch System

  • qsub -A kaf26_a_g_sc_default <yourrunscript>
  • alternately, add #PBS –A kaf26_a_g_sc_default to your run script
  • to use the open queue, replace kaf26_a_g_sc_default with open

Using the Modules System

The modules system controls which software packages are available in your current environement.

  • to list all available modules: module avail
  • to search available modules: module avail <keyword>
  • to load a specific software package: module load <modulename>
  • to show modules currently loaded in your environment: module list
  • to show paths for executables and libraries associated with a particular module (whether it is loaded or not): module show <modulename>
  • load newer GNU compilers, including GNU Fortran: module load gcc/5.3.1 (system default is version 4.7.7)

In addition to the systemwide modules, PCHES also has its own custom software stack containing additional modules. You can make these modules available with the following command:

module use /gpfs/group/kaf26/default/sw/modules

Specific software packages can be loaded as follows:

  • Anaconda (a full “scientific Python” environment): module load anaconda2 (Python 2.7.x) or module load anaconda3 (Python 3.x)
  • Git Cola (a graphical Git client): module load git-cola/2.10
  • ImageMagick: module load ImageMagick/7.0.5-2
  • Julia: module load julia/0.5.1
  • libGD: module load libgd/2.2.4
  • NCAR Command Language (NCL): module load ncl_ncarg/6.3.0_gcc447
  • ncview: module load ncview/2.1.7
  • Panoply: module load panoply/4.7
  • Perl Data Language (PDL): module load perl/5.24.1
  • PyCharm (a Python IDE): module load pycharm/2016.3.3
  • RStudio Desktop: module load rstudio/0.98.1103
  • uvcdat: module load uvcdat/2.4.1

Note that CDO, GDAL, gnuplot, TeX/LaTeX, NCL, NCO, and Pandoc are also available and are being provided as part of the two Anaconda distributions (see above).

You may also access several additional software packages (including a nice, terminal-based, mouse-enabled file manager and text editor called Midnight Commander) by adding /gpfs/group/kaf26/default/public/sw to your $PATH in .bashrc.

Monitoring the Batch System

The current state of the batch queue may be obtained using the showq command. To show the queue state for just the PCHES allocation, use:

showq –w acct=kaf26_a_g_sc_default

To show the status of the open queue, use:

showq –w acct=open

Getting Help

General information on system usage can be found in the ACI User Guide; a set of ACI video tutorials are also available.

You can obtain professional support for ICS-ACI resources in the following ways:

  • via the iASK web portal
    • weekday responses within 24 hours
    • please CC Robert Nicholas (ren10@psu.edu) on all of your support requests
  • by phone at (814) 865-4275
    • phone support hours are Monday-Friday 9:00am-6:00pm and Saturday-Sunday 1:00pm-4:00pm