Support and tools > ARC-cluster

23/04/2022

The ARC-cluster


The ARC is equipped with a dedicated computer server connected to a high-speed optical fiber network to the outside world, allowing fast data transfer (10 Gbit/sec).

We have 318 TB of disk space and one 13-node cluster (136 cores) dedicated to the ARC, with 64-256 GB of dedicated RAM.

ALMA and CASA users can request access to the server and the disk space by sending an e-mail to arc-support@ira.inaf.it indicating the reason for the request.

User policy


Blade.jpgNew ARC users can access the Italian ARC node computing facilities by requesting a face-2-face visit (ALMA users only, through the ALMA Helpdesk) or by visiting the ARC node in Bologna (for any data-reduction-related issue to be solved in collaboration with the ARC staff).
In both cases, they are requested to send an e-mail to help-desk@alma.inaf.it indicating the reason for the request.

Please notice that the request for a new account for a new requesting user implies that he/she (and/or his/her collaborators) visits the ARC for an introduction on the ARC facilities usage and on issues related to data reduction with CASA both for ALMA or any other telescope. If the request is positively evaluated the visit details will be arranged via e-mail.

The account will guarantee the usage of the facilities and the support for 6 months. Once the account expires the access to the data will be suspended and, after 1 month of quarantine, all data will be removed. Only one gentle reminder will be sent on account of expiration. Extensions of the account duration period could be considered on request (via e-mail). No visit is needed in case of account renewal.

The ARC members support is guaranteed for any ALMA-related issue. For data-reduction-related issues that do not involve ALMA, the support (other than the technical support in the usage of the ARC computing facilities) is limited to the knowledge/experience and availability of the ARC members. The same rules apply also to IRA staff members. IRA collaborators with temporary positions (i.e. students) can have an account for the entire duration of their position.

To ensure a well-balanced load on the cluster nodes please follow instructions about accessing the computer cluster.

Queries can be issued via e-mail to help-desk@alma.inaf.it

Users will be automatically added to the arc-cluster-users@ira.inaf.it mailing list that will be used for any communication from our side.

Accessing the computer cluster


SSH

Once you have obtained an ARC account at IRA, you can access the computer cluster nodes from everywhere through host scheduler.ira.inaf.it. Using graphical applications on the cluster is possible through remote X access. The accessible working nodes are listed in the table below. You can enter a node for interactive work by typing:

ssh -X ''<node>''.ira.inaf.it

Useful tip: by typing ‘hostname’ you can know on which node you are

The only nodes accessible to almaf2f accounts are 19 and the 22.

Here you can find some statistics about resources consumption on the arcblXX nodes.

VPN

You need to install OpenVPN (for IRA staff go to https://wiki.ira.inaf.it/index.php/OpenVPN_VPN)

The VPN (Virtual Private Network) allows you to access IRA internal services in a secure and transparent manner. Once connected, you are assigned an address on the private network 192.168.50.x.

Download an OpenVPN client

Download the configuration:

Procedure:

  • 1 – Start the customer;
  • 2 – From the linux command line, as root user:
$ openvpn --config <name_of_the_downloaded_file>
  • 3- Provide IRA credentials; the username must be indicated without the domain (@ira.inaf.it)

Please turn off VPN on your computer when not needed.

Using ARC storage


You need to change the directory to access your ARC storage on the cluster:

cd /iranet/groups/arc/homesarc/''username'' 

Beware that disks have no redundancy at all, never leave important data on them

Mounting ARC storage on you workstation

  • On IRA workstations ARC home filesystem can be accessed on /iranet/homesarc
  • On your laptop ARC filesystems can be seamlessly accessed with fuse-sshfs:
  • as root, install the package sshfs
# on RedHat/Centos/ScientificLinux
yum install fuse-sshfs
# on Debian/Ubuntu
apt-get install sshfs
  • then, as user
sshfs storage.alma.inaf.it:/iranet/groups/arc/homesarc/yourhome /your/local/mount/point/

By omitting /remote/path you can mount you home directory.Be aware that this method is suboptimal for heavy input/output loads. Running disk-intensive applications directly on the arc cluster will result in a file access speed 10-50 times faster.

Software packages activation and availability – irainit


Irainit is a script that initializes the execution environment for programs installed in directories (typically exported via NFS). Irainit searches for “inifiles” (files named env.ini) in the subdirectories of the directory set by the $irainit_search_dir environment variable (for example, /iranet/soft). Inifiles are text files in “INI” format.

Initialization

Irainit can be initialized by typing the commands:

$ export irainit_search_dir=/iranet/soft
$ . $irainit_search_dir/irainit/irainit_setup-alias.sh

The previous two lines can also be added to the personal .bashrc file, in case the user wants irainit to be automatically activated at login.

By calling:

$ irainit load <software>

irainit load-env is executed, performing the following operations:

  • Sources pre-env.sh
  • Configures the values it recognizes in the inifile
  • Sources post-env.sh
  • Sources the execute.sh file

If the package is already loaded, only the execute.sh file is executed.Inside post-env.sh and execute.sh, the following variables can be used:

  • PYTHON_VERSION (if present)
  • PYTHON3_VERSION (if present)
  • version_basedir

At the end of the procedure, the environment variables PATH, LD_LIBRARY_PATH, MANPATH, and PKG_CONFIG_PATH are updated. A SOFTWARE_NAME_DIR variable is also created to reference the software in other scripts.

Other useful irainit functions

Software and packages available on the ARC cluster can be listed by typing the command:

$ irainit available

The available versions of a specific available package or software can be shown using the option “display”:

$ irainit display <software>

The user can load one specific version of the software among those available using the command:

$ irainit load <software> <version>

Example 1: activating the default version of “casa”:

In this example, we initially activate irainit, then we check the available software, we activate the default version of casapy and finally, we run casa. In this example, we assume the first two commands are not included in the personal .bashrc file and therefore they were not already run at login.

$ export irainit_search_dir=/iranet/soft
$ . $irainit_search_dir/irainit/irainit_setup-alias.sh
$ irainit available
$ irainit load casapy 
$ casa

Example 2: activating a specific version of “casa”:

In this second example, after activating irainit and checking the available software, we also display the available versions of casapy. Then, we activate one specific version of casapy and finally, we run casa. Also in this example, we assume the first two commands are not included in the personal .bashrc file and therefore they were not already run at login.

$ export irainit_search_dir=/iranet/soft
$ . $irainit_search_dir/irainit/irainit_setup-alias.sh
$ irainit available
$ irainit display casapy
$ irainit load casapy 6.1.1-15-pipeline-2020.1.0.40
$ casa

Some of the available packages and software

As said above, software available on the ARC cluster can be listed by typing the command:

$ irainit available

Here below, we report an incomplete list of the available software packages:

Software package setup command launch command notes
CASA irainit load casapy casa data reduction package (link)
Miriad irainit load miriad miriad data reduction package (link)
aips irainit load aips Astronomical Image Processing System (link)
analytic infall irainit load analytic_infall
astron irainit load astron
Coyote library irainit load coyote
fits Viewer irainit load fv (link)
GCC Compiler irainit load gcc (link)
Gildas irainit load gildas (link)
Healpix irainit load healpix (link)
IDL irainit load idl (link)
heasoft irainit load heasoft (link)
QA2 irainit load qa2
Starlink irainit load starlink (link)

Computing Nodes


Name RAM CPU¹ Cores Clock Data Net Work Disk Scratch Disk Sch.² Group³ notes
arcbl01 256G [C] 6/12 3600 10GbE 15T 57G N (a,b)
arcbl02 256G [C] 6/12 3600 10GbE 15T 57G N (a,b)
arcbl03 256G [C] 6/12 3600 10GbE 11T 65G N (a,b)
arcbl04 256G [C] 6/12 3600 10GbE 11T 65G N (a,b)
arcbl05 256G [C] 6/12 3600 10GbE 11T 65G N (a,b)
arcbl06 256G [C] 6/12 3600 10GbE 11T 65G N (a,b) VM
arcbl07 256G [C] 6/12 3600 10GbE 11T 65G N (a,b,c)
arcbl08 256G [C] 6/12 3600 10GbE 11T 65G N (a,b)
arcbl09 64G [B] 4/8 3600 10GbE 15T 57G N (a,b)
arcbl10 64G [B] 4/8 3600 10GbE 11T 57G N (a,b)
arcbl11 64G [B] 4/8 3600 10GbE 11T 57G N (a,b,c,d) NFS server
arcbl12 64G [B] 4/8 3600 10GbE 22T 57G N (a,b) Data-transfer
arcbl13 64G [A] 8/16 3600 1GbE 3,5TB N (a,b)

¹ RAM: [A] AMD Ryzen 7 1800X; [B] Intel Xeon E3-1275 v6; [C] Intel Xeon E5-1650 v4;
² SCH.: Scheduler
³ Group: (a) arc-staff, (b) arc-vlbi, (c) arc-f2f, (d) arc-user; Blades are always dedicated to (a) and (b);

Storage Nodes


Name RAM CPU Cores Clock Data Net RAID Space Storage export
arcnas2 32G Intel Xeon Silver 4108 8/16 1800 10GbE ARC-1883IX-24 91 12x10TB (HGST HUH721010ALE600) RAID6 /lustre/arcfs0/ost3
arcnas3 32G Intel Xeon Silver 4108 8/16 1800 10GbE ARC-1883IX-24 72,8 12x8TB (HGST HUH728080AL5200) RAID6 /lustre/arcfs0/ost0
arcnas4 16G Intel XeonE5-2603v3 6/6 1600 10GbE ARC-1284ML-24 36,4T 12x4TB (WDC WD4000F9YZ-09N20L1) RAID6 /lustre/arcfs0/ost1
91T 12x10TB (ST10000NM0086-2AA101) RAID6 /lustre/arcfs0/ost2
arcnas5 32G Intel Xeon E5-2640 v4 10/20 2400 10GbE Broadcom/LSI MegaRAID SAS-3 3108 255G 2x255GB RAID1