The ARC-cluster
The ARC is equipped with a dedicated computer server connected to a high-speed optical fiber network to the outside world, allowing fast data transfer (10 Gbit/sec).
We have 318 TB of disk space and one 13-node cluster (136 cores) dedicated to the ARC, with 64-256 GB of dedicated RAM.
ALMA and CASA users can request access to the server and the disk space by sending an e-mail to arc-support@ira.inaf.it indicating the reason for the request.
User policy
New ARC users can access the Italian ARC node computing facilities by requesting a face-2-face visit (ALMA users only, through the ALMA Helpdesk) or by visiting the ARC node in Bologna (for any data-reduction-related issue to be solved in collaboration with the ARC staff).
In both cases, they are requested to send an e-mail to help-desk@alma.inaf.it indicating the reason for the request.
Please notice that the request for a new account for a new requesting user implies that he/she (and/or his/her collaborators) visits the ARC for an introduction on the ARC facilities usage and on issues related to data reduction with CASA both for ALMA or any other telescope. If the request is positively evaluated the visit details will be arranged via e-mail.
The account will guarantee the usage of the facilities and the support for 6 months. Once the account expires the access to the data will be suspended and, after 1 month of quarantine, all data will be removed. Only one gentle reminder will be sent on account of expiration. Extensions of the account duration period could be considered on request (via e-mail). No visit is needed in case of account renewal.
The ARC members support is guaranteed for any ALMA-related issue. For data-reduction-related issues that do not involve ALMA, the support (other than the technical support in the usage of the ARC computing facilities) is limited to the knowledge/experience and availability of the ARC members. The same rules apply also to IRA staff members. IRA collaborators with temporary positions (i.e. students) can have an account for the entire duration of their position.
To ensure a well-balanced load on the cluster nodes please follow instructions about accessing the computer cluster.
Queries can be issued via e-mail to help-desk@alma.inaf.it
Users will be automatically added to the arc-cluster-users@ira.inaf.it mailing list that will be used for any communication from our side.
Accessing the computer cluster
SSH
Once you have obtained an ARC account at IRA, you can access the computer cluster nodes from everywhere through host scheduler.ira.inaf.it. For example, for almauser0:
ssh -X -P 22 almauser0@scheduler.ira.inaf.it
Using graphical applications on the cluster is possible through remote X access. The accessible working nodes are listed in the table below. You can enter a node for interactive work by typing:
ssh -X ''<node>''.ira.inaf.it
Useful tip: by typing ‘hostname’ you can know on which node you are
See Table at the end of this page to know the nodes accessible to different users.
VPN
You need to install OpenVPN (for IRA staff go to https://wiki.ira.inaf.it/index.php/OpenVPN_VPN)
The VPN (Virtual Private Network) allows you to access IRA internal services in a secure and transparent manner. Once connected, you are assigned an address on the private network 192.168.50.x.
Download an OpenVPN client
- Debian GNU/Linux text: openvpn-auth-ldap and openvpn package
- Mac OSX: tunnelblick
- Windows: OpenVPN Guide and Download
Download the configuration:
- VPN to connect to IRA networks, leaving navigation to other sites unchanged (https://www.ira.inaf.it/vpn/ira-nogw.ovpn)
- VPN for browsing to all sites. Use this configuration only if strictly necessary (https://www.ira.inaf.it/vpn/ira-gw.ovpn)
- VPN for VoIP phone use (https://www.ira.inaf.it/vpn/ira-nogw-tel.ovpn)
Procedure:
- 1 – Start the customer;
- 2 – From the linux command line, as root user:
$ openvpn --config <name_of_the_downloaded_file>
- 3- Provide IRA credentials; the username must be indicated without the domain (@ira.inaf.it)
Please turn off VPN on your computer when not needed.
Using ARC storage
You need to change the directory to access your ARC storage on the cluster:
cd /iranet/groups/arc/homesarc/''username''
Beware that disks have no redundancy at all, never leave important data on them
Mounting ARC storage on you workstation
- On IRA workstations ARC home filesystem can be accessed on /iranet/homesarc
- On your laptop ARC filesystems can be seamlessly accessed with fuse-sshfs:
- as root, install the package sshfs
# on RedHat/Centos/ScientificLinux
yum install fuse-sshfs
# on Debian/Ubuntu
apt-get install sshfs
- then, as user
sshfs storage.alma.inaf.it:/iranet/groups/arc/homesarc/yourhome /your/local/mount/point/
By omitting /remote/path you can mount you home directory.Be aware that this method is suboptimal for heavy input/output loads. Running disk-intensive applications directly on the arc cluster will result in a file access speed 10-50 times faster.
Software packages activation and availability – irainit
Irainit is a script that initializes the execution environment for programs installed in directories (typically exported via NFS). Irainit searches for “inifiles” (files named env.ini) in the subdirectories of the directory set by the $irainit_search_dir environment variable (for example, /iranet/soft). Inifiles are text files in “INI” format.
Initialization
Irainit can be initialized by typing the commands:
$ export irainit_search_dir=/iranet/soft
$ . $irainit_search_dir/irainit/irainit_setup-alias.sh
The previous two lines can also be added to the personal .bashrc file, in case the user wants irainit to be automatically activated at login.
By calling:
$ irainit load <software>
irainit load-env is executed, performing the following operations:
- Sources pre-env.sh
- Configures the values it recognizes in the inifile
- Sources post-env.sh
- Sources the execute.sh file
If the package is already loaded, only the execute.sh file is executed.Inside post-env.sh and execute.sh, the following variables can be used:
- PYTHON_VERSION (if present)
- PYTHON3_VERSION (if present)
- version_basedir
At the end of the procedure, the environment variables PATH, LD_LIBRARY_PATH, MANPATH, and PKG_CONFIG_PATH are updated. A SOFTWARE_NAME_DIR variable is also created to reference the software in other scripts.
Other useful irainit functions
Software and packages available on the ARC cluster can be listed by typing the command:
$ irainit available
The available versions of a specific available package or software can be shown using the option “display”:
$ irainit display <software>
The user can load one specific version of the software among those available using the command:
$ irainit load <software> <version>
Example 1: activating the default version of “casa”:
In this example, we initially activate irainit, then we check the available software, we activate the default version of casapy and finally, we run casa. In this example, we assume the first two commands are not included in the personal .bashrc file and therefore they were not already run at login.
$ export irainit_search_dir=/iranet/soft $ . $irainit_search_dir/irainit/ irainit_setup-alias.sh $ irainit available $ irainit load casapy $ casa
Example 2: activating a specific version of “casa”:
In this second example, after activating irainit and checking the available software, we also display the available versions of casapy. Then, we activate one specific version of casapy and finally, we run casa. Also in this example, we assume the first two commands are not included in the personal .bashrc file and therefore they were not already run at login.
$ export irainit_search_dir=/iranet/soft $ . $irainit_search_dir/irainit/ irainit_setup-alias.sh $ irainit available $ irainit display casapy $ irainit load casapy 6.1.1-15-pipeline-2020.1.0.40 $ casa
Some of the available packages and software
As said above, software available on the ARC cluster can be listed by typing the command:
$ irainit available
Here below, we report an incomplete list of the available software packages:
Software package | setup command | launch command | notes |
---|---|---|---|
CASA | irainit load casapy | casa | data reduction package (link) |
Miriad | irainit load miriad | miriad | data reduction package (link) |
aips | irainit load aips | Astronomical Image Processing System (link) | |
analytic infall | irainit load analytic_infall | ||
astron | irainit load astron | ||
Coyote library | irainit load coyote | ||
fits Viewer | irainit load fv | (link) | |
GCC Compiler | irainit load gcc | (link) | |
Gildas | irainit load gildas | (link) | |
Healpix | irainit load healpix | (link) | |
IDL | irainit load idl | (link) | |
heasoft | irainit load heasoft | (link) | |
QA2 | irainit load qa2 | ||
Starlink | irainit load starlink | (link) |
Computing Nodes
Name | RAM | CPU¹ | Cores | Clock | Data Net | Work Disk | Scratch Disk | Sch.² | Group³ | notes |
---|---|---|---|---|---|---|---|---|---|---|
arcbl01 | 256G | [C] | 6/12 | 3600 | 10GbE | 15T | 57G | N | (a,b) | |
arcbl02 | 256G | [C] | 6/12 | 3600 | 10GbE | 15T | 57G | N | (a,b) | |
arcbl03 | 256G | [C] | 6/12 | 3600 | 10GbE | 11T | 65G | N | (a,b) | |
arcbl04 | 256G | [C] | 6/12 | 3600 | 10GbE | 11T | 65G | N | (a,b) | |
arcbl05 | 256G | [C] | 6/12 | 3600 | 10GbE | 11T | 65G | N | (a,b) | |
arcbl06 | 256G | [C] | 6/12 | 3600 | 10GbE | 11T | 65G | N | (a,b) | VM |
arcbl07 | 256G | [C] | 6/12 | 3600 | 10GbE | 11T | 65G | N | (a,b,c,d) | |
arcbl08 | 256G | [C] | 6/12 | 3600 | 10GbE | 11T | 65G | N | (a,b,d) | |
arcbl09 | 64G | [B] | 4/8 | 3600 | 10GbE | 15T | 57G | N | (a,b) | |
arcbl10 | 64G | [B] | 4/8 | 3600 | 10GbE | 11T | 57G | N | (a,b) | |
arcbl11 | 64G | [B] | 4/8 | 3600 | 10GbE | 11T | 57G | N | (a,b,c) | NFS server |
arcbl12 | 64G | [B] | 4/8 | 3600 | 10GbE | 22T | 57G | N | (a,b) | Data-transfer |
arcbl13 | 64G | [A] | 8/16 | 3600 | 1GbE | 3,5TB | N | (a,b,d) |
¹ RAM: [A] AMD Ryzen 7 1800X; [B] Intel Xeon E3-1275 v6; [C] Intel Xeon E5-1650 v4;
² SCH.: Scheduler
³ Group: (a) arc-staff, (b) arc-vlbi, (c) arc-f2f, (d) arc-user; Blades are always dedicated to (a) and (b);
Storage Nodes
Name | RAM | CPU | Cores | Clock | Data Net | RAID | Space | Storage | export |
---|---|---|---|---|---|---|---|---|---|
arcnas2 | 32G | Intel Xeon Silver 4108 | 8/16 | 1800 | 10GbE | ARC-1883IX-24 | 91 | 12x10TB (HGST HUH721010ALE600) RAID6 | /lustre/arcfs0/ost3 |
arcnas3 | 32G | Intel Xeon Silver 4108 | 8/16 | 1800 | 10GbE | ARC-1883IX-24 | 72,8 | 12x8TB (HGST HUH728080AL5200) RAID6 | /lustre/arcfs0/ost0 |
arcnas4 | 16G | Intel XeonE5-2603v3 | 6/6 | 1600 | 10GbE | ARC-1284ML-24 | 36,4T | 12x4TB (WDC WD4000F9YZ-09N20L1) RAID6 | /lustre/arcfs0/ost1 |
91T | 12x10TB (ST10000NM0086-2AA101) RAID6 | /lustre/arcfs0/ost2 | |||||||
arcnas5 | 32G | Intel Xeon E5-2640 v4 | 10/20 | 2400 | 10GbE | Broadcom/LSI MegaRAID SAS-3 3108 | 255G | 2x255GB RAID1 |