8. Cluster Guidelines
PICLas has been tested on numerous different Linux- and CPU-based clusters and is continuously been developed to run efficiently on high-performance systems.
8.1. Simulating at HLRS
Unfortunately, the GitHub and GitLab servers are not available on machines at the HLRS, such as the Hawk, due to restricted internet access. The workaround is to use ssh tunneling and proxy jump or remote forwarding to access the repositories.
8.1.1. Cloning with the SSH protocol
Two methods for checking out the code are described in the following.
8.1.1.1. Method 1 (Proxy Jump)
To clone a repository from, e.g., github.com on a HLRS system, the ssh proxy jump must first be set up. Simply connect to
the system via ssh and add the following lines to the ssh configuration file under ~/.ssh/config
on the cluster
Host github.com
HostName github.com
ProxyJump RemoteHost
where the RemoteHost has internet access and can be accessed via ssh from the HLRS system. It is also defined
Host RemoteHost
User username
HostName hostname.something.com
ForwardX11 yes
Then simply clone the repository via
git clone git@github.com:mygroup/myproject.git
and all the ssh traffic via github.com
is automatically re-routed over the RemoteHost.
8.1.1.2. Method 2 (Remote Forwarding)
You can use the SSH protocol to clone the repository. You have to connect to the cluster with the RemoteForward
option
ssh -R 7777:github.com:22 username@hazelhen.hww.de
To avoid using the above command every time, you can add the following to your .ssh/config
file:
host hlrs
hostname hazelhen.hww.de
user username
RemoteForward 7777 github.com:22
and login with ssh hlrs
. Now you can clone the repository when logged onto the cluster by
git clone ssh://git@localhost:7777/piclas/piclas.git
Note that if you experience problems when connecting, e.g., when the warning
Warning: remote port forwarding failed for listen port 7777
is issued,
you have to choose a different port, e.g., 1827 (or any other 4-digit number) and re-connect.
If the code has already been cloned using the original port, the number of the port must be changed
in ./git/config
to the new number for git fetch/pull operations, which would then look like
[remote "origin"]
url = ssh://git@localhost:1827/piclas/piclas.git
fetch = +refs/heads/*:refs/remotes/origin/*
8.1.2. Compiling and executing PICLas
For building on the hazelhen cluster, certain modules have to be loaded and included in the .bashrc or .profile:
module unload PrgEnv-cray
module load PrgEnv-intel
module load cray-hdf5-parallel
An example submit script for the test queue is then
#!/bin/bash
#PBS -N Testcase
#PBS -M email@university.de
#PBS -m abe
#PBS -l nodes=4:ppn=24
#PBS -l walltime=00:24:59
#PBS -q test
# number of cores per node
nodecores=24
# switch to submit dir
cd $PBS_O_WORKDIR
# get number of total cores
ncores=$(cat $PBS_NODEFILE | wc -l)
module unload craype-hugepages16M
# restart
aprun -n $ncores -N $nodecores -j 1 ./piclas parameter.ini DSMC.ini restart.h5 1>log 2>log.err
More information on using the queue system can be found in the HLRS wiki.
Section last updated: 27.03.2019