Atlas FDR¶
End of Life
Atlas FDR has been in production since May 2015 and is approaching end of life. The cluster operates with minimal maintenance and will be gradually decommissioned during 2026. New users should use Hyperion or Atlas EDR instead.
Atlas FDR is a DIPC cluster based on Intel Xeon Haswell, Broadwell, and Skylake processors. It uses an InfiniBand FDR interconnect. Atlas FDR has no GPUs.
Atlas FDR and Atlas EDR share a single SLURM instance (same partitions and QoS limits) but have separate login nodes, compute nodes, /scratch filesystems, and software stacks. Jobs submitted from Atlas FDR login nodes run exclusively on FDR compute nodes.
-
6,116 cores
225 compute nodes with Haswell, Broadwell, and Skylake processors.
-
60 TB RAM
From 64 GB to 1.5 TB per node.
-
FDR InfiniBand
56 Gb/s interconnect.
-
No GPUs
CPU-only cluster for general-purpose computing.
Connecting and login nodes¶
ssh <username>@atlas-fdr.sw.ehu.es
You can also connect directly to a specific login node:
ssh <username>@atlas-fdr-login-01.sw.ehu.es
ssh <username>@atlas-fdr-login-02.sw.ehu.es
| Login node | Hostname | Processor | Cores | Memory |
|---|---|---|---|---|
| atlas-fdr-login-01 | atlas-fdr-login-01.sw.ehu.es |
2x Intel Xeon E5-2609 v3 (Haswell) | 12 | 64 GB |
| atlas-fdr-login-02 | atlas-fdr-login-02.sw.ehu.es |
2x Intel Xeon E5-2609 v3 (Haswell) | 12 | 64 GB |
Warning
Login nodes are for file management, editing, and compilation only. Do not run computations on login nodes. If a resource-intensive process is detected, all your processes will be terminated and your account will be suspended until you contact support-hpc@dipc.org.
For detailed SSH configuration, key-based authentication, and troubleshooting, see Connecting via SSH.
Specifications¶
Compute nodes |
Count |
Processor |
Cores |
Memory |
|---|---|---|---|---|
| atlas-[001-018] | 18 | Intel Xeon E5-2680 v3 (Haswell) | 24 | 512 GB |
| atlas-[019-073] | 55 | Intel Xeon E5-2680 v3 (Haswell) | 24 | 128 GB |
| atlas-[074-106] | 33 | Intel Xeon E5-2680 v3 (Haswell) | 24 | 256 GB |
| atlas-[107-170] | 64 | Intel Xeon E5-2683 v4 (Broadwell) | 32 | 256 GB |
| atlas-[171-194] | 24 | Intel Xeon Gold 6140 (Skylake) | 36 | 384 GB |
| atlas-[195-199] | 5 | Intel Xeon Gold 6140 (Skylake) | 36 | 1536 GB |
| atlas-[200-201] | 2 | Intel Xeon E5-2683 v4 (Broadwell) | 32 | 256 GB |
| atlas-[202-209] | 8 | Intel Xeon Platinum 8164 (Skylake) | 52 | 128 GB |
Compute nodes |
Count |
Processor |
Cores |
Memory |
|---|---|---|---|---|
| atlas-[211-216] | 6 | Intel Xeon E5-2609 v3 (Haswell) | 36 | 64 GB |
| atlas-217 | 1 | Intel Xeon Gold 6226R (Cascade Lake) | 32 | 384 GB |
| atlas-218 | 1 | Intel Xeon Silver 4208 (Cascade Lake) | 16 | 256 GB |
| atlas-[219-226] | 8 | Intel Xeon Gold 6248R (Cascade Lake) | 48 | 256 GB |
Filesystems¶
| Filesystem | Mount point | Quota | Purpose | Compute nodes | Backup |
|---|---|---|---|---|---|
| Home | /dipc |
None | Permanent storage, configuration files | Not accessible (symlink to /scratch) |
Daily (up to 2 months) |
| Scratch | /scratch |
500 GB / 3M files | Job working directory, I/O | Read-write | No |
| Lscratch | /lscratch |
None | Fast local I/O for single-node jobs | Read-write (local disk, ~700 GB) | No |
Checking your quota¶
getquota
/dipc (home)¶
Your home directory (/dipc/<username>) is where you land when you log in. It runs on a Ceph parallel filesystem (~800 TB total, shared with Atlas EDR) and currently has no per-user quota, though we ask users to keep usage reasonable.
Daily backups are performed and retained for up to 2 months. If you lose data, contact support-hpc@dipc.org and we will help recover it.
/dipc is not accessible from compute nodes
On Atlas FDR, /dipc is only mounted on the login nodes. On compute nodes, /dipc is a symbolic link pointing to /scratch. This means you cannot use your home directory as a working directory for jobs. Always submit jobs from and direct output to /scratch.
Tip
Home directories are private by default. You can share access using chmod or POSIX ACLs.
/scratch¶
/scratch is a shared filesystem for job working directories. A personal directory /scratch/<username> is created automatically with your account. The per-user quota is 500 GB.
All job I/O should go through /scratch. Submit your jobs from here and direct output here.
Not permanent storage
/scratch is not for long-term storage. When occupancy exceeds 80%, performance degrades for all users. Move results you want to keep to /dipc and clean up finished jobs promptly.
Avoid many small files
The filesystem performs poorly with high volumes of small files. Avoid having more than a few thousand files in a single directory. The per-user file limit is 3,000,000.
/lscratch¶
/lscratch is a node-local filesystem (~700 GB per node). It is faster than /scratch for single-node jobs with heavy local I/O, but the data is only visible to the node where the job runs.
Warning
No user directory is created automatically in /lscratch. Create your own at the start of your job and clean it up when done:
mkdir -p /lscratch/$USER/$SLURM_JOB_ID
# ... use it for I/O ...
rm -rf /lscratch/$USER/$SLURM_JOB_ID
Running jobs¶
-
SLURM workload manager
Partitions, QoS limits, batch script templates, and all SLURM commands.
-
Your first job
Step-by-step tutorial to get started with job submission.
Use mpirun instead of srun for MPI jobs
Atlas FDR does not have full SLURM-MPI integration. Use mpirun -np ${SLURM_NTASKS} to launch MPI programs instead of srun. See the MPI template in the SLURM guide for details.
Software¶
Atlas FDR provides scientific software through environment modules.
module avail # list all available modules
module spider VASP # search for a specific package
module load intel/2019b # load a specific module
Info
The software stack on Atlas FDR is different from Atlas EDR and Hyperion. Some newer toolchains and applications may only be available on the newer clusters.
For details on compilers, optimization flags, and architecture-specific targets, see Compilers.