Data transfer¶
DIPC Supercomputing Center has a shared file system that is available from all computers: Home directories. This file system, whose mount point is /dipc, is ideal for sharing data among different systems. In addition, Atlas a has high-speed local scratch file system.
One of the most common applications for securely transferring data between two hosts are Secure Copy (SCP) and Secure FTP (SFTP).
To transfer files into or out of HPC systems using SCP or SFTP, you need a SSH client:
- GNU/Linux: command ssh, scp or sftp or a GUI tool like GFTP.
- Windows: GUI tools such as WinSCP,
- macOS: Command-line tools ssh, scp or sftp or a GUI tool like FileZilla.
How to use scp/sftp¶
In order to get a file from your home directory to your local machine, it can be done either from Atlas EDR:
$ scp username@atlas-edr.sw.ehu.es:/dipc/username/file /local/machine/path
$ scp username@atlas-fdr.sw.ehu.es:/dipc/username/file /local/machine/path
Same way, files can be transferred to your home directory from both Atlas EDR and Atlas FDR:
$ scp /local/machine/path/my_file username@atlas-edr.sw.ehu.es:
$ scp /local/machine/path/my_file username@atlas-fdr.sw.ehu.es:
or
$ scp /local/machine/path/my_file username@atlas-edr.sw.ehu.es:/dipc/username/mydir
$ scp /local/machine/path/my_file username@atlas-fdr.sw.ehu.es:/dipc/username/mydir
You can also transfer your files to/from your /scratch directory, but keep in mind Atlas EDR's and Atlas FDR's /scratch directories are different filesystems.
$ scp /local/machine/path/my_file username@atlas-edr.sw.ehu.es:/scratch/username/mydir
$ scp /local/machine/path/my_file username@atlas-fdr.sw.ehu.es:/scratch/username/mydir
$ scp username@atlas-edr.sw.ehu.es:/scratch/username/file /local/machine/path
$ scp username@atlas-fdr.sw.ehu.es:/scratch/username/file /local/machine/path
GUI clients¶
Just choose the SSH protocol (default port is 22), type in your credentials and drag and drop the files or directories you want to transfer.