Lesson Objectives¶
- review ssh configuring between an HPC and Github
- miniconda instilation and execution
- brief conversation about SLURM scheduler
- brief intro to module systems
- remote development with VSCode!
Some Links¶
these will be included in the useful links document as well:
ask help from your HPC admins!!!!!
WARNING¶
your mileage may vary
both your project and its requirements/limitations and the HPC you run on will determine your experience. There are no hard and fast rules
Minconda Install¶
We rarely have admin privileges on HPCs but we still need to install and run our software and its dependencies. We could build from source (often times for compiled codes using cmake) or we could use non-sudo package managers, in python this is commonly miniconda
Install miniconda in your root directory:
mkdir -p ~/miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm -rf ~/miniconda3/miniconda.sh
Add miniconda to your bash/zhs scripts
~/miniconda3/bin/conda init bash
Other package managers purpose-built for HPCs are available---further reading on spack
SLURM¶
- Launch an interactive node:
srun -N 1 --pty bashorsalloc - See the run queue:
squeue <USER_NAME> - Cancel a Job:
scancel <JOB_ID> - Schedule a batch script
sbatch <BATCH_SCRIPT>
There are other batch schedulers (e.g., LSF) but most have a similar syntax
Module Systems¶
HPCs often have wayyyyyyyy more installed code than you need
modules simplify this by altering your path to configured packages
often this is for compilers (e.g., clang, gcc, cuda)
Remote work with VSCode¶
while vim and emacs exist---why drive a chevy when you can drive a cadillac?
support for any file under the sun, scp/ftp abilities, file browsing, multi-terminal functionalities, etc. etc.
note: is not supported in IBM POWER9 systems