Access to Abel
Our reference computing environment will be the Norwegian national Abel supercluster. All course participants will be granted access to Abel, where some 10,000 CPUs, compute nodes with up to one terabyte of main memory, and a (small) rack of massively parallel GPUs are available. To register your UiO account for Abel usage, please submit an on-line form, where you should ask for association with Notur project NN9447K (Nordic Language Processing Laboratory; NLPL) until December 31, 2018. It will usually take a day or two before account activation is complete, and you will receive status updates by email and text messages.
Once you have received confirmation of account activation on Abel, you need to connect using ssh (e.g. from the Linux command-line, or any suitable secure shell client), e.g.
ssh -Y abel.uio.no
This will establish an interactive session on one of the Abel login nodes, which we can use for development, debugging, and testing. In a nutshell, moderate computation is fair game on the interactive login nodes, where we interpret ‘moderate’ as, say, using at most a handful of cores, up to 16 gigabytes of main memory, and run-times best measured in minutes.
If you feel there is a need to familiarize yourself with working in Linux command line environment, we recommend reading this FAQ and/or take some Linux basics course.
Python Modules
Python 3 is the main and recommended programming language for this course. Once logged into Abel, there is a course-specific installation of Python 3.5 with TensorFlow, Keras, Gensim, and other relevant add-on modules included. To activate this environment, one needs to execute the following commands:
module purge module use -a /projects/nlpl/software/modulefiles/ module add nlpl-inf5820
You can check that you have a sane working environment by issuing the commands above and then running our sanity test script. It will try to import all the necessary Python packages. In case the test it produces some warnings or errors, ask one of the teachers, send a question to our collective mailbox, or raise an issue in our UiO Github repository.
Please do not try to install any Python modules locally to your user directory on Abel. This will conflict with the environment we prepared for you. In other words, make sure that the ~/.local/ directory inside your home directory on Abel is empty (unless you are absolutely sure you know what you are doing).
Abel Foundations
Computations that will run for more than a few minutes, require multiple CPUs or very large amounts of memory, we need to be submitted through the Abel queue management system, using SLURM files. To learn how to deal with this system, read the Abel FAQ and the Abel User Guide. We will have a quick workshop on Abel technicalities during one of the first group sessions. We also provide an example of a SLURM file, which you can use as a template.
Recommended Editors
You can either develop your code on your computers and then copy it to Abel for larger runs, or you can work with the code on Abel itself. There is no shortage of text editors with various levels of Python support, and we will respect whatever choices you make. Andrey and Stephan are fond of the Vim or Emacs editors, respectvely, and will be happy to assist in getting maximum productivity in these environments.
If you need help in working with Vim, run the vimtutor command. An example of Vim configuration for convenient working with Python code can be found at /projects/researchers/researchers01/andreku/.vimrc or in the repository. Simply copy this file to your home directory on Abel, if you do not yet have a file by that name already, or merge the contents of our file into yours.
For emacs users, we provide a pre-configured integrated development environment (IDE), based on the ElPy package. To launch this environment (once you have executed the above module commands), simply type elpy at the command line.