Step1 – Load the anaconda module
When you login to Opuntia or Sabine, type the command
modula avail
Pick an Anaconda module that fits your needs and load it, for instance:
module load Anaconda3/python-3.6
Step 2- Create a conda environment
Let's first take a look at why you need to create environments. You can skip this if you already know why you have to create your own enviroment.
Many of you may have tried to run "conda install" without creating a conda environment at least once.
conda install package-to-install
Don't forget to replace "package-to-install" with an actual conda package name. The above command would have given you lots of permission denied errors.
What happened is that the command conda install tried to download the package to the path where the system-wide anaconda is installed. These directories are locked down and have the permission to read and execute only. To get around this problem, we instead recommend building a virualenviroment locally and install packages there.
Now, lets actually create a conda environment.
conda create -y -n conda-env
Replace the name conda-env with any name you want to give the environment. I generally use the package name I am installing with "-env" added to the name. For example "spring2018class-env", or "rnaquast-env".
Now, within this environment you can download as many conda packages you like and delete them- it will not delete or add these changes to anaconda module installed system-wide.
Step 3- Activate the environment you just created
source activate conda-env
Don't forget to replace conda-env with the name you gave the conda environment from Step 2. Once the environment is activated you will see the env name listed before your command prompt as shown below
(spring2019-env) [plindner@opuntia ~]$
Step 4- Download and install the conda package in the environment
This command may vary based on the program you are installing- lookup the documentation of the program or script. For example, to download ipyrad (
ipyrad documentation), the command is
conda install -c ipyrad
"-c" flag is telling conda to install the package from that channel. ipyrad will be installed from ipyrad channel. The programs documentation will tell you which channel to download their program from.
To confirm that the program installed type
conda list #the package you installed should be listed here now and its dependencies.
Step 5- Run your analysis
Run your commands for the program here within the environment.
Do not run your analysis on the front-end nodes. If you request a node for interactive usage, please make sure you followed the steps for activating your python environment and make sure you are within the environment, look for the conda-env name before your command prompt.
If you don’t see the conda environment’s name listed, this means you have not activated the environment. Rerun Step 3.
Step 6- Deactivating environment
Once you are done with your analysis- you can deactivate the conda environment by running the command
source deactivate
The command prompt will now go back to the normal one.
Submitting these commands as a job, just add the above lines to your job script. For example
#!/bin/bash
#SBATCH -J analysis_job
#SBATCH -o analysis_job.o%j
#SBATCH -t 00:01:00
#SBATCH -N 1 -n 20
#Load modules required
module load Anaconda3/python-3.6
#activate conda environment
source activate conda-env
#enter your commands here
python my_script
source deactivate
#end of the job script
Activating a conda environment already installed
The next time you would like to run the program you installed in this conda environment, you can simply run
source activate conda-env
Run the commands.
Deactivate the environment
source deactivate
You don’t have to install the package every time you activate the environment, it should already be installed.
