.. _python_env_conda: Anaconda Python 3 with conda ---------------------------- Anaconda is a special *distribution* of Python that is purposed for scientific workloads. Anaconda claims to provide everything you need for data science development and experimentation, including it's own specialized package manager. Some researchers prefer to use Anaconda for their Python workloads. The following steps describe the recommended method for creating an Anaconda (conda) virtual environment ... Step 1: Housekeeping ++++++++++++++++++++ First, ensure that you are in your home directory and have a clean environment, with no other Python-specific modules loaded... .. hpc-prompt:: hpcterm > cd module list If you see any Python modules loaded, it might be a good idea to log out, then back in to reset your environment. Step 2: Load Anaconda +++++++++++++++++++++ Once you have confirmed that your environment is clean, you can load one of the Anaconda Python modules from which you can begin configuring your virtual environment. Here, we load the default (latest) version... .. hpc-prompt:: hpcterm > module av python .. admonition:: Console: /bin/bash :class: terminal_small | (env2) hpcuser@easley01:~ > module av python | | ----------------------------------- Programming Languages & Environments ---------------------------------------- | python/3.8.6 python/anaconda/2.7.14 ... python/anaconda/3.7.0 python/anaconda/3.8.6 (L,D) | python/3.9.2 python/anaconda/2.7.15 ... python/anaconda/3.7.4 python/intel/3.7.9 | .. hpc-prompt:: hpcterm > module load python/anaconda Step 3: Create a New Virtual Environment ++++++++++++++++++++++++++++++++++++++++ With Anaconda Python loaded, your environment should now be set to use the Python binaries. Because the Python installation is purposed for a multi-user environment, the path for the default package location is not writable by normal users. In order to use our own location for packages, we need to set the ``CONDA_PKGS_DIRS`` environment variable so that ``conda`` does not attempt to write to the shared location... .. hpc-prompt:: hpcterm > export CONDA_PKGS_DIRS=~/.conda/pkgs Now we can create a new virtual environment using ``conda create`` using the ``-n`` argument to give it a specific name. For this example, the virtual environment name will be ``env1``. ... .. hpc-prompt:: hpcterm > conda create -n env1 .. admonition:: Console: /bin/bash :class: terminal_small | hpcuser@easley01:~ > conda create -n env1 | | Collecting package metadata (current_repodata.json): done | Solving environment: done | | ## Package Plan ## | | environment location: /home/hpcuser/.conda/envs/env1 | | Proceed ([y]/n)? | | Preparing transaction: done | Verifying transaction: done | Executing transaction: done | | # | # To activate this environment, use | # | # $ conda activate env1 | # | # To deactivate an active environment, use | # | # $ conda deactivate We can see from the output that conda wants to create the environment in a location in our home directory ``~/.conda/envs/env1``. If ``conda create`` was successful, we should see some new files and directories in that location ... .. admonition:: Console: /bin/bash :class: terminal_small | $ ls ~/.conda/ | environments.txt envs pkgs | | $ ls ~/.conda/envs/env1/ | conda-meta Step 4: Activate the Environment ++++++++++++++++++++++++++++++++ Also from the ``conda create`` output, we can see that Anaconda Python has advised us to use ``conda activate env1`` to use the environment. However, if you have not yet used Anaconda Python, you might see an error when running that command ... .. admonition:: Console: /bin/bash :class: terminal_small | $ conda activate env1 | | CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'. | To initialize your shell, run | | $ conda init | | Currently supported shells are: | - bash | - fish | - tcsh | - xonsh | - zsh | - powershell | | See 'conda init --help' for more information and options. | | IMPORTANT: You may need to close and restart your shell after running 'conda init'. .. warning:: We *could* certainly run ``conda init``, but doing this will alter our ``~/.bashrc`` file with settings specific to the loaded version of Python. Because ``.bashrc`` is sourced every time we log on to the cluster, this could potentially cause a conflict if we ever want to change our Python environment. Our recommended method for activating the environment is to use ``source activate`` ... .. hpc-prompt:: hpcterm > source activate env1 If the activation is successful, your command prompt should change to indicate the environment in which you are running ... .. admonition:: Console: /bin/bash :class: terminal_small | hpcuser@easley01:~ > source activate env1 | (env1) hpcuser@easley01:~ > Step 5: Install Packages ++++++++++++++++++++++++ Now, we should be able to install packages into our virtual environment using ``conda install ``. Let's see if we can install the curl module ... .. hpc-prompt:: hpcterm > conda install curl .. admonition:: Console: /bin/bash :class: terminal_small :: (env1) hpcuser@easley01:~ > conda install curl Collecting package metadata (current_repodata.json): done Solving environment: done ## Package Plan ## environment location: /home/hpcuser/.conda/envs/env1 The following packages will be downloaded: package build ------------------------------------------------------------ _libgcc_mutex-0.1 main 3 KB ca-certificates-2021.4.13 h06a4308_1 114 KB curl-7.71.1 hbc83047_1 140 KB ... ------------------------------------------------------------ Total: 13.9 MB The following NEW packages will be INSTALLED: _libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main ca-certificates pkgs/main/linux-64::ca-certificates-2021.4.13-h06a4308_1 curl pkgs/main/linux-64::curl-7.71.1-hbc83047_1 ... Proceed ([y]/n)? y Downloading and Extracting Packages curl-7.71.1 140 KB ###################### 100% ... Preparing transaction: done Verifying transaction: done Executing transaction: done Step 6: Exiting +++++++++++++++ To exit the execution context of the virtual environment and get back to our regular shell, ``conda deactivate`` ... .. admonition:: Console: /bin/bash :class: terminal_small :: (env1) hpcuser@easley01:morgaia > conda deactivate hpcuser@easley01:morgaia >