FASRC DOCS

Tag: anaconda

Using Spyder

Using Spyder

The following may be incorrect after June 9, 2023

Spyder is available through Anaconda Navigator on the FAS-RC Remote Desktop application on the Open OnDemand dashboard. To access it:

  • go to the VDI dashboard at https://vdi.rc.fas.harvard.edu/pun/sys/dashboard/ (or https://fasseood.rc.fas.harvard.edu/pun/sys/dashboard if you are working on FASSE)
  • start a new Remote Desktop session
  • connect to the Remote Desktop session
  • open a Terminal in the Remote Desktop session and load the relevant Anaconda module. Check https://portal.rc.fas.harvard.edu/p3/build-reports/ for the most up-to-date modules.
    • Examples:
    • if you’re working with Python 2, type the following:
      • module load Anaconda/5.0.1-fasrc02
    • if you’re working with Python 3, type the following:
      • module load Anaconda3/2020.11
  • type the following:
    • anaconda-navigator
  • when Anaconda Navigator loads, you may choose to remain in the “base” environment, or load your preferred environment from the “Applications on” dropdown, or create a new environment using the Environments tab to the left.
  • when you are in the environment you need, click the Launch button under the icon for Spyder.

Screenshots of the process:

Python

Python

2023 Rocky 8 Transition Information

Python examples: https://github.com/fasrc/User_Codes/tree/master/Languages/Python

Python

What is Python?

Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation.

Python is dynamically typed and garbage-collected. It supports multiple programming paradigms, including structured (particularly procedural), object-oriented and functional programming. It is often described as a “batteries included” language due to its comprehensive standard library.

Examples:

References:

Mamba

Mamba is a fast, robust, and cross-platform package manager.

mamba is a drop-in replacement for Anaconda and uses the same commands and configuration options as conda. You can swap almost all commands between conda & mamba.

Mamba is available on the FASRC cluster as a software module either as Mambaforge or as python/3* which is aliased to Mamba:

$ module load python/3.10.12-fasrc01
$ python -V
Python 3.10.12

You can create conda environments with mamba in the same way as with conda:

$ mamba create -n ENV_NAME PACKAGES

NOTE: The major advantage of using mamba instead of conda is that the environment creation, and installing / uninstalling of packages is much faster with mamba, e.g.,

$ mamba create -n python_env1 python=3.10 pip wheel

                  __    __    __    __
                 /  \  /  \  /  \  /  \
                /    \/    \/    \/    \
███████████████/  /██/  /██/  /██/  /████████████████████████
              /  / \   / \   / \   / \  \____
             /  /   \_/   \_/   \_/   \    o \__,
            / _/                       \_____/  `
            |/
        ███╗   ███╗ █████╗ ███╗   ███╗██████╗  █████╗
        ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
        ██╔████╔██║███████║██╔████╔██║██████╔╝███████║
        ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
        ██║ ╚═╝ ██║██║  ██║██║ ╚═╝ ██║██████╔╝██║  ██║
        ╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚═════╝ ╚═╝  ╚═╝

        mamba (0.25.0) supported by @QuantStack

        GitHub:  https://github.com/mamba-org/mamba
        Twitter: https://twitter.com/QuantStack

█████████████████████████████████████████████████████████████


Looking for: ['python=3.10', 'pip', 'wheel']

pkgs/main/noarch                                              No change
pkgs/r/linux-64                                               No change
pkgs/r/noarch                                                 No change
pkgs/main/linux-64                                   5.3MB @   4.5MB/s  1.2s
conda-forge/noarch                                  11.5MB @   3.8MB/s  3.1s
conda-forge/linux-64                                30.1MB @   4.4MB/s  7.1s
Transaction

  Prefix: /scratch/pkrastev/conda/python_env1

  Updating specs:

   - python=3.10
   - pip
   - wheel


  Package                Version  Build            Channel                  Size
──────────────────────────────────────────────────────────────────────────────────
  Install:
──────────────────────────────────────────────────────────────────────────────────

  + _libgcc_mutex            0.1  main             pkgs/main/linux-64     Cached
  + _openmp_mutex            5.1  1_gnu            pkgs/main/linux-64     Cached
  + bzip2                  1.0.8  h7b6447c_0       pkgs/main/linux-64     Cached
  + ca-certificates   2023.01.10  h06a4308_0       pkgs/main/linux-64     Cached
  + certifi            2022.12.7  py310h06a4308_0  pkgs/main/linux-64     Cached
  + ld_impl_linux-64        2.38  h1181459_1       pkgs/main/linux-64     Cached
  + libffi                 3.4.2  h6a678d5_6       pkgs/main/linux-64     Cached
  + libgcc-ng             11.2.0  h1234567_1       pkgs/main/linux-64     Cached
  + libgomp               11.2.0  h1234567_1       pkgs/main/linux-64     Cached
  + libstdcxx-ng          11.2.0  h1234567_1       pkgs/main/linux-64     Cached
  + libuuid               1.41.5  h5eee18b_0       pkgs/main/linux-64     Cached
  + ncurses                  6.4  h6a678d5_0       pkgs/main/linux-64     Cached
  + openssl               1.1.1t  h7f8727e_0       pkgs/main/linux-64     Cached
  + pip                   23.0.1  py310h06a4308_0  pkgs/main/linux-64        3MB
  + python                3.10.9  h7a1cb2a_2       pkgs/main/linux-64       28MB
  + readline                 8.2  h5eee18b_0       pkgs/main/linux-64     Cached
  + setuptools            65.6.3  py310h06a4308_0  pkgs/main/linux-64        1MB
  + sqlite                3.40.1  h5082296_0       pkgs/main/linux-64     Cached
  + tk                    8.6.12  h1ccaba5_0       pkgs/main/linux-64     Cached
  + tzdata                 2022g  h04d1e81_0       pkgs/main/noarch       Cached
  + wheel                 0.38.4  py310h06a4308_0  pkgs/main/linux-64       66kB
  + xz                    5.2.10  h5eee18b_1       pkgs/main/linux-64     Cached
  + zlib                  1.2.13  h5eee18b_0       pkgs/main/linux-64     Cached

  Summary:

  Install: 23 packages

  Total download: 32MB

──────────────────────────────────────────────────────────────────────────────────

Confirm changes: [Y/n] Y
wheel                                               65.6kB @ 337.5kB/s  0.2s
setuptools                                           1.2MB @   5.4MB/s  0.2s
pip                                                  2.7MB @  10.7MB/s  0.3s
python                                              28.1MB @  73.0MB/s  0.4s
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

To activate this environment, use

     $ mamba activate python_env1

To deactivate an active environment, use

     $ mamba deactivate

To use the environment, do:

$ source activate python_env1

You can list the packages currently installed in the conda environment with:

$ mamba list

You can install new packages with:

$ mamba install -y numpy

To uninstall packages, use:

$ mamba uninstall PACKAGE

When you finish using the conda environment, you can deactivate it with:

$ conda deactivate

For additional features, please refer to the Mamba documentation.

Conda environments in Lab space

(needs review and update)

By default conda/mamba installs your packages in your home directory under ~/.conda. While a safe place to keep this data it is also very low performance due to filesystem latency. It also makes it hard to share your conda environments with other people on the cluster.

You can tell conda to install to a different location by setting up your /.condarc. We recommend using your holylabs directory space as it should have low latency. Before setting up your .condarc you will need to make the directory you want to put the environments into (ex. /n/holylabs/LABS/jharvard_lab/Lab/envs). Then in /.condarc set:

envs_dirs:
  - /n/holylabs/LABS/jharvard_lab/Lab/envs

For more on .condarc see the official documentation.

Troubleshooting

Interactive vs. batch jobs

If your code works in an interactive job, but fails in a slurm batch job,

  1. You are submitting your jobs from within a mamba/conda environment.
    Solution 1: Deactivate your environment with the command mamba deactivate and submit the job or
    Solution 2: Open another terminal and submit the job from outside the environment.

  2. Check if your ~/.bashrc or ~/.bash_profile files have a section of conda initialize or a source activate command. The conda initialize section is known to create issues on the FASRC clusters.
    Solution: Delete the section between the two conda initialize statements. If you have source activate in those files, delete it or comment it out.
    For more information on ~/.bashrc files, see https://docs.rc.fas.harvard.edu/kb/editing-your-bashrc/

  • © The President and Fellows of Harvard College
    Except where otherwise noted, this content is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license.