AI Extensions on FASRC Clusters

AI extensions are tools that are typically used to integrate generative AI capabilities (or models) directly into interfaces, such as browsers, integrated development environments (IDEs) or interactive computing platforms like code editors, Jupyter notebook, JupyterLab, etc. They are used to enhance productivity while utilizing an AI model of your choice. For example, browser-based AI extensions, such as Sider, Harpa AI, can be used to summarize content, automate tasks through sidebars, or translate texts particularly for Chrome. On the other hand, IDE-based AI extensions, such as Jupyter-AI, GitHub Copilot, JetBrains AI Assistant can be used for accelerating full-stack software development, pair-programming, and analysis to name a few while enabling a powerful way to explore generative AI models in your applications.      

Here, we describe the method to install some of the IDE-based AI extensions on FASRC clusters, how to use them with a few of our in-built applications while ensuring that the AI model being utilized is being accessed in a safe manner and abides by Harvard’s guidelines for the use of generative AI (Gen AI) tools at Harvard.

Jupyter-AI Extension

The Jupyter-AI extension is an open-source tool designed to integrate generative AI models directly into the JupyterLab and Jupyter Notebook interfaces. Once the extension is installed, the `%ai` magic turns the notebook into a reproducible Gen AI playground and can work anywhere the IPython kernel is available.

Install Jupyter-AI Extension

In order to install jupyter-ai extension, one needs to follow the instructions for installing Python packages on FASRC clusters along with those for jupyter-ai. Following are the combined steps to illustrate the installation procedure, starting from requesting a compute node on the cluster, and creating a vanilla mamba environment with a specific version of Python. This is followed by activating the environment and installing the minimal jupyter-ai extension (reason explained below) using `pip`. Notice, `ipykernel` is needed so that the environment can be loaded as a Python kernel in a Jupyter notebook or JupyterLab downstream.

* salloc --partition test --nodes=1 --cpus-per-task=2 --mem=8GB --time=02:00:00 
* module load python 
* mamba create -y -p /n/holylabs/rc_admin/Lab/jupyter-ai-liteenv python=3.13 
* mamba activate /n/holylabs/rc_admin/Lab/jupyter-ai-liteenv 
* pip install ipykernel 'jupyter-ai==3.0.0b9' 'jupyter-ai-magics==3.0.0b7' boto3  
* mamba deactivate

As suggested in our Python package installation document, we recommend users create the environment to install the jupyter-ai package in a location other than their `$HOME`, preferably on the `holylabs` location of your lab folder. Our tests have shown that by doing that, enabling the extension in the notebook downstream is nearly 3-15 times faster (depending on the host and the interface used – Jupyter notebook vs JupyterLab) than creating an environment in `$HOME` to install the extension.

In the installation steps above, we have illustrated the minimal installation method for the jupyter-ai extension (also see Jupyter AI docs). This way ensures that Bedrock-based models, such as Anthropic, can be accessed via HUIT-provided AWS Bedrock, which is recommended for Harvard users. This also includes installing additional dependencies needed to run the available Anthropic models, e.g. boto3 that is needed to provide added support for AWS Bedrock models. OpenAI models, however, are installed by default. You can execute the `mamba list` command to see the packages installed in the environment after activating it.

API key For AI model

After the installation of the extension is complete and prior to utilizing it on the cluster, you need to ensure that the API key of the Gen AI model, you are intending to use, has been obtained and has been associated with Harvard’s agreement with the corresponding company.

OpenAI

The details to create an OpenAI app and obtain an API key that is associated with the Harvard agreement are provided as Option #2 (and Option #3 for limited free usage by FAS) of FASRC OpenAI guidelines. This procedure is also applicable to other API products available on HUIT API portal that can be registered as an app. See HUIT’s AI API service offerings for more details on this topic. 

Bedrock Products

For Gen AI models available via AWS Bedrock, such as Anthropic, you need to follow these instructions to set up Anthropic access that is secured and compliant. These instructions are similar to the ones provided in [OpenAI] with the exception that you would need to configure your shell environment appropriately to use Harvard’s API endpoint. More details regarding requesting access to Bedrock API, examples to set up client tools to connect to AI API(s) via Harvard’s gateway, and how to use those APIs to access the corresponding LLMs can be found on HUIT AI services – AWS Bedrock. 

Jupyter-AI extension using OOD

Once the jupyter-ai extension is installed (minimal or the full version), you can start using it via our Jupyter notebook/JupyterLab OOD app. If you are new to OOD, then refer to our OOD apps Jupyter notebook section to learn how to launch it and load the jupyter-ai mamba environment, you created above, as a kernel. Below, we link to a sample Jupyter notebook that you can run on OOD to see how to use the jupyter-ai extension with LLMs provided by OpenAI and Anthropic while ensuring that the models are being accessed securely.

Examples

See FASRC User Codes repo for AI Tools to access a sample notebook showing how to access an available provider’s LLM securely and leverage the jupyter-ai extension to write a simple Python code using that LLM. It also shows how to interact with an LLM without the jupyter-ai extension.